If you want to decent job of it, creating and using weapons in fiction can be quite tricky. There are a lot of pitfalls that newbies and even long time veterans of creating fiction fall into.

1. Number one is simply copying what others have done without understanding it yourself. “So what” you may think, maybe that person did something great and you're simply borrowing a little bit of that greatness so your stuff will be cool too… But it doesn't always work that way, often the person you're copying from is just one in a long line of idiots making the same mistake without knowing why (big swords being drawn from the back, bullets knocking people off their feet, anybody ever carrying and firing a minigun etc), or else it's someone who really understood their choices well but they ONLY apply specifically to the context in their work and not outside of it (Masamune Shirow's Tachikomas from Ghost in the Shell).

2. Keep It Simple Stupid (Kiss)
The most used weapons in reality are the mundane, boring, standard ones, like simple knives, machetes, standard 9mm automatic pistols, pump action shotguns, bolt action rifles and so on. The reason is that they're amazingly versatile and can be used in many different situations very easily. The more specialised a weapon is, like a gigantic great sword, or huge pistol that fires 50cal Browning machine gun ammo, the less situations it can logically be used in. That doesn't just go for comically out-sized stuff, it's just the same of assault rifles tricked out with laser sights, grenade launchers, high capacity magazines etc etc- that is a weapon of the battlefield, very awkward, heavy, dangerous (to the user), clumsy and attention getting anywhere else.

Look at what real people in similar roles to your character would use and why.
That said, highly specialised and unusual weapons can be used amazingly if they are used specifically FOR their speciality and in a way that advances the plot. The anti-tank fighter in the anime Pumpkin Scissors carries a huge specially made bolt action pistol that fires 50cal machine gun ammunition. In normal circumstances that would be a very impractical, silly weapon, but in the setting of the anime he uses it point blank to defeat tank armour (there was a time when you could actually do this with tank armour).
The No-dachi is a huge Japanese sword. It looks great but it's only of any real use on the battlefield or in personal duels where it gives the user a huge reach advantage to strike their enemy first or to keep more than one opponent at bay. Simply carrying the weapon in every day life is impractical- it's too long and so has to be hauled around like crappy but more easily damaged version of a quarterstaff. It's even too long to be carried on the back in a normal environment and utterly useless in a fight unless you have a LOT of open space and high ceilings around you at the time. Katanas are boring and common in comparison but they make more sense in far more situations.

So: Keep it simple, UNLESS you have a clever use for it.

3. Carrying too much:
Decide the minimum tools that your character needs and stick to them. A pistol and a rifle with 5 magazines of spare ammunition each maybe? Guns are heavy and take up a lot of room, ammunition takes up less but it's still heavy and bulky. Soldiers have to carry it all on special belts called webbing, normal people would look out of place doing that so consider that your character might have limited fire power and maybe work that into your plot. Even things like arrows and throwing knives are limited and heavy. Vehicles carry a lot more ammunition for things like mounted machine guns but even THAT is burned through very fast because that's precisely how machine guns work.

Lasers and energy weapons can seem to have unlimited firepower but they don't- their energy sources are drained and they heat up till they degrade. Even spells should make your wizard tired, forgetful, or wear out their wands. Consider using that in your plot.


4. Weapon systems:
This applies to mecha, tanks, ships planes and so on. Think of their role and arm them accordingly. Voltron and the Zords from the Power rangers are great examples of terribly thought out weapon systems: every single attack and weapon they have is close to useless apart from their main sword attack.
“So what? They're just fun shows for little kids.”
The trouble is that this sort of thing makes stories that are so predictable and stupid that even children recognise it once they get over the cool factor. It would also be amazingly boring to write them, so it's best not to fall into that trap in YOUR writing.

Peace time configurations of weapons systems are designed for multiple roles- they carry minimal basic armaments that could be used in many different situations. In war time maximum fire power tends to be the main thing: the weapons get as big and powerful as is practically possible, carrying as many at the same time as is physically possible. Weapons systems for paramilitary like police are extremely neutered compared to the military because they don't need to face the same sorts of threats. So keep in mind the roles and situations when coming up with this stuff.

5. Consider having a look at how real weapons are used and how they handle. Stats in a wikipedia article are great but it's another thing to handle them yourself or at least see them used on youtube. Some great resources are:
https://www.youtube.com/user/scholagladiatoria Extremely knowledgeable expert on swords, armour, bows and even black power weapons.
https://www.youtube.com/user/SkallagrimNilsson A more general, amateur enthusiast on various swords and modern firearms.
https://www.youtube.com/user/ForgottenWeapons An expert on unusual and rare firearms, including all types of full sized machine gun. - This man has expert engineering knowledge, not a fake persona like FPS Russia.

Why should you care?
Why not just do whatever you feel like? Well, the more thought you put into details like this the more real and interesting your work can feel (don't take it too far and give us a lecture though). Clever and knowledgeable use of weaponry can make your story more immersive. And as with the Voltron example, poorly thought out weapons, if they're as important to the story as that was, end up degrading your plot and story quality.
Remember the classic line “Do ya feel lucky, punk? Well, do ya?”. This was from on a scene where we're expected to remember how many bullets Dirty Harry had fired and whether he still had one left to kill the bad guy. It added a lot of drama and tension to the scene as both the characters and the audience tried to work it out.
The final part of chapter 7 of Pinky TA (starting here http://www.theduckwebcomics.com/Pinky_TA/5322902/) revolves around the specific capabilities of their mecha weaponry and how much ammunition each character has left. Not as successful or as dramatic as Dirty Harry, but it was a useful conceit to write that part of the story around!

There are some even cleverer takes on this subject on the DD forums where I originally posted a thread about this subject.

Link
Original forum thread - http://www.theduckwebcomics.com/forum/topic/177409/
Inspired by a Gizmodo article based on this book - http://www.amazon.com/gp/product/1599638150