Shiz,
There are a couple things you can check with your mic volume:
1. In the Steam client, click on the Steam pulldown menu in the upper left, and select Settings. Click over to the Voice tab and hit Test Microphone and play with the Microphone Volume slider.
2. In Windows XP, you can open your control panel and go to Sounds and Audio Devices, under the Volume tab, hit the Advanced button, and check your Master Volume and Mic Volume sliders (you may need to unmute your mic, although you shouldn't have to) Then hit the Advanced Button under Mic Volume and make sure your DB Microphone Boost box is checked.
Then go to the Options pull down menu in the upper left, and hit Properties. In the new window, under Mixer Device, select each item that says "Input" and check the Mic sliders there as well.
In Vista/Win7, the settings are similar but you may have slightly different navigation.
3. If you have a discrete sound card or a more advanced sound system which comes standard on some of the better motherboards, you may also have a 3rd party sound controller. This would be things like Realtek Audio Manager, Sound Blaster Control Center, etc. Each of these programs has it's own audio controls that may also be affecting your sound input.
4. You may also have a crappy mic. A lot of the cheap mics available from the retail stores don't have a very long useful life. It may be time to spend another $20 and get an upgrade.