That would be an excellent choice for someone that has a few parts laying around. You will not, however, be able to eliminate resistors for this application. This will eliminate the need to use resistors to create a voltage drop on the Vout side of the Vreg only.
Here's a simple schematic to provide 5vdc on the Vout side of that Vreg.
I have to warn you, however, that this IC is just as inefficient as the 7805. The only benefit of using a variable reg (in this case) is to adjust the needed voltage to compensate for start-up draw.
In this schematic, a varistor is used but a static resistor may be used in it's place if needed.
Another issue.....If you use a variable Vreg and don't have a meter to test the voltage, you
WILL blow your router into tiny little pieces. The above schematic will net you proper voltage (5vdc on your 4.5vdc input) just don't deviate from the design.
A better option for battery use is the PT5100. It's a low-loss Vreg and I have used them more than once for battery powered devices. The company is now owned by Texas Instruments so I am unsure of the new part number. I'm sure Radio Shack's cross-reference guide will get you what you need. They are, however, close to $15USD a piece last I bought them.
Here's an example schematic:
As far as your connector goes, I usually grab a wall-wort (sure, we'll say that's the technical term...) power supply out of my drawer and cut the end off. The best way is to take your device into the store and take a few out of the package. Any sensible Radio Shack will allow you to do this and MAY have a few in the back for you to test. If it fits, most likely it will work just fine.
BE SURE to get the polarity correct, especially with the above designs. If you happen to get it backwards, there are no 'oops' checks at all.
Normally, I'd draw you up a lovely schematic to work from. Unfortunately, a recent Raid 0 crash has lost my AutoCad. I'll get it installed and show you what I mean over the next few days.
Just to clarify, these are
NOT my designs. These are designs of others that I have used before as a base. I usually don't post schematics of my personal designs. Most people don't either. When I get ACD re-installed, I guess I'll be nice and design something for you.
You're also going to need to look at board etching kits. You can't just solder it all together and pray nothing shorts. Radio Shack (I'm just referencing them because it seems that's where you'll buy from?) has project cases to put the board in, PCBoards, and kits to etch the board with.
I hope this has helped a little. It's really hard to explain years of schooling and training when you don't know the experience level of the student.

I'll continue to try to help and I hope you're adventurous enough to try this. Most any Elect. savvy tech started by 'trying' things out! Best of luck to you and feel free to ask if you have any more questions!