It would seem that your router is doing its job - keeping outsiders from getting into your network.
Gettings into the firewall may be hard depending on which router it is.
As for myself I use a dual router system including a Cisco 806 broadband router and a Linksys (now a division of Cisco) Instant Wireless Broadband router (802.11b).
Most people seem to own the Linksys routers.
First off, before you start following my details - read your manual!
If you know the IP address of your router (Linksys uses 192.168.1.1) all you should need to do is open Internet Explorer (or any Web browser you like) and goto
http://192.168.1.1 (or the IP address that is correct).
This may bring up a logon dialog box asking for a user name and password. Linksys uses a blank user name and the default password is admin (you should change it immediately if it is). Click OK.
The page should now load showing various information. The firewall may be listed under the "Applications and Gaming" link or "Security" or some other relavent link - check the manual or online documentation.
Check port and MAC filtering, NAT/PAT, VPN, Setup and other links (depending on router make/model).
You may see an item that says something like "Allow Remote access" - you would want to enable this - but then enable filtering to only allow your remote (home) computer to access the local (office) computer. Most ISPs use dynamic addresses so your address might be valid only for a small window - between 2 hours and 24 hours after that you'd have to edit the filters to allow for the new IP address.
You could also (if able to) expose the remote desktop service as a running service like a Web server on the network by telling the router to allow access in and out for your home computers IP address and port number (I think it's 63553 or something, it is listed in other posts on the news group).
If you have an expensive Cisco router like I have, you'd have to either get the 806 with its Web-based configuration or pay me to come and configure the IOS access-lists and firewall.