Hi there,
Firstly, before overclocking, you should be aware that by OCing you will find that you void most of your components warranties (although it would be difficult for a manufacturer to prove that OCing was the cause if a component went wrong), also as mentioned by other members you will probabally have to upgrade your cooling to better air cooling or even a watercooling setup depending on the scale of the OC you want to achieve.
In order to OC a P4, you normally have to increase the fsb speed via BIOS, as P4's have locked multiplier settings (unlike AMD's). By increasing the fsb speed this means that you will also be increasing the operating speed of your memory, so you may also have to look into upgrading your memory also (to something like DDR500, instead of the DDR400 that I presume you're already using).
It can be a long and laborious process to achieve a stable OC!
You should go into BIOS and raise the fsb speed 1MHz at a time, then save settings, restart, boot into windows and see if your computer is running stably. If it is stable, then you can go back to BIOS, increase another 1MHz, save settings and boot back into windows again. Keep doing this until Windows crashes (where you'll have to go back to BIOS and change back to the last stable fsb Speed.
Depending how much you try to OC, you may also have to increase the CPU's core voltage (vcore), to keep the system running stably.
Throughout your OC process, do constantly moniter your CPU temp to see if it gets too hot or not.
So, you will find your main limits are CPU temp, how far your mobo will allow you to increase the fsb, and the speed of your memory.
Happy OCing!