A P4 1.8Ghz is adequate for MPEG-2 and lossless/uncompressed AVI captures. I used one for years.
Lossless and uncompressed video is mostly heavy on the hard drive (I/O resources), and light on RAM and CPU. It's mostly moving data off the video capture chips directly into data files with no further processing.
MPEG-2 capturing would depend on the card. Hardware compression cards would perform the same as the AVI, mostly being the movement of data. Because MPEG-2 is smaller, there's less I/O overhead. Software or hybrid hardware/software compression involves some CPU cycles, so you'll want to verify that Windows is running lean, with few open programs or running background services (and disconnected offline, in many cases).
RAM (memory) is not really used for video capture. The VPU/chipset on the video capture card handles that.
The difference in 352x and 720x is twice the bandwidth. You may have an issue with either I/O or CPU, depending on the capture format.
I/O issues on a computer from that era often means the drive has jumped out of DMA/UDMA mode, and is running as PIO. This is discussed in the dropped frames guide, because most "messed up" video is the result of dropped frames. See that here:
How to Prevent Dropped Frames and Audio Sync Problems
If you figure out what's wrong, reply with the solution.