To update on what I've done and what's been happening, I decided to fully reinstall Windows 10 after realizing the original install was done for.
Upon reinstalling captures worked as they should, and I got several good captures out of it. However, upon installing the driver for my GT 710, for some reason the same issue started happening again. The firewire input would basically cut out after about 30 seconds, come back, Virtualdub
would try to catch up, rinse and repeat.
I opted to disable the GT 710's Nvidia driver back to the basic display adapter, and while it worked for a while, when I came back to check on my capture about an hour later, the same issue had happened.
If I had to venture a guess, I'd say it's possibly a PCI-E bandwidth issue (the firewire card is in a PCI-E x1 slot)? Though I've never known the GT 710 to be a big bandwidth hog since it's not very powerful to start.
That or Windows 10 is just an awful OS. Regardless, if anyone has any more ideas, feel free to let me know about them, since trying to work around all of these things has gotten a bit annoying.