For video capture, use
VirtualDub 99%+ of the time.
But for video conversion, VirtualDub2 tends to be better, as that fork was created with more input/output formats in mind. However, I'm not sure how H.265 reacts. Probably not.
For H.265 > whatever conversion, Hybrid is probably what you want.
Why are you wanting to use
VirtualDub here?
Note that VirtualDub2 vs. VirtualDub gets stickier when it comes to the restoration filters. Lots of filters broke in VirtualDub2, and no longer work correctly. (I've notified the dev of these problems, long ago, but nothing was address. I wasn't even acknowledged. This wasn't in an email, either, but the official VH forum post where the dev visits.)