After near going crazy trying to figure out why IVTC doesn't work with mencoder, I found out why. Please note that this refers to IVTC a captured movie AVI or hard-telecined DVD, not MPEG2 with pulldown flags.
The problem is this that mencoder does not duplicate frames correctly with MPEG2. This is needed for IVTC when there's pattern change. mencoder actually says it's duplicating, but doesn't do so. Because of that, telecined streams with lots of scene changes (music videos, my test matherial) get out of sync very fast. Mencoder does that because with AVI you don't need to repeat the whole frame , writing a zero-byte chunck tell the decode to repeat the frame. Not true with MPEG. I found out about this here:
http://www1.mplayerhq.hu/pipermail/m...er/002970.html
So I tested, when outputting an AVI, IVTC works perfectly, with MPEG, it gets out of sync. I am currently applying the patch the guy posted in the above link (well sort of, the patch no longer works and I had to manually change sections) to see if it solves. Then I'll do a bug report. Anyway, if anyone needs IVTC right now, just transcode first to a lossless AVI (huffyuuv,FFV1) and then encode to mpeg1/2.
PS:As a side-effect, I managed to compile "transcode" in windows (with some hacks), another linux based encoding package. But it seems to use tons of shared libs and doesn't want to build them static, so I am giving up. The reason I tried this is because transcode has a decomb port.