Encoding NTSC Interlaced Fullscreen Video?

[DanoSaurus]

First I'd like to thank everyone here for their hard work, I have been lurking for a couple of weeks and learning a whole lot about making KVCDs. So far I've got all the tools I need and everything's working fine. However, I need some suggestions about the best way to deal with my source material. I would like to encode about 80-90 minutes of NTSC Interlaced 29.97fps 4:3 fullscreen video from DVD on one CD-R. So far I've found my DVD player is compatible with 352x240 and 480x480 resolutions, it will NOT play 528x480 or 544x480. It will also play 352x480 but will jump to double-speed playback after scanning or switching chapters so I guess that resolution is out. I have not tested anything higher. It will playback mpeg-2 streams burned as VCD but will not play SVCD. Questions:

Which TmpgEnc template should I use?

Should I de-interlace and encode mpeg-1 or keep interlaced with mpeg-2?

What is the best method to de-interlace?

Which script should I use, MA or non-MA?

Any suggestions are appreciated!

[DanoSaurus]

Okay, here is my method so far. Source material is 80-90 minutes full-frame NTSC interlaced video @29.97fps.

1) Rip DVD to HD with DVD Decrypter

2) Create D2V project with DVD2AVI.

3) Encode audio @128k MP2 with HeadAche.

4) Frameserve with Avisynth 2.5 using the recommended MA Script. Note I added TomsMoComp to de-interlace.

## DLL Section ## 

LoadPlugin("C:\Filters25\MPEG2Dec3.dll") 
LoadPlugin("C:\Filters25\GripFit_YV12.dll")
LoadPlugin("C:\Filters25\TomsMoComp.dll") 
LoadPlugin("C:\Filters25\STMedianFilter.dll") 
LoadPlugin("C:\Filters25\asharp.dll") 
LoadPlugin("C:\Filters25\unfilter.dll") 
LoadPlugin("C:\Filters25\undot.dll")
LoadPlugin("C:\Filters25\Sampler.dll") 

## Main section and static filters ### 

Mpeg2Source("D:\Video\Video.d2v")

undot() 
Limiter() 
asharp(1, 4)
TomsMoComp(-1, 15, 1)
GripCrop(480, 480, overscan=2, source_anamorphic=false) 
GripSize(resizer="BicubicResize") 
STMedianFilter(3, 3, 1, 1 ) 
MergeChroma(blur(1.5)) 
MergeLuma(blur(0.1)) 

## Linear Motion Adaptive Filtering ## 
# 
# ( Portions from AviSynth's manual ) 
# This will apply variable temporalsoften 
# and variable blur. 
# Both filters are active at all times, and work inversely proportional to the 
# activity, measured from current frame to next frame. 

ScriptClip(" nf = YDifferenceToNext()" +chr(13)+ "unfilter( -(fmin(round(nf)*2, 100)), -(fmin(round(nf)*2, 100)) ).TemporalSoften( fmin( round(2/nf), 6), round(1/nf) , round(3/nf) , 1, 1)  ") 
# 

GripBorders() 
#LetterBox( Your_Values_Here ) # Depends on situation. Use MovieStacker! 
Limiter() 
 
## Functions ### 

function fmin( int f1, int f2) { 
  return ( f1<f2 ) ? f1 : f2 
} 
  
# 
####

5) Load .avs into TMPGEnc (2.521), load KVCDx3 template. Change the following settings:

Under Video settings change resolution to 480x480, framerate to 29.97.

Advanced settings change Video Source Type to "Non-interlace (progressive)" and Source Aspect Ratio to "4:3 525 line (NTSC)"

GOP structure change Max Number of Frames in a GOP to 30.

Save as text project.

6) Use CQMatic for file prediction. This is giving me the most trouble right now. Both CQMatic and ToK are all over the place with predicted file size, either way under or a little over. I am trying x3 prediction with CQMatic right now using a custom size of 785mb in CalcuMatic to determine average bitrate. I get CQs in the 50-60 range. Any hints?

7) Mux mpeg stream with bbMpeg as VCD.

8) Create bin/cue with chapters in VCDEasy, burn with Nero.

That's it. Seems to work well and quality is good. Is there anything I can add to make it any better? Is there anything I don't need to do or could do differently? Any advice on getting more accurate results with CQMatic?

[mistermickster]

I guess you get an average bitrate of between 1062 & 1211 Kbps, right

What size does your final video file come out at?

You will of course get a lower CQ with 480x480 than you would with 352x240

[Dialhot]

You should encode in MPEG2, 23.976fps and set the interlace mode to "3:2 pulldown during playback". The video will be smaller and so the CQ will be better.

[DanoSaurus]

Quote:

Originally Posted by mistermickster

I guess you get an average bitrate of between 1062 & 1211 Kbps, right

What size does your final video file come out at?

You will of course get a lower CQ with 480x480 than you would with 352x240

Yes on my current project my average bitrate is 1183.

I think I may have worked out the prediction thing. On my last run, using CQMatic with CalcuMatic set to a custom file size of 795mb I got a final muxed mpg around 800mb, so I'm very close. After authoring with EasyVCD the .bin was around 811mb, which Nero could not burn without overburning. My current run I set CalcuMatic to 785mb, to allow a little more room for muxing and authoring. We'll see how it turns out. These are only the first two runs I've made with x3 prediction, before I was using x1 prediction and my results were all over the place, too high, too low.

As for the resolution, yes I've tried 352x240 with a higher CQ but I feel that I get better detail @ 480x480. I think it's worth the trade off in CQ but I could be wrong.

[DanoSaurus]

Quote:

Originally Posted by Dialhot

You should encode in MPEG2, 23.976fps and set the interlace mode to "3:2 pulldown during playback". The video will be smaller and so the CQ will be better.

Thanks Dialhot, I think I will try this. If I use these settings, do I keep TomsMoComp in my script? Also, would I use the SKVCD template?

[Dialhot]

Quote:

Originally Posted by DanoSaurus

Thanks Dialhot, I think I will try this. If I use these settings, do I keep TomsMoComp in my script? Also, would I use the SKVCD template?

No, you must continue to deinterlace the source or you have to adapt the script to work in a field basis mode (to handle correctly the interlaced frames). So continue to use TomsMoComp (even if I prefer personally FieldDeinterlace).

For the template, do not bother with that. Load the KVCDx3 template and juste change the setting you need to change in tmpgenc as you already do.
In fact the purpose of templates, in my eyes, if to avoid to have newbies lost with all the settings. But it seems you aren't lost .

[DanoSaurus]

Hmm prediction is still a problem...

On my first run I set Calcumatic to a custom filesize of 795mb, used the average bitrate in CQMatic with x3 prediction and got a final muxed mpg of 800mb.

On my second run, I changed Calcumatic to filesize of 785mb, used the new lower average bitrate in CQMatic again with x3 prediction, but this time my finished muxed mpg is 816mb! How can this be? Is it the source material? I know I should probably post this over in the prediction forum...

[Dialhot]

Quote:

Originally Posted by DanoSaurus

On my second run, I changed Calcumatic to filesize of 785mb, used the new lower average bitrate in CQMatic again with x3 prediction, but this time my finished muxed mpg is 816mb! How can this be?

This is known as the "unpredicable non-linear CQ curve" of tmpgenc. In other words : CQMatic can do wonders but it is not 100% sure because of how tmpgenc manage the CQ.

[Icarus3000]

Dialhot:

Can you clarify one thing for me please:

Quote:

You should encode in MPEG2, 23.976fps and set the interlace mode to "3:2 pulldown during playback". The video will be smaller and so the CQ will be better.

In another thread there is a script for converting 30i to 24p. I don't think that is what you mean here, is it?

If I have a 30i source, can you tell me the steps to do what you say above. I think you mean:

1. Include the line Fielddeinterlace(full=true) in my avisynth script
2. Don't include anything else related to changing fps or interlacing/deinterlacing in the script
3. In TMPGenc encode as 23.976 fps MPEG2
4. In TMPGen encode with "3:2 pulldown during playback" enabled

Can you confirm that the steps above are what you had in mind?

Thanks!

Icarus

[Dialhot]

Quote:

Originally Posted by Icarus3000

In another thread there is a script for converting 30i to 24p. I don't think that is what you mean here, is it?

Yes it is.

Quote:

1. Include the line Fielddeinterlace(full=true) in my avisynth script
2. Don't include anything else related to changing fps or interlacing/deinterlacing in the script
3. In TMPGenc encode as 23.976 fps MPEG2
4. In TMPGen encode with "3:2 pulldown during playback" enabled

No, in this case you have tu use a special way to deinterlace. So the #1 is replace by :

1. Include the lines Telecide() and Decimate()

These two line will do the detinterlace and the change in fps (you will see that when you load the avs script, now Tmpgenc tells you it is a 24 fps source and not a 30 one).

All the end is okay.

[DanoSaurus]

Quote:

Originally Posted by Dialhot

This is known as the "unpredicable non-linear CQ curve" of tmpgenc. In other words : CQMatic can do wonders but it is not 100% sure because of how tmpgenc manage the CQ.

Interesting. Are there any methods to minimize this unpredictability? Is there any more reliable prediction method, besides CQMatic? Manual prediction?

BTW I am encoding a project right now, using your suggestion with mpeg-2 @ 23.976fps with "3:2 pulldown during playback". CQMatic gave me a CQ of 74.74, much higher than what I was getting before, around 63. I'll post the results later.

[Dialhot]

Quote:

Originally Posted by DanoSaurus

Interesting. Are there any methods to minimize this unpredictability? Is there any more reliable prediction method, besides CQMatic?

1. Using min = 0.57*avg and max=2000.
2. Using x3 insteed of x1
3. Using both

Quote:

Manual prediction?

CQMatic is nothing but a manual prediction done automatically

[Icarus3000]

Dialhot:

Very interesting....

I have tried a similar approach, where I convert 30i to 24p, but WITHOUT the 3:2 pulldown (as I normally encode in MPEG1).

The result looked a little "strange". Too soft for my liking - gave everything a "dream" like appearance (not sure how else to desribe it!).

If I encode in MPEG2 with the pulldown, will it look closer to the original video? If not, what is the benefit of using the 3:2 pulldown, if it is possible to encode as 23.976 directly?

Thanks,
Icarus

[Dialhot]

Quote:

Originally Posted by Icarus3000

If I encode in MPEG2 with the pulldown, will it look closer to the original video? If not, what is the benefit of using the 3:2 pulldown, if it is possible to encode as 23.976 directly?

Incredible can explain you that much better than I can. Or go to see the 100fps.com site.

But remember that we are starting from 30 interlaced and we go to 24 progressive. This way we do not lose quality.

What you did in MPEG1 is starting from 30i and going to 24i. I guess that you lost some fields during the process (the way you describe it seems to correpond to that).

But again, Incredible is the guy; I never do interlaced NTSC sources.

[incredible]

Quote:

If I encode in MPEG2 with the pulldown, will it look closer to the original video? If not, what is the benefit of using the 3:2 pulldown, if it is possible to encode as 23.976 directly?

I didn't have the time to go through the whole thread now BUT ....


- CASE 1:

If your video is a Hollywood movie you have to inspect fast moving scenes in DVD2avi to be shure that there does no combing appear!
Just the message of 29.976 interlaced in the right window MEANS NOTHING YOU CAN TRUST IN! Only trust your eyes!

If you see interlacing artefacts (combing) in fast scenes you got a bad mastered hollywood movie, telecined to 29.976 including phase shift (shifted Fields = interlace look purposed to TV).
In this case do use your script above nut put below the first import line "mpeg2sorce...." the lines suggested by dialhot:

Telecide() # performs a reverse phase shift
Decimate() # does eliminate the added frames during 3:2 pulldown

and continue wit your script inkl. encoding using the pulldown-flag as explained here:
www.incredible.de.tf/pulldown.html

... audio will be in sync! afterwards.



- CASE 2:

If you see in DVD2AVI interlacing artefacts and its NOT a hollywoodmovie you deal with an real interlaced source.

In that case use this Script:

Mpeg2Source("D:\Video\Video.d2v") 
# We do get rid of limiter() as it only CUTS the luma range= MEGA RISKY
Fielddeinterlace(full=false)
convert60ito24p(2,0)
undot() 
asharp(1, 4) 
GripCrop(480, 480, overscan=2, source_anamorphic=false) 
GripSize(resizer="BicubicResize") 
STMedianFilter(3, 3, 1, 1 ) 
MergeChroma(blur(1.5)) 
#MergeLuma(blur(0.1))  # dont use this anymore cause the new MA already blurs from the beginning on as it performs now continously and linear
#
ScriptClip(" nf = YDifferenceToNext()" +chr(13)+ "unfilter( -(fmin(round(nf)*2, 100)), -(fmin(round(nf)*2, 100)) ).TemporalSoften( fmin( round(2/nf), 6), round(1/nf) , round(3/nf) , 1, 1)* ") 
# 
GripBorders() 
#
## Functions ### 
#
function fmin( int f1, int f2) { 
* return ( f1<f2 ) ? f1 : f2 
} 
#
function convert60ito24p (clip video, int mode, int offset) 
{ 
work = assumefieldbased(video) 
out = (mode==2) ? interleave( 
\selectevery( 
\layer(trim(work, 1, 0), 
\layer(work, trim(work, 2, 0), "fast"), 
\"fast"), 5, 0 + offset), 
\selectevery( 
\layer(work, trim(work, 1, 0), "fast"), 5, 3 + offset)) : 
\* * (mode==1) ? interleave( 
\selectevery(trim(work, 1, 0), 5, 0 + offset), 
\selectevery(layer(work, trim(work, 1, 0), "fast"), 5, 3 + offset)) : 
\* * (mode==0) ? selectevery(work, 5, 1 + offset, 4 + offset) : work 
assumeframebased(out) 
}
#

Instead of the line
Fielddeinterlace(full=false) instead of this you can use
Smoothdeinterlace(doublerate=true, tff=true, blend=true).selectodd()
... my personal favorite

This script contains a function of sharfis_brain, to do a conversion from 29.976 real interlaced (that means NOT telecined) to 23.976 progressive frames.
And here also the audio will be in sync when doing the pulldown-flag encode mode as shown in my link/site above.


- CASE 3

If you see in DVD2AVI no interlacing/combing artifacts even it says in the right window 29.976 NTSC you deal with a well treated/mastered Hollywoodmovie encoded at 23.976 FPS and a pulldown-flag added (like explained in my link) and thats the way 95% of the hollywood movies will be mastered to NTSC DVDs. (Btw: 23.976 with puldown flag JUST appears titeled as 29.976 but IT ISN'T in real!)
In this case just use in DVD2AVI "Forced Film", use the standard MA script and encode again using 23.976 incl. Pulldown Flag as shown in my link above.

Conclusion:
Even!! on a real "SHOT ON VIDEO" interlaced source (as case 2), since sharfis_brain came up with his innovative convert30ito24p function ...
Its no longer needed to encode KVCDs at 29.976!
= Much more Quality cause of LESS frames to encode!

And thats the point:

Movies !shot on 23.976 FPS FILM! (=Blockbusters/Hollywood) where never "real" interlaced.
- they are just encoded at 23.976 FPS using the Pulldown-flag which gives your standalone the order to do a 3:2 Pulldown internally
- or they are treated using a "Telecine" Process so they got shifted fields (interlace look) and frames added 3:2 Pulldown already inside the commercial source, but this ONLY makes sense if the Movie will be broadcasted to TVs, and thats why I said it maybe could be a bad mastered Hollywoodmovie (if its one) still telecined and produced on DVD.

Movies !shot on 29.976 FPS video! are REAL interlaced real 60 fields per second, interleaved to 29.976 interlaced frames = interlacing/combing artifacts.
This is the case in a lot of Music Concerts, some Music Videos (but much are shot on Film), almost all TV station Trailers, Documentations, etc.....
Well, said: Everything recorded on Video!

I think we should write a separate "sticky" which does contain all this

[Icarus3000]

Incredible:

Very informative, thank you!

It sounds like I may be blaming the wrong function for my poor output.

The sources I am dealing with at the moment are all from a DV camcorder, so 29.976 interlaced.

I used the 60ito24p function, BUT also added Pixiedust(), so it is probably the Pixiedust that is causing the "too soft" appearance, and not the deinterlacing.

I will have to do some more experimenting on some small portions of my video. On my P-III 733 it took 30 hours to encode the entire 83 minutes!

Thanks,
Icarus

[incredible]

In case of DV yoou have to watch out as they are standard "Bottom Field first"!!
That means in case of Smoothdeinterlace that you have to use tff=false!

Another point:
Try to get a new machine, I know thats not what you wanna hear, but there are very much cheap machines on the market, like an Athlon 2400 system or an Intel System which will be a difference from night to day if compared with your existing one.
Because once you get in video stuff, you'll get addicted!
And Videorendering is one of the hardest real-works for a PC.