HuffYUV/Lagarith YUY2 not working with VirtualDub/Premiere?

[hysteriah]

Okey, here's the situation. I'm used to work with DV footage but I'm trying to move over to HuffYUV/Lagarith now... but it doesn't work with the same workflow as I'm used to... so I really need some help here.

Here's the workflow that I'm used to:
1. Capture from VHS tapes to AVI DV PAL using my Panasonic NV-HS960 VCR, Datavideo TBC-1000 and Canopus ADVC-300.
2. Edit DV footage using Adobe Premiere and/or VirtualDub
3. Convert from DV to MPEG-2 using Canopus ProCoder 3

Now I've set up an dedicated capture computer with my old ATI AIW 7500 capture card and I'm trying to capture with HuffYUV or Lagarith codec using VirtualDub. I've managed to set up the computer properly now so I'm able to capture both HuffYUV and Lagarith for more than 3 hours without any dropped frames or audio synch problems. So far, so good, BUT! My workflow doesn't work anymore. Canopus ProCoder have some serious issues with YUY2 footage, so I can't use it for Conversion anymore, and Adobe Premiere CS 5.5 won't work with the HuffYUV codec!

Lagarith seems to work better with Adobe. The lagarith files import fine and I'm able to edit them, but right now this evening I ended up with a corrupt file after exporting to Lagarith from Adobe Premiere :-( This happened without any warning from Adobe Premiere or Adobe Media Encoder. From Adobes point of view, everything during the export process went fine. But when I'm trying to convert the exported file to MPEG-2 using MainConcept Reference, MC Reference simply "shuts down" unexpextly during the process. The same happens in VirtualDub . When watching the file in VD, there are some corrupted frames and VD also shuts down at the same timecode as MC Reference :-( Now I have to do the whole work all over again... and maybe even more of my files exported from Premiere is also corrupt.

Right now I'm just so extremely sick and tired of this piece of SH#¤$£@/&(¤IT and I just want to go back to my old workflow, working with DV footage instead. It's SO MUCH EASIER and it just WORKS all the time without problems! I've never in my entire life experienced anything like this before with DV. I think it's really soon a question of going back to DV or going crazy!

But before I do, I give this thread a try.
Please people, let me know; WHAT AM I DOING WRONG? Why does this happen? How can I work with HuffYUV or Lagarith YUY2 footage in Adobe Premiere without experiencing these kind of problems? ...Or what other software should I use for editing my footage? What workflow are you guys using for capturing, editing and converting HuffYUV/Lagarith footage?

Here is a sample of one of the corrupted frames:


This is the error from VirtualDub:

[sanlyn]

Quote:

Originally Posted by hysteriah

Okey, here's the situation. I'm used to work with DV footage but I'm trying to move over to HuffYUV/Lagarith now... but it doesn't work with the same workflow as I'm used to... so I really need some help here.

Here's the workflow that I'm used to:
1. Capture from VHS tapes to AVI DV PAL using my Panasonic NV-HS960 VCR, Datavideo TBC-1000 and Canopus ADVC-300.
2. Edit DV footage using Adobe Premiere and/or VirtualDub
3. Convert from DV to MPEG-2 using Canopus ProCoder 3

Well, yeah, new methods and new media usually bring on a case of the startup stumbles. Nothing most of don't overcome, but certainly not amusing.

Your old workflow was OK, but....don't you do any cleanup of bad Levels or color correction? Denoise? The last step #3 is interesting, in that you say you "convert" to MPEG-2 from DV, but what you really do is re-encode from DV to MPEG-2. That makes at least two lossy encodes in your workflow. If you used any filters or made image mods in Adobe, add more lossy encodes.

Quote:

Originally Posted by hysteriah

Lagarith seems to work better with Adobe. The lagarith files import fine and I'm able to edit them, but right now this evening I ended up with a corrupt file after exporting to Lagarith from Adobe Premiere :-( This happened without any warning from Adobe Premiere or Adobe Media Encoder.

Don't you test small segments first to iron out glitches in your workflow and processing? BTW, you save unencoded Avi work files with Lagarith, you don't encode with Media Encoder and try to compress on top of that with another compressor. But I guess you realize that.

The main purpose behind capturing to lossless media is that you encode only once -- after you do any cleanup, color correction, or edits. Changes made to lossless media are saved as new, lossless output files. With DV, however, if you do any filtering or color work, then some cleanup, etc., you re-encode DV every time, period. No way around it. No exceptions. By the time you've re-encoded your DV captures, you've lost so much data and piled up so many compression artifacts, you might as well throw the results in the garbage. Use whatever software you want, or use any free software you want, but multiple re-encodes is the pits.

A question from my end: other than simple cut and join, what are you doing as "edits" in Adobe? Are you repairing the defects usually seen with VHS and VHs->DV captures -- like, e.g, line twitter, dropouts, halos, dct ringing, shimmer and other disturbed motion, etc. ? If so, how are you doing that with Adobe? Are you doing it in Avisynth? VirtualDub? Are you fixing invalid video levels and blown out brights in YUV before going to something like RGB for other work in Adobe? Adobe can work with Lagarith, so use Lagarith for lossless compression. The final results should be converted to YV12 for encoding (YV12 conversions are weak spots with Adobe, despite the Pro label. Avisynth and even VirtualDub do it better). The encoders sound OK, but even with a premium label they work only with YV12, which is default colorspace for MPEG/BluRay/AVCHD. And a lot of other encoders will insist on the same thing.

Quote:

Originally Posted by hysteriah

How can I work with HuffYUV or Lagarith YUY2 footage in Adobe Premiere without experiencing these kind of problems? ...Or what other software should I use for editing my footage? What workflow are you guys using for capturing, editing and converting HuffYUV/Lagarith footage?

I'll concede that huffyuv comes in so many flavors, it even gives my other PCs problems....though huff never gave me problems with my AfterEffects. The first thing I do is inspect the capture, open the file in Avisynth and do some preliminary denoise or levels testing, and save the work as iLagarith. No problems. Yeah, the question is, why not capture with Lagarith in the first place? Well, my ancient AMD capture PC is bit old and tired these days, and Lagarith ain't quite as fast as huffyuv for capture. It doesn't matter because the captures will go thru more than one phase of open/process/save-as-lossless anyway. Remember, I intend to use lossy encoding once, and only once, as the very last step before author and burn. So far in my workflow here, I have zero lossy encodes.

My lossless AVI's are run through any cleanup needed with Avisynth and/or VirtualDub (with VHS, it's always needed!). Sometimes it's cut into segments that need TLC or intensive care. I seldom run an entire video thru all processing in one-button "auto" mode, which is the Walmart/BestBuy/PCMag method and seems to be the way Joe Q. Consumer uses Premiere Pro. Cleanup and repair often require deinterlace and re-interlace operations (which Adobe doesn't do very well) for certain filters and certain problems, or working with field separation methods and/or inverse telecine (again, I wouldn't use Adobe for that, nor even my prized copy of AfterEffects), and involves colorspace changes from YUY2 to YV12 and/or RGB (Adobe doesn't do those very well, either).

Then the Lagarith stuff goes to any of several programs I use for color correction: VirtualDub's color filters, the multiple-colorspace filters in TMPGenc Plus 2.5 or TMPGenc Mastering Works, or AfterEffects Pro with Color Finesse and Colorista plugins. (Since you have Premiere Pro, you have excellent color features available in PP's advanced menus). If I need a multi-level timeline (I usually don't) I feed lossless to TMPGenc Mastering Works, whose MPEG/x264 encoders I often use. If I need fancier timeline work or any serious mods like color layer work or whatnot, lossless Lagarith goes to AfterEffects, but is saved as lossless for a better encoder. Several good encoders at hand: TMPGenc, and both HCenc and x264 in several other apps.

I don't know what the VirtualDub error concerning Lagarith is about. Mainly, I don't know what you were trying to open in VirtualDub or what had been done to it. VirtualDub has no problem opening properly compressed formats, providing the decompressor is in the system. I assume that when you save those lossless files earlier you're telling Lagarith what colorspace you want, right?

Oh, what do I use for cut band join? I can use all of the apps mentioned for those operations, in one way or another. For cutting up already-encoded video I use TMPGEnc Smart Renderer.



On the other hand.....If your purpose is more limited: "capture -> Adobe cut-join -> encode", you can always go back to DV and your usual methods. Of course, you'll get the same results. You can also use lossless AVI, as well. But you don't say what you're doing in Premiere that gets messages from media encoders. Aren't you just saving your output as lossless AVI? If you're working with lossless and you want to go straight to encoding, then you don't need Lagarith for that output.

[lordsmurf]

Quote:

Originally Posted by hysteriah

Here's the workflow that I'm used to:
1. Capture from VHS tapes to AVI DV PAL using my Panasonic NV-HS960 VCR, Datavideo TBC-1000 and Canopus ADVC-300.
2. Edit DV footage using Adobe Premiere and/or VirtualDub
3. Convert from DV to MPEG-2 using Canopus ProCoder 3

This is a good workflow. Keyword = "workflow".

The weak link here in the ADVC-300, which can actually be worse than the lowly 50/55 or 100/110
models. You'd think that a $300+ devices would be better than a $100+ devices, but you'd be wrong.

Quote:

Canopus ProCoder have some serious issues with YUY2 footage, so I can't use it for Conversion anymore,

I'm not sure if Procoder will still export directly from Premiere CS5.5 as a plugin, like it did with earlier Premiere versions (I've not tested that combo), but I never do that anyway. Premiere has always been somewhat unstable -- all NLEs are -- so I prefer to export as uncompressed or lossless AVI first. Then encode the export AVI separately in Procoder (my old method) or MainConcept (my new method). There's not going to be a quality loss from doing this.

Quote:

and Adobe Premiere CS 5.5 won't work with the HuffYUV codec!

This isn't right. Something is wrong. Premiere has zero issues with Huffyuv. Are you using Huffyuv, or the MT/64 versions. Go back to basics if trying the advanced versions.

Quote:

Lagarith seems to work better with Adobe. The lagarith files import fine and I'm able to edit them, but right now this evening I ended up with a corrupt file after exporting to Lagarith from Adobe Premiere

It's been years, but I vaguely recall running into this issue myself. I'm not fond of Lagarith.

Quote:

Adobe Media Encoder.

FYI, AME is an Adobe version of MainConcept. Reference is the full version, and AME is a limited SDK version. It does work well, especially if you don't have the full formerly $500 (now $2k!) Reference. FYI, again, Reference is now called Rovi Total Encode.

Quote:

Now I have to do the whole work all over again... and maybe even more of my files exported from Premiere is also corrupt.

Save, as save often. Every little step -- save it. I know it makes a huge glut on the drive, but that's the only way to not have to redo work. I often have mere single DVD project eat 300gb or more! I don't delete anything until it's done. I learned that the hard way.

Quote:

What workflow are you guys using for capturing, editing and converting HuffYUV/Lagarith footage?

- ATI AIW > Huffyuv (basic, non-MT/non-64) captured with VirtualDub
- Huffyuv > Premiere CS4 > uncompressed or Huffyuv output
- edited output to MainConcept Reference for DVD/BD MPEG or H.264 streaming

It works flawlessly, and has for several years now. Nothing is broke, so not fixing (upgrading) it.

Edit/add: I also tend to wonder if your error has nothing to do with codecs at all, and is actually a hard drive issue. Fragmentation can especially cause these issues. Also make sure it's not being saved to the OS drive, but a secondary internal (IDE/SATA) or eSATA drive, and not USB.

Quote:

Originally Posted by sanlyn

..don't you do any cleanup of bad Levels or color correction? Denoise?

This really is the main benefit of lossless capture. If the source tapes are perfect as-is, then DV is not a terrible choice. Still not best, but it can be fine. The worst aspect here is color loss, especially when that already-ruined 4:1:1 (DV) is converted to 4:2:0 (DVD-Video MPEG).

Quote:

but what you really do is re-encode from DV to MPEG-2. That makes at least two lossy encodes in your workflow. If you used any filters or made image mods in Adobe, add more lossy encodes.

^ This.

Quote:

test small segments first to iron out glitches in your workflow and processing

^ This.

Quote:

you don't encode with Media Encoder and try to compress on top of that with another compressor.

AME also exports to Huffyuv. So maybe, maybe not.

Quote:

The main purpose behind capturing to lossless media is that you encode only once -- after you do any cleanup, color correction, or edits. Changes made to lossless media are saved as new, lossless output files.

Not quite. You technically do re-encode every filter pass export, but it's nowhere near as harmful as the lossy DV/MPEG/H.264/etc encoding. Going from lossless to lossless, with filtering, should actually improve the quality of the video. You do want to keep re-encodes to a minimum, but lossless allows for some.

Quote:

With DV, however, if you do any filtering or color work, then some cleanup, etc., you re-encode DV every time, period. No way around it. No exceptions. By the time you've re-encoded your DV captures, you've lost so much data and piled up so many compression artifacts, you might as well throw the results in the garbage.

You can export DV to lossless, but some damage is already done. Same for any other format. Restoring becomes that much harder when digital compression has been applied to analog errors. Many are 100% impossible to remove when that happens.

Quote:

A question from my end: other than simple cut and join, what are you doing as "edits" in Adobe?

I would assume assembly ("make movie"), as Premiere has zero restoration effects beyond color correction.

Quote:

conversions are weak spots with Adobe, despite the Pro label.

"Pro" = they sell it for more money now. The end. I actually like Final Cut Pro better, and that's one of the reasons I have a Mac workflow. It's also more stable. Heck, even Premiere on Mac is more stable! And understanding that I'm mostly a Window user, and a Premiere user, that's saying something!

Quote:

seems to be the way Joe Q. Consumer uses Premiere Pro. Cleanup and repair often require deinterlace and re-interlace operations (which Adobe doesn't do very well)

After my stroke, when I was unable to work (and in some ways, still cannot), the studio where I was working had to hire somebody new. They went in-house, instead of doing the contract thing, like they'd done with me. My workflow for all of the content had involved a mix of QTGMC, NNEDI versions, and Yadif, when deinterlace was needed. What they do now is rely solely on Premiere, and some of their current work looks terrible as a result. At least they don't handle much older source anymore.

So it's not just John Q. Consumer, but "pros" that really are not experienced video pros at all.

Quote:

Then the Lagarith stuff goes to any of several programs I use for color correction: VirtualDub's color filters, the multiple-colorspace filters in TMPGenc Plus 2.5 or TMPGenc Mastering Works, or AfterEffects Pro with Color Finesse and Colorista plugins. (Since you have Premiere Pro, you have excellent color features available in PP's advanced menus). If I need a multi-level timeline (I usually don't) I feed lossless to TMPGenc Mastering Works, whose MPEG/x264 encoders I often use. If I need fancier timeline work or any serious mods like color layer work or whatnot, lossless Lagarith goes to AfterEffects, but is saved as lossless for a better encoder. Several good encoders at hand: TMPGenc, and both HCenc and x264 in several other apps.

It's nice to see that some still realize the value of TMPGEnc Plus 2.5.

Quote:

For cutting up already-encoded video I use TMPGEnc Smart Renderer.

Same here, for H.264. I still use Womble for MPEG.

[sanlyn]

Yes, lordsmurf mentioned (but I forgot) that DV can always be decoded into lossless media for detail work before final encoding later. And there's always the huffyuv-vs-lagarith-vs-other guys debate. I recall myself seeing issues with Win7/8 and huffyuv. I went to Lagarith because it seemed to fit everywhere.

But those are minor compared to the problems you describe. I'd start by asking for more step-by-step info...not the entire process, but what would be the first step or two that you want to work with your lossless captures? Better to address issues a step at a time.

I can't say I've seen the effects you posted, but you might want to ensure that your entire process with anything in Avisynth or VirtualDub is 32-bit. 64-bit might be nice (not necessarily faster), but there just isn't enough 64-bit filter development just yet.

[hysteriah]

Okey. Thank you VERY MUCH, both of you for trying to help me out. I'll try to give you some more details.

Quote:

Originally Posted by sanlyn

Don't you test small segments first to iron out glitches in your workflow and processing? BTW, you save unencoded Avi work files with Lagarith, you don't encode with Media Encoder and try to compress on top of that with another compressor. But I guess you realize that.

Yes, I do test small segments first. But this time, everything was looking fine. This is a more than 3 hour long videofile, digitized from an E-195 tape. This error with the corrupt frame occurs at timecode 02:07:12.240 and it's the first error I've noticed so far. Just a few seconds later (at timecode 02:07:34.800) MC Reference and VD shuts down unexpectedly. But you know, I can't go through all my videofiles frame by frame looking for errors like these every time I save/export my files??? Is there a way to automatically search through videofiles for errors like these, f.ex. with VirtualDub? I think I have to do something like that from now on before deleting the source files to be sure this doesn't happen again.

Quote:

Originally Posted by sanlyn

The main purpose behind capturing to lossless media is that you encode only once -- after you do any cleanup, color correction, or edits. Changes made to lossless media are saved as new, lossless output files. With DV, however, if you do any filtering or color work, then some cleanup, etc., you re-encode DV every time, period. No way around it. No exceptions. By the time you've re-encoded your DV captures, you've lost so much data and piled up so many compression artifacts, you might as well throw the results in the garbage. Use whatever software you want, or use any free software you want, but multiple re-encodes is the pits.

Yes, I understand this. That's the whole reason why I'm now dumping my whole DV workflow, moving over to working with lossless HuffYUV/Lagarith footage instead

Quote:

Originally Posted by sanlyn

A question from my end: other than simple cut and join, what are you doing as "edits" in Adobe? Are you repairing the defects usually seen with VHS and VHs->DV captures -- like, e.g, line twitter, dropouts, halos, dct ringing, shimmer and other disturbed motion, etc. ? If so, how are you doing that with Adobe? Are you doing it in Avisynth? VirtualDub? Are you fixing invalid video levels and blown out brights in YUV before going to something like RGB for other work in Adobe? Adobe can work with Lagarith, so use Lagarith for lossless compression. The final results should be converted to YV12 for encoding (YV12 conversions are weak spots with Adobe, despite the Pro label. Avisynth and even VirtualDub do it better). The encoders sound OK, but even with a premium label they work only with YV12, which is default colorspace for MPEG/BluRay/AVCHD. And a lot of other encoders will insist on the same thing.

Well, my edits in Adobe Premiere is in fact mainly simple cut and join actually. But I very often need to insert footage from one file over the audio from another and also mixing different audio tracks together. My projects can become quite complex so it's very convenient to use Adobe Premiere where I can work with multiple audio and video tracks. Usually, that's all I ever do in Adobe Premiere, then I export my sequence to AVI DV (probably without any loss because all that's really been done to the footage is simple cut and join), then I import it to VirtualDub where I add filters and do some more cutting (spliting it up in shorter pieces). The only filtering I do at this point is mainly adding the NeatVideo Noise Reduction filter, but I'm going to keep the lossless file now for adding more filters in the future.

The exception to this workflow may be a few clips where I need to do color corrections. I prefer doing that in Adobe Premiere. Then I usually do both the color correction and adding NeatVide filter in Adobe Premiere and export directly to MPEG-2 from the Adobe Premiere sequence.

I tried doing all the cutting in VirtualDub, but after appending a hugh number of AVI segments from a whole bunch of different files, I ended up with audio and video out of synch for some reason. Why?

Quote:

Originally Posted by sanlyn

I'll concede that huffyuv comes in so many flavors, it even gives my other PCs problems....though huff never gave me problems with my AfterEffects. The first thing I do is inspect the capture, open the file in Avisynth and do some preliminary denoise or levels testing, and save the work as iLagarith. No problems. Yeah, the question is, why not capture with Lagarith in the first place? Well, my ancient AMD capture PC is bit old and tired these days, and Lagarith ain't quite as fast as huffyuv for capture. It doesn't matter because the captures will go thru more than one phase of open/process/save-as-lossless anyway. Remember, I intend to use lossy encoding once, and only once, as the very last step before author and burn. So far in my workflow here, I have zero lossy encodes..

I started capturing as HuffYUV, then transform to lagarith when editing... but it seems my Intel P4 2,8Ghz capture PC can handle capturing straight to lagarith without problems. During capture, the CPU usage is stable around 50% and I haven't yet had any repported dropped frames.

Quote:

Originally Posted by sanlyn

My lossless AVI's are run through any cleanup needed with Avisynth and/or VirtualDub (with VHS, it's always needed!). Sometimes it's cut into segments that need TLC or intensive care. I seldom run an entire video thru all processing in one-button "auto" mode, which is the Walmart/BestBuy/PCMag method and seems to be the way Joe Q. Consumer uses Premiere Pro. Cleanup and repair often require deinterlace and re-interlace operations (which Adobe doesn't do very well) for certain filters and certain problems, or working with field separation methods and/or inverse telecine (again, I wouldn't use Adobe for that, nor even my prized copy of AfterEffects), and involves colorspace changes from YUY2 to YV12 and/or RGB (Adobe doesn't do those very well, either).

I understand that. I'm not very good at restoration work on footage from VHS tapes right now at this moment. So all I ever do at this point is adding some Noise Reduction filter, just to make it "viewable"... but I'm storing the lossless videofiles on external HDD for more Advanced restoration work in the future. I don't have time for that now anyway. Just have to digitize my whole VHS collection with best possible quality now while there's still possible to get VCR's with deasent quality and my ATI AIW card still works

Quote:

Originally Posted by sanlyn

Then the Lagarith stuff goes to any of several programs I use for color correction: VirtualDub's color filters, the multiple-colorspace filters in TMPGenc Plus 2.5 or TMPGenc Mastering Works, or AfterEffects Pro with Color Finesse and Colorista plugins. (Since you have Premiere Pro, you have excellent color features available in PP's advanced menus). If I need a multi-level timeline (I usually don't) I feed lossless to TMPGenc Mastering Works, whose MPEG/x264 encoders I often use. If I need fancier timeline work or any serious mods like color layer work or whatnot, lossless Lagarith goes to AfterEffects, but is saved as lossless for a better encoder. Several good encoders at hand: TMPGenc, and both HCenc and x264 in several other apps.

Thank you SO MUCH for sharing your workflow

Quote:

Originally Posted by sanlyn

I don't know what the VirtualDub error concerning Lagarith is about. Mainly, I don't know what you were trying to open in VirtualDub or what had been done to it. VirtualDub has no problem opening properly compressed formats, providing the decompressor is in the system. I assume that when you save those lossless files earlier you're telling Lagarith what colorspace you want, right?

All I did in Adobe Premiere is this I told you. Simple cut and join on a file captured with the lagarith codec. Then I export the sequence from Adobe Premiere using the sequence settings and export settings on the attached images. The only reason for my problem that I can think of is that I was editing the file on a external USB 3.0 HDD, exporting it to internal drive. I've never had any problems working on external drives when working with the DV format... but maybe lossless is to big? But anyway, I don't understand why Adobe Premiere didn't return any error during the export process? After experiencing this problem, the Whole process feels very unreliable

Quote:

Originally Posted by sanlyn

On the other hand.....If your purpose is more limited: "capture -> Adobe cut-join -> encode", you can always go back to DV and your usual methods. Of course, you'll get the same results. You can also use lossless AVI, as well. But you don't say what you're doing in Premiere that gets messages from media encoders. Aren't you just saving your output as lossless AVI? If you're working with lossless and you want to go straight to encoding, then you don't need Lagarith for that output.

Yes, but adding Noise Reduction and more filtering together with fixing tape errors and more advanced color correction will be necessary in the future to make these footage look good. As you say: with VHS, it's always needed!. If I'm ever going to do that in the future, only time will tell... but I really want to keep that option ;-)

I will answer "lordsmurfs" questions in another post

[hysteriah]

Quote:

Originally Posted by lordsmurf

This is a good workflow. Keyword = "workflow".

The weak link here in the ADVC-300, which can actually be worse than the lowly 50/55 or 100/110
models. You'd think that a $300+ devices would be better than a $100+ devices, but you'd be wrong.

Thanks. Yes, I've noticed that the ADVC-300 isn't such a hi quality unit as I thought it was when bying this piece of crap I also have the TwinPact 100 and a Sony camcorder that also converts analog to DV, but they're still DV and not much better

Quote:

Originally Posted by lordsmurf

I'm not sure if Procoder will still export directly from Premiere CS5.5 as a plugin, like it did with earlier Premiere versions (I've not tested that combo), but I never do that anyway. Premiere has always been somewhat unstable -- all NLEs are -- so I prefer to export as uncompressed or lossless AVI first. Then encode the export AVI separately in Procoder (my old method) or MainConcept (my new method). There's not going to be a quality loss from doing this.

I think the Procoder plugin for Premiere was quit after Premiere version CS3 or CS4 or something. Anyway, this part of my workflow isn't the biggest problem cos I think I'm moving over to MainConcept instead of Canopus

Quote:

Originally Posted by lordsmurf

This isn't right. Something is wrong. Premiere has zero issues with Huffyuv. Are you using Huffyuv, or the MT/64 versions. Go back to basics if trying the advanced versions.

Are you sure? I've tried both the 64-bit version, the basic version and the MultiThread version of HuffYUV. None of them will work with my version of Adobe Premiere CS 5.5. When importing a HuffYUV clip, all I ever see in Premiere is the audio This is on a Windows 7 64-bit computer with 64-bit version of Adobe Premiere. I think it works fine on 32-bit OS and software?

Quote:

Originally Posted by lordsmurf

It's been years, but I vaguely recall running into this issue myself. I'm not fond of Lagarith.

Hmmmm... so you think it the lagarith codec that's causing my problem? What to do when HuffYUV won't work???

Quote:

Originally Posted by lordsmurf

FYI, AME is an Adobe version of MainConcept. Reference is the full version, and AME is a limited SDK version. It does work well, especially if you don't have the full formerly $500 (now $2k!) Reference. FYI, again, Reference is now called Rovi Total Encode.

Thanks. I think I'm going to skip Canopus ProCoder and move over to AME or MainConcept now

Quote:

Originally Posted by lordsmurf

- ATI AIW > Huffyuv (basic, non-MT/non-64) captured with VirtualDub
- Huffyuv > Premiere CS4 > uncompressed or Huffyuv output
- edited output to MainConcept Reference for DVD/BD MPEG or H.264 streaming

It works flawlessly, and has for several years now. Nothing is broke, so not fixing (upgrading) it.

That's the exact same workflow as I want to use... if only my HuffYUV would work on my 64-bit system *ARGH*

Quote:

Originally Posted by lordsmurf

Edit/add: I also tend to wonder if your error has nothing to do with codecs at all, and is actually a hard drive issue. Fragmentation can especially cause these issues. Also make sure it's not being saved to the OS drive, but a secondary internal (IDE/SATA) or eSATA drive, and not USB.:

Hmm... why not USB 3.0? I don't have eSATA on any of my computers and it's a pain in the a#¤%&$£@ss moving internal drives from one computer to another. I'm using a dedicated PC for capturing and another PC for editing. I must move the file from my capture PC to my editing PC one way or another. I guess editing video over ethernet using gigabit ethernet adapters won't be any better than using USB? These lossless files are so hugh it takes ages just to copy the file from one internal drive to another. What to do???

Quote:

Originally Posted by lordsmurf

This really is the main benefit of lossless capture. If the source tapes are perfect as-is, then DV is not a terrible choice. Still not best, but it can be fine. The worst aspect here is color loss, especially when that already-ruined 4:1:1 (DV) is converted to 4:2:0 (DVD-Video MPEG).

Well, I'm on pal so the worst aspect of it (the 4:1:1 to 4:2:0 conversion) is not a big issue in my case, since DV PAL is already 4:2:0. But anyway, it is required with noise reducion and also correcting other errors on most of my recordings. Most of my tapes are in pretty bad shape

Once again; THANK YOU SO MUCH BOTH OF YOU for your great input. I would never been able to do this without the help from you guys

[hysteriah]

Quote:

Originally Posted by sanlyn

Yes, lordsmurf mentioned (but I forgot) that DV can always be decoded into lossless media for detail work before final encoding later. And there's always the huffyuv-vs-lagarith-vs-other guys debate. I recall myself seeing issues with Win7/8 and huffyuv. I went to Lagarith because it seemed to fit everywhere.

But those are minor compared to the problems you describe. I'd start by asking for more step-by-step info...not the entire process, but what would be the first step or two that you want to work with your lossless captures? Better to address issues a step at a time.

I can't say I've seen the effects you posted, but you might want to ensure that your entire process with anything in Avisynth or VirtualDub is 32-bit. 64-bit might be nice (not necessarily faster), but there just isn't enough 64-bit filter development just yet.

I think I've already answered most of it, but I guess it's much clearer if I list up a step by step list of what I'm doing this way:
1. I capture with my dedicated capture computer using VirtualDub - YUY2 lagarith codec (P4 2,8Ghz, 1GB RAM, ATI AIW 7500, SoundBlaster Audigy, 200GB IDE harddrive)
2. I open the captured file in VirtualDub on my dedicated editing computer (64-bit Win7 Intel I7). I open it straight from the capture PC using gigabit network connection. Then I simply use VirtualDub to copy the file (direct stream copy) from my capture PC to my editing PCs internal harddrive for "editing" with Adobe Premiere.
3. THIS PROCESS ACTUALLY SEEMS TO WORK FINE! I haven't noticed any problems with my workflow so far.
4. Then I import that file into Adobe Premiere for editing (advanced cutting) using the sequence settings as the attachement in my 2nd post (post #5).
5. After the editing in Adobe Premiere, I export my sequence using the export settings like the attachement in my 2nd post (post #5). I export the file directly to my External USB 3.0 HDD for permanent storage. From this file I want to do more advanced editing/restortion in the future. it's not yet added any filters, only cuting and joining so it's still "untouched" footage as far as I know.
6. THIS IS WHERE IT WENT WRONG. THIS FILE ON MY External HDD is "corrupt" without any warning from Adobe Premiere!
7. This file should now be my "master file" that I want to work with in VirtualDub. Now I want to add filters, and cut into pieces and/or convert to MPEG-2 using MainConcept Reference or AME. And I have to make sure this file works properly so I can delete the source file from my dedicated capture computer so I can capture new stuff to the very limited 200GB IDE disk in that computer. This is a old computer with AGP port and no S-ATA connection

[sanlyn]

You made mention several times of your input "capture file". Is this capture from PAL VHS? I noted on your AP output image that you specify bottom field first. PAL and NTSC VHS are usually top field first, not bottom field. DV is almost always bottom field first. Your output is sized to 704x576 -- is this the frame size you captured?

You also mentioned something about MC Reference and VD shutting down at the same point. Do you mean both are running at the same time, i.e, encoding to an encoder right out of VDub's output? People do that, but I never cared for it. I run VDub and encoders separately. Or maybe I'm confused about how you're using them.

People have reported problems with Win7 and USB 3.0 external drives (file corruption, including data files). If your external USB drive is USB-powered instead of an A.C. powered external container, that could be another problem. All I do with external drives is copy to or from. I don't feed files to external drives during processing.

NeatVideo works only in RGB. There's nothing wrong with correcting color in Premiere, but the problem becomes the sloppy way Premiere converts to RGB. VirtualDub doesn't get it exactly right either, depending on certain factors related to the source file. I always make initial colorspace conversions in Avisynth, whiuch has settings for frame structure and color matrix (yes, these do matter when converting colorspace).

Audio sync with joined files depends a lot on how your initial cuts were made. Audio is usually stored as an offset to video (audio occurs earlier than video, similar to the way it does on movie film). Different files, different captures, etc., have different audio offsets and delay. I make cuts and joins together in the same app. If I do have to have such a cut made at different times in differemnt apps, I include some trailing frames to make sure the joiner can match the audio between sources. Avisynth has similar concatenation settings; they differ depending on how you want audio to match up between segments. Avisynth has provision to adjust audio sync uniformly between segments. How Adobe does this is not familiar to me, and how VirtualDub does it depends, as I said, on audio delay characteristics from different segments.

I tend to make all of my cuts in either Avisynth or VirtualDub, but I keep each segment separate for processing, leave some trailing space to catch any audio glitches, then join them later. Yes, this takes more time. But I don't have audio sync or color matching problems between segments. This is especially true of segments joined from different captures, less so with segments from the same capture. If I'm working with a timeline I join segments in the timeline (after all other processing), then encode from there.

Can't come up with more detail offhand. Just started another capture and have to keep my eyes on it for now.

[Oops. Forgot again!] Audio is usually captured as uncompressed PCM. It stays that way until the final encode. Avoid capping or changing audio to compressed formats like C3 or mp2 during processing. I don't demux audio and video during cutting. Had too many problems doing it that way.

[hysteriah]

Quote:

Originally Posted by sanlyn

You made mention several times of your input "capture file". Is this capture from PAL VHS? I noted on your AP output image that you specify bottom field first. PAL and NTSC VHS are usually top field first, not bottom field. DV is almost always bottom field first. Your output is sized to 704x576 -- is this the frame size you captured?

Yes, it's PAL capture from PAL VHS tapes at 704 X 576 resolution which I understand is the native hardware resolution of my ATI AIW 7500 capture card... and yes, it's upper field first. If you see the "AP Sequence settings.jpg" and "AP Export settings.jpg" in post #5, they are both set to "Field Type: Upper First".

Quote:

Originally Posted by sanlyn

You also mentioned something about MC Reference and VD shutting down at the same point. Do you mean both are running at the same time, i.e, encoding to an encoder right out of VDub's output? People do that, but I never cared for it. I run VDub and encoders separately. Or maybe I'm confused about how you're using them.

Oh, I see this is a littlebit confusing the way I explained it. No, I didn't run MC Reference and VD at the same time. I wasn't using VD at all in this process. I was just trying to convert the file exported from Adobe Premiere with MC Reference. But when MC Reference crashed at timecode 02:07:34.800, then I just rebooted my computer and tried opening the same file in VirtualDub just to try identify the problem, trying to solve out if it was a problem with my MC Reference software or if it was a error in my videofile causing the software to crash. When VD also crashed at the exact same timecode just by playing the file at the specific timecode, then I think it's pretty clear that it's something terribly wrong with my file at that specific timecode.

Quote:

Originally Posted by sanlyn

People have reported problems with Win7 and USB 3.0 external drives (file corruption, including data files). If your external USB drive is USB-powered instead of an A.C. powered external container, that could be another problem. All I do with external drives is copy to or from. I don't feed files to external drives during processing.

I see. Guess I have to try skipping the external HDD and just working on internal HDD to see if that eliminates the problem, and then copy the file to external HDD for permanent archiving at the end. Gosh, I was hoping for a way to work with these extremely large files without having to copy them to many times from one drive to another... that's extremely timeconsuming... but I guess it's not possible?

Quote:

Originally Posted by sanlyn

NeatVideo works only in RGB. There's nothing wrong with correcting color in Premiere, but the problem becomes the sloppy way Premiere converts to RGB. VirtualDub doesn't get it exactly right either, depending on certain factors related to the source file. I always make initial colorspace conversions in Avisynth, whiuch has settings for frame structure and color matrix (yes, these do matter when converting colorspace).

Yes, I realize that I have to learn more about AviSynth. That's the best way to get decent quality out of these shitty old VHS tapes. But right now I don't have time for that. Right now I just have to concentrate on getting my tapes digitized with best possible quality and store them as lossless files on external HDD until I'm able to do more advanced restoration work on them later

Quote:

Originally Posted by sanlyn

Audio sync with joined files depends a lot on how your initial cuts were made. Audio is usually stored as an offset to video (audio occurs earlier than video, similar to the way it does on movie film). Different files, different captures, etc., have different audio offsets and delay. I make cuts and joins together in the same app. If I do have to have such a cut made at different times in differemnt apps, I include some trailing frames to make sure the joiner can match the audio between sources. Avisynth has similar concatenation settings; they differ depending on how you want audio to match up between segments. Avisynth has provision to adjust audio sync uniformly between segments. How Adobe does this is not familiar to me, and how VirtualDub does it depends, as I said, on audio delay characteristics from different segments.

I tend to make all of my cuts in either Avisynth or VirtualDub, but I keep each segment separate for processing, leave some trailing space to catch any audio glitches, then join them later. Yes, this takes more time. But I don't have audio sync or color matching problems between segments. This is especially true of segments joined from different captures, less so with segments from the same capture. If I'm working with a timeline I join segments in the timeline (after all other processing), then encode from there.

Okey, I see. It doesn't seems like there's any audio synch issues when I'm using Adobe Premiere, and that's really the NLE I want to use for cutting my videos the way I do... that's what I'm used to when working with DV also... so if audio synch issues occure in VD really doesn't bother me that much... if only Adobe Premiere would work properly without giving me corrupt files

Quote:

Originally Posted by sanlyn

[Oops. Forgot again!] Audio is usually captured as uncompressed PCM. It stays that way until the final encode. Avoid capping or changing audio to compressed formats like C3 or mp2 during processing. I don't demux audio and video during cutting. Had too many problems doing it that way.

Yes, I'm also keeping both Audio and Video uncompressed (without demux) until the final encode

I'm gonna try skipping using the external USB 3.0 HDD now... see if that works better
Thank you once again so much for your wonderful help

Just one more thing: Is there a reliable way to automatically scan through videofiles for errors?

[hysteriah]

Just a short update:

I stoped using my external USB3.0 HDD. After capturing, I simply copied the file from my capture PC to my editing PC's internal harddrive. Then edited with Adobe Premiere. But after exporting from Adobe Premiere, this is what happened:


Then we know it's not related to using external USB3.0 HDD.
I guess it must be the lagarith codec that is causing it for some strange reason?

DAMNIT! How can I get HuffYUV to work with Adobe Premiere CS5.5 64-bit? Anyone knows???

[sanlyn]

Obviously, then, something's screwy. On the machine where you have Adobe installed, how does that same AVI look in media players and in VirtualDub? How did the files transferred from your capture PC to the Adobe PC? Do you get any error messages? Do you have a MediaInfo report on the AVI shown in the image posted?

An acquaintance uses huffyuv and Lagarith, both, in Adobe Premiere Pro, and I use both in AfterEffects. We're using the two compressors in other editors as well (VirtualDub, TMPGenc TVMW), and via Avisynth to HCenc encoder and the Avisynth plugin for AfterEffects as well as AE direct import.

[hysteriah]

What version of Adobe Premiere pro are you using? I think I can get HuffYUV to work with Premiere CS4 and earlier that is 32-bit software... but I've been "adicted" by Adobe Premiere's "Mercury Playback Engine" that is integrated to the later 64-bit versions of Premiere Pro. But I can'can't get it to work with the HuffYUV codec. II've tried both the regular 32-bit version, the 64-bit version and the mulithreding version of HuffYUV. None of them is working with my Premiere Pro CS5.5

[NJRoadfan]

Using avisynth to handle the decoding usually works when NLEs screw it up.

I was going to recommend the AviSynth plug-in for Premiere, but it may not work with newer versions: http://pwolfamv.com/programs/csavs64/

This may work if the above plug-in is incompatible with CS6: http://www.videohelp.com/tools/Avisy...al-File-System

The Mercury Playback Engine is of marginal use with SD video, the machine is more then fast enough to decode and playback SD video without a problem.

[hysteriah]

Thanks!

When I've digitized a tape, I like to fast farwarding through the whole file on 4X speed, looking for errors. If I find any errors (I usually do), then I digitize those parts once more. Most of the times, the errors gets less anoying the 2nd time.

I've tried to use Adobe Premiere Pro CS4, but it has very jerky playback at 4X speed :-( In Premiere Pro CS5.5 with "The Mercury Playback Engine", it's flawless!

[NJRoadfan]

Personally I use VirtualDub for scrubbing capture files, its one thing it does well. CS4 is suited for NLE tasks only here. I discovered the Avisynth plug-in after I encountered problems with CS4 not being able to properly speed up video footage for a time lapse driving video. I landed up creating an AviSynth script to do it and a de-interlace pass (another thing CS4 screws up) and then imported that output into my CS4 timeline.

[hysteriah]

I think it's easier in Premiere. Then I can just import the new clip and add it to the sequence right away.

How is it possible to play at 4X speed with sound in VirtualDub? I have to listen for "audio-drops" also (switching between stereo and mono)

[NJRoadfan]

One other thing you can try. DebugMode Frameserver: http://www.debugmode.com/frameserver/

It allows you to "export" the Premiere timeline to another application without saving a file. You can feed the raw output directly into VirtualDub (or any other video program, compressor, etc) this way. That avoids having to use HuffYUV or Lagarith. I use DebugMode to compress to formats not supported by Premiere's built in compressor, or to do a final de-interlace pass.

[hysteriah]

Okey. Looks great!
I'm gonna check them out
Thanks a lot!