The effect of noise on compression?

[dpalomaki]

As an illustration of the effect of noise on compression of a file I tried a wee experiment.
I used audio files and WinZIP for this test. While these are not the same as video and lossy video compression schemes like DV or MPEG, it will give a feel for the impact of noise on file compressibility, and they were easy to use.

I used Adobe Audition 3.0 to create a 10-second, 400 Hz, tone and saved it as a 44.1 kHz stereo WAV file.
The tone was recorded at -6 dB.
I then added pink noise to the tone. The pink noise had average amplitude of -40 dB for one file, and -47 dB for another, about the same and better than the S/N ratio for S-VHS. And saves the file

The three .WAV (uncompressed) files were 1,726 KB size.
I then used WINZip to (losslessly) compress the files.

The clean tone file (no added noise) compressed to 459 KB.
The file with pink noise at -40 dB added compressed to 1,672 KB.
The file with pink noise at -47 dB added compressed to 1,659 KB, better but not drastically so.

The bottom line is that a file containing noise is more difficult to compress than a file without noise. This illustrates why lossy capture formats are not good for noisy sources such as consumer analog video, and why a simple dump of home video to bandwidth limited distribution media, like DVD, is often disappointing.

[lordsmurf]

The same is true of video, not just audio.

[dpalomaki]

As a continuing experiment I did a quick look at the impact of noise on on video compression, as measured by the resulting size of a lossless compressed video file.
Conditions were using Edius 9 Workgroup NLE
5 seconds of standard NTSC (SD) color bars and tone as generated by Edius.
Tested Pure bars and tone and Ignite Noise filter applied to the color bars at levels of 0, 10, and 20.
And a video of humans with the Ignite Noise filter for a more complex image

The clips were exported to an AVI file using Grass Valley Lossless codec.
The resulting file sizes:
Pure bars & tone - 18 MB
Bars & tone With Ignite noise set to 0 -- 21 MB
With Humans, Ignite Noise set to 0 -- 42 MB
With Ignite Noise (both cases) set to 10 -- 58 MB
With Ignite Noise (both cases) set to 20 -- 69 MB

Bars and tone are a simple image that will compress nicely.
The Ignite filter, even at 0 noise setting adds some shadow background to the bars that can be seen in the image.
A JPG of Ignite Noise set to 10 is attached to give a feel for the amount of noise added the image. Subjectively I don't believe this is an unrealistic amount of noise for typical home video shot with less than good lighting.
As a point of reference, exporting to DV format AVI results in a 18 MB file no matter the content.

Conclusion. In this simple test with human image content the 10 setting noise added nearly 40 to the bandwidth/storage space required for lossless digitization. If bandwidth is constrained there can be a corresponding loss of image information to noise.

[Feedbucket]

This is pretty much compression 101, no? The less complex the data is, the easier it is to compress.

Basically, a string like "AAAAAAAAAA" can be represented as "Ax10" whereas "ABCDEFGHIJ" needs to remain intact if you want to retain integrity. "Lossy" compression could turn it into something smaller like "Ax5,Fx5" but then you no longer have the BCDEGHIJ info and it would render as "AAAAAFFFFF".

[dpalomaki]

A bit like the old PCX file type compression.

Or maybe "Cx5, Hx5" which reduces the overall error.
Ax5 has a total error over the run of 0+1+2+3+4 = 10
Cx5 has a total error over the run of 2+1+0+1+2 = 6
thus compression method, and data being compressed makes a difference.

Since video is complex, and noise is an added unpredictable factor I thought it would be interesting to see the effect of noise, and better appreciate why lossy compression of old consumer analog formats looks so bad.