Cleaning up VHS capture

[fberreth]

Hi there,

I'm converting my VHS movies into MPEG4 movies. As you know, VHS movies have some problems with picture quality, mainly rainbow effects (color bars overlapping the video) and noise.
I read a lot also in this forum and tried pretty much all available virtualdub filters.
I noticed temporal filters (smoothers etc) will produce noticable artifacts in some scenes. Example: You have a face filling the screen moving a little up/down causes a lot of artifacts since the skin color is close frame/frame compared but from different locations in the face.
I noticed flaXen VHS filter or chroma noice red. also cause artifacts, especially on scene changes.
The best noise red. filter I could find is the smart smoother HiQ but even that one I'm not sure if I should use, it's very slow and really doesn't reduce the filesize that much if you also resize the movie.

So currently I decided to capture at 768x576 (PAL) and resize/crop to 480x564. No filters. Maybe I use smart smoother HiQ which decreases file size by about 10% but increases encoding time 4x. The percieved quality I think is better without any smoothing (somehow I have the feeling there is more detail in the picture even though there is quite some noise).

I would like to know if there is something I missed? Would you recommend just a resize (lacz...somthing3 method) or applying some filters too? I really would like to get rid of the color artifacts too ...

What are your experiences?

[wotef]

make a small but representative raw, unfiltered clip available

[^^-+I4004+-^^]

and wotef will denoise it for you!
aren't you lucky or what?
hehe

make it small enough and i'll have a go too......
(smal enough for modem that is...)

[fberreth]

OK, I've created a small movie clip. Please have a look at

http://www.winisp.net/phoenix/doom9.avi for a 35MB Huffyuv AVI
http://www.winisp.net/phoenix/doom9q2.avi for the same movie encoded with XviD quant. 2 (3.2MB)
http://www.winisp.net/phoenix/doom9q1.avi for the same movie encoded with XviD quant. 1 (8.2MB)

Thanks for your help,
Frank.

[scharfis_brain]

I've tested a little bit, and found those scripts useful:
For DivX-encoding
LoadPlugin("c:\dvdrip'in\convolution3d.dll")
LoadPlugin("c:\dvdrip'in\chromashift.dll")

avisource("D:\temp\doom9.avi")
converttoyuy2()
crop(8,4,-8,-12)
convolution3d(0,3,7,5,12,10,0)
levels(0,1.3,248,0,255)
chromashift(L=-2)
lanczosresize(512,384) #or any other Rez you want

or this one for SVCD-encodingLoadPlugin("c:\dvdrip'in\convolution3d.dll")
LoadPlugin("c:\dvdrip'in\chromashift.dll")

avisource("D:\temp\doom9.avi")
converttoyuy2()
crop(6,4,0,-12)
convolution3d(0,3,7,5,12,10,0)
lanczosresize(480,560)
levels(0,1.3,248,0,255)
chromashift(L=-2)
addborders(0,8,0,8)

IMO SVCD is more suitable for archiving VHS, because it preserves the most resolution of VHS.

[wotef]

ha ha ivo, i took you off my plonk list to read your post :D

actually this clip isn't too bad for vhs, so i think this is ok

loadplugin("c:\program files\avisynth2\plugins\dustv5.dll")
loadplugin("c:\program files\avisynth2\plugins\unfilter.dll")
avisource("E:\doom9.avi")
converttoyuy2()
unfilter(20,20)
pixiedust()
crop(8,0,752,564)
bilinearresize(576,432)

surprised there are no interlacing artifacts, though

[scharfis_brain]

surprised there are no interlacing artifacts, though
why should it interlace?
if the film is correct mastered then you'll never get interlace with PAL.

[wotef]

i was referring to combing artifacts, which you can certainly get on VHS pal

[fberreth]

So I was experimenting with those avi-scripts. It's easier than I thought (never did avisynth before).
Convolution3D at least with these settings doesn't seem to produce the artifacts that other temp. smoothers produce and it looks like it eliminates a little the color problems that the face for instance has.
I noticed that you use chromashift. Is that for all movies the same chroma shift (2 vertical lines up)?
Also, you use level to adjust the gamma. Why don't you use ColorYUY2? wotef, you didn't use gamma correction, would you do that at playback? For me the movie looks right without gamma correction but maybe I have my monitor's brightness/contrast too much cranked up?

[^^-+I4004+-^^]

>i was referring to combing artifacts, which you can certainly get on VHS pal

no,wotef,scharfis is right:proper film->pal conversion and every field gets to proper spot in particular frame and there's no combing!
ie.usually i don't need telecide() on my film-source analog captures!
it's captured without combing.......
(a small explanation for your "i was surprised"..hehe)

off course video sources pal is always combed!

>I noticed that you use chromashift. Is that for all movies the same chroma shift (2 vertical lines up)?

i was wondering about it too;
scharfis,who shifted your chroma so you're bringing it back now?

>For me the movie looks right without gamma correction but maybe I have my monitor's brightness/contrast too much cranked up?

for me too,but it's the "scharfis_brain"..heheh
also i still say that "tweak" can do all that levels can and it's easier to use....(what input/output?come on.......)
image too bright/too dark and levels can't repair it too.....

i'm dloading xvid sample so i'll give it a shot..probably will do few screenshots comparison..........

huffyuv on cca. 10MB/sec...huh,definitely NOT for modem..hehe
on the other hand mjpeg on 2-3MB/s would do much better,but doesn't matter now.............

[wotef]

well, i got lots of badly mastered film pal tapes here then, all kinds of crap, the majority of which have combing...

anyhoo, re: gamma and so on, i think this clip is fine colour-wise

if i thought it did need tweaking, i would have done it here, not in post-processing - i hardly ever use post-processing filters, actually...i tend to prefer to sort out as many quality issues in filtering prior to encoding

now waiting for ivo's contribution :cool:

[scharfis_brain]

Nearly EVERY VHS-Tape introduces a chromshift of 2 or 4 lines vertikal. This is why the PAL-Standard delays the chroma-information of one line. With every copy from VHS to VHS (The next generation, hehe) you'll get you chroma shifted a second time down by 2 or 4 lines. I've seen 4th generation copied tapes. The chroma was shifted extremely.:mad: .

In this Videos, I've used the yellow Traffic-sign to apply this chromshift. I use it everytime, when I capture VHS.

hm.. Levels.. That's up to you, whether you use it, or not.

[^^-+I4004+-^^]

>I've seen 4th generation copied tapes

how did you saw anything at all on 4th generation?
(hehe)

>This is why the PAL-Standard delays the chroma-information of one line.

hmmm..are you sure?PAL takes 2 lines(one has 180 phase offset) and produces "good line" by merging these 2..i think "wrong" color lines never get to the screen...
this lousy PAL way (that lets color mix in viewers eye) is only done in "simple PAL" way..i belive no one uses it these days....

perhaps vhs chroma shift is caused by "delay line" imperfections;dl is used because luma and chroma don't have same processing duration so luma needs to be slowed down etc.
anyhow,i didn't spot this few lines offset,but if you say it exists,then it exists...(at least on your VCR...hehe)
also,vhs gives luma less bandwidht so anything can happen...
if you have any interesting links,please provide.....
(i'll check my books on this)

>well, i got lots of badly mastered film pal tapes here then

interestingly,most vhs rent tapes i tried were prety good in that manner...(no combing after the capture..no progr. frames recovery needed)


>now waiting for ivo's contribution

as i said,i dloaded the xvid sample (quant2) because it was smallest..hehe
and (wait for it...hehe)...i would only do some CNR on this clip and some SHARPENING for sure!
my god this is blurry!even for pre-recorded vhs seems bit too soft (if this xvid can be used for any reference? is it at least mpeg quantization model? let me hope it's not h263...)

i see only very mild chroma noise and soft image....my encodings usually look much sharper even after extensive denosing.......
what capture device was used?
philips or ati chips perhaps?

i would:
1-cnr
2-resize to 574x432 or 640x480
3-unsharp mask to sharpen it a bit.....

if your huff and xvid samples differ greatly on sharpness,please provide some screenshots so i can take that into account (it's expected that xvid will blur somewhat,but this seems too much..)
768x576 must be sharp as hell for most of the time!
and you said:
>I'm converting my VHS movies

so i presume it's your tv recordings you're ripping(?)

on the other hand pre-recorded vhs' tends to be blurry......

(i have used directshowsource and relied on ffdshow to do proper decoding as i'm not an xvid fan)

also,there's constant motion in the clip,so hard to get any denoiser to do it properly....denoisers should be tweaked for still scenes and then you shoudl count on soem ghosting on fast ones..(ie. when still scene loses all the noise,you're something around good value)

i tried "dust" filters and it's slow (internal 2 pass) and it still blurred (probably i expected too much ...) so i'm staying on grapes and peaches....

huh gotta hurry,i have to provide samples for graft to improve a nice filter!

[scharfis_brain]

In every PAL-TVset exists a chroma-delay, to add the 180-degree-line to the 0-degree-line (you have to memory this one line), because of this there will be a chromashift.

Recording: PAL -> VCR -> demodulator (chromashift by one line) -> tape
Playback: tape -> modulator(next chromashift by one line) -> VCR -> PAL

I hope, that will clearup this a little bit.

[fberreth]

Let's see if I can answer all questions:
^^-+I4004+-^^, I did encode with MPEG quantization (since it also does not have those blocks in dark scenes), this particular scene is from a bought VHS, but many movies I want to convert are from TV.

scharfis - thanks for the info on chroma shift. You never stop learning ...

So I tried a lot more filters ...
CNR2, DNR2, TemporalCleaner & PeachSmoother all cause horrid artifacts in the face.
Convolution3D is very fast but sometimes "orders" the pixels in a way that you have small areas of same-color-pixel which doesn't look that good (though, I think it removes noise pretty good). SmartSmootherHiQ is also OK but the best settings I like so far are form wotef.
Unfortunately, wotef settings take a loooooong time ...

I did a lot of reading but could not find a qualitative comparison of denoisers for fast or slow scenes. Can somebody give me a link to it or did his own research? wotef, for instance, you use pixiedust for a reason (otherwise why stay with such a slow denoiser). What is your personal experience with denoisers?

One time I saw a still picture of an "a3d" denoiser (nice ordering of pixels). Where can I find that one?

Thanks again for your help,
Frank.

[wotef]

the deen filter has several convolution modes, one of which is "a3d"
...it's good and fast

i know pixiedust is damned slow, but it's very powerful and i think it does a beautiful job, you could swap it, though, for the faster and kind of similar spacedust()

i'm happy to leave a one-pass to huffyuv running overnight, and then queue the two-pass mpeg4 (off the filtered huffyuv) after that

cause dust is so slow, it's quicker to save lossless to an intermediate file, rather than doing two-passes of the filtering directly to xvid or whatever

anyway, here's a thread for you

http://forum.doom9.org/showthread.php?s=&threadid=51181

[^^-+I4004+-^^]

>I did a lot of reading but could not find a qualitative comparison of denoisers for fast or slow scenes.

most of them do nice job on slow/no motion scenes.....
most of them DON'T on motion scenes.....
soon you'll see that perfect filtering (as i see you are geared up quite a bit...hehe) is non existant....
so you better use slo-mo scenes as reference...fast motion will not look good anyhow....
but there's a theorem in which we gave up denoising the image completely to preserve sharpness...(and we cranked the bitrate too..hehe) and i'm all for sharpness.......

>Recording: PAL -> VCR -> demodulator (chromashift by one line) -> tape
Playback: tape -> modulator(next chromashift by one line) -> VCR -> PAL
I hope, that will clearup this a little bit.

after a bit of a research ;

[ http://groups.google.com/groups?num=50&hl=hr&inlang=pl&lr=&ie=ISO-8859-2&q=vhs+chroma+recording

http://groups.google.com/groups?hl=hr&lr=&ie=UTF-8&inlang=pl&th=769af90b7b9a9c16&rnum=8

http://groups.google.com/groups?hl=hr&lr=&ie=UTF-8&inlang=pl&th=b7792ec4dcb26829&seekm=4b3c984408Phil%40argonet.co.uk#link11

http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&th=b7792ec4dcb26829&rnum=1 ]


it does make a sense!
nice one!
(and usenet helped much more than my lousy books....)

but still,4th generation?/hehe
(i have some stuff that i'm doing for somebody else,and the color is all over the place....luma too...,but the girls are nice! lol! )

ohh,and fberreth what's the device ,again?
also,be carefull what you wish,and keep some distance from your display device too...usually looks better that way(hehe)

/ivo

[fberreth]

Ahh, the device. A "Blaupunkt" upper-line VCR & a bt848 capture card (the cheapest one I could get - Pinnacle PCTV or so).

About filters - well, I was so frustrated that I started this thread saying I currently prefer resize "only" with no filters. Resize does smooth out the noise already a bit ...

I have to do more comparison, try this conv3d, spacedust, deen & pixiedust a little more and then go for the encoding marathon.

PS: Ivo, what did I wish for?!?

[^^-+I4004+-^^]

Originally posted by fberreth
PS: Ivo, what did I wish for?!?

perhaps too much if you're denoising the video that's already blurred?
but tastes differ,so no one is wrong really.....
(as i said,i would cnr,downsize and sharpen that video...ie. i didn't saw the noise there as the main problem(infact that's as clean as vhs gets....),but i could show you what noise really is..hehe)

>So currently I decided to capture at 768x576 (PAL) and resize/crop to 480x564

thinking about playback on (possible) standalones?(huh i never asked that bloke if divx hardware players could correct AR..i really should as i also think these resolutions are to be oreffered (352 or 480 x 576 or simillar...but i will...will save me some bitrate and will have peace of mind that it'll be compatible with devices....in the long term that's a good thing....probably very long term,but non the less)

[sync]

Where can I get the PixieDust plugin? I've done a number of google searches and nothing comes up. Which versions of AviSynth is it compatible with?

[jggimi]

At the moment attachments are currently broken, but you can usually find it here (http://forum.doom9.org/showthread.php?s=&threadid=42749&highlight=pixiedust).

[sync]

Originally posted by jggimi
At the moment attachments are currently broken, but you can usually find it here (http://forum.doom9.org/showthread.php?s=&threadid=42749&highlight=pixiedust).
Thanks. I found that post earlier and now that I know that "attachments are currently broken" I understand why I didn't see any download.

[sync]

I found dust4. I tried pixiedust and unfilter and the result looks really good. But it took 65 minutes to encode a two and half minute avi on a 1 Ghz P3. That was using 3-pass VBR in CCE. That means encoding two hours of material would take 52 hours!

[wotef]

find dust5, try spacedust (faster), save to intermediate huffyuv file before encoding to cce, should save muchos time

[sync]

Originally posted by wotef
find dust5, try spacedust (faster), save to intermediate huffyuv file before encoding to cce, should save muchos time
I wish I could find dust5.

I keep reading about huffyuv. Yesterday, I took a 584 MB DV avi and loaded it into VirtualDub. I choose huffyuv compression and then did a 'save as avi'. The resulting file was 1,831 MB. So, unless I did something wrong, I don't see how that would save any time.

[fberreth]

First, get DustV5 from here (called "Dust"):
http://www.avisynth.org/~warpenterprises/

Second, DV is lossy that's why it takes less space than HuffYuv which is lossless.

Third, if you make a 3-pass encode, you basically "encode" 3 times your file with all the filters applied. Only the last encode however does the real encode, the other runs just try to figure out how much space each frame would take.
If you however run the filters once, store in HuffYuv so that the temporary result is lossless and then 3-pass encode the HuffYuv file total encoding time should be faster since you only run the filters once.

[sync]

I got dustv4 from there, but they don't have v5.

Thanks for the explanation. Now I understand why it took so long.

[sync]

Originally posted by fberreth
First, get DustV5 from here (called "Dust"):
http://www.avisynth.org/~warpenterprises/

Second, DV is lossy that's why it takes less space than HuffYuv which is lossless.

Third, if you make a 3-pass encode, you basically "encode" 3 times your file with all the filters applied. Only the last encode however does the real encode, the other runs just try to figure out how much space each frame would take.
If you however run the filters once, store in HuffYuv so that the temporary result is lossless and then 3-pass encode the HuffYuv file total encoding time should be faster since you only run the filters once.
If I start out with a 20GB DV AVI, then I'll need another 60GB just for the temporary file. I appreciate the value of storing the temp file in a lossless format, but I'm wondering if there might be another way to approach this that doesn't require so much disk space.

[fberreth]

Instead of HuffYuv you can use any other filter, but it will be lossy. For instance you could use XviD with quantizer setting of '1'.

You have to make your own decision here - if you want to stick with your filters - to either spend a lot of time (running the filters 3 times with a 3-pass encoding mode), or to use a temporary file for processing in HuffYuv (which is lossless) or any other format which would be lossy (MJPEG for instance works too, many use it).

I personally don't use any x-pass modes since I don't care about the file size but go with quality modes (I use quantizer settings).

[fberreth]

Hi,

I thought it might be useful if I share my final script that I use to process the movie (see link on pg 1). Please try it out and tell me what you think. Could I improve it somewhere? Maybe not sharpening that much (XviD with H263 will loose some detail that is "exaggerated", so with MPEG matrix I think the sharpening should be a little reduced). My goal is to have it run without postprocessing in high quality since me CPU is fully loaded anyway on normal playback. I encode with XviD, quantizer 4, search precision ultra high, VHQ 1, chroma motion & optimization, H263 or MPEG (I'm not sure yet which one I should choose). All other options are off (no B-frames).


LoadPlugin("c:\VideoTools\AVISynthYV12Plugins\asharp.dll")
LoadPlugin("c:\VideoTools\AVISynthYV12Plugins\LoadPluginEx.dll")
LoadPlugin("c:\VideoTools\AVISynthYUY2Plugins\chromashift.dll")
LoadPlugin("c:\VideoTools\AVISynthYUY2Plugins\wnr.vdf")

# Load the file
#AviSource("D:\doom9.avi")
AssumeFrameBased ()
ConvertToYUY2() # Just in case

# Shift chroma
ChromaShift (L=-2)

# Correct for color levels
saturation = 1.04
cu = - (1-saturation)*256
ColorYuv (off_y=-15, gain_y=50, cont_u=cu, cont_v=cu, gamma_y=5)

# Process the frames
WNR ("C:\VideoTools\wavelet_weak.vcf")

# Resize and convert to YV12 for encoding
LanczosResize (768, 576*3)
ConvertToYV12 ()
aSharp (3,6)
LanczosResize (768,576)

# Cut borders & center
Crop (6,0,0,-10)
AddBorders(3,5,3,5)

# Our final format is 480x576
LanczosResize(480,576)

# For testing purpose
#histogram ()
#ColorYuv (analyze=true)
bilinearresize(768,576)


The file wavelet_weak.vcf contains:
VirtualDub.video.filters.Add("wavelet noise reduction (0.1)");
VirtualDub.video.filters.instance[0].Config(4624, 3600, 3598, 4384, 5156, 8244, 3858, 4373, 5664);


Thanks for your input,
Frank.

[wotef]

you've lost me now -

too many colourspace conversions (esp using a vdub filter means RGB)
seems like a lot of work just to bump up the saturation (i'd just use tweak)
too many resizes (you want SVCD? use FITCD)
have no idea why you'd want an XviD fixed quant of 4

[fberreth]

Hi Wotef,

not really a lot of colorspace conversions - just one. The original is in YUY2 (althought the link to the download AVI is RGB) and at the end I convert to YV12 which has to be done anyway. WNR acts here as an avisynth plugin, so it does not convert to RGB but operates in YUY2.
The ColorYuv statement does not only bump the saturation but also adjustst the luminance range from 16-234 to 0-255 and tweak very slightly the gamma. I'm currently still wondering if the bt8x8 gave me the right ADC conversions for dark picture parts. It seems like bt8x8 might not convert very dark areas properly.
The resizes are because aSharp is very strong and causes artifacts "vertically", that's why I resize UP (and later down). A weaker setting of aSharp would have nearly no effect horizontally though - kind of unusual solution but works.
According to docu the down resize statements
LanczosResize (768,576)
LanczosResize (480,576)
are exactly the same as a LanczosResize (480,576) from the higher resolution, but since I had to crop (after sharpening) I have 2 resizes.
And I use q4 simply because I want to archive the video and not squeeze excatly on 1 or 2 CDs, although it should be <=1.5GB just to not eat up too much space.

-Frank.