Unexpected Results Processing PAL DV with Avisynth (Newbie) Advice Sought

[Gerard V]

I made my first DVD about 6 months ago. Captured PAL DV from a borrowed Sony VX2000 using Windows Movie Maker, and then edited and made a DVD with Ulead. Having since discovered avisynth, virtualdub and DVD Lab Pro I have gone back to that material to try to improve the picture quality but I am getting some unhappy results that I don’t understand. I would appreciate some explanation of what is going wrong and advice for processing techniques to avoid or correct the problems.

I have uploaded a sample of the original AVI (just 100) frames compressed with Xvid using virtualdub. (Why is it still nearly 8 Mb for 100 frames of compressed PAL?)

Sample of original AVI http://www.thathypnoshow.com/Video/T2Pt0001CutX.avi (http://www.thathypnoshow.com/Video/T2Pt0001CutX.avi)

This site’s newbie guide offers the following script in its DV to DVD section but the results are terrible in this particular case.

AVISource("Pt0001CutX.avi")
ConvertToYUY2()
SeparateFields()
odd=SelectOdd.Convolution3D(1, 6, 10, 6, 8, 2.8, 0)
evn=SelectEven.Convolution3D(1, 6, 10, 6, 8, 2.8, 0)
Interleave(evn,odd)
Weave()
DoubleWeave.SelectOdd()

My results: http://www.thathypnoshow.com/Test/Process_1_2_A.jpg and http://www.thathypnoshow.com/Test/Process_1_2_B.jpg show scan lines as if not deinterlaced.

I have had good success with the following script on a subsequent project:

M = AVISource("D:Pt0001CutX.avi")
M = M.ConvertToYUY2(interlaced=true)
M = M.DGbob(order = 1, Mode = 0)
M

Better results for the case in point too. But I don’t understand what is happening with the blocky pixelisation on the left sleeve where the curtains run into the suit, and the staircase on the shoulder where it occludes the screen (both present on the “original”)?

Example results: http://www.thathypnoshow.com/Test/Process_1_1_A.jpg and http://www.thathypnoshow.com/Test/Process_1_1_B.jpg

What could I do about remedying those effects? (Suggestions anyone?) BTW – I cannot go back to my original source – those tapes are wiped – the version you see is the earliest version I now have (damn).

The “original” was captured from the camera with WMM, is it likely I did something at that point which caused the pixelisation effect?

[Boulder]

Processing interlaced material has been discussed quite thoroughly, a quick search should help you with that. One route to go is MVBob->filtering->reinterlacing. It'll be slow but the quality is worth it.

It's normal to see the combing when viewed on the PC, if the field order is set correctly (DV is BFF), it will not be visible on the TV. You definitely should keep the material interlaced.

(FYI, the field order is incorrect in your DGBob line)

[Gerard V]

Thanks for your reply. I have been lurking for about 4 months. Perhaps I have misunderstood the consensus, but I had concluded that in general the better visual quality for DVD was obtained by deinterlacing. I will experiment with the sequence you suggest, though my previous efforts have all been based on material from these forums (fora?) as well.

I'm still unsure what to do about the pixelisation and the staircase - these remain after deinterlacing.

Perhaps I wont notice them on the TV, but is that always the case? I have seen thread here and examples where people have gone to great lengths to remove these artifacts to improve the whole visual outcome and I am keen to learn ways to do that with this particular material.

Oddly, the frame order I chose for DGBOB was based on comparing the output from the two options (I tried both). Will also try again - perhaps I missed something.

[Gerard V]

OK, I have compared the DGbob options, and the order I used gives (at least in my eyes) a better result example below (photoshopped to show only detail sections).

http://www.thathypnoshow.com/Test/DGbobCompare.jpg

Script:

M = AVISource("D:\Video\Edits\Raw\T2Pt0001CutX.avi")
M = M.ConvertToYUY2(interlaced=true)
M1 = M.DGbob(order = 1, Mode = 0)
M2 = M.DGbob(order = 0, Mode = 0)
Stackhorizontal(M1,M2)

[Boulder]

Thanks for your reply. I have been lurking for about 4 months. Perhaps I have misunderstood the consensus, but I had concluded that in general the better visual quality for DVD was obtained by deinterlacing. I will experiment with the sequence you suggest, though my previous efforts have all been based on material from these forums (fora?) as well.

I think that progressive DV has more detail but that means shooting the material as progressive, not making it progressive by deinterlacing. Deinterlacing always includes interpolation (or simply discarding) of information. You'll also lose the smoothness of motion which is irritating IMO. (I hate the film look the live TV broadcasts nowadays have way too often)

I'm still unsure what to do about the pixelisation and the staircase - these remain after deinterlacing.
Those are some side-effects of deinterlacing. One more reason not to do so;)

Perhaps I wont notice them on the TV, but is that always the case?

Yes, provided that you treat the material correctly and set the field order in the encoder correctly. In any case, you should check your results on your TV if that's what you'll use in the end.

DV is BFF so the order parameter should be 0. However, I strongly recommend you to try MVBob. It's dead slow but very good for your purposes - it'll also get rid of some of the noise as a side-effect. Using Cedocida as the DV codec is also highly recommended.

Try this script and see if you like it:

AVISource("path\clip.avi")
AssumeBFF()
MVBob()
#place the filters here
AssumeBFF()
SeparateFields()
SelectEvery(4,0,3) # or SelectEvery(4,1,2) if you want TFF output
Weave()

[Boulder]

Is the original AVI the one that you got from the camera? You can doublecheck the field order by opening this script in VirtualDub.

AVISource("path\clip.avi")
AssumeTFF()
SeparateFields()

If the motion is smooth, then the clip is TFF. If it's not, try AssumeBFF() instead of AssumeTFF(). If the motion is then smooth, the field order is BFF.

[communist]

DV is always BFF :)
Keep it interlaced if its going to DVD.
- Smooth motion
- no fiddling with deinterlacers
- no artifacts on TV

[Gerard V]

You can doublecheck the field order by opening this script in VirtualDub.

Thanks, I tried that just now. The difference is not very noticeable to me (should it be or am I blind?). Anyway the BFF option looked slightly smoother (I tried both TFF & BFF).

Is the original AVI the one that you got from the camera?

Yes

Try this script and see if you like it:

I have tried it on a few seconds of video and it looks OK. But now I really need to make up a longer segment trying out some options and filters to see which one I prefer in the end. Thanks for pointing me in the right direction. I’ll try both deinterlaced and not. To see how that goes.

Thanks both of you for your help.

[Boulder]

Thanks, I tried that just now. The difference is not very noticeable to me (should it be or am I blind?). Anyway the BFF option looked slightly smoother (I tried both TFF & BFF).

The difference should be quite noticable in scenes with motion. The TFF one should jerk back and forth and the BFF one should be very smooth. In scenes with little motion, the difference is much smaller.

[bb]

The difference should be quite noticable in scenes with motion. The TFF one should jerk back and forth and the BFF one should be very smooth. In scenes with little motion, the difference is much smaller.
I recommend to try a clip with horizontal motion, because that's where the difference becomes obvious.

bb

[communist]

Sample of original AVI
This one is already filtered. For future reference please always try to post unprocessed material (in this case a small snipplet of the DV avi). Since its usual to post the used script everyone can reproduce your results and try other scripts and then just post the script that yields better results.

Its a bit hard to suggest things if we dont know what exactly you're working with ;)

[Gerard V]

Thanks communist and boulder. The version I posted is in fact the earliest generation I still retain (hence my problem), the original tapes are gone, alas, so recapture is not possible. So in this case what you see is what I am working with. However the clip I uploaded is cut from what I think is the original capture (made with WMM). Simply loaded it into Virtualdub and selected a few frames. I did compress with Xvid, or the upload would have been too big. However you have hinted at the answer to one of my questions. When you say that the clip is already filtered - how do you know that? I assume that either WMM automatically, or I have accidentally, done something - and I'm trying to figure out what that was (so as not to do it again).

With regard to "not deinterlacing" for DVD. Am I still correct in thinking that if for some reason I need to crop/resize, then I should first deinterlace?

[Boulder]

WMM doesn't filter, it gives you the raw unprocessed stream when you capture DV with it. What comes to the sample: if you haven't used interlaced encoding in XviD, it will screw things up. I'm not sure but it also could be that XviD only encodes interlaced as top field first.

If you need to crop and/or resize, use a smart bobber first - MVBob is one of those. It will give you a double framerate. Then you do all the processing and as the last item, you reinterlace the video which will also restore the framerate to the original one. The reinterlacing is done by the AssumeBFF().SeparateFields().SelectEvery(4,0,3).Weave() part of the script.

[communist]

Simply loaded it into Virtualdub and selected a few frames. I did compress with Xvid, or the upload would have been too big. However you have hinted at the answer to one of my questions. When you say that the clip is already filtered - how do you know that? I assume that either WMM automatically, or I have accidentally, done something - and I'm trying to figure out what that was (so as not to do it again).
In your original post you had 2 pictures showing interlaced footage (probably the real original material).
If you load it via this simple (AviSource().AssumeBFF().SeparateFields()) script into VirtualDub you should see slight movement in each frame within a scene that has motion. If it is already deinterlaced (progressive) then you will see only new motion with each 2nd frame. The latter is the case with the sample you linked to. XviD will not preserve interlacing if you dont check the "Interlaced Encoding" box, though I'm sure this isnt the problem since XviD then would treat the "mice teeth" as picture detail and try to preserve - but in the linked sample its not there at all.

/EDiT
See these 2 samples:
Original interlaced material (http://www.stud.uni-goettingen.de/~s304280/D9/sample_interlaced.avi)
Deinterlaced with Tdeint (http://www.stud.uni-goettingen.de/~s304280/D9/sample_tdeint.avi)

Apply this script and single step through it in VirtualDub:
AviSource("...")
AssumeBFF()
SeparateFields()

You will see motion in each frame with the interlaced sample, while in the deinterlaced sample there is only motion each 2nd frame - since there are no more fields.

If you still wondering what this all is about see http://100fps.com/

With regard to "not deinterlacing" for DVD. Am I still correct in thinking that if for some reason I need to crop/resize, then I should first deinterlace?
If the output is supposed to be interlaced aswell, BobDeinterlacer -> Crop/Resize -> Reinterlace maybe a way. See Boulder's post ;)

[Gerard V]

Guys, you are reassuringly correct. I’m not sure how/what I did all that time ago, but the following is now clear. The video sequence I have been using (and uploaded) is already deinterlaced, this explains the stop go stop go effect when scrolling through frame by frame with virtualdub. Somehow I must have deinterlaced (not sure how) when extracting the cut to place on the web for this post because I can now see that the true original is interlaced (courtesy of communist’s kind examples).

Using boulder’s original suggestion I have examined a clip from the interlaced original trying both TFF and BFF and getting it “wrong” is now clearly visible. Cool. This is still not visible in DGbob unless I set mode=1. With mode=0 DGbob was removing every other frame anyhow so they came out the same. With mode=1 it is clear that the video is BFF.

So that’s that. I don’t know where I went wrong to start with, but no matter, as I can start again cleanly with this video. It is interlaced and BFF. So to process it I can now (as suggested), use MVbob to deinterlace, filter/crop/resize etc., then re-interlace with AssumeBFF().SeparateFields().SelectEvery(4,0,3).Weave() part of the script.

Thanks again Boulder & communist. This has been really useful and I appreciate that you have taken time to help out a noob. Ta.

[dosdan]

If you need to crop and/or resize, use a smart bobber first

1. If cropping Mod 2 off the top and/or bottom, and the destination will be DVD, why deinterlace?

2. If only resizing horizonally (I do a lot of VHS-to-half D1 PAL DVD using HorizontalReduceBy2) there is no need to deinterlace.

[Boulder]

You need to crop mod-4 since we're dealing with interlaced YV12 here.

The reason why I suggested bobbing and reinterlacing is that the OP had done some noise reduction and I figured he would like to do that in the final encode as well. And yes, there are better options than Convolution3D..I would use MVTools to do motion compensated denoising especially if the material is shot without a tripod.