HDV to DVD with Avisynth/Quenc picture judders badly.

[Gerard V]

I desperately need help with this. We recorded a show using a Sony FX1E PAL HDV camera 1440 x 1080i. Captured the video using CapDVHS. This gives us an MPG file which we indexed with DG Index.

(Using Avisynth I have cut 6 frames as an AVI (compressed using huffyuv) and posted them http://www.thathypnoshow.com/Video/Cut.avi.
The frames were cut using Mpegsource and trim, no other processing.
)


We processed the whole video with a script like this one below (but this is just a minimal sample that gives the same problem).

function HDV2WS(clip V)

{
AssumeBFF(v)
SeparateFields()
ConvertToYUY2
BicubicResize(720,288)
PeachSmoother(noiseReduction=40, stability=25, spatial=100, dot=FALSE)
ConvertToYV12
Odd=SelectOdd()
Evn=SelectEven()
Interleave(odd,Evn)
Weave()
}

M1 = AviSource("E:\THS4\Cut.avi")
M1.HDV2WS()


Encoded to Mpeg using Quenc, 2 pass, interlaced, VBR=6500, Max=8000.

The resulting Mpeg plays fine on the PC, but on the DVD player the picture stutters very very badly when there’s lots of motion, especially if the subject is moving left to right across the scene and the camera is panning. It is unwatchable.

What have I done wrongly? What can I do to fix this?

I’m posting here because I’m not sure where I have created the problem, but I do most of the processing with AviSynth.

[krisq]

are you sure it's bff? i have some hdv (.m2t) material from fx1 pal and it's definitely tff.
this kind of artifacts are mainly produced by setting the wrong field order.

//edit
just checked your sample, it's tff.

[Gerard V]

Thanks for your input. Although you are indeed correct, it is TFF, changing the line in the script to assumeTFF() makes no difference to the jittery DVD produced. In order to be sure I had not just made a mistake I did it twice.

I suspect that regardless of the assumeBFF or TFF, since the processed resized frames are put together the same way they were taken apart that the motoin would not be greatly affected. Could it be a bitrate issue? Since when panning the whole scene is moving? Looks OK on the PC though.

:(

[krisq]

leave AssumeTFF().
and get rid off this

Odd=SelectOdd()
Evn=SelectEven()
Interleave(odd,Evn)

just weave() your video
i don't know Quenc, but i'm pretty sure you have to set the correct field order there also.


//edit
sorry for this edit, i thought you were 'bobbing' your source somewhere in your script. :o

[Boulder]

Did you try with AssumeTFF and Interleave(evn,odd)?

(though I'd still go for the smart bobber route ;))

[Gerard V]

krisq, Boulder - thank you again. I have tried removing the section as recommended krisq. I have tried Evn, Odd and Odd, Evn. In fact I have tried all combinations of BFF, TFF, Evn, Odd. All to no avail. The resulting DVD plays relatively well on my PC, and very badly on my DVD player regardless of the combination used.

Boulder - could you kindly please suggest a smart bobber script option and I'll try that when I get home tomorrow.

[Pookie]

Gerard V - Can you post a few seconds of the original Transport Stream ?

This article might be applicable to your situation.

http://adamwilt.com/HDV/cineframe.html

and/or

http://dvfilm.com/cgi-bin/board/main.cgi?board=HD;action=display;num=1111243659

Which leads to : http://www.khm.de/~alec/share/fx1_handing%20interlaced.html

[hoozdapimp]

Are you sure you are encoding it correctly (i.e. not setting it to be BFF?)

[krisq]

krisq, Boulder - thank you again. I have tried removing the section as recommended krisq. I have tried Evn, Odd and Odd, Evn. In fact I have tried all combinations of BFF, TFF, Evn, Odd. All to no avail. The resulting DVD plays relatively well on my PC, and very badly on my DVD player regardless of the combination used.


are you setting Quenc to encode TFF (odd)?
because if you do then something strange is going on here :)

[Boulder]

I see absolutely no reason why your script should cause stuttering after the AssumeBFF has been changed to AssumeTFF(). The field order remains TFF and the video plays smoothly in VDub. The only reason I can think of is what hoozdapimp said, incorrect field order. If it is correct, you might want to upload a sample of the QuEnc encoded video so we can take a close look at that too.

About smart bobbers: since you are resizing, IMHO good ones are TDeint(mode=1), SecureDeint() and MVBob(). The last two are in the MVBob package, TDeint is the plugin of the same name. TDeint should be the fastest and MVBob is the slowest one.

You could try this script:

AVISource() # or MPEG2Source()
AssumeTFF() # with MPEG2Source no need for this line
TDeint(mode=1)
BicubicResize(720,576) # resize before colorspace conversion for much faster processing
ConvertToYUY2()
PeachSmoother(noiseReduction=40, stability=25, spatial=100, dot=FALSE)
ConvertToYV12()
AssumeTFF()
SeparateFields()
SelectEvery(4,0,3)
Weave()

[krisq]

SelectEvery(4,0,3)


shouldn't it be


SelectEvery(4,1,2)

?

[Boulder]

Nope, if you do that in that script, it'll change the field order from TFF to BFF.

[krisq]

this is form Avisynth Help

SeparateFields.SelectEvery(4,0,3) # Undo Bob, even field first. Use SelectEvery(4,1,2) for odd field first.


my english is poor but odd means tff, right?

[Boulder]

Even is 0,2,4,6... and odd is 1,3,5,7... so in the docs (in that specific part) even field first means TFF, odd field first is BFF. It does sound a bit confusing, agreed :)

[krisq]

ok, always learning :D thanks.

[Gerard V]

Thanks all. I am away from home for the next 24 hours. So I cannot test some of the things you suggested. But I have tried the following based on my original script and they ALL gave the same result when played on my standalone DVD player:

See postings above for this to make sense.
Basic Script BFF, Odd, Evn, then Quenc Interlace, Not TFF
Script BFF, Odd, Evn, Quenc NotInterlace, Not TFF
Script BFF, Evn, Odd, Quenc Interlace, Not TFF
Script BFF, Evn, Odd, Quenc NotInterlace, Not TFF
krisq's suggetion to leave out those lines, TFF, Quence TFF
Boulders suggestion TFF and EVn, Odd etc.
The thing is that they ALL stutter badly when played on the4 DVD player, but on the PC they are not as bad but still stutter. For completeness, I am playing the actual DVD disk on the PC, using Zoomplayer (full version).

I'll post up a section of the mpg file (post Quenc) that shows the problem (as soon as I get home). Since I am not confident I know how to cut an edit from an Mpg without reencoding etc., could someone advise which tools I should/could use to cut a piece out of the mpg file and how many frames would be enough to get the gist of this? I have TMPEG DVD Source Creator (can I cut MPG with that?).

I'll try the other suggestions too when I am back at my own PC.

(BTW Thanks again for all your help - it is much appreciated).

[Boulder]

One other thing I would try is to use HC for encoding. That way we can check if it's actually a problem with the encoder.

You can for example open the m2v or mpg file in DGIndex and use the [ and ] buttons to select a small range and then choose "Save project and demux video". It will give you the sample clip without re-encoding. I would say something like 25-50 frames should be enough at least for viewing on the PC, just try to get the troublesome panning there. I have no problems downloading a large clip so the sky is the limit on my part ;)

[BigDid]

... Since I am not confident I know how to cut an edit from an Mpg without reencoding etc., could someone advise which tools I should/could use to cut a piece out of the mpg file ...
Mpg2cut2 here: http://www.geocities.com/rocketjet4/
I use it all the time for cutting commercials from my captures without any problems :)

Did

[Gerard V]

Ok, will do both, when I get home about 36 hours from now.

Incidentally, I have used the same Avisynth script before and it worked OKish. There was mild judder but not "too bad" even though I was unhappy with it. This last show the camera was much closer to the stage, with audience on the left and right of the stage, so there was more fast panning as the camera panned across following a subject who was running. The first time I thought the juddering reflected a basic problem encoding that much info at the given bitrate because with DV the entire background is in sharp focus (unlike film). Now I can clearly see that I am doing something way wrong.

Since you have access to reasonable broadband (unlike us in New Zealand), I'll post up a few frames of the original capture and the same as they turn out. Provided I can sort out the space for them, and my upload speed is 128Kbps so it'll be slo-o-o-w.

Please check back in a day or so.

[BigDid]

... Since you have access to reasonable broadband (unlike us in New Zealand), ... my upload speed is 128Kbps so it'll be slo-o-o-w...
Welcome to the club, it seems a common burden for us Southerners, I have a 128/128 expecting to upgrade to a 256/128 when prices drop :devil:

Did

[wiak]

why do you want to downgrade 1920x1080 or 1280x1080 or 1280x720 HDV to 720x480 NTSC or 720x576 PAL ?
you will LOSE a huge amount of you monkeys!
you should have buyd a DVD Camcorder not a HDV High Def Camcorder

oh well my little humble option*
:)

[krisq]

if you ask me, downsampled hdv material looks unmatched to sd. btw why bother with dv when bray/hddvd is around the corner :)

[Boulder]

Since you have access to reasonable broadband (unlike us in New Zealand), I'll post up a few frames of the original capture and the same as they turn out. Provided I can sort out the space for them, and my upload speed is 128Kbps so it'll be slo-o-o-w.

You can save a little bandwidth by encoding at a lowish bitrate, something like 2Mbps should be enough. It doesn't matter if it's blocky but it'll save you some time :)

I'll encode the sample clip you provided earlier with CCE and HC and post the clips here so we can compare results.

[Boulder]

btw why bother with dv when bray/hddvd is around the corner :)
And the prices are very high..

[krisq]

And the prices are very high..
That's true. But Sony announced bray recorder yesterday (well, they're taking pre-orders, august 18th is the ship date). Retail price is 750$ which is not that much if video is your profession. But I agree, it's still too much for a casual user :D

[foxyshadis]

why do you want to downgrade 1920x1080 or 1280x1080 or 1280x720 HDV to 720x480 NTSC or 720x576 PAL ?
you will LOSE a huge amount of you monkeys!
you should have buyd a DVD Camcorder not a HDV High Def Camcorder

oh well my little humble option*
:)
It's not like downsampling the video is the end of the line for it. The original can be archived until HD-DVD or Bluray is cheap enough to produce a version for it, but in the meantime if it's going to be played it needs to be an SD DVD.

[Boulder]

I just remembered another important thing we need to know : what program do you use to author and burn the DVD?

I suggest trying MuxMan out if you already haven't done so, it's free and it'll make a perfectly compliant DVD structure for you. Then you can use ImgTool Classic to create an ISO image to burn (preferably with ImgBurn). These procedures should exclude any errors in the authoring phase.

[Boulder]

@Gerard:
Here's the six-frame clip as encoded with HC : www.saunalahti.fi/sainki/sample_hc.m2v . I haven't had the chance to use CCE yet as I'm running a long encode right now but it shouldn't look any different.

Do you get the same output when you use QuEnc on your machine?

[Gerard V]

(wiak) why do you want to downgrade 1920x1080 or 1280x1080 or 1280x720 HDV to 720x480 NTSC or 720x576 PAL ?
you will LOSE a huge amount of you monkeys!
you should have buyd a DVD Camcorder not a HDV High Def Camcorder

I want to make a PAL DVD to share with others, so downsampling is what’s required. Also HDV gives me more options for editing/cutting, as I can crop half the picture or zoom to 50% centre after shooting and still have better than PAL. The FX1E will output PAL at 720x576 if I select that option. I didn’t this time and want to solve the problem here. But I have that option as a fallback too. BTW - FX1E produces 1440 x 1080i so I'm reducing that to 720x576.

(krisq) if you ask me, downsampled hdv material looks unmatched to sd. btw why bother with dv when bray/hddvd is around the corner

Because its just around the corner and has been for a while and I am alive now.

(foxiyhadis) It's not like downsampling the video is the end of the line for it. The original can be archived until HD-DVD or Bluray is cheap enough to produce a version for it, but in the meantime if it's going to be played it needs to be an SD DVD.

That was my thinking too. Buy 1 good camera and in 2007 be able to do HD-DVDs etc. Not that anyone in NZ has a player or that they’ll be affordable until 2009.

(Boulder) I just remembered another important thing we need to know : what program do you use to author and burn the DVD?

DVD Lab Pro – But I have news – its not the Mpeg encoding – it is the AVS script that is doing it. (See below)

(Boulder) Here's the six-frame clip as encoded with HC . . .

Tried your clip, and then coded a longer clip with HC myself – same problem in all cases. The helpful news is that I think I have demonstrated that it is not the DVD or the Mpeg that is the issue.

Using DGindex I clipped some fast motion frames from the original clip at 1440 x 1080i.
I also trimmed and coded the AVS to an AVI file using DivX. (720 x 576 16:9)
I played both through my TV from a hard drive video device (TViX).

Both versions had the problem. So the problem is created between the MPEG original and the output of the AVS script – i.e. – in the AVS script.

I still don’t know what the problem is, but this is progress of a sort. I have clipped 10 frames of the original and put them here (http://www.thathypnoshow.com/Video/cut_orig.demuxed.m2v).

[Gerard V]

Oh, and thanks everyone for your ideas and advice. It has helped narrow things down a bit.

[Gerard V]

Posted by Boulder
You could try this script:

AVISource() # or MPEG2Source()
AssumeTFF() # with MPEG2Source no need for this line TDeint(mode=1)
BicubicResize(720,576) # resize before colorspace conversion for much faster processing
ConvertToYUY2() PeachSmoother(noiseReduction=40, stability=25, spatial=100, dot=FALSE)
ConvertToYV12()
AssumeTFF()
SeparateFields()
SelectEvery(4,0,3)
Weave()


OK. I tried that script (much slower), coded with DiVX to an AVI and played it using the TVix Hard drive player and the stuttery jerky playback is just as bad as my original attempt. (Pity). Thanks for the suggestion, but where to from here?

[krisq]

damn, but that just impossible :D
it has to be the encoder then. can you post some screenshots or anything with encoder's settings? is there a chance that you have to set input and output field order in your encoder? oh, and can you give us a sample of your final encode?

[Boulder]

My next suggestion is to make sure that the source preparation doesn't cause the issue. That is, I think neuron2 should be consulted as he's the author of DGIndex. He's probably seen this thread so hopefully he'll respond here if he has some ideas as what to try next. Have you tried creating the d2v project file with the latest DGIndex beta?

Looking at the sample of the original, DGIndex v1.4.8 beta 7 shows nothing weird - it's TFF and 100% video as it's truly interlaced.

Did you enable interlaced encoding when you encoded to DivX and most importantly, does your hard drive player output interlaced encodes properly?

Here's the second sample encoded with HC (at constant quant 4, MPEG standard matrix): www.saunalahti.fi/sainki/sample_hc_clip2.m2v

Here are the settings for HC:
http://www.saunalahti.fi/sainki/hc_settings.gif

[krisq]

just for the record, i've made an encode using quenc 0.7 (5000kbps, 1 pass VBR, Top field first, interlaced), your .m2v source and this script.

mpeg2source("E:\test.d2v")
AssumeTFF()
SeparateFields()
ConvertToYUY2
BicubicResize(720,288)
#PeachSmoother(noiseReduction=40, stability=25, spatial=100, dot=FALSE) had to drop it, i don't have this plugin)
ConvertToYV12
Weave()


output is perfect. is there a chance that peachsmoother is doing something bad?

[Gerard V]

Posted by Krisq
damn, but that just impossible
it has to be the encoder then. can you post some screenshots or anything with encoder's settings? is there a chance that you have to set input and output field order in your encoder? oh, and can you give us a sample of your final encode?

Since I get the same stuttering result with Quenc and HC whether cut to DVD or played from HD recorder, and since I also get that stuttering result playing an AVI made with DiVX from the avs file I am almost certain its not the encoder. So pursuing the encoder line doesn't make sense at thispoint. But if you can see why that might be it then I'll post some output up.

Posted by Boulder
My next suggestion is to make sure that the source preparation doesn't cause the issue. That is, I think neuron2 should be consulted as he's the author of DGIndex. He's probably seen this thread so hopefully he'll respond here if he has some ideas as what to try next. Have you tried creating the d2v project file with the latest DGIndex beta?


This is a good idea. First I'll try minmal processing from the original - I'll post up the script and result when I've done it. That way we can find out where in the script the problem starts.

Looking at the sample of the original, DGIndex v1.4.8 beta 7 shows nothing weird - it's TFF and 100% video as it's truly interlaced.


Did you enable interlaced encoding when you encoded to DivX and most importantly, does your hard drive player output interlaced encodes properly?

Yes, I have tried all options, interlaced and not.

By krisq
just for the record, i've made an encode using quenc 0.7 (5000kbps, 1 pass VBR, Top field first, interlaced), your .m2v source and this script.

mpeg2source("E:\test.d2v")
AssumeTFF() S
eparateFields()
ConvertToYUY2
BicubicResize(720,288)
#PeachSmoother(noiseReduction=40, stability=25, spatial=100, dot=FALSE) had to drop it, i don't have this plugin)
ConvertToYV12
Weave()

output is perfect. is there a chance that peachsmoother is doing something bad?

I'll try this too.

Has anyone been able to recreate the stuttering?

BTW - I recaptured the tapes using the camera with the i.link set to automatically downsample to PAL. So it is captured like a regular PAL DV. Used WinDV. Cut the same piece and played as AVI - 100% perfect so far. So I am using quenc with the same settings as before and will process the whole thing and have a look.

Summary
1. The AVI test and the fact that it occurs qith HC, QuEnc, and from DVD and the HD Device means that DVD Lab and the Final Mpeg encoders are eliminated.
2. It could be the original MPEG processes - Boulder's idea
3. It could be Peach - krisq's suggestion.

I'll be back.

[Gerard V]

Boulder - your screenshot for HC shows the video as being 720 x 576, yet the clip I posted was 1440 x 1080i. What script did you use to reduce it to PAL? Please post it and I will do the exact same thing here.

Gerard V

[Gerard V]

We have a solution owing to the fact that krisq doesn't have peachsmoother.

krisq's script without peach (above) works fine. I put peach back in (uncommented his 1 line) and disaster struck.

Phew! Knowing the problem is 90% of finding a solution.

Thanks everyone for your ideas and help. I am very grateful to now be able to move on with this project.

The problem is only apparent when there is lots of motion as with a panned shot. Don't exactly know what is going wrong but I can live without peach.

[Boulder]

Ha! I was just about to post as I remembered that Peach had some weird problems with motion, though I don't remember it being extreme stuttering but some weird artifacts.

I strongly recommend trying out MVDegrain1 or MVDegrain2 which are found in the latest MVTools release. They are very efficient and it's possible that MVDegrain1 isn't much slower than PeachSmoother - and you don't need to go the YV12->YUY2->YV12 route.

Good to know you got it going now :)