DVB-C - HD H264 stream with blurred frames

[asteri]

Hello ALL,

after lot of try I decided to post the question here. I think there is no better place in the internet to ask video capturing/encoding related questions :)

I have this sample DVB-C HD video (1920*1088) (~20MB):
http://rghost.net/private/50812938/e1fec4a5f9837e4c6b045048a69ec7e4

If I try playback it I see that some parts of the video are not comfortable for watching. Especialy the scenes where the camera moves horizontaly over the screen. It looks like the video motion is jerky. I cant explain it better also because english is not my native language. If you look on the test.ts you will see it easy between 11s - 14s. Quite well it is shown if the camera goes over the glasses and then over the paper(book). Video motion is jerky.

If I check the video more closely frame by frame, I see that some of the frames are blurred. Is this normal?? Is this the reason why the video motion looks jerky?

I think the problematic scenes are even more visible after encode. Is it because I have used wrong settings for deinterlacing?
Interesting thing is that I think it is problematic mainly with documentary films.

Here is encoded test.ts into mp4 using x264:
http://rghost.net/private/50813088/20488105ebc5ad9b16779d7ce1658caf

I have used this avs(cropping is already defined in the index file):
LoadPlugin("***********\DGDecNV\DGDecodeNV.dll")
DGSource("************\test.hdtv.x264.dgi", deinterlace=1).Spline36Resize(720,406)#.Trim(0,100)

For capturing I had used Technisat Cablestar Combo HD + DVBViewer 5.2.9(RS1.27). PC specification are good: core i5-2500, 12GB ram, nvidia quadro k600 graphics, 80GB ssd for OS and 1TB for recordings. So the computer is strong enough.

My questions are:

1) Is the jerky video motion problem of the capturing DVB-C stream by my tuner?
2) Or is it problem of some kind of wrong conversion of the documentary film from NTSC(30p) to PAL(50i->25i) made by DVB-C broadcast provider?
3) What kind deinterlacing method should I use for this source?
4) Will the correct deinterlacing filter with correct settings eliminate these problems in video stream?
5) What other should I do to solve it ;)

Thank you for any info, I am currently quite desperade with finding any solution.

[kleen]

Don't expect BR quality from a TV broadcast. The file you get it's exact copy of the stream, unless you have errors (see DVB Viewer log). Software deinterlacing will degrade the quality so better leave it to your TV/hardware.

[asteri]

I dont expect BR quality. I am totally satisfied with the quality. I just need to do something with jerky video motion. There was no errors in part of jerky video motion. So it is not problem of bad signal.

I dont use sw deinterlacing. I use hw deinterlacing by Nvidia (Quadro K600 graphics).

And also I want to have the video after conversion deinterlaced. Dont ask me why, simply I need it.

So my question still remain unanswered. What could I do better in conversion process? What deinterlace filter and what settings should I use?

Thanks

[ChiDragon]

2) Or is it problem of some kind of wrong conversion of the documentary film from NTSC(30p) to PAL(50i->25i) made by DVB-C broadcast provider?

It appears to be a standards conversion with field blending, yes.

You can try disabling the hardware deinterlacing and using Srestore (http://avisynth.nl/index.php/Srestore) in your Avisynth script.

[asteri]

Hi Chiragon!

I will try this Srestore(). It looks promissing. I hope it will help.

Can you explain me how it know which frames should be replaced? I am worry about the scenes where this blended frames are not presented. Should I use after Srestore hw deinterlace again? Or it will not be anymore interlaced after Srestore end?

Can you recommend me correct parameters of Srestore() for my case?

[asteri]

Hello,

I have tried several settings for SRestore and also for bobber (hw nvidia, yadif, QTGMC).

I think I am on good way, I have removed blured frames, but still I have feeling that the motion isnt perfect. I think currently the best settings for me is follows (quality and speed):

LoadPlugin("K:\_prevod\_pgm\DGDecNV\DGDecodeNV.dll")
DGSource("K:\_prevod\_tmp\test.hdtv.x264.dgi", deinterlace=2).SRestore(frate=25,speed=-25).Spline36Resize(720,406)#.Trim(0,100)

If you want check some other variation I have tested download this package:
http://rghost.net/private/50979438/c4b6daa4fc9b5f7bc9d8ec132ec30752

I have an important question. Should I leave SRestore to set output framerate to 23.976fps or is better to set frate=25fps as the original recorded ts ? I guess I cant have another framerate because audio will get out of sync, right?

Could you recommend me any other settings to tweak to get better results?

Oh and last question. I had to use older SRestore v2.7e instead of v2.7g. Newer version doesnt work fo me even if I have all dependencies installed. I am getting error message about :
"Script error: expected '{'"
"srestore.avsi, line 319, column 0"
Currently I dont use MT version of Avisynth. In future I would like to use MT version for SRestore but I guess I will need for MT SRestore v2.7g+, right?

Thanks for any help!

[Guest]

Too many questions in one post! People will be scared off as you make it a major project to help you. Please post your most important question, get it resolved, and move on to the next.

Also, "I have feeling that the motion isnt perfect" is meaningless. You have not told us specifically the problem, we can't guess it, and perfection is unattainable anyway.

I think if you post again with one clear specific question we can help you with it.

[asteri]

Hello neuron2,

thanks for attention :)

My main interest and still unanswered question is If I have used correct technique(settings) to remove blended frames and if I could do more better. Blended frames have been removed successfuly but the video motion is still jerky (not smooth). Of course less then before. Do you see it or is it just in my mind? It is good to see between 11s-14s and later.

Original HD stream .TS ~20MB:
http://rghost.net/private/50812938/e1fec4a5f9837e4c6b045048a69ec7e4

Converted as usual without SRestore ~1MB:
http://rghost.net/private/50813088/20488105ebc5ad9b16779d7ce1658caf

Converted with SRestore using different settings ~9MB:
http://rghost.net/private/50979438/c4b6daa4fc9b5f7bc9d8ec132ec30752

I have choosen this settings as best for me. But maybe someone could tell me better ;)
LoadPlugin("K:\_prevod\_pgm\DGDecNV\DGDecodeNV.dll")
DGSource("K:\_prevod\_tmp\test.hdtv.x264.dgi", deinterlace=2).SRestore(frate=25,speed=-25).Spline36Resize(720,406)#.Trim(0,100)

Thanks!

[Guest]

I have no experience with Srestore. I posted only to advise you about your approach to getting help. Now that you have made your concern more concrete, hopefully some Srestore experts may now weigh in.

Your links mention different settings but you don't say what they are. I also don't know why you are messing with the frame rate. If it is an NTSC to PAL belended conversion, then you don't force it to a PAL rate. Yes, to keep sync you need to convert the audio also. The experts will likely give you additional advice.

[asteri]

Both SRestored video 25fps and 23.976fps are still jerky. So from this point of view it is better use frate=25 and dont need to recode audio. If the 23.976fps version will be better I would convert the audio. It is not about being lazy ;)

[Guest]

Fine, ignore the advice that you asked for. I won't lose sleep over it.

BTW, where did you get this stream?

[asteri]

Maybe I didnt write it clearly, but the SRestored video with 23.976fps has the same jerky motion as 25fps version, so why use 23.976fps if i will have to convert also the audio?

It has been recorded with Technisat CableStar Combo HD CI using latest DVBViewer. It is from cable TV provider in Czech republic (UPC).

I need solve this because almost each document from this broadcast has this problem :(

Did anoone tried look at SRetored video? It has only two downloads and i have downloaded it once and my friend also once. There cant be any discussion if noone want to download it and watch it.

[Guest]

Maybe I didnt write it clearly, but the SRestored video with 23.976fps has the same jerky motion as 25fps version, so why use 23.976fps if i will have to convert also the audio? If the original before standards conversion was NTSC, then forcing it to PAL is incorrect theoretically and guaranteed to produce jerkiness.

Srestore is not perfect and you should not expect to obtain perfect results. Can any of our members do better than you did? We'll have to wait and see. My personal policy is usually to stay miles away from field-blended stuff. Some of our members find it interesting, however. Your job is to get them excited about it.

[ChiDragon]

I don't know whether he's allowed to start a new thread specifically to address correcting the field blending, but it doesn't help that this is in the seldom-used HDTV/DVB/Tivo subforum and the topic didn't actually turn out to be a capturing issue.

asteri: It may be 59.94Hz interlaced video converted to 50Hz. I don't know enough Srestore to know whether it can deal with that. See if a search pulls up anything.

[Guest]

asteri: It may be 59.94Hz interlaced video converted to 50Hz. I don't know enough Srestore to know whether it can deal with that. See if a search pulls up anything. That seems likely to me. Worth trying frate=29.97? I don't know enough about Srestore to know if that makes any sense.

[asteri]

SRestore(frate=29.97,speed=-25)
LoadPlugin("K:\_prevod\_pgm\DGDecNV\DGDecodeNV.dll")
DGSource("K:\_prevod\_tmp\test.hdtv.x264.dgi", deinterlace=2).SRestore(frate=29.97,speed=-25).Spline36Resize(720,406)#.Trim(0,100)

http://rghost.net/private/51044383/db5345e355d15378115ed6355e638b98

I think it looks the same as 23.976fps AND 25fps. Still it is not smooth , is it?

[Didée]

You cannot choose "at free will" Srestore's output framerate. It must match the actual source fps before norm conversion. Choose a lower fps, you get skipped frames. Choose a higher fps, you get duplicate frames.

This source fps is 23.976.
(edit - at least from what can be seen in the given sample. Since it seems to be a documentary, maybe the source was 59.4 fields/s indeed [presence cutscenes], with the film sections being 3:2 pulldown)

At least when using an external deinterlacer, it might be necessary to manually specify source field order (here: TFF). I'm not sure if actual DGDecodeNV delivers field order to Avisynth -- I'm a couple of revisions behind, and my older version does not report fieldorder.

The result of the following script, to me, is 99% smooth:

dgsource("test_(asteri).ts.dgi")

AssumeTFF() # without this, all the rest will be _broken_

yadif(mode=1)

sm=bicubicresize(960,540,-.5,.75)

srestore(frate=23.976,speed=-1,dclip=sm)

return(last)

... While this is for keeping the 1920x1080 resolution. If you really plan to reduce to that tiny 720x406 resolution (but, why capture fullHD in the 1st place then?), you can do

DGSource(...)
AssumeTFF()
yadif(mode=1)
bicubicresize(720,406,-.2,.6)
srestore(frate=23.976,speed=-1)

[Guest]

Is there some reason to prefer yadif here over CUVID for the bobbing?

The latest version of DGDecNV correctly passes the field order to avisynth.

[asteri]

Hello Didee!

It makes sense. I dont know if it is important but I forgot to tell you that my .dgi already contain info about cropping (exactly 4 4 0 8). I am cropping during generating the index file.

I prefer use hw bob, so let me choose as best avs this, ok?:

LoadPlugin("K:\_prevod\_pgm\DGDecNV\DGDecodeNV.dll")
DGSource("K:\_prevod\_tmp\test.hdtv.x264.dgi", deinterlace=2).AssumeTFF().SRestore(speed=-1).Spline36Resize(720,406)


1) Should I set explicitly frate=23.967 ? I see SRestore() makes output correctly as 23.967fps.
2) Should I define also dclip? I am not sure how exactly this settings work. You have used it in first avs but not in the second.
3) I see you have put resize filter before the SRestore() function (at least in the second avs script). Should I also put Spline36Resize before SRestore() ?

Thanks for all valuable info! I think I am getting close to the end of this project :)

[martin53]

I had to use older SRestore v2.7e instead of v2.7g. Newer version doesnt work fo me even if I have all dependencies installed. I am getting error message about :
"Script error: expected '{'"
"srestore.avsi, line 319, column 0"
Currently I dont use MT version of Avisynth. In future I would like to use MT version for SRestore but I guess I will need for MT SRestore v2.7g+, right?

1) The AviSynth version you use seems to stumble over comments between function head and body. Remove the string starting with the hash sign in line 319, and the whole line 320; or move the { character from line 321 between the ) and # characters in line 319.
I will not publish a 2.7h for that purpose because I just coiped that function and other users did not seem to have any problem with the script syntax as it is.

2) The decision to use 2.7e or 2.7g has nothing to do with MT.
The only restriction of 2.7e over 2.7g is you can not have more than one Srestore call in your script - e.g. you can not make a script with two different parameter sets displayed side by side.

[martin53]

asteri,
trust Didée's experience in framerate conversion, IVTC etc.

Since you also PM-ed me, find my contributions here.

(I don't own DGDecNV, nor a NVidia card. DGAVCIndex+AVCSource() (DGAVCDecode.dll, all march 2009) gave errors "Found NALU type 13, len 2 undefined, ignore NALU, moving on". The output was buggy and unusable. So I transcoded the AVC to huffyuv (XMediaRecode) and tested with AviSource() ).

HD, Deinterlace, downscaling
Because you downscale below one half of the original height 1080, I suggest you leave away all deinterlacing/bobbing and just take one field.

Motion smoothness reconstrucion
Mediainfo tells me that the example from the first post is 25fps. With telecined or otherwise framerate-crafted footage, I know three effects: duplicated fields/frames, blended frames and missing frames. In general, inverse telecine depends on the cuts and the assembly of the original. For broadcasts assembled from 50fps or 60fps interlaced video and 24fps film, obviously there is no solution that fits both without any quality loss. And if a clip was assembled after 24->30 or 24->25 or 30->25 or ... fps conversion, the inserted frames may be at different positions of the group over a cut.

The given example does indeed show blended frames, but it also puzzled me. The mentioned scene with the bottles, papers etc. shows a frame-by-frame motion scheme migrating between [normal-normal-normal] and [big-small-big] (~ frames 265, 288 and 311). Jerkyness seems to come from that: with 25fps and two frames nearly identical, then a big move, you have in fact 12.5fps.

-> I don't think Srestore can help here. Frame replacement looks even more jerky in pan scenes, and i found no suitable pattern for frame removal in the clip.

Instead, I figured out a frame realignment method with MVTools (would have proposed SVPflow, but that throws an exeption on my V2.6 x64 system), see example.
The idea is to take every 2nd frame, calculate the motion vectors and move the contents of the middle frame to the middle of the motion vectors. Pan looks good, station logo not. I didn't succeed with scene changes - so I leave that as an exercise for you :p

function FlowComp(Clip c) {
# Function takes every 2nd frame and calculates motion vectors.
# Then, MFlowInter is the result of the frames, with pixels moved to the position of the removed frame.
# Because not always all objects are visible in the first, second and third frame, there are typically atrefacts.
c
SelectEvery(2,0)
super = MSuper()
backward_vectors = MAnalyse(super, isb = true, delta=1)
forward_vectors = MAnalyse(super, isb = false, delta=1)
MFlowInter(super, backward_vectors, forward_vectors, time=50, ml=70)
# The artefacts are reduced a bit by the second process: Motion vectors between the removed frame and the synthesized frame are calculated
super = MSuper()
vectors = MAnalyse(super, isb = false, delta=1)
# And now the pixels of the removed frame itself are moved to the positions in the synthesized frame.
# So, most of the content of the 'removed' frame is in fact not removed, but it is moved to positions 50% between previous and following frame
MCompensate(c.SelectEvery(2,1), super, vectors)
Interleave(c.SelectEvery(2,0), last)
}

function CompareHor(clip c0, clip c1) {
# Function is just there to show differences between to similar clips
d= mt_makediff(c1,c0)
d= mt_lutxy(c1, c0, expr="x y - 5 * 128 +", chroma="process")
StackHorizontal(c0, c1, d)
}
#==============================================
# Input
AVIsource("test.avi")
trim(254,0)

AssumeTFF()
SeparateFields()
# Remove one field - is in fact similar to VerticalReduceBy2()
SelectEvery(2,0)

# Correct aspect ratio
HorizontalReduceBy2()

# Remove following line for final processing
CompareHor(last,FlowComp())

[Didée]

I suggest you leave away all deinterlacing/bobbing and just take one field.

If the source is field blended, the one thing you must not do is discarding one field. There will be vital information in even and in odd fields (alternating by phase of blending pattern). If you start out with discarding half the fields, I suppose that's the cause of the jerky- or jumpy-ness that you mention.

IMO Srestore does a reasonable job on this source, the script I posted gives a smooth result. There are no motion states missing in the source, and there are neither skipped nor duplicate frames in the result.

[asteri]

Hi, I am little bit busy before Christmas I guess as anybody else ;)

I think Didees script is fine BUT... I have try this method on whole documentary. The test scene is looking quite well now, unfortunatelly I have found others which are maybe worse than without using SRestore. Or maybe I am so confused from making the comparison again and again that I dont see well :D I need someone who can confirm it.

I will try prepare some more test scenes from the same documentary and show what I am talking about. Thanks for now to all. I really apreciate your help!

[martin53]

If the source is field blended.
Agree if. This test clip did not look that way to me after SeparateFields(), I think I can judge that from the motion of objects.
If they originally shot it at 60i, the fields origin from t=0 (top field) and t=8.3ms (bottom field).
If they now left away one frame to convert 60->30 fps, there is 8.3ms difference between the fields in one frame, but then 25ms to the next top field, to achieve 33.3ms between frames.
Maybe this is where the appearingly odd motion comes from. To my eyes, there was no or substantially less motion between the fields of one frame than between bottom field of preceding frame and top field of next frame.
Could be worthwile to analyse by synthesizing full frames from the fields, but I won't :p

[Didée]

Agree if. This test clip did not look that way to me after SeparateFields(), I think I can judge that from the motion of objects.
Dunno, perhaps you didn't look close enough. I looked at the source after bob-deinterlacing, and can clearly see that 1/4th~1/3rd of all bob frames are blended.The only odd thing is the actual pattern ... in this kind of norm conversion, usually one would expect to be almost half of all fields (or bob-frames) to be blended, and 2 fields (bob-frames) per second to be duplicates (or weave-able fields). In this sample there are less blends than usual, and much more dup-things than usual. (Perhaps they have discovered a forgotten knob on the normconverter box, enabling an alternative blending pattern ... :D )

The test scene is looking quite well now, unfortunatelly I have found others which are maybe worse than without using SRestore.
That's quite possible. As already mentioned - if the original source actually was a mix of 24p film sections and 60i (maybe 30p) NTSC-sections, then after normconversion you're facing a real problem. In this case you can't get the thing done in one go by an automated tool. You would have to isolate the different sections manually, treat them seperately, and finally either create a VFR (variable framerate) MKV file, or speed the unblended 24p section up to 25p, to merge them with the NTSC sections (which would be left in their blend-converted 25i stage).