HCEnc Progressive PAL for a Progressive PAL DVD

[jclampy]

I got an old capture that my brother got via an analog capture 7 years ago. It looks like it was originally interlaced NTSC and then deinterlaced when converted to PAL before being broadcasted. I assume this because there are blended frames. When captured it is in interlaced form because of analog signal but when seperated into fields it shows every pair of fields as the same like progressive. There is no combing with fast movement, just blurring as mentioned earlier.

The capture has strong artifacts so I have used QTGMC to deinterlace the footage and then SelectEven after. I had to hit it with 2x derainbow filters and a Temporal & Spatial denoiser, I also made a minor tweak to colour calibration.

I want to keep the footage progressive as I have it now and put it on a DVD so it is also DVD compliant.

In HCGUI 0.26 I am using these settings;
'interlacing options' = progressive
'chroma downsampling' = progressive
'progressive sequence' = ticked
bff (Looks like field order was changed after QTGMC deinterlace, this keeps text from jittering on CRT TV set, no difference when viewed on LCD display)

1. Can somebody please tell me if these are the correct settings for the best method of creating a Progressive PAL DVD that is DVD compliant?

2. I saw a post mentioning "Pulldown Table of Truth" and guess what I am doing above is 'number 29', but should I be doing 'number 13' or 'number 5' instead? If so what HCEnc settings should I change?

3. I see some examples that talk about RFF I assume that is no use to me. I guess that since my material is 25fps and played at 25fps I won't be using RFF. Out of Curiosity for the future though, where would one set that because I couldn't find a setting in HCEnc or ReStream? Would it be set in a pulldown tool?

Thankyou in advance for any help.

[Richard1485]

'progressive sequence' = ticked

Despite the fact that having the progressive sequence checked is, apparently, technically valid for PAL, I have never been able to create a PAL DVD that plays with this option checked. You don't need it, so I would uncheck it. You don't need a pulldown if your source is as you say it is.

TFF or BFF shouldn't matter if your video is now progressive -- regardless of the fact that is was interlaced in the past. As I understand it, DVD is usually TFF, but there's nothing wrong with its being BFF. The other settings look fine to me. I believe that the encoder sets RFF flags for you without your having to do anything in terms of settings.

[TheSkiller]

I have never been able to create a PAL DVD that plays with this option (progressive sequence) checked.That's interesting, everything progressive that I have ever put on DVDs had this flag and "frametype progressive" (but not TFF) set (using Restream) and I have never experienced any problems with many different players. Actually I noticed one important thing about the "progressive sequence" flag:

You don't need it, so I would uncheck it.In my experience it is the only flag that really effectively completely disables a progressive scan DVD player's deinterlacing. Without it (only "frametype progressive" with or without TFF flag set) progressive scan players still try to deinterlace the video, that is I have yet to find one which doesn't do that without progressive sequence set.

There are things to consider:
"frametype progressive" + "progressive sequence" + "TFF" is illegal, TFF must be false (unset) if progressive sequence is set. I haven't tried it with TFF set, but maybe that's what was causing you trouble with the progressive sequence flag?

I know for a fact that "frametype progressive" + "progressive sequence" without TFF flag is a legal combination for DVD video (it has been discussed here), it should cause no problems.


Edit:
There is no combing with fast movement, just blurring as mentioned earlier.

The capture has strong artifacts so I have used QTGMC to deinterlace the footage and then SelectEven after.
Well but in that case it is not interlaced therefore needs no deinterlace. If you want to use QTGMC to enhance the image quality, use InputType=1 (read the documentation).

[mp3dom]

Yes, if you set the progressive sequence it means that ALL the stream is indeed progressive so a progressive scan player doesn't ever need to deinterlace. If you mark only frametype progressive, it means that *that* frame is progressive but the sequence can potentially have a mixture of both progressive and interlaced contents.
Progressive sequence is in specs as long as frametype progressive is ON and TopField is OFF. I've made a lot of commercial dvds in that way and none of them have created any problem.

[jclampy]

Thankyou TheSkiller and mp3dom for your excellent responses you have cleared up the situation and answered my question perfectly. :cool:


Despite the fact that having the progressive sequence checked is, apparently, technically valid for PAL, I have never been able to create a PAL DVD that plays with this option checked. You don't need it, so I would uncheck it. You don't need a pulldown if your source is as you say it is.


I have tested it on a few players here now and it works fine with 'progressive sequence ticked'. Only problem is making sure the maximum bitrate doesn't peak too high especially for the old players or it becomes unwatchable.


TFF or BFF shouldn't matter if your video is now progressive -- regardless of the fact that is was interlaced in the past. As I understand it, DVD is usually TFF, but there's nothing wrong with its being BFF. The other settings look fine to me. I believe that the encoder sets RFF flags for you without your having to do anything in terms of settings.


Although I can confirm this is true when shown on a progressive display like LCD TV or LCD PC. It can have an effect on old Analog TV's. I only noticed it where the intro text was jumping up and down, I couldn't notice any issues after watching about 30 seconds of video footage although it must effect everything throughout. Footage was shot with hand cams and viewed on a small 13inch screen so not easiest to see the problems. Once I flicked TFF off to make it BFF the text was completely smooth on the Analog TV same as when viewed on LCD TV or LCD PC.

[Richard1485]

That's interesting, everything progressive that I have ever put on DVDs had this flag and "frametype progressive" (but not TFF) set (using Restream) and I have never experienced any problems with many different players.

That is indeed interesting, because I tested discs with this flag set on many players, and they all refused to play it. I shall experiment again. Thank you for your information.

In my experience it is the only flag that really effectively completely disables a progressive scan DVD player's deinterlacing.

Yes, I agree that that is the idea behind the progressive flag.

There are things to consider:
"frametype progressive" + "progressive sequence" + "TFF" is illegal, TFF must be false (unset) if progressive sequence is set. I haven't tried it with TFF set, but maybe that's what was causing you trouble with the progressive sequence flag?


I don't recall setting TFF (because it's indeed illegal) when I tried to use the progressive sequence, but I suppose it is possible.

I know for a fact that "frametype progressive" + "progressive sequence" without TFF flag is a legal combination for DVD video

Oh, I agree that it's legal; it's simply that I never had discs work when I did it. I'll try again and see what happens.

Although I can confirm this is true when shown on a progressive display like LCD TV or LCD PC. It can have an effect on old Analog TV's.

That's interesting. I understood that if the stream is fully progressive it shouldn't matter which you select. I have never experienced this myself, but it's been a long time since I had an analog TV and I almost always encode NTSC, so thanks for the information.

[hank315]

Just informational, from DVD Demystified by Jim Taylor:

Technically,DVD-Video can only be stored in interlaced format. Signals from standard video cameras are already in interlaced format.
Film, which is inherently progressive, is encoded into MPEG-2 as paired fields.
Even though the encoding is field-based, the frame-based nature of the source can be preserved.

In MPEG-2 encoding, the decision between progressive and interlaced format can be made all
the way down at the macroblock level.The DVD-Video specification limits MPEG-2 video to nonprogressive
sequences, which can include both progressive and interlaced frames.
Progressive frames are still encoded for display as two fields, but they are identified as progressive.
Interlaced frames can further include both progressive and interlaced macroblocks. Since progressive
macroblocks are more efficient (using one motion vector instead of two), even interlaced source
is often encoded with more than 50 percent progressive macroblocks. However, each frame is represented
as two fields of 720 x 240 pixels each for NTSC or 720 x 288 pixels each for PAL/SECAM.

[Richard1485]

Hank, I know you just posted that for information, but what do you do when you encode progressive PAL?

[jclampy]

I decided to put this response to seperate post as I have tested and now report on results.



There is no combing with fast movement, just blurring as mentioned earlier.

The capture has strong artifacts so I have used QTGMC to deinterlace the footage and then SelectEven after.



Well but in that case it is not interlaced therefore needs no deinterlace. If you want to use QTGMC to enhance the image quality, use InputType=1 (read the documentation).


After checking this it looks like the footage was deinterlaced when converted from NTSC to PAL which is where the 'blurred' frames come from. Then was broadcasted as interlaced, whether just stream or footage I am not experienced enough to say. My brother captured it using Analog connection at height 576 with HFYU so I guess this part is interlaced anyway. So at this point when played it has interlacing lines not just on moving parts but what looks like the whole screen. Maybe it has two lots of interlacing I dunno know. You can also see certain blurred frames on fast movement panning.

If seperatefields() is performed the interlacing lines are not visible as far as I can tell. The footage becomes 50 framerate of 25 identical pairs so there is no strange sequence pattern. Fast movement shows blurred pairs. I take it this means to get it back to progressive frames I have to do a 'full deinterlace' and cannot IVTC.

Anyway, after running;
QTGMC(Preset="medium",NoisePreset="medium",InputType=1)
The interlacing lines are still present.

After running;
QTGMC(Preset="medium",NoisePreset="medium")
All interlacing lines are gone but has become 50 framerate of 25 identical pairs. So I put SelectEven() afterwards.

I hope that is a bit clearer to understand what is going on. You probably understand it more than me, I'm just happy it appears to be working and I hope I am doing it right. :o

PS: I only deinterlace because I need to do some heavy filtering to clean the capture up. It has vertical rainbow stripes and chroma artifacts plus do some denoising for compressibility to avoid macroblocking.

[Richard1485]

So at this point when played it has interlacing lines not just on moving parts but what looks like the whole screen. Maybe it has two lots of interlacing I dunno know.

Could his source be phase-shifted?

[hank315]

@Jeff B
In HCenc, I would untick progressive sequence in settings2 and tick progressive and BFF in settings1.
Setting progressive sequence doesn't improve encoding efficiency, setting interlaced for progressive sources certainly lowers encoding efficiency and not only for the reasons mr. Taylor mentions.

[Richard1485]

In HCenc, I would untick progressive sequence in settings2 and tick progressive and BFF in settings1.

Thank you. Apart from BFF, that is basically how I have always encoded progressive PAL.

setting interlaced for progressive sources certainly lowers encoding efficiency and not only for the reasons mr. Taylor mentions.

Oh, yeah. I certainly wouldn't encode it as interlaced in terms of the interlacing options in settings 1.

[jclampy]

@Jeff B
In HCenc, I would untick progressive sequence in settings2 and tick progressive and BFF in settings1.
Setting progressive sequence doesn't improve encoding efficiency, setting interlaced for progressive sources certainly lowers encoding efficiency and not only for the reasons mr. Taylor mentions.

Hi Hank, this sounds interesting I will give it a try and see what happens. I don't know what the output will look by reading settings I am still new so will only see difference after I have encoded a test. I am wanting to try and keep it looking progressive for playback on PC while still be DVD compliant to play on old Analog TVs & LCD TVs.

Thanks will let you know what happens, I appreciate it. :D

[TheSkiller]

Could his source be phase-shifted?Possibly, it's hard to say without a sample. If it is phase-shifted then there would be "interlacing" or whatever jclampy sees only when there is movement, still parts would be fine.


So at this point when played it has interlacing lines not just on moving parts but what looks like the whole screen.

Hm, maybe the fields are simply swapped. This can happen with capture cards or during decoding of the captured file (especially if the video was captured years ago and is now decoded with a different version of the codec that encoded it during capture).
Does SwapFields() fix it? I mean just SwapFields(), no QTGMC.

[jclampy]

Hm, maybe the fields are simply swapped. This can happen with capture cards or during decoding of the captured file (especially if the video was captured years ago and is now decoded with a different version of the codec that encoded it during capture).
Does SwapFields() fix it? I mean just SwapFields(), no QTGMC.

SwapFields() made the interlacing more pronounced.

I might post a sample if it is not too hard for me to do. What would be the easiest way, screenshots or video sample? If video then what is quick method to get a small unedited piece for you to have a look at please? Source is a 24GB Huffy .avi file.

[Richard1485]

Jclampy, you can load the capture into Virtualdub and mark out and save a small sample.

[TheSkiller]

I see.

Regarding a sample, video is preferable, 10 seconds is plenty. Screenshots usually don't help much.
An easy way to do it:
Open the original Huffyuv video in VirtualDub,
go to video and click on "Direct stream copy" (very important), go to audio and click "No audio".
Mark a sample to output using the "Home" and "End" buttons (beneath the numpad).
Save as AVI.
Upload to mediafire and post the link.

[jclampy]

Ok here is a link to the sample; http://www.mediafire.com/?fjd08d2na26jot0

Here is my script as it stands at the moment;
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
AviSource("H:\capture.avi",false,pixel_type="RGB32") # [AUDIO DISABLED]
converttoYV12()
AssumeFrameBased()
AssumeTFF() # [CAPTURE IS TOP FIELD FIRST]
ChromaShiftSP(0.15,1.50)
QTGMC(Preset="medium",NoisePreset="medium",EZDenoise=0,EZKeepGrain=0,NoiseProcess=0,ShowSettings=false)
SelectEven()
DeFreq(fx=14.5,fy=0.0,dx=1.5,dy=1.0,sharp=1.0,plane=2,show=0,info=false) # [SET FOR CHROMA (V) ONLY]
SmoothUV(6,60) # [ATTACKS CHROMA ONLY (UV), NOT LUMA (Y)]
SmoothLevels(smooth=100,input_low=16,gamma=2.0,input_high=235,output_low=16,output_high=235,limiter=2,Lmode=2,show=false)
#SmoothTweak(smooth=100,saturation=1.3,hue1=3,hue2=3,limiter=true,show=false) # [DEACTIVATED SEE EDIT BELOW]
FluxSmoothT(6) # [TEMPORAL DENOISER]
BilateralFilter(1,3,3.0) # [SPATIAL GPU BILATERAL DENOISER]
#DctFilter(1,1,1,1,1,1,.5,0) # [DEACTIVATED SEE BELOW]
Letterbox(14, 14, x1=4, x2=4, color=$000000)
#histogram(mode="color2")
#histogram(mode="levels")
#info()
#
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Note: I would have activated DctFilter but I get error message when used with BilateralFilter so I don't bother to use DctFilter;
Traceback (most recent call last):
File "F:\AvsPmod\src\AvsP.py", line 7136, in OnMenuVideoNextFrame
File "F:\AvsPmod\src\AvsP.py", line 11385, in ShowVideoOffset
File "F:\AvsPmod\src\AvsP.py", line 11135, in ShowVideoFrame
File "F:\AvsPmod\src\AvsP.py", line 11734, in PaintAVIFrame
File "pyavs.pyo", line 343, in DrawFrame
File "pyavs.pyo", line 320, in _GetFrame
File "avisynth.pyo", line 277, in GetFrame
WindowsError: exception: access violation writing 0x00000000

I think the script at the moment does a pretty good job but I'm open to improvements.
Also, if you need more samples I could do a few more as the rainbow interference gets real bad at some parts. Looks like it goes into the Luma channel sometimes which I haven't been able to eliminate without ruining the picture completely.

Edit:
I decided to deactivate SmoothTweak in my script above at the moment because the settings I was using blur's the colours too much on my LCD PC screen and makes it hard to watch. In the future I may finetune my settings with that filter or just not bother with saturation and hue adjustments at all.

[jclampy]

@Jeff B
In HCenc, I would untick progressive sequence in settings2 and tick progressive and BFF in settings1.
Setting progressive sequence doesn't improve encoding efficiency, setting interlaced for progressive sources certainly lowers encoding efficiency and not only for the reasons mr. Taylor mentions.

Sorry Hank, I know you were responding to Jeff there but I tried that on my footage just to compare and I am not sure if it was better suited or not. I guess I got myself a bit confused when you say "untick progressive sequence" and then "setting interlaced for progressive sources certainly lowers encoding efficiency". You are referring to 'number 13' on the "Pulldown Table of Truth" list right?

So just to be sure, since I deinterlaced before encoding then I should use these settings in HCEnc right;
'interlacing options' = progressive
'progressive sequence' = ticked
bff
Which would be 'number 29' on the "Pulldown Table of Truth".

Or should I use 'number 13' because it would show full 'temporal motion' (sorry can't remember correct term) whereas 'number 29' is only giving me half 'temporal motion' or something like that?

With 'number 13' I couldn't see interlacing on playback on my LCD PC with a test encode so I guess unticking 'progressive sequence' doesn't interfere in that regard atleast.

PS: Is anything that is 'legal' on the "Pulldown Table of Truth" list mean that it is also fully compatible with DVD compliance and playback?

[manono]

It was capped without the use of a line TBC which is why it seems to be slightly interlaced. Unless you can go back and capture it from the VHS tape properly, I don't guess you do any better than bob it followed by selecting every other frame. It's not out-of-phase but looks to me to have been from a progressive source, before the capture was screwed up. I think. :)

[jclampy]

Don't have a line TBC so couldn't help that.

I am relatively sure it was not captured from video tape but was captured directly from TV broadcast so can't redo the capture. Also, was captured in Jan 2005.

So I guess you are saying I am converting what I have to progressive via an ideal method then, if so, thankyou for confirming that. :cool:

Although I am not quite sure of where you are exactly refering to 'progressive source' as was captured through old TV antenna of a VH band of UHF signal broadcast. So is analogue so can't be 'progressive source' right? If that is correct then it would mean was screwed up somewhere before being broadcasted (like maybe when it was converted from NTSC to PAL, if it was). Or ofcourse I could be completely wrong? ;)


PS: I will ask my brother and see if he can remember if it was recorded to video tape before he captured it. I don't think he did because he had started mucking around with a PC TV Tuner around that time. Out of curiosity may I ask what setting/s do you think were set wrong when capturing if it was from video tape as I am contemplating backing up some old video tapes I have in storage in the future (hopefully not too far in the future).

Thanks.

[manono]

Ok, maybe it wasn't from a VHS tape (although it looks like it to me), but somehow parts of the fields got slightly offset from each other which created that slight interlaced effect.

It was from a progressive source (probably encoded as interlaced). It's not interlaced (except for that screwy little bit of interlacing from the fields not lining up exactly right), wasn't shot using video cameras, and therefore has to have come from a progressive source once upon a time. It even has something called Roax Films on screen during about two-thirds of your sample. If it was shot on film, by definition it was progressive. It would have been broadcast, of course, as 25i.

I'm far from an expert when it comes from capping VHS tapes, but almost a requirement for successful capturing, I think, is the addition of both a line TBC and a full frame TBC into the capture chain. Perhaps others around here can chime in with more informed opinions as to what's going on with that sample of yours.