HCenc problem with "partially interlaced" source?

[simonhowson]

I have spent the last day trying to convert a 16:9 anamorphic 720 x 576 (PAL) dvr-ms (digital TV) source to a DVD compatible MPEG2 file. I have followed these steps:

1) Use DVRMSToolbox to convert (without re-encoding) the dvr-ms file to a standard MPG file.

2) Load this MPG file in MPG2cut2 to cut out the start and end of the clip so that I only encode the movie that I want. I then write this out to a new MPG file which is only the edited section I want.

3) I load this cut file into the D2V creator in MeGUI which demxes the audio and video.

4)

a) I then load this D2V file in MeGUI AviSynth script creator which detects the source as "partially interlaced" and suggests the use of YADIF to deinterlace it. I follow this, and add a basic filter to smooth the image a bit.

b) I use Autocrop to crop the sides of the image, and resize to enlarge it to 720 X 576 (PAL) non-anamorphic.

(The movie I want to convert is an old 4:3 film, however this is set within the 16:9 anamorphic broadcast image so it has big black bars down the sides. So the cropping and resizing is designed to convert it to a 720 x 576 non anamorphic image that takes up the entire 720 x 576 non-anamorphic format.)

5) I run HCenc23 and load this AVIsynth file. But here I am confused. I don't know if HCenc will follow the AVIsynth script and deinterlace the source using yadif before encoding it, or do I need to change the interlacing options on the "Settings 1" tab? I left it at automatic, but this seems to have left the output as "partially interlaced" which is how MeGUI detected it.

I noticed this when I played back the output file in Nero Showtime, it played back mainly interlaced, but sometimes changed to progressive, which caused horrible stuttering at those points. Ideally I want the output to be completely progressive, because I think this will look the best.

Other questions: Is there an easy way to add chapters every 5 minutes? I don't care where they go. Just some chapters would be better than none.

I ran HCenc via HCENC^n.exe for my dual core machine. Is this necessary? Or does HCenc now support dual CPUs natively? On the Main tab under "miscellaneous" there is an option called SMP, does this mean muti-threading?

Once I finished encoding the video with HCenc, I have a demuxed audio track (from MeGUI d2v creator) that says it has a -16 ms delay. Is there anyway to remove the delay from this file without re-encoding it? I tried muxing the file using ImagoMPEGMuxer, but the audio was not synchronised. I do not know if it was because of the delay in the audio file, or if it was a side effect of the partially interlaced problem.

This is quite a complicated process to get this all to work. But if it does I will be very happy because it will be high quality, fast, and free. I am currently encoding the same clip using a trial of TMPGenc 4 XPress to see how it deals with the partial interlacing problem. But I would prefer to get it to work using HCenc, because it encoded the file in 80 minutes, whereas TMPGenc 4 doesn't seem to use both cores of my CPU, and thus is going to take about 2.5 hours, and it costs US$100 for a full version!

If I can get this to work in HCenc I will donate to the developers. It seems to be a very fast encoder, but I just can't the partial interlaced problem solved, and don't know how to mux the audio properly.

So to sumamrise:

How do I get HCenc to follow the AviSynth script and deinterlace a partially interlaced source, and output a progressive file?

How do I mux the audio, do I have to re-encode the audio to remove the delay? Or is there a way to do this automatically without re-encoding?

My encode using TMPGenc 4 just finished. The video has been properly deinterlaced, but it doesn't look good as the HCenc output. However the big problem is that the file doesn't have any sound, even though I told it to encode the sound!

[smok3]

uhmm, i would skip yadif and encode the file as interlaced, using HCenc (carefully check your avs skript, as some filters may not work fine with interlaced content), and yes hc will 'follow' your skript, it really has no idea about what was there before the script processing (if that is your question)?

about cropping: only crop width, leave height as it is, so you don't break the interlacement (at least thats the simplest approach), also scale only width as well.

about chapters: if you are using say muxman for authoring, it will accept a 'chapter' text file like this:
176
15186
30175
45175
60163
(frame count)

[simonhowson]

uhmm, i would skip yadif and encode the file as interlaced, using HCenc (carefully check your avs skript, as some filters may not work fine with interlaced content), and yes hc will 'follow' your skript, it really has no idea about what was there before the script processing (if that is your question)?
So encoding it interlaced is the best option, even though it seems to have some progressive sections?

I am getting this weird stuttering at the progressive parts, especially noticeable on quickly moving objects. NeroShowtime has can display file properties as the M2V plays, and it shows the bitrate fluctuating, and the format fluctuating between interlaced and progressive.

Could it be that DVRMSTools isn't demuxing the file from dvr-ms to MPEG properly? Is there something else in the file interfering with the image?

about cropping: only crop width, leave height as it is, so you don't break the interlacement (at least thats the simplest approach), also scale only width as well.
Ahh, thanks. I had no idea this could be a problem. Autocrop was taking 2 (pixels?) from both the top and bottom because there was a thin black line.

I'll make another script without any top cropping, and without any deinterlacing and see what happens.

Thanks for your help.

[simonhowson]

about chapters: if you are using say muxman for authoring, it will accept a 'chapter' text file like this:
176
15186
30175
45175
60163
(frame count)

Does it make a difference if I do the chapters in HCenc using the GUI? I have tried to generate the chapter list by scanning the video using the scene cutting detection, but it is very slow - I'd rather spend that time encoding the file!

If I want a chapter every 5 minutes, is that every 7500 frames?

5 minutes = 5 x 60 = 300 seconds. 300 x 25 fps (PAL) = 7500

So I just make a list that says:

7500
15000
22500
30000

etc?

[smok3]

i think chapters in HC are only so that the encoder knows that thats the place to force an I frame and close the GOP (but yes you can use the HC as the GUI and then generate a text file with the same chapter info which you use later in muxman).

p.s. yes, your 5 minute approach should work.

[simonhowson]

i think chapters in HC are only so that the encoder knows that thats the place to force an I frame and close the GOP

Ooops! OK I'll have to encode again, and leave the chapters to after.

[smok3]

p.s. another thing, check the field order in interlaced parts, can be TTF or BFF (i think there is an 'auto' switch in HC now thought?)

[simonhowson]

p.s. another thing, check the field order in interlaced parts, can be TTF or BFF (i think there is an 'auto' switch in HC now thought?)

According to the analysis scan in MeGUI it is Top Field First, so that is the option I used in HCEnc. But I selected INTERLACED rather than leaving it at Auto.

I was worried if I left it at auto that it would be confused by the progressive parts.

Basically this time I am encoding from a script that will crop the sides of the image, that is all. I wanted to see if this works before complicating it with other filters.

[simonhowson]

Damn! Encoding it interlaced didn't work. It still stutters and changes from interlaced to progressive during playback!?

I think the problem is the way the audio is arranged (interleaved?) in the file. I think the video is interlaced all the way through, but the position of the audio relative to the audio seems to change, which gets interpreted as making the file progressive.

Does anyone have any idea how to fix this?

[smok3]

a. how did you mux it?
b. what kind of audio is it? samplerate, # of channels, ect?

[simonhowson]

a. how did you mux it?
b. what kind of audio is it? samplerate, # of channels, ect?

I didn't mux it, I just played back the M2V file from HCenc.

The audio out of the dvr-ms is MPEG 1 Layer II 224 Kbps Stereo 48 KHz.

Here is a very short sample of the clip.

http://www.megaupload.com/?d=WFGAIXNO

It would be great if anyone can figure out what is wrong with it. It stutters (switches between interlaced and progressive) in every app I play it in VLC, Nero Show Time. I think it plays properly in Windows Media Player, but it plays back without sound!

This clip is a small edited fragment taken from after the file was converted from dvr-ms to MPEG by DVRMSTools.

[manono]

Hi-

It was converted from the original 24fps film to 25fps PAL by field blending. To restore it to 24fps and rid yourself of the interlacing and blending you might try:

Ord = last.getparity() ? 1 : 0
Yadif(Mode=1,Order=Ord)
MRestore(Numr=24,Denm=50)

[MrC]

Let's see if an all-in-one converter (like AVStoDVD) can do properly that job.

1) Use DVRMSToolbox to convert (without re-encoding) the dvr-ms file to a standard MPG file.

2) Load this MPG file in MPG2cut2 to cut out the start and end of the clip so that I only encode the movie that I want. I then write this out to a new MPG file which is only the edited section I want.

Ok, even if clip trimming can be done directly in AVStoDVD ('Edit Title'/'AviSynth'/'Trim')

3) I load this cut file into the D2V creator in MeGUI which demxes the audio and video.

Not necessary. Load your mpg directly in AVStoDVD: it will handle both audio and video.

4a) I then load this D2V file in MeGUI AviSynth script creator which detects the source as "partially interlaced" and suggests the use of YADIF to deinterlace it. I follow this, and add a basic filter to smooth the image a bit.

Look at video info in AVStoDVD: if you read "PPF" source is progressive, if you read "TFF" source is interlaced Top Field First and if you read "BFF" is interlaced Bottom Field First. You can go to 'Title Edit'/'Video' and customize output settings.

b) I use Autocrop to crop the sides of the image, and resize to enlarge it to 720 X 576 (PAL) non-anamorphic.
(The movie I want to convert is an old 4:3 film, however this is set within the 16:9 anamorphic broadcast image so it has big black bars down the sides. So the cropping and resizing is designed to convert it to a 720 x 576 non anamorphic image that takes up the entire 720 x 576 non-anamorphic format.)

Set 'Title Edit'/'Video'/'Video Aspect Ratio' to 4:3 and then play with 'Crop' in 'Edit Title'/'AviSynth'

5) I run HCenc23 and load this AVIsynth file. But here I am confused. I don't know if HCenc will follow the AVIsynth script and deinterlace the source using yadif before encoding it, or do I need to change the interlacing options on the "Settings 1" tab? I left it at automatic, but this seems to have left the output as "partially interlaced" which is how MeGUI detected it.

In 'Title Edit'/'Video' you can select HCenc (0.23) or QuEnc (0.75). HCenc is automatically choosen if bitrate is low.

In main window press 'Output Setup' and select output type. Then START.

Let me know if you give a try to AVStoDVD and if the result is according to your expectations.

:)

Bye

[simonhowson]

Hi-

It was converted from the original 24fps film to 25fps PAL by field blending. To restore it to 24fps and rid yourself of the interlacing and blending you might try:

Ord = last.getparity() ? 1 : 0
Yadif(Mode=1,Order=Ord)
MRestore(Numr=24,Denm=50)

Shouldn't I leave it at 25 fps, because I want to end up putting this on a PAL 4:3 DVD?

Can you explain in a bit more detail what the full script should look like. I'm not very good with scripting, I only know the absolute basics.

Is Ord a built in fuction or do I download it somewhere?

What would a full script look like to get rid of the partial progressive sections, and crop the video?Let's see if an all-in-one converter (like AVStoDVD) can do properly that job.
Thanks for suggesting AVStoDVD I had never heard of it before.

Unfortunately I have two problems:

1) it crashes whenever I try to preview the output.
2) It won't let me change the output DVD format from 16:9 to 4:3.

The capture format is 720 X 576 anamorphic. But it is a 4:3 movie within this widescreen image. I want to crop away the black bars, but then end up with a 720 x 576, 4:3 DVD.

[manono]

Hi-
Shouldn't I leave it at 25 fps, because I want to end up putting this on a PAL 4:3 DVD?
After encoding it for 24fps and 4:3, run the MPV created through DGPulldown set for 24->25fps to get it to output 25fps. Here's a PAL VOB file of your sample encoded with HCEnc, with pulldown applied by DGPulldown, and authored to DVD by Muxman:

http://www.mediafire.com/?tcjwmyy9zx1

To keep the size down (11.77MB) I didn't give it as good a quality as I might otherwise.
Can you explain in a bit more detail what the full script should look like. I'm not very good with scripting, I only know the absolute basics.
Here's what I used:

LoadPlugin("D:\AviSynth Stuff\Dlls\DGDecode.dll")
LoadPlugin("D:\AviSynth Stuff\Dlls\MT_MaskTools.dll")
LoadCPlugin("D:\AviSynth Stuff\Dlls\Yadif.dll")
LoadPlugin("D:\AviSynth Stuff\Dlls\TIVTC.dll")
MPEG2Source("E:\Test\test.d2v")
Import("D:\AviSynth Stuff\plugins\MRestore.avs")
Ord = last.getparity() ? 1 : 0
Yadif(Mode=1,Order=ord)
MRestore(Numr=24,Denm=50)
Crop(96,0,-98,-2)
Lanczos4Resize(704,Height)
AddBorders(8,0,8,2)
Greyscale()

Adjust the paths and names for your needs. You'll need the R_Pack and some .dlls:

http://forum.doom9.org/showthread.php?p=673243#post673243

Stick the included MRestore.avs somewhere and import it into the script. Make sure to have the latest TIVTC. Ord is built in (sort of). You probably don't even need to use it but it protects you in case of field order changes, or if you don't know how to find the field order.

[simonhowson]

Your sample VOB playsback smoothly, that is exactly how I want it to work out!

Thank you so much for that detailed script. I think I downloaded all the right filters. It is currently encoding, it is going to take 2 hours.

However I have a question. Why did you resize it to 704, rather than 720? Why was it necessary to add borders to the image?

Was "Greyscale" used to speed up the encoding, or is that the easiest way to get rid of the blue TV station watermark?

[MrC]

Thanks for suggesting AVStoDVD I had never heard of it before.

Unfortunately I have two problems:

1) it crashes whenever I try to preview the output.
2) It won't let me change the output DVD format from 16:9 to 4:3.

The capture format is 720 X 576 anamorphic. But it is a 4:3 movie within this widescreen image. I want to crop away the black bars, but then end up with a 720 x 576, 4:3 DVD.

1) That may depend from your DirectShow codec installed. You can try installing ffdshow (and enable MPEG2 in supported codec). If not go to 'Settings'/'Path' and toggle on 'Use External Player'
2) I forgot to tell you to go to 'Settings'/'Audio/Video' and toggle off 'Keep/Transcode DVD compliant video'. Then set 'Title Edit'/'Video'/'Video Aspect Ratio' to 4:3 and play with 'Crop' and/or 'Vertical Resize' in 'Edit Title'/'AviSynth'

;)

Bye

[manono]

Why did you resize it to 704, rather than 720
Well, it is 720x576, but with small black bars added on the left and right. I'll happily bow to the AR experts on this, but I seem to remember that when converting 4:3 to 16:9, you resize to 540 width and add bars to bring it up to 720. This one, with the black bars removed, was 524 wide, and I figured it was supposed to be either a 704 width, or a 720 width with black added to fill it out from 704. But like I said, I'm no expert on that subject. The crop (and AddBorders) along the bottom was to get rid of the bobbing/unblending artifacts.
Why was it necessary to add borders to the image?

You don't have to, but I like the more common 720 width, even though 704x576 is perfectly legit.
Was "Greyscale" used to speed up the encoding, or is that the easiest way to get rid of the blue TV station watermark?
It neither speeds up the encoding nor gets rid of any watermark. It turns the watermarks grey which, to me, looks better than those color things. If I were doing it myself I'd get rid of them entirely, but that's not a suject for this forum.

[simonhowson]

Thanks for your help, however, unfortunately my encode didn't work. It still stutters, and the audio seems to be very slightly out of sync.

This is the script I used:

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\DGDecode.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MT_MaskTools.dll")
LoadCPlugin("C:\Program Files\AviSynth 2.5\plugins\Yadif.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\TIVTC.dll")
MPEG2Source("f:\devilthumbs\devilcut.d2v")
Import("C:\Program Files\AviSynth 2.5\plugins\MRestore.avs")
Ord = last.getparity() ? 1 : 0
Yadif(Mode=1,Order=ord)
MRestore(Numr=24,Denm=50)
Crop(96,0,-98,-2)
Lanczos4Resize(704,Height)
AddBorders(8,0,8,2)
Greyscale()


Are you able to upload your copies of the relevent filters in a zip? It is possible I've downloaded the wrong versions, which is why it didn't work properly. It still seemed to stutter, and played back at 23.8 (!?) fps, rather than 25.

[manono]

Out of synch audio is easy to fix. That's the least of your worries. I had to resynch mine as well. After loading the audio into Muxman set the delay for -300ms. It's somewhere around there.

If you got the script to work then you have all the required filters. When you play back your encode of the section I did of your sample, does it play jerky also? Because what might work for one section (the sample, for example), may not work for other sections.

And rather than wait for the whole thing to encode, you can encode parts to see if the script is working. Use the Trim command.

Trim(10000,20000)

added to the bottom of the script will encode just 10000 frames starting at frame number 10001.

I don't know what settings you used in HCEnc. Make sure you don't hit that "Make DVD Compliant" button, and ignore the warnings at the bottom about framerate not being compliant and a resolution/frame rate mismatch. In fact, you want to see those warnings.

[simonhowson]

Out of synch audio is easy to fix. That's the least of your worries. I had to resynch mine as well. After loading the audio into Muxman set the delay for -300ms. It's somewhere around there. OK I'll worry about that later.

If you got the script to work then you have all the required filters. When you play back your encode of the section I did of your sample, does it play jerky also? Because what might work for one section (the sample, for example), may not work for other sections. OK. The script worked, but I was thinking that maybe I had older versions or something.
I don't know what settings you used in HCEnc. Make sure you don't hit that "Make DVD Compliant" button, and ignore the warnings at the bottom about framerate not being compliant and a resolution/frame rate mismatch. In fact, you want to see those warnings.
Ahh! Of course. I encoded it at 25 fps, but obviously it needs to be encoded at 24 fps. Hopefully this is the problem!

I checked the "progressive" box under the interlacing options. Is this right? Or should I have left it at AUTO?

It is currently encoding with:

scan zigzag
interlaced no
TFF/BFF NA

Are there any other HCenc settings I need to adjust?

It definately gave me the non compliant and frame mismatch warnings.

The test VOB you made played back perfectly on my PC in VLC and Nero Showtime. No stutters at all. That section you encoded is indicative of the entire film.

Thanks so much for your help.

[manono]

Although most PAL DVDs are encoded as interlaced, and although your source file is encoded as interlaced, because DGPulldown wants a progressive source to apply pulldown, I'd suggest encoding as progressive. Other than that, I'd prefer not to make encoder setting recommendations for several reasons. I usually use CCE and don't claim to be an HCEnc expert. There are many here, including the developer himself, who are far more qualified to give HCEnc recommendations. The settings I use for my own encodes aren't universally approved of anyway. And I joined this thread to tell you what was wrong with your source and how to fix it, not to tell you how to set up your encoder. Good luck.

However, if you are still having trouble getting a smooth playing result, upload a small 10 second section of your jerky-playing video, and I (and others) can probably figure out where you went wrong.

[hank315]

I also tested it using the Avisynth script from manono.

Settings for encoding this with HCenc:

*INFILE yourAVS.avs
*OUTFILE yourOUT.m2v
*BITRATE 3000
*MAXBITRATE 8000
*PROFILE best
*ASPECT 4:3
*AUTOGOP 12
*AQ 2
*PROGRESSIVE
*AVSRELOAD
*MATRIX mpeg
As manono said, ignore the warnings, HCenc can't do 24 --> 25 pulldown, so use DGPulldown after encoding for setting the pulldown flags which will make it DVD-compliant.

The result plays without stuttering here.