Discussion: Generating encodes suitable for HDTV media players

[SeeMoreDigital]

The idea behind this particular thread is to enquire about methods of converting Mpeg2 1080/88i "interlaced" content to Mpeg4 1280x720p "progressive" or even "interlaced", for playback via high-def/res media players.

A few of us over on the AVS forum started discussing this subject with the idea of finding encoding applications (such as Nero Recode2 and VirtualDubMod) that could generate quality looking encodes without using AVS scrips. However, it quickly became apparent that the conversion process was not as straight forward as hoped...


The Mpeg2 Source
potus (over on AVS) has provided the following Mpeg2 source (https://home.comcast.net/~owenplanet/mpeg/copper_clip.mpg)
Resolution: 1920x1088
FPS: 29.970
Frame Type: Interlaced
Field Order: Top Frame First
Bit-rate: 14,500Kbps (approx)
Run-time: 6 seconds (approx)
Total Frames: 207

The clip also contains 6ch AC3 audio @ 384bps

The Problem
Some progressive encoding attempts have generated unwanted effects such as the image on the left: -

http://img170.exs.cx:81/img170/1977/test6ao.jpg


Primary Goal
To generate an 1280x720 progressive image at 29.970 that looks more like the source...

Secondary Goal
To generate an 1280x720 interlaced image at 29.970 that looks more like the source...

Third Goal
To generate an 1920x1080/88 interlaced image at 29.970 that looks more like the source...


Does anybody have any suggestions please?


Cheers

EDIT: Third goal added

[Wilbert]

What's the problem? For your first goal, you don't know how to apply a deinterlacer? :)

[Ark]

Yes:

Third Goal
To generate an 1280x720 ghosting image at 29.970 that looks less like the source... :D

Well, seriously i don't think it's all that much different from doing a standard NTSC interlaced to progressive encoding, there's the plus of resizing and much more work to do for the encoder (and also time for us to spend waiting...).

So i think that all the "guides" already out in Avisynth forum can come in help nicely (expecially the ones regarding proper deinterlacing/ghosting artifacts)

[CruNcher]

http://cruncher.mufflastig.com/XviD/samples/720p.avi

fast test vdubmod + smartdeinterlacer

[SeeMoreDigital]

Originally posted by Ark
So i think that all the "guides" already out in Avisynth forum can come in help nicely (expecially the ones regarding proper deinterlacing/ghosting artifacts) Thanks Ark but the idea was to find applications that could encode without creating AVS files!


> Cruncher,
Many thanks... that's the kind of thing.


> All
I guess in some respects I'm having a slight dig at NeroDigital's Recode2, as it was this application that generated the above left image!

Has anybody tried using AutoGK?


Cheers

[temporance]

Originally posted by SeeMoreDigital
Secondary Goal
To generate an 1280x720 interlaced image at 29.970 that looks more like the source...
AFAIK, 720i30 is not a standard HDTV format. Why go there? No hardware player or, more importantly, display is going to support it.

If you want to reduce resolution but retain motion fluidity, try 720p60. Of course this still leaves you with a high pixelrate, but...

[SeeMoreDigital]

Originally posted by temporance
AFAIK, 720i30 is not a standard HDTV format. Why go there? No hardware player or, more importantly, display is going to support it.

If you want to reduce resolution but retain motion fluidity, try 720p60. Of course this still leaves you with a high pixelrate, but... As far as I've been able to determine Sigma's current EM8620 chip-set does support 720i @ 25, 29.970 etc fps.

But the Mpeg4 720i option was to find out if various encoding applications could generate it and how well.

I think it might also be useful to suggest experimenting with converting Mpeg2 1080/88i to Mpeg4 1080/88i.


Cheers

[SeeMoreDigital]

Hi CruNcher,

Could you please generate another Mpeg4 1280x720p "video only" encode without Qpel and B-VOP?


Cheers

[nexx]

mencoder.exe -vf kerndeint,scale=1280:720 -ovc xvid -xvidencopts pass=1:bitrate=4000:vhq=1:max_bframes=0 -nosound -o output.avi input.mpg
mencoder.exe -vf kerndeint,scale=1280:720 -ovc xvid -xvidencopts pass=2:bitrate=4000:vhq=1:max_bframes=0 -nosound -o output.avi input.mpg

(yes I know bitrate isn't required for first pass, but it keeps the code nice and even ;) )

http://nexxteh.spymac.net/dvb/720p_30fps_kerndeint.avi

For some reason mencoder screws up the colours slightly. Anybody got a fix?
Anyway I just stick to blend (-vf pp=lb) for my low motion HDTV captures and bob (-vf tfields=1 -fps 50 -ofps 50) for my fast motion ones. ( you guys would need 59.94fps I guess for american hdtv)

[Andrey]

>>To generate an 1280x720 progressive image at 29.970 that looks
more like the source...
I'm stupid today. :(
Such a source (if it is a properly done encode) should have 23.976 full frames. I think.
What is the point to do it as 29.970 progressive ?
(or HDTV specs simply do not allow 23.976 frame rate ?)

[guada 2]

Hello SeeMoreDigital,

Your bet is dared,
Unless that you find a progressive scan recorder with DD (possessing a HD+ scaler), that will be in charge to convert your source 1080i in 720p with compensation of movement + application of the 3:2 pulldown.

Good luck.

[SeeMoreDigital]

Originally posted by guada 2
Your bet is dared... Not really!

There's seems to be quite a bit of plenty genuine 29.970 1920x1080/88 interlaced Mpeg2 content out there...

Not all 1080i content started it's life at film speed you know. And as a result does not require converting to 23.976 progressive frames!


Cheers

[*.mp4 guy]

Well it's possible with graphedit and ffdshow, it was a bit unstable tho. Imo it just makes more sense to use AviSynth...

http://img42.exs.cx/img42/4302/untitled5vn.gif

[SeeMoreDigital]

Originally posted by *.mp4 guy
Imo it just makes more sense to use AviSynth...True... but the idea was to try and find out how many of todays popular encoding tools/applications could cope with such a task!

Anybody tried AutoGK yet?

[scharfis_brain]

what about this?

http://home.arcor.de/scharfis_brain/720p24.avi

[L'il Jerry]

Originally posted by scharfis_brain
what about this?

http://home.arcor.de/scharfis_brain/samples/720p24.avi

All I get is a 47KB file that doesn't play. :(

[scharfis_brain]

whoops.
did a mistake while upping that file.

link corrected.

[guada 2]

Very strange, L'il Jerry
The link is correct, It functions very well.
size: 1280*720 2.66 mo.

Sorry, Seemoredigital I had'nt understood well.

[Andrey]

>>Not all 1080i content started it's life at film speed you know.
Such a video should be 30 fps then, not 29,97... Right ?

[SeeMoreDigital]

Originally posted by scharfis_brain
what about this?

http://home.arcor.de/scharfis_brain/720p24.avi Sadly, it's not very good I'm afraid :eek:

When you capture a still frame from the trains "2003" number you see this effect: -

http://img68.exs.cx/img68/3617/01scharfisbrain7xg.jpg

So converting to 24fps (23.976), did not work so well...


Cheers

[obieobieobie]

Okay, I fired up AutoGK 1.93.

Fixed width to 1280. Custom size to 3 MB. Audio to AC3.

Download here: http://www.putfile.com/media.php?n=copperclipbyautogk

Edit: Bah! stupid site embeds videos.

Here's a direct link: http://x3.putfile.com/videos/6502494287.avi

[obieobieobie]

Tried an encode with vdub(mod).

Filters: SmartDeinterlace (2.8 beta1) and bicubicRezise 1280x720

Xvid settings: Unrestricted, two pass, target size=2742, CQM=jawor's 1cd, chroma optimizer, motion search presicion=6, vhq=4, chroma motion, max i-frame interval=300, trellis, min/max quants=1/31

Settings for DG's SmartDeinterlace filter: Frame-and-field differencing, edge-directed interpolation, motion map denoising, motion threshold=10 and scene change threshold=100.

That's it, I think. It's undersized, though.

Download it here: http://www.putfile.com/media.php?n=copperclipvdubmodsmartdeinterlace

Direct link to file: http://x3.putfile.com/videos/6503341267.avi

[SeeMoreDigital]

Originally posted by obieobieobie
Okay, I fired up AutoGK 1.93.

Fixed width to 1280. Custom size to 3 MB. Audio to AC3. That's not bad at all. However, I've noticed that some encoder settings leave this effect: -

http://img55.exs.cx/img55/8803/01obieobieobie9nd.jpg


Originally posted by obieobieobie
Tried an encode with vdub(mod).

Filters: SmartDeinterlace (2.8 beta1) and bicubicRezise 1280x720Not bad too... but you can still see the following de/interlacing effect: -

http://img69.exs.cx/img69/9458/02obieobieobie8ok.jpg

Weird isn't it?


Cheers

[obieobieobie]

Yeah, it's weird. Maybe some gurus can shed some light. :)

[scharfis_brain]

@SMD: this 2003-Effect was intended.
I blurred the high shutter speed zo 1/24 sec to avoid ghosting and to retain as much smoothness as possible at 24p.

btw.: the other images with their deinterlacing artifacts are looking ugly :D

[SeeMoreDigital]

Originally posted by scharfis_brain
@SMD: this 2003-Effect was intended.
I blurred the high shutter speed zo 1/24 sec to avoid ghosting and to retain as much smoothness as possible at 24p. Can you have a go at generating another encode (without audio) at the same fps as the source please?

Originally posted by scharfis_brain
@btw.: the other images with their deinterlacing artifacts are looking ugly :D True :eek:

This is what I managed to come up with at just 1590Kbps: -

http://img198.exs.cx/img198/1395/021280x720p29970xvid1590kbps4b.jpg


And another at 3065Kbps: -

http://img115.exs.cx/img115/4723/041280x720p29970xvid3065kbps0x.jpg


Cheers

[scharfis_brain]

Can you have a go at generating another encode (without audio) at the same fps as the source please?

with this kind of artificial motion blur?
or just with deinterlacing applied?

btw.: the creation of that mo-blurred 720p24 clip took about 80 minutes, so it is not usable for real world applications.

[SeeMoreDigital]

Originally posted by scharfis_brain
with this kind of artificial motion blur?
or just with deinterlacing applied? That's entirely up to you... What ever you feel looks good and does not exceed 3500Kbps!


Thanks

[guada 2]

Hello,

Say me SMD, why did you chose the Xvid codec for this conversion?

Whereas the last version of the codec DivX fusion to the advantage on the macroblocks.

Why make not a true test with the codec RV10 hfe2.1 and the WMV9.
You will see to equivalent size, the RV10 is sovereign.
But i prefer the method of treatment of the WMV9 to this bitrate; because it delivers a picture more clear and more detailed.
More than, Which i prefer with this codec, it gives life to matter: as this train, one would say a true train, very materialized.

RV10 hfe2.1: 2.50mo for a bitrate of 2500
WMV9: 2.64mo for a bitrate of 2500

To this bitrate the 2 codecs is out contest.
It's my report.

[SeeMoreDigital]

Hi guada,

Any Mpeg4 (ISO 14496) codec is fine for this exercise. As it can be played using an HDTV media player (http://forum.doom9.org/showthread.php?s=&threadid=84003)... So yes, is suppose 1280x720 WMV9 video can be a option too!

RealMedia can't be an option because there are currently no stand-alone players, media streaming devices or chip-sets that support hardware decoding!


Cheers

[Andrey]

>>Whereas the last version of the codec DivX fusion to the advantage
>>on the macroblocks.
You should easily obtain the same (more smooth, less details) by selecting h263 quant in XViD. DivX has it set by default, if something have not changed since I last used it.

P.S. Yeah, SMD, now I understand your problem :), but still no luck solving it :( I hate interlaced sources :angry:

[guada 2]

Sorry, For the RM you are right perfectly.

In fact, The avel link player2 is equipped with your chipset.
But it is expensive: 249 dollards.
Available solely in America.

[guada 2]

Andrey, you seemed to know well the Divx codec, what are by default your settings and those that are optimal of the Divx.
Thank you to confirm.

[Andrey]

>>Andrey, you seemed to know well the Divx codec, what are by
>>default your settings and those that are optimal of the Divx.
Hmmm ?
As I uderstand your post, you get less macroblocks with DiVX than XViD ? Below is one of the explanations why it could possibly happen:

The problem with DiVX was that codec always used h263 quantization scheme, until new (can't remeber which, one of the latest) version which allow you to choose between h263 and MPEG quantization schemes appears on the scene...(anyway, h263 is selected by default)
h263 was designed for low bitrate videos and smoothes picture a lot, comparing to mpeg matrix. That's why DiVX encoded video usually has less macroblocks, than XViD one (encoded with MPEG or custom matrixes), but also has less detailes.
You can select h263 quant in XViD too to obtain the same effect, that DiVX has.

I think, it is offtopic a bit :(

[guada 2]

ANDREY,

You don't answer my question, and you think that it is out subject.
I think that it is not the pain to discuss since you chose not to study the question.

Note: I never have make comparison between these two codecs, I noted rightly a make strange at one time on the treatment of the Xvid codec.

Make a few test thrusts and you will see the opposite maybe: cards graphic 9800 pro + converter DVI/HDMI + Scaler (vigatec) + spotlight (barco).

" H.263 is not the H.263 Optimized "

[SeeMoreDigital]

Originally posted by Andrey
I think, it is offtopic a bit :( Although interesting... It would be off topic...

For those wanting to convert say, 1080i to 720p, the primary goal is to obtain the correct level of deinterlacing before moving the discussion onto the codecs used.

For instance, while fiddling about with latest version of Smart DeInterlacer, these are just some of the different effects I obtained: -

http://img56.exs.cx/img56/9675/fieldonlydifferencinglinearint.jpg

http://img100.exs.cx/img100/9848/frameandfielddifferencinglinea.jpg

http://img196.exs.cx/img196/1068/frameonlydifferencinglinearint.jpg


Cheers

[L'il Jerry]

Originally posted by Andrey
Such a video should be 30 fps then, not 29,97... Right ?

No. NTSC is 29.97 but people just round it up and refer to it as 30fps. Same with film framerate where people refer to it as 24fps when its really 23.97.

[scharfis_brain]

@SMD: this is a very common problem with motion adaptve deinterlacers!
I asked tritical whether there could be a solution.

The field-only mode has a real big downside:
it only weaves non-detailed areas (look for the remnant combing!)
detailed and pure static areas are always getting deinterlaced

[temporance]

Originally posted by L'il Jerry
No. NTSC is 29.97 but people just round it up and refer to it as 30fps. Same with film framerate where people refer to it as 24fps when its really 23.97. Allow me to out-pedant you!

NTSC frame rate is actually 29.97002997002997002997002997002997002997 fps. (=30/1.001)

But people round this up and refer to it as 29.97fps!!

[*.mp4 guy]

Hmm. I just thought of Xmpeg and flaskmpeg. I don't have flaskmpeg however Xmpeg works extremely well for this type of thing. Using Xmpeg + ffdshow video encoder (because of encoder side deinterlacing and resizing) I got pretty nice results. Oh and about the green blocks...there because the first gop isn't closed they go away after the broken gop is passed.

http://putfile.com/media.php?n=Output Video File

[L'il Jerry]

Originally posted by temporance
Allow me to out-pedant you!

NTSC frame rate is actually 29.97002997002997002997002997002997002997 fps. (=30/1.001)

But people round this up and refer to it as 29.97fps!!

Thanks troll, I guess your e-penis is bigger now. He was asking about if 30 fps should be used instead of 29.97 and I was just trying to explain why that wouldn't be correct. I wasn't trying to be pedantic at all, and maybe if you had actually read the post I was responding to instead of being a dick you would have realized that. Here it is for you:

Such a video should be 30 fps then, not 29,97... Right ?

[Backwoods]

Originally posted by L'il Jerry
Thanks troll, I guess your e-penis is bigger now. He was asking about if 30 fps should be used instead of 29.97 and I was just trying to explain why that wouldn't be correct. I wasn't trying to be pedantic at all, and maybe if you had actually read the post I was responding to instead of being a dick you would have realized that. Here it is for you:

Only jerk responce I see here is your's.

[Andrey]

[Offtopic]
>>guada 2: " H.263 is not the H.263 Optimized "
1. See here for a good comments on h263 and mpeg matrixes in DiVX: http://forum.doom9.org/showthread.php?s=&threadid=89919&perpage=20&highlight=H.263%20Optimized&pagenumber=2
2. h263 scheme is not a matrix in common terms. It simply has all coefficents set to 2. (to avoid ringing, to smooth image and etc. A lot of effects are achieved) So, what they "optimized" ? :)
If they changed coefficients, it is simply custom matrix. But as far as I understand it, mpeg matrix must be the best for wide range of different video sources at any bitrate (that was the goal when creating it), while any custom matrix designed for special purpose...
Any more questions ?
EDIT:
>>Note: I never have make comparison between these two codecs
Doom9 did.
I do it regulary, because I use XViD but my friends use DiVX (easier) to encode home-grown videos and I advice them on settings...
So what ? :(
My conclusions come from my own expirience, if you disagree, make your own test. That's simple.
Read that thread and len0x comments. He is expirienced person :P

[Andrey]

No. NTSC is 29.97 but people just round it up and refer to it as 30fps. Same with film framerate where people refer to it as 24fps when its really 23.97.
Oh, guys...
Just to clarify it a bit to you. :cool:

Film is 24 fps.
It is slowed to 23.976 to be telecined (*5/4) to 29.97 fps for NTSC transmission. This mode is specially designed for film sources.
NTSC video is filmed to 30 full fps or 60 fps interlaced.
So, NSTC has 3 standart frame rates: 29.97i, 30 and 60i.
Sometimes I saw 29.97 full fps but I wonder, if it is a standart mode...
EDIT:
>>He was asking about if 30 fps should be used instead of 29.97 and I
>>was just trying to explain
:)
SMD tells something of video, not film sources. NTSC filmed on NTSC video camera should be 30/60i fps. That's what I asked :)
If the source was film camera, it should be 29.97...

[SeeMoreDigital]

Originally posted by scharfis_brain
@SMD: this is a very common problem with motion adaptve deinterlacers!
I asked tritical whether there could be a solution.

The field-only mode has a real big downside:
it only weaves non-detailed areas (look for the remnant combing!)
detailed and pure static areas are always getting deinterlaced Agreed.

I guess the combing effect is not so apparent due to the source and the encode being high-def!

In theory, what should be the ideal de-interlacing solution for genuine 30fps 1080i content?


Cheers