Proposed Hi-Def encoding 'Codec Shoot-Out' discussion

[SeeMoreDigital]

Ok I've started this thread as an extension to Doom9's most excellent latest 'Codec Shoot-Out' (http://www.doom9.org/index.html?/codecs-203-1.htm) and this thread (http://forum.doom9.org/showthread.php?s=&threadid=67816&perpage=20&pagenumber=1)started by twist3d as a response to the afore mentioned 'Codec Shoot-Out'.

Well, for those members and visitors who followed twist3d's thread, I posted this observation (http://forum.doom9.org/showthread.php?s=&postid=423904#post423904) (as well as others) and Doom9 posted this response (http://forum.doom9.org/showthread.php?s=&postid=423985#post423985) (as well as others)!

Anyway, I think it's fair to say we are both now interested in the next evolutionary step for the current codec manufacturers to aspire to, which is High-Definition encoding.

So if anybody has any suggestions regarding the standardisation of such a test together with proposed methods, please post the information here.

Cheers

EDIT: Just put 'Proposed' at the beginning of the title - that's all!

[Versa]

One question, why?
The codec should scale with resolution. I encode HDTV to Xvid and it does wonderfully, I have also tried wme9 and it does good also, the main reasons I went with Xvid were AC3 compatibility, stability (wme9 crashed alot when I used it), and ease of use for the encoding tools.

I do 42 minute tv shows in 700-1024 megs typically. And movies usually go for 4-6 gigs.

[vio]

In Australia we have some decent hdtv. I dont have that great reception but I have got a 1440x1088 mpeg2 clip that works out ~100mb/min. Thats way too high to store on a dvd so if i wanted to keep it at a high resolution I would need about a third of the bitrate to store a movie on a dvd-r.

Maybe the codecs can be stretched to a half hd dvd rip.

I have done zero testing on this, I may do some tonight.

[SeeMoreDigital]

Originally posted by Versa
One question, why?
The codec should scale with resolution. I encode HDTV to Xvid and it does wonderfully, I have also tried wme9 and it does good also, the main reasons I went with Xvid were AC3 compatibility, stability (wme9 crashed alot when I used it), and ease of use for the encoding tools.

I do 42 minute tv shows in 700-1024 megs typically. And movies usually go for 4-6 gigs. This is exactly the kind of information that may be of interest to others!

So, you've ascertained that capturing with the WME9 encoder did not work reliably enough for you. But it might be interesting to find out why. And if others have had more success!

What image pixel frame size are you using for your captures?

[vio]

OK played around a bit with some codecs

Bitrates: 5000kbps and 2500kbps, this would equate to 1 or 2, two hour movies per dvd. My clip was much shorter.

Resolution: 1440x704 (interlaced anamorphic mpeg2 source, cropped with avisyth 2.5)

TomsMoComp and lanczos resize.

Xvid, best out of the lot, comparable quality to a 2 cd dvd rip, (per pixel size that is) @ 5000kbps, still very good quality @ 2500 - some blocks. My guess is a lot more quality could be obtained I used the default settings. probably 2500 could be comparable to a 1cd rip. [a third slower than divx5]

Divx5.1.1, second. Just a little worse, still very good. I did not use b frames, gmc or qpel. Also no psy. Standard speed. [Fastest encoding time]

tie for third depending on how you like things:

blurred; divx3, just tried this one for fun. turned out not so bad, bitrate did not affect quality that noticably. Plenty of blurring, smears and little edge atifacts. [almost as fast as divx5]

blocky; wmv9, heaps(!) of macroblocks. I dont use wmv9 that much but used the highest quality (ie slowest settings) went out and had dinner because it was so slow. Very disappointing [Slowest by a factor of 10]


I have never used rv9 and have only looked at vp6 for a few minutes so I didnt try these codecs.


I think the hardest part about a codec shoot out would be some standardized sources. I have no idea where you could find some.

[SeeMoreDigital]

Hi vio

Can I have a little more info about your source please? You mentioned that it's 1440x1088 which equates to an 4:3 image!

Does your source contain any black mattes or is it all image?

Is it a 4:3 image or is it 16:9

Before things get too confused I think it may be best to stick to the following 'full frame' image pixel frame sizes when encoding: -

For 16:9 sources use either 1280x720p or 1920x1088i
For 4:3 sources use either 960x720p or 1440x1088i

Obviously some 16:9 sources may not be all image (ie they might also contain black mattes) and as such have aspect ratios of their own, such as 1.85:1, 2.35:1, 2.40:1 etc.

In which case you could elect to crop away the mattes and use the following image pixel frame sizes: -

For 720p sources at 1.85:1 you could crop to 1280x688
For 720p sources at 2.35:1 you could crop to 1280x544
For 720p sources at 2.40:1 you could crop to 1280x528

For 1080i sources at 1.85:1 you could crop to 1920xx1040
For 1080i sources at 2.35:1 you could crop to 1920xx 816
For 1080i sources at 2.40:1 you could crop to 1920xx 800

Cheers

[twist3d]

So if anybody has any suggestions regarding the standardisation of such a test together with proposed methods, please post the information here.

idea 1: I think that highdef-tests shouldn't be done with already lossy & compressed material, like dvd's. (it's like recompressing mp3 and making audio quality tests). what do you think?

idea 2: maybe closer to the user interest would be anamorphic encodes at dvd-resolution or little lower?

[Doom9]

idea 1: I think that highdef-tests shouldn't be done with already lossy & compressed material, like dvd's. (it's like recompressing mp3 and making audio quality tests). what do you think?
And we're back to the source problem. Where would you get an uncompressed high def master of a movie to encode? You can't take the T2 DVD because it's already been WMV9 compressed. How good is HDTV? Do you get studio quality 10+mbit average, 20mbit tops sources without any transmission errors?

[SeeMoreDigital]

Originally posted by twist3d
idea 1: I think that highdef-tests shouldn't be done with already lossy & compressed material, like dvd's. (it's like recompressing mp3 and making audio quality tests). what do you think? Yes, that is a good idea. However, it's not going to be that easy for us all to obtain the same source files for testing. I've just downloaded the latest T2 trailers from the M$ web site. One is at 720p the other is at 1080p (wow, we now have 1080p as well as 1080i) and each of these is around 100MB!

Originally posted by twist3d
idea 2: maybe closer to the user interest would be anamorphic encodes at dvd-resolution or little lower? Ah! One of the main advantages behind high-def is its use of 'true 16:9 frame' sizes and not 'anamorphic (1.25:1 or 1.50:1) frame' sizes, like the ones we currenly have with DVD.

That said, one of the problems we have with HDTV is the confusing amount of trasmission standards (around 20 or so I think). There is even a 864x480p standard ie a 'true 16:9 frame' version of NTSC.

Another advantage of not encoding using anamorphic frames is that the images will always appear at the correct aspect ratio when viewed!

Cheers

[Wilbert]

Why would you encode 'uncompressed high def masters'?

What's wrong with the T2 1080p (1440x816) and the Corel Reef 1080p WMV9 encodings from the microsoft site?

edit: sorry I didn't read SeeMoreDigital's post

On this site: http://206.159.116.24/public.htm, hdtv captures (~15Mb, including 2.0 AC3) are posted regularly.

[SeeMoreDigital]

Originally posted by Wilbert
Why would you encode 'uncompressed high def masters'? There will be a time when forum members may want to convert their HDTV Mpeg2 captured broadcasts to say WMV9, Mpeg4, RV9 etc using larger image pixel frame sizes than the ones many of us currently use when 'backing up' our DVD's

Originally posted by Wilbert
What's wrong with the T2 1080p (1440x816) and the Corel Reef 1080p WMV9 encodings from the microsoft site?

On this site: http://206.159.116.24/public.htm, hdtv captures (~15Mb, including 2.0 AC3) are posted regularly. There is nothing wrong with any of these encodes, they look great. But even the WMV9 version of T2 exceeds 6GB and therefor will not fit onto a single layered DVD~R!

Cheers

[Wilbert]

I guess you misunderstood my post.


There will be a time when forum members may want to convert their HDTV Mpeg2 captured broadcasts to say WMV9, Mpeg4, RV9 etc using larger image pixel frame sizes than the ones many of us currently use when 'backing up' our DVD's
Of course, that's the reason for this test. But since when are 'HDTV Mpeg2 captured broadcasts' uncompressed?

There is nothing wrong with any of these encodes, they look great. But even the WMV9 version of T2 exceeds 6GB and therefor will not fit onto a single layered DVD~R!
What I meant, is that you could use them in your test. Of course, it might be that you can find better candidates.

I think it makes only sense, if you encode them using a lower resolution.

[Versa]

Originally posted by SeeMoreDigital
What image pixel frame size are you using for your captures? [/B]
I use full frame 1280x720 (@24 or 30 fps depending) for 720p broadcasts and I deinterlace and bicubic resize down to 1280x720p for 1080i broadcasts.

I managed to fit toystory hdtv to 1.5 gigs and mission to mars down to 3 gigs, looking very good, but I think I am going to up the filesize a little for the next movies. The hardest compression was 2.5 hours of hdtv hockey that was 18 gigs mpeg2, I could only get it down to 6 gigs.

[Mtz]

I think that only the live transmissions can have the real quality of HDTV. The movies are DVD's at high bitrate and bigger resolutions.
So, for what we need more space for the same movie? Great quality? I don't think so, just wasting space.

[SeeMoreDigital]

Originally posted by Wilbert
Of course, that's the reason for this test. But since when are 'HDTV Mpeg2 captured broadcasts' uncompressed? Ofcourse you are correct, they are already compressed. But they are massive too!


Originally posted by Wilbert
What I meant, is that you could use them in your test. Of course, it might be that you can find better candidates.

I think it makes only sense, if you encode them using a lower resolution. Absolutely. I did just this with the T2 clip. I lowered the bitrate of the original clip so the resulting encode would have a file size suitable for for storage on a single sided DVD-R!

At 152mins, the WMV9 version of T2 Extreme would make a great test source. Shame it's encrypted.... bummer!

[vio]

SeeMoreDigital, all the digital tv I get is 16:9.

Clip is 1440x1088, a bit of letter boxing so cropped down to 1440x1036. And Aspect Ratio is 2.025 so then resized to 1440x704, not perfect AR but closet mod16.

I can get access to alot of 1080i tv, but I dont know whether its 576i upconverted or just a bad source. The TV guides dont say. The clip I tested is a true 1080i source but I have to move my antenna to get it.

[Doom9]

Well.. having a WMV9 source sure would itch me.. I'd consider that an unfair beginning.. I'd expect the source to be available in a format that is not one of the codecs to be tested. A high bitrate MPEG-2 HDTV broadcast seems the most suitable source imho.

The movies are DVD's at high bitrate and bigger resolutions.That is not quite correct. In fact, most studios today make one high def digital master of a movie (unfortunately I don't recall the resolution and actual format.. but it's at least as good as what you can get via HDTV). From that, they make the HDTV broadcast, DVD and DVB broadcast. It is all the same source. So, DVD is a resized and recompressed version of the original digital master. Taking the HDTV version would get us much closer to the actual source.

[SeeMoreDigital]

Originally posted by Doom9
Well.. having a WMV9 source sure would itch me.. I'd consider that an unfair beginning.. I'd expect the source to be available in a format that is not one of the codecs to be tested. A high bitrate MPEG-2 HDTV broadcast seems the most suitable source imho. Agreed. Unfortunately, Euro1080 our European HD broadcaster, no longer transmits such images 'in the clear' (a case of real bad timing). So I am looking into the prospect of purchasing the necessary PCI card and card reader. The good news is, Euro1080 is available to anybody throughout Europe with none of the usual 'out of country' licensing restrictions.

Originally posted by Doom9
That is not quite correct. In fact, most studios today make one high def digital master of a movie (unfortunately I don't recall the resolution and actual format.. but it's at least as good as what you can get via HDTV). From that, they make the HDTV broadcast, DVD and DVB broadcast. It is all the same source. So, DVD is a resized and recompressed version of the original digital master. Taking the HDTV version would get us much closer to the actual source. Agreed again. Even in my old telecine days, the equipment I had access to, in order to create analog tape transfers (from film), were capable of producing higher quality images than any of the early DVD's I own.

However, there's no denying that DVD has taken off faster than anybody in the industry could ever have hoped for. And as a result the new transfer equipment and techniques used to transfer the film image to DVD have improved too. I guess one of the main reasons why todays DVD images look so good is because the production of the DVD is planned at the same time as the film. Also having direct access to the original film (instead of the copies) is the biggest plus of all!

Owners of the T2 Extreme DVD will get an insight of how 'film to digital' transfer techniques have changed after reading the sleeve notes. In fact the newly created digitised master (which at 24fps equals film speed) contains so much detail and resolution that only a very small percentage is carried over to the Mpeg2 DVD... To underline what I'm talking about, here's a direct quote from the sleeve notes.. "Because the video compression for DVD strips away nearly 98% of the original bits used on the uncompressed DVD master, THX reviews every shot to verify that the compression matches the source as closely as posible..."

98% eh, I don't think we'll be downloading this source master for our lossy tests then!

Cheers

[sxd]

I can only think of one source of uncompressed digital video off the top of my head - video games. Too bad they wouldn't be very comparable to live-action or animated content. Unless somebody has either a really nice digital camcorder, or somebody has access to a transfer device, I imagine it won't be easy getting higher-than-HDTV quality stuff to encode.

EDIT: That 98% figure is incredible. If we assume the maximum DVD bitrate is 9.8Mbps (That is correct, isn't it?), then the master would be 490 Mbps/61.25MBps.

[Wilbert]

I looked at suitable transport streams I have:

1) matrix trailer (~17 Mbps, 1920x1088, 88s)

2) performance of Michella Branch (~18 Mbps, 1920x1080, 53s)

What do you think about it?

[gooki]

Here's your uncompressed source files:

http://www.moonlight.co.il/down_video_high.htm

"Order High Definition HDTV Clips

Moonlight features 12 original 15-second High Definition HDTV clips shot with a Sony 900 camera.

The HD clips are featured in 1920x1080 resolution , progressive or interlaced, and are available in the following formats:

Targa Uncompressed
Avi Uncomressed
MPEG-2 80 Mbps

To purchase the High Definition clips or for additional information about the HDTV clips, please contact sales@moonlight.co.il"

[gooki]

And some free to download ones:

http://www.dvico.com/hdtv/download.asp

[SeeMoreDigital]

Originally posted by Wilbert
I looked at suitable transport streams I have:

1) matrix trailer (~17 Mbps, 1920x1088, 88s)

2) performance of Michella Branch (~18 Mbps, 1920x1080, 53s)

What do you think about it? What are the file sizes for each of your files?

And the more movement in them the better!

[Wilbert]

I have actually two matrix clips. This one is better (where Neo stops the bullets from the three agents):

matrix clip (~14.3 Mbps, 1920x1088, 94s, 160 Mb)

The interesting part (much movement) is in the second 80 Mb of the clip. (The other matrix clip contains much scene changes.)

The other one, Michelle Branch performing a song with her band at some talk show. Very detailed, but of course less movement as the Matrix clip. Size: 113 Mb.

I can put them on my webpage for download. How many clips do you want to use for the test? 3 or 4, and every does a comparison for one clip?

Btw, I'm downloading some of the clips of gooki.

[SeeMoreDigital]

Originally posted by Wilbert
... I'm downloading some of the clips of gooki. I have downloaded 'transport stream' clip 4 (1920x1088) which appears to be a full 1.77:1 aspect ratio image.

I have not looked at all of it yet, I'll do that tomorrow but your Matrix clips sound interesting.

Cheers

[Wilbert]

This site

ftp://ftp.cmf.nrl.navy.mil/pub/iHDTV/

contains also some cool hd clips. I downloaded the endavour (you know one of the space shuttles) one. 1280x720p and 45 Mbps. You can also add this one to the candidate list.

[bilu]

http://forum.doom9.org/showthread.php?s=&threadid=68693

May be useful.


Bilu

[trbarry]

I regularly capture HDTV shows, trim out the commercials, re-encode, and burn them to DVD-R.

But to get around the encoding artifacts and save a bit of space I may filter at the 1080 size and then re-encode at whatever resolution is square pixels of 1280 width. I may use an even lower rez if the movie is soft anyway.

But this can be used to get a standard starting point using the downloads from M$ of Coral Reef, Step Into Liquid, or whatever. Download a 1080i version but assume the benchmarks will be encoded in some lower resolution, leaving room for filtering, deblocking, possible source softness etc.

After going through resize or other filtering the clips are unlikely to correspond much with the original codec.

- Tom

[SeeMoreDigital]

Originally posted by Wilbert
This site

ftp://ftp.cmf.nrl.navy.mil/pub/iHDTV/

contains also some cool hd clips. I downloaded the endavour (you know one of the space shuttles) one. 1280x720p and 45 Mbps. You can also add this one to the candidate list. Damn!

I've tried several times today to download the 'space shuttle' file but the connection keeps failing -which is a bit of a pisser when you only have an 150k connection!

Shame... as this clip sounds like a good one!

Cheers

[Wilbert]

I've tried several times today to download the 'space shuttle' file but the connection keeps failing -which is a bit of a pisser when you only have an 150k connection!
It's 800Mb (almost 6 minutes)! I can upload it for you on my webpage, but I have to decimate it to 50 seconds.

@SeeMoreDigital,

What about the following setup:

We consider three clips (M$'s Terminator, my Matrix clip and the endavour clip). We will look for a third person to help us doing the encodings, so that each one can do the encodings of one clip.

The codecs which will be used: XviD, DivX, RV10, WMV9, FFVFW and VP6. We will have to decide on the encoder settings, and on the AviSynth scripts.

About the comparison itself. We can do it like Doom9 did for his comparison. But it could also be nice to do a blindtest, so that users can give ratings (similar as the listening tests at the hydrogenaudio forum) and decide which codec is best. That means people have to compare screenshots instead of parts of the movie itself.

I know that you can't compare anything looking at one frame, but maybe you can select a range of frames instead and compare them. This should be done at multiple places in the clip.

I would like to know what other people think about such a blindtest, and if it is possible at all to do it.

[sh0dan]

If someone is able to set up a bittorrent tracker we might be able to distribute it without killing off any webservers. Maybe corecodec.org could host the torrent, I seem to remember they have a bittorrent tracker.

[SeeMoreDigital]

Hi Wilbert,

With reference to your most recent post and suggestions, I think they will be perfect for our requirements!

I would be most grateful if you could post the Endavour clip on your WebSite or post it to me on a CD (or CD's)!

Talking about WebSites, I've finally managed to get my WebSite (www.SeeMoreDigital.net) up and running, on which I hope to post some HD encodes for download very soon!

Thanks again. Cheers

[Tommy Carrot]

When i'm trying to load the Endevour clip into virtualdub via avisynth, the image is totally garbled, green blocks everywhere. I'm totally noob to hdtv, any advices how should i open it?

[Wilbert]

Works fine for me.

Just use the script:

DirectShowSource("E:\endavour.mpg")

The following dshow filters are used to play them (in my case):

Elecard MPEG2 Demultiplexer
Moonlight Odio Decoda (to decode the MP2 audio)
DirectSound Device
Elecard MPEG2 Video Decoder

[Tommy Carrot]

Thanks!

[trbarry]

Maybe there should be some discussion about the criteria for evaluating an HDTV codec. When doing DVD rips it seems to be the fashion to value those codecs that degrade most gracefully when there is maybe not quite enough bit rate to do the job but maybe you want to fit a movie onto a CD at 'almost' great quality.

But for HDTV many might feel instead that it is all about getting almost perfect quality and high detail at higher bit rates. After all, you can always reduce to DVD or lower resolution if you want a 1 or 2 CD rip.

So maybe this might seem obvious but we should expect to test at a higher BPP than otherwise, and grade down for artifact free but soft looking encodes. And yet everyone has to encode at the same bit rates for a fair comparison, so it's something that sort of should be discussed first.

This also should exclude any idea of testing with any soft upconverted material.

- Tom

[trbarry]

There is also the consideration that few of us can use softare to play much more than 1280x720 with most of the advanced codecs. That's another reason I've mostly (though not exclusively) stuck with Xvid. It just seems to play faster at high resolutions.

- Tom

[31 Flavas]

Originally posted by Wilbert

Works fine for me. The endevor.mpg file should play in a software dvd player right? Or it is an mpeg4 file?

It is not playing for me in PowerDVD.

[Tommy Carrot]

It doesn't seems to be a normal mpeg2 file (but it's still some kind of mpeg2, that's sure), because VdubMod cannot open it directly, and the dvd2avi gave me garbled image. But Wilbert's method worked.

[sh0dan]

To open it using MPEG2DEC3, use DVD2AVIT3 v1.83 from this page (http://forum.doom9.org/showthread.php?s=&threadid=49349). The streams in this movie are numbered PID 21 (video) and ffff (audio). You need to set these in the DVD2AVI.INI.

Then you can open the d2v using MPEG2DEC3 (I used v. 1.10 in this case).

[Wilbert]

It is not playing for me in PowerDVD.
I couldn't play it in powerdvd v3.0, although I could play it in windvd 4+.

@trbarry, SeeMoreDigital

I will come back to you tomorrow.

[trbarry]

To open it using MPEG2DEC3, use DVD2AVIT3 v1.83 from this page.

I think that page references MPEG2DEC3 from Marc FD and there have been a couple HDTV related fixes since then.

While they don't appear in my downloads section there are still a couple versions of HD supporting modules on my site that work:

www.trbarry.com/DVD2AVIT3.zip and
www.trbarry.com/MPEG2DEC3.zip

However I'm pretty sure these are NOT the current and best versions for HDTV. Nic made improved versions from both of these though I'm not sure what or where those are now.

And I don't know if Neuron2's new versions support HD yet or not, so I'd probably avoid them unless told differently.

- Tom

edit: I guess the current and best MPEG2DEC3.dll for HDTV is still the one on Nic's site. I'm still not sure about DVD2AVIT3.

[SeeMoreDigital]

Originally posted by Wilbert
@trbarry, SeeMoreDigital

I will come back to you tomorrow. "I..... thank you"!

[Wilbert]

Maybe there should be some discussion about the criteria for evaluating an HDTV codec. When doing DVD rips it seems to be the fashion to value those codecs that degrade most gracefully when there is maybe not quite enough bit rate to do the job but maybe you want to fit a movie onto a CD at 'almost' great quality.

But for HDTV many might feel instead that it is all about getting almost perfect quality and high detail at higher bit rates. After all, you can always reduce to DVD or lower resolution if you want a 1 or 2 CD rip.

So maybe this might seem obvious but we should expect to test at a higher BPP than otherwise, and grade down for artifact free but soft looking encodes. And yet everyone has to encode at the same bit rates for a fair comparison, so it's something that sort of should be discussed first.

Just a proposal, since SeeMoreDigital is a bit quiet:

We assume we put 2:30 on one DVD-R. Assuming the soundtrack is 5.1 AC3 (about 422 MB, if I'm not mistaken), that leaves 4.3 GB - 422 MB = 3980 MB. Subtracting the overhead and subtitles, leaves, let's say 3.8 GB. That should be more than enough. The bitrate becomes 3378 kbit/s.

The clips will be downsized to 1280x720 (the endaviour clip is already at this size).

What about it?

[trbarry]

I guess it would depend upon the movie. I can fit most movies on a DVD-R but most are probably not 2:30. But if it compressed well and was a 2.25:1 or wider ration with the black bars trimmed off then it might work.

Also my own feel for it is likely wrong since I now do a lot of 1 pass CBR, simply because I'm lazy and they play better. But of course that takes more space. Usually I don't care since a DVD-R is a DVD-R anyway.

Though I used 2 DVD-R's for a 3:00 movie recently.

- Tom

[Wilbert]

@Tom

1) Do you use additional filtering in AviSynth?

2) So, we should do 2 pass VBR :)


And I don't know if Neuron2's new versions support HD yet or not, so I'd probably avoid them unless told differently.

No, it doesn't

edit: I guess the current and best MPEG2DEC3.dll for HDTV is still the one on Nic's site.

Yup.

I'm still not sure about DVD2AVIT3.
There is a version from TimeCop which demuxes aac streams. I'm not sure which version it is based on. Besides that, dvd2avit3 is the most recent one.

[SeeMoreDigital]

Originally posted by Wilbert
Just a proposal, since SeeMoreDigital is a bit quiet: Sorry about this....

I've recently been struck down with ME/CFS, so from time to time I don't have much energy.

Over this week for instance, I've managed to completely knacker myself by building my WebSite!

Originally posted by Wilbert
We assume we put 2:30 on one DVD-R. Assuming the soundtrack is 5.1 AC3 (about 422 MB, if I'm not mistaken), that leaves 4.3 GB - 422 MB = 3980 MB. Subtracting the overhead and subtitles, leaves, let's say 3.8 GB. That should be more than enough. The bitrate becomes 3378 kbit/s.

The clips will be downsized to 1280x720 (the endaviour clip is already at this size).

What about it? Yes, I like this!

From the start I thought it a sensible idea to have a target file size to aim for and a DVD~R/RW seems to offer that!

Reading up a little more about Hi-Def encodes and the equipment currently available for viewing such transmissions/encodes, I think 'encoding' to 1080i (1088) may not be all that straight forward, as it will require very fast PC's for both and encoding and decoding. And there will always be the temptation to encode at 1080p instead of 1080i, which could seriously complicate the issue!

So for me it looks like generating 720p (progressive) encodes would offer the most sensible all round option at the moment, using the entire 1280x720 image pixel frame size, regardless of the picture aspect ratio.

Cheers

[trbarry]

1) Do you use additional filtering in AviSynth?

2) So, we should do 2 pass VBR

1) I throw in Undot() and UnFilter(-5,-5) in most everything these days (after deinterlace or IVTC), just for good luck. But that's just me.

2) 2 pass VBR is certainly more efficient. But the results may stutter during playback on some machines and high bit rate scenes, and in turn cause sound sync playback problems for HDTV.

And yep, I think 720p (actually 1280 wide p) is probably a good target. ;)

My own stuff I usually still stick close to .25 bpp but that's because it usually fits on a DVDR anyway, not because it is absolutely needed.

- Tom

[SeeMoreDigital]

I've managed to get hold of a very short Mpeg2 TS 1280x720 clip running at 60fps.

Is it normal practice for hi-def TS streams to run at 60fps (over in NTSC 60Hz land), as I assumed they would be 30fps!

The reason why I ask, is because I've been busy re-encoding some of the HD WMV9 encodes (available on the M$ website) which all seem to be running at (or around) 24 fps!

Instead of using scripts I thought I would see how well applications such as Dr DivX, Helix Producer (basic) and Win MediaEncoder 9 would fair..... and I have to say they all coped very well!

Cheers

[trbarry]

Typically ATSC HDTV will be sent either at 1920x1080i at 60 fields / second or 1280x720p at 60 frames / second.

1080i acts just like 480i when from video source (not movies). You can deinterlace it to make 30 or 60 progressive frames. I usually keep only 30.

For movies 1080i can be processed through Decomb/Decimate to 24 fps just like DVD movies.

720p material is almost always 60 fps progressive frames. If it is movies then there are just a lot of duplicate frames so you can

SelectEven()
Decimate(5)

down to 24 progressive frames / second. There are a few rare things that actually come from 60p cameras, like ABC sports (football). Here there are really 60 different frames / second and you have to decide whether you are going to keep all of them, or decimate to something else.

BTW, most of them are not properly flagged, so Force Film usually won't work.

- Tom

[SeeMoreDigital]

Wilbert,

Hey thanks very much for sending the disc's, I received them this morning.

There's some interesting files on disc #2 which I think will encode very well.

The little TSdemux tool is great fun and very quick. Shame there is'nt a tool that will half the frame rate of a TS stream!

Okay then.... I suppose it's testing time.

Thanks again

[Wilbert]

Hey thanks very much for sending the disc's, I received them this morning.
Finally :)

There's some interesting files on disc #2 which I think will encode very well.
I forgot exactly which clips I send you. But one is ts+aac (Bikini clip) and there is also one with two video streams (Faith Hill). I use TSdemux to demux the aac stream.

[Kon]

SeeMore - IMHO part of your test should include a transmission and reception test of the codecs you're testing with HD encoding. The easiest way to do this is to datacast them in MPEG sections. This is a good way to test if the CBR codecs really are CBR, how well they fare in reception, and what the playback error correction capabilities are. It is also a big eye opener for people used to seeing a single HD stream and maybe one SD stream in (for example) a single ATSC broadcast channel. When you show people you can do one HD and almost a dozen SD channels using WMV as a comparison (in the same channel space), it is a big eye-opener. Testing image quality, encoding aspect, and so on is just one aspect of such a comparison.

Just my 2c...

[SeeMoreDigital]

Time to re-ignite this thread me thinks!

> Wilbert and Tom
Since discovering the wonders of XviD's (and 3viX's) ability to add an anamorphic switching signal to the Mpeg4 data stream I decided to see what would happen to an 1080high (or 1088high) source when it's resized to 720high.

Also, because I have an Xcard (sorry Doom9 to mention it again) which is able to detect anamorphic signalling automatically I elected to place the XviD encode an an MP4 container.

The only trouble with this is, the Xcard, like any other hardware player, does not decode streams that have an image pixel frame size larger than 720wide x 576high (or 414'720 total) pixels.

So, I decided to see what would happen if I generated 3000kbps 2pass XviD encodes at 567wide x 720high!

And.... to my surprise the images are very watchable. Even when viewed via an 1366x768 plasma screen. And of course not only can my Xcard 'easily' play them but so can most PC's.

All is not perfect however, because as yet I can't manage to encode 60fps sources properly only 30fps.

And because I've been unable to strip out the AC3 audio stream from the .ts source I've been unable to re-encode it. That said the AAC audio stream that was in the 'JapanBikini_aac.ts file' that Wilbert sent me was no problem!

Anyways, I just thought I'd let you know I was actually doing something!

Cheers

[bond]

anyone tried mainconcepts h.264 codec for hd aready?

[SeeMoreDigital]

Originally posted by bond
anyone tried mainconcepts h.264 codec for hd aready? Not yet!

I hear reports that it's a little slow to encode - a bit like VP6. But I will test it soon!

But you know me and Mpeg4. I just can't resist testing new versions of Mpeg4 codecs and containers. And the new XviD build was too much of a temptation!

Cheers

[Wilbert]

... about AC3 demuxing. Did you try the stuff Sh0dan mentioned here (DVD2AVIT3 v1.83):

http://forum.doom9.org/showthread.php?s=&postid=427717#post427717

If so, which clip do you have problems with?

[SeeMoreDigital]

Originally posted by Wilbert
... about AC3 demuxing. Did you try the stuff Sh0dan mentioned here (DVD2AVIT3 v1.83):

http://forum.doom9.org/showthread.php?s=&postid=427717#post427717

If so, which clip do you have problems with? I'll check out this link. But it's a shame TSDemux (v0.18) is only able to strip out .aac audio streams and not AC3 audio (or what ever he audio stream is) from .TS sources!

I really can't believe how good the Bikini Girls, Faith Hill, Matrix, Michelle Branch and the Wild Things clips look. Even the wife commented how good the image was!

That said, if you guys had used the PAL colour system instead of NTSC, I cant help feeling they would look enev better ;)

Cheers

[SeeMoreDigital]

OK, for all those interested I've provided a link to an anamorphic 576wide x 720high 2pass XviD .mp4 encode (http://82.2.167.24/Uploaded_Files/Doom9_Forum_files/Bikini_Girls.zip) @ 2775kbps (11.9MB). The source was an 16:9 1440x1080 Mpeg2 .ts stream (67.3MB).

Please note that this encode has been generated to suit the Sigma card and should ideally be viewed on a TV screen, not a PC screen!

If however you are viewing the encode via a PC screen, Nero's ShowTime Player should automatically display the encode at the correct AR.

And last but not least. Generating encodes with an horizontal quantity of just 576 pixels is not advisable, as 1280 pixels should really be the norm.

Comments please. Remember it's just a test!


Cheers

EDIT: Source details corrected

[bond]

looks nice :D

tough i get choppy playback, i guess its my 866mhz pentium3 :(

Originally posted by SeeMoreDigital
If however you are viewing the encode via a PC screen, Nero's ShowTime Player should automatically display the encode at the correct AR.the 3ivx decoder also handles this

it doesnt really belong here, but how did you create the .mp4? i ask because it doesnt have a bifs and od stream, which are mandatory in .mp4 afaik!?

[SeeMoreDigital]

Originally posted by bond
...it doesnt really belong here, but how did you create the .mp4? i ask because it doesnt have a bifs and od stream, which are mandatory in .mp4 afaik!? I thought this would interest you!

After creating the anamorphic encode with MPEGmediator. I created a copy of an old 'audio only' aac.mp4 file. I opened the file with MP4UI, imported the XviD video stream. Deleted the audio stream, the bifs and the OD. And optimised the file!

Shame you have not got an Xcard... It looks really quite good on a big screen.

Cheers

EDIT: Yes I thought the bifs and the OD files were mandatory. But I keep deleting them and the files keep working.... weird in it!

[bond]

Originally posted by SeeMoreDigital
Yes I thought the bifs and the OD files were mandatory. But I keep deleting them and the files keep working.... weird in it!yep, cause dshow doesnt need it, but to be spec compliant you have to keep it

[SeeMoreDigital]

Originally posted by bond
yep, cause dshow doesnt need it, but to be spec compliant you have to keep it Makes me wonder if the encodes will work in standalone hardware player... if they ever build/upgrade one that can spin/stream .MP4 encodes!

These manufacturers don't know what they are missing!

Cheers

[SeeMoreDigital]

Hello again

Here's another link to an anamorphic 576wide x 720high 2pass XviD .mp4 encode (http://82.2.167.24/Uploaded_Files/Doom9_Forum_files/Wild_Things.zip) @ 2980kbps (14.5MB). The source was an 2.35:1 1920x1088 Mpeg2 .ts stream (video element @ 47.6MB).

Wilbert, can you confirm that the source is in slow motion?

Cheers

[Wilbert]

Wilbert, can you confirm that the source is in slow motion?
It plays fine on my Athlon 3000+.

[bond]

nice :)

may i ask where the source is from?

[Wilbert]

Sure, I got them from http://206.159.116.24/public.htm The site is not accesible atm.

He uploads new clips (about) every week. Most of them with AC3 2.0 ch, some of them with AAC 2.0 ch.

[SeeMoreDigital]

Originally posted by bond
nice :)

may i ask where the source is from? Wilbert mailed them to me on CD!

As you can see I've had no trouble converting the 30fps 1080i source material to 720 anamorphic XviD.

However, try as I might, I can't successfully convert 60fps 720p source material to 720 anamorphic XviD.

It would be nice to get hold of some PAL HiDef source material from the Euro1080 satellite... can anyone oblige, as I have'nt had chance to replace my equipment yet, after it was struck by lightning...

Bummer!

[Wilbert]

However, try as I might, I can't successfully convert 60fps 720p source material to 720 anamorphic XviD.
Are you talking about the spaceshuttle launch? What's the problem here, the framerate or the width?

[SeeMoreDigital]

The frame rate is the biggest problem. It would seem my Xcard (along with most other hardware players) does not like 60fps encodes

Admittedly I guess I could use a script to convert 60fps to 30fps but I thought I would try encoding without them!

Bye the way, have you come across any other wonderful tools like TSDemux that can demux AC3 streams from .ts files. Or convert 60fps to 30fps?

Cheers

[bond]

Originally posted by Wilbert
Sure, I got them from http://206.159.116.24/public.htm The site is not accesible atm.

He uploads new clips (about) every week. Most of them with AC3 2.0 ch, some of them with AAC 2.0 ch.hm ic :)
are you sure that these are not some self made upsized movies?

[Wilbert]

are you sure that these are not some self made upsized movies?
No. I'm quite sure they are captures from satellite though. I will mail him about the upsizing ...

[SeeMoreDigital]

Hi

Here's another link to a Faith Hill (http://82.2.167.24/Uploaded_Files/Doom9_Forum_files/Faith_Hill.zip) anamorphic .mp4 encode: -

Encode - 576wide x 720high 2pass XviD @ 2800kbps (12.3MB).

Source - 1920x1088 Mpeg2 .ts stream (video element @ 83.9MB).


I'm sorry the download speed is so slow. I hav'nt yet worked out how to upload files to NTL's server!

Cheers

[bond]

great stuff SMD :)

what settings and .avs script did you use for the encodings?

Originally posted by Wilbert
I will mail him about the upsizing ..would be great,
its not that important if all codecs use the same source, but surely its good to know if a non upsized source gets used :)

[Wilbert]

Bye the way, have you come across any other wonderful tools like TSDemux that can demux AC3 streams from .ts files.
Like I said in a previous post: DVD2AVIT3 v1.83: http://www.trbarry.com/DVD2AVIT3.zip

Don't forget to specify the video and audio pid's in the DVD2AVI.ini file.

convert 60fps to 30fps?
tbarry said he used just SelectEven for that. I don't think you have other options besides Decimate(cycle=2).

[SeeMoreDigital]

Thanks Wilbert,

DVD2AVIT3 (v1.83) works well at demuxing AC3 streams from .ts files.

The problem I now have is that all the demuxed AC3 streams (and aac streams for that matter) incorporate an slight ms delay - all different!

Is nothing easy these days?

Cheers

[SeeMoreDigital]

Hi Wilbert,

I've been experimenting again with the clips you gave me!

This time I've been trying to convert and fix the delay in the AC3 streams prior to generating AAC streams.

I've been using DVD2AVI (v1.83) to demux the AC3 stream from the (Mpeg2/AC3) .TS file. And then a new tool called DelayCut (http://forum.doom9.org/showthread.php?s=&threadid=71545) to reduce the delay in the AC3 stream.

I then used foobar2000 to convert the AC3 file to an AAC file. And then MP4UI to mux the AAC stream with the previously made anamorphic XviD 576x720 video stream, into an MP4 container!

It all worked very well.... it's just a damned shame there's not an easier way to do it all!

Cheers

[crahak]

For what it's worth...

I'm taking 720p material and I take it down to 24fps (sometimes 30 when it looks too choppy).

Choppyness wise, in avisynth, I use

Decimate(5)
SelectEven()

reversing them leads to choppy video.

Other than that, I crop it in avisynth as well, then compress it to fit on a single DVD-R with xvid (using virtualdubmod). divx looks ok but has slightly less detail. WM9 might be good (and supposedly the video cards should help at some point with the decoding???) but it never turns out right. RVx and VPx are out of the question as far as I'm concerned.

It looks great. One could sometimes (esp if the titles are short) fit 2 per disc but I'd rather keep better pic quality.

I also tried reencoding the source into mpeg2 @ 24fps (curiosity I guess) also because it seems to be decodable on lower end computers (most video cards I've tried did work well using the decoding assist)... Anyways, lower CPU usage of course, but the picture didn't look quite as good as the xvid encodes.

[Selur]

just as a little side note:
I normally use Project X (aka. ds.jar) to demux hdtv clips, after this they also seem to work fine in DVD2AVIdg ;)

Cu Selur

[SeeMoreDigital]

Hi Selur,

I gave ProjectX a go last week, the only thing is, it's a massive application, that can obviously do way more than demux .TS files.

That said, it could not demux all the Mpeg2/AC3/AAC .TS clips Wilbert gave me to test. With some it would only demux the video. The audio was left somewhere in cyberspace!

By contrast, DVD2AVI (v1.83) was able to demux the audio from all of Wilberts Mpeg2/AC3 .TS files.

So I'm left with a bit of a .TS file quandary....

TSdemux (v0.18): can demux both the video and audio streams, from an Mpeg2/AAC file. But the video stream only, from an Mpeg2/AC3 file.

DVD2AVI (v1.83): can demux the audio stream only from an Mpeg2/AC3 file. But doesn't like Mpeg2/AAC files at all.

PVAStrumento (v2.1.0 RC4): has it's own peculiarities and was unable to demux some of the files.

ProjectX: is overly complicated.

In conclusion, if there was a tool based on TSdemux that could demux both AAC and AC3 audio streams, I for one would be in 'demux heaven' - But as yet, I havn't found such a tool!


Hi crahak,

My experiments with 720p files, using non script applications, have not been very successful at all. I can demux the audio and video streams okay. But re-encoding 60fps video to 30fps (or even 24fps) has not worked!

I'll just have to keep on looking... I guess!

[Wilbert]

There exists a dvd2avi_aac by timecop:

http://pbx.mine.nu/dvd2avi/

Maybe you can try it out?

[SeeMoreDigital]

Originally posted by Wilbert
There exists a dvd2avi_aac by timecop:

http://pbx.mine.nu/dvd2avi/

Maybe you can try it out? Just tried it.

But for some reason it will only demux Mpeg2/AAC files. It can't demux any of the Mpeg2/AC3 files you gave me!

So still no 'holy grail' yet!

Can you try it please, just in case I've missed something?

Cheers

[Xleon]

Owners of the T2 Extreme DVD will get an insight of how 'film to digital' transfer techniques have changed after reading the sleeve notes. In fact the newly created digitised master (which at 24fps equals film speed) contains so much detail and resolution that only a very small percentage is carried over to the Mpeg2 DVD... To underline what I'm talking about, here's a direct quote from the sleeve notes.. "Because the video compression for DVD strips away nearly 98% of the original bits used on the uncompressed DVD master, THX reviews every shot to verify that the compression matches the source as closely as posible..."


I know this is a few pages ago, but to add to this, a lot of film work is now digitized to 2K format 4K. All of the professional movie effects systems can now operate in these two formats.



Here is some official details on it:



2k

Short for the image size of 2048 x 1556. This is almost the same as the QXGA computer image resolution, has 3.19 Mpixels and a 1.316:1 aspect ratio – the same as full frame 35 mm film. This image size is increasingly used for digitising full frame 35 mm motion
picture film sampled in RGB colour space – making each image 12 MB. Sampling is usually at 10-bit resolution and may be linear or log, depending on the application.

Note that the sampling includes 20 lines of black between
frames because of the use of a full-frame camera aperture. Thus the actual ‘active’ picture area is 2048 x 1536 and has a 4:3 aspect ratio. Removing the aperture creates an ‘open gate’ format which may have no black bar between frames – so all 1556 lines carry picture information.

[SeeMoreDigital]

Thanks Xleon and welcome to the Doom9 forum

Yes, it would appear that things have moved on quite considerably in the world of film to digital image transferring.

Anyways, for those who are interested. Here's a link to the T2 Extreme sleeve notes (http://82.2.167.24/Uploaded_Files/Doom9_Forum_files/T2_Sleeve_Notes.zip)!

How times change eh!

[Xleon]

Cheers SeeMore, I seem to be following you around on these forums :D

Typed that a bit quick with a few typos. I left out that the 2K material means a storage requirement of 287Mbytes/Sec at 24fps !!
(Though I'm sure you already worked that out)

The (not so popular) 4K size (4096 x 3112), equates to 1.1Gbytes/Sec but the film industry has settled on 2K which was what the latest StarWars films were shot on.

[virus]

hi!

to all you HD content lovers... I did a search on this forum but found no thread matching, so I'd like to outline this link:

http://www.ist-metavision.com

the Metavision project is an European effort to design a new hi-def camera with embedded lossless compression.
AFAIK the prototype captures at 1920x1080 @ 72fps :cool:
This way you can do 3:1 slow-motion keeping the smoothness of a 24 fps video. This should be fantastic for sport-related and, generally, all high-motion material.
Obviously they need in-place compression to reduce the outstanding bandwidth: a few gigabits per second... :eek:

cheers
virus

[SeeMoreDigital]

For all those who would like to see some more HD XviD encodes (from an 1080i HiDef source), please look here: -

http://forum.doom9.org/showthread.php?s=&postid=467527#post467527

Cheers

[SeeMoreDigital]

For all those who may be interested how the film to HD digital transfer of Terminator 2 was made, I finally got round to converting the DVD's sleeve notes into an web page.

For more info please look here (http://www.SeeMoreDigital.net/03_Video_Only_Info/T2_HD_Restoration.html).


Cheers

[Soulhunter]

Thanks, nice info... :)


Bye

[minolta]

@SeeMoreDigital

I failed when conducting my own HD shootout, but think you might learn from results anyway:

http://hdtvshootout.tripod.com (better have popup-blocker...I'd host myself but 11MB too big)

This might be a "how not to conduct an HD shootout" page. I archive my favorite HD shows to DVD regurally (just Smallville actually). I liked my sources, sizes, and codecs at first, but tests took too long and results were not very exciting (wrote 80% of homepage during first encode job). Then I lost my stats, and called it quits. Well, any experience is good experience, so take what you will.
-Minolta

[SeeMoreDigital]

Nice one :D, thanks Minolta,

I will give it a read. But why all these pop-ups!

... I hate pop-ups :devil:


Cheers