First whack at that uncompressed source clip

[benwaggoner]

Folks,

It's about six months late, but I've finally gotten a rough cut of that 24p film soure content that I can distribute source for.

Here's a QHD (960x528) verison of it on Silverlight Streaming to give a taste of it.

http://silverlight.services.live.com/invoke/31260/TallShipScratch720p/iframe.html

I did this edit in a crazy rush on my laptop in order to make some DNLA conformance test streams, so it's pretty ragged. Among other things I'll want to do:

Get some more kinds of transitions in there
Color correct to bring out different kinds of textures. There is NO color/image processing of any kind in this edit, which is painfullly obvious.
Speed up the credits in order to fit some more shots in
Do some something interesting with color motion graphics for the title sequence


My beefy HD editing workstation shiould be arriving this week, finally, so I can hopefully make a real effort to finish this off.

I also have a 1080i30 edit from the same shoot (fortunately edited by someone other than me!) I can share as well.

Anyway, here's a taste. I appreciate any feedback and suggestions on what can be done to make this a more relevant test clip of film source content for codecs.

[CruNcher]

Wow nice it looks beautiful ok some color stuff i would do differently (but thats personal taste) Mr. Billups and the daylight gave it already a wonderful mod to begin with :)
I miss some longer lasting closeups here the only one that could be seen as a closeup is the pan between the 2 Woman tough that's a little short, also the exposure there is not very good to see the detail richness of their faces :)
Does everything only Plays on the Deck or do you also have some interior shots with the Crew you could add in like sitting around @ a table or on the Deck in a Circle talking to each other? :)
Most of what it shows currently seems to be the work of the Crew on board only a small part like the beautiful sun scenes and the guy playing the guitar is somekind of resting :)

[benwaggoner]

That's a pretty good demo of the kinds of shots in the source I have (the clip was edited from about 45 minutes of developed film). Good point about the lack of closeups; skin texture is an important codec scenario. When I get my workstation set up, I'll see if I can find a few more close-up shots to use. There are certainly some more of the guy wih the guitar near the end.

No interior shots at all, alas. But I'll spend some time finishing in After Effects to match more closely what a color-timed film would look like.

[Golgot13]

Can you post a Full HD version of your video?

[Biggiesized]

This is a great encode, but the transfer isn't terribly sharp. And, as you mentioned, there needs to be some color grading done.

[benwaggoner]

Can you post a Full HD version of your video?
Full HD compressed, or the raw stuff? The raw is about 20 GB.

At this point, I'd like to get feedback on the edit and content, and will do another edit post-NAB at render that out at full rez.

[Schrade]

Direct link to download the WMV (Unless the path gets randomized): http://msbluelight-0.agappdom.net/e1/d/31260/1171734/63343504800/0.C0De-sTpqj0uf1KJhuJIs7aKzS8/Lady_Washington_DLNA_1080p24_stereo_44.wmv (100.22MB)

[Golgot13]

Hi Ben,

about the FullHD version you can use a lossless codec to reduce the size of video.

[Inventive Software]

Double post!!!!! :readrule:

You keep asking about the source, have you looked at his 1/4HD sample and given feedback?

[Golgot13]

Double post!!!!! :readrule:

Yes, MS software have bug sometime...


You keep asking about the source, have you looked at his 1/4HD sample and given feedback?

Yes, video with boat.

Feedback: give the FullHD video (with lossless codec) to show the potential of last development of H264 tool !!

[benwaggoner]

Yes, video with boat.

Feedback: give the FullHD video (with lossless codec) to show the potential of last development of H264 tool !!
You really don't have any thoughts whatsoever on what can be done to make this a more relevant test?

Let me put it this way: if VC-1 won the test hands down, what features of the clip would you then object to as being unrealistic for the scenario we're testing?

Some things I've been thinking about:

What kinds of color correction do we want?
How much low-luma shadow detail to look for blocking?
What kinds of transitions are good to include? Whacky stuff like Barn Doors interesting?
Anything interesting we should do for credits?
How about opening titles? Anything in particular that would be good for a motion graphics test? Maybe something with smooth gradients?

[Sagittaire]

Let me put it this way: if VC-1 won the test hands down, what features of the clip would you then object to as being unrealistic for the scenario we're testing?


Realistic scenario is the large majority of studio don't use complex pre-process (No time and no money for make that).

Realistic scenario is the large majority of studio don't use advanced setting like segment reencoding (from pep, cinevision or cinevisionPSE) simply because actual compressionist don't have the knowledge or time for make that. For example the Sonic Customers don't have support (120 000$ for sonic suite) for use correctly the Mainconcept SDK codec (low bitrate profil, high bitrate profil, grain profil ...).


What kinds of color correction do we want?

just yv12 bt709 ...


How much low-luma shadow detail to look for blocking?

well it's pre-process ... don't change anything for the encoding process itself.


Anything interesting we should do for credits?

Well just classical pre-precess like dithering. Anyway don't change anything too for the encoding process itself.


Objective here is not to make the best possible pre-process but compare codec efficiency. Anyway if you want make the best possible pre-process it's not prohibited. The source must be simply exactly the same for all the codec in direct input if you want real comparison. Imply for example that internal filter pre-process (like denoiser or dark noise filter for cinevisionPSE) are prohibited.

[benwaggoner]

Realistic scenario is the large majority of studio don't use complex pre-process (No time and no money for make that).
Oh, studios absolutely do a lot of of preprocessing for major titles, including advanced dithering technologies for 10-bit to 8-bit. I'm planning on following tha workflow as closely as possible.

But my question is about color correction/color timing, and all Hollywood films get that. What kinds of images are hard? Saturated? I want to do some intersting stuff to test gradient quality in the sunset and blue sky shots, for example.


just yv12 bt709 ...
Yes, that's what my deliverable will be in, of course.

Objective here is not to make the best possible pre-process but compare codec efficiency. Anyway if you want make the best possible pre-process it's not prohibited. The source must be simply exactly the same for all the codec in direct input if you want real comparison. Imply for example that internal filter pre-process (like denoiser or dark noise filter for cinevisionPSE) are prohibited.
...within the scenario of feature film content sourced on 35mm.

The place I'm looking for feedback on is making sure we've got a good mix of shots and edits that match the "interesting" stuff in movie sources. Making it a good test of rate control, shadow detail, etcetera.

If you wanted a clip to test different encoders, what would you want in it?

[Sagittaire]

If you wanted a clip to test different encoders, what would you want in it?

I have the ideal source for make that: Casino Royal.

- color: N&B and highly satured part
- grain: all possible type are in practice here
- scene type: really clean CGI, N&B with heavy grain, explosion, fire, water, low motion, very high motion, gradient ... you can find all the possible type of scene in this movie.

If you have the same complex source it's ideal.
Anyway your 1080p source seem really good for make test.

[benwaggoner]

I have the ideal source for make that: Casino Royal.

- color: N&B and highly satured part
- grain: all possible type are in practice here
- scene type: really clean CGI, N&B with heavy grain, explosion, fire, water, low motion, very high motion, gradient ... you can find all the possible type of scene in this movie.
I don't think I have the time to do good CGI, but maybe some opening credits of some sort. Suggestions?

We don't have any explosions or fire in my source, but we make it up in water!

[CruNcher]

@benwaggoner
btw what for options did you used to create this in VC-1 i find it rather strange that it's still very easy to transcode and also the problems with the chaotic movement scenes "watter" heavily blocking seems not like a very high compression factor was used here?
Actually this is the second time now after your What happens in Vegas encode i see this kind of heavy blocking now, what are the restrictions for a DLNA compatible stream?

[benwaggoner]

@benwaggoner
btw what for options did you used to create this in VC-1 i find it rather strange that it's still very easy to transcode and also the problems with the chaotic movement scenes "watter" heavily blocking seems not like a very high compression factor was used here?
It was an encode with the beta of Expression Encoder 2. Maybe 3 Mbps CBR :)? I don't remember exactly what I used.

Actually this is the second time now after your What happens in Vegas encode i see this kind of heavy blocking now, what are the restrictions for a DLNA compatible stream?
The content was edited for some DLNA encoding tests, but this isn't a DLNA compliant encode. It was just something I whippped out quickly for this review.

[Sagittaire]

I don't think I have the time to do good CGI, but maybe some opening credits of some sort. Suggestions?

Well the source like that is good. Open/End credit are not difficult to encode by definition.

challenge could be in too part:

1) Codec evaluation: source must be used without any pre-process (Dark noise filtering, dithering, noise/grain optimisation are possible but must be the same in input for all the codec)

2) Best looking evaluation: all pre-process are free. All codec are free (H264, VC1 and MPEG2 implementation).

[benwaggoner]

Well the source like that is good. Open/End credit are not difficult to encode by definition.
Closing, certainly. Although including them is a good test for rate control for 2-pass VBR; good to be able to squeeze the bitrate down for those exactly as low as they can go, and no more.

challenge could be in too part:

1) Codec evaluation: source must be used without any pre-process (Dark noise filtering, dithering, noise/grain optimisation are possible but must be the same in input for all the codec)

2) Best looking evaluation: all pre-process are free. All codec are free (H264, VC1 and MPEG2 implementation).
Interesting. Although it can be a hard to determine what's strictly preprocessing and what's not in a particular preprocessing implementation, like a QP-adaptive low-pass filter).

[benwaggoner]

Also, I put up a sample of the 1080i source over on my blog if folks are interested in that. Not film source, obviously, but it's already edited and everything. I just need to figure out how I can share around a 15 GB file.

http://www.on10.net/blogs/benwagg/Sample-Encoder-Test-Clips/

[nm]

I just need to figure out how I can share around a 15 GB file.
With BitTorrent? I think there would be enough interested people here to make it work.

[benwaggoner]

With BitTorrent? I think there would be enough interested people here to make it work.
Fair enough! I've got a Server 2008 machine at home with a static IP. What's everyone's favorite tracker software these days?

[benwaggoner]

Try this one; it's a ATSC compliant .ts of the 1080i source. It'll give you a sense of what the video looks like, and that my tracker is working.

http://216.99.212.233:6969/torrents/TallShip_1080i_ATSC.zip.torrent?9BA0395FC0E7B5D2203F9BEF4CB3AB54B0B8D367

[nm]

Tracker seems to work fine and the video looks very nice indeed :)

[Inventive Software]

Not a problem with the torrent. Nice fast download speeds. :)

[Sagittaire]

Seem really good source for make test

[benwaggoner]

Seem really good source for make test
Okay. I can put up a torrent of the 1080i this weekend sometime (15 GB as Lagarith 4:2:2!).

Do people have a preference of whether I mux in the PCM 5.1 24-bit soundtrack into the AVI or not?

[Sagittaire]

Okay. I can put up a torrent of the 1080i this weekend sometime (15 GB as Lagarith 4:2:2!).

Do people have a preference of whether I mux in the PCM 5.1 24-bit soundtrack into the AVI or not?

1080i source is unfortunaly completely useless for this test ...

[benwaggoner]

1080i source is unfortunaly completely useless for this test ...
True, but it could be useful for plenty of other tests, and it has the benefit of not having to wait for me to finish it :).

[Inventive Software]

At a push, do both 1080i and 1080p? HD Content delivery to TV sets is almost always 1080i because 720p looks great on 720p displays and a bit crap on 1080p displays. 1080i is in the middle because it's got about the same bandwidth per encoded frame as 720p. Especially true for sports content.

[MfA]

HD content delivery is almost always 1080i because 1080 is a higher number than 720 ... after deinterlacing everything looks "a bit crap".

[benwaggoner]

At a push, do both 1080i and 1080p? HD Content delivery to TV sets is almost always 1080i because 720p looks great on 720p displays and a bit crap on 1080p displays. 1080i is in the middle because it's got about the same bandwidth per encoded frame as 720p. Especially true for sports content.
ATSC doesn't support 1080p60, so 720p60 and 1080i30 are actually in the same pixels/second ballpark.

HD content delivery is almost always 1080i because 1080 is a higher number than 720 ... after deinterlacing everything looks "a bit crap".
I've always thought it was because the old video engineers had a hard time imagining life without interlaced.

[MfA]

Whereas the young engineers and display manufacturers can imagine it all too well ... a live without interlacing means better pictures with less hardware and more straightforward algorithms, ie. less chance for them to distinguish themselves.

Interlacing is great for engineers, lousy for viewers.

[CruNcher]

Whereas the young engineers and display manufacturers can imagine it all too well ... a live without interlacing means better pictures with less hardware and more straightforward algorithms, ie. less chance for them to distinguish themselves.

Interlacing is great for engineers, lousy for viewers.

Hehe yeah it's the same with Digital Cinema it will come as the day will come every channel is broadcasted Progressively tough it will still need some time till the New Young Generation will take fully over the OLD, but the time will definitely come :)

[Inventive Software]

Whereas the young engineers and display manufacturers can imagine it all too well ... a live without interlacing means better pictures with less hardware and more straightforward algorithms, ie. less chance for them to distinguish themselves.

Interlacing is great for engineers, lousy for viewers.

So THAT's why some TVs are absolutely fabulous, and others are complete and utter gobs**te at handling the "i"...

Simplicity almost always wins. In this case, it's been beaten by something that gets abused far too often. :(

[benwaggoner]

Okay, I've actually got Premiere Pro CS3 loaded on my new workstation, and have spent time on the 24p edit today! I don't know that I'll get it fixed this side of Streaming Media East, but I've got a ocuple of questions.


For credits, what's a typical percentage of feature film length they take up?
What's a typical font size for scrolling credits?
For opening title, any thoughts on what would be interesting and challenging?

[benwaggoner]

I got around to sticking up the 1080i30 source with 5.1 audio:

http://216.99.212.233:6969/torrents/TallShip_lag_YUY2_5.1.avi.torrent?96548FD06299672814021EB0D5BD9FA826024E68

I'm out of town for most of the week, so here's something for my bandwidth to do.

[Golgot13]

Hi Ben,

the download is too slow with my ISP.
I don't understand why 1080i30, your source is 1080p24 ?
The video is to test the efficient of codec on HD media,
so we can put 1080p24 (or 23.976) it's work well on BD or HDDVD.

You talked about grain preservation, video quality,... and
you convert your source for 24P to 30i !!! It's really strange...

So I will upload a true film source (with movie grain, 35mm)
at the end of week. This small video will be in 1080p24 (no audio)
in PNG and enough to test the efficient of three codec.

[nm]

the download is too slow with my ISP.
Works pretty well here (30-40 kbps), considering that there is currently only one seed and a few peers.

I don't understand why 1080i30, your source is 1080p24 ?
Ben said that this is the source, so it is originally 30i. The MPEG-2 encode he put up earlier already confirmed this. Although this is not what some people expected, it is still a very nice video for comparing encoders, deinterlacers and other filters.

So I will upload a true film source (with movie grain, 35mm)
at the end of week. This small video will be in 1080p24 (no audio)
in PNG and enough to test the efficient of three codec.
Sounds great!

[benwaggoner]

Works pretty well here (30-40 kbps), considering that there is currently only one seed and a few peers.
Yeah, I've only got 898 up, which is being sliced pretty thin right now.

Ben said that this is the source, so it is originally 30i. The MPEG-2 encode he put up earlier already confirmed this. Although this is not what some people expected, it is still a very nice video for comparing encoders, deinterlacers and other filters.
Actually these are two different sources.

This 1080i30 clip was completed some time ago by a professional editor, and I thought might be interesting for people to work with for other tests. The 1080p24 content is from the same shoot but from 35mm film cameras, and I'm slowly hacking away at it time permitting.

[Biggiesized]

How about getting that 1080p24 source up in a .PNG image sequence?

[Inventive Software]

Why? Lagarith offers far better compression than PNG, is quicker to load. The only caveat is it's Windows only.

[Biggiesized]

Because I'll want to mess around with the clip quite a bit and encode small segments for personal testing. Can you even edit Lagarith?

[GodofaGap]

It's just as editable as any other avi file...

[Golgot13]

I will use a lossless codec because it's hard to share
a big file on the web (I will not use any P2P).

[Biggiesized]

It's just as editable as any other avi file...
Do you need to use VirtualDub or will any program work with a DirectShow filter?

[benwaggoner]

Do you need to use VirtualDub or will any program work with a DirectShow filter?
It works pretty much everywhere I've tried, including VfW and DirectShow apps. I use Lag all the time with Premiere Pro, After Effects, VDub, Expression Encoder, etcetera. The only stuff it doesn't work with are things with their own media pipeline (like VLC, IIRC).

[Inventive Software]

I've spent about 3 weeks trying to download the 1080i30 lossless source, and got about 5 GB through it, when no seeds appears. If anybody's downloaded it, could they help by reseeding please? :) I appreciate my download bandwidth's not quite as quick as I'd like (though I was getting around 100 KB/sec on another torrent this morning), but if anybody's got *fast* uploading speeds, that'd help.

[nm]

I have been seeding, but the tracker seems to be currently down.

[benwaggoner]

I have been seeding, but the tracker seems to be currently down.
Oops, looks like it went down in the thunderstorm we had night before last. I'm rebooting it now.

[Inventive Software]

What do people know about adding trackers, i.e when you've downloaded it? Would be useful if it goes down again. Not ribbing, just looking to make it a lil more reliable.

[benwaggoner]

What do people know about adding trackers, i.e when you've downloaded it? Would be useful if it goes down again. Not ribbing, just looking to make it a lil more reliable.
I thought I set up the file to use distributed trackers, so it should have kept working after the server went down. Not sure what went wrong there.

[Inventive Software]

*bump*

I figured out how to use the uni's enormous internet speeds for my use on BitTorrent, but the tracker again seems to be down, and I can only see 1 peer........:confused:

[benwaggoner]

*bump*

I figured out how to use the uni's enormous internet speeds for my use on BitTorrent, but the tracker again seems to be down, and I can only see 1 peer........:confused:
Hmmm. I'm not at home right now, but I can't ping that machine, so it's down. That power switch is very toddler-compelling...

I'll kick it off again tonight.

And anyone better at hosting than me is welcome to share it as well!

[benwaggoner]

I'll kick it off again tonight.
...and it's back up!

[TEB]

I have it on at work constantly.. @ 85% now i belive @ on a 50mbit link

[benwaggoner]

Glad to hear it's working again.

And again, the license on the last page is for "any use whatsoever" - anyone's welcome to transcode it into another source format, distribute it on their own, or whatever.

[benwaggoner]

Hey all,

I got a DSL update that (yeah!) has taken me up to 12 Mbps, but (no!) took away all my static IP addresses. Thus the tracker is down.

Anyone still trying to download? I presume there's some way to do it via NAS, but I haven't looked into it yet.

Anyone else is welcome to share the file out, of course.

[Sharktooth]

use a dynamic dns ... :p
http://www.dyndns.com/

[benwaggoner]

use a dynamic dns ... :p
http://www.dyndns.com/
Medium-term I'll just stick it up on connect.microsoft.com when I'm on campus and can upload it.

[Inventive Software]

@Ben: please do that. You have no idea how fast our university can download from Microsoft's website! :D

[CruNcher]

Argh i missed it, does someone have a torrent to it that's still seeded ?

[benwaggoner]

Argh i missed it, does someone have a torrent to it that's still seeded ?
Ugh, I still haven't set up dynamic DNS here, sorry.

I'll try to get around it in a few days. Anyone else is welcome to seed it, of course.

That said, I've been staring at the clip a bunch the last few weeks, and I'm thinking about going back and doing some color correction. Some parts are too dark, and others are inexplicable magenta-tinged.

[CruNcher]

Thx Ben for your help :)

[benwaggoner]

Thx Ben for your help :)
Did you find it yet?

I didn't get a chance to get it done before I left on this week's trip to Boston. Alas, I didn't find a good place to stick something that large on Microsoft.com.

Any preference between me getting the current one hosted versus doing a new tweaked version from the 10-bit source?

[CruNcher]

Did you find it yet?

I didn't get a chance to get it done before I left on this week's trip to Boston. Alas, I didn't find a good place to stick something that large on Microsoft.com.

Any preference between me getting the current one hosted versus doing a new tweaked version from the 10-bit source?

It doesn't hurry :) take your time and paint it like you want it to be first ;) i actually have the TallShip_1080i_ATSC.ts one if that is also the uncompressed version could you maybe look for a longer lasting closeup scene of one of the Crew Members @ 2:23 there is not a super closeup but in that direction (perfect would be a conversation like http://s7.directupload.net/file/d/1499/dzr87ssc_png.htm closeup), tough this one is to short and the one that follows it of that woman @ 2:24 is underexposed and again to short :(

[benwaggoner]

It doesn't hurry :) take your time and paint it like you want it to be first ;) i actually have the TallShip_1080i_ATSC.ts one if that is also the uncompressed version could you maybe look for a longer lasting closeup scene of one of the Crew Members @ 2:23 there is not a super closeup but in that direction (perfect would be a conversation like http://s7.directupload.net/file/d/1499/dzr87ssc_png.htm closeup), tough this one is to short and the one that follows it of that woman @ 2:24 is underexposed and again to short :(
I don't have any different edits of the 1080i source, and given how long it's taken me to get around to finishing the 1080p24 source, I wouldn't want to have to go back to tape :).

The main thing I'd do in a "remaster" would be to add a title sequence, rerender the end credits, and fix some exposure and color timing issues in the 10-bit master.

[CruNcher]

I don't have any different edits of the 1080i source, and given how long it's taken me to get around to finishing the 1080p24 source, I wouldn't want to have to go back to tape :).

The main thing I'd do in a "remaster" would be to add a title sequence, rerender the end credits, and fix some exposure and color timing issues in the 10-bit master.

OK thats fine then any estimation when we can expect the release of the 1080p24 version :) ?

[benwaggoner]

OK thats fine then any estimation when we can expect the release of the 1080p24 version :) ?
Ugh.

So, here's a quick question:

What's an appropriate percentage of the file to be spent on the credits? Also, anyone know what's ballpark "standard" for font size and scroll speed in film credits?

I also should do some opening credits something. Any thoughts as to what that could be?

Those are the things I think about when I sit down to start working on it, before I get distracted and do something else :).

[Sharktooth]

use helvetica (Arial). it's readable. Or a free readable font.
credits lenght depends on you. usually movies have from 1 to 10 mins of credits.
if you put all the credits on opening then the shorter the better.

[benwaggoner]

use helvetica (Arial). it's readable. Or a free readable font.
credits lenght depends on you. usually movies have from 1 to 10 mins of credits.
if you put all the credits on opening then the shorter the better.
I want to have about the same proportion of credits/main content in the clip as in a typical movie, of typical movie style.

[Koadic]

Doing some quick research on what I have available, I have found that credits make up around 10% of a 2 hour movie. 1.5-2 minutes at the begining (about 1.5%) with some sort of studio identifier(s) of around 30 seconds and then video overlaid with text fading in and out for another 1 to 1.5 minutes. Then about 10 minutes (about 8.5%) of scrolling text on a black background at the end with a scroll rate of about 10% per second (full 16/9) or about 13% per second (2.35:1).

If I have read correctly, this is the kind of information your looking for.

[Sagittaire]

I want to have about the same proportion of credits/main content in the clip as in a typical movie, of typical movie style.

Well 5% is a good deal ...

[TEB]

Hi. I was tinkering with avisynth and this clip for feeding x264 with.
I see that the clip is YUV2 1080i30 right? Could a good soul here with avisynth knowledge print out an example on how to properly make this clip progressive and in the right colourspace handled by x264?

thx

TEB

[Sagekilla]

Easy:

Deinterlacer()
ConvertToYV12()


Replace Deinterlacer() with any one of the number of deinterlacers available: Yadif, mcbob, TGMC, etc.

[benwaggoner]

Oh, and am I ever embarassed that I haven't done anything on the 24p version in ages. Nice to see someone working with the 1080i, though.

I wish I could promise something soon. But I've got 1 and 1/3rd books to get written in the next six months...

[Biggiesized]

It sounds like this is never gonna happen.

[benwaggoner]

It sounds like this is never gonna happen.
I delude myself that I'll get to it one of these days...

The good thing is Moore's Law makes it easier to do the longer I wait :). Once the new Cineform Prospect integration with Premiere Pro CS4 is done, it should be an afternoon's work instead of a week's work.

[Biggiesized]

And what tools do you need to get your second book out in a month's time rather than the better part of a decade? :p

[benwaggoner]

And what tools do you need to get your second book out in a month's time rather than the better part of a decade? :p
I've actually completed the book, and it should be in stores in November!

[Lyris]

Let us know when/where to get it!

[Biggiesized]

How revised is it compared to your first edition? I remember reading about codecs that, for all intents and purposes, no longer exist in today's production use.

On-topic: that clip is great, but we need the film only version :p

[benwaggoner]

How revised is it compared to your first edition? I remember reading about codecs that, for all intents and purposes, no longer exist in today's production use.
It's pretty radically different; probably 80% of the text is new.

On-topic: that clip is great, but we need the film only version :p
Yeah...