x264 settings for NTSC DV to 4:2:2?

[FredThompson]

I'd like to archive NTSC DV using x264. Which colorspace conversions (via AviSynth) should be used and where in the x264 settings do I specify 4:2:2?

[Blue_MiSfit]

First things first, make sure you use a decoder that outputs YUY2 for DV, and does a nice job of resampling the chroma. I recall Cedocida being good at this. It's VFW if memory serves. That probably means you can just decode directly into AviSynth using AVISource(...). I wouldn't use any avisynth filters unless you want to do noise reduction or something along those lines. I'd say keep the sources as-is without any filtering.

Regarding x264 configuration, the manual (x264.exe --fullhelp) contains the following:

--output-csp <string> Specify output colorspace ["i420"]
- i420, i422, i444, rgb


So... x264.exe [settings] --output-csp i422

If you're looking to archive without intent to edit directly then keeping the defaults and picking a CRF that suits your requirements is probably all you'd need to do. If you intend to edit directly (verify your editing app can decode 4:2:2 H.264), then I'd use a short keyint, maybe 15 to 30 frames. This will keep things simple.

In any case, code interlaced bottom field first, assuming your sources are standard NTSC DV (480i60 BFF)

[Atak_Snajpera]

x264.exe --crf 20 --bff --output-csp i422

[Blue_MiSfit]

I'd go lower than CRF20 if you're archiving. Maybe 14-16 depending on your bitrate tolerance...

[benwaggoner]

In any case, code interlaced bottom field first, assuming your sources are standard NTSC DV (480i60 BFF)
Quite a lot of DV cameras supported progressive, so any given bitstream (not just tape) could be 480p30 or 480i30.

[Blue_MiSfit]

You're quite right, but 480i is by far more common - no?

[benwaggoner]

You're quite right, but 480i is by far more common - no?
Yeah, it's probably more common. But not overwhelmingly so. It's worth checking each file.

[jfcarbel]

I wouldn't use any avisynth filters unless you want to do noise reduction or something along those lines. I'd say keep the sources as-is without any filtering

Why would you not advise any cleanup? DV video can be of the home video type, thus shaky and noisy. Is there a string reason you would not want to touch the video and is it because of the Vfw based codec?

Also, would it be advised to de-interlace using QTGMC?

[Blue_MiSfit]

If you want to do cleanup, stabilizing etc, that's fine. Personally I would keep the unfiltered files as well, since technology may improve in a few years, at which point you may want to re-process the video.

The same goes for deinterlacing. If you're going to do something with these videos that requrie a progressive output, then yes by all means use the best technology currently available. This would be QTGMC - at least in the AviSynth world.

[Asmodian]

Why would you not advise any cleanup? DV video can be of the home video type, thus shaky and noisy. Is there a string reason you would not want to touch the video and is it because of the Vfw based codec?

Also, would it be advised to de-interlace using QTGMC?

Speaking for myself, I think it is a matter of archive vs "to watch".

I used to believe a very good QTGMC was a good idea even for a very low crf x264 archive. I have, after a little more experience, decided a good, as untouched as possible, encode is preferred. That one scene that didn't deinterlace well or some other artifact you miss is very annoying if it is in your best quality file and you know you caused it while using lots of CPU time :p.

If you are making a clip to upload for people to watch directly or doing a quality/size optimized "to watch" clip then QTGMC is a good idea.

[jfcarbel]

If you want to do cleanup, stabilizing etc, that's fine. Personally I would keep the unfiltered files as well, since technology may improve in a few years, at which point you may want to re-process the video.

The same goes for deinterlacing. If you're going to do something with these videos that requrie a progressive output, then yes by all means use the best technology currently available. This would be QTGMC - at least in the AviSynth world.

Ok, I see your point the OP mentions archive so I see the reason to store it untouched.

If you're going to do something with these videos that requrie a progressive output

I suppose I can't think of anything that really requires progressive. I suppose if uploading a video to YouTube or maybe for a mobile device. But every HDTV can handle interlaced video and h.264 can encode interlaced. I do think it requires that you put your HDTVs motion enhancement option on for this feature.

Even if storing the encoded video to a Blu-ray, even BD supports 720x480 (http://en.wikipedia.org/wiki/Blu-ray_Disc#Video) NTSC interlaced. However, for H.264 SD, the 8-bit (4:2:0) Colorspace is only allowed. Thus I think the x264 parm would need to look like this for bluray compatible video: -output-csp i420

Now what about if someone does have shaky video and noise they want to cleanup, then I thought I read its best to de-interlace so the filters can work better. Is that true?

Someone in another thread I created on this topic had stated that just about any advances in real-time de-interlacing would never match that of a computer processed de-interlacing that can better analyze the video. It probably is better then what most HDTVs offer today.

But then again we see something like this (http://gizmodo.com/5818238/vreveal-uses-military+grade-image-stabilization-to-fix-your-shaky-home-movies) using GPU for real time processing and one wonders how far off we are from having TVs with dedicated GPUs.

[benwaggoner]

I suppose I can't think of anything that really requires progressive. I suppose if uploading a video to YouTube or maybe for a mobile device. But every HDTV can handle interlaced video and h.264 can encode interlaced. I do think it requires that you put your HDTVs motion enhancement option on for this feature.
You may need motion enhancement to get 120 fps interpolated out of 480i or something. But any HDTV is going to display 30i as 60p, displaying each field deinterlaced.

This won't look as good as a high quality Bob deinterlacing in AVISynth. But 480p60 also isn't as widely playable.

The proper archive of an interlaced source is an interlaced file.

Even if storing the encoded video to a Blu-ray, even BD supports 720x480 (http://en.wikipedia.org/wiki/Blu-ray_Disc#Video) NTSC interlaced. However, for H.264 SD, the 8-bit (4:2:0) Colorspace is only allowed. Thus I think the x264 parm would need to look like this for bluray compatible video: -output-csp i420 [/QUOTE]
Correct. 4:2:2 would only be useful as an archive, and won't play back on many devices.

With MBAFF, I suspect a 4:2:0 480i encode is going to look pretty darn good with most content. The biggest problem with DV 4:1:1 color is the one chroma sample for every four pixels horizontally.

I've heard from multiple contemporaneous sources that this was actually a feature from the Japanese camera manufacturers to keep DV from competing with professional cameras too much. Shady backroom deals later kept the 4:2:2 DV50 from becoming a prosumer standard.

Now what about if someone does have shaky video and noise they want to cleanup, then I thought I read its best to de-interlace so the filters can work better. Is that true?
Depends on the filter. There are good interlaced-mode denoising filters in existence. I'm not sure about in AVISynth.

Someone in another thread I created on this topic had stated that just about any advances in real-time de-interlacing would never match that of a computer processed de-interlacing that can better analyze the video. It probably is better then what most HDTVs offer today.
The bigger problem with deinterlacing is that going to 30p throws out half your temporal sample: 30i is 60 unique fields a second. A good bob to 60p gets around that problem, but can also make a difficult to play back file. And I would argue would be a mezzanine, not an archive. But that may be fine, or even preferable for some scenarios.

[jfcarbel]

...But 480p60 also isn't as widely playable.

...With MBAFF, I suspect a 4:2:0 480i encode is going to look pretty darn good with most content. The biggest problem with DV 4:1:1 color is the one chroma sample for every four pixels horizontally.

...The bigger problem with deinterlacing is that going to 30p throws out half your temporal sample: 30i is 60 unique fields a second. A good bob to 60p gets around that problem, but can also make a difficult to play back file. And I would argue would be a mezzanine, not an archive. But that may be fine, or even preferable for some scenarios.

All very good points, thanks for illustrating. I think for ultimate compatibility and playability it seems encoding as interlaced is the best route. And for archiving the best route since you are not throwing anything out.

But, lets say I decided that I would archive one copy but the next one I made would be to focus on the best quality I could get at this time. And playback support would not be an issue as I would either use a network device (Roku, WD TV Live) or an HTPC which should be able to play back almost any format, resolution, etc. With that said what would be the best route considering I would use the x264 for encode, source is DV from 8mm tape or MiniDV, and would play then on a modern HDTV?

You also mentioned earlier.
If you intend to edit directly (verify your editing app can decode 4:2:2 H.264), then I'd use a short keyint, maybe 15 to 30 frames.

Can most top quality editing apps like Sony Vegas or Adobe Premiere decode 4:2:2? Is the short keyint to make frame accurate editing easier? I assume this will increase the size significantly since more full resolution I-frames are including.

[Blue_MiSfit]

I imagine Premiere / Vegas can decode and edit AVC Intra, which is 10 bit H.264 in either subsampled (1440x1080) at 50mbps with 4:2:0, or full res (1920x1080) at 100mbps with 4:2:2.
That being said, I doubt those decoders would be available for / work with any old 4:2:2 bitstream.

[benwaggoner]

But, lets say I decided that I would archive one copy but the next one I made would be to focus on the best quality I could get at this time. And playback support would not be an issue as I would either use a network device (Roku, WD TV Live) or an HTPC which should be able to play back almost any format, resolution, etc. With that said what would be the best route considering I would use the x264 for encode, source is DV from 8mm tape or MiniDV, and would play then on a modern HDTV?
If your devices can decode 480p60, definitely go for that.

Can most top quality editing apps like Sony Vegas or Adobe Premiere decode 4:2:2? Is the short keyint to make frame accurate editing easier? I assume this will increase the size significantly since more full resolution I-frames are including.
Yes, I believe the editing tools can do that, at least for some cases. I recommend trying a test file.

Yes, the keyint is to make editing easier. At archival bitrates, the overhead of extra I-frames isn't that bad.

That said, DV is only 25 Mbps. The simplest and risk-free archival strategy is to just save the DV bitstreams as is, and DEFINITELY everything can edit those!

[Blue_MiSfit]

Yeah, the more I think about it, keeping the DV as-is makes the most sense from an archival perspective. If taking things down to ~10-20 Mbps is really worth it, then the comments in this thread are quite relavent.

Also, yes, intra-only doesn't have such a huge impact at high bitrates.

[jfcarbel]

I do understand the DV is the best archival, but I have so much footage to capture that I am just not going to have the space. I also want to back up some to BD-R to store in a safe. I will keep most of the most important ones in DV on the drive but most will need to find another well suited format. Since x264 is one of the best lossy encoders out there, I wanted to go that route.

I did some more research and also found that a constant frame rate (not bit rate) is also suggested since it enables better video editing. The setting is --force-cfr

Also I think most editors are only going to support 4:2:0

[Blue_MiSfit]

--force-cfr will be redundant in this case.

[jfcarbel]

--force-cfr will be redundant in this case.

Why is that? I suggested it because I might in future want to non-destructively edit it using VideoRedo and they make this suggestion here (http://www.videoredo.net/msgBoard/showthread.php?t=28392).

Here is what I had come up with in terms of the command line options:

--crf 13 --preset slower --tune film --bff --output-csp i420 --keyint 24 --min-keyint 1 --force-cfr

also wondering if I need to force level to 3.1 or set color matrix.

[Blue_MiSfit]

DV input will have a constant frame rate output. Guaranteed.

If you want to edit later, use a short keyint (and don't set min-keyint). I'd suggest going all I-frames. In other words, set --keyint 1.

If you must do interlaced encoding at 4:2:0, I'm not sure where the downconversion is best achieved. There are 3 possibilities that I can see:

1) Have Cedocida output YV12, with the chroma samples moved for interlaced MPEG encoding. I _think_ this is the correct setting for x264, but I'm not sure
2) Have Cedocida output YUY2, and have AviSynth do ConvertToYV12(interlaced=true)
3) YUY2 all the way, and let x264 do the YV12 conversion.

My hunch is that method 1 is probably ideal, but I'm really not sure.

Derek

[jfcarbel]

1) Have Cedocida output YV12, with the chroma samples moved for interlaced MPEG encoding. I _think_ this is the correct setting for x264, but I'm not sure

Anyone know?

Also is the usage of Cedocida due to the DV chroma bug?

NTSC DV has 4:1:1 chroma subsampling and if you don't do any post-processing before encoding, you end up with chroma bug error.

But then I saw this comment in an avs forum:
Hi-8 and SVHS is around 400 lines of resolution in the luminance and only around 30 lines of resolution in the colour-under chrominance. DVD resolution in XP and SP is 720x480 in the luminance and half of that in the chrominance. So DVD is more than capable of capturing the full resolution of SVHS and Hi-8 video.

So is it really an issue?

So I am a bit confused, here by DV codecs and chroma sampling issues. So let me try to explain my intended workflow and perhaps others can clarify what I need to do.

Input sources NTSC VHS, MiniDV, Video8
Capture via Canopus ADVC-110 (except MiniDV which is over as Firewire digital save)
Canopus will capture 4:1:1 DV format
Edit DV AVI file with NLE
Intent is to save as interlaced H.264

So if after NLE edits, I can render directly to the encoder be it Mainconcept or x264, will I still have these Chroma issues? I think so because something is going to have to decode the DV and it transcodes to H.264 even the MiniDV AVI.

Is there perhaps a NLE that will decode the DV correctly or does it always use the DV codec filter installed on the OS (Canopus in this case)? I had read that Premiere Pro correctly handles chroma (with interpolation) internally and when exporting. Not sure on Vegas. [source (http://forum.videohelp.com/threads/335134-So-is-any-DV-codec-better-than-MainConcept-at-decoding-Does-it-matter)]

I think someone mentioned that the Canopus codec had issues but if install ProCoder software that it installs a new DV codec or some files that alleviate the issue.

The other suggestion was to use Cedocida DV codec, but this is a VFW codec so will it be used when I edit in NLE like Vegas or PP? Or do I need to use avisynth to frameserve to NLE?

[FredThompson]

Your chroma will be limited by whatever internal formats are used by your editing apps.

For archiving DV source, encode to 4:2:2 or you will lose half the chroma. Yes, this means you will have a lot of duplicated chroma information but that's better than losing half the vertical chroma resolution of NTSC DV. 4:2:0 is the most common use of MPEG2 and 4 but not a limit. I used to take NTSC DV to 4:2:2 MPEG2 at 8000 bitrate as an archival format. Now I take it to 4:2:2 x264 at 5000 bitrate. HD camcorder is kept in native formats for archiving. Hard drive space is cheap but 13G for an hour of NTSC DV is waste and it takes longer to parse.

[FredThompson]

I do understand the DV is the best archival...Not true. NTSC is 4:1:1. If your source is 4:2:0 you will lose half the chroma information by converting to DV. 4:1:1 uses a 1x4-shaped group of pixels for each chroma sample. 4:2:0 uses a 2x2-shaped group.

4:1:1 has twice the vertical chroma resolution of 4:2:0 but only half the horizontal resolution.

A lot of people also get confused and assume DV is lossless because it is all I-frames. It IS all i-frames but re-encoding loses information. Multi-generational DV encodes start to look very degraded, similar to multi-generational analog photocopies.

[jfcarbel]

Your chroma will be limited by whatever internal formats are used by your editing apps.

For archiving DV source, encode to 4:2:2 or you will lose half the chroma. Yes, this means you will have a lot of duplicated chroma information but that's better than losing half the vertical chroma resolution of NTSC DV. 4:2:0 is the most common use of MPEG2 and 4 but not a limit. I used to take NTSC DV to 4:2:2 MPEG2 at 8000 bitrate as an archival format. Now I take it to 4:2:2 x264 at 5000 bitrate. HD camcorder is kept in native formats for archiving. Hard drive space is cheap but 13G for an hour of NTSC DV is waste and it takes longer to parse.

I can certainly encode to 422 color space when I take the edited DV AVI in NLE and output to x264. But I can't control the capture as its going thru my ADVC-300 device and captures as DV 411. So won't there be a conversion from DV 411 to MPEG4 422 when doing the decode/encode process during NLE transcode? That is why I thought someone mentioned to use the Cedocida VFW filter so that it corrects this issue as the Canopus DV codec on the OS has this chroma bug issue, but not sure how I would used Cedocida in my workflow, its settings, and if it was even necessary.

Also I am not sure if my NMT device like WD TV Live will play a 422 colorspace, but I will try a test.

Also what is the correct way to have a NLE editor use the x264 encoder? It looks like most NLE support "Debugmode FrameServer (http://www.debugmode.com/frameserver/usage.htm)" as the output type. However instead of directly serving to x264, I will open signpost file in MeGUI and add some noise correction since my source is older VHS tapes. I think the MiniDV and Video8 should be clean enough for MPEG4 encoding.