Red on black enhancement?

[FBX]

One thing that's been bothering me while working on my Transformers restoration project is how red with black outlines has always look rather messy from the DVD source files. All other color combinations look nice and sharp, so the red/black combo really sticks out like a sore thumb. Here's a classic example:

http://www.firebrandx.com/downloads/tfexample.jpg

Is there a plugin that deals specifically with this problem or perhaps a setting in one that limits alteration to only areas with red next to black? Much appreciated for any advice the experts have on this!

[Lyle_JP]

It looks to me that at one point this video was fed through an MPEG2 decoder with the CUE (Chroma Upsample Error) bug (http://www.cockam.com/vidbug2.htm). Sorry, I'm not sure how to correct that.

The other problem could be that your chroma is pushed up too high in the red spectrum. Perhaps easing off a bit on the red could help.

[FBX]

I'm using AviSynth with DGDecode and Decomb plugins to frame-serve into VirtualDub the raw MPEG2 source. Is there a setting I need to make/adjust in my AVS file to avoid that CUE issue?

Edit: I don't think it's CUE. I looked at the original DVD and it has the same messy red detail as my output does.

[FBX]

Well it turns out some of the artifacts were indeed from a setting I left in my AVS file: upConv=1. Turning that off got rid of the general jaggies around the red edges, but the detail is still muggy. Here's the updated pic with upsampling turned off:

http://www.firebrandx.com/downloads/tfexample2.jpg

So now all I need is a line sharpen filter that limits itself to just red outlines :-P

[FBX]

I found mfToon gets very close to the results I want:

http://www.firebrandx.com/downloads/tfexample3.jpg

Effectively doubling the apparent color resolution around the red edges. Unfortunately it also over-sharpens everything else in the image. These are just default settings, so I'm hoping to learn enough to tweak it to my needs.

[Soulhunter]

Wouldn't working in RGB (yes, i know...), to limit the sharpening and warping to the fuzzy red channel, do the trick?

[FBX]

I'm sort of a noob so I don't know how to go about that. Can you give me some pointers on how to do that? Thanks for your help!

[Soulhunter]

Either use RGBHack (http://soulhunters-crappy-website.com/data/RGBHack.zip), or Avisynth's inbuild ShowRed() + ShowGreen() + ShowBlue() + MergeRGB()

[FBX]

Unfortunately I get that CUE problem when I use the upConv function to convert the video into RGB. Using the built-in fix for this error only made it worse. If it weren't for that, I could use ShowRed to make a separate video file of the red tones for cleanup and then possibly merge that back into the remaining GB video file.

I don't know, it seems a a difficult task to accomplish, especially since the damn CUE problem is preventing me from doing it.

[Gavino]

Unfortunately I get that CUE problem when I use the upConv function to convert the video into RGB. Using the built-in fix for this error only made it worse.
I don't know what upConv is, but can't you use ConvertToRGB(interlaced=true) instead?

[Wilbert]

upConv samples to YUY2 following the progressive frame flag: http://forum.doom9.org/showthread.php?p=1378117#post1378117 Whether the flag is reliable is another question.

Try AutoYUY2: http://neuron2.net/autoyuy2/autoyuy2.html

FBX, is your source progressive or interlaced?

[FBX]

The source is interlaced and part of my restoration project is using Decomb's Telecide command with my own text file containing exact frame re-combination patterns (yes, I manually correct all 41,000+ frames in each episode to ensure 100% accuracy). Here's how my script is currently serving to VirtualDub (where I then use resizing and "neatvideo" for cleanup):

LoadPlugin("C:\DGIndex\DGDecode.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\filters\Decomb.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\filters\aWarpSharp.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\filters\mt_masktools.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\filters\warpsharp.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\filters\Repair.dll")
LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\filters\MaskTools.dll")

Import("C:\Program Files (x86)\AviSynth 2.5\filters\mfToon-v0.52.avs")
Import("C:\Program Files (x86)\AviSynth 2.5\filters\DeHalo_alpha.avs")

MPEG2Source("TF_S1_01.d2v", cpu2="xxxxxx", upConv=0)

AssumeTFF()
Telecide(ovr="customtelecide01.txt",guide=0,gthresh=100,vthresh=255,post=4,blend=true)
Decimate(cycle=5)

mfToon( strength=128, wdepth=24, wthresh=0.3)
DeHalo_alpha(rx=2,ry=2,darkstr=0.0)

The "customtelecide01.txt" is a text file containing field combining patterns for all 41,000+ frames. It makes for flawless 24fps playback after Decimate removes every 5th duplicate frame.

[FBX]

Just wanted to update in that I was able to develope a working system of extracting the red channel, applying mfToon directly to it, and then using MergeRGB to layer it back into the original footage. Turns out my problem wasn't exactly CUE, but rather the red chroma channel shifting to the left by a pixel upon conversion to RGB. In VirtualDub, I just use the ChromaShift filter to shift it back to the right and everything looks kosher.

So now I can fix the red detail with this work-flow, but it means I'll be doing lossless encodes three times instead of just once:

1st lossless encode: Custom IVTC and various restoration filters.
2nd lossless encode: Extracting and mftooning red channel.
3rd lossless encode: Re-inserting corrected red channel.

Final lossy encode: Blockbusting

Ugh, what a load of extra work! But I can't pass up the ability to fix the muggy reds, so it's gotta be done.

[poisondeathray]

Turns out my problem wasn't exactly CUE, but rather the red chroma channel shifting to the left by a pixel upon conversion to RGB. In VirtualDub, I just use the ChromaShift filter to shift it back to the right and everything looks kosher.


Is this something peculiar to your source, or from the method of converting to RGB ? How did you end up doing the RGB conversion ?

[pandy]

Try to play with median filter (GeneralConvolution?) on red channel - especially after resizing frame at least twice (use PointResize 2x)

[Soulhunter]

Tried RGBHack instead of MergeRGB?

[FBX]

So I sort of figured out why the red tones look so bad on these shows. Upon looking at each of the Y, Cr, and CB channels, I discovered that any robots with red parts have virtually no luminance information in those areas (looks blacked out). Because of the YV12 colorspace storing color at basically 25% of the luminance resolution, any areas that are relying entirely on a single chrominance channel for detail end up looking like shit compared to the rest of the image. Further proof I found is a couple of scenes where the red paint was incorrectly colored slightly orange. In those scenes, the line detail was much clearer and sharper due to having some amount of luminance information.