RemoveBlend - new plugin

[violao]

Recently I became involved in progressive video restoration from badly telecined source. At first I tried the available plugins deblend/unblend and published scripts and realized that none fitted my needs. The files I'm dealing with have no blending pattern, original frame rates are not known and blending certainly doesn't satisfy unblend's assumption that:

Frame = (1 - N)* Previous + N * Next

I found that weight coefficients for previous and next frame are in no way related to each other and that they can both be > 1.

Another thing I didn't like with currend deblend techniques is that they have no support for interlaced source. The idea widely used is to bob the source and then unblend, that obviously results in replacing top blended fields with bottoms and vice-versa.

So using unblend as a start point I designed my own blend remover from scratch. It works by trying to find weight coefficients by examining most different pixels from previous and next frames and using some statistic calculations (medians, means, standard deviation, variance, etc.) to detrermine the reliability of results. I tested it succsessfully with very clean artificially blended progressive clip as well as with my camcorder telecined noisy super8 projections.

Currently default parameters are set for what I expect to be the best values for "clean" blended source and for noisy unblending my parameters were:

threshold = 10
dthresh = 0.5
cthresh = 0.075
pixels = 100
mode = 0

As manual states it is expected to work only on sources that include all clean fields/frames + some blends. If you however have interlaced source with orphan tops or bottoms you may still try the bob approach.

Needless to say removeblend works with both progressive and interlaced and currently only with YV12 TFF. If you have BFF you can convert it with:

separatefields().trim(1, 0).weave()

Although it appears to work in my initial tests I have some concerns about replacing decisions and since I have limited access to varios types of telecined material I would appreciate any feedback. From what I have seen in this forum lately there seems to be a lot of interest in this area.

Last words, I don't intend to make this a full progressive restorer, that is to decomb and decimate the result. There are great tools for that around and I don't even dream of making a better software for that purpose.

Finally, this is my first plugin, so be nice and :thanks:

Download (http://bossanovaguitar.com/video) is at the bottom of man page.

[scharfis_brain]

tried it, but it doesn't seem to have a better blend detection than deblend :(.

the interlaced mode is really messed up.

I'll try to explain:

lets assume the following sequence:


A B C D E EF FG GH
ab bc cd de e f g h


then the output after blend removal has to be similar to this:


A B C D E F G H
a b c d e f g h


where italics are resampled (either shifted 1/2 pixel up or down) fields taken from the prev or next field.

you filter actually replaces WHOLE frames. this completely messes up the interlaced structure and fluidity.
Also I can't use its interlaced output for decimation etc.

this is, why only progressive deblending makes sense (to me)


(smart)bob()
de/un/remove blend()
alreadybobbed=last #Didée's idea
separatefields()
selectevery(4,0,3) # (4,1,2) depends on input's fieldorder,
#to retain most crispness as possible
weave()
smartdecimate(...,bob=alreadybobbed)

[violao]

Originally posted by scharfis_brain
lets assume the following sequence:


A B C D E EF FG GH
ab bc cd de e f g h


You're right. That's why I wrote in manual that it's not supposed to work with subsequent blended frames/fields. Interlaced deblending need something like this:


A AB B C CD D E EF f ...
a b bc c d de e f fg ...


In this example it is supposed to replace AB with B, bc with b, etc.

With clip you suggested you may only try bob/progressive approach.

Thanks for trying.

[scharfis_brain]

ah, I understand, so interlaced deblending may be intended for deblending

17 fps progressive blended into 60fps NTSC

[violao]

Originally posted by scharfis_brain
17 fps progressive blended into 60fps NTSC

Hopefully. This is in fact 16.67 fps -> PAL blended (2 frames in 6 fields)

[scharfis_brain]

here a idea, how to improve blend detection:

some small kind of patter guidance

the user defines a framerate ratio that decribes how many unique frames the input stream has.

a fieldblended PAL Film arriving from NTSC has commonly a ratio of 24:50 = 12/25
(nearly every other frame is a blend)

your 16,66 fps blended into 50fps has a ratio of 1:3
(one blend every three frames)

if you know what I mean.

this could significantly help detection IMO.

[scharfis_brain]

okay after playing with it a while and I found out HOW to tweak its parameters, I found it being useful.

1) could you add a noise map, like deblend's show=true to set up the noise-threshold properly?

2) could you add a functioanlity never replacing directly 2 times?
it sometimes trys to replace 2 follwowing frames with its prev. frames. this makes me (unnessecary lowering dthresh).
I modification would be cool.
this means it only should replace every other frame, not more.
if there is the case it 'could' replace two following frames it only should replace the one with the lower dthresh.

3) on progressive mode, info=true sets show=true automatically.
info=true & show=false doesn't work.

[violao]

1) You don't have to worry much about noise. The nature of statistics calculations protect from noise. At first I use medians that are not affected much by marginal elements and then I discard 10% outliers prior to calculation. Therefore threshold parameter is not much important.

2) For now I disabled 2nd replacing in progressive mode. In interlaced I left 1st top following 2nd bottom, as well as bottom replacement following decombed frame and decombing following any replacement. I have also added 2 more parameters:

bool decomb (default=true) - try to decomb if possible
float bthresh (default = 1.1) - same as cthresh, but applies to duplicate and near duplicate fields

Any field that is near duplicate will have mean value of corresponding coefficient around 1, so if you don't want to detect that as possible blend just lower bthresh to something less than 1.0.

If you're still concerned about possible subsequent replacement just set decomb=false and all subsequent replacement will be disabled (except from top following bottom which is important). Also no decombing.

3) Fixed.

As for pattern guidance, can you suggest any algorithm that will be used for decision if a blend is in-pattern or not? In other words, what happens if we detect out-of-pattern blend and what if we don't detect in-pattern blend?

Thanks for your help and suggestions.

BTW, I managed to clean my test samples to a ratio of around 1 blend / 1000 frames left. Do you feel this would be sufficient for your application?

Download (http://www.bossanovaguitar.com/video/)

[Chainmax]

Is anyone up to the task of making a comparison between this and unblend on both progressive and interlaced video?

[gizmotech]

Alright. I give up.

I've spent the last hour poking and proding the various settings (which make absolutely no sense to me. Perhaps more lamen descriptions are in order) in an attempt to remove blends from a sample vob I recieved a few months back.

Locating an interlaced blend, which could be cleanly matched via telecine to the previous frame to remove it (which is currently being matched to c/n for testing), and for all intents and purposes fidling w/ settings, I was unable to change the results of the debug output any signifigant amount (ie at all).

Could a more simplified explination be provided for the filters use? I even got a collegue to look at it and he was unable to determine what the settings were describing. Also, descriptions on how to deal w/ specific blend scenarios (ie single field risidual blends from previous frame)

Thanks.

Gizmo.

[violao]

Originally posted by gizmotech
...in an attempt to remove blends from a sample vob I recieved a few months back.
First check for combined variance in info mode:

combined variance = prev variance * next variance

Basically you need to find a value for dthresh such that

for progressive blends:
combined variance < dthresh

for interlaced blends:
combined variance < pow (dthresh, 2)

Previous, next and combined variances are display in VAR column.

If a single (prev or next) variance is under dthresh then this field is considered duplicate (only interlaced mode).

Can you upload a sample of few hundreds frames somewhere?

[grua]

http://bossanovaguitar.com/video/ ist down. Where can I get RemoveBlend now?

cu, grua

[violao]

Originally posted by grua
http://bossanovaguitar.com/video/ ist down.
Up again. Problems with ISP. Sorry.