Best deblocker for heavily compressed video?

[feisty2]

The concept is, the block matching idea is great, and there're many ways of implementing it to a specific algorithm
NLMeans does it pixel-wise, so it performs great on high frequency components, as they are sparse in time domain and good for refining
BM3D does it block-wise, filtering in frequency domain, so non-high frequency components (sparse in frequency domain) resulted in better quality than NLMeans and high frequency components are inferior than NLMeans and good for general cleaning
MDeGrain does it block-wise, filtering in time domain, brutally blends all matched blocks and blurs everything out but also removes most of the crap, so it works good as the pre-filter for other later finer noise estimations

Oyster simply gets the best parts of all 3 together.

[MysteryX]

If you were to not depend on existing components but were to write a DLL yourself, probably great optimizations and shortcuts would be possible. You'd have a lot more control over the details of the implementations.

[feisty2]

I ported the floating point MVTools my very self, which is an unpleasant experience as my major is cosmic physics not computer science, and I even took some extra C++ programming classes for it..
BM3D has been SIMD optimized
knlmeans works on gpu
End of story

[MysteryX]

How about a pre-processor to determine the best processing path for each pixel? NNEDI3's pre-processor is an example of that.

How about 16-bit or 32-bit UINT processing instead of float?


You're right that using both KNLMeans and SMDegrain works much better than just one. Unfortunately, BM3D isn't available in AviSynth...

[real.finder]

fine
https://mega.nz/#!qh13jYrC!6o17_1yw53i3MVIhNXjgCAehLDYIEUrZJuklNatQTnI

http://i.imgur.com/1hLzh3o.png

frame 7


ivtc
DeHaloH()
RemoveDirtMC_SE(gpu=false,twopass=false,clmode=0)
funkydeblock(quant=10)
mc_spudsmod(frames=3,starfield=true,chroma=true)
dfttestMC()
GradFun3()
ConvertToY8()

[feisty2]

How about a pre-processor to determine the best processing path for each pixel? NNEDI3's pre-processor is an example of that.

How about 16-bit or 32-bit UINT processing instead of float?


You're right that using both KNLMeans and SMDegrain works much better than just one. Unfortunately, BM3D isn't available in AviSynth...

I never said I'll make oyster work on avisynth, I'm done with avisynth
and floats are irreplaceable as they are more mathmatically correct (=better quality), floats will (probably) never overflow, and integers will, you see floats don't even need no clamping at all and integers need it bad. And clamping introduces mathmatical errors.
The pre screener of nnedi3 is a neural network algorithm, entirely different systems, not possible, again

[MysteryX]

Oh, you wrote bm3d? ok

What's the difference between Oyster and BM3D?

[feisty2]

http://i.imgur.com/1hLzh3o.png

frame 7


ivtc
DeHaloH()
RemoveDirtMC_SE(gpu=false,twopass=false,clmode=0)
funkydeblock(quant=10)
mc_spudsmod(frames=3,starfield=true,chroma=true)
dfttestMC()
GradFun3()
ConvertToY8()


that frame is so shitty and far beyond saving, I was just placing it to prove that hello_hello's statement was not completely right, by any means trying to fix that frame will end up with similar results, you raise the denoising strength up till the blocks are gone, how about a fairer example, the one at #39, I'll upload the sample soon

EDIT1: I do notice some differences now.

is there some kind of sharpening in your script? Britney's hair looks sharper in your result but also a bit cartoonized, which is very bad from my personal perspective of view, I hate algorithms that tend to cartoonize photos.

your result actually has an even more severe detail loss than mine, take a look at the flecks on her boobs, my result weakened them and yours removed them completely

[real.finder]

that frame is so shitty and far beyond saving, I was just placing it to prove that hello_hello's statement was not completely right, by any means trying to fix that frame will end up with similar results, you raise the denoising strength up till the blocks are gone, how about a fairer example, the one at #39, I'll upload the sample soon

EDIT1: I do notice some differences now.

is there some kind of sharpening in your script? Britney's hair looks sharper in your result but also a bit cartoonized, which is very bad from my personal perspective of view, I hate algorithms that tend to cartoonize photos.

your result actually has an even more severe detail loss than mine, take a look at the flecks on her boobs, my result weakened them and yours removed them completely

this just a quick test, yes mc_spudsmod use contra sharpening or LimitedSharpen, you can turn it off http://avisynth.nl/index.php/MC_Spuds

about detail loss, yours has others (big things like shadows and lines, etc...)

[real.finder]

http://i.imgur.com/qyIKpPX.png


IVTC
DeHaloH()
funkydeblock(quant=10)
mc_spudsmod(frames=3,starfield=true,chroma=true,sharpp=0)
dfttestMC()
GradFun3()
ConvertToY8()


and mfilter in SMDegrain can be used in this case to not loss details in static scenes

[Reel.Deel]

Oh, you wrote bm3d? ok

What's the difference between Oyster and BM3D?

Oyster is just a script, it includes these main functions Destaircase(), Deringing(), and Deblocking(). Deblocking uses BM3D, as the main denoiser. For more information on BM3D see here: http://forum.doom9.org/showthread.php?t=172172

[feisty2]

http://i.imgur.com/qyIKpPX.png


IVTC
DeHaloH()
funkydeblock(quant=10)
mc_spudsmod(frames=3,starfield=true,chroma=true,sharpp=0)
dfttestMC()
GradFun3()
ConvertToY8()


and mfilter in SMDegrain can be used in this case to not loss details in static scenes

your first result looks better to me, this one has some obvious residual blocking...

anyways, I found some algorithmic similarities between our methods, since I'm not using NLMeans in "Deblocking" (cuz using pixel-wise algorithm to deblock is insane)

yours -----> mine
DeHaloH() -----> X
funkydeblock(quant=10) ------>X
mc_spudsmod ------> MDegrain
dfttestMC() ------->BM3D (both are block matching enhanced frequency filter, but BM3D is superior)
GradFun3() ------> X

[real.finder]

your first result looks better to me, this one has some obvious residual blocking...

anyways, I found some algorithmic similarities between our methods, since I'm not using NLMeans in "Deblocking" (cuz using pixel-wise algorithm to deblock is insane)

yours -----> mine
DeHaloH() -----> X
funkydeblock(quant=10) ------>X
mc_spudsmod ------> MDegrain
dfttestMC() ------->BM3D (both are block matching enhanced frequency filter, but BM3D is superior)
GradFun3() ------> X

you forget Speed factor :devil:

more than 1 fps in worst case ------> 0.0xfps (http://forum.doom9.org/showpost.php?p=1770885&postcount=7)

the RemoveDirtMC_SE(gpu=false,twopass=false,clmode=0) was did good job but case the remove of "the flecks on her boobs"

If I encode this I will did more tweaking of course, this just quick tests as I say

[feisty2]

you forget Speed factor :devil:

more than 1 fps in worst case ------> 0.0xfps (http://forum.doom9.org/showpost.php?p=1770885&postcount=7)

the RemoveDirtMC_SE(gpu=false,twopass=false,clmode=0) was did good job but case the remove of "the flecks on her boobs"

If I encode this I will did more tweaking of course, this just quick tests as I say

well, BM3D is way more complicated than dfttestmc, and that extra complexity brings better quality (and lower speed), you might find there's no huge difference in this extremely shitty case, but BM3D generally works a LOT better, looking for another example to prove my point...

[real.finder]

I edit Deblock_QED_MT2 http://pastebin.com/MuajpPfm

will play with it in free time

[MysteryX]

I edit Deblock_QED_MT2 http://pastebin.com/MuajpPfm

will play with it in free time
I just tried this on my heavily blocked videos compared to SMDegrain(thsad=2000, contrasharp=true).

For me it's just adding distortions and ghosting effects.

First I run KNLMeans, then SuperResXBR to double the image size, then I perform this just before Interframe and running a frame double again. So the difference is getting extrapolated and is more visible.

Deblock_QED(100, 120)
https://s32.postimg.org/vqjt1g2dt/Deblock_QED_MT2.png (https://postimg.org/image/vqjt1g2dt/)

SMDegrain(thsad=2000, contrasharp=true)
https://s32.postimg.org/h8mltgb2p/SMDegrain.png (https://postimg.org/image/h8mltgb2p/)

[hello_hello]

Aside from MysteryX's examples, was it ever decided why we're comparing black and white deblocking?

[feisty2]

Aside from MysteryX's examples, was it ever decided why we're comparing black and white deblocking?

Because luma is the dominant factor of quality

[feisty2]

@real.finder
try this one, blocky frame with sharp edges, see what you can do in avisynth
http://i.imgur.com/rsWKcT2.png

ref = Oyster.Basic(clp, Oyster.Super(clp))
clp = Oyster.Destaircase(clp, ref, block_step=2, sigma=6.4, thr=2/256, elast=1/256)
clp = Oyster.Deringing(clp, ref, block_step=2, lowpass=12)

http://i.imgur.com/lRnLoo0.png

sample: uncompressed 720x480 Y8 raw, frame 26
https://mega.nz/#!jktxFQqS!ZVRYFjREMuIAONtSg6k6hQp7j61nCi2QcAqwGdwft1U

[real.finder]

@real.finder
try this one, blocky frame with sharp edges, see what you can do in avisynth
http://i.imgur.com/rsWKcT2.png

ref = Oyster.Basic(clp, Oyster.Super(clp))
clp = Oyster.Destaircase(clp, ref, block_step=2, sigma=6.4, thr=2/256, elast=1/256)
clp = Oyster.Deringing(clp, ref, block_step=2, lowpass=12)

http://i.imgur.com/lRnLoo0.png

sample: uncompressed 720x480 Y8 raw, frame 26
https://mega.nz/#!jktxFQqS!ZVRYFjREMuIAONtSg6k6hQp7j61nCi2QcAqwGdwft1U

http://i.imgur.com/IBAGl2Z.png

with this quick test

RawSource("sample.rgb", 720, 480, "y8")
ConvertToYV12()
o=last
DeHaloH()
Deblock_QED(exdeblock1="funkydeblock(quant=10)",exdeblock2="RemoveDirtMC_SE(gpu=false,twopass=false,clmode=0)")
dfttestMC(pp=blur(1.53))
deb=last
o
SMDegrain(mfilter=deb,prefilter=3,lsb=true)

[feisty2]

so I appended another low frequency de-noising to match the result above

ref = Oyster.Basic(clp, Oyster.Super(clp))
clp = Oyster.Destaircase(clp, ref, block_step=2, sigma=6.4, thr=2/256, elast=1/256)
clp = Oyster.Deringing(clp, ref, block_step=2, lowpass=12)
ref = core.bm3d.VBasic(clp,sigma=12,radius=6,block_size=8, block_step=2, group_size=32, bm_range=24, bm_step=1, ps_num=2, ps_range=8, ps_step=1).bm3d.VAggregate(6,1)
low = core.bm3d.VFinal(clp,ref,sigma=12,radius=6,block_size=8, block_step=2, group_size=32, bm_range=24, bm_step=1, ps_num=2, ps_range=8, ps_step=1).bm3d.VAggregate(6,1)
clp = Oyster.helpers.freq_merge(low,clp,8)

http://i.imgur.com/tYSnBvb.png

anyways, I'm not seeing any significant difference, maybe the theoretically inferior dfttestmc is enough for blocking?
that's an interesting one

and how's the speed going, Oyster runs at 0.08fps here (i7-4790k)

[real.finder]

so I appended another low frequency de-noising to match the result above

ref = Oyster.Basic(clp, Oyster.Super(clp))
clp = Oyster.Destaircase(clp, ref, block_step=2, sigma=6.4, thr=2/256, elast=1/256)
clp = Oyster.Deringing(clp, ref, block_step=2, lowpass=12)
ref = core.bm3d.VBasic(clp,sigma=12,radius=6,block_size=8, block_step=2, group_size=32, bm_range=24, bm_step=1, ps_num=2, ps_range=8, ps_step=1).bm3d.VAggregate(6,1)
low = core.bm3d.VFinal(clp,ref,sigma=12,radius=6,block_size=8, block_step=2, group_size=32, bm_range=24, bm_step=1, ps_num=2, ps_range=8, ps_step=1).bm3d.VAggregate(6,1)
clp = Oyster.helpers.freq_merge(low,clp,8)

http://i.imgur.com/tYSnBvb.png

anyways, I'm not seeing any significant difference, maybe the theoretically inferior dfttestmc is enough for blocking?
that's an interesting one

and how's the speed going, Oyster runs at 0.08fps here (i7-4790k)

3 fps on this http://i.imgur.com/911VwD1.png

with no mt at all (even avstp)

[feisty2]

3 fps on this http://i.imgur.com/911VwD1.png

with no mt at all (even avstp)

i7 x980, that's a much better CPU....
dunno how to compare the speed now..

[MysteryX]

How about CPU Benchmark (https://www.cpubenchmark.net/)?

[feisty2]

@real.finder
How about mosquito noise, any idea to slay that in avisynth?
this sample suffers from MPEG2-style mosquito noise, I'll post another sample that suffers from H264-style mosquito noise if you could work this one out

http://i.imgur.com/Paim7CR.png

ref = Oyster.Basic(clp, Oyster.Super(clp))
clp = Oyster.Destaircase(clp, ref, block_step=2)
clp = Oyster.Deringing(clp, ref, block_step=2)

http://i.imgur.com/dDjdWdR.png

sample: uncompressed 720x576 Y8 raw, frame 12
https://mega.nz/#!31lniDDL!FgM8h9IaZxWLs2Fw3gxMgkHDfQmOe6M-xHqcKerIfbw

[MysteryX]

RawSource("sample.rgb", 720, 480, "y8")
ConvertToYV12()
o=last
DeHaloH()
Deblock_QED(exdeblock1="funkydeblock(quant=10)",exdeblock2="RemoveDirtMC_SE(gpu=false,twopass=false,clmode=0)")
dfttestMC(pp=blur(1.53))
deb=last
o
SMDegrain(mfilter=deb,prefilter=3,lsb=true)

I wanted to give it a try but... damn this thing uses every single plugin ever developed!!

I needed to add this to my existing script to get it to work

Import(P+"deblock_qed_mt2.avsi")
LoadPlugin(P+"deblock.dll")
LoadPlugin(P+"TBilateral.dll")
LoadPlugin(P+"SmoothUV.dll")
Import(P+"DeHaloHmod.avsi")
LoadPlugin(P+"deen.dll")
Import(P+"FunkyDeblock.avsi")
LoadPlugin(P+"dgdecode.dll")
Import(P+"removedirtmc_se.avsi")
LoadPlugin(P+"FFT3DFilter.dll")
LoadPlugin(P+"RemoveGrainS.dll")
LoadPlugin(P+"RepairS.dll")
LoadPlugin(P+"FluxSmooth.dll")
LoadPlugin(P+"RemoveDirtSSE2.dll")
LoadPlugin(P+"DctFilter.dll")
LoadPlugin(P+"dfttest.dll")
Import(P+"dfttestmc.avsi")


Here's the output when integrated into my more complex script after the first frame double, just before InterFrame and a 2nd frame double, which extrapolates the details and makes them more visible.

SMDegrain(thsad=2000, contrasharp=true)
https://s31.postimg.org/yn343i9fr/everything.png (https://postimg.org/image/yn343i9fr/)

Everything on Earth and from unknown Universes
https://s31.postimg.org/ba52kztc7/SMDegrain.png (https://postimg.org/image/ba52kztc7/)

*THAT* is a "throw it all on the wall and see what sticks" approach

[real.finder]

@real.finder
How about mosquito noise, any idea to slay that in avisynth?
this sample suffers from MPEG2-style mosquito noise, I'll post another sample that suffers from H264-style mosquito noise if you could work this one out

http://i.imgur.com/Paim7CR.png

ref = Oyster.Basic(clp, Oyster.Super(clp))
clp = Oyster.Destaircase(clp, ref, block_step=2)
clp = Oyster.Deringing(clp, ref, block_step=2)

http://i.imgur.com/dDjdWdR.png

sample: uncompressed 720x576 Y8 raw, frame 12
https://mega.nz/#!31lniDDL!FgM8h9IaZxWLs2Fw3gxMgkHDfQmOe6M-xHqcKerIfbw

http://i.imgur.com/wsuknBt.png


nnedi3_rpow2(rfactor=2).DeHaloH(5,dr_str=40,threshold=200).PointResize(Width(),Height())
SMDegrain(3,110,refinemotion=true,truemotion=false,mfilter=admfilter(),prefilter=3,lsb=true)

[feisty2]

I knew it.
it just won't always be so lucky, your result is a lot worse this time
http://i.imgur.com/RKnDcgj.png
there's a lot of residual mosquito noise (marked within blue boxouts)
and more detail loss/less sharp (marked within the red boxout)

[real.finder]

and more detail loss/less sharp (marked within the red boxout)

your
http://i.imgur.com/ooQKNZu.png

my
http://i.imgur.com/onFLEuy.png

anyway, this satisfactory result for fast old deen that used in dehaloh

I think if I put xNLMeans in the game it will did better result, but if any script speed less than 1 fps I will change it to something faster, waste a lot of time in lossy source that will end in another lossy encode makes no sense

so yeah good luck in VapourSynth and coding in python :)

[feisty2]

I didn't tweak the parameters, all default values
First, that looks more like residual low frequency noise than detail to me, and assume that's the actually detail, this could be easily fixed by using a larger lowpass value

And bring xnlmeans to the game, quality first. more about speed when the quality factor is settled

[feisty2]

And PS, I do not do ANY kind of lossy compression, got an OCD about quality, lossy compression is toxic

[feisty2]

and H264 compression artifacts

http://i.imgur.com/m1oZT1x.png

ref = Oyster.Basic(clp, Oyster.Super(clp))
clp = Oyster.Destaircase(clp, ref, sigma=24.0, block_step=4, lowpass=4)
clp = Oyster.Deringing(clp, ref, sigma=24.0, h=10.8, block_step=4, lowpass=4)

http://i.imgur.com/IOr8RXz.png

h264 generally generates less artifacts than mpeg2, but, once the compression artifacts emerges in an h264 vid, it will be very hard to remove,
way harder than mpeg2 artifacts, you'll know that if you ever tried to remove compression artifacts from some crappy h264 vid...

[MysteryX]

floats are irreplaceable as they are more mathmatically correct (=better quality), floats will (probably) never overflow, and integers will, you see floats don't even need no clamping at all and integers need it bad. And clamping introduces mathmatical errors.
When working with YUV data the 16-235 TV range standard was adopted to prevent such overflows. Thus, the problem of overflows is already taken care of.

[feisty2]

When working with YUV data the 16-235 TV range standard was adopted to prevent such overflows. Thus, the problem of overflows is already taken care of.

yeah, the "math" your learned is obviously different from the one I did, maybe, say 235x2 falls in the range of [0, 255] by your definition of "math"

anyways, a short piece of the nnedi3 source code

template <typename PixelType, typename TempType>
int32_t processLine0_C(const uint8_t *tempu, int width, uint8_t *dstp8, const uint8_t *src3p8, const int src_pitch, const int max_value, const int chroma) {
PixelType *dstp = (PixelType *)dstp8;
const PixelType *src3p = (const PixelType *)src3p8;

TempType minimum = 0;
TempType maximum = max_value - 1;
// Technically the -1 is only needed for 8 and 16 bit input.

int count = 0;
for (int x = 0; x < width; ++x) {
if (tempu[x]) {
TempType tmp = 19 * (src3p[x + src_pitch * 2] + src3p[x + src_pitch * 4]) - 3 * (src3p[x] + src3p[x + src_pitch * 6]);
if (std::is_same<TempType, float>::value) {
dstp[x] = tmp / 32;
} else {
tmp = (tmp + 16) / 32;
dstp[x] = std::max(std::min(tmp, maximum), minimum);
}
} else {
memset(dstp + x, 255, sizeof(PixelType));
++count;
}
}
return count;
}


and clamping is evil

[pandy]

When working with YUV data the 16-235 TV range standard was adopted to prevent such overflows. Thus, the problem of overflows is already taken care of.

This is not entirely true as 16 is black analogue signal level (hypotehetical 0 Volts) and 235 is white analogue signal level (hypothetical 0.7 Volt) but all video standards allow to have under and over shoots (due nature of analog signal - real analogue unless design in particular way tend to ring at sharp signal change - every transition suffer from under and overshoots).

I never said I'll make oyster work on avisynth, I'm done with avisynth
and floats are irreplaceable as they are more mathmatically correct (=better quality), floats will (probably) never overflow, and integers will, you see floats don't even need no clamping at all and integers need it bad. And clamping introduces mathmatical errors.
The pre screener of nnedi3 is a neural network algorithm, entirely different systems, not possible, again

And floats are generally not better than integer - it is common misconception that float will give more freedom to developer than integer, sometimes they may give false confidentiality and under some real signal aspects provide worse results than properly used integers (classical example are high order IIR filters where usually integers provide better results than float with same word length).

Clamping or arithmetic with saturation works like normal reallife systems - signal level can't be higher than power supply providing power to amplifier.
In 32 bit float accuracy is lower than in properly used 32 bit INT however overall dynamic for float is higher than for INT.

[feisty2]

In 32 bit float accuracy is lower than in properly used 32 bit INT however overall dynamic for float is higher than for INT.

I'd be glad to give uint32_t a try if such pixel format does exist, and integers give a highest precision of 16 bpc among all popular pixel formats, which leaves single precision floating point the existing highest precision pixel format (23bits vs 16bits) among all widely used image formats.


And floats are generally not better than integer - it is common misconception that float will give more freedom to developer than integer, sometimes they may give false confidentiality and under some real signal aspects provide worse results than properly used integers (classical example are high order IIR filters where usually integers provide better results than float with same word length).

Clamping or arithmetic with saturation works like normal reallife systems - signal level can't be higher than power supply providing power to amplifier.

clamp the final output whatever you like, any kind of clamping MUST NOT happen during the process to make sure the intermediate math works correctly, integers don't give a shit if it's the intermediate math or not, it just gotta be clamped.
https://helpx.adobe.com/after-effects/using/color-basics.html




High dynamic range (HDR), 32-bpc floating-point color values can represent brightness levels much greater than white, including objects as bright as a flame or the Sun.


https://helpx.adobe.com/after-effects/using/color-basics/_jcr_content/main-pars/image_1.img.png/co_03.png
Glow effect and Gaussian Blur effect applied to image in 32-bpc project (left) and 16-bpc project (right)




Set the project color depth to 32 bpc to work with HDR footage or to work with over-range values—values above 1.0 (white) that aren’t supported in 8- or 16-bpc mode. Over-range values preserve the intensity of highlights, which is as useful for synthetic effects such as lights, blurs, and glows as it is for working with HDR footage. The headroom provided by working in 32 bpc prevents many kinds of data loss during operations such as color correction and color profile conversion.

Even if you’re using 8-bpc footage and are creating movies in 8-bpc formats, you can obtain better results by having the project color depth set to 16 bpc or 32 bpc. Working in a higher bit depth provides higher precision for calculations and greatly reduces quantization artifacts, such as banding in gradients.


edit:

Additional resources about high dynamic range color



Chris Meyer explains what floating-point, 32-bpc, HDR color is good for in a video overview on the Lynda.com website (http://movielibrary.lynda.com/html/modPage.asp?ID=483).

Kert Gartner provides some visual examples and a brief explanation on his VFX Haiku website (http://vfxhaiku.com/2009/10/intro-to-32bpc-in-after-effects/)that demonstrate the benefits of working with 32-bpc color.

Andrew Kramer provides a video tutorial on his Video Copilot website (http://www.videocopilot.net/tutorials/earthquake_with_32bpc/)in which he demonstrates the advantages of using 32-bpc color with motion blur.

On his fnord website (http://fnordware.blogspot.com/2007/11/hdr-tone-mapping-using-film-profiles.html), Brendan Bolles explains how to use the Color Profile Converter effect and film color profiles to adjust colors and perform tone mapping to make an HDR image appear as if it were shot on motion picture film.


clamping is evil, yet again

[geometer]

I love the results I see.

* someone might look at the algorithms in a more holistic way, like the guy who invented FFT by turning the sort order inside out. some numeric math guys might have a highly efficient idea.

* someone with money might want to throw it at one of the very big processing farms, to get done some videos he loves.

* parallel computing is still growing, and quantum computing will be a "big bang" in that.

In just a few years, "oystering" might be the most popular term in the denoising business. :)

---
BTW
the 3:2 pulldown is solved within the program, or with the usual preprocessing? (decomb telecide, best match, use chroma to decide... / 2 from 5 frames the chroma is lagging behind. chroma fields different than luma fields.)

[pandy]

I'd be glad to give uint32_t a try if such pixel format does exist, and integers give a highest precision of 16 bpc among all popular pixel formats, which leaves single precision floating point the existing highest precision pixel format (23bits vs 16bits) among all widely used image formats.


We can't mix accuracy with dynamics range.
With same number of bits INT will always provide higher accuracy than float but overall dynamic range will be lower.
Nowadays floats are almost same speed as int - sometimes you have dedicated float path when int path is shared with regular data - as such float can be faster than int even if they are slower by sequence of operations (mantissa normalization/denormalization is step unknown in int world).
I know that lot of people think that float is like "fire and forget" but this is totally untrue - floats suffer from own problems (https://en.wikipedia.org/wiki/Floating_point#Accuracy_problems ) and for well developed algorithms int behave more stable...


clamp the final output whatever you like, any kind of clamping MUST NOT happen during the process to make sure the intermediate math works correctly, integers don't give a shit if it's the intermediate math or not, it just gotta be clamped.
https://helpx.adobe.com/after-effects/using/color-basics.html
edit:
clamping is evil, yet again

Fully agree, clamping intermediates is wrong but this mean that your algorithm is simply wrong...
In audio world going for 48 - 56 bits is nothing unusual and intermediate can be stored with even 96 bits - with proper algorithm you have accuracy and real life signal dynamics (assuming that your signal resolution is 24 - 32 bits).
Float are useful when code must be frequently changed and there is no time (or other reason) for proper testing.
So i would say for real pro int are OK and they are preferred over float.

[feisty2]

We can't mix accuracy with dynamics range.
With same number of bits INT will always provide higher accuracy than float but overall dynamic range will be lower.

fine, like I said before, I would love to try uint32_t(same size as float) if such pixel format does exist and unfortunately it does not, the highest precision integer pixel type out there is uint16_t, and float offers 23bits of precision, 23>16, float wins obviously.


I know that lot of people think that float is like "fire and forget" but this is totally untrue - floats suffer from own problems (https://en.wikipedia.org/wiki/Floati...uracy_problems ) and for well developed algorithms int behave more stable...


floats lose precision as the calculation goes on, everyone knows that, 23bits is way more precise than 16bits still, and floating point math works at 80bits long double precision under x86 so that's no big deal.

besides, integers lose precision as well,

//c++
[]() {auto num = 3; return num / 2 * 2 == num;}();

mathematically it should be true, but anyone that knows any basic stuff about programming will know that's gonna be false


Fully agree, clamping intermediates is wrong but this mean that your algorithm is simply wrong...
you could use some large data types like uint64_t and stuff to store intermediate things to avoid clamping, but that won't always work, you will have to write all the intermediate data to some video frame when the algorithm is complex enough, like that nnedi3 code overhead, there are steps of different interpolations going on and you could merge them all within one single function and avoid temporary video frames and thus avoid clamping, and sadly human programmers won't do things like that


In audio world going for 48 - 56 bits is nothing unusual and intermediate can be stored with even 96 bits - with proper algorithm you have accuracy and real life signal dynamics (assuming that your signal resolution is 24 - 32 bits).

I don't see how that's any better than float, I would just take boost and I then got floating points at ANY precision if, the calculation requires a lot of precision, what's the point of complicating your algorithm by using only integers?

[feisty2]

@johnmeyer
if you're still interested, I just finished writing the doc for oyster
https://github.com/IFeelBloated/Oyster/blob/master/README.md

[johnmeyer]

Thanks! I skimmed through it, but will go back and read it thoroughly as time permits.

[feisty2]

Oh, and I did quite a few modifications to oyster, so you'll have to redownload oyster.py and replace the old one if you have played with it for a while

[TCmullet]

I have hours of footage that looks horrible like this. Looks like someone encoded 1080p30 footage at a bitrate meant for 240p. HORRIBLE. But I am desperate to salvage it all. I've tried hours and hours of experimenting with lots of tools, including some (but not all) of the ones mentioned. Everything seems to work at a "micro" level and never at a "big block" level. I feel I need someone who knows what they're doing to SEE a sample. Can ANYTHING be done to improve it at all??

[raffriff42]

Can't see the attachment, but have a look at this, it can be amazing This script is surprisingly effective :eek: It even works on blocky gradients, strangely enough, even though the original author designed it for cartoons.

Honorable mention: DGDecode/BlindPP

EDIT I see your attachment now. Ouch! I can't help you.

[feisty2]

I have hours of footage that looks horrible like this. Looks like someone encoded 1080p30 footage at a bitrate meant for 240p. HORRIBLE. But I am desperate to salvage it all. I've tried hours and hours of experimenting with lots of tools, including some (but not all) of the ones mentioned. Everything seems to work at a "micro" level and never at a "big block" level. I feel I need someone who knows what they're doing to SEE a sample. Can ANYTHING be done to improve it at all??

lost cause
there's nothing you can do about crap like that

[johnmeyer]

Feisty is correct that your video is never going to look very good, but you can certainly smooth out the blocks a bit. Take a look at this thread where I wanted to restore the only copy of a live story telling performance by the late Geoffrey Lewis:

Ideas for restoring early generation (really old) digital video (http://forum.doom9.org/showthread.php?t=163014)

I not only had to smooth out the blockiness in a first-generation (early 1990s) 160x120 digital video, but also wanted to increase the frame rate from 12 fps to 30 fps. You can compare the before & after (both links are still live). Also, I received a lot of really useful advice and suggestions and even if you don't do what I did, perhaps some of the other ideas will be more appropriate for your current video.