Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 13th November 2017, 17:59   #1  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
ResampleHQ causes weird bands?

I wanted to compare the output of resamplehq and standard bicubic resizing.

Downloaded resamplehq for VS here:
https://gist.github.com/4re/64642122e359c37543fe

I added parameters a1 and a2 to the function so that I could utilize the b and c values of Bicubic resizing.

First I tested comparing this set:
Code:
org = core.fmtc.bitdepth(clp, bits=16)
clp = core.resize.Bicubic(org, width=1280, height=688, filter_param_a=-0.7, filter_param_b=0.35)
clp2 = core.resize.Bicubic(org, width=1280, height=688, filter_param_a=-0.4, filter_param_b=0.2)

planes = [0]
expr = ('{median} x + y -').format(median=1<<(clp.format.bits_per_sample-1))
expr = [(i in planes) * expr for i in range(1)]
result = core.std.Expr([clp, clp2], expr)
result = core.std.Levels(result, min_in=125*256, max_in=132*256, min_out=0, max_out=255*256, planes=0)
It created a nice comparison:


Then I replaced the second clip by clp2 = rhq.resamplehq(org, width=1280, height=688, a1=-0.4, a2=0.2)

This created some strange banding:


Is this an issue of fmtconv, which resamplehq uses extensively, or something else? Is there some other alternative to try to do gamma aware resizing?
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...

Last edited by Boulder; 13th November 2017 at 18:19.
Boulder is offline   Reply With Quote
Old 13th November 2017, 19:49   #2  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
The problem seems to disappear if I resize to 1280x720. I probably should report this in the fmtconv thread, although I don't know if cretindesalpes is active anymore.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 14th November 2017, 15:03   #3  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: void
Posts: 2,633
try with floating point precision
feisty2 is offline   Reply With Quote
Old 14th November 2017, 15:30   #4  |  Link
Are_
Registered User
 
Join Date: Jun 2012
Location: Ibiza, Spain
Posts: 321
I already tested with it and bands disappeared, but is it normal for them to not be there just by changing the resolution while maintaining 16 bit integer precision? Does zimg use an internal float precision when dealing with 16bit inputs?

EDIT: And yes, this is a fmtc issue because you can see it when you use it raw too. Gamma aware resizing just makes it more obvious.

Last edited by Are_; 14th November 2017 at 15:33.
Are_ is offline   Reply With Quote
Old 14th November 2017, 15:38   #5  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
So for now, the performance-wise best option would be to use the internal zimg-based resizer inside resamplehq. The difference is not really visible to the eye as I always compare the original and downsized&reupscaled output and I've not noticed anything weird.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 14th November 2017, 16:34   #6  |  Link
Are_
Registered User
 
Join Date: Jun 2012
Location: Ibiza, Spain
Posts: 321
In the past when dealing with 16bit inputs fmtc was noticeable faster, that and being nicer when wanting to deal with many options made me not use zimg. Did you benchmark to make sure it is faster -float vs 16bit?
Are_ is offline   Reply With Quote
Old 14th November 2017, 19:19   #7  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
The speed difference between the resize methods inside resamplehq seem to be negligible when encoding to x265, which is the bottleneck anyway. The funny thing is that I also tried "core.resize.Bicubic(clp, width=1280, height=688, filter_param_a=-0.6, filter_param_b=0.3)" without anything else and it was just about as fast/slow as using resamplehq.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 14th November 2017, 19:29   #8  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
Quote:
Originally Posted by Stephen R. Savage View Post
Code:
orig_format = clip.format.id
tmp_format = vs.RGBH if is_cpu_ivy_bridge_or_later() else vs.RGBS

clip = core.resize.Bicubic(clip, format=tmp_format, transfer='linear')
clip = core.resize.Bicubic(clip, width=w, height=h)
clip = core.resize.Bicubic(clip, format=orig_format, transfer='709')
In fact, it should be faster than fmtconv, because it creates fewer temporary images. According to the link, the fmtconv equivalent uses 8 temporary images.
Trying that one gives an odd error:

Code:
 in 
clp = core.resize.Bicubic(clp, format=tmp_format, transfer='linear')
File "src\cython\vapoursynth.pyx", line 1702, in vapoursynth.Function.__call__ (src\cython\vapoursynth.c:34634)
File "src\cython\vapoursynth.pyx", line 527, in vapoursynth.typedDictToMap (src\cython\vapoursynth.c:12806)
ValueError: invalid literal for int() with base 10: 'linear'
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 14th November 2017, 23:45   #9  |  Link
Are_
Registered User
 
Join Date: Jun 2012
Location: Ibiza, Spain
Posts: 321
Boulder, if you want to test stuff I updated the gist with a version that makes use of zimg resizer. Not really much tested but it should work.
Are_ is offline   Reply With Quote
Old 15th November 2017, 20:27   #10  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
Thanks, will give that a go. I see that you have also included a descaler which could also prove to be useful.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 25th November 2017, 20:02   #11  |  Link
Are_
Registered User
 
Join Date: Jun 2012
Location: Ibiza, Spain
Posts: 321
Well, I went ahead and tested/fixed it myself. It is roughly 50% faster than fmtc version with higher quality because it uses single float instead 16-bit integer. Overall a nice finding.
Are_ is offline   Reply With Quote
Old 25th November 2017, 20:19   #12  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
Sorry, forgot to report that I noticed no ill effects. Were there any critical changes in the last version compared to the first fixed one?
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 26th November 2017, 20:33   #13  |  Link
Are_
Registered User
 
Join Date: Jun 2012
Location: Ibiza, Spain
Posts: 321
Nothing that I can remember, the range_in value was not used at some point, but that would be something you will notice if went wrong in your encode.
Are_ is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:51.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.