Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 16th April 2010, 08:54   #341  |  Link
Hiritsuki
Novice of AVS
 
Join Date: Oct 2009
Posts: 156
I found a filter this can help someone video non-waste bitrate, it's Dedup to using to making VFR video.
and it's no x64 ver.
__________________
My PC
Hiritsuki is offline   Reply With Quote
Old 16th April 2010, 13:38   #342  |  Link
Stephen R. Savage
Registered User
 
Stephen R. Savage's Avatar
 
Join Date: Nov 2009
Posts: 327
Quote:
Originally Posted by Andrey /MAG/ View Post
I have done frame-by-frame comparing with 'subtract' and found that difference still present.
This was addressed on page two or three, and is the result of SoftWire performing rounding differently from JoshyD's native assembly.

Quote:
Originally Posted by Hiritsuki View Post
I found a filter this can help someone video non-waste bitrate, it's Dedup to using to making VFR video.
and it's no x64 ver.
I've found that Dedup doesn't contribute significantly at all to compression ratio.

Last edited by Stephen R. Savage; 16th April 2010 at 13:40.
Stephen R. Savage is offline   Reply With Quote
Old 16th April 2010, 14:08   #343  |  Link
Hiritsuki
Novice of AVS
 
Join Date: Oct 2009
Posts: 156
Quote:
Originally Posted by Stephen R. Savage View Post
I've found that Dedup doesn't contribute significantly at all to compression ratio.
Oh, using big bitrate to compress video like anime ,dedup is good use.
__________________
My PC
Hiritsuki is offline   Reply With Quote
Old 16th April 2010, 15:45   #344  |  Link
Pho3n1x
Registered User
 
Join Date: Jun 2009
Posts: 2
i have a problem about the ffms2.dll from
Quote:
kemuri-_9's FFMS2 (The Fabulous FM Source 2)
when i open the avs in virtualdib64 it comes no funtion about ffvideosource.

my script is

Code:
loadplugin("C:\Program Files (x86)\AviSynth 2.5\plugins64\ffms2.dll")
ffvideosource("E:\Encode\0_2.mkv")
crop(0,22,-0,-22)
The same script with directshowsource works well but not with the ffms2.dll.
Pho3n1x is offline   Reply With Quote
Old 16th April 2010, 16:48   #345  |  Link
Zathor
Registered User
 
Join Date: Nov 2009
Posts: 2,405
Replace LoadPlugin with LoadCPlugin and everything should be fine.
Zathor is offline   Reply With Quote
Old 16th April 2010, 18:44   #346  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,364
Quote:
@Avs developers: I notice that resizing RGB images causes the hue/saturation/brightness to shift. Is this normal, and can it be fixed?
Does that also happen with the regular v2.58? Could you post a script which shows this?
Wilbert is offline   Reply With Quote
Old 16th April 2010, 21:19   #347  |  Link
Stephen R. Savage
Registered User
 
Stephen R. Savage's Avatar
 
Join Date: Nov 2009
Posts: 327
Quote:
Originally Posted by Wilbert View Post
Does that also happen with the regular v2.58? Could you post a script which shows this?
I can confirm this with 2.58 standard.

The script I used was:

Quote:
a = ImageSource("test.png")
b = Spline36Resize(a, 1280, 720).Spline36Resize(a.width, a.height)
Interleave(a, b)
I tested this with the following image (after decoding to PNG format):

http://img684.imageshack.us/img684/7910/testjc.png

After resizing, the image is noticeably darker:

http://img651.imageshack.us/img651/446/shiftcxifb.png

This also appears to happen with BicubicResize and LanczosResize, though it is a bit difficult to see with bicubic due to all the blur. Converting to YV12 first does not result in the same problem.
Stephen R. Savage is offline   Reply With Quote
Old 16th April 2010, 22:04   #348  |  Link
Hiritsuki
Novice of AVS
 
Join Date: Oct 2009
Posts: 156
Some anime had few stationary frame only music runing.
So, can anyone make this "dedup" to x64 ver. ? please ...
__________________
My PC
Hiritsuki is offline   Reply With Quote
Old 17th April 2010, 00:00   #349  |  Link
osgZach
Registered User
 
Join Date: Feb 2009
Location: USA
Posts: 676
You've already asked once. If someone has the time and energy they will eventually get to it. But you are currently the only one asking for it. Repeating your request will not make it happen faster, you'll just make people ignore you.
osgZach is offline   Reply With Quote
Old 17th April 2010, 00:38   #350  |  Link
Hiritsuki
Novice of AVS
 
Join Date: Oct 2009
Posts: 156
Quote:
Originally Posted by osgZach View Post
You've already asked once. If someone has the time and energy they will eventually get to it. But you are currently the only one asking for it. Repeating your request will not make it happen faster, you'll just make people ignore you.
Sorry about my post I just explain what that filter is useable
__________________
My PC
Hiritsuki is offline   Reply With Quote
Old 17th April 2010, 01:39   #351  |  Link
JoshyD
Registered User
 
Join Date: Feb 2010
Posts: 84
@Stephen

Ah, I know why it happens in both . . . the resize code for RGB is pretty much left stock by myself, just pulled out into an asm file and written for all local register usage. It's never been performance optimized by anyone, to my knowledge. RGB resize doesn't come up that often I guess?

Found the problem, as well, I think. I'm uploading a new build for you to run through: Here's the test build.

Slowly stepping through the interleave appears to be improved, but the original problem was subtle upon first inspection by my eyes. It's not completely gone . . .

32bit difference map:



64bit difference map:



Red means that the pixels are strictly different between adjacent frames of the image stream, using your script in VDub32 and VDub64. As you can see, it's better, but not perfect.

I've been playing around with the code seeing how much performance I could eek out of the core . . . let me know if anything breaks before I go update the first post.

For the devs, from the 2.58 source directly (and in 2.6 apparently):
Code:
// RGB24 is not recommended. 75% of all pixels are not aligned.
    int y = vi.height;
    int w = vi.width * 3;
    int fir_filter_size = pattern_luma[0];
    int* pattern_lumaP1 = pattern_luma+1 - fir_filter_size;
    static const __int64 xFF000000 = 0xFF000000;
    __asm {
	  push        ebx
      mov         esi, srcp
      mov         edi, dstp
      pxor        mm2, mm2
      movq        mm4, xFF000000
      align 16
    yloop24:
      xor         ecx, ecx
      mov         edx, pattern_lumaP1       ;cur - fir_filter_size
      align 16
    xloop24:
      mov         eax, fir_filter_size
      lea         edx, [edx+eax*4]          ;cur += fir_filter_size
      mov         ebx, [edx]
      lea         ebx, [ebx+ebx*2]          ;ebx = ofs = *cur * 3
      add         edx, 4                    ;cur++
      pxor        mm0, mm0                  ;btotal, gtotal
      pxor        mm1, mm1                  ;rtota
      lea         edx, [edx+eax*4]          ;cur += fir_filter_size
      add         ebx, esi                  ;ebx = srcp + ofs*3
      lea         eax, [eax+eax*2]          ;eax = a = fir_filter_size*3
      align 16
    aloop24:
      sub         edx, 4                    ;cur--
      sub         eax, 3
      movd        mm7, [ebx+eax]            ;mm7 = srcp[ofs+a] = 0|0|0|0|x|r|g|b
      punpcklbw   mm7, mm2                  ;mm7 = 0x|0r|0g|0b
      movq        mm6, mm7
      punpcklwd   mm7, mm2                  ;mm7 = 00|0g|00|0b
      punpckhwd   mm6, mm2                  ;mm6 = 00|0x|00|0r
      movd        mm5, [edx]                ;mm5 =    00|co (co = coefficient)
      packssdw    mm5, mm2
      punpckldq   mm5, mm5                  ;mm5 =    co|co
      pmaddwd     mm7, mm5                  ;mm7 =  g*co|b*co
      pmaddwd     mm6, mm5                  ;mm6 =  x*co|r*co
      paddd       mm0, mm7
      paddd       mm1, mm6
      jnz         aloop24
      pslld       mm0, 2
      pslld       mm1, 2                    ;compensate the fact that FPScale = 16384
      packuswb    mm0, mm1                  ;mm0 = x|_|r|_|g|_|b|_
      psrlw       mm0, 8                    ;mm0 = 0|x|0|r|0|g|0|b
      packuswb    mm0, mm2                  ;mm0 = 0|0|0|0|x|r|g|b
      pslld       mm0, 8
      psrld       mm0, 8                    ;mm0 = 0|0|0|0|0|r|g|b
      movd        mm3, [edi+ecx]            ;mm3 = 0|0|0|0|x|r|g|b (dst)
      pand        mm3, mm4                  ;mm3 = 0|0|0|0|x|0|0|0 (dst)
      por         mm3, mm0
      movd        [edi+ecx], mm3

      add         ecx, 3
      cmp         ecx, w
      jnz         xloop24

      add         esi, src_pitch
      add         edi, dst_pitch
      dec         y
      jnz         yloop24
      emms
	  pop         ebx
Put the rounder in those registers, also, specifically why not change:

Code:
movd        mm5, [edx]                ;mm5 =    00|co (co = coefficient)
packssdw    mm5, mm2
punpckldq   mm5, mm5                  ;mm5 =    co|co
to

Code:
movd        mm5, [edx]                ;mm5 =    00|co (cocoefficient)
pshufw       mm5, mm5,0xCCh
You're only interested in the low word anyhow, high double is zeroed on a movd.

Last edited by JoshyD; 17th April 2010 at 01:49.
JoshyD is offline   Reply With Quote
Old 17th April 2010, 02:21   #352  |  Link
Stephen R. Savage
Registered User
 
Stephen R. Savage's Avatar
 
Join Date: Nov 2009
Posts: 327
Good work! Let's hope this update gets included in the main Avisynth trunk. I don't see any red on flat areas anymore, so I'm guessing it's about as fixed as it can get. Out of curiosity, what was the source of the bug?

Also, I hate to pester, but any chance of updating MaskTools to v2.0a37? The changes should be trivial.

Edit: Also, what did you use to make the difference map?
Edit2: Also, any chance you can port the Average() plugin (or write your own)?

Last edited by Stephen R. Savage; 17th April 2010 at 03:44.
Stephen R. Savage is offline   Reply With Quote
Old 17th April 2010, 03:04   #353  |  Link
JoshyD
Registered User
 
Join Date: Feb 2010
Posts: 84
@Stephen
The source of the problem is those two lines of highlighted inline ASM for 2.58, or in my case similar lines of code living in my .asm file of a comparable function. Basically, the resizers are working with an intent of doing the calculation of the new pixel value as a floating point value, and then converting it back to an integer quantity. For this, it's best to adopt some sort of rounding methodology, where I noted above. The rounding value is used in YV12 and YUY2 calculations, it was just omitted from the RGB functions, probably because they're pretty well neglected.

The masktools v2.0a37 changes are pretty trivial, I'll see what I can cook up in the next few days.

The program I was using is from an image analysis pack called ImageMagick. I'll just go back to their compare function to give some basic statistical analysis of changes I make to speed up the algorithms. Essentially, to weigh speed vs quality.

This error was a blatant error and actually was rather trivial to find, I was going to submit a SourceForge bug to the Avisynth guys, but apparently the site is down at the moment.

As for average, I'm not getting any hits from the filter collection or the web in general, is this something
a) in 2.58 that I didn't port and am totally drawing a blank on
b) in 2.6 that can be added
c) has the source tucked away in a location you are familiar with
d) closed source

If it falls under those categories a-c, then probably feasible, I just need to know where to find the basis. If it falls under the dreaded category d (nnedi2!) then well, I can try to recreate it, but ugh, I've been busy. I was considering writing a comparable plugin to nnedi2, but this conversion may be easier in the short term.

The busy factor also means I have limited time to set up an RSS feed, document well, etc. The downside of OSS is that it's generally poorly documented . . . I can see why now. Please, if you get a chance, put the posted build through the paces, it has a number of small tweaks which generate a measurable performance difference. I've been playing with a lot of instruction ordering and code size variability, but can only test it with my machine. It may throw up on itself under another environment. This would be good to know in advance to prevent another total mess like last time.

Last edited by JoshyD; 17th April 2010 at 03:09.
JoshyD is offline   Reply With Quote
Old 17th April 2010, 03:26   #354  |  Link
Stephen R. Savage
Registered User
 
Stephen R. Savage's Avatar
 
Join Date: Nov 2009
Posts: 327
Average plugin: http://www.wilbertdijkhof.com/mg262/Average_v11.zip

It's actually very simple. It takes an arbitrary number of clips and weights, then calculates the weighted average (w_a * a + w_b * b + w_c * c [...]).

Edit:
@Avs developers: Why is the YV12 luma channel centered on 126?

@JoshyD: I can't tell a difference between the Y-channel differences for RGB and YUV resizing now. I think this problem is fixed for certain. Good work.

Edit2:

@JoshyD: Also, what happened to the TCPDeliver/Source plugin that ships with normal Avisynth?

Last edited by Stephen R. Savage; 17th April 2010 at 04:30.
Stephen R. Savage is offline   Reply With Quote
Old 17th April 2010, 17:57   #355  |  Link
manoj4986
Registered User
 
Join Date: Sep 2006
Posts: 8
Guys also need following 64 bit plugins
1)HDRAGC
2)dctfilter
3)medianblur
4)nnedi2
manoj4986 is offline   Reply With Quote
Old 17th April 2010, 19:03   #356  |  Link
Stephen R. Savage
Registered User
 
Stephen R. Savage's Avatar
 
Join Date: Nov 2009
Posts: 327
Quote:
Originally Posted by manoj4986 View Post
Guys also need following 64 bit plugins\
4)nnedi2
Good luck with that.
Stephen R. Savage is offline   Reply With Quote
Old 17th April 2010, 19:13   #357  |  Link
squid_80
Registered User
 
Join Date: Dec 2004
Location: Melbourne, AU
Posts: 1,963
pshufw isn't used because it's an SSE instruction, not MMX.
squid_80 is offline   Reply With Quote
Old 17th April 2010, 22:05   #358  |  Link
turbojet
Registered User
 
Join Date: May 2008
Posts: 1,840
JoshyD: Yeah you can use the benchmark results wherever you want. It was done on an Athlon II 620 at 3.4 ghz. The deadlock didn't display anything, x264.exe was using 25% cpu with no status updates. During the benchmark it seemed to me the goal with avisynth mt is hitting 90%+ CPU with the fewest number of threads with a truly multi-threaded encoder (x264 with fairly fast settings). Deciding between MT() and SetMTMode() seems to be a crap shoot and both may result in a deadlock. With a lot of research and some coding it might be possible to automatically set the optimal setmtmode or mt when it's helpful and works.

Stephen: For update notifications you can get an rss feed from the googlecode download page.
turbojet is offline   Reply With Quote
Old 18th April 2010, 19:42   #359  |  Link
levi
Registered User
 
Join Date: Mar 2003
Posts: 116
Quote:
Originally Posted by turbojet View Post
I did some benchmarks of the common filters I use and a few slower filters here's the numbers (messy I know but I couldn't think of a better way). Trends and notes below may make more sense
Could you update your post with the full .avs scripts you used?

Quote:
Originally Posted by turbojet View Post
During the benchmark it seemed to me the goal with avisynth mt is hitting 90%+ CPU with the fewest number of threads with a truly multi-threaded encoder (x264 with fairly fast settings).
Agreed. I found the same in my testing.

Quote:
Originally Posted by turbojet View Post
Deciding between MT() and SetMTMode() seems to be a crap shoot and both may result in a deadlock. With a lot of research and some coding it might be possible to automatically set the optimal setmtmode or mt when it's helpful and works.
That would be awesome! It would also be nice to have a repository to submit & compare speeds among various filters. This database could be helpful in manually and/or programatically selecting the optimal setting.

Last edited by levi; 18th April 2010 at 20:01.
levi is offline   Reply With Quote
Old 18th April 2010, 20:31   #360  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,988
I'm getting some nasty crashes in virtualdub x64 or x264 x64 when using the 4/12 release on an MDegrain2 script...

Code:
setmtmode(2,0)

directshowsource("E:\BluRay Rips\Extract\Extract.mkv", audio=false, fps=23.976)

assumefps("ntsc_film")

spline36resize(1280,720)
md2(150,300)

distributor
MD2 is just a little wrapper for MDegrain2... the two parameters are thSAD and thSADC respectively.

Has anyone had luck getting something like this working with the current x64 releases?

Thanks
~MiSfit
__________________
These are all my personal statements, not those of my employer :)
Blue_MiSfit is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 10:29.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.