Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 27th March 2015, 02:09   #1  |  Link
zerowalker
Registered User
 
Join Date: Jul 2011
Posts: 1,100
Decoding multiple threads?

Just wondering if it's possible to decode a video on more threads. I don't mean the hackish way MTMode etc.
Normally a decoder has support for multiple threads nowadays, but with Avisynth it's locked to 1.

Why i want this is because it's the bottleneck when i am encoding, the decoding is slowest so i would like to make it faster. (Good thing is that i can waste more time on encoding quality i guess).

I am talking about AviSynth 2.6.0 RC1.

Many Thanks
zerowalker is offline   Reply With Quote
Old 27th March 2015, 04:04   #2  |  Link
Mounir
Registered User
 
Join Date: Nov 2006
Posts: 714
decoding or encoding ?
for decoding there is DGDecNV (not free) see here: http://neuron2.net/dgdecnv/dgdecnv.html
Mounir is offline   Reply With Quote
Old 27th March 2015, 05:00   #3  |  Link
zerowalker
Registered User
 
Join Date: Jul 2011
Posts: 1,100
Decoding, as it says;P

Should have mentioned, Codecs are MagicYUV, Lagarith etc, those types.
zerowalker is offline   Reply With Quote
Old 27th March 2015, 07:12   #4  |  Link
colours
Registered User
 
colours's Avatar
 
Join Date: Mar 2014
Posts: 312
Why would Avisynth have any control over how many threads a source filter wants to use?

Where do your lossless video files come from?

Have you tried FFV1, UtVideo, lossless H.264, etc.?
__________________
Say no to AviSynth 2.5.8 and DirectShowSource!
colours is offline   Reply With Quote
Old 27th March 2015, 07:57   #5  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 3,636
How did you determine Avisynth is "locked" to a single decoding thread? ffms2 doesn't specify a need for MT Avisynth to use more than one thread for decoding. At least no mention of it I could see.

https://ffmpegsource.googlecode.com/...-avisynth.html
int threads = -1
The number of decoding threads to request from libavcodec. Setting it to less than or equal to zero means it defaults to the number of logical CPU's reported by Windows. Note that this setting might be completely ignored by libavcodec under a number of conditions; most commonly because a lot of decoders actually do not support multithreading.
hello_hello is offline   Reply With Quote
Old 27th March 2015, 16:34   #6  |  Link
zerowalker
Registered User
 
Join Date: Jul 2011
Posts: 1,100
My only clue to single thread is using Avisource with a lossless codec (pretty sure i have tried UtVideo and had same results). And it's always 25% Max (4 cores).
So that's why my assumption is 1 thread. So it's probably the Avisource that's the culprit i guess?
Haven't tried DirectShow, but had some bad experiences with it at times being non-frame-accurate etc.
zerowalker is offline   Reply With Quote
Old 27th March 2015, 16:48   #7  |  Link
Groucho2004
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Posts: 3,786
Quote:
Originally Posted by zerowalker View Post
My only clue to single thread is using Avisource with a lossless codec (pretty sure i have tried UtVideo and had same results). And it's always 25% Max (4 cores).
So that's why my assumption is 1 thread. So it's probably the Avisource that's the culprit i guess?
Avisource with an UTVideo encoded file utilizes all the cores of my i5-2500K (~90% usage).
Groucho2004 is offline   Reply With Quote
Old 27th March 2015, 16:49   #8  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,460
How is decoding a bottleneck? What resolutions are you using ? You should get 100's of FPS with UT Video and 1080p on a typical computer
poisondeathray is offline   Reply With Quote
Old 27th March 2015, 19:02   #9  |  Link
creaothceann
Registered User
 
Join Date: Jul 2010
Location: Germany
Posts: 360
Quote:
Originally Posted by zerowalker View Post
Haven't tried DirectShow, but had some bad experiences with it at times being non-frame-accurate etc.
Use DSS2.
creaothceann is offline   Reply With Quote
Old 27th March 2015, 20:53   #10  |  Link
zerowalker
Registered User
 
Join Date: Jul 2011
Posts: 1,100
hmm wait it might be avs4x264 that's the bottleneck here.

(I am doing simple 2D encodings, so it's super fast, pixelated low res stuff).
It's not super bottleneck, but yeah, it's fairly close on Medium.

Last edited by zerowalker; 27th March 2015 at 21:05.
zerowalker is offline   Reply With Quote
Old 27th March 2015, 21:06   #11  |  Link
hello_hello
Registered User
 
Join Date: Mar 2011
Posts: 3,636
Quote:
Originally Posted by zerowalker View Post
hmm wait it might be avs4x264 that's the bottleneck here.

(I am doing simple 2D encodings, so it's super fast, pixelated low res stuff).
I've seen a couple of "why is MeGUI so slow?" type threads recently. Maybe at VideoHelp rather than doom9, but I did seriously wonder if avs4x264 was the cause.

I use XP so I can't test it myself, but if you uncheck the 64 bit x264 encoder setting in MeGUI's options I believe MeGUI will stop using avs4x264 (it's only needed for "32 bit Avisynth" -> "64 bit x264", the way I understand it).
And I guess I just made an assumption you're actually using MeGUI..... sorry.

I only let ffms2 use a single thread for decoding and it doesn't seem to slow things down much. CPU usage still sits between 90% and 100% most of the time (4 cores) unless I'm using slow Avisynth filters.
hello_hello is offline   Reply With Quote
Old 27th March 2015, 22:42   #12  |  Link
Sparktank
47.952fps@71.928Hz
 
Sparktank's Avatar
 
Join Date: Mar 2011
Posts: 848
Taro_06 made an updated versoin of avs4x264, now avs4x26x (as it supports x265).
http://tmod.nmm-hd.org/avs4x26x/

If one wants to try that intead of old avs4x264.
__________________
Win10 (x64) build 17134 | GPU Caps Viewer v1.39.0.0
NVIDIA GeForce GT 640 (GK107) 2047MB/DDR3 | (R398.11)
NTSC | DVD: R1 | BD: A
Sparktank is offline   Reply With Quote
Old 28th March 2015, 06:44   #13  |  Link
zerowalker
Registered User
 
Join Date: Jul 2011
Posts: 1,100
I am not using MeGUI but my own made "MeGUI", but it's basically the same just super simple, so it uses avs4x264 to pipe to x264 as i must have the 32bit version of Avisynth (which i could use 64bit, but sadly i need the new Avisource track support).
Tried the new version avs4x26x, same results.

But it's not really a deal breaker. But i guess if it was possible to use 64bit it might go faster, as 64bit decoders tend to better.
zerowalker is offline   Reply With Quote
Old 28th March 2015, 06:54   #14  |  Link
colours
Registered User
 
colours's Avatar
 
Join Date: Mar 2014
Posts: 312
Please post your full script and command line if you want any of us to help you rather than take stabs in the dark.
__________________
Say no to AviSynth 2.5.8 and DirectShowSource!
colours is offline   Reply With Quote
Old 28th March 2015, 07:40   #15  |  Link
zerowalker
Registered User
 
Join Date: Jul 2011
Posts: 1,100
Not really much of a script, but here it is.

Remember, it's not that it's Slow. It's that it's limited to 1 Thread that bothers me a bit.

Quote:
a=Avisource("Z:\VisualBoyAdvance-M_2015_03_26_14_02_00_585.avi",atrack=1).GetChannels(1,1).SyncAudio( 0.999979)
b=Avisource("Z:\VisualBoyAdvance-M_2015_03_26_14_02_00_585.avi",atrack=0).SyncAudio( 0.999979)
#~ a.MixAudio(b, clip1_factor=1, clip2_factor=0.04)
a
PointResize(800, 720, src_left=0.0, src_top=0.0, src_width=0.0, src_height=0.0)
ConvertToYV24(matrix="Rec709", interlaced=false)
Quote:
function SyncAudio(clip c, Float sync)
{
c
ConvertAudioToFloat()
AR=AudioRate()
TR=float(AR)*sync


ResampleAudio(ContinuedNumerator(TR), ContinuedDenominator(TR))
AssumeSampleRate(AR)

}

function SyncAudioToVideo(c){
c
ConvertAudioToFloat()
AR=AudioRate()
TR=Float(AR)*AR*FrameCount()/(AudioLengthF()*FrameRate())
ResampleAudio(ContinuedNumerator(TR), ContinuedDenominator(TR))
AssumeSampleRate(AR)
}

Last edited by zerowalker; 28th March 2015 at 07:49.
zerowalker is offline   Reply With Quote
Old 28th March 2015, 09:04   #16  |  Link
Groucho2004
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Posts: 3,786
Quote:
Originally Posted by zerowalker View Post
It's that it's limited to 1 Thread that bothers me a bit.
How do you determine that?

I suggest the following:
1. Remove all audio processing from the script and run it through AVSMeter:
Code:
Avisource("Z:\VisualBoyAdvance-M_2015_03_26_14_02_00_585.avi")
2. Use the full script and also run it through AVSMeter.

Use the "-log" switch for AVSMeter and post the log files. If it's a very long clip you can limit the frame range with "-range=x,x".
Groucho2004 is offline   Reply With Quote
Old 28th March 2015, 12:01   #17  |  Link
Pat357
Registered User
 
Join Date: Jun 2006
Posts: 419
Unless you're have used a very old codec to create those .AVI, the decoding should support more cores.

What codec is used to decode the AVI ? What's the resolution ?
Pat357 is offline   Reply With Quote
Old 28th March 2015, 22:13   #18  |  Link
zerowalker
Registered User
 
Join Date: Jul 2011
Posts: 1,100
Seems to be superfast with only the Decoding (even if it doesn't seem to use 50%, it's at like 1000fps).
The HDD Didn't bottleneck, but at that speed i don't really care if it's 1 thread or not, so decoding isn't the issue i guess.

Quote:
[General info]
Log file created with: AVSMeter 1.9.8.0 (x86)
Avisynth version: AviSynth 2.60, build:Jan 14 2015 [09:58:31] (2.6.0.5)


[Clip info]
Number of frames: 249948
Length (hh:mm:ss.ms): 01:09:25.800
Frame width: 640
Frame height: 576
Framerate: 60.000 (60/1)
Colorspace: RGB32
Audio channels: 2
Audio bits/sample: 2
Audio sample rate: 48000
Audio samples: 199948000


[Runtime info]
Frames processed: 10001 (0 - 10000)
FPS (min | max | average): 411.6 | 2063 | 1149
Memory usage (phys | virt): 33 | 29 MB
Thread count: 4
CPU usage (average): 31%
Time (elapsed): 00:00:08.706


[Script]
Avisource("Z:\VisualBoyAdvance-M_2015_03_27_16_36_24_407.avi")

[Performance data]
Frame interval Frames/sec Time/frame(ms) CPU(%) Threads PhysMEM(MB) VirtMEM(MB)
0-19 1414.005 0.7072 0 4 31 27
20-39 1229.132 0.8136 0 4 31 27
40-59 454.435 2.2005 0 4 31 27
60-79 1129.949 0.8850 0 4 31 27
80-99 1394.569 0.7171 28 4 33 29
100-119 1505.480 0.6642 28 4 33 29
120-139 1386.771 0.7211 28 4 33 29
140-159 1492.680 0.6699 28 4 33 29
160-179 1250.698 0.7996 28 4 33 29
180-199 1144.013 0.8741 28 4 33 29
200-219 1121.031 0.8920 28 4 33 29
220-239 1134.955 0.8811 32 4 33 29
240-259 1128.806 0.8859 32 4 33 29
260-279 1324.400 0.7551 32 4 33 29
280-299 1215.761 0.8225 32 4 33 29
300-319 1235.058 0.8097 32 4 33 29
320-339 1355.985 0.7375 32 4 33 29
340-359 1587.015 0.6301 32 4 33 29
360-379 1121.916 0.8913 25 4 33 29
380-399 966.981 1.0341 25 4 33 29
400-419 936.753 1.0675 25 4 33 29
420-439 911.505 1.0971 25 4 33 29
440-459 1005.669 0.9944 25 4 33 29
460-479 1306.263 0.7655 25 4 33 29
480-499 1157.317 0.8641 25 4 33 29
500-519 999.020 1.0010 25 4 33 29
520-539 706.702 1.4150 25 4 33 29
540-559 1003.679 0.9963 25 4 33 29
560-579 988.415 1.0117 25 4 33 29
580-599 957.875 1.0440 32 4 33 29
600-619 1236.035 0.8090 32 4 33 29
620-639 1524.458 0.6560 32 4 33 29
640-659 1043.036 0.9587 32 4 33 29
660-679 813.758 1.2289 32 4 33 29
680-699 863.593 1.1580 32 4 33 29
700-719 867.352 1.1529 31 4 33 29
720-739 411.591 2.4296 31 4 33 29
740-759 842.799 1.1865 31 4 33 29
760-779 864.128 1.1572 31 4 33 29
780-799 871.065 1.1480 21 4 33 29
800-819 867.410 1.1529 21 4 33 29
Codec is Lagarith in this case. (As the size is so small for 2D, i did use MagicYUV first but i see no benefit).

EDIT: Checked, and PointResize really does have quite a performance hit, didn't think it would be that great, that explains it.

Last edited by zerowalker; 28th March 2015 at 22:16.
zerowalker is offline   Reply With Quote
Old 28th March 2015, 22:44   #19  |  Link
Groucho2004
 
Groucho2004's Avatar
 
Join Date: Mar 2006
Posts: 3,786
Quote:
Originally Posted by zerowalker View Post
Checked, and PointResize really does have quite a performance hit, didn't think it would be that great, that explains it.
Really, pointresize? That should be very fast. What CPU is this running on?
Groucho2004 is offline   Reply With Quote
Old 28th March 2015, 23:38   #20  |  Link
zerowalker
Registered User
 
Join Date: Jul 2011
Posts: 1,100
Quote:
Originally Posted by Groucho2004 View Post
Really, pointresize? That should be very fast. What CPU is this running on?
FPS (min | max | average): 53.72 | 387.5 | 316.5
That's with PointResize.

I got i5 760 @4ghz

Though this is easily solvable as i can just record in the resolution i resize to.
I only used PointResize is i thought it would be faster compared to a higher resolution decoding.
zerowalker is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 05:08.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2018, vBulletin Solutions Inc.