Decoding multiple threads?

[zerowalker]

Just wondering if it's possible to decode a video on more threads. I don't mean the hackish way MTMode etc.
Normally a decoder has support for multiple threads nowadays, but with Avisynth it's locked to 1.

Why i want this is because it's the bottleneck when i am encoding, the decoding is slowest so i would like to make it faster. (Good thing is that i can waste more time on encoding quality i guess).

I am talking about AviSynth 2.6.0 RC1.

Many Thanks

[Mounir]

decoding or encoding ?
for decoding there is DGDecNV (not free) see here: http://neuron2.net/dgdecnv/dgdecnv.html

[zerowalker]

Decoding, as it says;P

Should have mentioned, Codecs are MagicYUV, Lagarith etc, those types.

[colours]

Why would Avisynth have any control over how many threads a source filter wants to use?

Where do your lossless video files come from?

Have you tried FFV1, UtVideo, lossless H.264, etc.?

[hello_hello]

How did you determine Avisynth is "locked" to a single decoding thread? ffms2 doesn't specify a need for MT Avisynth to use more than one thread for decoding. At least no mention of it I could see.

https://ffmpegsource.googlecode.com/...-avisynth.html
int threads = -1
The number of decoding threads to request from libavcodec. Setting it to less than or equal to zero means it defaults to the number of logical CPU's reported by Windows. Note that this setting might be completely ignored by libavcodec under a number of conditions; most commonly because a lot of decoders actually do not support multithreading.

[zerowalker]

My only clue to single thread is using Avisource with a lossless codec (pretty sure i have tried UtVideo and had same results). And it's always 25% Max (4 cores).
So that's why my assumption is 1 thread. So it's probably the Avisource that's the culprit i guess?
Haven't tried DirectShow, but had some bad experiences with it at times being non-frame-accurate etc.

[Groucho2004]

Originally Posted by zerowalker

My only clue to single thread is using Avisource with a lossless codec (pretty sure i have tried UtVideo and had same results). And it's always 25% Max (4 cores).
So that's why my assumption is 1 thread. So it's probably the Avisource that's the culprit i guess?

Avisource with an UTVideo encoded file utilizes all the cores of my i5-2500K (~90% usage).

[poisondeathray]

How is decoding a bottleneck? What resolutions are you using ? You should get 100's of FPS with UT Video and 1080p on a typical computer

[creaothceann]

Originally Posted by zerowalker

Haven't tried DirectShow, but had some bad experiences with it at times being non-frame-accurate etc.

Use DSS2.

[zerowalker]

hmm wait it might be avs4x264 that's the bottleneck here.

(I am doing simple 2D encodings, so it's super fast, pixelated low res stuff).
It's not super bottleneck, but yeah, it's fairly close on Medium.

[hello_hello]

Originally Posted by zerowalker

hmm wait it might be avs4x264 that's the bottleneck here.

(I am doing simple 2D encodings, so it's super fast, pixelated low res stuff).

I've seen a couple of "why is MeGUI so slow?" type threads recently. Maybe at VideoHelp rather than doom9, but I did seriously wonder if avs4x264 was the cause.

I use XP so I can't test it myself, but if you uncheck the 64 bit x264 encoder setting in MeGUI's options I believe MeGUI will stop using avs4x264 (it's only needed for "32 bit Avisynth" -> "64 bit x264", the way I understand it).
And I guess I just made an assumption you're actually using MeGUI..... sorry.

I only let ffms2 use a single thread for decoding and it doesn't seem to slow things down much. CPU usage still sits between 90% and 100% most of the time (4 cores) unless I'm using slow Avisynth filters.

[Sparktank]

Taro_06 made an updated versoin of avs4x264, now avs4x26x (as it supports x265).
http://tmod.nmm-hd.org/avs4x26x/

If one wants to try that intead of old avs4x264.

[zerowalker]

I am not using MeGUI but my own made "MeGUI", but it's basically the same just super simple, so it uses avs4x264 to pipe to x264 as i must have the 32bit version of Avisynth (which i could use 64bit, but sadly i need the new Avisource track support).
Tried the new version avs4x26x, same results.

But it's not really a deal breaker. But i guess if it was possible to use 64bit it might go faster, as 64bit decoders tend to better.

[colours]

Please post your full script and command line if you want any of us to help you rather than take stabs in the dark.

[zerowalker]

Not really much of a script, but here it is.

Remember, it's not that it's Slow. It's that it's limited to 1 Thread that bothers me a bit.

a=Avisource("Z:\VisualBoyAdvance-M_2015_03_26_14_02_00_585.avi",atrack=1).GetChannels(1,1).SyncAudio( 0.999979)
b=Avisource("Z:\VisualBoyAdvance-M_2015_03_26_14_02_00_585.avi",atrack=0).SyncAudio( 0.999979)
#~ a.MixAudio(b, clip1_factor=1, clip2_factor=0.04)
a
PointResize(800, 720, src_left=0.0, src_top=0.0, src_width=0.0, src_height=0.0)
ConvertToYV24(matrix="Rec709", interlaced=false)

function SyncAudio(clip c, Float sync)
{
c
ConvertAudioToFloat()
AR=AudioRate()
TR=float(AR)*sync


ResampleAudio(ContinuedNumerator(TR), ContinuedDenominator(TR))
AssumeSampleRate(AR)

}

function SyncAudioToVideo(c){
c
ConvertAudioToFloat()
AR=AudioRate()
TR=Float(AR)*AR*FrameCount()/(AudioLengthF()*FrameRate())
ResampleAudio(ContinuedNumerator(TR), ContinuedDenominator(TR))
AssumeSampleRate(AR)
}

[Groucho2004]

Originally Posted by zerowalker

It's that it's limited to 1 Thread that bothers me a bit.

How do you determine that?

I suggest the following:
1. Remove all audio processing from the script and run it through AVSMeter:

Avisource("Z:\VisualBoyAdvance-M_2015_03_26_14_02_00_585.avi")

2. Use the full script and also run it through AVSMeter.

Use the "-log" switch for AVSMeter and post the log files. If it's a very long clip you can limit the frame range with "-range=x,x".

[Pat357]

Unless you're have used a very old codec to create those .AVI, the decoding should support more cores.

What codec is used to decode the AVI ? What's the resolution ?

[zerowalker]

Seems to be superfast with only the Decoding (even if it doesn't seem to use 50%, it's at like 1000fps).
The HDD Didn't bottleneck, but at that speed i don't really care if it's 1 thread or not, so decoding isn't the issue i guess.

[General info]
Log file created with: AVSMeter 1.9.8.0 (x86)
Avisynth version: AviSynth 2.60, build:Jan 14 2015 [09:58:31] (2.6.0.5)


[Clip info]
Number of frames: 249948
Length (hh:mm:ss.ms): 01:09:25.800
Frame width: 640
Frame height: 576
Framerate: 60.000 (60/1)
Colorspace: RGB32
Audio channels: 2
Audio bits/sample: 2
Audio sample rate: 48000
Audio samples: 199948000


[Runtime info]
Frames processed: 10001 (0 - 10000)
FPS (min | max | average): 411.6 | 2063 | 1149
Memory usage (phys | virt): 33 | 29 MB
Thread count: 4
CPU usage (average): 31%
Time (elapsed): 00:00:08.706


[Script]
Avisource("Z:\VisualBoyAdvance-M_2015_03_27_16_36_24_407.avi")

[Performance data]
Frame interval Frames/sec Time/frame(ms) CPU(%) Threads PhysMEM(MB) VirtMEM(MB)
0-19 1414.005 0.7072 0 4 31 27
20-39 1229.132 0.8136 0 4 31 27
40-59 454.435 2.2005 0 4 31 27
60-79 1129.949 0.8850 0 4 31 27
80-99 1394.569 0.7171 28 4 33 29
100-119 1505.480 0.6642 28 4 33 29
120-139 1386.771 0.7211 28 4 33 29
140-159 1492.680 0.6699 28 4 33 29
160-179 1250.698 0.7996 28 4 33 29
180-199 1144.013 0.8741 28 4 33 29
200-219 1121.031 0.8920 28 4 33 29
220-239 1134.955 0.8811 32 4 33 29
240-259 1128.806 0.8859 32 4 33 29
260-279 1324.400 0.7551 32 4 33 29
280-299 1215.761 0.8225 32 4 33 29
300-319 1235.058 0.8097 32 4 33 29
320-339 1355.985 0.7375 32 4 33 29
340-359 1587.015 0.6301 32 4 33 29
360-379 1121.916 0.8913 25 4 33 29
380-399 966.981 1.0341 25 4 33 29
400-419 936.753 1.0675 25 4 33 29
420-439 911.505 1.0971 25 4 33 29
440-459 1005.669 0.9944 25 4 33 29
460-479 1306.263 0.7655 25 4 33 29
480-499 1157.317 0.8641 25 4 33 29
500-519 999.020 1.0010 25 4 33 29
520-539 706.702 1.4150 25 4 33 29
540-559 1003.679 0.9963 25 4 33 29
560-579 988.415 1.0117 25 4 33 29
580-599 957.875 1.0440 32 4 33 29
600-619 1236.035 0.8090 32 4 33 29
620-639 1524.458 0.6560 32 4 33 29
640-659 1043.036 0.9587 32 4 33 29
660-679 813.758 1.2289 32 4 33 29
680-699 863.593 1.1580 32 4 33 29
700-719 867.352 1.1529 31 4 33 29
720-739 411.591 2.4296 31 4 33 29
740-759 842.799 1.1865 31 4 33 29
760-779 864.128 1.1572 31 4 33 29
780-799 871.065 1.1480 21 4 33 29
800-819 867.410 1.1529 21 4 33 29

Codec is Lagarith in this case. (As the size is so small for 2D, i did use MagicYUV first but i see no benefit).

EDIT: Checked, and PointResize really does have quite a performance hit, didn't think it would be that great, that explains it.

[Groucho2004]

Originally Posted by zerowalker

Checked, and PointResize really does have quite a performance hit, didn't think it would be that great, that explains it.

Really, pointresize? That should be very fast. What CPU is this running on?

[zerowalker]

Originally Posted by Groucho2004

Really, pointresize? That should be very fast. What CPU is this running on?

FPS (min | max | average): 53.72 | 387.5 | 316.5
That's with PointResize.

I got i5 760 @4ghz

Though this is easily solvable as i can just record in the resolution i resize to.
I only used PointResize is i thought it would be faster compared to a higher resolution decoding.