Darksoul71
27th September 2007, 19:27
Hi all,
I havenīt used AutoGK for almost 2 years since Iīve switched back to MPEG2 for most of my encodings.
Based on my experiences when coding QuEnc^n and HCEnc^n I did some experiments to see wether
multiinstance encoding makes sense. To my surprise Iīve seen quite impressive increases due to multi-instance encoding. Iīve use the PAL version of The Punisher with 177648 Frames and the AVS file which was generated by AutoGK.
My system:
Opteron 170
2 GB GSKILL DDR-400 Memory in Dual Channel mode
Software used:
AVISynth 2.5.7
XviD-1.2.-127-25022006 _Alpha Build_
XVid Encraw - built at 31 Aug 2007
I didnīt use any special settings for Encraw. So most settings were as defaulted by encrawl (?). Anyway, this comparison was only to get a feeling about performance. I think those increases would be also visible when e.g. using VirtualDubMod and DivX.
Here are the values:
+---------------------------+--------+-----------+-------+----------------
| Details | Frames | Runtime | FPS | Increase in %
+---------------------------+--------+-----------+-------+----------------
| One instance (Threads=1) | 177648 | 02:02:00 | 24,27 | 0%
| One instance (Threads=2) | 177648 | 01:32:14 | 32,1 | 32%
| Two instances (Threads=1) | 177648 | 01:07:20 | 43,97 | 81,20%
| Two instances (Threads=2) | 177648 | 00:59:13 | 50 | 106%
+---------------------------+--------+-----------+-------+----------------
The increase for one instance singlethreaded to one instance multihreaded was quite limited though.
You get 30% (what it nice) but are still far away from the 100% increase from two multithreaded instances.
What I found even more interesting is the fact that even with two instances running singlethreaded there is
still some room for increase when enabling multithreading.
My questions @lenox:
Are there any chances that AutoGK (in one of the next releases) could include multi-instance encoding ?
Will the sources be available at some day somewhere ?
I could imagine writting a middleware between AutoGK and VirtualDubMod to enable multi-instance encoding but this means a lot of problems such as the codec-specific settings within the VCF file are not very easy to decode and alter. Esp. since they can change with any version of the codec. I can remember vaguely that they are encoded in MIME BASE64 and you have to try out which setting from the codec dialog translates to which part of the "codec data". Thewef was once so kind to explain this to me when Iīve asked how GK does this. Now since tools such as XVid Encraw are available you can directly specify the settings via CLI in clear text.
Generation of AVS segment files is really easy. I adopted code for a more simplified fully AVISynth-based segment generation from Mr_Odwin and his tools Multicoreenc. Here are a few examples:
Original script generated from AutoGK:
LoadPlugin("C:\PROGRA~1\AutoGK\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\PROGRA~1\AutoGK\filters\autocrop.dll")
LoadPlugin("C:\PROGRA~1\AutoGK\filters\ColorMatrix.dll")
LoadPlugin("C:\PROGRA~1\AutoGK\filters\RemoveGrainSSE3.dll")
movie = mpeg2source("D:\Punisher\agk_tmp\Punisher.d2v")
cropclip = autocrop(movie,mode=0,wmultof=4,hmultof=4,samples=10,aspect=0,threshold=34,samplestartframe=0,leftadd=0,rightadd=0,topadd=0,bottomadd=0)
fixed_aspect = 1.42222222222222
c_width = width(cropclip)
c_height = round(height(cropclip) / fixed_aspect)
input_par = float(c_width)/float(c_height)
input_par = (input_par > 1.4) || (input_par < 1.25) ? input_par : (4.0/3.0)
out_width = 608
out_height = round(float(out_width) / input_par)
hmod = out_height - (floor(out_height / 16 ) * 16)
out_height = (hmod > 4) ? (out_height + (16 - hmod)) : (out_height - hmod)
new_aspect = (float(out_width) / float(out_height)) / fixed_aspect
autocrop(movie,mode=0,wmultof=4,hmultof=4,samples=10,aspect=new_aspect,threshold=34,samplestartframe=0,leftadd=0,rightadd=0,topadd=0,bottomadd=0)
LanczosResize(out_width,out_height)
RemoveGrain(mode=2)
ColorMatrix("Rec.709->Rec.601",opt=0,hints=false,threads=0)
Script for segment#1:
..as above plus...
SegLen = Floor(FrameCount()/2)
SegStart = 0
SegEnd = SegLen
Trim(SegStart,SegEnd)
Script for segment#2:
..as above plus...
SegLen = Floor(FrameCount()/2)
SegStart = 1+SegLen
SegEnd = FrameCount()
Trim(SegStart,SegEnd)
Merging can be easily done by using VirtualDubMod itself:
VirtualDub.Open("e:\\segment_1.avi");
VirtualDub.Append("e:\\segment_2.avi");
VirtualDub.video.SetMode(0);
VirtualDub.SaveAVI("e:\\movie.avi");
I could write a middleware similar to QuEnc^n or HCEnc^n for XVid encraw with a pretty similar CLI. This would make implementation of multi-instance encoding easier for anyone willing to support it.
Let me hear what you think !
Best regards,
D$
I havenīt used AutoGK for almost 2 years since Iīve switched back to MPEG2 for most of my encodings.
Based on my experiences when coding QuEnc^n and HCEnc^n I did some experiments to see wether
multiinstance encoding makes sense. To my surprise Iīve seen quite impressive increases due to multi-instance encoding. Iīve use the PAL version of The Punisher with 177648 Frames and the AVS file which was generated by AutoGK.
My system:
Opteron 170
2 GB GSKILL DDR-400 Memory in Dual Channel mode
Software used:
AVISynth 2.5.7
XviD-1.2.-127-25022006 _Alpha Build_
XVid Encraw - built at 31 Aug 2007
I didnīt use any special settings for Encraw. So most settings were as defaulted by encrawl (?). Anyway, this comparison was only to get a feeling about performance. I think those increases would be also visible when e.g. using VirtualDubMod and DivX.
Here are the values:
+---------------------------+--------+-----------+-------+----------------
| Details | Frames | Runtime | FPS | Increase in %
+---------------------------+--------+-----------+-------+----------------
| One instance (Threads=1) | 177648 | 02:02:00 | 24,27 | 0%
| One instance (Threads=2) | 177648 | 01:32:14 | 32,1 | 32%
| Two instances (Threads=1) | 177648 | 01:07:20 | 43,97 | 81,20%
| Two instances (Threads=2) | 177648 | 00:59:13 | 50 | 106%
+---------------------------+--------+-----------+-------+----------------
The increase for one instance singlethreaded to one instance multihreaded was quite limited though.
You get 30% (what it nice) but are still far away from the 100% increase from two multithreaded instances.
What I found even more interesting is the fact that even with two instances running singlethreaded there is
still some room for increase when enabling multithreading.
My questions @lenox:
Are there any chances that AutoGK (in one of the next releases) could include multi-instance encoding ?
Will the sources be available at some day somewhere ?
I could imagine writting a middleware between AutoGK and VirtualDubMod to enable multi-instance encoding but this means a lot of problems such as the codec-specific settings within the VCF file are not very easy to decode and alter. Esp. since they can change with any version of the codec. I can remember vaguely that they are encoded in MIME BASE64 and you have to try out which setting from the codec dialog translates to which part of the "codec data". Thewef was once so kind to explain this to me when Iīve asked how GK does this. Now since tools such as XVid Encraw are available you can directly specify the settings via CLI in clear text.
Generation of AVS segment files is really easy. I adopted code for a more simplified fully AVISynth-based segment generation from Mr_Odwin and his tools Multicoreenc. Here are a few examples:
Original script generated from AutoGK:
LoadPlugin("C:\PROGRA~1\AutoGK\DGMPGDec\DGDecode.dll")
LoadPlugin("C:\PROGRA~1\AutoGK\filters\autocrop.dll")
LoadPlugin("C:\PROGRA~1\AutoGK\filters\ColorMatrix.dll")
LoadPlugin("C:\PROGRA~1\AutoGK\filters\RemoveGrainSSE3.dll")
movie = mpeg2source("D:\Punisher\agk_tmp\Punisher.d2v")
cropclip = autocrop(movie,mode=0,wmultof=4,hmultof=4,samples=10,aspect=0,threshold=34,samplestartframe=0,leftadd=0,rightadd=0,topadd=0,bottomadd=0)
fixed_aspect = 1.42222222222222
c_width = width(cropclip)
c_height = round(height(cropclip) / fixed_aspect)
input_par = float(c_width)/float(c_height)
input_par = (input_par > 1.4) || (input_par < 1.25) ? input_par : (4.0/3.0)
out_width = 608
out_height = round(float(out_width) / input_par)
hmod = out_height - (floor(out_height / 16 ) * 16)
out_height = (hmod > 4) ? (out_height + (16 - hmod)) : (out_height - hmod)
new_aspect = (float(out_width) / float(out_height)) / fixed_aspect
autocrop(movie,mode=0,wmultof=4,hmultof=4,samples=10,aspect=new_aspect,threshold=34,samplestartframe=0,leftadd=0,rightadd=0,topadd=0,bottomadd=0)
LanczosResize(out_width,out_height)
RemoveGrain(mode=2)
ColorMatrix("Rec.709->Rec.601",opt=0,hints=false,threads=0)
Script for segment#1:
..as above plus...
SegLen = Floor(FrameCount()/2)
SegStart = 0
SegEnd = SegLen
Trim(SegStart,SegEnd)
Script for segment#2:
..as above plus...
SegLen = Floor(FrameCount()/2)
SegStart = 1+SegLen
SegEnd = FrameCount()
Trim(SegStart,SegEnd)
Merging can be easily done by using VirtualDubMod itself:
VirtualDub.Open("e:\\segment_1.avi");
VirtualDub.Append("e:\\segment_2.avi");
VirtualDub.video.SetMode(0);
VirtualDub.SaveAVI("e:\\movie.avi");
I could write a middleware similar to QuEnc^n or HCEnc^n for XVid encraw with a pretty similar CLI. This would make implementation of multi-instance encoding easier for anyone willing to support it.
Let me hear what you think !
Best regards,
D$