avs -> avi -> HCenc... 2 Pass Speed Boost.

[mikeytown2]

I made a script that will take an avs file, outputs an avi, then use that avi in HCenc. It's written in AutoIt, Thread for it
http://www.autoitscript.com/forum/in...howtopic=58691

It's far from complete, but it will take input from the gui or drag and drop files on top of the exe. It can handle multiple files or even a directory as input. Uses HC.ini for HC settings, thus it uses the same settings for all the avs files.

This in in response to my own post
http://forum.doom9.org/showthread.ph...01#post1072001

[squid_80]

There's an even better way. Do a search for an avisynth filter called twriteavi. With some creative scripting you can make it write the avi file and do the first pass in HC at the same time, then when the HC second pass starts you can use the avi file for the source.
It sounds pretty cryptic, but is possible - I just don't have a script handy (or the time to write one out) at the moment. The advantage is you remove one instance of avi decoding by performing the encode and the first pass at the same time.

[G_M_C]

Why exactly dont you use the AviSynth for both passes ?

I do, and on my E6750 i get avg 50 ~55 fps on a PAL DVD encode (720x576).

[Blue_MiSfit]

sometimes your AviSynth script is very slow. I mean _VERY_ slow. In these cases, you actually save time by rendering out a lossless AVI ahead of time, and then using that as a source.

~Misfit

[G_M_C]

Originally Posted by Blue_MiSfit

sometimes your AviSynth script is very slow. I mean _VERY_ slow. In these cases, you actually save time by rendering out a lossless AVI ahead of time, and then using that as a source.

~Misfit

Hmmm, i get it;

It can save you 50% time in extreme cases (need to encode the slow AVS script only once). I' ll have to remember that trick

[mikeytown2]

@squid_80 Thanks for the tip!

This thread has a lot of talk but no "real" solution
http://forum.doom9.org/showthread.php?t=121869

If we could make a script that has 2 outputs by using twriteavi, then have the script select the lossless source for pass 2. We can accomplish this by using
a try catch block
http://avisynth.org/mediawiki/Control_structures
Or Writing a temp file
http://avisynth.org/mediawiki/WriteFile
Then reading that file
http://avisynth.org/mediawiki/Import
http://avisynth.org/mediawiki/ConditionalReader

If twriteavi could accept FourCC like avs2avi does then this script could be run without user input.
In avisynth Is there a way to get the scripts own name and is there a way see if a file exists?


Another way would be to use
http://avisynth.org/mediawiki/ImageWriter
http://avisynth.org/mediawiki/ImageSource
But this would use even more space then a lossless codec.


Any Thoughts?

[squid_80]

Originally Posted by mikeytown2

If twriteavi could accept FourCC like avs2avi does then this script could be run without user input.

Having twriteavi run without user input would be nice, but there needs to be some way to pass parameters to the compression codec.

In avisynth Is there a way to get the scripts own name and is there a way see if a file exists?

Exist(filename): returns TRUE or FALSE after checking if the file exists.

So you'd do something like

exist("output.avi") ? avisource("output.avi") : twriteavi("output.avi")

to have it use output.avi as the source if it already exists (second pass) or use twriteavi to write it (first pass). You'd probably want to do a bit more checking though, like making sure output.avi has the correct number of frames (also make sure HC closes and re-opens the .avs file on second pass start otherwise it won't work at all).

[mikeytown2]

Originally Posted by squid_80

Having twriteavi run without user input would be nice, but there needs to be some way to pass parameters to the compression codec.

Because this is a lossless codec, and its a temp avi; passing parameters to the codec IMHO is not a high priority. In my sucky AutoIt Script i didn't pass compression settings to avs2avi.

Sweet looks like most of the stuff i need is in here
http://avisynth.org/mediawiki/Intern...lean_functions
http://avisynth.org/mediawiki/Operators
http://avisynth.org/mediawiki/Clip_properties
http://avisynth.org/mediawiki/Block_statements

A delete file function might be useful? In any case i think i got this working

Global LFileName = "TempLossless.avi"
Global SourceFile = AVISource("OrginalInput.avi")

exist(LFileName) ? ChkLosslessFile(LFileName) : RunScriptAndOutputLosslessFile(SourceFile)


function ChkLosslessFile(string f)
{
	LossLessFile = AVISource(f)
	(Framecount(SourceFile) == Framecount(LossLessFile)) ? UseLosslessFile(LossLessFile) : RunScriptAndOutputLosslessFile(SourceFile)
}

function UseLosslessFile(clip s)
{
	s = Spline36Resize(s,320,240)
	return s
}

#Put your normal script functions in this function
function RunScriptAndOutputLosslessFile(clip c)
{
	c = Spline36Resize(c,720,480).TWriteAvi(fname=LFileName, overwrite=true, showAll=true)
	Return c
}

TWriteAvi DLL I used
http://bengal.missouri.edu/~kes25c/TWriteAvi.zip

[mikeytown2]

k i was looking into the source code for TWriteAvi. Seems like all thats needed is to add a new variable to the public class declaration and TWriteAvi function call (const char* _userFourCC) . After this we change this code

	// select compressor and fill COMPVARS cvar info
	cvar.cbSize = sizeof(COMPVARS);
	if (showAll)
	{
		if(!ICCompressorChoose(NULL, ICMF_CHOOSE_ALLCOMPRESSORS | ICMF_CHOOSE_DATARATE | 
			ICMF_CHOOSE_KEYFRAME | ICMF_CHOOSE_PREVIEW, &bih, NULL, &cvar, (LPSTR)fname)) 
			env->ThrowError("TWriteAvi:  ICCompressorChoose failed!");
	}
	else
	{
		if(!ICCompressorChoose(NULL, ICMF_CHOOSE_DATARATE | ICMF_CHOOSE_KEYFRAME | ICMF_CHOOSE_PREVIEW, 
			&bih, NULL, &cvar, (LPSTR)fname)) env->ThrowError("TWriteAvi:  ICCompressorChoose failed!");
	}

to something like this

	// select compressor and fill COMPVARS cvar info
	cvar.cbSize = sizeof(COMPVARS);
	if (_userFourCC)
	{
		cvar.fccHandler = *(DWORD*)_userFourCC;
	}
	else if (showAll)
	{
		if(!ICCompressorChoose(NULL, ICMF_CHOOSE_ALLCOMPRESSORS | ICMF_CHOOSE_DATARATE | 
			ICMF_CHOOSE_KEYFRAME | ICMF_CHOOSE_PREVIEW, &bih, NULL, &cvar, (LPSTR)fname)) 
			env->ThrowError("TWriteAvi:  ICCompressorChoose failed!");
	}
	else
	{
		if(!ICCompressorChoose(NULL, ICMF_CHOOSE_DATARATE | ICMF_CHOOSE_KEYFRAME | ICMF_CHOOSE_PREVIEW, 
			&bih, NULL, &cvar, (LPSTR)fname)) env->ThrowError("TWriteAvi:  ICCompressorChoose failed!");
	}

Any idea if this would work? Anyone want to try and compile this? I found this code in avs2avi... looks like we might need it.
http://www.avs2avi.org/

void ChooseOneCompressorOnFourCC(const char* FourCC, COMPVARS *cvar)
{
	ZeroMemory(cvar, sizeof(COMPVARS));
	cvar->cbSize = sizeof(COMPVARS);
	cvar->dwFlags = ICMF_COMPVARS_VALID;
	cvar->fccType = ICTYPE_VIDEO;
	if (stricmp(FourCC, "null") == 0) {
		// No Recompression
		cvar->fccHandler = FCC_NULL;
		cvar->hic = NULL;
	}
	else {
		cvar->fccHandler = *(DWORD*)FourCC;
		cvar->hic = ICOpen(cvar->fccType, cvar->fccHandler, ICMODE_COMPRESS);
		if(cvar->hic == NULL) {
			ShowErrorAndExit("There is no codec with FourCC \"%s\"!", FourCC);
		}
	}
	cvar->lKey = 15;
	cvar->lQ = ICQUALITY_DEFAULT;
}

[squid_80]

The avs2avi code is nearly perfect, except it sets lKey to 15 for some odd reason. Better to set it to 0 to let the codec decide. MSDN is typically vague when if you look up the COMPVARS structure; it says the lpbiOut member must be filled out, but setting it to NULL is acceptable.
Code would need to be added to make sure the codec returned by ICOpen actually accepts the input format (bih). This could be done with a call to ICCompressQuery.

If nobody else is up to it, I'll be able to do these changes in a few hours time.

[LigH]

katjarella reported in the german board:

She gets a temporary result which is 2 frames longer than the original.
__

P.S.:

Also she reports that AVS2AVI reliably crashes when she uses an AviSynth script as source which only contains

AviSource("lossless.avi")

so she can only use VirtualDubMod (and the joblist) to automate 2 passes.

[mikeytown2]

@LigH: Thanks for the info! AVS2AVI has given me problems as well, but mainly when i use SoundOut(output="AC3"...). As for the issues, what codec is being used, and what are the codec settings? AviSynth version?

Using the google language translator i was able to understand
http://translate.google.com/translat...hl=en&ie=UTF-8
I wonder if this AVS2AVI bug has anything to do with the color space of the video? What if they pass the video as a clip variable using
AVISource("S01E00.avs").TWriteAvi("TempLossless.avi",overwrite=true,showAll=true)

Also does the error still show up if you put TWriteAvi at the end of S01E00.avs? If S01E00.avs uses DirectShowSource(), this could be causing the problem.


@squid_80: I've never made an AviSynth dll, so if you have some spare time, and you're up to the challenge, go for it!


@Moderator: I'm not sure, but this thread is starting to look like it might need to go into one of the Avisynth ones.

[katjarella]

@mikeytown2
AVISynth Version: 2.57, Build:Dec 31 2006 [13:16:28]

Sorry not 2 frames longer, 1 frames longer.

source.avs:

BlankClip(width=512,height=368,fps=25,length=800,pixel_type="YUY2")
ShowFrameNumber(x=40,y=40,size=32)

source_P1.avs:

LoadPlugin("***path***\TWriteAvi\TWriteAvi.dll")
AVISource("source.avs",false).TWriteAvi("TempLossless.avi",overwrite=true,showAll=true)

VDubJop:

VirtualDub.Open("source_P1.avs","",0);
VirtualDub.RemoveInputStreams();
VirtualDub.video.DeleteComments(1);
VirtualDub.video.AdjustChapters(1);
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(1);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetCompression(0x75796668,0,10000,0);
VirtualDub.video.filters.Clear();
VirtualDub.SaveAVI("final.avi");
VirtualDub.Close();

Codec Settings is HFYU or Lagarith

The source.avs has 0-799 == 800Frames OK
The source_P1.avs has 0-799 == 800Frames OK
The source_P1.avs has 0-799 == 800Frames OK
The final.avi has 0-799 == 800Frames OK
The TempLossless.avi has 0-799,0 == 801Frames Fail

[LigH]

"0-799,0" reads like the first frame is added again while "rewinding" the video?
__

P.S.: Instead of AviSource("*.avs"), may Import("*.avs") work too? Of course, respecting the possible side effects.

[squid_80]

I think one of the modifications I made to twriteavi was to make sure each frame of the original video is only written out to the avi once. Otherwise it writes a frame every time the host app pulls one from the script, which is probably what is happening here.
There's also the issue of frames being pulled out of order, I made the idxname option for that to be used with stickboy's remapframes when loading the temp avi back in.

(see this thread or direct download. This isn't the new "auto fourcc" version, just the original one modified slightly.)

[squid_80]

OK I've made some changes: http://members.optusnet.com.au/squid_80/twriteavi.zip

- New parameter called fourcc, to specify a codec without user intervention
- Moved the index file creation so it's not created if compression can't be started
- Once all frames have been written out, close the avi file and index. This stops programs which re-open the script before they close it from complaining (I'm talking about you, virtualdub). Only works if all frames have been processed!

Here's the script I'm using:

# loadplugin("blah\blah\blah\twriteavi.dll")
# loadplugin("blah\blah\blah\remapframes.dll")

function CachedFile(clip c, string filename, string "_fourcc") {
	_fourcc = default(_fourcc, "")
	s = (exist(filename) ? ChkLosslessFile(c, filename): NOP())
	IsClip(s) ? s \
		: twriteavi(c, filename, overwrite=true, idxname=filename+".idx", fourcc=_fourcc)
}

function ChkLosslessFile(clip c, string filename) {
	LossLessFile = avisource(filename)
	c.framecount == LossLessFile.framecount ? LossLessFile.RemapFrames(filename+".idx") \
		: NOP()
}

### START OF MAIN SCRIPT

avisource("fixed.avi")
trim(0, -2000)

### END OF MAIN SCRIPT

CachedFile("test.avi", "hfyu") #huffyuv

[katjarella]

@mikeytown2 and squid_80
My first tests, function wonderful (inc Batch). BIG Thanks.

avs2avi source_TMP.avs final.avi -P 2 -p 0 -w -l DIVX2P.cfg

AVS2AVI v1.40 (c) 2002-2006:
Christophe Paris, David Leatherdale, int21h, Moitah
Modified by Argos, http://www.sasteam66.org/
http://www.avs2avi.org/

Source:
  * Filename: "***\TWriteAvi\source_TMP.avs"
  * FourCC: YUY2
  * Frames: 2000
  * Resolution: 512x368
  * Frame rate: 25.000 FPS
Compressor:
  * Name: DivX« 6.8 Codec (2 Logical CPUs)
  * FourCC: divx
Destination:
  * Filename: "***\TWriteAvi\final.avi"
  * Pass 1/2: Finished in 00:04:07.589 (8.08 FPS)
  * Pass 2/2: Finished in 00:01:20.385 (24.88 FPS)
  * Frames: 2000 (49 keyframes)
  * Size: 11.49 MB

[mikeytown2]

Sweet!!! Thanks squid_80, your script and dll work great! Also thanks to everyone else for their time and effort!

remapframes
http://avisynth.org/stickboy/

Redid my AutoIt post. Made it into a very simple batch encoder
http://www.autoitscript.com/forum/in...dpost&p=444330

[squid_80]

Hmm I left something very minor out of my script...

function ChkLosslessFile(clip c, string filename) {
	LossLessFile = avisource(filename)
	c.framecount == LossLessFile.framecount ? LossLessFile.RemapFrames(filename+".idx").audiodubex(c) \
		: NOP()
}

[EasyStart]

I don't know what I've done wrong. Dowloaded mikeytown2's hc_batch.exe and squid_80's script. When I try to run HC to do a 2 pass encoding job from hc_batch.exe, my second pass is only 2 seonds faster than the first pass. My pc is an overclocked Core2Quad Q6600 with 2gb ram running win xp pro.

Easystart