Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-2 Encoding

Reply
 
Thread Tools Search this Thread Display Modes
Old 4th December 2007, 10:46   #1  |  Link
mikeytown2
Resize Abuser
 
mikeytown2's Avatar
 
Join Date: Apr 2005
Location: Seattle, WA
Posts: 623
avs -> avi -> HCenc... 2 Pass Speed Boost.

I made a script that will take an avs file, outputs an avi, then use that avi in HCenc. It's written in AutoIt, Thread for it
http://www.autoitscript.com/forum/in...howtopic=58691

It's far from complete, but it will take input from the gui or drag and drop files on top of the exe. It can handle multiple files or even a directory as input. Uses HC.ini for HC settings, thus it uses the same settings for all the avs files.

This in in response to my own post
http://forum.doom9.org/showthread.ph...01#post1072001
mikeytown2 is offline   Reply With Quote
Old 4th December 2007, 13:36   #2  |  Link
squid_80
Registered User
 
Join Date: Dec 2004
Location: Melbourne, AU
Posts: 1,963
There's an even better way. Do a search for an avisynth filter called twriteavi. With some creative scripting you can make it write the avi file and do the first pass in HC at the same time, then when the HC second pass starts you can use the avi file for the source.
It sounds pretty cryptic, but is possible - I just don't have a script handy (or the time to write one out) at the moment. The advantage is you remove one instance of avi decoding by performing the encode and the first pass at the same time.
squid_80 is offline   Reply With Quote
Old 4th December 2007, 14:38   #3  |  Link
G_M_C
Registered User
 
Join Date: Feb 2006
Posts: 1,076
Why exactly dont you use the AviSynth for both passes ?

I do, and on my E6750 i get avg 50 ~55 fps on a PAL DVD encode (720x576).
G_M_C is offline   Reply With Quote
Old 4th December 2007, 18:14   #4  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,528
sometimes your AviSynth script is very slow. I mean _VERY_ slow. In these cases, you actually save time by rendering out a lossless AVI ahead of time, and then using that as a source.

~Misfit
Blue_MiSfit is offline   Reply With Quote
Old 4th December 2007, 19:37   #5  |  Link
G_M_C
Registered User
 
Join Date: Feb 2006
Posts: 1,076
Quote:
Originally Posted by Blue_MiSfit View Post
sometimes your AviSynth script is very slow. I mean _VERY_ slow. In these cases, you actually save time by rendering out a lossless AVI ahead of time, and then using that as a source.

~Misfit
Hmmm, i get it;

It can save you 50% time in extreme cases (need to encode the slow AVS script only once). I' ll have to remember that trick
G_M_C is offline   Reply With Quote
Old 4th December 2007, 22:09   #6  |  Link
mikeytown2
Resize Abuser
 
mikeytown2's Avatar
 
Join Date: Apr 2005
Location: Seattle, WA
Posts: 623
@squid_80 Thanks for the tip!

This thread has a lot of talk but no "real" solution
http://forum.doom9.org/showthread.php?t=121869

If we could make a script that has 2 outputs by using twriteavi, then have the script select the lossless source for pass 2. We can accomplish this by using
a try catch block
http://avisynth.org/mediawiki/Control_structures
Or Writing a temp file
http://avisynth.org/mediawiki/WriteFile
Then reading that file
http://avisynth.org/mediawiki/Import
http://avisynth.org/mediawiki/ConditionalReader

If twriteavi could accept FourCC like avs2avi does then this script could be run without user input.
In avisynth Is there a way to get the scripts own name and is there a way see if a file exists?


Another way would be to use
http://avisynth.org/mediawiki/ImageWriter
http://avisynth.org/mediawiki/ImageSource
But this would use even more space then a lossless codec.


Any Thoughts?
mikeytown2 is offline   Reply With Quote
Old 5th December 2007, 07:17   #7  |  Link
squid_80
Registered User
 
Join Date: Dec 2004
Location: Melbourne, AU
Posts: 1,963
Quote:
Originally Posted by mikeytown2 View Post
If twriteavi could accept FourCC like avs2avi does then this script could be run without user input.
Having twriteavi run without user input would be nice, but there needs to be some way to pass parameters to the compression codec.
Quote:
In avisynth Is there a way to get the scripts own name and is there a way see if a file exists?
Exist(filename): returns TRUE or FALSE after checking if the file exists.

So you'd do something like
Code:
exist("output.avi") ? avisource("output.avi") : twriteavi("output.avi")
to have it use output.avi as the source if it already exists (second pass) or use twriteavi to write it (first pass). You'd probably want to do a bit more checking though, like making sure output.avi has the correct number of frames (also make sure HC closes and re-opens the .avs file on second pass start otherwise it won't work at all).
squid_80 is offline   Reply With Quote
Old 5th December 2007, 09:38   #8  |  Link
mikeytown2
Resize Abuser
 
mikeytown2's Avatar
 
Join Date: Apr 2005
Location: Seattle, WA
Posts: 623
Quote:
Originally Posted by squid_80 View Post
Having twriteavi run without user input would be nice, but there needs to be some way to pass parameters to the compression codec.
Because this is a lossless codec, and its a temp avi; passing parameters to the codec IMHO is not a high priority. In my sucky AutoIt Script i didn't pass compression settings to avs2avi.

Sweet looks like most of the stuff i need is in here
http://avisynth.org/mediawiki/Intern...lean_functions
http://avisynth.org/mediawiki/Operators
http://avisynth.org/mediawiki/Clip_properties
http://avisynth.org/mediawiki/Block_statements

A delete file function might be useful? In any case i think i got this working

Code:
Global LFileName = "TempLossless.avi"
Global SourceFile = AVISource("OrginalInput.avi")

exist(LFileName) ? ChkLosslessFile(LFileName) : RunScriptAndOutputLosslessFile(SourceFile)


function ChkLosslessFile(string f)
{
	LossLessFile = AVISource(f)
	(Framecount(SourceFile) == Framecount(LossLessFile)) ? UseLosslessFile(LossLessFile) : RunScriptAndOutputLosslessFile(SourceFile)
}

function UseLosslessFile(clip s)
{
	s = Spline36Resize(s,320,240)
	return s
}

#Put your normal script functions in this function
function RunScriptAndOutputLosslessFile(clip c)
{
	c = Spline36Resize(c,720,480).TWriteAvi(fname=LFileName, overwrite=true, showAll=true)
	Return c
}
TWriteAvi DLL I used
http://bengal.missouri.edu/~kes25c/TWriteAvi.zip

Last edited by mikeytown2; 5th December 2007 at 10:06.
mikeytown2 is offline   Reply With Quote
Old 6th December 2007, 03:59   #9  |  Link
mikeytown2
Resize Abuser
 
mikeytown2's Avatar
 
Join Date: Apr 2005
Location: Seattle, WA
Posts: 623
k i was looking into the source code for TWriteAvi. Seems like all thats needed is to add a new variable to the public class declaration and TWriteAvi function call (const char* _userFourCC) . After this we change this code

Code:
	// select compressor and fill COMPVARS cvar info
	cvar.cbSize = sizeof(COMPVARS);
	if (showAll)
	{
		if(!ICCompressorChoose(NULL, ICMF_CHOOSE_ALLCOMPRESSORS | ICMF_CHOOSE_DATARATE | 
			ICMF_CHOOSE_KEYFRAME | ICMF_CHOOSE_PREVIEW, &bih, NULL, &cvar, (LPSTR)fname)) 
			env->ThrowError("TWriteAvi:  ICCompressorChoose failed!");
	}
	else
	{
		if(!ICCompressorChoose(NULL, ICMF_CHOOSE_DATARATE | ICMF_CHOOSE_KEYFRAME | ICMF_CHOOSE_PREVIEW, 
			&bih, NULL, &cvar, (LPSTR)fname)) env->ThrowError("TWriteAvi:  ICCompressorChoose failed!");
	}
to something like this

Code:
	// select compressor and fill COMPVARS cvar info
	cvar.cbSize = sizeof(COMPVARS);
	if (_userFourCC)
	{
		cvar.fccHandler = *(DWORD*)_userFourCC;
	}
	else if (showAll)
	{
		if(!ICCompressorChoose(NULL, ICMF_CHOOSE_ALLCOMPRESSORS | ICMF_CHOOSE_DATARATE | 
			ICMF_CHOOSE_KEYFRAME | ICMF_CHOOSE_PREVIEW, &bih, NULL, &cvar, (LPSTR)fname)) 
			env->ThrowError("TWriteAvi:  ICCompressorChoose failed!");
	}
	else
	{
		if(!ICCompressorChoose(NULL, ICMF_CHOOSE_DATARATE | ICMF_CHOOSE_KEYFRAME | ICMF_CHOOSE_PREVIEW, 
			&bih, NULL, &cvar, (LPSTR)fname)) env->ThrowError("TWriteAvi:  ICCompressorChoose failed!");
	}
Any idea if this would work? Anyone want to try and compile this? I found this code in avs2avi... looks like we might need it.
http://www.avs2avi.org/

Code:
void ChooseOneCompressorOnFourCC(const char* FourCC, COMPVARS *cvar)
{
	ZeroMemory(cvar, sizeof(COMPVARS));
	cvar->cbSize = sizeof(COMPVARS);
	cvar->dwFlags = ICMF_COMPVARS_VALID;
	cvar->fccType = ICTYPE_VIDEO;
	if (stricmp(FourCC, "null") == 0) {
		// No Recompression
		cvar->fccHandler = FCC_NULL;
		cvar->hic = NULL;
	}
	else {
		cvar->fccHandler = *(DWORD*)FourCC;
		cvar->hic = ICOpen(cvar->fccType, cvar->fccHandler, ICMODE_COMPRESS);
		if(cvar->hic == NULL) {
			ShowErrorAndExit("There is no codec with FourCC \"%s\"!", FourCC);
		}
	}
	cvar->lKey = 15;
	cvar->lQ = ICQUALITY_DEFAULT;
}
mikeytown2 is offline   Reply With Quote
Old 6th December 2007, 05:54   #10  |  Link
squid_80
Registered User
 
Join Date: Dec 2004
Location: Melbourne, AU
Posts: 1,963
The avs2avi code is nearly perfect, except it sets lKey to 15 for some odd reason. Better to set it to 0 to let the codec decide. MSDN is typically vague when if you look up the COMPVARS structure; it says the lpbiOut member must be filled out, but setting it to NULL is acceptable.
Code would need to be added to make sure the codec returned by ICOpen actually accepts the input format (bih). This could be done with a call to ICCompressQuery.

If nobody else is up to it, I'll be able to do these changes in a few hours time.
squid_80 is offline   Reply With Quote
Old 6th December 2007, 09:25   #11  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,929
katjarella reported in the german board:

She gets a temporary result which is 2 frames longer than the original.
__

P.S.:

Also she reports that AVS2AVI reliably crashes when she uses an AviSynth script as source which only contains

AviSource("lossless.avi")

so she can only use VirtualDubMod (and the joblist) to automate 2 passes.

Last edited by LigH; 6th December 2007 at 09:55.
LigH is offline   Reply With Quote
Old 6th December 2007, 12:05   #12  |  Link
mikeytown2
Resize Abuser
 
mikeytown2's Avatar
 
Join Date: Apr 2005
Location: Seattle, WA
Posts: 623
@LigH: Thanks for the info! AVS2AVI has given me problems as well, but mainly when i use SoundOut(output="AC3"...). As for the issues, what codec is being used, and what are the codec settings? AviSynth version?

Using the google language translator i was able to understand
http://translate.google.com/translat...hl=en&ie=UTF-8
I wonder if this AVS2AVI bug has anything to do with the color space of the video? What if they pass the video as a clip variable using
AVISource("S01E00.avs").TWriteAvi("TempLossless.avi",overwrite=true,showAll=true)

Also does the error still show up if you put TWriteAvi at the end of S01E00.avs? If S01E00.avs uses DirectShowSource(), this could be causing the problem.


@squid_80: I've never made an AviSynth dll, so if you have some spare time, and you're up to the challenge, go for it!


@Moderator: I'm not sure, but this thread is starting to look like it might need to go into one of the Avisynth ones.
mikeytown2 is offline   Reply With Quote
Old 6th December 2007, 12:41   #13  |  Link
katjarella
woman @ german doom9
 
katjarella's Avatar
 
Join Date: Feb 2003
Location: Germany
Posts: 117
@mikeytown2
AVISynth Version: 2.57, Build:Dec 31 2006 [13:16:28]

Sorry not 2 frames longer, 1 frames longer.

source.avs:
Code:
BlankClip(width=512,height=368,fps=25,length=800,pixel_type="YUY2")
ShowFrameNumber(x=40,y=40,size=32)
source_P1.avs:
Code:
LoadPlugin("***path***\TWriteAvi\TWriteAvi.dll")
AVISource("source.avs",false).TWriteAvi("TempLossless.avi",overwrite=true,showAll=true)
VDubJop:
Code:
VirtualDub.Open("source_P1.avs","",0);
VirtualDub.RemoveInputStreams();
VirtualDub.video.DeleteComments(1);
VirtualDub.video.AdjustChapters(1);
VirtualDub.video.SetDepth(24,24);
VirtualDub.video.SetMode(1);
VirtualDub.video.SetFrameRate(0,1);
VirtualDub.video.SetIVTC(0,0,-1,0);
VirtualDub.video.SetCompression(0x75796668,0,10000,0);
VirtualDub.video.filters.Clear();
VirtualDub.SaveAVI("final.avi");
VirtualDub.Close();
Codec Settings is HFYU or Lagarith

The source.avs has 0-799 == 800Frames OK
The source_P1.avs has 0-799 == 800Frames OK
The source_P1.avs has 0-799 == 800Frames OK
The final.avi has 0-799 == 800Frames OK
The TempLossless.avi has 0-799,0 == 801Frames Fail
__________________
katjarella by Creyskull Visuell Design
katjarella is offline   Reply With Quote
Old 6th December 2007, 12:59   #14  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,929
"0-799,0" reads like the first frame is added again while "rewinding" the video?
__

P.S.: Instead of AviSource("*.avs"), may Import("*.avs") work too? Of course, respecting the possible side effects.

Last edited by LigH; 6th December 2007 at 13:01.
LigH is offline   Reply With Quote
Old 6th December 2007, 13:34   #15  |  Link
squid_80
Registered User
 
Join Date: Dec 2004
Location: Melbourne, AU
Posts: 1,963
I think one of the modifications I made to twriteavi was to make sure each frame of the original video is only written out to the avi once. Otherwise it writes a frame every time the host app pulls one from the script, which is probably what is happening here.
There's also the issue of frames being pulled out of order, I made the idxname option for that to be used with stickboy's remapframes when loading the temp avi back in.

(see this thread or direct download. This isn't the new "auto fourcc" version, just the original one modified slightly.)
squid_80 is offline   Reply With Quote
Old 6th December 2007, 17:26   #16  |  Link
squid_80
Registered User
 
Join Date: Dec 2004
Location: Melbourne, AU
Posts: 1,963
OK I've made some changes: http://members.optusnet.com.au/squid_80/twriteavi.zip

- New parameter called fourcc, to specify a codec without user intervention
- Moved the index file creation so it's not created if compression can't be started
- Once all frames have been written out, close the avi file and index. This stops programs which re-open the script before they close it from complaining (I'm talking about you, virtualdub). Only works if all frames have been processed!

Here's the script I'm using:
Code:
# loadplugin("blah\blah\blah\twriteavi.dll")
# loadplugin("blah\blah\blah\remapframes.dll")

function CachedFile(clip c, string filename, string "_fourcc") {
	_fourcc = default(_fourcc, "")
	s = (exist(filename) ? ChkLosslessFile(c, filename): NOP())
	IsClip(s) ? s \
		: twriteavi(c, filename, overwrite=true, idxname=filename+".idx", fourcc=_fourcc)
}

function ChkLosslessFile(clip c, string filename) {
	LossLessFile = avisource(filename)
	c.framecount == LossLessFile.framecount ? LossLessFile.RemapFrames(filename+".idx") \
		: NOP()
}

### START OF MAIN SCRIPT

avisource("fixed.avi")
trim(0, -2000)

### END OF MAIN SCRIPT

CachedFile("test.avi", "hfyu") #huffyuv
squid_80 is offline   Reply With Quote
Old 6th December 2007, 20:37   #17  |  Link
katjarella
woman @ german doom9
 
katjarella's Avatar
 
Join Date: Feb 2003
Location: Germany
Posts: 117
@mikeytown2 and squid_80
My first tests, function wonderful (inc Batch). BIG Thanks.
Code:
avs2avi source_TMP.avs final.avi -P 2 -p 0 -w -l DIVX2P.cfg

AVS2AVI v1.40 (c) 2002-2006:
Christophe Paris, David Leatherdale, int21h, Moitah
Modified by Argos, http://www.sasteam66.org/
http://www.avs2avi.org/

Source:
  * Filename: "***\TWriteAvi\source_TMP.avs"
  * FourCC: YUY2
  * Frames: 2000
  * Resolution: 512x368
  * Frame rate: 25.000 FPS
Compressor:
  * Name: DivX 6.8 Codec (2 Logical CPUs)
  * FourCC: divx
Destination:
  * Filename: "***\TWriteAvi\final.avi"
  * Pass 1/2: Finished in 00:04:07.589 (8.08 FPS)
  * Pass 2/2: Finished in 00:01:20.385 (24.88 FPS)
  * Frames: 2000 (49 keyframes)
  * Size: 11.49 MB
__________________
katjarella by Creyskull Visuell Design
katjarella is offline   Reply With Quote
Old 6th December 2007, 22:50   #18  |  Link
mikeytown2
Resize Abuser
 
mikeytown2's Avatar
 
Join Date: Apr 2005
Location: Seattle, WA
Posts: 623
Sweet!!! Thanks squid_80, your script and dll work great! Also thanks to everyone else for their time and effort!

remapframes
http://avisynth.org/stickboy/

Redid my AutoIt post. Made it into a very simple batch encoder
http://www.autoitscript.com/forum/in...dpost&p=444330
mikeytown2 is offline   Reply With Quote
Old 7th December 2007, 09:17   #19  |  Link
squid_80
Registered User
 
Join Date: Dec 2004
Location: Melbourne, AU
Posts: 1,963
Hmm I left something very minor out of my script...
Code:
function ChkLosslessFile(clip c, string filename) {
	LossLessFile = avisource(filename)
	c.framecount == LossLessFile.framecount ? LossLessFile.RemapFrames(filename+".idx").audiodubex(c) \
		: NOP()
}
squid_80 is offline   Reply With Quote
Old 8th December 2007, 04:36   #20  |  Link
EasyStart
Registered User
 
Join Date: Apr 2002
Posts: 47
I don't know what I've done wrong. Dowloaded mikeytown2's hc_batch.exe and squid_80's script. When I try to run HC to do a 2 pass encoding job from hc_batch.exe, my second pass is only 2 seonds faster than the first pass. My pc is an overclocked Core2Quad Q6600 with 2gb ram running win xp pro.

Easystart
EasyStart is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 04:29.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.