xvid_encraw - Patched with AviSynth input support

[squid_80]

If it helps you any, -o will soon become the only option needed for output i.e. -o file.avi will create an avi file and -o file.mkv will create a mkv file. -o any_other_extension.xyz will still create a raw m4v stream.
-avi and -mkv will still be there, this is just to help out MeGUI users who keep getting stuck with raw m4v files which look like something else thanks to incorrect file extensions.

[Brother John]

Thx for the new version, squid_80. I've one question, though.

In your example above you mention -packed to enable packed bframes. Encraw's online help only mentions -nopacked to disable packed streams. No I'm wondering what is the proper way to en/disabled packed bitstream?

[SeeMoreDigital]

Thanks, when I've got a couple of hours time I'll make a TreeView based GUI for StaxRip.If it becomes your intension to incorporate this into StaxRip, by all means please let me know by PM and I'll be happy to test it for you before going live :)


Cheers

[squid_80]

In your example above you mention -packed to enable packed bframes. Encraw's online help only mentions -nopacked to disable packed streams. No I'm wondering what is the proper way to en/disabled packed bitstream?
Deja vu... (http://forum.doom9.org/showthread.php?p=803819#post803819) Packed bitstream is off by default unless outputting to an .avi file, which is not the case in my first pass. But as my second pass target is a packed .avi file, for best results the stats file figures SHOULD include the fake n-vops produced by packed bitstream. It's not absolutely necessary, filesize and quality will probably be more or less the same either way but I'm pedantic.

Also -packed was an option of the original xvid_encraw, hence it remains for compatability.

[SeeMoreDigital]

If say, I wanted to generate an encode with b-frames straight to MP4 (ie: packed bitstream free). How will the b-frames be ordered, in relation to the I and P frames?


Cheers

[squid_80]

Umm, however mp4creator does it? That's where the code is coming from.

[foxyshadis]

mp4 always has to be in jumbled mpeg order anyway, splitters would probably crash if you somehow shoehorned packed into mp4.


IPBBPBBPBIPPBPP
031264587902134

[SeeMoreDigital]

Umm, however mp4creator does it? That's where the code is coming from.
mp4 always has to be in jumbled mpeg order anyway, splitters would probably crash if you somehow shoehorned packed into mp4.


IPBBPBBPBIPPBPP
031264587902134
So to confirm... the B frame order for MP4 will not be the same as it is when they are hacked into AVI?


Cheers

[squid_80]

No, that is how they're stored in avi. Except in packed mode, the first B frame in a run is stored packed with the preceeding reference frame and an extra P (n-vop) is stored at the tail of the run.

Packed mode only exists for avi, for mkv and mp4 there is only one way to do things.

[SeeMoreDigital]

Thanks Squid.... :D

[Brother John]

Thanks Squid.... :D
Thx from me as well. Your answer saved me some boring testing. ;)

Btw: I've updated the enraw mirror on my site (http://brother-john.net) to the current version.

[bond]

Umm, however mp4creator does it? That's where the code is coming from.is xvid_encraw now supposed to support .mp4 output? if yes how?

[squid_80]

is xvid_encraw now supposed to support .mp4 output? if yes how?
Not yet. "That's where the code is coming from," means it's on its way but not quite here yet. But it would be just like me to release a new version and not say anything. :)

[SeeMoreDigital]

Not yet. "That's where the code is coming from," means it's on its way but not quite here yet. But it would be just like me to release a new version and not say anything. :)Great... I'm looking forward to an .MP4 output support :)

[bond]

:thanks:

[MatMaul]

I think I found a very minimal bug :
C:\Documents and Settings\MatMaul\Bureau\xvid_encraw>xvid_encraw -i "D:\lotr1\vi
deo.avs" -type 2 -pass2 -bitrate 2863 -max_bframes 2 -bquant_ratio 162 -bquant_o
ffset 0 -zones 0,w,1,O -quality 6 -vhqmode 4 -bvhq -qpel -qmatrix "SixOfNine.cqm
" -nopacked -imin 2 -imax 5 -pmin 2 -pmax 5 -bmin 2 -bmax 5 -chigh 10 -clow 3 -m
ax_key_interval 250 -progress -mkv "D:\lotr1\video\lotr1.mkv"
xvid_encraw - raw mpeg4 bitstream encoder written by Christoph Lampert 2002-2003


Trying to retrieve width and height from input header
Input colorspace is YV12
xvidcore build version: xvid-1.2.0-dev
Bitstream version: 1.2.-127
Detected CPU flags: ASM MMX MMXEXT SSE SSE2 TSC
Detected 1 cpus, using 1 threads.
288651 frames(100%) encoded, 6.68 fps, Average Bitrate = -113kbps
Tot: enctime(ms) =43188075.00, length(bytes) = -163231213
Avg: enctime(ms) = 149.62, fps = 6.68, length(bytes) = -565
I frames: 3977 frames, size = 53748/213756687, quants = 2 / 2.33 / 3
P frames: 119693 frames, size = -12651/-1514325311, quants = 2 / 2.66 / 4
B frames: 164825 frames, size = 6900/1137336171, quants = 3 / 3.64 / 5
N frames: 155 frames, size = 8/ 1240

The avg bitrate reported by the log is negative.
The real bitrate of this encoded video is good, so it's not very important.

[squid_80]

Yes, the variables that hold the sizes are signed int32s so when the output grows larger than 2gb they wrap around to negative values. This throws out all the average calculations. The sensible thing to do would probably be for me to change them to uint64s.
The bitrate should be pretty spot on though, I'm guessing the final size was around 3940M? And the source has a framerate of 25fps?

[MatMaul]

yes the real final bitrate is good.

[berrinam]

Hi there squid_80,

Would it be possible to make the messaging system in xvid_encraw more informative? I have something in mind like how x264.exe does it:
x264 [error/warning/info]: message
where error means a fatal error, warning means a non-fatal error, and info means info. Then it would mean that a parser (like MeGUI) could tell whether the encode error'd out or not.

It could also be a good way of choosing the verbosity of messages: the user can choose which level of message they want in the commandline, and xvid_encraw.exe then only prints messages of that level or of more importance.

[squid_80]

Certainly. I've been meaning to do that for a while, the only reason I haven't changed any of the output messages was in case some programs used the original encraw and modifying the messages would break compatability. I'll go poke around in x264 and figure out what constitutes a fatal/non-fatal/info message so things match up nicely.

[acrespo]

I don't know if you saw the MEGUI bug report thread but I will post here again as I think that is a xvid_encraw problem. I receive this error when try to encode a avisynth script:

comand line:
-i "C:\encodes\ygh1.AVS" -pass1 "C:\encodes\ygh1.stats" -bitrate 606 -kboost 100 -overhead 0 -turbo -vhqmode 1 -closed_gop -imax 31 -max_bframes 2 -bmax 31 -threads 1

Result:
Trying to retrieve width and height from input header
Wrong Framerate 0.000000
xvid_encraw - raw mpeg4 bitstream encoder written by Christoph Lampert 2002-2003

I don't know what's wrong, because I can open this script in virtualdubmod and media player classic.

[squid_80]

I saw it, but I really need to know exactly what is in the avisynth script.

[acrespo]

Here is the script:


SetMemoryMax(64)
LoadPlugin("c:\encodes\filtros\EEDI2.dll")
LoadPlugin("c:\encodes\filtros\tdeint.dll")
LoadPlugin("c:\encodes\filtros\dgdecode.dll")
LoadPlugin("c:\encodes\filtros\tivtc.dll")
LoadPlugin("c:\encodes\filtros\dup.dll")
LoadPlugin("c:\encodes\filtros\RemoveDirt10SSE2.dll")
LoadPlugin("c:\encodes\filtros\RemoveGrain10SSE2.dll")
LoadPlugin("c:\encodes\filtros\RepairSSE2.dll")
LoadPlugin("c:\encodes\filtros\masktools-v2.0a28.dll")
LoadPlugin("c:\encodes\filtros\gradfun2db.dll")
LoadPlugin("c:\encodes\filtros\frfun7.dll")

import("c:\encodes\filtros\RemoveDust.avs")
import("c:\encodes\filtros\FastLineDarken_1.3.avs")

MPEG2Source("c:\encodes\2006-04-01\ygh2.d2v",cpu=6)

AssumeTFF()
Interp = SeparateFields().EEDI2(field=-2)
Deinted = TDeint(mode=2,edeint=Interp)
TFM(PP=7,Clip2=Deinted)
TDecimate(mode=1)

ColorYUV(levels="PC->TV")
Crop(8,8,-8,-8,align=true)
Spline16Resize(320,240)
RemoveDirt()
frfun7()

FastLineDarken2(thinning=0)
dup(threshold=1,blend=true,maxcopies=8)
gradfun2db()

str = "WAS Hub - dchub://wasbr.com - " + Time("%d") + "/" + Time("%m") + "/" + Time("%Y")
Subtitle (str,15,10,0,240,"Courier New", 14)
Subtitle ("WAS",258,202,0,300000,"Courier New",20)

[berrinam]

squid: Basically, the fatal errors in x264 are the ones in which there is no way to continue, like the video is stuffed or the log file is missing, etc. The warnings tend to be settings that are wrong for some reason, but can be fixed by x264, so the warning just tells you that it isn't encoding quite with the settings you asked for.

[squid_80]

acrespo: I can't find anything wrong. Can you do me a favour, add info() to the bottom of that script, open it in virtualdub(mod) and tell me what it says on the Frames per second line? Specifically what the numbers in the brackets are.

[acrespo]

Here is the exact information extracted from info() filter in avisynth:

Frame: 0
ColorSpace: YV12
Width: 320 pixels, Height: 240 pixels.
Frames per second: 23.9760
FieldBased (Separated) Video: NO
Parity: Assumed Top Field First
Video Pitch: 640 bytes
Has Audio: NO
Audio Channels: 0
Sample Type: NONE
Samples Per Second: 0
CPU detected: x87 MMX ISSE SSE SSE2 SSE3

[squid_80]

The only thing I can suggest is updating avisynth and dgindex/dgdecode; I think this might be related to the way avisynth used to give some framerates large numerator/denominator values.

[bond]

The only thing I can suggest is updating avisynth and dgindex/dgdecode; I think this might be related to the way avisynth used to give some framerates large numerator/denominator values.avisynth 2.5.7alpha doesnt do this anymore

[squid_80]

True. It also displays dwrate and dwscale via info(), which is what I really need because I can't reproduce this error. Closest I can get is a crash in xvidcore when dwscale is >2^31.

[sysKin]

I want to put it in CVS.
However, I was crazy enough to try to compile it first >_< and, well, it doesn't because it's trying to use c++ syntax in c file.

What do I do about IID_IAvisynthClipInfo to make it C? :)

[squid_80]

I just compile it as c++ :D
I did try to convert that code to C, but couldn't get it to work. However I'll try again and see how it goes.

[squid_80]

Looks like this works, but I dunno if it's completely correct. First define the interface inside the #ifdef XVID_AVI_INPUT block:

#ifdef XVID_AVI_INPUT
/* Avisynth related */
const GUID IID_IAvisynthClipInfo // {E6D6B708-124D-11D4-86F3-DB80AFD98778}
= {0xe6d6b708, 0x124d, 0x11d4, {0x86, 0xf3, 0xdb, 0x80, 0xaf, 0xd9, 0x87, 0x78}};

#undef INTERFACE
#define INTERFACE IAvisynthClipInfo
DECLARE_INTERFACE_ (INTERFACE, IUnknown)
{
STDMETHOD (QueryInterface) (THIS_ REFIID, void **) PURE;
STDMETHOD_ (ULONG, AddRef) (THIS) PURE;
STDMETHOD_ (ULONG, Release) (THIS) PURE;
STDMETHOD_ (INT, GetError) (THIS_ char**) PURE;
STDMETHOD_ (LONG, GetParity) (THIS_ INT) PURE;
STDMETHOD_ (LONG, IsFieldBased) (THIS) PURE;
};
#endif

(Bool wouldn't work, so I used LONG for the return type of GetParity and IsFieldBased. Shouldn't be relevant, they're not used here.)
Then change the avisynth checking stuff to match the new interface:

if (FAILED(avi_file->lpVtbl->QueryInterface(avi_file, &IID_IAvisynthClipInfo, (void **)&pAvisynthClipInfo)))
pAvisynthClipInfo = NULL;
else {
const char *s;

if (pAvisynthClipInfo->lpVtbl->GetError(pAvisynthClipInfo, &s)) {
fprintf(stderr, "xvid [error]: Avisynth open failure:\n%s\n", s);
pAvisynthClipInfo->lpVtbl->Release(pAvisynthClipInfo);
AVIFileRelease(avi_file);
AVIFileExit();
return (-1);
}
}


if (pAvisynthClipInfo != NULL)
pAvisynthClipInfo->lpVtbl->Release(pAvisynthClipInfo);

Then there were a few spots where I've declared and assigned variables in the middle of code which plain C doesn't like. Easy enough, just move the declarations up.

[sysKin]

Looks like this works, but I dunno if it's completely correct.
Well, it didn't work :/

I removed the avisynth error catching and commited the file.

I officially declare this is the ugliest piece of code in XviD's CVS. It's technically incorrect (static variables), hackish to the bone, and probably not cross-platform at all.

It needs an urgent rewrite. If doing it properly requires xvid interface changes (wrt packed bitstream and userdata) then so be it. Better that than hackish removal of userdata from encoded frames.

Volunteers?....

[shpitz]

i'm using latest build and for some reason it always says detected 0 cpus and using 0 threads.

if i force -threads 2 or 4 it will still be using only 1 cpu as my cpu usage is stuck on 25%.

i have a dual xeon 3ghz with HT enabled.

Detected 0 cpus, using 4 threads.
701 frames( 31%) encoded, 7.19 fps, Average Bitrate = 14231kbps

that's 1st pass, 2nd pass goes down to 3fps.

i'm using the batch file made by henry_hk in the settings thread by teegedeck.

any ideas about what i'm doing wrong?

[SeeMoreDigital]

any ideas about what i'm doing wrong?It would be handy if you could confirm the resolutions you are encoding to!!!

[shpitz]

oh, sorry, i was encoding a 720p clip to a 720p output resolution.

what was suspicious to me was that encraw reported 'xvidcore build version: xvid-1.1.0'

so i figured it must be using an old version of xvid that is not multithreaded.

i installed a newer build and now it is working great, getting 2x the speed, and # of cpus is detected properly now.

so that's all it was, using an older version of xvidcore.dll .

i will do another test with 2 threads, right now i'm doing with 4 threads.

[mod]

Hi all.
I'm not able to limit the number of frames encoded. Arguments:

xvid_encraw -i pippo.avs -type 2 -frames 200 -custom_par 80 81 +chroma_opt -quality 6 -max_bframes 1 -framerate 25 -bitrate 2000 -pass1 pippo.log -iquants 2 31 -pquants 2 31 -bquants 2 31 +trellis -lumimasking -vhq 4

it simply doesn't stop at frame 200. Am I making some error?
Thanks!

[buzzqw]

Hi mod !

first of all: you are using the old sintax...
second: latest build of xvid_encraw doesn't have -frame parameter... just -start for specifying the START frames...

i can suggest to use zones... to lower quant

BHH

[mod]

Really thanks for the info.
I'm using the version linked a 1st page, and the syntax is the one I got with

xvid_encraw -help ..

EDIT: I found this (http://forum.doom9.org/showthread.php?p=800912#post800912) one and supports -frames. Is this the latest?

I'm really sorry if I missed some link.

[kurt]

latest is this (AFAIK): http://forum.doom9.org/showthread.php?p=822253#post822253

[buzzqw]

please mod ... don't complain !

i don't know older built fupporting this features... but... may i suggest to use the TRIM feature in your avs script ?

BHH

[SeeMoreDigital]

Excuse for my ignorance but I have to ask....

Is one of the goals of XviD EncRAW to create an non VfW, ie: CLI version of XviD?


Cheers

[imcold]

yes /I believe :D

[buzzqw]

yes ! for sure... but what's the matter ? :confused:

BHH

[john27]

small bug:
#include <ctype.h> is missing //toupper(c)

[JoeBG]

Excuse for my ignorance but I have to ask....

Is one of the goals of XviD EncRAW to create an non VfW, ie: CLI version of XviD?


Cheers

Yes...and it works great. I use it for all my xvid encodes. You can create a proper raw or *.mkv stream without packet bitstreams.

[shpitz]

Yes...and it works great. I use it for all my xvid encodes. You can create a proper raw or *.mkv stream without packet bitstreams.

Joe, and your encodes have no issues with fast forwarding (seeking)? if so what do you use to mux the encoded video with the audio?

[SeeMoreDigital]

Yes...and it works great. I use it for all my xvid encodes. You can create a proper raw or *.mkv stream without packet bitstreams.That's encouraging news :)

I wonder if Stax can code it into StaxRip before he abandons it altogether?


Cheers

[foxyshadis]

Is it possible to split this thread from page 10 or so, or better have a mod edit the first post to include some information on getting current versions of encraw?

[mod]

I made a little GUI for xvid_encraw, maybe someone finds it useful.

here (http://forum.doom9.org/showthread.php?t=112808)

I use it for SAP, so not really a lot of options (ie only avi output), but if somene wants I can add what needed, no problem :D

In the next days I'll try to write something in english, but it's really easy, doesn't need any real explanation..