xvid_encraw - Patched with AviSynth input support

[S_O]

I already made a more or less working of xvid_encraw with AviSynth input support (I simply copied the code from avs2yuv) several month ago (when 1.1.0 beta1 was released), but I somehow borked the source and I never published it.
Now I copied the changes into XviD 1.1.0 beta2, but more cleanly now and did some other changes:
-Added VBV-Buffer options (don´t ask me how to use them - I have no idea)
-Disabled the statistic display for every frame by default
-Added progress display
-Added stats at the end that displays how many I/P/B/S-VOPs with avg. quant and size.

How to use it:
For example: xvid_encraw -i script.avs -type 2 -asm -quality 6 -qpel -single -bitrate 1000 -max_bframes 2 -o rawmpeg4.m4v
For mor details use xvid_encraw -h
It only outputs raw streams, no MP4! You can use mp4box to mux the stream into mp4:
mp4box -add rawmpeg4.m4v video.mp4

Known bugs:
-Time/fps in statistics is ..hmm.. not realistic (I don´t know if I introduced the bug or if it has always been there)
-With B-Frames enabled stat says that more than 100% of the source frames were encoded.

If you like to compile it yourself using VC++, copy the xvid_encraw.cpp and the avs2yuv-dir in the examples-dir of xvidcore, open the main project, delete the xvid_encraw.c from the xvid_encraw project and add xvid_encraw.cpp.

I noticed another problem (not in this release): I had many for me unexplainable bugs in stats display, every time I fixed one another display broke. I hunted bugs over 3 hours. I couldn´t explain why a the value of variable which I set to 0 is somewhere else outputted as 875644. But when I add messagebox in between, that displays another value - it works.
I fixed the problem by changing compiler from "Optimize for speed" to "Standard" (this has only affected the frontend, not xvidcore.dll). Now it works exactly as it´s supposed to work. I don´t know if I write crappy code or Microsoft crappy compilers... (or both)

Edit: Release 3 is ready:
for changes goto:
http://forum.doom9.org/showthread.php?p=698181#post698181
http://forum.doom9.org/showthread.php?p=698555#post698555
Attachment has been approved by Koepi, please download and test, and post here how it works for you (or if you have found any bugs).

[Doom9]

thank you. Now we just need somebody to pick up the slack and add the missing options and this will be an awsome tool.

[Sirber]

Once more stable, I will add it to RealAnime :D Keep the good work!

[S_O]

thank you. Now we just need somebody to pick up the slack and add the missing options and this will be an awsome tool.What options are missing exactly? I´ll try to add them.
Once more stable, I will add it to RealAnime Keep the good work!Thanks

Edit: I yust looked into xvid.h and the vfw interface, there are lot´s of options missing. I´ll try to add them.

[Sirber]

Can mkvmerge merge M4V stream into MKV?

[Doom9]

What options are missing exactly?
interlaced: bff
quantization type
adaptive quantization
par
1 pass mode: reaction delay factor, averaging period, smoother
2 pass mode, 2nd pass: iframe boost, iframes closer than, reduced by, overflow control strength, max overflow improvement, max overflow degradation, high bitrate scenes degradation, low bitrate scenes improvement
vhq mode
vhq for b-frames
chroma motion
turbo
frame drop ratio
trellis
min/max quantizer for i/p/b frames
and zones also have: begin with keyframe, cartoon mode, grayscale, chroma optimizer and bvop sensitivity
last but not least custom quantization matrices

phew, quite a long list, isn't it?

[S_O]

Can mkvmerge merge M4V stream into MKV?No, but it shouldn´t be that difficult to add.

It seems that xvid.h offers much more options than the vfw-interface uses (some of them might be grouped toghether). I´ll try to add all.

Edit: @Doom9: The list is even longer.

[Doom9]

I'd have some more ideas but I'll keep them to myself for now, as I've already flooded you.

[Sirber]

Well, thanks a lot for your efforts! I won't bug you longer either :)

[bond]

really great stuff, thx a lot! :)

[S_O]

I yust noticed that the quality presets automatically activate features. At quality 6 for example Trellis, Inter4MV, Halfpel and Chroma Motion are always used.

Yust for curiosity: It seems that XviD allows a new Quant Matrix at every I-VOP (if I read the code correctly I can even force a new quant matrix without a new I frame).
In theory it should be possible if the VOL would be written again. But for seeking, the VOL should be repeated at every I-VOP, otherwise the decoder would need to scan the entire file back until it finds a quant matrix.
Is that allowed by the specs? Also I found informtion how inverse quantisation with the different methods work (H.263; MPEG) and how this is indicated in the bitstream, I couldn´t find one sentence saying it is allowed or not (maybe it´s more general, like "VOL is not allowed to change in a stream"). I remember XviD had an option some time ago to automatically switch between H.263 and MPEG quant, but it was removed, because its not ISO compliant. Where in the specs is that written? In theory, where a VOL can be (are they only allowed in front of a I-VOP?), there could be new quantisation.

[sysKin]

maybe it´s more general, like "VOL is not allowed to change in a stream"Yes this is exactly it. I don't remember where it is exactly, but although VOL can be repeated accross the stream, it must not change.

[edit]oh, I almost forgot: great work, I'll commit it to cvs when it's ready :)

[bond]

as syskin said its not allowed to have varying vols

eg when placing the raw m4v stream in .mp4 the vol is placed only once in the file (as it cant change anyways) and its also not placed together with the i-frames

[IgorC]

it's sumary usefull tool. Many settings were mentioned to apply. And Full quality first pass?
I don't care much about -debug , but it crashes here.

-bitrate (bps) . -bitrate 800000 . NOT -bitrate 800

[multicone]

Can this tool be patched with Haali MKV writing lib from x264, so it will write native MPEG4 MKV ?

[Doom9]

@multicone: sure... are you volunteering your time? We've dumped too many things on poor S_O already.. I have a feeling the more it gets the more likely it'll never get done - it's kinda annoying to have a classroom full of people clamoring I want this and that ;)

[S_O]

Ok, that´s all switches I want to implement, have I missed something (except zones). Note that the description is either taken from vfw or directly from xvid.h, what means I don´t what they are all good for. I also reorderd them in the help display:
Input options:
-i string : input filename (default=stdin)
-type integer: input data type (yuv=0, pgm=1, avisynth=2)
-w integer: frame width ([1.2048])
-h integer: frame height ([1.2048])
-frames integer: number of frames to encode

Output options:
-dump : save decoder output
-save : save an Elementary Stream file per frame
-o string: save an Elementary Stream for the complete sequence

Bitstream options:
-prf profile : indicated profile ([S0.S3] or [AS0.AS5])
(default=unrestricted)
-par aspect_ratio : pixel aspect ratio ([VGA11; PAL43; NTSC43; PAL169;
NTSC169]) (default=VGA11)
-custom_par X Y : custom pixel aspect ratio
-divx_ud : write DivX-userdata
-vol_ivop : repeat VOL at every I-VOP

General encoding options:
-quality integer: quality ([0..6]) presets
-hqacp : high quality ac prediction
-inter4v : use 4 motion vectors per MB
-gmc : use global motion compensation
-interlaced : use interlaced encoding (this is NOT a deinterlacer!)
-i_tff : top field first (interlaced encoding)
-i_alt : alternate scan (interlaced encoding)
-reduced_res : enable reduced resolution
-force_rrv : force all frames to be reduced resolution
-greyscale : greyscale mode, all chroma is ignored
-cartoon : use 'cartoon' mode
-chroma_opt : enable chroma-optimizer pre-filter
-frame_drop integer: frame drop ratio (in percent)

BFrames options:
-max_bframes integer : max bframes (default=0)
-bquant_ratio integer : bframe quantizer ratio (default=150)
-bquant_offset integer : bframe quantizer offset (default=100)
-packed : packed mode (DO NOT USE! WILL CORRUPT OUTPUT!)
-closed_gop : closed GOP mode

Rate control options:
-framerate float : target framerate (>0 | default=25.0)
-bitrate integer : target bitrate
-single : single pass mode
-pass1 filename : twopass mode (first pass)
-pass2 filename : twopass mode (2nd pass)
-zq starting_frame float : bitrate zone; quant
-zw starting_frame float : bitrate zone; weight
-max_key_interval integer : maximum keyframe interval

Single Pass options:
-cquant float : target quantizer (use instead of bitrate)
-reac_delay integer : reaction delay factor
-avg_period integer : averaging period
-smoother integer : smoother

Pass 1 of 2 options:
-full_1p : Full first pass

Pass 2 of 2 options:
-size integer : target size of video (use instead of bitrate)
-keyframe_boost integer : keyframe boost (in percent)
-close_i_red intA intB : I frames closer than A frames are reduced by B%
-ccp_high integer : high bitrate degradation (in percent)
-ccp_low integer : low bitrate improvement (in percent)
-max_oi integer : max overflow improvement (in percent)
-max_od integer : max overflow degradation (in percent)
-overf_cs integer : overflow control strength (in percent)
-dxn_prf profile : apply DXN-Profile ([HH; PPAL; PNTSC; HTPAL;
HTNTSC; HDTV])
-vbv_size integer : buffer size (bits)
-vbv_initial integer : initial buffer occupancy (bits)
-vbv_maxrate integer : max processing bitrate (bps)
-vbv_peakrate integer : max average bitrate over 3 seconds (bps)

Zone options:
-zq starting_frame float : bitrate zone; quant
-zw starting_frame float : bitrate zone; weight

Quantization options:
-iquants min max : I-frame quantizer restriction ([1.31])
-pquants min max : P-frame quantizer restriction ([1.31])
-bquants min max : B-frame quantizer restriction ([1.31])
-trellis : use trellis based R-D "optimal" quantization
-mpeg_quant : use MPEG quantization instead of H.263
-mpeg_cqm filename : use custom MPEG quantization matrix (intra & inter)
-mpeg_intra_cqm filename : use custom MPEG quantization matrix (intra only)
-mpeg_inter_cqm filename : use custom MPEG quantization matrix (inter only)
-lumimasking : use lumimasking/adaptive quantization algorithm

Motion estimation options:
-hpel : use half pixel ME
-qpel : use quarter pixel ME
-chromap : use P-Frame chroma for ME
-chromab : use B-Frame chroma for ME
-me_advd16 : use advanced diamonds as search pattern
-me_advd8 : use advanced diamonds for extended 8x8 search
-me_sqr16 : use squares as search pattern
-me_sqr8 : use squares for extended 8x8 search
-me_hpelr16 : enable halfpel refine 16
-me_qpelr16 : enable quarterpel refine 16
-me_hpelr8 : enable halfpel refine 8
-me_qpelr8 : enable quarterpel refine 8
-me_gmer : enable GME refine
-me_exts16 : extend PMV by more searches
-me_exts8 : extended 8x8 search

Rate disortion options:
-vhq integer : R-D presets ([1.4])
-rated : enable R-D
-rd_simple : use simplified R-D mode decision
-rd_bf : enable R-D for B-frames
-rd_hpelr16 : use R-D halfpel refine 16
-rd_qpelr16 : use R-D quarterpel refine 16
-rd_hpelr8 : use R-D halfpel refine 8
-rd_qpelr8 : use R-D quarterpel refine 8
-rd_chk_pred : check vector equal to prediction
-rd_exts : perform R-D-based search using square patterns

Turbo options:
-turbo : turbo preset
-t_fr16 : low-complexity 16 refinement
-t_fr8 : low-complexity 8x8 sub-block refinement
-t_skipds : skip b-frame delta search
-t_fmi : partly skip interpolate mode
-t_bfes : stop b-frame search early
-t_dsm : detect stationary scenes

Other options
-asm : use assembly optmized code
-stats : print stats about encoded frames
-debug : activates xvidcore internal debugging output
-vop_debug : print some info directly into encoded frames
-help : prints this help message

NB: You can define 64 zones repeating the -z[qw] option as many times as needed.Please tell if I seem to have forget anything.
I´ll have a look how to add zones properly, with the flexibility to change all parameters that are allowed by the specs and are possible with xvid.
All the extended zone stuff (cartoon, greyscale, b-vop threshold...) is handled directly by the vfw and not by xvidcore and manually applied every frame.

Is there only one format for quant-matrix (I don´t know the english plural of matrix), the raw format (128 Byte)?
I don't care much about -debug , but it crashes here.Was it the last parameter in your commandline? It seems to requires some kind of argument, but it doesn´t check if there is one.
Can this tool be patched with Haali MKV writing lib from x264, so it will write native MPEG4 MKV ?There was C++ class for matroska writing once (based on libebml/libmatroska), i don´t know if it´s still up-to-date. It should be rather simple to add mkv support with it (as compile time option like avisynth).
eg when placing the raw m4v stream in .mp4 the vol is placed only once in the file (as it cant change anyways) and its also not placed together with the i-framesThat would of course brake changing quant-matrix.

Edit: I´m yust thinking that the option -asm should become -noasm, because most probably want to use asm and many will forget to set -asm.

[S_O]

@multicone: sure... are you volunteering your time? We've dumped too many things on poor S_O already.. I have a feeling the more it gets the more likely it'll never get done - it's kinda annoying to have a classroom full of people clamoring I want this and thatCurrently I have (summer!?) holidays (also temperature is more like autumn) and so I have time.
I´ll first add all the other stuff before I start thinking about mkv.

[bond]

Please tell if I seem to have forget anything.hm the -packed option should be removed, as it only borks the stream with raw output

afaik the reduced resolution option is borked too, but i dunno the details

the dxn profiles should be grouped together with the normal profile settings, as its not possible to have both, a mpeg-4 and a dxn profile
the point of profiles is (dis)allowing specific options and setting the vbv (and setting the profile/level indication in the stream of course)
so if the tool isnt able to take this into account when the user sets a profile its better to not allow the setting of the profiles at all (and leave this up to a gui)

I´ll have a look how to add zones properly, with the flexibility to change all parameters that are allowed by the specs and are possible with xvid.probably its a good idea to allow the same features to be set via zones as the vfw gui allows
most things (like qpel, gmc, matrix aso) are not allowed to vary in the stream anways

Is there only one format for quant-matrix (I don´t know the english plural of matrix), the raw format (128 Byte)?yes, there is
if you search around you will find lots of cqm for xvid (i assume the vfw code handles the parsing of these)
people definitely dont use different cqm files for inter and intra

[S_O]

the dxn profiles should be grouped together with the normal profile settings, as its not possible to have both, a mpeg-4 and a dxn profile
the point of profiles is (dis)allowing specific options and setting the vbv (and setting the profile/level indication in the stream of course)
so if the tool isnt able to take this into account when the user sets a profile its better to not allow the setting of the profiles at all (and leave this up to a gui)
What I thought by separating the DXN profiles from the normal one:
The normal profiles are indicated in the VOL, DXN are not. So I can apply a DXN profile which restricts VBV buffer while at the same time the stream can be marked as AS@L3. This can be perfectly fine, if the encoding parameters are reduced to the least common denominator.
Also profile indication cannot be left to gui (except DXN), since it´s stored in the bitstream and xvidcore does not automatically set the profile, xvid_encraw (or vfw) has to tell it which profile to indicate. Currently it is hardcoded to AS@L4 (better would have been unrescricted, I haven´t done it), xvidcore doesn´t care if the stream complies to it or not. If I set profile to 0xc4, it probably won´t complain either. This is yust 1 byte information in the bitstream. It´s up to the interface if it checks the parameters to profile compliance and xvid_encraw currently does not. But I´ll add a warning if a profile is indicated thoghether with a not compliant feature.
probably its a good idea to allow the same features to be set via zones as the vfw gui allows
most things (like qpel, gmc, matrix aso) are not allowed to vary in the stream anwaysIt seems that many things like turbo, ME options, trellis, some rd options, force reduced resolution can be chnaged during encode, but of course the most important things are the ones included in vfw.
hm the -packed option should be removed, as it only borks the stream with raw outputYes, you´re probably right. Also I added a warning, people might think "do not use - this must be the expert-only switch for super-high-ultra-quality, yust like -k is in lame"
if you search around you will find lots of cqm for xvid (i assume the vfw code handles the parsing of these)
people definitely dont use different cqm files for inter and intraAll I found were the normal 128 byte files, which xvid vfw seems to handle.
I also haven´t found different files for intra/inter, but maybe somebody only wants to use a custom inter matrix and leave the intra matrix untouched, or use intra and inter from different files. I´ll add it this way:
Intra/inter-matrix: only 128 byte files accepted
intra-matrix only: 64/128 byte files accepted, for 128 byte first matrix will be used.
inter-matrix only: 64/128 byte files accepted, for 128 byte second matrix will be used.
What I originally meant, there is no format containing several cqms with the names, designed goals etc. stored in the file, only raw.

[bond]

What I thought by separating the DXN profiles from the normal one:
The normal profiles are indicated in the VOL, DXN are not. So I can apply a DXN profile which restricts VBV buffer while at the same time the stream can be marked as AS@L3. This can be perfectly fine, if the encoding parameters are reduced to the least common denominator.
Also profile indication cannot be left to gui (except DXN), since it´s stored in the bitstream and xvidcore does not automatically set the profile, xvid_encraw (or vfw) has to tell it which profile to indicate.well people could set a mpeg-4 profile (like simple profile) together with a dxn profile (using for example b-frames) which wouldnt fit

i think the way how its handled currently in vfw is fine: set the mpeg-4 profile being equivalent to the dxn profile
eg dxn home theather profile is equivalent to asp@l5 (it makes no sense to allow a different mpeg-4 profile to be set)

Currently it is hardcoded to AS@L4 (better would have been unrescricted, I haven´t done it), xvidcore doesn´t care if the stream complies to it or not. If I set profile to 0xc4, it probably won´t complain either. another easy possible way would be to signal simply always the highest possible profile: asp@l5

This is yust 1 byte information in the bitstream. It´s up to the interface if it checks the parameters to profile compliance and xvid_encraw currently does not. But I´ll add a warning if a profile is indicated thoghether with a not compliant feature.well i am no dev, but if encraw is already able to detect that a feature is not part of the set profile, how much work would it be to disable this feature automatically?

[yokem55]

Is there a way to build this on linux? I don't need the avs input as setting up mencoder to dump into a raw yuv fifo works fine, but the extra options would be nice to have available/play with.

[S_O]

Is there a way to build this on linux? I don't need the avs input as setting up mencoder to dump into a raw yuv fifo works fine, but the extra options would be nice to have available/play with.Yes, that will be possible, you yust need to delete/comment "#define AVISYNTH_INPUT" at the beginning.
well people could set a mpeg-4 profile (like simple profile) together with a dxn profile (using for example b-frames) which wouldnt fit

i think the way how its handled currently in vfw is fine: set the mpeg-4 profile being equivalent to the dxn profile
eg dxn home theather profile is equivalent to asp@l5 (it makes no sense to allow a different mpeg-4 profile to be set)What I mean is, that DXN restriction is partly different than the normal is (VBV buffer), and DXN profiles also only work in 2pass mode, because the vbv restriction stuff is part of the 2pass-plugin. With single pass the output probably won´t comply to the DXN specs.
That´s why I added the DXN stuff to 2pass mode in help: It only makes sense there! And the iso profiles are added to bitstream options and described "indicate profile", because it´s yust one byte in the VOL for xvidcore. I can set this byte to whatever I like (except to 0, then xvidcore sets it automatically):

if reduced_resolution -> ARTS@L4
if GMC or QPEL -> ASP@L5
everything else will get SP@L3

that´s all. In other words when I create a stream 60fps, 1440x960 with 3 bframes at 4mbps, it will automatically get SP@L3 (max: 352x288; 15fps; 384kbps; no b-frames).
The switch is yust to set the byte, not to force the profile. Of course, that means files with a wrong profile may be created, but that can happen with vfw as well, because it also doesn´t check everything.
But I will add a warning, like with " -dxn_prf XY -prf AS5 -par PAL43 -reduced_res":
Warning: Aspect Ratio for DXN-Profiles must be VGA 1:1. Stream will not comply to DXN XY Profile
Warning: Reduced resolution is not supported by Profile AS@L5. Stream will not comply to indicated profile!

Unfortunately I have no complete list of the profiles and their ids. There is a list here http://www.m4if.org/resources/profiles/ with more than 50 profiles/levels (like "Core Studio" or "Advanced Core"). Unfortunately the ids are not included.

well i am no dev, but if encraw is already able to detect that a feature is not part of the set profile, how much work would it be to disable this feature automatically?encraw is not able to detect it, I have to tell him what complies to specs and what not. Of course I can also disable stuff like Qpel, GMC or B-Frames or reduce bitrate, but fps or frame size is not easy to change (and vfw doesn´t check that either).

[bond]

What I mean is, that DXN restriction is partly different than the normal is (VBV buffer), and DXN profiles also only work in 2pass mode, because the vbv restriction stuff is part of the 2pass-plugin. With single pass the output probably won´t comply to the DXN specs.
That´s why I added the DXN stuff to 2pass mode in help: It only makes sense there!exactly the same goes for the mpeg-4 profiles...
i dunno if you know it but if you signal asp@level5 you also have to use the vbv if you want to be spec compliant...
the mpeg-4 profiles/levels influence the vbv too, not only the encoding options. basically the dxn profiles are private mpeg-4 profiles, nothing less, nothing more

check out what mpeg-4 profile/level gets signalled when a dxn profile is set! afaik the dxn home theater profile sets the asp@l5 profile/level, now what if the user additionally tries to set a different mpeg-4 profile/level
it doesnt make sense to seperate the two and its also not differentiated in the vfw gui

again, checkout how things are handled in xvid vfw! its uptodate and the people who wrote the vfw codec knew what they did (cause after all practically the xvid codec is the vfw codec)

another point showing this i brought up before too, which you seem to ignore, is that reduced resolution is not working correctly in xvid
the code might still be there but if you look at the vfw gui you will see that reduced resolution (and the arts profile) is not available, this also happens for a reason

And the iso profiles are added to bitstream options and described "indicate profile", because it´s yust one byte in the VOL for xvidcore. I can set this byte to whatever I like (except to 0, then xvidcore sets it automatically):

if reduced_resolution -> ARTS@L4
if GMC or QPEL -> ASP@L5
everything else will get SP@L3

that´s all. In other words when I create a stream 60fps, 1440x960 with 3 bframes at 4mbps, it will automatically get SP@L3 (max: 352x288; 15fps; 384kbps; no b-frames).
The switch is yust to set the byte, not to force the profile. Of course, that means files with a wrong profile may be created, but that can happen with vfw as well, because it also doesn´t check everything.no, with vfw this cant happen, as the gui doesnt let the user enable incompliant settings (of course framerate, bitrate and resolution gets ignored, but i dunno any player which cares about these three)

But I will add a warning, like with " -dxn_prf XY -prf AS5 -par PAL43 -reduced_res":
Warning: Aspect Ratio for DXN-Profiles must be VGA 1:1. Stream will not comply to DXN XY Profile
Warning: Reduced resolution is not supported by Profile AS@L5. Stream will not comply to indicated profile!

encraw is not able to detect it, I have to tell him what complies to specs and what not. Of course I can also disable stuff like Qpel, GMC or B-Frames or reduce bitrate, but fps or frame size is not easy to change (and vfw doesn´t check that either). ok but here is my point: how hard is it to make xvid automatically disable incompliant settings? i mean if you already have the goodie implemented that it tells the user that a chosen setting is incompatible

something like a message saying "b-frames disabled as not complying with chosen profile/level"

[S_O]

no, with vfw this cant happen, as the gui doesnt let the user enable incompliant settings (of course framerate, bitrate and resolution gets ignored, but i dunno any player which cares about these three)
I yust tested and confirmed result in hexeditor:
Set profile to unrescricted, enable some ASP features (except Qpel / GMC) like B-Frames and mpeg quant, now encode your file:
The stream is marked as SP@L3 also it contains B-Frames and MPEG quantization!

In my opionen the best place for automatic profile decision is xvidcore, not the frontend.
i dunno if you know it but if you signal asp@level5 you also have to use the vbv if you want to be spec compliant...
the mpeg-4 profiles/levels influence the vbv too, not only the encoding options. basically the dxn profiles are private mpeg-4 profiles, nothing less, nothing moreSeems you are right, I thought the ISO profiles were yust limiting encoding features and fps/bitrate/resolution and that´s why DXN introduced their profiles.
But since they both have this vbv stuff you are of course right and separating doesn´t makes much sense.
But I wouldn´t call them "private profiles", because they are not indicated in bitstream, if they were using some reserved values to indicate their profiles this trem would be adequate, but in my opinion it´s yust some extra rescrictions for DXN promotion that do not interfere with the ISO standard at all.
dxn home theater profile sets the asp@l5 profile/levelNo, it is set to "auto-detect" by xvidcore (see vfw source (config.c, line 149) and xvidcore-source (bitstream.c, line 1076 - 1117)!), which means, if you don´t use Qpel/GMC it´s marked as SP@L3.
another point showing this i brought up before too, which you seem to ignore, is that reduced resolution is not working correctly in xvid
the code might still be there but if you look at the vfw gui you will see that reduced resolution (and the arts profile) is not available, this also happens for a reasonSorry, I must have missed that sentence. In case it´s not working correctly it should of course be removed.
The reason why reduced resolution has been removed:
From sysKin:
* it's now a long time we planned removing support for RRV as it
adds complexity to the ME, to the decoder and this feature fits
nowhere in any MPEG4 profile we plan to support.I don´t know if it has only been removed from vfw yet, or if it´s already removed (or will be ever removed) from xvidcore and only the flags are still there.
again, checkout how things are handled in xvid vfw! its uptodate and the people who wrote the vfw codec knew what they did (cause after all practically the xvid codec is the vfw codec)That´s what I do all the time, and as far as I can tell you, profiles are not handled perfectly (see demonstration at the beginning of this post)
ok but here is my point: how hard is it to make xvid automatically disable incompliant settings? i mean if you already have the goodie implemented that it tells the user that a chosen setting is incompatible

something like a message saying "b-frames disabled as not complying with chosen profile/level"As already said, in my opionen the best way to handle this would be if xvidcore checks the features and automatically sets the profile. That ensures that every frontend sets the right profile.
When profile is set by frontend xvidcore yust confirms that the encoding features are allowed by the profile, and in case not allowed features are used it refuses to encode and exits (except and override flag is set like "XVID_VOL_IGNOREPROFILE").

The question is now: What profile has to be set when no profile matches the encoding features? The maximum resolution even for AS@L5 is 720x576.

[sysKin]

RRV has been removed from xvid 1.1. Flags will stay, so that API remains backward compatible with 1.0's API.

[bond]

I yust tested and confirmed result in hexeditor:
Set profile to unrescricted, enable some ASP features (except Qpel / GMC) like B-Frames and mpeg quant, now encode your file:
The stream is marked as SP@L3 also it contains B-Frames and MPEG quantization!hm, thats not good :/

As already said, in my opionen the best way to handle this would be if xvidcore checks the features and automatically sets the profile. That ensures that every frontend sets the right profile.indeed, but i assume its hard to code taking everything into account
and than there is the issue of enforcing the vbv too (but i assume the profile code can do that already, at least in the case of the dxn profiles its a must, or is this done by the vfw gui?)

When profile is set by frontend xvidcore yust confirms that the encoding features are allowed by the profile, and in case not allowed features are used it refuses to encode and exits (except and override flag is set like "XVID_VOL_IGNOREPROFILE").cool

The question is now: What profile has to be set when no profile matches the encoding features? The maximum resolution even for AS@L5 is 720x576. well i would simply ignore the framerate and resolution issue, cause i really dunno any player who needs this to be correct

if you ignore the two than the profile to be set should be imho simply asp@l5, cause it covers everything (except reduced resolution, which is borked anyways)

[S_O]

RRV has been removed from xvid 1.1. Flags will stay, so that API remains backward compatible with 1.0's API.Unfortunately this is not written in xvid.h. The flags are there yust like all others, without any notice that they are only for compatibility and rrv doesn´t work. Also there is still code inside xvidcore to deal with rrv, the function BitstreamWriteVolHeader checks for rrv being enabled and sets the flags in bitstream.
and than there is the issue of enforcing the vbv too (but i assume the profile code can do that already, at least in the case of the dxn profiles its a must, or is this done by the vfw gui?)Yes, everything is done by the frontend, xvidcore doesn´t know at all a DXN profile is applied (it doesn´t even know that they exist).
When a profile is selected, vfw enables/disables the features in the dialog and sets the vbv buffer stuff (and for DXN profile also maximum peak-bitrate). In case of ISO profiles it also sets the profile id (the 5th byte in a raw stream, directly after the video object sequence start code).
Without 2pass mode the vbv buffer is not set, since this feature is part of 2pass plug-in.
well i would simply ignore the framerate and resolution issue, cause i really dunno any player who needs this to be correct

if you ignore the two than the profile to be set should be imho simply asp@l5, cause it covers everything (except reduced resolution, which is borked anyways)OK, I yust looked in the specs and this page: http://www.m4if.org/resources/profiles/ again. fps is nowhere mentioned, and resolution is called "typical resolution" not "maximum resolution". In other words, we don´t need to care about that. Unfortunately I can´t find ISO 14496-2 AMD2 containing advanced simple profile defiition.
If I get a full definition of the profiles I´ll see how difficult it is to implement working auto-profile setting in xvidcore.

Oh, in case you haven´t notice: I uploaded a new version (contains only changed files, if you do not have XviD 1.1 installed you need xvidcore.dll from zip of first post).
Chnages:
Lot´s of (nearly all) options added. See -h for details. You may notice that some options are not -xy but +xy: "+" means enable, "-" disable. So you can use presets (-quality, -vhq, -turbo), but you can easily disable some options again like "-quality 6 -trellis", means use -quality 6 (which enables trellis), but disable trellis. Disable switches always override enable switches.
I also added this switches, because I have an idea how to handle advanced zones and there I also need them.

Note 1: cartoon is not exactly the same here and in vfw! When you check cartoon in vfw, it enables cartoon mode and "detect static motion". So "+cartoon +t_dsm" equals vfw cartoon mode (I may change it to avoid confusion).

Note 2: I coded several hours and did one test. I mean one 30 sec clip (but that was fine). So the chances for bugs are very high. Consider this as very alpha. And please report all bugs here. The frame display is not yet fixed. And I said the encoding time is screwed up: It´s not: It´s _only_ the encoding time. If encoding takes 3 minutes but encraw says 1 minute, it means that the other 2 minutes was avisynth processing!

Still missing: Profile settings, fix quant mode, full 1st pass and advanced zones.

Edit: Outdated, see first post for latest

[buzzqw]

would be possible to implement an automatic 2 pass mode to hit final size (mb)

thanks

BHH

[Doom9]

would be possible to implement an automatic 2 pass mode to hit final size (mb)That's a feature for a GUI. MeGUI and RealAnime will do that for you when the time is right.

[S_O]

would be possible to implement an automatic 2 pass mode to hit final size (mb)I not quite sure what you mean, but you can use -size instead of -bitrate for 2pass mode, allowing you to set the final size of the video (note: Frame overhead is 0, but in vfw it´s set to 24 bytes (AVI), so you have calculate the container overhead yourself)

[Sirber]

That's a feature for a GUI. MeGUI and RealAnime will do that for you when the time is right.
So far, RealAnime don't produce a fixed size and work by bitrate or quality only. It include at least a bitrate calculator :).

[Doom9]

note: Frame overhead is 0, but in vfw it´s set to 24 bytes (AVI), so you have calculate the container overhead yourselfdoes that have any influence in bitrate mode? in the VfW and mencoder, the bitrate contains 24 byte container overhead.

[S_O]

does that have any influence in bitrate mode? in the VfW and mencoder, the bitrate contains 24 byte container overhead.No, it doesn´t seem so. I looked into xvid source and it seems that this parameter is only used with filesize parameter in 2pass mode, very simple, like this:
newsize = oldsize - frames*overhead.
(see plugin_2pass2.c; line 404).
Because xvid_encraw outputs raw stream I set this to 0 (with vfw the avi-file have the filesize enterd, with encraw the raw-file will have the filesize). So this parameter isn´t special, you can calculate it yourself.

Has nobody noticed that I uploaded a new version? No feedback until now... I don´t think that´s why because it works perfect and does not contain bugs.

[CiNcH]

No, it is because download doesn't work ;) ... still not approved.

[S_O]

http://forum.doom9.org/attachment.php?attachmentid=4477
Works for me, also after log-out.

[CiNcH]

Not for me.

Invalid attachment specified. If you followed a valid link, please notify the webmaster

[S_O]

You seem to be right. It seems to work because of cookies, when I try another browser it doesn´t work.
I´ll see if I can upload it somewhere else.

[Doom9]

It might just be that you can see your own attachments even while they're still pending.. I know that happens with mine but then again as admin I get to see everything. I have just approved the attachment. It might not be a bad idea to have just one attachment in the first post and edit the thread with changelog updates rather than to have multiple attachments all over the place.

[S_O]

It might not be a bad idea to have just one attachment in the first post and edit the thread with changelog updates rather than to have multiple attachments all over the place.Yes, I´ll do so next time and delete the others then.

[S_O]

OK, next release is ready:
Added advanced zones:
Now you have -z "zone_type start_frame quant/weight [advanced options]"
To see which options are allowed as advanced zones options see -h.
Note: Like in VirtualDub first frame is 0, not 1!

Fixed bugs:
Frame display finally works, you encode 100% of your movie.

Known bugs:
-Advanced zones will not start exactly at the specified frame in case B-Frames are enabled
-N-VOPs are not displayed correctly in stats (always 0 N-VOPs).

Missing features:
-Single pass fixed quant encoding
-Full 1st pass
-Profiles

Download is in first post!

[IgorC]

VHQ for b-frames? is it -rd_bf?

[S_O]

VHQ for b-frames? is it -rd_bf?Yes.
-rd_bf : enable R-D for B-framesR-D is Rate Disortion, "VHQ" is just another name for it (in fact the "VHQ"-modes are some presets for the R-D switches).
Edit:
How I can disable a setting which is enabled by default?Nothing is enabled by default. But if you have a preset activated like -quality 6, you can disable parts of it again, with, for example "-trellis" (disable trellis). Only the switches marked with "+" in the help screen can be disabled.
Tomorrow (or today, when I wake up again), I´ll post the normal switches that eqauls to.

[IgorC]

chroma optimization is bugy with this coomand line

xvid_encraw -i mxskal.avs -o raw.m4v -type 2 -asm -quality 6 -bitrate
900 -max_bframes 2 -croma_opt

[S_O]

xvid_encraw -i mxskal.avs -o raw.m4v -type 2 -asm -quality 6 -bitrate
900 -max_bframes 2 -croma_optYou disabled chroma optimizer (it is already disabled by default).
chroma optimizer is enable/disable switch, to enable it you have to use +chroma_opt.

Edit: it is +chroma_opt, not +croma_opt

[IgorC]

ok , I get it.

chroma was my fault. Thanks to point

[S_O]

+trellis
Yes. Trellis is enabled with this switch
+lumimasking
No. Lumimasking cannot be enabled/disabled. It´s yust -lumimasking
+chroma opt
Yes, it´s +chroma_opt to enable it and, if you like to disable it, for example inside a zone: use -chroma_opt as advanced parameter in the zone.

[IgorC]

I think a good GUI will be nice :)

[S_O]

I think a good GUI will be nice Yes.
I have taken this idea with "+" and "-" switches from a Real/Helix tool, I think it was dt_drive, it has also this kind of options.

[Doom9]

I think a good GUI will be niceMeGUI will eventually support it.
Speaking of which, two things that will be requested at some point will be MP4 and MKV output. Considering that such output is currently broken at 23.976fps in x264, and that that kind of output is most likely going to be integrated in the very same way, I'd hold off until these bugs are fixed. On the other hand, AVI output might be something where those problems won't ocurr, and with AVI output, encraw can serve as a full mencoder replacement in MeGUI.