Packed bitstream in XviD workaround for MT1389 players

[oddball]

At present players like the Elta 8883 will not playback XviD with packed bitstream correctly. If you have already got encodes in this format and do not wish to re-encode here is what you need to do in order to strip the packed bitstream out LOSSLESSY.

Tools you will need

3divx from http://www.3ivx.com
VirtualDubMod from http://virtualdubmod.sourceforge.net
Graphedit (Google it not hard to find)

Install 3ivx etc

First you need to demux the audio in VirtualDubMod. Open the AVI file in VirtualDubMod. Select menu 'Streams', 'Stream list' select audio file and click on 'demux' give it a name and remember where you saved it.

After that is demuxed close VirtualDubMod.

Open Graphedit. Drag the avi file onto Graphedit. It opens a graph. Delete all the boxes except the first two boxes (One says open file in one box and AVI Splitter in the other).

Go to menu 'filters' and add filter from 3ivx '3ivx muxer' and from DirectShow section 'filewriter' and give it name video.mp4 (Must give extension or it will bork).

right click on top out pin of AVI Splitter box and click render pin. It should connect itself to the 3ivx muxer and then to the file writer box. Right click on the 3ivx filter and select properties. Tick the Click on play button. Exit Graphedit when file completes.

Now click on 'File' 'New' in Graphedit. Drag the newly created MP4 file into Graphedit. Delete stuff after 3ivx splitter. Go to filters again and add AVI Mux and Filewriter under the DirectShow tree list. Right click on '3ivx Splitter' properties. Tick 'Allow Unsupported Decoders' and apply/OK it. Rick click on '3ivx Splitter and select 'Render pins'. Now click on play in Graphedit. This will render the file back to an AVI file ready for the final step.

Open VirtualDubMod again and open the newest AVI file. Go to stream list menu and select 'Add' and add back the audio file you demuxed earlier. If AC3 don't forget to right click the newly added audio file, select 'Interleave' and set audio interleave to 64 or 96 milliseconds (Preload can be left as is).

OK it and then make sure on the 'Video' menu it is set to 'Direct Stream Copy'. Go to 'File', 'Save as' and save to a new filename. This is your finished movie file with packed bitstream removed.

Done.

Word of warning. Check the audio is in sync. If not you may need to remux again in VirtualDubMod adjusting the audio delay slightly (That's another topic though). Also make sure 'Direct Stream Copy' is set under VirtualDubMod's 'Video' menu or it will try to re-encode the video (You don't want to do that). :)

Longwinded but works. Waiting for a tool that removes packed bitstream for you in one easy step.

EDIT: Updated. No need for MP4UI (And I heard it can cause probs).
EDIT: Corrected filter to use on muxing MP4 back to AVI. Thanks for the heads up.

[bond]

Originally posted by oddball
Waiting for a tool that removes packed bitstream for you in one easy step. waiting for a hardware player which doesnt only handle this old outdated avi container, which this packed bitstream hack was invented for, to be able to use the .mp4 file directly :D

[monty0815]

Does anybody know, how I can tell, if packed bitstream was turned on during the encode? Is there any app that shows this kind of information?
I simply can't remember, on which movies I had this enabled, since starting of with Xvid 1.0.
cheers, monty

[Stux]

Originally posted by oddball
Go to filters again and add AVI Splitter and Filewriter under the DirectShow tree list

I think you mean AVI Mux

You can also download GraphEdit from the 3ivx windows download page

[bond]

Originally posted by monty0815
Does anybody know, how I can tell, if packed bitstream was turned on during the encode?no way

[manono]

DRF Analyzer (http://www.geocities.com/analyzerDRF/)

[bond]

cool, didnt know that :)

[ashley]

I've just tried DRF analyser, and in part of the report it says "The file has Packeted Frames"

Is this the part that means that packed bitstream was used?

[manono]

Yes, and if Packed Bitstream isn't used, then it says nothing at all about it.

[monty0815]

thanks, manono

You just saved me a lot of burning on dvd-rw and testing on my standalone.

[oddball]

Stux corrected. Thanks.

[FZero]

does packed bitstream cause problems on all standalones? I have the liteon lvd-2002.

[oddball]

I don't know. But XvID with Packed bitstream or QPEL or GMC enabled can cause the MT1389 chips to b0rk.

[bond]

:search: guys

it also depends on whether you used 1 or more than 1 consecutive b-frames with packed bitstream

[oddball]

I'd rather they not use any of those options at all at the moment. Then again I'd rather they fix the damned firmware and hurry up and release it!

[Sharro]

Originally posted by oddball
... or QPEL ... can cause the MT1389 chips to b0rk.

Either I read it wrong... or you are wrong ... QPEL plays flawlessly on MT1389DE.

All the best.

Sharro

[oddball]

I have an XviD encoded with QPEL and all I get is sound and no video playback.

[Zhnujm]

Then surely the problem is not qpel.

Wrong muxing ? You could try to remux with Virtualdub
Custom Matrices ? Some will not play.
Horizontal resolution above 720 ? Does not work also.

[SeeMoreDigital]

All in all... very nice work oddball

Cheers

[oddball]

Hoing back to the QPEL issue. I had a file with packed bitstream AND QPEL. I did the usual to get rid of the packed bitstream but now that file will not play video in the Elta. It just plays the audio and remains stuck on the file selection screen. If the file is played on the computer it plays fine (But if you seek you have to wait until it catches up to the next keyframe).

Does QPEL have various settings?

[net83it]

hello oddball

please, can you write a guide with images?
i wrong something with graphedit, i don't realize what... :(

[bond]

avi (packed bitstream) -> mp4 -> avi (no packed bitstream) is now also possible with mp4box
read more about it here (http://forum.doom9.org/showthread.php?s=&threadid=75631)

[CrashUK]

I made my own way to do this so they work on my yamada 6600.. But should work for other players too.. its same as doom9 one but it comes with all needed software and all graph files are premade so you dont need to make them every time and it comes with checking tools so you can check for packeted bitstream..


Change Log - v1.0 to v1.1

Added: missing file 3vix test.GRF

Download Here:: remover packeted bitstream_v1.1.rar (http://homepage.ntlworld.com/david.allison33/packeted_bitstream_remover_v1.1.rar)

I premade all the graph files you need so they no need to make any graphs like in the doom9 one..

Also this is how long it should take per 700 meg avi

Part 1 - 1-2 Mins
Part 2 - 1-2 Mins
part 3 - 2-3 mins
Part 4 - 5 secs

Overall 4-7 mins maybe 10 mins on older systems


Also this is the readme below:

How to remove packeted bitstream from XVID Files



Unpack this zip to your C:\

Now before you start you need to install 3VIX "3ivx_d4_451_win.exe" which you can get from http://www.3ivx.com

when you installed that you need to load up graphedit and open "3vix test.GRF"

click on 3vix D4 Media Spiltter

and put Allow Unsupported Decodes On and now and then click Ok..

Ok now we get started....

now get your non-working avi and copy it to c:\avi_files\ and rename it bad.avi

now load up graphedt and open "Part One - avi to mp4.GRF"
now click play should take about 1-2 mins to do this bit you know when it's done when the play icon go black..

now that you done that...





open "Part Two - mp4 to avi.GRF" in graphedit and click on play..

Now when thats done we go to Part3



part3 open "bad.avi" in VirtualDubMod

and On the put down menu click Streams/steam list... now click demux and save sound file as "sound"

Now Open "Fixed_with_out_sound.avi" in VirtualDubMod

On the put down menu click Streams/stream list... and click add and open soundfile and then click ok.

"NOTE: on VBR MP3 Only if it ask you to rewite vbr header always click no"

Now on the put down menu click Video/save as in the box where it says video mode change it
from Full processing mode to Direct stream copy. now enter file name "FIXED" and click save




Now Part 4 this last part..

load up avic.exe and open fixed.avi

now put xvid on the box that says "FourCC Description Code:"
and Put XVID in the box that says "FourCC Used Codec:"

now click Apply..

now your done.. :)

rename fixed.avi to the real file name e.g. my movie disc 1.avi

you can run "delete_temp_files.bat" and it will delete all temp
files you made.

NOTE: Part4 Maybe Only Be Needed On Yamada 6600.. Not Tested This on Other Players

[glfinish]

The procedure seems to work, no complaints from any of the programs... but... the resulting AVI has some frames in wrong order if I change the fourcc to XVID !
If I leave the fourcc as mp4v it is played (correctly) by the 3ivx codec, but I'm not sure what will happen on the standalone player.
The original AVI had 2 conssecutive B-frames, I think the error is that those 'doubles' are swapped.
Anyone see this problem?

[pogo stick]

Originally posted by glfinish
... the resulting AVI has some frames in wrong order if I change the fourcc to XVID !
That's because of XviD decoder.
See this thread (http://forum.doom9.org/showthread.php?threadid=77455).

[monty0815]

Someone just posted another way to do this to the mt13x9 group, thought this might be interesting for some here:

Get the Mpeg4Modifier from here (needs the .NET framework)
http://www.moitah.net/
1. Load avi
2. Check the unpack option
3. In the user Data field either delete the whole Line which says something like "DIVX...b...p" or just edit the line and replace the last "p" with an "n"
4. Save the avi back to disk

The program doesn`t support OpenDML and I only tried it on one movie until now, but it seems to work quite well.
EDIT:
Just saw that GSpot identifies the movie as "OpenDML (AVI 2.0)", still worked though.

cu, monty

[bond]

mpeg4modifier was been developed by a doom9 member and it was also heavily tested in the xvid forum afaik already

[SeeMoreDigital]

Originally posted by bond
mpeg4modifier was been developed by a doom9 member and it was also heavily tested in the xvid forum afaik already How very true (http://forum.doom9.org/showthread.php?s=&threadid=78050)!

However, Moitah's thread was posted in the XviD forum... and I can't help feeling that many people may have missed reading about this excellent tool!


Cheers

[undercover_dancer]

Originally posted by monty0815
Someone just posted another way to do this to the mt13x9 group, thought this might be interesting for some here:

cu, monty

Monty,

I'm the guy who asked the question...wow you are fast I wanted to post here myself...

The one comment I have which is strange...

I "fixed" a file and changed it divx descriptor from p to n...then opened both the original file and the new file in differnet DRF analyzer windows.

The original one showed DRF average of 2.768, with 653.73 Frames/S
while the "fixed" one showed average of 3.289 with 2603.89 Frames/S....

Here are the logs:
Original:

FourCC: XVID
Codec: XviD0035
Resolution: [ Width: 576 Height: 240 ]
Frame Rate: 23.976 frames per second
The Video has 147064 frames [ 01:42:13 ]

Average Frame quality is HIGH [Average DRF/quantizer is 2.77]
Standard Deviation: Quality is MEDIUM [Std. Deviation is 1.00]
Image Resolution is MEDIUM

The file has Packeted Frames!

There are NO frame drops ( NO drops is better )

Recomended Resolution: [576x240] (Target DRF/quantizer=2.8)

Performance Caracteristics:
Macroblocks per frame: 540
The Width is multiple of 32

Kilobits per Second: 776.32
Kilobits per Frame: 32.38
Kilobits per Macroblock: 0.060
Bits per Pixel: 0.24

Frame Type Statistics :
I Frames: 1.07%
P Frames: 64.15%
B Frames: 34.78%
S Frames: 0.00%
N Frames: 0.00%
(More Advanced Codecs use B and S frames)
Frame Quality Statistics :

DRF=1&2: 73561 50.6%
DRF=3: 25173 17.3%
DRF=4: 40655 27.9%
DRF=5: 5163 3.5%
DRF=6: 7 0.0%
DRF=7: 609 0.4%
DRF=8: 321 0.2%
DRF=9: 0 0.0%
DRF>9: 0 0.0%
KeyF/DeltaF: 1.08%
KeyDRF<4: 1358
KeyDRF=4: 17
KeyDRF>4: 200

AverageKeyDRF: 1.90
MAXDRF: 8
AverageDRF: 2.77
Deviation: 1.00


Fixed:
FourCC: XVID
Codec: XviD0035
Resolution: [ Width: 576 Height: 240 ]
Frame Rate: 23.976 frames per second
The Video has 147064 frames [ 01:42:13 ]

Average Frame quality is MEDIUM [Average DRF/quantizer is 3.29]
Standard Deviation: Quality is MEDIUM [Std. Deviation is 1.01]
Image Resolution is MEDIUM

There are NO frame drops ( NO drops is better )

Recomended Resolution: [528x208] (Target DRF/quantizer=2.8)

Performance Caracteristics:
Macroblocks per frame: 540
The Width is multiple of 32

Kilobits per Second: 775.87
Kilobits per Frame: 32.35
Kilobits per Macroblock: 0.060
Bits per Pixel: 0.24

Frame Type Statistics :
I Frames: 1.07%
P Frames: 35.74%
B Frames: 63.19%
S Frames: 0.00%
N Frames: 0.00%
(More Advanced Codecs use B and S frames)
Frame Quality Statistics :

DRF=1&2: 41282 28.4%
DRF=3: 21085 14.5%
DRF=4: 73002 50.2%
DRF=5: 8283 5.7%
DRF=6: 16 0.0%
DRF=7: 1194 0.8%
DRF=8: 627 0.4%
DRF=9: 0 0.0%
DRF>9: 0 0.0%
KeyF/DeltaF: 1.08%
KeyDRF<4: 1358
KeyDRF=4: 17
KeyDRF>4: 200

AverageKeyDRF: 1.90
MAXDRF: 8
AverageDRF: 3.29
Deviation: 1.01



Anyone has any idea who is wrong here? I think the DRF analyzer is wrong (http://www.geocities.com/analyzerDRF/)...but maybe I am wrong...

:rolleyes:

[bond]

undercover_dancer, the fixed file should have the same amount of frames as the input one, what framenumber does virtualdubmod report when you open the two files in it?
also try the 3ivx and mp4box way to get rid of packed bitstream, maybe mpeg4modifier is buggy

[undercover_dancer]

how do I check in VirtualDubMod ?
:confused:

[monty0815]

However, Moitah's thread was posted in the XviD forum... and I can't help feeling that many people may have missed reading about this excellent tool!


I for one never heard about it before.
Worked for me, couldn`t find any difference between the avi files.
Thanks to Moitah for the nice tool.

cheers

[glfinish]

I think I identified another xvid setting which can cause non-smooth playback on the MT1389.
If you encode with max B-VOPs 2 (or more) , AND without packed bitstream, but with adaptive quantization, you get non-smooth playback.
Not as bad as B-VOPs >=2 with packed bitstream, but not as it should play either. Maybe it misses playing just one of the B-VOPs, not sure.
(I think packed bitstream causes the whole B-VOP sequence not to play)
Anyone can confirm or contradict this?
Any idea what these things have in common that 'fools' the MT1389?

[stegre]

I've added recognition of "packed bitstreams" to GSpot 2.51 beta, both to the container and MPEG sections. A message in the container info window is displayed whenever an AVI file exhibits one or more 6-byte chunks containing a single DivX 5 compatible "N_VOP" (specifically, a "not-coded P_VOP with a time_base of zero"). Additionally, an indicator in the MPEG secion is activated in the presence of these N_VOPS, even in the absence of an AVI container. Finally, an unrelated update was made to allow viewing userdata that's inside an MPEG which itself is inside an AVI, so now you can read those headers. See screenshot (http://gspot.headbands.com/nvop.png). The features should appear in beta 3, probably in another week or so.

Steve G

[lark]

Originally posted by undercover_dancer
how do I check in VirtualDubMod ?
:confused:

open the avi and click file - file information.
but as you can see, both have the same amount of frames (The Video has 147064 frames [ 01:42:13 ])

regards
t :)

[xmicky]

I've tried Oddballs workround for removing XviD packed bitstreams on three Xvid movies encoded with Gordian Knot and Xvid Koepi's 1.0.1 codec with packed bitstreams. My Elta 8883 will not playback the workround generated 3ivx video. I get sound and no video.
Any comments?

Any views on THE PROS & CONS on whether useing DIVX with packed bitstreams & maybe GMC (which play on the 8883) rather than XVID with no packed bitstreams and no GMC?

[xmicky]

Of course MPEG4MODIFIER unpacks the stream and allows the the XVID to play properly on the Elta 8883.

[masterleo66]

Don't work. Final avi=sound but no picture!(on DivX player, ofcourse)
Any sugestions, please?

[monty0815]

which method did you use?

[masterleo66]

Originally posted by CrashUK
I made my own way to do this so they work on my yamada 6600.. But should work for other players too.. its same as doom9 one but it comes with all needed software and all graph files are premade so you dont need to make them every time and it comes with checking tools so you can check for packeted bitstream..


Sorry, don't work. When i load up graphedt and open "Part One - avi to mp4.GRF, a message error say: "some files the filter graph uses are no longer available". Eny sugestions?




Change Log - v1.0 to v1.1

Added: missing file 3vix test.GRF

Download Here:: remover packeted bitstream_v1.1.rar (http://homepage.ntlworld.com/david.allison33/packeted_bitstream_remover_v1.1.rar)

I premade all the graph files you need so they no need to make any graphs like in the doom9 one..

Also this is how long it should take per 700 meg avi

Part 1 - 1-2 Mins
Part 2 - 1-2 Mins
part 3 - 2-3 mins
Part 4 - 5 secs

Overall 4-7 mins maybe 10 mins on older systems


Also this is the readme below:

How to remove packeted bitstream from XVID Files



Unpack this zip to your C:\

Now before you start you need to install 3VIX "3ivx_d4_451_win.exe" which you can get from http://www.3ivx.com

when you installed that you need to load up graphedit and open "3vix test.GRF"

click on 3vix D4 Media Spiltter

and put Allow Unsupported Decodes On and now and then click Ok..

Ok now we get started....

now get your non-working avi and copy it to c:\avi_files\ and rename it bad.avi

now load up graphedt and open "Part One - avi to mp4.GRF"
now click play should take about 1-2 mins to do this bit you know when it's done when the play icon go black..

now that you done that...





open "Part Two - mp4 to avi.GRF" in graphedit and click on play..

Now when thats done we go to Part3



part3 open "bad.avi" in VirtualDubMod

and On the put down menu click Streams/steam list... now click demux and save sound file as "sound"

Now Open "Fixed_with_out_sound.avi" in VirtualDubMod

On the put down menu click Streams/stream list... and click add and open soundfile and then click ok.

"NOTE: on VBR MP3 Only if it ask you to rewite vbr header always click no"

Now on the put down menu click Video/save as in the box where it says video mode change it
from Full processing mode to Direct stream copy. now enter file name "FIXED" and click save




Now Part 4 this last part..

load up avic.exe and open fixed.avi

now put xvid on the box that says "FourCC Description Code:"
and Put XVID in the box that says "FourCC Used Codec:"

now click Apply..

now your done.. :)

rename fixed.avi to the real file name e.g. my movie disc 1.avi

you can run "delete_temp_files.bat" and it will delete all temp
files you made.

NOTE: Part4 Maybe Only Be Needed On Yamada 6600.. Not Tested This on Other Players

[monty0815]

Did you change the FourCC Descriptors as described in step 4?
Or did you give mpeg4modifier a try? It's faster and easier to use.

cheers, monty

[masterleo66]

Mpeg4Modifier is an excelent tool for bitstream depacked. Work 100%.
The other method don't work(sound but no picture).
Thanks.

[Krack]

hi,
Mpeg4modifier works fine with a lot of files but with some I have:
"Error no dummy N-Vop for packed frames"

I have tried to direct streamed it with nandub but it changes nothing.

Any help for understanding and bypassing this error?

[Krack]

ok I've found the 1.1.0 version of mpeg4modifier and it's showing me no error.
I'll watch the new encoded ep to see if everything's allright.

[Krack]

ok it works but there are some speeding moments specially where video parts have been pasted.

[thop]

Does anyone know of a linux tool that can remove the packed bitstream? MP3Box can do the job of removing the packed bitstream, albeit in 2 steps (AVI->MP4, MP4->AVI) and unfortunately it will only tell you if the file has packed bitstream/>2 BVOPs _after_ it converted the file. Is there any tool that just scans through the file without converting?

Hmm maybe i'll be able to fiddle with the source a bit and add an extra switch.

Ok made a check switch.

[monty0815]

Give Avidemux2 a try, it works quite well for me, but I haven't tested it a lot. Website is http://fixounet.free.fr/avidemux/
There's a nice guide to unpacking bitstreams on the page.

cheers, monty

[thop]

Thanks i'm gonna look into it :)

Unfortunately it's no command line tool :/

[monty0815]

You can use it as a command line tool, something like this should do the trick:
avidemux2 --force-unpack --load $SRC/%f --audio-map --save-unpacked-vop $TGT/fixed_%f --quit

but you still need X11 and gtk running.

cheers, monty

[thop]

Originally posted by monty0815
but you still need X11 and gtk running.

unfortunately that's the problem :/