conversion from msmpeg4v3 to iso mpeg4 attempt

[mat]

Hi someone post a code that try to do the conversion on ffmpeg mailling list : http://sourceforge.net/mailarchive/forum.php?thread_id=5929420&forum_id=1506

If someone is interested in reviewing/improving his code...

[bond]

wow, cool stuff, hopefully someone will pick it up :)
would be great to be able to convert my old divx3.11 collection to brand new spec compliant mpeg-4 files...

thanks to koepi for moving the thread from the xvid to the new a/v forum. this topic isnt really related to xvid

[PowerMacG4]

Who was the person who said he was doing this exact thing.. and then disappeared

[BoNz1]

Originally posted by PowerMacG4
Who was the person who said he was doing this exact thing.. and then disappeared

You mean -h (http://forum.doom9.org/showthread.php?s=&threadid=42253) ? I don't think this is the same person though.

[jggimi]

A new user on the forum, pa. He's sent me the executable, which I've made available at <link deleted>.

He should be joining this thread as soon as his waiting period completes.

[bond]

thanks a lot for the link jggimi :)

use the tool with the following commandline:
ffmpeg -i inputdivx3.avi -vcodec mpeg4 -acodec copy -mstomp4 outputmpeg4.avii also talked to him and here is a brief summary of the tool up to now given to me by him:
- it's a hack which passes quantized coeffs, mb types and vectors from the msmpeg4 decoder to the mpeg4 encoder directly
- it works for most msmpeg4v3 files, often with only a 3% size penalty
- it converts about 80% of the msmpeg4v3 videos he tried perfectly, but he can't figure out what's wrong with the rest
- all nandub rips seem to convert fine
- there are still bugs (some files fail to convert, it's no faster than a regular re-encode, mmx must be disabled, etc.)
- on some problem samples, it looks like certain inter frames aren't having their intra blocks copied properly
- it's almost as if certain blocks are being decoded differently for no apparent reason
- it all looks ok in terms of mb types, coeffs, etc.
- the problems only happen with files he doesnt know how they have been created
- it doesnt seem to be related to old nandub versions, as he, in order to do the conversion, stepped through every line of code that was involved, and there weren't any different workarounds being invoked between files that worked and didn't work
- if you tried "ffmpeg input.avi output.avi" it was fine (albeit re-encoded lossily), but if you tried "ffmpeg -mstomp4 input.avi output.avi" the decoded mb types, vectors and coeffs are the same as before, but it is displayed differently (-mstomp4 tells ffmpeg that the conversion will be lossless. without it ffmpeg does a regular decode->encode)
- one thing he hasn't done is decode the original and the buggy converted streams side-by-side and check that way
- he suspects that there's something ffmpeg is doing that he hasn't noticed till now

my findings till now:
- i tried decoding a problem sample myself with various decoders (ffdshow, 3ivx, xvid, divx5) in both the divx3 and mpeg-4 version and the used decoder shows no difference for the output
- maybe this has nothing to say but when i compressed a divx3.avi and the correctly converted resulting mpeg4.avi of it with winzip i got a 1% additional winzip compression
when i compressed a divx3.avi and the not correctly converted resulting mpeg4.avi of, winzip compressed the mpeg4.avi with 7%, which is pretty high i think?
- trying to place the resulting mpeg-4.avi to .mp4 via mp4box works fine (mp4box doesnt handle importing divx3 streams with "uncorrect" mpeg-4 fourccs!), still of course the distortions are still there. also mp4box detects a not existing audio stream, one time mpeg-1 (with the working sample), another time mpeg-2 (with the not working sample), which it never did before, maybe the .avi writing is not 100% correct?


and of course he would be happy to recieve some support of devs knowing libavcodec if possible for being able to make this tool even better :)

[Selur]

Nice,.. hope this guy and the libavcodec codec guys get together and fix and tune this tool. DivX3.11;) => iso compliant mpeg4 would be great :D

Cu Selur

[thoralf]

Why on earth would one convert divx3.11 to mpeg4?
Admittedely, msmpeg4 was the first codec that allowed reasonably good looking DVD backups. Then again, it's quite ancient nowadays and quality-wise definitely not on par with xvid or h264. In addition, it would even make our favourite mpeg4-codec looking bad - a native mpeg4-encode will look definitely better than a transcoded one.
By converting those files to mpeg4, one would gain no quality boost at all, and most standalone DVD players are capable of decoding divx3 directly. I just don't get the point of this ...

[ReinerSchweinlin]

For me this makes perfect sense, since i have a lot of redorded tv-shows which I stored in DIVX3.11 in the old days. Now, when played back on my KISS 500, the Sound ist not processed in 16Bit but in 8 Bit, which makes it impossible to use my Denon 1602 to do the sound.
I get ur point, but there IS a great use in this!

[Selur]

I personally would like to convert some old home clips to a standard so that mpeg4 compatible player will have no problem with it. And no, reencoding the vhs tapes to xvid etc. is no option since it's long gone ;).

Cu Selur

[bond]

so did anyone test this tool already? it should work already on most files

Originally posted by Selur
I personally would like to convert some old home clips to a standard so that mpeg4 compatible player will have no problem with it.indeed :)

[jggimi]

Bond sent me pa's original divx 3 .avi that will show artifacts when converted to mpeg-4 using this tool. I haven't tried it yet, but have made the 3MB file available at:

<link deleted>

[gotaserena]

I did a few conversions... Almost all files crashed gabest's avi splitter when I tried to play them.

[bond]

Originally posted by jggimi
Bond sent me pa's original divx 3 .avi that will show artifacts when converted to mpeg-4 using this tool. I haven't tried it yet, but have made the 3MB file available at:
http://jggimi.webhop.net/divx3-to-mpg4_sample.zipthanks jggimi :)

would be great if people try this tool on this sample and share their opinions on what could cause these artifacts and maybe pointing to hints to fix this problem

[Shtirlits]

I did a conversion on a ~700 Mb file. After the conversion video played alright, but sound didnt. I dunno if its intended.

[bond]

this tool converts divx3 video to mpeg-4 video losslessly, it doesnt deal with audio

[pa]

yes the tool has some problems, and the ffmpeg cvs it was built from has a problem with pts/dts generation which makes bad .avi files.

i plan on working on this further but it is hard without help from people familiar with libavcodec h263 coding. there are a number of other fixes that must be made for this conversion to work (frame skipping, re-quant frames where quant>25), perhaps next week some time. i don't expect this to make it into the official ffmpeg cvs so it will always be a standalone tool.

[mat]

Originally posted by bond
thanks jggimi :)

would be great if people try this tool on this sample and share their opinions on what could cause these artifacts and maybe pointing to hints to fix this problem

that's strange : it look like the first keyframe is wrong : in msmpeg it is black, in the mpeg4 it start with the star.


If I remenber well some divx3 suffer from frozen frame, could the program that fix the problem make the stream invalid ?
Or on the contrary apply it could fix the problem ?

Do you know how was made this divx ?

[mat]

i have checked the not-working avi with avidemux (~virtualdub for linux) and it seem the first 60 are null and not valid...


So it seem in order to convert this file you have to remove all the null frame (avidemux saw 66 bad frame), another solution could to add a check that skip these bad frame during the conversion.


Pat, I hope you will continue to improve it, and perhaps when it would be stable, it could be merged with ffmpeg.

Matthieu

[jggimi]

Due to space requirements, I am removing the software and sample links by Friday 17 December.

[chilledoutuk]

this is very cool.

I remember being impressed with this codec when i was bored and wen through the codecs in ffdshow.

I just did a backup of a music video and then used this tool to convert it and it plays back fine with the xvid decoder I may have to try some other decoders but so far it seems to work well.

I dont like people who imediatly assume that because a codec is not new its a pos comparing this codec to divx 5.2 and a few others it seems to me to be a very useful codec indeed.
The clarity and sharpness are its main features that come to mind gives xvid a run for its money also dont forget that its fast.

Thinking about it would it not be possible to integrate this into the ffdshow divx3 encoder as an option say a check box for making video mpeg-4 compliant. ( thay would be cool)

[bond]

Originally posted by chilledoutuk
Thinking about it would it not be possible to integrate this into the ffdshow divx3 encoder as an option say a check box for making video mpeg-4 compliant. ( thay would be cool)indeed, this tool opens lots of possibilites, i hope someone will continue the work on it :)

[akupenguin]

Originally posted by chilledoutuk
Thinking about it would it not be possible to integrate this into the ffdshow divx3 encoder as an option say a check box for making video mpeg-4 compliant. ( thay would be cool)
Then it isn't divx3 anymore. This tool converts divx3 into mpeg4 Simple Profile. If you want mpeg4 SP, you can already do that without going through Microsoft's perversion in the middle.

The only purpose of this tool is to standardize old videos that you don't have the option to re-encode.

[ReinerSchweinlin]

Wow.

Just tested this tool. It worked great on an old TV-Recording of mine. It plays back perfectly on:
- mplayer für Windows.
- Kiss DP 500


Video seems to play flawless when decoded with ffdshow (hard to tell, it´s anime, will have to try a more smoother source, but there is noch audio through ffdshow-audio section. Same with Nero Showtime and it´s filters. xvid dshow and vfw decoder also decodes picture fine, but all windows mediaplayers based on dshow give no sound.

Thrown into virtualdub, no errors in the stream appear. Trying to play back (xvid and fhi-mp3), virtualdub crashes. Tried re-saving in stream-copy mode, still crashing. After demuxing audio and re-inserting, directstream copy works fine. Now the file plays perfect in windows-mediaplayer with different decoders....
Original File : 143mb
mstomp4: 148mb
vdbub remux: 147mb

Opening the old and the new files in drf_analyzer shows exact the same values, both for the divx3.1 and the new version (well, as it should..) :)

Great tool, hope someone will continue development. A simple GUI would be great to batch-convert files and automaticaly remux them with vdub oder avimux :)
Thanx for this!!

[opsis81]

Try working with audio and video seperately

SET INPUT=C:\movie.avi
SET VIDEO=C:\video.avi
SET AUDIO=C:\audio.mp3

ffmpeg -i "%INPUT%" -vcodec mpeg4 -an -mstomp4 -f avi -y "%VIDEO%"
ffmpeg -i "%INPUT%" -acodec copy -vn -f mp3 -y "%AUDIO%"

Then mux audio and video with virtualdub.
It will work this way.


I convert old DivX3/mp3 videos to mp4 with great success

SET INPUT=C:\movie.avi
SET OUTPUT=C:\movie.mp4

ffmpeg -i "%INPUT%" -vcodec mpeg4 -an -mstomp4 -f avi -y "%OUTPUT%.avi"
ffmpeg -i "%INPUT%" -acodec copy -vn -f mp3 -y "%OUTPUT%.mp3"
MP4Box -add "%OUTPUT%.avi" -add "%OUTPUT%.mp3" "%OUTPUT%"
del "%OUTPUT%.avi" & del "%OUTPUT%.mp3"

[ReinerSchweinlin]

@opsis81
Thanx for the hint. What´s mp4box? The final product will be a .mp4, right? I think, my kiss plays these, too. So that would be fine. Is there a way to do this in a batch? (I´m not that much into CLI).

Greets Reiner

[bond]

note that ffmpeg should also be able to output to .mp4 directly itself (dunno if these outputted .mp4 files are 100% mpeg-4 spec compliant tough, never really tested it)

:search: mp4box is a very good opensource commandline .mp4 tool of the gpac project (muxing and a lot of other stuff)

[yakima]

hi,
is the div3->mpeg4 tool still online somewhere? or could someone make it available?
i thought i had it dled, but, well, can't find it ...
thanks!

[ReinerSchweinlin]

I´v hosted the file here:

http://www.hardware-test.de/doom9/ffmpeg-mstomp4.rar

Enjoy :)

[yakima]

thanks!
my first test went fine - i don't see anything strange, no artifacts that aren't in the source avi - i didn't have a chance to see the faulty one that was posted, so could someone tell me how obvious the problems with faulty p frames should be, are they everywhere or is it just a few frames? did anyone encounter other problems (aside from the buggy avi writing)?
thanks to pa for this great tool!

[bond]

i just did a bunch of tests on divx3 streams i had and i have to say: "this tool really kicks ass" :)

[JLOcioso]

Where i can find the source code of this app to compile it for macos?

I want to convert some DIV3 to DX50, cause my dvd player doesn't support DIV3....but I want a losslessly transformation....

[bond]

the code for the ffmpeg patch is linked to in the first post

[planet1]

Would patching the "latest" ffmpeg CVS, improve this div3->mpeg 4 asp process even more ?

[planet1]

obiously the tool does DivX3 -> mpeg4 SP not ASP :o conversions

the outputted avis mux greatly with mp4muxer

and mplayer handles the mp4 (SP@L1, AAC) without a problem :p


btw MPEG4 Modifier also works on the new avi (fourcc=divx), in case there is need for manipulating the ratio or "user data".

[ReinerSchweinlin]

After looking at my old archive of TV-Clips, i figured it would be a nice way to write a little gui for this nice tool. Sadly, I´m not into programming. In the old days, I did some Basic and Logo-Stuff, so basic undrestanding of logical structures should be available and I like a challenge. Can anybody suggest a nice Dev-Kid (free, open source..) to build a little GUI? I guess calling some tools and file-handling should be enough as the "work" is done by external tools.
Any hints ?

[Elias]

Why can't KurtNoise write a GUI for this tool?

[ReinerSchweinlin]

Don´t know :) We have to ask him :) I´ll write him a PM :)

[Kurtnoise]

Why can't KurtNoise write a GUI for this tool?
Désolé, je ne suis pas une vache à lait...In other words, no way. :rolleyes:


btw, you can try with UniversalFront (http://unifront.boereck.de/). It's very easy for non developpers and simple users like you. You create your own frontend via some scripts. It's dedicated for audio but it should work with msmpeg4 tool.

[ReinerSchweinlin]

Thanx, I´ll take a look at it tomorrow :)

[ReinerSchweinlin]

Ok, here is a quick hack, if anyone wants to try this:

Load Universal Front from:
http://unifront.boereck.de/download.html
Install it
Load Plugin and ffmpeg.exe in a zip from:
http://www.hardware-test.de/doom9/msmpeg4tompeg4sp.zip
Unzip in Universal Front Dir
Run Universal Front
Choose "MSMPEG4 to MPEG4SP"
Choose different destination dir from source
drop files in list
RUN :)



This takes (drag & drop) a series of MSMPEG4 Files (like DIVX 3.11, MSMPEG42, etc.) files an converts them to mpeg4SP via ffmpeg.exe. This really doesn't do much, just passes one file to the .exe and then takes the next one. Things that would be nice to have in the future:

- A check, if the AVI really ist MSMPEG4/Divx3.xx (migth be possible with gspot export-funktion?)
- Demulitplexing of Audio
- Mulitplexing of converted stream with demulitplexed Audio via virtualdub, so streams are really compliant to play on Standalones.

Don´t have much time... we´ll see if there will be more on this...

[Elias]

Tried it. Not working.

[bond]

Tried it. Not working.is that supposed to be a bug report? ;)

here it works fine

[ReinerSchweinlin]

Here too :)

[madman1980]

Ok, here is a quick hack, if anyone wants to try this:

Load Universal Front from:
http://unifront.boereck.de/download.html
Install it
Load Plugin and ffmpeg.exe in a zip from:
http://www.hardware-test.de/doom9/msmpeg4tompeg4sp.zip
Unzip in Universal Front Dir
Run Universal Front
Choose "MSMPEG4 to MPEG4SP"
Choose different destination dir from source
drop files in list
RUN :)



This takes (drag & drop) a series of MSMPEG4 Files (like DIVX 3.11, MSMPEG42, etc.) files an converts them to mpeg4SP via ffmpeg.exe. This really doesn't do much, just passes one file to the .exe and then takes the next one. Things that would be nice to have in the future:

- A check, if the AVI really ist MSMPEG4/Divx3.xx (migth be possible with gspot export-funktion?)
- Demulitplexing of Audio
- Mulitplexing of converted stream with demulitplexed Audio via virtualdub, so streams are really compliant to play on Standalones.

Don´t have much time... we´ll see if there will be more on this...

Unfortunately audio is still missing in the resulting file :(

[foxyshadis]

I think that's what

- Mulitplexing of converted stream with demulitplexed Audio via virtualdub, so streams are really compliant to play on Standalones.

meant. You'll probably have to mux the audio in yourself (via virtualdub for avi or yamb for mp4).

[ReinerSchweinlin]

Read the whole thread, it´s in fact the case that audio is not dealt with correctly, u have to demux it from the original file and then mux it into the new avi. I wanted do script something to automate this, but since this type of file is rare nowadays, I was too lazy :)

[madman1980]

Ok, I misunderstood the description so I thought it meant it would demux and remux the audio.

The file I'm trying to do is Divx3 OpenDML and has something that Gspot calls "Mpeg2 Layer3 audio". That might be the reason that I don't seem to be able to demux it, because Vdub, vdub mp3, vdub mpeg2, nandub and every version i've tried won't demux the audio....

[foxyshadis]

That's just a low-bitrate mp3. VDubMod should be able to demux it through its streams list. Alternatey, you can use graphedit.

[sneaker_ger]

Sorry for digging out this old thread, but are the described problems fixed now? Are there any news about an updated tool?

[ReinerSchweinlin]

The patched ffmpeg works just fine. AFAIK nobody took the effort to fix the muxing-problem since it´s easy solvable via vdub(mod).

[sneaker_ger]

I mainly meant this problems:


- it converts about 80% of the msmpeg4v3 videos he tried perfectly, but he can't figure out what's wrong with the rest
- there are still bugs (some files fail to convert, it's no faster than a regular re-encode, mmx must be disabled, etc.)
- on some problem samples, it looks like certain inter frames aren't having their intra blocks copied properly
- it's almost as if certain blocks are being decoded differently for no apparent reason
- the problems only happen with files he doesnt know how they have been created
- it doesnt seem to be related to old nandub versions, as he, in order to do the conversion, stepped through every line of code that was involved, and there weren't any different workarounds being invoked between files that worked and didn't work
- if you tried "ffmpeg input.avi output.avi" it was fine (albeit re-encoded lossily), but if you tried "ffmpeg -mstomp4 input.avi output.avi" the decoded mb types, vectors and coeffs are the same as before, but it is displayed differently (-mstomp4 tells ffmpeg that the conversion will be lossless. without it ffmpeg does a regular decode->encode)
- he suspects that there's something ffmpeg is doing that he hasn't noticed till now