fixMKV v2 *updated* - MKV Header Compression fix

[7ekno]

Hi all,

New version in post 19 BELOW! (http://forum.doom9.org/showpost.php?p=1526954&postcount=19), stuff kept here for prosperity and continuity of replies :P

I have a couple of standalone media players that don't like the recent MKVToolnix v4.1.1 or greater MKVmerge default of using header compression ;)

So as an interim fix (until firmware upgrades available, etc) I have coded this quick and dirty BAT file to convert all problem MKVs in a directory into good behaving citizens on media players!

@ECHO OFF
SET HOMEDIR=%~dp0
REM SET %MKVTOOLPATH=C:\Program Files\MKVToolnix
SET KEY="HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\App Paths\mmg.exe"
FOR /F "tokens=2*" %%A IN ('REG QUERY %KEY%') DO SET MKVTOOLPATH=%%B
IF "%MKVTOOLPATH%" == "" (
DIR "%HOMEDRIVE%\mkvmerge.exe" /s /b >"%HOMEDIR%findmkvmerge"
SET /P MKVTOOLPATH=<"%HOMEDIR%findmkvmerge"
DEL "%HOMEDIR%findmkvmerge"
)
IF "%MKVTOOLPATH%" == "" (
ECHO MKVToolnix NOT found!!!>"%HOMEDIR%logging.txt"
GOTO STATUS
) ELSE (
ECHO MKVToolnix found at "%MKVTOOLPATH%">"%HOMEDIR%logging.txt"
ECHO.>>"%HOMEDIR%logging.txt"
)

:STARTFILES
IF "%~dpnx1" == "" GOTO DRAGDONE
ECHO PROCESSING ALL FILES DRAGGED AND DROPPED!!
ECHO PROCESSING ALL FILES DRAGGED AND DROPPED>>"%HOMEDIR%logging.txt"
ECHO.>>"%HOMEDIR%logging.txt"
SET dropped=yes
CALL :MKVFIXMUX %1 "%MKVTOOLPATH%"
SHIFT
GOTO STARTFILES

:DRAGDONE
IF "%dropped%" == "yes" GOTO STATUS
ECHO PROCESSING ALL FILES IN THIS DIRECTORY AND ALL SUBDIRECTORIES!!
ECHO PROCESSING ALL FILES IN THIS DIRECTORY AND ALL SUBDIRECTORIES>>"%HOMEDIR%logging.txt"
ECHO.>>"%HOMEDIR%logging.txt"
FOR /F "delims=*" %%A IN ('dir /b /s *.MKV') do CALL :MKVFIXMUX "%%A" "%MKVTOOLPATH%"

:STATUS
START "" "%HOMEDIR%logging.txt"
ping 1.1.1.1 -w 1000 -n 2$1>nul
del "%HOMEDIR%logging.txt"
GOTO END

:MKVFIXMUX
"%~dp2mkvmerge.exe" -o "%~dpn1_fix.MKV" -A -S --compression -1:none "%~dpnx1" -D -S --compression -1:none "%~dpnx1" -A -D "%~dpnx1">>"%HOMEDIR%logging.txt"
if not errorlevel 0 (
ECHO FAILED! on "%~dpnx1" remux!>>"%HOMEDIR%logging.txt"
)
if exist "%~dpn1_fix.MKV" (
del "%~dpnx1"
rename "%~dpn1_fix.MKV" "%~nx1"
ECHO SUCCESSFUL! processing of "%~dpnx1">>"%HOMEDIR%logging.txt"
)

:END

Copy and paste the above code as FIXMKV.CMD into your Top Level Directory that requires fixing (it will parse everything in that directory and everything in ALL subdirectories) OR create the file on your Desktop (for drag & drop functionality)!

It requires MKVToolnix v4.1.1 or later INSTALLED or UNZIPPED onto the Windows Install drive (if you didn't install MKVToolnix and it's NOT on the same drive as Windows, you will have to remove the 'REM' command and manually specify where the MKVToolnix directory is located) ...

7ekno
EDIT: Code fixed for inadvently removing subtitle header compression :P
EDIT2: Code rewritten for Drag & Drop and Double Click function with better logging and self cleanup :P

[tebasuna51]

I don't found info in mkvmerge help about:

--engage keep_bitstream_ar_info

For what is necessary?

For me work for all tracks:

--compression -1:none "%~dpnx1"

instead:

-A -S --compression -1:none "%~dpnx1" -D -S --compression -1:none "%~dpnx1" -A -D --compression -1:none "%~dpnx1"

What is the difference selecting video, audio and subs separately?

[canTsTop]

there is special tool for this purpose, mkWDclean http://www.matroska.org/downloads/mkclean.html

[sneaker_ger]

I don't found info in mkvmerge help about:

--engage keep_bitstream_ar_info

For what is necessary?

In older versions mkvmerge would delete the SAR info from H.264 streams. As it does not do that anymore in newer versions this option has become obsolete.

[Atak_Snajpera]

For me work for all tracks:

--compression -1:none "%~dpnx1"
Frankly i wouldn't disable compression on subtitles (vobsub)

[b66pak]

there is special tool for this purpose, mkWDclean http://www.matroska.org/downloads/mkclean.html

a .bat like this will remove header compression for audio and video (tested with a file generated with mkvmerge v4.2.0 default settings for compression):

for %%a in ("*.mkv") do mkWDclean --optimize %%a
_

[roozhou]

I don't found info in mkvmerge help about:

--engage keep_bitstream_ar_info

For what is necessary?

There are 18 secret hack options in mkvmerge, all started with --engage. The author provides no documentation about these options, you can only learn it from source code.

keep_bitstream_ar_info keeps the original SAR information in SPS for AVC when you specify a DAR for the video stream. It is on by default now so it's not needed any more.

[sneaker_ger]

There's a short explanation for most these options in mmg under "mux">"add command line options".

[Mosu]

Those options are not supported (meaning I will not give support for questions regarding the use of or the result of the use of those options). They're development hacks, hacks used for testing, stuff that simply doesn't work correctly yet etc etc.

[tebasuna51]

Frankly i wouldn't disable compression on subtitles (vobsub)

There are now support for compressed Vobsub subtitles (zlib I suppose) in standalone players like WD TV?

@all
Thanks for your tips.

[Mosu]

VobSub subtitles have always been compressed in Matroska -- so either a player supports zlib compressed VobSubs or it doesn't support VobSubs in Matroska at all. That would be my guess.

[shaxs]

Hi Everyone,

So my 720p MKV tv shows used to work fine until recently when streamed to my Patriot Box Office or my Xbox 360. Now neither can play more recent MKV files. I tried your .bat fix and my xbox still cannot play it. I have no tried the Patriot yet. Any clue how to get MKVs to work on the Xbox again using windows media center on the xbox?

[tormento]

Tried to mux a mkv with MKVToolnix 4.3.0 and then applied MKWDClean with no parameters. The files still does not work on WDTV Live.

What am I doing wrong?

[Video Dude]

Tried to mux a mkv with MKVToolnix 4.3.0 and then applied MKWDClean with no parameters. The files still does not work on WDTV Live.

What am I doing wrong?
Use 4.0.0 and see if it works.

[tormento]

Use 4.0.0 and see if it works.
With < 4.1.0 I don't even need to clean or remux.. I know that..

[piratburner]

Would it be possible to update the CMD file to work on network share ?

I have all my MKV on a unraid server.

//Peter

[7ekno]

Would it be possible to update the CMD file to work on network share ?

I have all my MKV on a unraid server.

//Peter

BAT/CMD do not support UNC (http://compnetworking.about.com/od/windowsnetworking/g/unc-name.htm) names ;)

The only way to achieve that is to map a share to network drive ...

7ek

[jeff3611]

Thanks 7ekno, just the tool I need to simplify things.

However, is that possible to include a checking routine before fixing every mkv in the dir tree. What I meant was fix only the mkv with compressed header in it.

Cool scripts, thanks.

[7ekno]

However, is that possible to include a checking routine before fixing every mkv in the dir tree.

Sure! I bring you fixMKV v2 :P

Tools needed:
MKVToolnix (http://www.bunkus.org/videotools/mkvtoolnix/downloads.html#windows) - Either installed or unzipped to the same drive as Windows (if not, you will have to remove the "REM" statement and manually enter the file path to mkvmerge.exe)
MediaInfo CLI version (http://mediainfo.sourceforge.net/en/Download/Windows) - Use the same version as your OS (so 64bit for 64bit OS, 32bit for 32bit OS), extract "MediaInfo.dll" and "MediaInfo.exe" to the same folder as fixMKV v2 (if not, you will have to manually enter the file path to MediaInfo.exe by deleting the %~dp0 in the "SET MEDIAINFOPATH=%~dp0" line and replacing it with your file path)

@ECHO OFF
SET HOMEDIR=%~dp0
SET MEDIAINFOPATH=%~dp0
SET LOGFILE=%~dpn0_log.txt
REM SET MKVTOOLPATH=C:\Program Files\MKVToolnix
SET KEY="HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\App Paths\mmg.exe"
FOR /F "tokens=2*" %%A IN ('REG QUERY %KEY%') DO SET MKVTOOLPATH=%%B
IF "%MKVTOOLPATH%" == "" (
FOR /F "tokens=*" %%I IN ('DIR "%HOMEDRIVE%\mkvmerge.exe" /s /b') DO SET MKVTOOLPATH=%%I
)
IF "%MKVTOOLPATH%" == "" (
ECHO %DATE%%TIME%: MKVToolnix NOT found!!!>"%LOGFILE%"
GOTO STATUS
) ELSE (
ECHO %DATE%%TIME%: MKVToolnix found at "%MKVTOOLPATH%">"%LOGFILE%"
)
IF EXIST "%MEDIAINFOPATH%MediaInfo.exe" (
ECHO %DATE%%TIME%: MediaInfo.exe CLI found at "%MEDIAINFOPATH%">>"%LOGFILE%"
) ELSE (
ECHO %DATE%%TIME%: MediaInfo.exe CLI NOT FOUND at "%MEDIAINFOPATH%">>"%LOGFILE%"
GOTO STATUS
)
IF (%1) == () GOTO CLICKED
ECHO FixMKV v2 removing Header Compression from all files DROPPED!!
ECHO %DATE%%TIME%: PROCESSING ALL FILES DRAGGED AND DROPPED>>"%LOGFILE%"

:DROPFILES
IF (%1) == () GOTO STATUS
CALL :MKVFIXMUX %1 "%MKVTOOLPATH%"
SHIFT
GOTO DROPFILES

:CLICKED
ECHO FixMKV v2 removing Header Compression from all files in %HOMEDIR% AND all Sub-Directories!!
ECHO %DATE%%TIME%: PROCESSING ALL FILES IN %HOMEDIR% + SUBDIRECTORIES>>"%LOGFILE%"
FOR /F "delims=*" %%A IN ('dir /b /s *.MKV') DO CALL :MKVFIXMUX "%%A" "%MKVTOOLPATH%"

:STATUS
START "" "%LOGFILE%"
PING 1.1.1.1 -w 1000 -n 2$1>nul
DEL "%LOGFILE%"
GOTO END

:MKVFIXMUX
"%MEDIAINFOPATH%mediainfo.exe" "%~dpnx1">"%HOMEDIR%%~n1.nfo"
FIND /C /I "Header stripping" "%HOMEDIR%%~n1.nfo">nul
IF NOT ERRORLEVEL 1 (
ECHO %DATE%%TIME%: "%~nx1" - Header Compression Found, processing ... >>"%LOGFILE%"
"%~dp2mkvmerge.exe" -o "%~dpn1_fix.MKV" -A -S --compression -1:none "%~dpnx1" -D -S --compression -1:none "%~dpnx1" -A -D "%~dpnx1">nul
IF ERRORLEVEL 1 (
ECHO %DATE%%TIME%: "%~nx1" - REMUX FAILED somehow!!>>"%LOGFILE%"
GOTO CLEAN
)
IF EXIST "%~dpn1_fix.MKV" (
DEL "%~dpnx1"
RENAME "%~dpn1_fix.MKV" "%~nx1"
ECHO %DATE%%TIME%: "%~nx1" - SUCCESSFUL remux.>>"%LOGFILE%"
GOTO CLEAN
)
) ELSE (
ECHO %DATE%%TIME%: "%~nx1" - SKIPPED - No Header Compression.>>"%LOGFILE%"
GOTO CLEAN
)
:CLEAN
DEL "%HOMEDIR%%~n1.nfo"

:END

The advantages of fixMKV v2 are:
- Bit more neat and concise logging with autocleanup
- The use of MediaInfo to determine if an MKV has header compression or not and skip the file if nothing needs doing
- Can be run from a top level directory and will automatically crawl all sub-directories
- Files can be dragged and dropped on the BAT file for processing (and will still be skipped if not needed)

Cheers,
7ekno

[jeff3611]

Sure! I bring you fixMKV v2 :P




Super cool!, thanks again.

[Devilman1]

Using MKVToolnix would not lose any TAG added to the different stream? like for example language or comment etc.?

[RedDwarf1]

It could be useful but it doesn't seem to recognise header compression correctly. It only recognises header stripping and not lzo (lzo1x) or zlib. Surely they are also compressed headers, although they do seem to make the file larger than having header compression disabled.

I don't know whether those types cause any problems with standalone media players. Has anyone tested by re-muxing an MKV?

[Mosu]

zlib compression has been in use since 2002. Practically since VobSubs have been supported in Matroska. If a player doesn't support zlib compression... oh well.

Also a note about the terms you're using. In Matroska a compression algorithm can apply to a) the track headers only (meaning the CodecPrivate element), b) to each and every block (each frame) or c) to both of them. zlib is usually used for VobSub subtitles and other subtitles that are actually not really compressed. In such cases zlib compression is used on c) both the track headers and each frame. Implying that zlib and lzo are "header compression" algorithms is therefore misleading.

Compressing highly compressed content, e.g. h264 video tracks, with zlib will of course increase file size, or at best not decrease it. That's normal, hence zlib compression is usually only used for subtitles.

[Xad]

a .bat like this will remove header compression for audio and video (tested with a file generated with mkvmerge v4.2.0 default settings for compression):

for %%a in ("*.mkv") do mkWDclean --optimize %%a
_Sorry to bump this thread. I'm using Windows XP and i tried using this code with mkWDclean but it doesn't work; it seems the spaces in a filename are causing issues? If i enter the pause command i can see that it says for example:
Unknown parameter Spider
Unknown parameter Man

However, if i enter one block name, for example spiderman, then it works fine. What do i need to add to that quote for it to work? I've tried placing quotation marks around different places but no luck.


7ekno, i also tried using fixMKV2 and i keep getting an error telling me remux has failed. I have MKVtoolnix unzipped on C drive (C:\mkvtoolnix\), have the 2 media info files, entered the location manually and removed REM, but still keep getting that error: "Header Compression Found, processing" and then: "REMUX FAILED somehow". I tried drag and drop but still the same error.

[sneaker_ger]

You need to put quotes around the "%%a" variable. The current mkvmerge versions have reverted to header removal compression off by default, btw, so you could just use the newest mkvmerge:
for %%a in (*.mkv) do mkvmerge -o "%%~na_processed.mkv" "%%a"

If you have one of those problematic players:
for %%a in (*.mkv) do mkvmerge --clusters-in-meta-seek --engage no_cue_duration --engage no_cue_relative_position -o "%%~na_processed.mkv" "%%a"
(https://trac.bunkus.org/wiki/FAQ%3AImprovingPlaybackCompatibilityWithPlayers)

[SeeMoreDigital]

It may be worth noting...

The new versions of MKVmerge don't add header compression 'by default' any more...

[Xad]

Thanks sneaker_ger, that works perfectly. Wanted it for some random old video files that i never fixed, but good to hear it's not default any more. Thanks again.

[doomer9er]

Bumping an old thread.

anyone care to make a fixmkvV3 that works today for the latest mkvtoolnix gui? Newbie here. Thanks.

[Liisachan]

Bumping an old thread.

anyone care to make a fixmkvV3 that works today for the latest mkvtoolnix gui? Newbie here. Thanks.

mkclean (https://sourceforge.net/projects/matroska/files/mkclean/) 0.8.10 (August 2017) does support the current matroska (v4) with several options (perhaps including what you want to do):
mkclean --doctype 6 input.mkv output.mkv

Note, --doctype 6 means v4, which is not well documented and confusing. You could convert a matroska v4 file to an old file like matroska v2, if that is what you want to do, with --doctype 2.

A win64 binary of mkclean is officially available from the download page. If you need Win32 binary, maybe you can try 0.8.9 instead (it's in ZIP).

If you want to make a Win32 build by yourself, then:

Download mkclean-0.8.10.tar.bz2.
Compile mkclean-0.8.10\corec\tools\coremake\coremake.c and get coremake.exe. You may have to compile it as C++ (not as C). You may have to tweak lines 1068, 1137 for const correctness, and in line 4941, you may have to change the last "0" to "(build_step)0".
Make your project files using this coremake.exe (see readme).
Compile mkclean.exe using the project files.

[doomer9er]

mkclean (https://sourceforge.net/projects/matroska/files/mkclean/) 0.8.10 (August 2017) does support the current matroska (v4) with several options (perhaps including what you want to do):
mkclean --doctype 6 input.mkv output.mkv

Note, --doctype 6 means v4, which is not well documented and confusing. You could convert a matroska v4 file to an old file like matroska v2, if that is what you want to do, with --doctype 2.

A win64 binary of mkclean is officially available from the download page. If you need Win32 binary, maybe you can try 0.8.9 instead (it's in ZIP).

If you want to make a Win32 build by yourself, then:

Download mkclean-0.8.10.tar.bz2.
Compile mkclean-0.8.10\corec\tools\coremake\coremake.c and get coremake.exe. You may have to compile it as C++ (not as C). You may have to tweak lines 1068, 1137 for const correctness, and in line 4941, you may have to change the last "0" to "(build_step)0".
Make your project files using this coremake.exe (see readme).
Compile mkclean.exe using the project files.


Thanks for replying, Lisa.

Not sure I understood some of that, but I'm too much of a newbie:stupid: to compile anything, plus the mkclean seems to have a warning flag about it being possible malware.
:thanks:

[Liisachan]

Thanks for replying, Lisa.

Not sure I understood some of that, but I'm too much of a newbie:stupid: to compile anything, plus the mkclean seems to have a warning flag about it being possible malware.
:thanks: I'm not sure either if I understand you - what exactly you want to do, so my comments might be irrelevant.

If you want to just remove the track compression, I think mkvtoolnixGUI itself can do that (Compression: no compression in the pull-down list). Or in mkvmerge --compression 3:none (if the track id is 3).

Mkclean binary for Win64 is already there, so you don't need to compile anything yourself. Just download EXE from sf and you can use it. It's on SF, so it's unlikely that it's malware.