Hybrid: Input -> x264/x265/Xvid/VP8/VP9

[Selur]

I then changed to 704x428 (mod4), but filtering kept greyed out.
I closed Hybrid, reopen and adjust cropping manually to 704x428, but filtering appears greyed out anyway. It seems like manual crop disables filtering?.
Nope, I just forgot to turn the filtering back on once the cropping is fine again. ;)

In last Hybrid build I still find it leaves an *.ac3 audio track in /tmp, and the subtitle track is not deleted, even if I select 'Delete subtitle...'
will look at the step-by-step guide

[digitall.h]

I suppose it helps nothing, since we cannot compare (and you know what my preferences are ;)), but for example Handbrake detects this source autocropping as 696x426 (not mod4, but they don't use mplayer but some internals) and PAR 5568x3905 (?), that is 1.426. The reported DAR is 2.32:1

Does this help?

[Selur]

Nope it does not help. Like I wrote, the mod4 thing is a mencoder problem and the par is detected through MediaInfo, can't do anything about it.

[digitall.h]

Well, if we can manually adjust the cropping and keep filtering, it would be enough for these difficult sources.
My problem now is how to overcome the problem when mediainfo reports a wrong PAR (probably because the flag in the source is wrong).

Did you take a look at the source sample?. Does it give you the same results as it did with me?.

[Selur]

Did you take a look at the source sample?. Does it give you the same results as it did with me?.
same for me PAR is detected as:
Width : 720 pixels
Height : 576 pixels
Pixel aspect ratio : 1.896
Display aspect ratio : 2.370
Display aspect ratio : 2.40:1

My problem now is how to overcome the problem when mediainfo reports a wrong PAR (probably because the flag in the source is wrong).
Other than telling Hybrid to use the aspect ratio you want: no clue :)
Problem might be rooted in:
1. Copy protection that writes wrong headers, which are not fixed by the ripper you use
2. MediaInfo simply misinterpreting the headers.
-> 1st thing I would try is to use another application to rip the dvd to hdd an check if the headers change.

Well, if we can manually adjust the cropping and keep filtering, it would be enough for these difficult sources.
will compile a new version that fixes the mod4 problem of not reenabling the filtering

[digitall.h]

same for me PAR is detected as:
Width : 720 pixels
Height : 576 pixels
Pixel aspect ratio : 1.896
Display aspect ratio : 2.370
Display aspect ratio : 2.40:1

Well, at least nothing wrong in my system.
Did you try autocrop?. Is autocrop done exclusively in mplayer/mencoder, or is there any detection algorithm in Hybrid to detect cropping?

-> 1st thing I would try is to use another application to rip the dvd to hdd an check if the headers change.
Few options in Linux...

will compile a new version that fixes the mod4 problem of not reenabling the filtering
Great! :)

[Selur]

Did you try autocrop?. Is autocrop done exclusively in mplayer/mencoder, or is there any detection algorithm in Hybrid to detect cropping?
mainly mplayer, but the auto crop call is adjusted depending on the source, working on it atm. (think I got it fixed)

Cu Selur

Ps.: uploaded new version for testing
PPs.: seems like ffprobe detects the aspect ratio properly,... same for mplayer during playback, but not with -frames 0 (where only the headers get checked), so seems like the headers are wrong, and the aspect ratio stored in the stream itself is correct,..

[digitall.h]

Good!, I will download new version and test.

Could this ffprobe thing be used in this case, for wrong headed streams?

[Selur]

How do you determine that the headers are wrong?

[digitall.h]

Don't ask me difficult questions... :rolleyes:
I'm in the (confortable) user side.
I would compare mplayer and ffprobe output, and in case of discrepancies, ask for user input. Of course it may lead to errors, it's better to offer a single solution and not depend on variable user skill or taste.

I assume that sometimes a bad PAR encoded signal in stream may be corrected with a correct PAR signal in header, isn't it?. Or will PAR in stream be always correct?

As you see I have no skills in this matter, just from the user point of view. Just thinking in loud voice...

[Selur]

adding ffprobe would mean adding another dependency,.. not really looking forward to that, if I find the time I'll look with a hex editor at the file to see if may be MediaInfo is just wrong :)
-> added a 'bug' to the mediainfo bug tracker (https://sourceforge.net/tracker/?func=detail&aid=3583398&group_id=86862&atid=581181) zenitram can probably tell better what is going on.

[digitall.h]

I tried last version.
When I load this problematic source and I set cropping to 704x426 it disables filtering.
When changing it to 704x428, filtering is back again.
Good!!

Some button in the GUI, as there's in the resizing part, to adjust vertical cropping to mod4?.
[I hope this did not sound as the typical irritating suggestion from an user 'asking' things to developer. :p Just a sugestion if you find it may improve Hybrid...]

[Selur]

Some button in the GUI, as there's in the resizing part, to adjust vertical cropping to mod4?.
automatically adjusting crop values does not make much sense, but you can tell Hybrid to crop to mod4 (Crop/Resize->Misc->Advanced Auto Crop Options->mod)

[digitall.h]

Yes, I already have it adjusted to mod4. But once I apply manual cropping, if I'm not cautious to do it mod4, this setting does not correct it.

[Selur]

Right, it does not hinder you from using non-mod4 values. Hybrid assumes that you know what you are doing. :)
I think it's a bad idea to add an 'adjust to modulo X' option, simply because sometime you might want to overcrop and sometime you might what to undercrop.
If it helps, I could add an option to color the width&height numbers red is non-mod-X values are selected,...

[digitall.h]

Well, it also could be, as a reminder.
It's your (very good) creation, and you're the best one to make it work the way you have it in your brain.

Related to Hybrid assuming every user know what is doing... I'm the first one that forgot once to adjust cropping to mod4...

[Selur]

I'm the first one that forgot once to adjust cropping to mod4...
the new version should enforce mod4 (iirc) for dvd input,...

[digitall.h]

And non-mod4 manual cropping in red numbers?

[Selur]

no, that was a suggestion for a feature I could add.
The latest version I uploaded should enforce mod4 for autocrop on dvd input, it does not hinder you from changing the detected crop values in any way.

[digitall.h]

Very good, so if autocrop works normally, it will apply mod4 to DVD sources.
We can then modify them manually, if needed.

My vote for red numbers in non-mod4 height in manual cropping :cool:

[Selur]

uploaded a new Linux64 version to the linux share, which should now also have an option to color width&height. (Crop/Resize->Misc->Color crop width/height red if they are not modulo->X)

[digitall.h]

And it gets red with non-mod4 values!!
(No surprise in my voice, just the confirmation of something I knew you were able to do :cool:)
I find it this way more noob-proof (myself)

Ps: I don't know if you changed anything else, but with my problematic source, after loading it and changing to cropping, now it does not change to PAR 16x15 as it did, and keeps PAR 256x135
Mysterious things that happen...
Ps(2): It happened again, I load the source, adjust x264 settings, go to cropping and... PAR 16x15 and source lasts 8:50 min... Aaargh

[Selur]

I uploaded a new version that should change 256x135 to 64/45 for DVD and vob input.

About the length:

according to the debugOutput you send me before, your DVD has 8 titles:
Title_01 of 8 has a length of 5780s
Title_01 of 8 has a length of 8s
Title_02 of 8 has a length of 15s
Title_03 of 8 has a length of 17s
Title_04 of 8 has a length of 8s
Title_05 of 8 has a length of 530s
Title_06 of 8 has a length of 280s
Title_07 of 8 has a length of 122s
Title_08 of 8 has a length of 831s
Hybrid than removes, title: 02, 03 and 04 which leaves 01, 05, 06, 07, 08.

Since title 01 and 03 are detected as PAR 1.896 and title 02,04,05,06,07,08 are detected as PAR 1.067

so you end up with:
1: Title_01, length: 5780, par: 1.896 (in the latest version, this should now be 1.422)
2: Title_05, length: 530, par: 1.067
3: Title_06, length: 280, par: 1.067
4: Title_07, length: 122, par: 1.067
5: Title_08, length: 831, par: 1.067
-> if you select Title 2 in the Main Tab you will end up with 530s of video with a par of 1.067,...

=> if Hybrid accidentally selected a wrong title, send me a debug output where this happens + a step-by-step how to reproduce it, otherwise I can't tell if it's a user error or a bug in Hybrid. :)

Cu Selur

[digitall.h]

OK, I will do this later, and will report back.
I now have an ongoing Hybrid encode (I would swear that x264 now encodes faster, even with filtering hqdn3d enabled, but as I have a more recent version of x264...).
I'm afraid I will have to redo it because it's being encoded as PAR 256x135...
But I want it to finish, and compare with an encode with your new version (the only difference will be PAR, but I can then compare)

[Selur]

About the par, according to zenitram MediaInfo follows the specification and the file headers are at fault, for more details see: http://sourceforge.net/tracker/?func=detail&atid=581181&aid=3583398&group_id=86862

[digitall.h]

Thank you for the info. Crystal clear (after several readings, this was not common language I use everyday).
So, as we thought, source was wrong.
I will try your last build in an hour or so.

[digitall.h]

selur, already tested your last build.
I'll send you the DebugOutput.
Steps:
I selected DVD folder
I browse to the folder and select VIDEO_TS folder
I then select english subtitle stream and click Add
After that it appears Title 1 selected, but now it's just 8:50 min long...

[Selur]

will look at the debugoutput once the email arrives :)

[digitall.h]

You've got an email :cool:

[Selur]

I see the problem, but it will take me a bit to fix it. :)

[digitall.h]

I'm sure you will.
Take your time.

[digitall.h]

selur,
with your last build, and following the steps posted above, I don't see the track switch after adding the subtitle stream. :)

I will do more tests to verify, but I think you fixed it :cool:

[Selur]

Hybrid rev. 2012.11.6.1

*fixed*

analyse: fixed stretch stream length analyses when no simple blocks were present
analyse: for multiple input, not all files were processed properly
analyse: handling of ac3 with 22.05kHz
analyse: Hybrid stayed grayed out when, the interlace analysis with mplayer crashed
avisynth: tivtc output frame rate calcluation for mode 2
avisynth: videoformat and interlaced result were mixed up :/
audio: when settings aac profile and than changing the channel count (e.g. by changing the downmix), aac profile changed, even when not needed
autocrop: didn't work properly for dvd input,...
batch: during intake not all files were processed properly
cut: cutting sometimes was ignored for the video stream (not sure if the problem is completely fixed ;))
extraction: tsMuxeR .meta files now are always coded with the current system text encoding
filtering: once dvd and non-mod4 resolution was selected filtering was disabled, but not enabled if the crop resolution was mpd4 again
input: auto audio loading was broken due to recent auto handling changes
input: batch - choosing 1st video stream for all sources
input: blu-ray input analysis
input: dvd input analyse finished not always called
input: crash on Windows XP with long file names
muxing: adding multiple idx/sub subtitle to an mp4 file added to many subtitles
mxuing: change speed for .avi output
muxing: ignore empty subtitles
muxing: tsmuxer meta files now are always coded with the current system text encoding
preview: window height
sample: file name mix-up during sample creation for mp4 files
srt2ttxt: fixed file naming
subtitle: accidentally the 'submit' and 'cancel' buttons in the subtitle modify dialog were hidden *gig*
x264: VUI color adjustment depending on input
x264: VUI color adjustment did only work if hardware restrictions were selected
xvid: missing quotes around stats file
Adjustment works this way:

check source contains color matrix info (if it does take the info)
if source has no color matrix info and is

MPEG2 assume input as BT.709
MPEG1 assume input as BT.601
width*height > 720 * 576, assume BT.709
width*height <= 720 * 576, assume BT.601


muxing: adding multiple idx/sub subtitle to an mp4 file added to many subtitles
subtitle: accidentally the 'submit' and 'cancel' buttons in the subtitle modify dialog were hidden *gig*

*changed*

analyse: only allow 16/15, 8/9, 64/45, 32/27 and 1/1 as aspect ratios for vob files (DVD input)
avisynth: use ffmpeg source for interlaced avc not DirectShowSource
checks: allow cutting if ffmpeg is set as extractor even if input is a transport stream
input: making sure ffmpeg is used instead of tsMuxeR to extract LATM muxed audio
input: 'Config->Input settings->Input Extensions->Only support' now also filters the 'Main->Input File'-dialog content.
muxing: mkvmerge: only set '--default-duration ...' if input and output fps differ, otherwise interlaced/progressive mixed content will cause problems
video: disable cuda encoder on systems without a cuda capable card

*added*

avisynth: option to use .d2v.bad instead of .d2v (workaround for, something that is probably a 'bug' in DGIndex)
crop: option to color the crop width&height spin boxes red if the widht/height value is not modulo X without rest.
decoder: option to flip rgb output when decoding is done with mencoder or avisynth
filter: option to overwrite input scan type
filter: "softpulldown + detc" and "softpulldown + filmdint" to handle mixed (progressive <> telecine) material

*removed*

filter: 'ivtc' got removed, since it seems to be broken (filmdint and detc are prefered)


-> downloads: http://www.selur.de/downloads

Cu Selur

[digitall.h]

Good!, a new version.

I don't know if you changed much things related to last test build (2012.11.04.2 if I recall well) but with that version I created a job and it aborted when Hybrid arrived to muxing, without errors in the GUI.
I send you the Debug to see if it helps.
I keep the temporal files, I will try to load them in Hybrid or tsMuxer and see if they are OK and if I can mux then in an mp4 file.
The video stream looks OK in mediainfo.
:)

[digitall.h]

selur,
I don't know if this is the expected behaviour, but when I try to load the *.264 I talked about in previous post, Hybrid (last version) stays greyed out and in GUI says something like analyzing input.

When I load the streams in tsMuxerGUI it does not complain about neither the *.264 stream nor the *.ac3, but it complains about the *.aac stream.

The aac stream looks good in mediainfo.

If you suggest a command to feed MP4Box I may try muxing in command line and see the errors MP4Box shows.

[digitall.h]

selur,
I added the files to MP4Box and when it has loaded 56% of the *.264 stream it stops with an I/O error.
Something wrong with the generation of the video stream?, on x264 side?, on GPAC side?

[Selur]

No clue, the debug output you send me only contains the reading of your source, not the debug regarding job creation and job processing, probably since the output folder changed,...
-> can't really say anything,...

The video stream looks OK in mediainfo.
MediaInfo only reads the first headers of the stream, so it doesn't say much about whether the stream is intact or not. :)

I don't know if this is the expected behaviour, but when I try to load the *.264 I talked about in previous post, Hybrid (last version) stays greyed out and in GUI says something like analyzing input.
Is FrameCounter present? Can you call it in a console? To get the frame count of a raw stream, FrameCounter is used,...

[digitall.h]

No clue, the debug output you send me only contains the reading of your source, not the debug regarding job creation and job processing, probably since the output folder changed,...
-> can't really say anything,...
I will redo the job tonight. :rolleyes: I'll send you the second half of the debug, but it just says ExitCode1.

MediaInfo only reads the first headers of the stream, so it doesn't say much about whether the stream is intact or not. :)
Is there anyway to analyse the stream and see what was wrong?. Of course I will have to redo the job, just to see if I selected strange settings that ended up with a wrong stream...

Is FrameCounter present? Can you call it in a console? To get the frame count of a raw stream, FrameCounter is used,...
Yes, it is present. And in Hybrid tool's path. When I call it in terminal I have to use ./FrameCounter for it to work. But in Debug I think there's some FrameCounter output.
From your answer I guess that Hybrid may load raw H264 streams, isn't it?. Not working with me (or just a problem of bad stream).

[Selur]

Yes Hybrid normally can handle raw h.264 just fine

[Selur]

Update to the latest Hybrid version, and since the calls all seem to look fine, please call:
"/usr/bin/MP4Box" -add "/tmp/El_submarino_20_29_54_601_08.264"#video:fps=25:par=64/45:delay=117 -brand avc1 -add "/tmp/El_submarino_es_aid_131__20_29_54_601_02_DELAY 117ms.ac3"#audio:lang="es":delay=117 -add "/tmp/El_submarino ms_en_aid_128__20_29_54_601_06.aac"#audio:lang="en" -mpeg4 -chap "/tmp/El_submarino_20_29_54_601_11.chp" -add "/home/ximo/Vídeos/Hybrid/Output/TITLE_1_0_en.idx" -itags name="El submarino":comment="Das Boot":genre="Accion, Aventura, Drama":tool="Hybrid 2012.11.4.2" -tmp "/tmp" -new "/home/ximo/_20_29_54_601_12.mp4"
in a console to see what MP4Box is complaining about.

Cu Selur

Ps.: if the audio is the problem, try if it helps to disable 'Config->Internals->Use mplayer for dvd audio extraction'.

[mogobime]

Wow, a lot of fixes/improvements! :cool:

I hope this didn't produce too much bugs :devil:

Time to encode some files.

[digitall.h]

selur,
I tried the code. MP4Box does not complain, but it begins to load the ac3 file... and it never stops, it continue loading after 100%, and the temporal file grows and grows...
I killed the process when it has loaded 2545/100 and the temporal file was 16.5 GiB :eek:
Something wrong in this ac3?. Because MP4Box loaded previously other ac3 without a problem. Or may the problem be it is a 3 hour long film?.
I tried to load it in Audacity but it does not load. It plays well in VLC.
Any idea how to analyse the ac3, or see why MP4Box loads it forever? :scared:

[digitall.h]

selur,
I loaded the ac3 in HeadAC3he (an old program from Windows times :rolleyes:) and it says stream is not perfect, that frame 357907 is incomplete...
It seems more clear that something in the ac3 creation went wrong, or in DelayCut. I passed it through DelayCut in Windows and it shows the uncomplete frame 357907 error. The total length of the file is 357910 frames.
Any clues? :o

[Selur]

helps to enabled:
1. 'Config->Internals->Use mplayer for dvd audio extraction'
and/or
2. 'Audio->Misc->Fix input with delay cut' (especially if like you say delaycut detects the stream as broken)
(before adding the audio to the audio queue)

[digitall.h]

selur,
the first option I had already enabled ('Use mplayer...').
The 'Fix input with delay cut' was not enabled. But DelayCut reports an error in the encoded ac3 stream when analysed in Windows, not the original.
I'm demuxing audio stream with Avidemux now, and I will analyse it with DelayCut then, and I'll report. Because I don't know if the error is in the original stream, or in the delay-corrected one.
I'm not even sure if MP4Box is failing due to this error in the stream, but it is better to verify that stream are Ok, isn't it?

[Selur]

"but it is better to verify that stream are Ok, isn't it? "
sure :)

[digitall.h]

I demuxed the audio track ac3 with avidemux, and in Windows DelayCut does not show any error.
So there was an error when mplayer extracted the audio or DelayCut adjusted delay (the stream was not processed, just copied and delay adjusted).
What can I do next?
Repeat the job (I erased :rolleyes:) and test the extracted ac3 stream (also erased:rolleyes::rolleyes:)?.
Is thre a way to say Hybrid: 'just do this task of the job', or 'execute job partially and stop when you arrive to...'?

[digitall.h]

selur, as I keep the streams to be muxed, I'm repeating the job, but the video settings are the worst possible to make it run faster.
I will then look at the ac3 stream and see which is wrong: the original that mplayer extracts, or the one created by DelayCut.
To begin with, there's something VERY estrange. The ac3 from the previous job (the one that MP4Box loaded for ever) showed a '..._02_DELAY_117ms' in its name. The ac3 in the job being redone just shows '..._02' :eek: :confused:
Looks like it was detecting a delay in the first job, that now doesn't detect...
Which tool detects the delay?, mplayer?, or DelayCut?

[Selur]

Which tool detects the delay?, mplayer?, or DelayCut?
1. delay is detected by MediaInfo by reading the headers of the input file (in your case 1st or last vob of the selected VTS). If the input file doesn't change the detected delay should not change either.
2. if you enable Config->Output settings->General Settings->Use delaycut to fix delays, Hybrid will try to use delaycut to fix the delay in the input file.

For DVD input either mplayer or mencoder is used to extract audio (if 'Use mplayer for dvd audio extraction' is enabled mplayer is used, if it's disabled mencoder).
If the audio stream mplayer/mencoder extracts is broken, there are two possibilities:
1. mplayer/mencoder is broken in regard of extracting DVD audio streams (which I doubt, since I use both without problems here)
2. the input is broken in some way, which might be caused by a copy protection that is nor properly circumvented.
In both cases, I sadly can't really to much if both mencoder&mplayer extract a broken ac3, I can't do anything much about it.

Is thre a way to say Hybrid: 'just do this task of the job', or 'execute job partially and stop when you arrive to...'?
if you postpone a job while it's running, Hybrid will halt the processing after the sub-job currently executed.

Cu Selur

[digitall.h]

Well, MediaInfo showed in first vob 0ms delay, but in last vob 120ms delay...
As I said above, I reencoded the audio, and this time the result was opposite: it detected 0ms delay for spanish stream and 117ms for english stream :eek: , with the same version of tools and Hybrid :eek:, :eek:

I muxed these streams with MP4Box, and this time everything worked OK.

I tried and saved GlobalDataModel, but it does not retain the 'Fix audio input with DelayCut' setting, that remains disabled, and I have to enable for every encode.

I will try later to load again a 264 raw stream and see if Hybrid loads it well