Hybrid: Input -> x264/x265/Xvid/VP8/VP9

[Jamaika]

Hi Selur
Do you know what is the function 'lag-in-frame'? When used 'end-used=q, cq-level=47, target-bitrate=0' doesn't matter which has a value. When used bitrate=10000kbps image is fatal to the 'lag-in-frame=1'.

[DJ-1]

Hi, I've got some old UK (BBC) PAL TV Dvds. The source i have is from a show shot on video, with analogue ghosting around outlines (people heads and edges in general). I know its not field blended ir anything like that, just not a great source...

Its being QTGMC, and avisynth resized (nnedi3).
Is there a good way to combat the ghosting effect...if at all?

Thanks.

Sent from my LG-H815 using Tapatalk

[Selur]

@DJ-1:
if the source was NTSC, sRestore is probably worth a try, other than that avisynth got some deghosting (http://avisynth.nl/index.php/External_filters#Ghost_Removal) filters, but I have non-experience with them and if you want to use them you would need to add them in the custom section.

@Jamaika:
When used 'end-used=q, cq-level=47, target-bitrate=0' doesn't matter which has a value.
Here's what I wrote in the tool-tip:
Max frams to lag:
The parameter defines an upper limit on the number of frames into the future that the encoder can look.

If set, this value allows the encoder to consume a number of input frames before producing output frames. This allows the encoder to base decisions for the current frame on future frames. This does increase the latency of the encoding pipeline, so it is not appropriate in all situations (ex: realtime encoding).
Note that this is a maximum value -- the encoder may produce frames sooner than the given limit. Set this value to 0 to disable this feature.
in x264&x265 this is the frame lookahead.


When used 'end-used=q, cq-level=47, target-bitrate=0' doesn't matter which has a value.
Yes target-bitrate doesn't matter when constant quantizer is used,... no surprise there.

[DJ-1]

@DJ-1:
if the source was NTSC, sRestore is probably worth a try, other than that avisynth got some deghosting (http://avisynth.nl/index.php/External_filters#Ghost_Removal) filters, but I have non-experience with them and if you want to use them you would need to add them in the custom section.
Awesome...
PAL source, Ghostbuster seems to be the one, was also rebuilt looking at the last post in http://forum.doom9.org/showthread.php?t=35339

Would that by default work with hybrid...
I have no idea 'what' im putting in the custom code part though

Cheers.

Sent from my LG-H815 using Tapatalk

[Selur]

Would that by default work with hybrid...
No since Hybrid doesn't support the filter by default, so there is no button to enable it. :)

I have no idea 'what im putting in the custom code part though
You would:
0. copy the filter dll into the Hybrid avisynthPlugins folder
1. enable "Filtering->Avisynth->Custom->Add custom content before" and select the part where the call should appear inside the script (you probably want it before resizing)
2. add

LoadPlugin("PATH TO \Hybrid\avisynthPlugins\Ghostbuster.dll")
ConvertToYUY2() # to convert to YUY2, since the filter only supports YUY2
Ghostbuster(0, 10) # no clue what are good values here, you probably should read the readme of Ghostbuster
ConvertToYV12() # to convert back to Yv12
with the parameters of your choice for offset and strength into the box and check the avisynth preview

Cu Selur

[DJ-1]

Cheers.
Got various result from preview window, but not removing the issue.

Maybe `ghosting` wasnt the right term for what it is, unsure now..
what would you call it?

all down the outline of his arm
https://www.dropbox.com/s/9kjxqiipsf7sk7g/1.png?dl=0

down the right edge of his jeans, and on the old guys shirt edge
https://www.dropbox.com/s/ep11edda0dsm0z3/2.png?dl=0

:thanks:

[Selur]

what I seem most prominent:
a. chroma displacement
b. ringing

[DJ-1]

Ok cool, so looking at http://avisynth.nl/index.php/External_filters#Chroma correction
ChromashiftSp seems best bet?

Cheers.

Sent from my LG-H815 using Tapatalk

[den78]

Hi, Selur.

In new version i get this error message when i select YadifMod.

http://i.imgur.com/Yv504F6.png

It's no big deal btw, deinterlacer works after clearing the message.

[alter4]

Can confirm the above and faced another issue.
I use https://db.tt/IPZrdexi configuration and put to input MOV camera with 30fps video and 48000Hz pcm audio.

Generated job looks this way:
ffmpeg -y -threads 8 -i "U:\photo\копи попи\DCIM\100_FUJI\DSCF0004.MOV" -map 0:1 -vn -sn -acodec pcm_s16le -map_metadata -1 -metadata handler_name="Hybrid 2016.02.13.1"

"F:\temp\iId_1_aid_0_lang_en_DELAY_-44ms_20_47_19_2810_01.wav"
delaycut -fixcrc fix -startcut 44 -o "F:\temp\iId_1_aid_0_lang_en_20_47_19_2810_02.wav" -i "F:\temp\iId_1_aid_0_lang_en_DELAY_-44ms_20_47_19_2810_01.wav"
cleanUp "F:\temp\iId_1_aid_0_lang_en_DELAY_-44ms_20_47_19_2810_01.wav"
cleanUp "F:\temp\iId_1_aid_0_lang_en_DELAY_-44ms_20_47_19_2810_01_log.txt"
ffmpeg -y -threads 8 -loglevel fatal -i "F:\temp\iId_1_aid_0_lang_en_20_47_19_2810_02.wav" -ac 2 -ar 48000 -acodec pcm_s16le -f wav - | qaac --threading --tvbr 75 --raw

- -o "F:\temp\iId_1_aid_0_lang_en_20_47_19_2810_05.aac"


The issue is in resulting mp4 file. Audio has muxed into mp4 container as 44100 and its length is more than video as result + distorted voice.. IMHO to fix the issue hybrid should generate last line in code block as

ffmpeg -y -threads 8 -loglevel fatal -i "F:\temp\iId_1_aid_0_lang_en_20_47_19_2810_02.wav" -ac 2 -ar 48000 -acodec pcm_s16le -f wav - | qaac --threading --tvbr 75 --raw-rate 48000 --raw - -o "F:\temp\iId_1_aid_0_lang_en_20_47_19_2810_05.aac"

[Selur]

calls look fine as far as I can see them (the muxing call is missing) would need a sample to reproduce that (btw. your global settings don't really help.
IMHO to fix the issue hybrid should generate last line in code block as
it wouldn't (no clue where the new lines in the jobs you post come from,..)

[alter4]

(the muxing call is missing)
I just omitted the rest. Sent you PM.

[Selur]

Hybrid rev 2016.03.09.1:

*fixed*

audio: mkv audio extraction with more than 10 streams
audio: qaac call, removed 'raw' parameter
avisynth: f3kdb - range parameter wasn't set properly
avisynth: qtgmc - progressive content wasn't handled properly
avisynth: don't use MPEG2source for mpls files
decode: ffmpeg - post processing
x264: small bug when resetting x264 medium profile
x264: allow negative chroma offsets
x265: 2pass controls grayed out
x265: version detection -> 12bit support was disabled
x265: restrict number of references to 8 by default (http://x265.readthedocs.org/en/latest/cli.html?highlight=references#cmdoption--ref)
qsvenc: feature detection
batch: frame rate adjustments not always properly kept

*added*

general: basic support for multi-angle blu-ray content (selection of angle should now be possible, DGDecNV users need to update their DGDecNV version to 2051+)
x264: 'reset to profile' option for tune

*changed*

audio: added additionally HE-AAC v2 stereo enforcement checks
analysis: mediainfo output is temporary saved to file for language compatibility
avisynth: switched back to old avisynth MT dll for compatibility and stability
avisynth: switched yadifmod with yadifmod2
gui: another ugly workaround for arch linux kde compatibility
gui: audio when a audio queue entry is clicked on the side bar will show a summary of the main settings

-> downloads: http://www.selur.de/downloads

Cu Selur

[alter4]

Absolutely clean install. Video preview (I tried camera mov and cellphone mp4) opened for preview button on main tab always freezes at first frame after opening.
Sample - https://dl.dropboxusercontent.com/u/7307234/VID_20151102_220011.mp4
Log - https://dl.dropboxusercontent.com/u/7307234/HybridDebugOutput.txt

Thanks

[Selur]

on a business trip atm. with REALLY shitty internet connection, will look at it mid next week when I'm back

[alter4]

Got it. Thanks, no urgency here. Have a safe trip!

[Selur]

@alter4: tried to reproduce the issue, but it works fine here.
Do you get any errors when calling:
"D:\Program Files\Hybrid\mplayer_64.exe" "F:\temp\nexus4\DCIM\Camera\VIA3AA~1.MP4" -forcedsubsonly -nosub -noautosub -mc 0 -nocache -aid 0 -ao win32 -vo direct3d -lavdopts threads=8
Loaded the video inside Hybrid and pressed the Preview-button in the main tab and playback started right away.
Also try if it helps to use the mplayer/mencoder binaries from:
http://mplayerwin.sourceforge.net/downloads.html
32bit executables need to be named: mplayer.exe and mencoder.exe
64bit executables need to be named: mplayer_64.exe and mencoder_64.exe

[alter4]

After endless doing something in background, preview opened and it works with binaries from http://mplayerwin.sourceforge.net/downloads.html
Some evil in old binaries. Who knows...
Thanks.

[Selur]

first call to mplayer always takes long since it normally comes with fontconfig which first needs to scan the fonts on the system. ;)

[alter4]

One more question. If I want to use cutting edge x265 binaries because of https://bitbucket.org/multicoreware/x265/issues/172/bitrate-control-zones-are-ignored-in-any
It it possible to replace x265 in hybrid folder with multilib version from http://msystem.waw.pl/x265/ ?

[Selur]

a. If you want it cutting edge better compile it yourself.
b. you can exchange the x265 binary as long as it's with a single multilib binary.

[digitall.h]

Hybrid rev 2016.03.09.1:

*fixed*

audio: mkv audio extraction with more than 10 streams
audio: qaac call, removed 'raw' parameter
avisynth: f3kdb - range parameter wasn't set properly
avisynth: qtgmc - progressive content wasn't handled properly
avisynth: don't use MPEG2source for mpls files
decode: ffmpeg - post processing
x264: small bug when resetting x264 medium profile
x264: allow negative chroma offsets
x265: 2pass controls grayed out
x265: version detection -> 12bit support was disabled
x265: restrict number of references to 8 by default (http://x265.readthedocs.org/en/latest/cli.html?highlight=references#cmdoption--ref)
qsvenc: feature detection
batch: frame rate adjustments not always properly kept

*added*

general: basic support for multi-angle blu-ray content (selection of angle should now be possible, DGDecNV users need to update their DGDecNV version to 2051+)
x264: 'reset to profile' option for tune

*changed*

audio: added additionally HE-AAC v2 stereo enforcement checks
analysis: mediainfo output is temporary saved to file for language compatibility
avisynth: switched back to old avisynth MT dll for compatibility and stability
avisynth: switched yadifmod with yadifmod2
gui: another ugly workaround for arch linux kde compatibility
gui: audio when a audio queue entry is clicked on the side bar will show a summary of the main settings

-> downloads: http://www.selur.de/downloads

Cu Selur

Hi Selur,
:)
Did you drop QT 4.8 builds?.
I cannot run 5.2.1 build in my system...
:rolleyes:

[Selur]

Yes, I dropped Qt4.8 in 'rev 2013.01.02.1' . (two month ago and since nobody complained I spend the last week removing compatibility code and rewriting stuff)
What system are you running?

[alter4]

Reproduced on absolutely clean install (removed new mplayer binaries and clean reinstalled hybrid to verify issue) (see logs for 2016.03.19)
Got crash with x265 encoding with any of my camera movs
samples sent via PM because it's private.

[Selur]

-> replace the mencoder/mplayer versions with older once (https://trac.mplayerhq.hu/ticket/2293); anything before r37826 should be fine.
From the looks of it mencoder is freeing some memory it shouldn't free at that point in time when piping to an external encoder.

[alter4]

Will try. Thank you.
What's wrong with this world? :-) Mplayer should be newer, mencoder should be older.

[Selur]

It's the same code base,... :)

[digitall.h]

Yes, I dropped Qt4.8 in 'rev 2013.01.02.1' . (two month ago and since nobody complained I spend the last week removing compatibility code and rewriting stuff)
What system are you running?
Thank you for answering Selur.
The last QT4.8 version I downloaded from your site was Hybrid_151223_64bit_binary_qt48

I run a Debian 8 based system: SolydXK 8 solydk. KDE version 4.14.2

It's true, I've been very busy lately.
Otherwise I would have complaint!
;)

[alter4]

Mencoder from revision https://sourceforge.net/projects/mplayerwin/files/MPlayer-MEncoder/r37817/
works for me

It's the same code base,... :)
That is exactly the thing that makes me funny

[Selur]

Yes, ...
r37825 and before work
r37826 and later crash

[digitall.h]

Yes, I dropped Qt4.8 in 'rev 2013.01.02.1' . (two month ago and since nobody complained I spend the last week removing compatibility code and rewriting stuff)
What system are you running?

Sorry me for bumping...
any chance for a new QT48 version?.

I can hardly imagine what I'm requesting, if you already uninstalled QT4...
:(

[Selur]

Not sure if the current code will still compile without problems under qt 4.8, but I'll look into it,..

[digitall.h]

Thank you Selur

[digitall.h]

Hi Selur,
I found a parameter switch in the command generated by Hybrid to send to x264 that may be normal but I find it strange.
First I'm using Hybrid 64bit QT4.8 2015.12.23.1 version. It may be important because I noticed that with this version I cannot set a negative chroma_qp_offset, but I see that it has been corrected recently.
When I try to make an encode from Bluray, I select crf encoding mode, I then select preset veryslow, tune film and when I run the job I see that the parameter chroma_qp_offset=-3 was sent to x264. Of course this parameter is not shown in the window when I'm selecting x264 switchs.
Is this normal given the combination of parameters I selected previously?

[Selur]

Hybrid only modifies:
a. stats file name (during 2pass encoding)
b. qp file (to adjust to chapters)
c. bitrate (during 2pass assuming you want to hit a specific bit rate)
It doesn't modify the chroma offset.
There might be a difference between the chroma_qp_offset you use and the one some tools report,.. (iirc. the output of h264_parse and the value one sets in x264 differs)

[digitall.h]

You're right.

The command Hybrid generates does not have any chroma_qp_offset adjustment. So it should stay default, chroma_qp_offset=0.
But MediaInfo reports chroma_qp_offset=-3

So, do you believe that it may be related to an error in MediaInfo when 'reading' chroma_qp_offset in x264 stream?

[Selur]

Can't reproduce this here, using MediaInfo Command line, MediaInfoLib - v0.7.83, when I encode something with Hybrid it reports:
Video
Count : 334
Count of stream of this kind : 1
Kind of stream : Video
Kind of stream : Video
Stream identifier : 0
Format : AVC
Format/Info : Advanced Video Codec
Format/Url : http://developers.videolan.org/x264.html
Commercial name : AVC
Format profile : High@L4.1
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, CABAC : Yes
Format settings, ReFrames : 4
Format settings, ReFrames : 4 frames
Internet media type : video/H264
Codec : AVC
Codec : AVC
Codec/Family : AVC
Codec/Info : Advanced Video Codec
Codec/Url : http://developers.videolan.org/x264.html
Codec profile : High@L4.1
Codec settings : CABAC / 4 Ref Frames
Codec settings, CABAC : Yes
Codec_Settings_RefFrames : 4
Bit rate : 1500000
Bit rate : 1 500 Kbps
Width : 640
Width : 640 pixels
Height : 480
Height : 480 pixels
Sampled_Width : 640
Sampled_Height : 480
Pixel aspect ratio : 1.000
Display aspect ratio : 1.333
Display aspect ratio : 4:3
Frame rate : 25.000
Frame rate : 25.000 fps
Resolution : 8
Resolution : 8 bits
Colorimetry : 4:2:0
Color space : YUV
Chroma subsampling : 4:2:0
Chroma subsampling : 4:2:0
Bit depth : 8
Bit depth : 8 bits
Scan type : Progressive
Scan type : Progressive
Interlacement : PPF
Interlacement : Progressive
Bits/(Pixel*Frame) : 0.195
Writing library : x264 - core 148 r2665 a01e339
Writing library : x264 core 148 r2665 a01e339
Encoded_Library_Name : x264
Encoded_Library_Version : core 148 r2665 a01e339
Encoding settings : cabac=1 / ref=3 / deblock=1:0:0 / analyse=0x3:0x111 / me=hex / subme=5 / psy=1 / psy_rd=1.00:0.00 / mixed_ref=1 / me_range=16 / chroma_me=1 / trellis=0 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=0 / threads=12 / lookahead_threads=3 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=3 / b_pyramid=2 / b_adapt=1 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=1 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=0 / rc=2pass / mbtree=0 / bitrate=1500 / ratetol=1.0 / qcomp=0.50 / qpmin=0 / qpmax=69 / qpstep=4 / cplxblur=20.0 / qblur=0.5 / vbv_maxrate=62500 / vbv_bufsize=78125 / nal_hrd=none / filler=0 / ip_ratio=1.40 / pb_ratio=1.30 / aq=0
Color range : Limited
colour_description_present : Yes
Matrix coefficients : BT.601
tested chroma offsets between -3 and 3 and all are reported correctly,....

[digitall.h]

I'm almost sure this is not related to Hybrid.
When I run in terminal:
./x264-r2665-a01e339 --preset veryslow --tune film --crf 20 --profile high --level 4.1 --ref 4 --chroma-qp-offset 0 --output-csp i420 --fps 24000/1001 --input-res 1920x1080 --output "/home/tests/cqo20.264" -

I get in MediaInfo (MediaInfoLib - v0.7.70):
Video
Count : 277
Count of stream of this kind : 1
Kind of stream : Video
Kind of stream : Video
Stream identifier : 0
Format : AVC
Format/Info : Advanced Video Codec
Format/Url : http://developers.videolan.org/x264.html
Commercial name : AVC
Format profile : High@L4.1
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, CABAC : Yes
Format settings, ReFrames : 4
Format settings, ReFrames : 4 frames
Internet media type : video/H264
Codec : AVC
Codec : AVC
Codec/Family : AVC
Codec/Info : Advanced Video Codec
Codec/Url : http://developers.videolan.org/x264.html
Codec profile : High@L4.1
Codec settings : CABAC / 4 Ref Frames
Codec settings, CABAC : Yes
Codec_Settings_RefFrames : 4
Width : 1920
Width : 1 920 pixels
Height : 1080
Height : 1 080 pixels
Pixel aspect ratio : 1.000
Display aspect ratio : 1.778
Display aspect ratio : 16:9
Frame rate : 23.976
Frame rate : 23.976 fps
Resolution : 8
Resolution : 8 bits
Colorimetry : 4:2:0
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8
Bit depth : 8 bits
Scan type : Progressive
Scan type : Progressive
Interlacement : PPF
Interlacement : Progressive
Writing library : x264 - core 148 r2665 a01e339
Writing library : x264 core 148 r2665 a01e339
Writing library/Name : x264
Writing library/Version : core 148 r2665 a01e339
Encoding settings : cabac=1 / ref=4 / deblock=1:-1:-1 / analyse=0x3:0x133 / me=umh / subme=10 / psy=1 / psy_rd=1.00:0.15 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=1 / chroma_qp_offset=-3 / threads=12 / lookahead_threads=2 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=8 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=23 / scenecut=40 / intra_refresh=0 / rc_lookahead=60 / rc=crf / mbtree=1 / crf=20.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / ip_ratio=1.40 / aq=1:1.00


So it seems not related to Hybrid.

I'm not sure is something wrong whether in MediaInfo reading chroma_qp_offset or in x264 or in this particular combination of parameters in x264 that forces chroma_qp_offset=-3?
:confused:


I also realized that even I select under "x264/VUI" Matrix, Tranfer, Primes and Range to automagically select them according to source, it does not recognize some sources as bt709 and does not send x264 the colormatrix=bt709 command.
But for this last issue you will probably need some additional information

[Selur]

I can reproduce this when I use your command line and, it seems to be related to crf encoding. (2pass, cq and abr are fine)
-> best ask the MediaInfo author and the x264 folks about this and I agree this is not Hybrids fault. :)

[digitall.h]

(...)and I agree this is not Hybrids fault. :)
Good to know!
:D

[Selur]

Okay it isn't crf related it comes from psy-rd as soon as it is enabled, the chroma-qp-offset is calculated in an to me unknown way. :)
-> http://forum.doom9.org/showpost.php?p=1283216&postcount=7

[digitall.h]

You're right!
:thanks:
Thank you for pointing me to this post.
It would have taken me ages, if I had ever managed to find it.
So it's most probably related to psy-rd, that is adjusted to 1.0:0.15 after selecting veryslow preset.

So I run a new x264 encode, but this time selecting chroma-qp-offset 3.
And the resulting file shows in MediaInfo a chroma-qp-offset 0
Meaning that this particular combination of parameters subtracts 3 to chroma-qp-offset: as it was default (0) the resulting chroma-qp-offset=-3

Great discover!

[Selur]

-> Will indicate the 'chroma-qp-modifier' in the next release version.

[digitall.h]

Hi Selur,
I'm not 100% sure but I have the feeling that when you encode with x264 and select to generate a qp file based on chapters, Hybrid previously generated a qp file, but now it does not. I'm not sure if it is working properly.
I will come back hopefully with more certainties...
:D

[Selur]

Hmm,... you are right, will look into it. Seems like the 'Create qt file for chapters' doesn't work any more.
-> will be fixed in the next release, happened to due to some changes I made for better ArchLinux/KDE compatibility,...

[alter4]

Hi Selur! The issue for mencoder/mplayer looks fixed. I am waiting for the new release soon)) I want to be lucky guy upcoming release. :rolleyes:

[Selur]

On that note: New release is planned for tomorrow, assuming no new bug with the dev version will be reported today.

[alter4]

Stupid question. Where I can get this version if I'd like to be beta tester?

[Selur]

I send links to those that report a bug so they can check if it is fixed, other than that there is no public way to get a dev version.

[alter4]

Thanks. Sad but true. I should find and report bug right now to get a chance to get dev version! :D