Ugly fades.

[shon3i]

I never saw that ugly fades in x264. I use buld 1046 from MeGUI Auto Updater and these settings

--pass 2 --bitrate 5381 --stats ".stats" --level 4 --keyint 24 --min-keyint 1 --ref 3 --mixed-refs --no-fast-pskip --bframes 3 --b-adapt 2 --weightb --direct auto --deblock -3:-3 --subme 9 --trellis 2 --psy-rd 1.0:1.0 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --vbv-bufsize 8000 --vbv-maxrate 15000 --me umh --merange 32 --threads 6 --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output "output" "input" --sar 1:1 --mvrange 511 --aud --nal-hrd without turbo activated.

Source

http://thumbnails3.imagebam.com/2066/a621d020651005.gif (http://www.imagebam.com/image/a621d020651005)

x264

http://thumbnails6.imagebam.com/2066/a927e420651006.gif (http://www.imagebam.com/image/a927e420651006)

I aslo tryed to use zone at begining with quantizer @ 10 but not helped much

Source is Austin Powers 2 Blu-ray, target 1080p BD5 :)

[Atak_Snajpera]

Why do you use maxbitarte 15000? Level 4.0 allows 25000! What happens if you remove --keyint 24 --min-keyint 1 ?

[shon3i]

Why do you use maxbitarte 15000? Level 4.0 allows 25000! What happens if you remove --keyint 24 --min-keyint 1 ?
Why do you use maxbitarte 15000? Level 4.0 allows 25000!Because i follow BD (BD-5) standard and not H264 :) anyway there is no point for higher max bitrate or buffer.

http://thumbnails13.imagebam.com/2066/236c6820651721.gif (http://www.imagebam.com/image/236c6820651721)

What happens if you remove --keyint 24 --min-keyint 1 I will test soon, thanks for suggestion.

btw i just finished encoding same scene with MC encoder, and no problems :)

[Dark Shikari]

Post an output stream.

Also, does the problem disappear if you don't use threads, or if you use this build (http://www.mediafire.com/download.php?cugmzznrmzz)?

[Terranigma]

I only noticed artifacts like the one you mentioned, shon3i, when I used vbv on a source before. try without vbv.

[Sagittaire]

I never saw that ugly fades in x264. I use buld 1046 from MeGUI Auto Updater and these settings

without turbo activated.

Source

http://thumbnails3.imagebam.com/2066/a621d020651005.gif (http://www.imagebam.com/image/a621d020651005)

x264

http://thumbnails6.imagebam.com/2066/a927e420651006.gif (http://www.imagebam.com/image/a927e420651006)

I aslo tryed to use zone at begining with quantizer @ 10 but not helped much

Source is Austin Powers 2 Blu-ray, target 1080p BD5 :)


Well fade is particular part for H264 ...

Try without --b-adapt 2 or --weightb. Fade is really particular problem for x264.

[Dark Shikari]

Well fade is particular part for H264 ...

Try without --b-adapt 2 or --weightb. Fade is really particular problem for x264.B-adapt 2 and weightb improve fades, not make them worse.

The problem here has nothing to do with H.264 or x264 in general, or even fades in general, but rather just that the frame quantizer is being shot up to QP51 by something (probably VBV).

[Sagittaire]

B-adapt 2 and weightb improve fades, not make them worse.


Well certainely bitrate efficiency but certainely not for the quality. All the other H264 encoder make history better job for fade (Ateme for exemple) with native WPred support for Pframe. For my BD encoding I must often use zone option for improve bitrate in fade part.

[Dark Shikari]

Well certainely bitrate efficiency but certainely not for the quality.Higher bitrate efficiency inherently means higher quality, if anything due to the fact that qcomp will lower quality more in the highest bitrate areas.All the other H264 encoder make history better job for fade (Ateme for exemple) with native WPred support for Pframe.All other H.264 encoders? Not in my experience, remember, almost all H.264 encoders suck. Though certainly one would expect the ones that make proper use of weighted P-frame prediction to do better.For my BD encoding I must often use zone option for improve bitrate in fade part.You could just set qcomp to 1.0 instead.

[shon3i]

Post an output stream.Here is (http://srbokuz.users.sbb.co.yu/vbv/chapter-0291.264)

but rather just that the frame quantizer is being shot up to QP51 by something (probably VBV). Yes it is VBV :( when is disabled fade is normal. What to do?? I mean i must use VBV, i aslo tryed to change to -vbv-buffersize 30000 -vbv-maxrate 33000 (by original Blu-Ray specs) to saw what should except from some normal values, and thig is same :(

[Terranigma]

I knew I was right.. and Sagittaire, blocking in fades really isn't a problem anymore, and if so, one could always opt to use a cqm, but you can see that I haven't even recommended that to him. The only time to use a cqm for fading, is if one chooses to use noob settings.

[Sagittaire]

Here is (http://srbokuz.users.sbb.co.yu/vbv/chapter-0291.264)

Yes it is VBV :( when is disabled fade is normal. What to do?? I mean i must use VBV, i aslo tryed to change to -vbv-buffersize 30000 -vbv-maxrate 33000 (by original Blu-Ray specs) to saw what should except from some normal values, and thig is same :(

Try with crf mode ...

[Fishman0919]

Wouldn't the Max bitrate for BD-5 be around 30000?

The max data transfer rate for DVD is about 11000. BD-5 has a spin speed of 3x's... so max would/should be about 33000.... account for audio.... about 30000

[Dark Shikari]

Yes it is VBV :( when is disabled fade is normal. What to do?? I mean i must use VBV, i aslo tryed to change to -vbv-buffersize 30000 -vbv-maxrate 33000 (by original Blu-Ray specs) to saw what should except from some normal values, and thig is same :(Did you follow my instructions and try both without threads, and with threads+the build I posted for you?

[shon3i]

Try with crf mode ...
Not helped :( only solution i found is to encode this first 300 frames with Mainconcept Encoder, and whole chapter now look good.

@Dark Shikari, can you do something about this? I remember when elecard encoder haved same issue in some of first versions of converter studio, and i reported it to Mutek, and they are soloved very quickly :)

[Blue_MiSfit]

I smell something familiar here...

[shon3i]

Did you follow my instructions and try both without threads, and with threads+the build I posted for you?
I tryed without --threads 6 option and leave on default, but is same ugly fade. I didnt tryed with you build because don't contain hrd-nal patch :)

[Dark Shikari]

I tryed without --threads 6 option and leave on default, but is same ugly fade. I didnt tryed with you build because don't contain hrd-nal patch :)What does the NAL-HRD patch have to do with anything? It has nothing to do with the actual encoding process, and certainly nothing to do with VBV. Again, I told you to test that build: that means you should test it, not complain about why you can't when you obviously can.

Also, if it turns out that build doesn't fix the problem, I'm going to need from you the smallest possible source segment and associated x264 build and x264 settings that replicate the problem.

[shon3i]

I can't do anything if you ignore every single post I make in this thread. I am less inclined to care about peoples' encoding problems if they refuse to read my posts when I'm trying to solve the problem.
I don't get it, it works with you build and --threads 6 :D, Sorry for ignoring you post :)

What does the NAL-HRD patch have to do with anything? I need it for BD compatability. I know that nal-hrd don't have anything in encoding, but VBV without NAL is usless

[Dark Shikari]

I don't get it, it works with you build and --threads 6 :D, Sorry for ignoring you post :)It works? As in, there is no VBV issue, and all problems are solved?

[LoRd_MuldeR]

I don't get it, it works with you build and --threads 6 :D, Sorry for ignoring you post :)

Obviously he included a fix in his build that is not in the official git yet and he asked you to test it in order to see whether it helps or not ...

[shon3i]

It works? As in, there is no VBV issue, and all problems are solved?
Correct, so how much i have wait to get new build nal-hdr patched :D

[Dark Shikari]

Correct, so how much i have wait to get new build nal-hdr patched :DSo it is that problem... I'm psychic, eh? :D

So, the problem here is that while I have a fix for the problem, it's a bit of a hack and not a fix for the actual underlying problem. But it works, so... :sly:

here's your build (http://www.mediafire.com/?3nozdw0ozn1).

[shon3i]

Thanks :)

btw here is problematic source clip if you need future testings

Source (VC-1) (http://srbokuz.users.sbb.co.yu/vbv/source.mkv)

[MasterNobody]

Dark Shikari
As I understand this bug is coursed by the misprediction of frame size (probably B-frames) in VBV (in function predict_size) so that predicted size becomes greater than vbv-bufsize. But I didn't find any good solution to fix this misprediction.

[Dark Shikari]

Dark Shikari
As I understand this bug is coursed by the misprediction of frame size (probably B-frames) in VBV (in function predict_size) so that predicted size becomes greater than vbv-bufsize. But I didn't find any good solution to fix this misprediction.Yes, I know that. A fix will go in soon.

[CruNcher]

so if that issues now gets fixed X264 VBV would be finally once for all stable also with multithreading (it seems somehow to be the last issue from all the reports) ?

[asarian]

Odd. I reencoded this Blu-Ray too, and got worried after reading this thread. Turns out 'x264 core:65 r1028M', which I used on my Vmware machine, did a perfect job. However, it seems 'x264 core:65 r1029M', which I use on my desktop machine, produces the same ugly fade.

I wouldn't even presume to guess what might cause it, but I just thought I let the powers that be know.

[Dark Shikari]

Odd. I reencoded this Blu-Ray too, and got worried after reading this thread. Turns out 'x264 core:65 r1028M', which I used on my Vmware machine, did a perfect job. However, it seems 'x264 core:65 r1029M', which I use on my desktop machine, produces the same ugly fade.There was no change that could have affected things in either of those revisions: I suspect the "M" might have more to do with it...

[asarian]

There was no change that could have affected things in either of those revisions: I suspect the "M" might have more to do with it...

I don't understand, they're both "M", right? (though, granted, I have no idea what the M stands for). The reencoded Blu-Ray is fine, but the sample posted here, which I tried with r1029M is messed up too.

[Dark Shikari]

I don't understand, they're both "M", right?M means modified, meaning that it isn't an official build and anything one could possibly imagine could have been done to it between the official source code and you.

[asarian]

M means modified, meaning that it isn't an official build and anything one could possibly imagine could have been done to it between the official source code and you.

Okay, thanks for explaining. These x264 versions came with RipBot264.

[LoRd_MuldeR]

Okay, thanks for explaining. These x264 versions came with RipBot264.

Try to reproduce the behavior with "unpatched" builds from here:
http://x264.nl/x264/

[asarian]

Try to reproduce the behavior with "unpatched" builds from here:
http://x264.nl/x264/

revision1057 produces the exact same, faulty rendering (tested inside RipBot264).

I'm nonetheless very confused now, because when I try and reencode the same sample, using 'x264 core:65 r1028M' again. on my desktop now, I get the same messed-up result. That's quite bewildering, of course. So, I'll have to examine my Vmware machine now (which also has 'x264 core:65 r1028M') to see what could be possibly different in that environment (maybe different ffdshow version?).

This is gonna take a bit of sorting out. Because the Blu-Ray I encoded on my Vmware machine definitely did a flawless fade. So, I'm kinda baffled.

[LoRd_MuldeR]

revision1057 produces the exact same, faulty rendering (tested inside RipBot264).

I'm nonetheless very confused now, because when I try and reencode the same sample, using 'x264 core:65 r1028M' again. on my desktop now, I get the same messed-up result. That's quite bewildering, of course. So, I'll have to examine my Vmware machine now (which also has 'x264 core:65 r1028M') to see what could be possibly different in that environment (maybe different ffdshow version?).

This is gonna take a bit of sorting out. Because the Blu-Ray I encoded on my Vmware machine definitely did a flawless fade. So, I'm kinda baffled.

Before you do any more encoding tests, make sure that the problem isn't in the source you feed into x264. Especially make sure the problem is not on the decoder side!

Also: Did you use 100% identical x264 parameters for your tests? You probably can't reproduce certain behavior if params have changed...

[asarian]

Before you do any more encoding tests, make sure that the problem isn't in the source you feed into x264. Especially make sure the problem is not on the decoder side!

Also: Did you use 100% identical x264 parameters for your tests? You probably can't reproduce certain behavior if params have changed...

For the original source I used the Blu-Ray I encoded myself (which went perfectly); it seems to go wrong on the sample.mkv posted above. I assume that's also a sample from the same Blu-Ray, but I may just wind up clipping that scene myself again.

Yes, I use the same x264 parameters:

--crf 16 --sar 1:1 --level 4.1 --aud --nal-hrd --vbv-bufsize 35000 --vbv-maxrate 35000 --filter 0,0 --ref 3 --b-pyramid --mixed-refs --bframes 3 --b-adapt 2 --weightb --no-fast-pskip --direct auto --subme 9 --trellis 1 --partitions all --8x8dct --me tesa --threads auto --thread-input --progress --no-psnr --no-ssim

Sans the --nal-hrd for the unpatched version.

[nm]

Yes, I use the same x264 parameters:

--crf 16 --sar 1:1 --level 4.1 --aud --nal-hrd --vbv-bufsize 35000 --vbv-maxrate 35000 --filter 0,0 --ref 3 --b-pyramid --mixed-refs --bframes 3 --b-adapt 2 --weightb --no-fast-pskip --direct auto --subme 9 --trellis 1 --partitions all --8x8dct --me tesa --threads auto --thread-input --progress --no-psnr --no-ssim
Do both environments have the same number of processors/cores and does x264 use the same number of threads in both cases? Threading may very well trigger the problem, so you should try setting the number of threads to 1 manually and then try 2 or 3 to see if there is a difference.

[Dark Shikari]

Do both environments have the same number of processors/cores and does x264 use the same number of threads in both cases? Threading may very well trigger the problem, so you should try setting the number of threads to 1 manually and then try 2 or 3 to see if there is a difference.Also note that VBV problems can in fact be nondeterministic with threads...

[shon3i]

@asarian, did you try with Dark Shikari modified build? http://www.mediafire.com/?3nozdw0ozn1, at least helped to me :) but with this buld i have little different problem :) now. I tryed to encode Austin Powers 1 BD and i get big undersize :) target size 4.2GB, i get 3.3GB with this build. I reencode with lastest 1057 form MeGUI, and i get prefered size??

[Gabriel_Bouvigne]

This is gonna take a bit of sorting out. Because the Blu-Ray I encoded on my Vmware machine definitely did a flawless fade. So, I'm kinda baffled.
It is likely because the number of threads used is lower within VMWare. The more threads, the more likely this problem will occur.

If you want to check it, you could explicitly specify the number of threads (--threads xxx instead of --threads auto)

[asarian]

Do both environments have the same number of processors/cores and does x264 use the same number of threads in both cases? Threading may very well trigger the problem, so you should try setting the number of threads to 1 manually and then try 2 or 3 to see if there is a difference.

Good call. :) No, the XP Vmware box only has 2 vCPU's, as opposed to my desktop machine, which has a quadcore.

And indeed, as you predicted, setting threads up to 4 on the quadcore makes the issue go away; but above that, like 5 or 6, and the problem returned again.

[asarian]

@asarian, did you try with Dark Shikari modified build? http://www.mediafire.com/?3nozdw0ozn1, at least helped to me :) but with this buld i have little different problem :) now. I tryed to encode Austin Powers 1 BD and i get big undersize :) target size 4.2GB, i get 3.3GB with this build. I reencode with lastest 1057 form MeGUI, and i get prefered size??

Yep, using 'x264 core:65 r1051+1' solves the issue! :) It doesn't have the built-in nal-hrd patch, but since I stream my reencoded discs from a media server anyway, I might as well do without it.

And thanks, Dark Shikari for the quick fix; you're the man! :)

EDIT: Actually, 'x264 core:65 r1051+1' does have the nal-hrd patch; so, even better. :)

[asarian]

@asarian, did you try with Dark Shikari modified build? http://www.mediafire.com/?3nozdw0ozn1, at least helped to me :) but with this buld i have little different problem :) now. I tryed to encode Austin Powers 1 BD and i get big undersize :) target size 4.2GB, i get 3.3GB with this build. I reencode with lastest 1057 form MeGUI, and i get prefered size??

Okay, pending this latest issue you report, I'll hold off on doing any major jobs with this version until we get word from Dark Shikari that it's safe.

[moviefan]

I have the same problem with a really ugly fade-in from black... The picture is completely messed up, extremely blocky, no details at all. Is there a way to encode the fade-in better?

[asarian]

I have the same problem with a really ugly fade-in from black... The picture is completely messed up, extremely blocky, no details at all. Is there a way to encode the fade-in better?

Regret to say, x264.1069M still has the same issue.

It was solved (with a makeshift solution) in a custom x264.1057M Dark Shikari posted, but apparently this change hasn't made it to the latest x264 version, it seems.

[moviefan]

Dark Shikari's patched x264 does not work for me. The fade scene still lacks a lot of detail, is blocky and looks ugly... Even using zones, I am not able to achieve a good looking picture. Any suggestions?

[nm]

Dark Shikari's patched x264 does not work for me. The fade scene still lacks a lot of detail, is blocky and looks ugly... Even using zones, I am not able to achieve a good looking picture. Any suggestions?Post a sample clip and encoding parameters so that we can duplicate the problem.

[moviefan]

sample clip: http://www.mediafire.com/?sharekey=508d6798bcda1276d2db6fb9a8902bda

My x264 settings are:

"path to x264.exe\x264.exe" --threads auto --thread-input --progress --no-psnr --no-ssim "%IN_TITLE%.AVS" --stats "%IN_TITLE%.stats" --output NUL --pass 1 --bitrate %IN_BITRATE% --vbv-bufsize 30000 --vbv-maxrate 24000 --level 4.1 --keyint 24 --min-keyint 1 --nal-hrd --aud --sar 1:1 --bframes 3 --b-adapt 2 --b-pyramid --weightb --ref 3 --mixed-refs --no-fast-pskip --ipratio 1.2 --pbratio 1.2 --direct auto --subme 2 --trellis 2 --psy-rd 0.8:0.8 --partitions none --me dia --merange 24 --mvrange 511 --aq-strength 1.0

"path to x264.exe\x264.exe" --threads auto --thread-input --progress --no-psnr --no-ssim "%IN_TITLE%.AVS" --stats "%IN_TITLE%.stats" --output "%IN_TITLE%.264" --pass 2 --bitrate %IN_BITRATE% --vbv-bufsize 30000 --vbv-maxrate 24000 --level 4.1 --keyint 24 --min-keyint 1 --nal-hrd --aud --sar 1:1 --bframes 3 --b-pyramid --weightb --ref 3 --mixed-refs --no-fast-pskip --ipratio 1.2 --pbratio 1.2 --direct auto --subme 9 --trellis 2 --psy-rd 0.8:0.8 --partitions all --8x8dct --me umh --merange 24 --mvrange 511 --aq-strength 1.0

[asarian]

Any suggestions?

Other than not letting threads go > 4 (see above), regrettably no. Dark Shikari is the man: only he can ultimately truly fix it. :)

But a sample would be nice, yes. Btw, did you also get the ugly fading with the Austin Powers sample posted in this thread? Dark Shikari's patch worked for that sample (here, at least).

[asarian]

sample clip: http://www.mediafire.com/?sharekey=508d6798bcda1276d2db6fb9a8902bda

Seems to go just fine here with Dark Shikari's patched version. :) (on a quadcore, with 6 threads).

[nm]

sample clip: http://www.mediafire.com/?sharekey=508d6798bcda1276d2db6fb9a8902bda
I also can't reproduce the issue with vanilla x264 r1070 when using your settings (I used 6 threads and a bitrate of 10 Mbps, and also tried 5 and 20 Mbps). My result is here (http://www.cs.helsinki.fi/u/mikkila/video/x264-r1070-fade.264).

[moviefan]

@nm: Your result is impressive, does the 5 Mbps version look more or less similar? When I encoded it with ~6 Mbps, I had an extremely washed up picture in the fading scene. Your 10 Mbps sample looks totally transparent... I will try an encode with r1070 and 6 threads.

Edit: Even with 10 Mbps I cannot reproduce your result. The picture lacks a lot of detail, is blurry and shows some blocks.

[shon3i]

@nm, what you processor have, aslo moviefan you hace quad-core right??

[moviefan]

I have an Intel Pentium Dual-Core E5200.

[nm]

I have T7200 (a dualcore) and there was no visible difference in the output between 3 and 6 threads.

@nm: Your result is impressive, does the 5 Mbps version look more or less similar?
Similar, but of course it wasn't quite as transparent (artifacts in the edges etc.)

Edit: Even with 10 Mbps I cannot reproduce your result. The picture lacks a lot of detail, is blurry and shows some blocks.
I'd suspect a problem in the input. Which decoder do you use and how is it set up? Make sure you haven't disabled inloop deblocking, for example. Have you tried playing the Avisynth script?

I used x264 through MEncoder on 64-bit Linux, without any additional filtering.

[moviefan]

I have opened the Avisynth-Script with VirtualDub and it looks fine. I don't use the --deblock option so it is set to 0:0 I guess. I encode the video via command line directly, so I have a batch-file in Windows that does the two passes automatically for me with the settings I posted a few posts before. Could you please post a sample of your 5 Mbps version? Just for comparison. I assume x264 allocates not enought bits to the fading scene. Even when the average bitrate is high, x264 allocates many bits to the final frames where there is no fading to make it look very good, but misses need bitrate in the fading, probably because it "thinks", there is almost no details, so get rid of the hard to see rest. But that's wrong. The only thing I am wondering about is why you get a good result...

Concerning the input, I use a current version of DGAVCDec and import the .dga-file in Avisynth via AVCSource.

[nm]

I have opened the Avisynth-Script with VirtualDub and it looks fine. I don't use the --deblock option so it is set to 0:0 I guess.
I meant skipping inloop deblocking in the decoder, but since you use DGAVCDec and the script plays fine, that's probably not the problem.

Could you please post a sample of your 5 Mbps version?
Sure, here you go (http://www.cs.helsinki.fi/u/mikkila/video/x264-r1070-fade-5mbps.264).

I assume x264 allocates not enought bits to the fading scene. Even when the average bitrate is high, x264 allocates many bits to the final frames where there is no fading to make it look very good, but misses need bitrate in the fading, probably because it "thinks", there is almost no details, so get rid of the hard to see rest. But that's wrong. The only thing I am wondering about is why you get a good result...
It is more likely a problem somewhere in your setup since asarian didn't see anything wrong either. Maybe an encoding log, stats file and the output video would get us further.

[Dark Shikari]

but misses need bitrate in the fading, probably because it "thinks", there is almost no details, so get rid of the hard to see rest.x264 goes out of its way to allocate more bits where there are few details, not less.

[moviefan]

@nm: I use MT 0.7 instead of the original avisynth.dll, could this be a problem? Do you use the original single-threaded avisynth?

[nm]

@nm: I use MT 0.7 instead of the original avisynth.dll, could this be a problem?
Hard to say (probably not, but better safe than sorry). What does your AviSynth script contain?

Do you use the original single-threaded avisynth?
Nope, just MEncoder:
I used x264 through MEncoder on 64-bit Linux, without any additional filtering.

[moviefan]

I installed the unmodified Avisynth 2.5.8, but it doesn't change anything. When I encode with the current version 1074M, I get this result (5 Mbps): http://www.mediafire.com/?sharekey=92e43124476f88d8d2db6fb9a8902bda

settings:

"%X264%" --threads auto --thread-input --progress --no-psnr --no-ssim "%IN_TITLE%.AVS" --stats "%IN_TITLE%.stats" --output NUL --pass 1 --bitrate %IN_BITRATE% --vbv-bufsize 30000 --vbv-maxrate 24000 --level 4.1 --keyint 24 --min-keyint 1 --aud --sar 1:1 --bframes 3 --b-adapt 2 --b-pyramid --ref 3 --mixed-refs --no-fast-pskip --ipratio 1.2 --pbratio 1.2 --direct auto --subme 2 --trellis 2 --partitions none --me dia --merange 24 --mvrange 511 --aq-mode 2 --aq-strength 1.0 --qcomp 0.6 --weightb --psy-rd 0.8:0.8 --nal-hrd

"%X264%" --threads auto --thread-input --progress --no-psnr --no-ssim "%IN_TITLE%.AVS" --stats "%IN_TITLE%.stats" --output "%IN_TITLE%.264" --pass 2 --bitrate %IN_BITRATE% --vbv-bufsize 30000 --vbv-maxrate 24000 --level 4.1 --keyint 24 --min-keyint 1 --aud --sar 1:1 --bframes 3 --b-pyramid --ref 3 --mixed-refs --no-fast-pskip --ipratio 1.2 --pbratio 1.2 --direct auto --subme 9 --trellis 2 --partitions all --8x8dct --me umh --merange 24 --mvrange 511 --aq-mode 2 --aq-strength 1.0 --qcomp 0.6 --weightb --psy-rd 0.8:0.8 --nal-hrd

What is going on here?? Note a few blocky and blurry frames in the sample.

Edit: I've found out that psy-trellis completely destroys the picture in the fading scene. When I go to a high value like 1.0, it becomes really obvious. Any explanation why that is happening?

[nm]

I installed the unmodified Avisynth 2.5.8, but it doesn't change anything. When I encode with the current version 1074M, I get this result (5 Mbps): http://www.mediafire.com/?sharekey=92e43124476f88d8d2db6fb9a8902bda
Something seems to be wrong with the link...

[moviefan]

link updated, try it now.

[Sharc]

Looks unexpectedly ugly indeed.
(I assume the decoder and source are ok).

The bit rate starts reasonably high during the black frames at the beginning of the scene and then decays dramatically during the fade-in, and finally recovers once the picture is at normal brightness.

I would once try with a different (simpler) profile for comparison (no b-pyramid) , like
...... --level 4.1 --keyint 24 --min-keyint 2 --ref 3 --mixed-refs --bframes 3 --weightb --filter -2:-2 --trellis 1 --psy-rd 0.7:0 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --ipratio 1.2 --pbratio 1.1 --vbv-bufsize 20000 --vbv-maxrate 20000 --qcomp 0.5 --me umh --threads auto --progress --sar 1:1 --no-psnr --no-ssim --mvrange 511 --aud --output .......

[moviefan]

Okay, I'll try that this evening and report back with a sample.

I assume, you consider the source "ok", if it looks ok when loading it to VirtualDub, right? If yes, the source is ok ;-).

Edit: sample is here http://www.mediafire.com/?yjymmkmzmzh
It looks (again) very ugly - I copied your settings exactly. What is going on here???
If you like to try a test encode yourself, you find the source sample here: https://forum.doom9.org/showthread.php?p=1231641#post1231641

[Rumbah]

Have you tried without VBV?

[Sharc]

.....It looks (again) very ugly - I copied your settings exactly. What is going on here???
If you like to try a test encode yourself, you find the source sample here: https://forum.doom9.org/showthread.php?p=1231641#post1231641
It looks as if you found a rather tricky scene. Surprisingly CRF single pass produces a much better - although not perfect - result at CRF=26.5 which results in about the same file size (approx. 5 Mbps average).
I guess the average bit rate is rather low with 5Mbps for a 1920x1080 picture anyway. Would probably be fine if you would resize to 1280x720.

[nm]

I had no such problems encoding the scene at 5 Mbps with 64-bit MEncoder and vanilla x264 on Linux, in 2-pass mode (check the earlier posts in this thread). Maybe the problem is somehow Windows- or AviSynth-specific. Did you use a patched x264 build?

[Sharc]

No, not a patched version.
The visibility of blocking / color banding depends much on the decoder one is using. Some decoders mask such effects better than others.

[nm]

The visibility of blocking / color banding depends much on the decoder one is using. Some decoders mask such effects better than others.
There is a huge difference between the clips encoded by moviefan (http://www.mediafire.com/?yjymmkmzmzh) and myself (http://www.cs.helsinki.fi/u/mikkila/video/x264-r1070-fade-5mbps.264) when using the same decoder.

[shon3i]

@Dark Shikari, did you have some plans about this in newer versions, because every new vanilla build have same problem when VBV used??

[Dark Shikari]

@Dark Shikari, did you have some plans about this in newer versions, because every new vanilla build have same problem when VBV used??My local build environment is broken with threads, so this makes it far more annoying to test things...

... I'll get around to it. But it'll come much faster if someone can tell me how to fix my local Cygwin install, where if I use -mno-cygwin (use mingw for compilation), gcc can't even find the right cc.exe... even though I completely wiped and reinstalled Cygwin and MinGW.

Or told me how to get pthreads to work under native cygwin (NOT mingw).

[moviefan]

- Encoding without VBV doesn't help.
- Encoding 1080p material at 5Mbps generally works well for me, if I preprocess the source to be more compressible.
- There is indeed a big difference between nm's encode and mine!

When playing my messed up samples frame by frame, I have noticed that every 5th or 6th frame (counting from a very good frame) is good again, the 4/5 frames between those are messed up with blocks, banding and very little details. Maybe this helps to get to the bottom of this problem. I have no deep knowledge of video compression but I know there are different frame types (I, P, B) where there are reference frames and frames that rely on this reference... Maybe the good looking frame is a reference frame or something like that and the others refer to that, but because fading is a difficult thing, they get messed up... Perhaps you guys with proper knowledge of the encoding techniques used here can explain that better.

[Dark Shikari]

- Encoding without VBV doesn't help.Then the problem is completely unrelated to the original topic of this thread.

[moviefan]

Maybe to the original matter but not unrelated to the topic ;). Just to be sure: encoding without VBV means leaving out the --vbv-options in the command line right?

[Sharc]

There is a huge difference between the clips encoded by moviefan (http://www.mediafire.com/?yjymmkmzmzh) and myself (http://www.cs.helsinki.fi/u/mikkila/video/x264-r1070-fade-5mbps.264) when using the same decoder.
nm: you are using x264-r1070
moviefan: you are using x264-r1074

Although you say having used exactly the same encoder settings, the bit allocation is totally different, as can be seen from attached pictures for the first 40 frames, changing from black into the fading scene (red=I, blue=P, green=B).
The vertical scale is the frame size in bytes, almost zero bytes for the fading frames in moviefan's clip! Why so much difference ??

[nm]

nm: you are using x264-r1070
moviefan: you are using x264-r1074
moviefan was having the same problem with earlier x264 versions.

Although you say having used exactly the same encoder settings, the bit allocation is totally different, as can be seen from attached pictures for the first 40 frames, changing from black into the fading scene (red=I, blue=P, green=B).
Well, I'd say that bit allocation is indeed the problem here, but the cause is unclear.

[moviefan]

Correct, and I wonder whether other decisions are made non-optimal as well in my encodes, apart from fading scenes... By the way, I tried various versions, modded and official.

[asarian]

@moviefan,

Like I said earlier, your sample encodes just fine here with 6 threads (4 cores) on Dark Shikari's specially patched "x264 core:65 r1051" version. I uploaded the result here:

http://www.mediafire.com/?sharekey=180a5a58b0132c1ad2db6fb9a8902bda

So, maybe there's something else in your environment at work?

[moviefan]

When looking at Sharc's graph, I wonder why there are no B-frames in my encode (I assume mine is the bottom one) and the I frames get very much bitrate at the right half. I will encode the same sample clip with exactly the same settings on my old Pentium 4 laptop to see if that gives a different result. Maybe then we make progress with finding out what is going on here.

@asarian: Hm, I can't imagine what should interfere the encoding in such a way... I have a pretty freshly set up system, Windows XP Professional SP3, Avisynth 2.58 final, DGAVCDec latest release, a number of filter plugins for Avisynth, the AVS script only contains AVCSource() and looks fine in VirtualDub.

My hardware:
- Intel Pentium Dual-Core E5200
- 4 GB Corsair DDR2-800 RAM
- Gigabyte EP35-DS4 mainboard
- Club3D ATI Radeon 4670 graphics card

The CPU does not support SSE4. x264 shows the capabilities used up to SSE3.

[Sharc]

Sorry, I forgot to add a legend.
moviefan: top graph, with almost zero size frames in the right half of the graph (fade scene) => blocky pictures
nm: bottom graph, without B frames and large I frames => better quality pictures

[moviefan]

Ah, okay, so B-frames are not a good idea in fading scene I conclude...? And why are the I-frames in nm's encode so extreme in comparison to the P-frames and mine don't show such a big difference? I have never seen such a graph, so I have no basis for comparison ;)

[Dark Shikari]

Ah, okay, so B-frames are not a good idea in fading scene I conclude...?No, B-frames are extremely good in fades.

[Sharc]

@moviefan,

Like I said earlier, your sample encodes just fine here with 6 threads (4 cores) on Dark Shikari's specially patched "x264 core:65 r1051" version. I uploaded the result here:

http://www.mediafire.com/?sharekey=180a5a58b0132c1ad2db6fb9a8902bda

asarian:
It's pointless to compare your 24 MB file, encoded at 38 Mbps, to moviefan's 3 MB file encoded at 5 ....6 Mbps.
Or did you upload the original clip by mistake?

[nm]

Ok, I found out why my encode was fine: I forgot to change pass=1 to pass=2 for the second pass, so it was encoded in cbr mode. Sorry!

I was able to duplicate the issue in 2-pass mode, and looks like psy-rd triggers most of the blocking.

[moviefan]

OK, thanks for reviewing your settings and finally finding out why we had so different outputs, although this is a bad news since it is not a fault on my side but on the encoder's. Now there pops up the question whether it can be solved and if yes, how. Can this problem be compensated by different settings or is it a major problem in an x264 algorithm concerning frame type decision, bitrate distribution etc.

In fact, I noticed particularly strong blocking when raising the psy-trellis value above 0.0

[asarian]

asarian:
It's pointless to compare your 24 MB file, encoded at 38 Mbps, to moviefan's 3 MB file encoded at 5 ....6 Mbps.
Or did you upload the original clip by mistake?

Not so pointless. :) Although I use considerably heftier encoder settings than moviefan, I compared my sample made with patched "x264 core:65 r1051" to any other x264 version (with the same encoder settings), and only the patched version prevents the ugly fading.

[Sharc]

Oh I see your point now. Means that all but DS's patch seem to have a problem there, even at very high bitrates. Would be interesting to see how the patch behaves at 5000 kbps.

[nm]

Oh I see your point now. Means that all but DS's patch seem to have a problem there, even at very high bitrates. Would be interesting to see how the patch behaves at 5000 kbps.
moviefan tried it earlier and I tried it now. Didn't see a difference to other builds at 5 Mbps: same heavy blocking with psy-rd and psy-trellis.

[moviefan]

Unless there will be a working solution in the near future (a bug fix, if it's a bug or a "fade-feature"), can I divide my video into three parts
1) part before the fading scene
2) fading scene (with CBR because nm's encode looks good)
3) part after the fading scene
without losing the overall bitrate distribution like it would be when encoding it in one part? If I encode the three parts separately, the first part will not take the bitrate of the third video into account and thus distribute the bits in a different way (and vice versa).

[Sharc]

Not sure if it would help you: The clip looked much better when encoding in 1-pass CRF mode. I used CRF=26.5 to produce about the same file size. Problem with CRF is (un)predictability of the file size, and perhaps VBV issues with your playback device.

[asarian]

Not sure if it would help you: The clip looked much better when encoding in 1-pass CRF mode. I used CRF=26.5 to produce about the same file size. Problem with CRF is (un)predictability of the file size, and perhaps VBV issues with your playback device.

I encoded the clip with 1-pass CRF 16. :)

[moviefan]

I have noticed that setting --me umh --partitions all --subme 9 also in the first pass, combined with --trellis 2 but psy-trellis 0.0 gives best results so far concerning my fade-in issue, still not perfect, but a lot better than the other test encodes I have done. Maybe one should consider setting the first pass options higher to these values at the cost of speed but for better quality.

[asarian]

With regard to the originally mentioned (and posted) Austin Powers sample, "x264 core:66 r1093M 1df50b9" no longer produces the ugly fades on that sample. Could the issue be resolved? :)

[LoRd_MuldeR]

With regard to the originally mentioned (and posted) Austin Powers sample, "x264 core:66 r1093M 1df50b9" no longer produces the ugly fades on that sample. Could the issue be resolved? :)

Probably a result of:
http://git.videolan.org/gitweb.cgi?p=x264.git;a=commit;h=71c5a8dca6e5f7bf2330d028989eeeab27701151

[moviefan]

I still have the problem I have mentioned some time ago with the scene I posted. Would it be possible to replace this short scene (start from the first I-frame of that scene to the frame before the first I-frame after the scene) with that scene consisting of I-frames only?

[LoRd_MuldeR]

Would it be possible to replace this short scene (start from the first I-frame of that scene to the frame before the first I-frame after the scene) with that scene consisting of I-frames only?

You could achieve this by using zones, I think.

Or by hacking the stats file, which may not be the best idea (as it hurts rate control).

[moviefan]

How do the zones have to look like if I want I-frames only for say 100 frames?

[kemuri-_9]

How do the zones have to look like if I want I-frames only for say 100 frames?

for this you wouldn't use zones but the --qpfile option.
see how to use it in --longhelp

[moviefan]

Does qpfile interfere with VBV and thus Blu-ray compliancy in my case?

[J_Darnley]

It's not the fact that you use a qpfile that will interfere with VBV and other restrictions, it is what you do with it. If you ask for 100 I-frames then VBV could start killing the picture because I-frames are large. This may also not work with Blu-Ray restrictions too. What is the minimum key-frame interval that is usually used?

[moviefan]

I use --min-keyint 1 and --keyint 24, --vbv-bufsize 30000 and --vbv-maxrate 24000. In fact, I am not keen on inserting I-frames only, but this is ONE possibility to increase the quality of that fade-in scene. The thing is that I don't see any other way to achieve that... Otherwise I am open for any ideas, but I'm afraid this problem has been discussed here without a helping result. So do 100 I-frames in a row interfere with said compliance considering the above mentioned settings?

Thanks for help!

[Dark Shikari]

I use --min-keyint 1 and --keyint 24, --vbv-bufsize 30000 and --vbv-maxrate 24000. In fact, I am not keen on inserting I-frames only, but this is ONE possibility to increase the quality of that fade-in scene. The thing is that I don't see any other way to achieve that... Otherwise I am open for any ideas, but I'm afraid this problem has been discussed here without a helping result. So do 100 I-frames in a row interfere with said compliance considering the above mentioned settings?

Thanks for help!Interfere with compliance? No. Interfere with quality? Maybe.

[nm]

I use --min-keyint 1 and --keyint 24, --vbv-bufsize 30000 and --vbv-maxrate 24000. In fact, I am not keen on inserting I-frames only, but this is ONE possibility to increase the quality of that fade-in scene. The thing is that I don't see any other way to achieve that...
How about just encoding the movie without psy-rd (the fade scene looked fine to me that way), or alternatively, use CRF instead of 2-pass?

[moviefan]

Hm, CRF won't work since I aim at a specific target size. Encoding without psy-rd, I could try that but psy-rd keeps grain well - maybe disabling it result in worse grain retention, but I will definitely try! In case grain looks bad without psy-rd: Could I use the encode I have so far, which is very pleasing (except for that fading scene) and replace the fade by a CRF encode of it (and then without psy-rd)?

[Dark Shikari]

Hm, CRF won't work since I aim at a specific target size. Encoding without psy-rd, I could try that but psy-rd keeps grain well - maybe disabling it result in worse grain retention, but I will definitely try! In case grain looks bad without psy-rd: Could I use the encode I have so far, which is very pleasing (except for that fading scene) and replace the fade by a CRF encode of it (and then without psy-rd)?You could also try using the same encoding settings on both passes... that tends to help bitrate mispredictions with psy-RD. Usually just using subme 7/8x8dct on the first pass is sufficient though.

[nm]

You could also try using the same encoding settings on both passes... that tends to help bitrate mispredictions with psy-RD. Usually just using subme 7/8x8dct on the first pass is sufficient though.
Indeed, using subme 7 and 8x8dct on the first pass fixes the problem in moviefan's clip.

[moviefan]

@nm: Have you tried it? I will only be able in a couple of hours...

[nm]

@nm: Have you tried it? I will only be able in a couple of hours...
Yep. I was just experimenting with using zones on the clip: --zones 1,50,q=20 also helped to avoid the problem.
But DS's suggestion worked perfectly.

[moviefan]

Unfortunately and without explanation, the fading scene still contains ugly frames when I look at it frame by frame. It does look better with --8x8dct --subme 9 (which is what I use... 9 instead of 7 shouldn't harm quality right?) but compared to the rest of the movie, it still looks very ugly.

@nm: Could you please upload the encoded sample which you consider good looking now? Also please post your command line to check the difference in order to hopefully optimize mine. Thanks!

[nm]

Unfortunately and without explanation, the fading scene still contains ugly frames when I look at it frame by frame. It does look better with --8x8dct --subme 9 (which is what I use... 9 instead of 7 shouldn't harm quality right?) but compared to the rest of the movie, it still looks very ugly.
Yes, if I look at it frame-by-frame, the first 10-20 frames seem to exhibit some blocking, but I can't see the issue in normal playback even if I look for it.
If this still bothers you, try using a constant QP zone as I described above.

A sample (http://www.cs.helsinki.fi/u/mikkila/video/x264-r1070-fade-5mbps.264) with psy-rd 1.0:0.5. Rest of the parameters are same as yours. I'm not sure if I encoded this with subme 7 or 9 in the first pass, but that didn't seem to make a difference.