Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 22nd July 2014, 11:24   #1101  |  Link
upyzl
zj262144
 
upyzl's Avatar
 
Join Date: Sep 2010
Posts: 105
@LigH

oh, that's it... I thought the cmd would just output only the human-readable info, and if that it could be saved by a string variable directly for other usage...
__________________
MPC-HC 1.7.8 / LAV Filters 0.64+ (tMod) / XySubFilter 3.1.0.705 / madVR 0.87.14

Direct264 Mod (src & win32 builds): code.google.com/p/direct264umod (maybe outdated)
upyzl is offline   Reply With Quote
Old 22nd July 2014, 11:28   #1102  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,984
The tool "strings" just filters longer sequences of readable characters, without knowing if it is actually text meant to be read by humans. "Random" sequences of data bytes "incidently" making readable character sequences pass it too.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 25th July 2014, 00:19   #1103  |  Link
xooyoozoo
Registered User
 
Join Date: Dec 2012
Posts: 199
Multi-pass encoding is now enabled through the cli.

Based on a few tests, the SSIM of CRF ~= Pass2 >= Pass1, so it seems to be working fine. However, "turbo first pass" doesn't exist yet.
xooyoozoo is offline   Reply With Quote
Old 25th July 2014, 01:04   #1104  |  Link
x265_Project
Registered User
 
Join Date: Jul 2013
Posts: 596
Yes, 2 pass encoding is enabled in the Dev build, but there are a few bugs that we are aware of and working on (dealing with compatibility with rate control options used in the first or second pass). Detailed feedback (ideally, with source clips available to us and steps to reproduce the results) is welcomed.

Tom
x265_Project is offline   Reply With Quote
Old 26th July 2014, 12:51   #1105  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,419
Quote:
Originally Posted by xooyoozoo View Post
Multi-pass encoding is now enabled through the cli.

Based on a few tests, the SSIM of CRF ~= Pass2 >= Pass1, so it seems to be working fine. However, "turbo first pass" doesn't exist yet.
well you can make yourself "turbo first pass". I make several test and that work very well ...
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9
Sagittaire is offline   Reply With Quote
Old 26th July 2014, 16:03   #1106  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,929
Quote:
I make several test and that work very well ...
What options did you disable/change during 1st pass?

----
btw. 2pass + qpfile -> crash (https://bitbucket.org/multicoreware/...frame-type-not)
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..
Selur is offline   Reply With Quote
Old 26th July 2014, 16:18   #1107  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,419
Quote:
Originally Posted by Selur View Post
What options did you disable/change during 1st pass?

----
btw. 2pass + qpfile -> crash (https://bitbucket.org/multicoreware/...frame-type-not)
I make that for good first pass speed:

Code:
x265.exe --input hp.yuv --output crf23aq.265 --input-res 720x304 --fps 25 --pass 1 --stats hp.log --bitrate 450 
--preset fast --me dia --ref 1 --aq-mode 0 --aq-strength 1.0 --psy-rd 0.0 --bframes 3 --b-adapt 2 --min-keyint 1 --ipratio 1.2 --pbratio 1.2 --psnr
Only frame decision is important (b-adapt 2). You can use very fast quality decision without problem (me, subme, ref, rd ...).
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9

Last edited by Sagittaire; 26th July 2014 at 19:50.
Sagittaire is offline   Reply With Quote
Old 26th July 2014, 18:19   #1108  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,929
thanks
__________________
Hybrid here in the forum, homepage
Notice: Since email notifications do not work here any more, it might take me quite some time to notice a reply to a thread,..
Selur is offline   Reply With Quote
Old 26th July 2014, 19:31   #1109  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,419
well some test with 2 pass mode:

- fast first pass mode with crf
- Npass with fast, medium, slow, slower, veryslow and placebo preset

Code:
|--------------|----------|----------|-----------|
| fast         | 449 kbps | 42.97 dB | 13.98 fps |
| medium       | 449 kbps | 43.10 dB | 12.13 fps |
| slow         | 449 kbps | 43.35 dB |  3.40 fps |
| slower       | 448 kbps | 43.72 dB |  1.22 fps |
| veryslow     | 449 kbps | 43.74 dB |  0.75 fps |
| placebo      | 449 kbps | 43.75 dB |  0.47 fps |
| balanced     | 449 kbps | 43.31 dB |  7.93 fps |
|--------------|----------|----------|-----------|
Conclusion:

- 2 pass mode work very well even if you use fast first pass.
- On my system (c2d Q6600) you have abyssal speed delta between medium and slow preset. It's certainely in the most important speed zone because BD Ripp 1080p will be done in this speed zone with performant CPU system. I propose alternative "balanced" preset setting in this really important zone.

balanced = slower + rd 4 + no-rect + no-amp + me hex
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9
Sagittaire is offline   Reply With Quote
Old 27th July 2014, 04:35   #1110  |  Link
x265_Project
Registered User
 
Join Date: Jul 2013
Posts: 596
Quote:
Originally Posted by Sagittaire View Post
well some test with 2 pass mode:

- fast first pass mode with crf
- Npass with fast, medium, slow, slower, veryslow and placebo preset

Code:
|--------------|----------|----------|-----------|
| fast         | 449 kbps | 42.97 dB | 13.98 fps |
| medium       | 449 kbps | 43.10 dB | 12.13 fps |
| slow         | 449 kbps | 43.35 dB |  3.40 fps |
| slower       | 448 kbps | 43.72 dB |  1.22 fps |
| veryslow     | 449 kbps | 43.74 dB |  0.75 fps |
| placebo      | 449 kbps | 43.75 dB |  0.47 fps |
| balanced     | 449 kbps | 43.31 dB |  7.93 fps |
|--------------|----------|----------|-----------|
Conclusion:

- 2 pass mode work very well even if you use fast first pass.
- On my system (c2d Q6600) you have abyssal speed delta between medium and slow preset. It's certainly in the most important speed zone because BD Ripp 1080p will be done in this speed zone with performant CPU system. I propose alternative "balanced" preset setting in this really important zone.

balanced = slower + rd 4 + no-rect + no-amp + me hex
Hi Sagittaire,
Thanks for the feedback. The big difference between medium and slow is that medium doesn't test rectangular blocks (--no-rect). Once you turn on --rect, you greatly increase the number of possible ways to partition a 64x64 CU. You can see what I mean if you look at the top of page 3 of this paper... https://www.ic.tu-berlin.de/fileadmi...12_HEVC-BM.pdf. At the first level of the quad-tree, you can partition a CU 8 different ways. With --no-rect, this drops to only 2 possibilities. The same is true at every level of the quad-tree except when you get down to 8x8 pixel blocks.

So, once you add --rect, there is a big increase in the number of possible partitions that must be checked to see which results in the most efficient way to encode the CU.
x265_Project is offline   Reply With Quote
Old 27th July 2014, 05:09   #1111  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,437
Quote:
Originally Posted by x265_Project View Post
Hi Sagittaire,
Thanks for the feedback. The big difference between medium and slow is that medium doesn't test rectangular blocks (--no-rect). Once you turn on --rect, you greatly increase the number of possible ways to partition a 64x64 CU. You can see what I mean if you look at the top of page 3 of this paper... https://www.ic.tu-berlin.de/fileadmi...12_HEVC-BM.pdf. At the first level of the quad-tree, you can partition a CU 8 different ways. With --no-rect, this drops to only 2 possibilities. The same is true at every level of the quad-tree except when you get down to 8x8 pixel blocks.

So, once you add --rect, there is a big increase in the number of possible partitions that must be checked to see which results in the most efficient way to encode the CU.
Maybe --rect could become a 0-3 argument, each testing more levels and partition possibilities at each level? Or make it depend on the rd level? Similar to how Steve talked about minimizing the number of intra tests on the mailing list a few days ago.
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order.
foxyshadis is offline   Reply With Quote
Old 27th July 2014, 06:19   #1112  |  Link
x265_Project
Registered User
 
Join Date: Jul 2013
Posts: 596
Quote:
Originally Posted by foxyshadis View Post
Maybe --rect could become a 0-3 argument, each testing more levels and partition possibilities at each level? Or make it depend on the rd level? Similar to how Steve talked about minimizing the number of intra tests on the mailing list a few days ago.
The same thought occurred to me as I typed the response above. I didn't want to raise the possibility publicly without checking on the scope of effort, so I emailed our Dev team asking if --rect 1 could be limited to 2N x N rectangles , with --rect 2 adding all of the 2N x n rectangles. We certainly have lots of other higher priorities right now, but let's see what they say in terms of the complexity that this would create and the scope of effort. It seems logical that it would create an intermediate level of partitioning that would enable a smaller step along the speed vs. efficiency tradeoff curve.
x265_Project is offline   Reply With Quote
Old 29th July 2014, 03:56   #1113  |  Link
fumoffu
Registered User
 
Join Date: May 2013
Posts: 90
I have a question (I think it has been touched before but that was a long time ago).
Why merange is so high in all presets? Does it work the same as x264 merange?
because as you can find on some forums and also trough testing:
"...a large merange is basically useless. merange is the distance to search from the best predictor, not the maximum motion vector range. The best predictor is generally close enough to the correct MV that a large search radius is a waste of time."
and so in x264 in almost all presets merange is only 16 and changing it doesn't really do much except increasing encoding time...
So what is the reason for setting it so high in x265?
fumoffu is offline   Reply With Quote
Old 29th July 2014, 04:27   #1114  |  Link
x265_Project
Registered User
 
Join Date: Jul 2013
Posts: 596
Quote:
Originally Posted by foxyshadis View Post
Maybe --rect could become a 0-3 argument, each testing more levels and partition possibilities at each level? Or make it depend on the rd level? Similar to how Steve talked about minimizing the number of intra tests on the mailing list a few days ago.
Our dev leads reminded me that --amp controls whether x265 checks the additional n x 2N rectangular partition possibilities. However, even with --amp we only ever check at most one AMP prediction, based on the results of the N x 2N rectangular predictions.

So, there are no intermediate levels that don't already exist.

There are other dials we can turn to spread out the distribution of performance presets, but we have some additional coding tools that aren't on by default yet (psy-rd and psy-rdoq), and we need to figure out where they should come into play, and the impact on performance and visual quality. We don't want to change things too often, so it's better to do this all at once.

Tom
x265_Project is offline   Reply With Quote
Old 29th July 2014, 04:43   #1115  |  Link
upyzl
zj262144
 
upyzl's Avatar
 
Join Date: Sep 2010
Posts: 105
Quote:
Originally Posted by fumoffu View Post
I have a question (I think it has been touched before but that was a long time ago).
Why merange is so high in all presets? Does it work the same as x264 merange?
because as you can find on some forums and also trough testing:
"...a large merange is basically useless. merange is the distance to search from the best predictor, not the maximum motion vector range. The best predictor is generally close enough to the correct MV that a large search radius is a waste of time."
and so in x264 in almost all presets merange is only 16 and changing it doesn't really do much except increasing encoding time...
So what is the reason for setting it so high in x265?
I'm not expert on it, but AFAIK, H.264/AVC coding unit should be MacroBlock, which size is 16x16; and H.265/HEVC coding unit is CTU, which default is 64x64

and here is the doc from MCW x265 Team:
http://x265.readthedocs.org/en/defau...ption--merange
__________________
MPC-HC 1.7.8 / LAV Filters 0.64+ (tMod) / XySubFilter 3.1.0.705 / madVR 0.87.14

Direct264 Mod (src & win32 builds): code.google.com/p/direct264umod (maybe outdated)

Last edited by upyzl; 29th July 2014 at 04:52.
upyzl is offline   Reply With Quote
Old 29th July 2014, 05:52   #1116  |  Link
x265_Project
Registered User
 
Join Date: Jul 2013
Posts: 596
Quote:
Originally Posted by fumoffu View Post
I have a question (I think it has been touched before but that was a long time ago).
Why merange is so high in all presets? Does it work the same as x264 merange?
because as you can find on some forums and also trough testing:
"...a large merange is basically useless. merange is the distance to search from the best predictor, not the maximum motion vector range. The best predictor is generally close enough to the correct MV that a large search radius is a waste of time."
and so in x264 in almost all presets merange is only 16 and changing it doesn't really do much except increasing encoding time...
So what is the reason for setting it so high in x265?
When we created our performance presets we ran many tests to see the effect on performance and quality for each parameter. Reducing the merange at most presets didn't produce any meaningful improvement on performance. It does have a slightly negative impact on encoding efficiency.

It's easy to run your own tests to verify this. Just run multiple encodes of the same clip with --ssim on (to measure quality), varying --merange for each test run. Your feedback and test results are welcomed!
x265_Project is offline   Reply With Quote
Old 29th July 2014, 11:01   #1117  |  Link
Kurtnoise
Swallowed in the Sea
 
Kurtnoise's Avatar
 
Join Date: Oct 2002
Location: Aix-en-Provence, France
Posts: 5,183
Quote:
Originally Posted by x265_Project View Post
Yes, 2 pass encoding is enabled in the Dev build, but there are a few bugs that we are aware of and working on (dealing with compatibility with rate control options used in the first or second pass). Detailed feedback (ideally, with source clips available to us and steps to reproduce the results) is welcomed.
Using any presets lower than "Fast" crash on the 2nd pass, here (well...not a crash but a lot of warnings/errors :

Quote:
x265 [warning]: specified frame type is not compatible with max B-frames
x265 [warning]: specified frame type is not compatible with max B-frames
x265 [error]: slice=P but 2pass stats say B
x265 [error]: slice=P but 2pass stats say B
x265 [warning]: specified frame type is not compatible with max B-frames

Are you able to reproduce this ?

Currently, I'm using version 1.2+349-8bab on Windows 8.1 64 bits - 8bpp.

I tried different sources with different length and different bitrates but the issue still appears...

Code:
--preset superfast --pass 1/2 --bitrate 2500 --stats "file.stats" --output "output.hevc" "input.avs"

Last edited by Kurtnoise; 29th July 2014 at 11:04.
Kurtnoise is offline   Reply With Quote
Old 29th July 2014, 11:40   #1118  |  Link
Sagittaire
Testeur de codecs
 
Sagittaire's Avatar
 
Join Date: May 2003
Location: France
Posts: 2,419
Perhaps that preset under "fast" have lower bframe number. try to have same frametype structure in first and second pass (min key frame, max key frame, bframe number, pyramidal bframe, closed or open GOP ...).
__________________
Le Sagittaire ... ;-)

1- Ateme AVC or x264
2- VP7 or RV10 only for anime
3- XviD, DivX or WMV9
Sagittaire is offline   Reply With Quote
Old 29th July 2014, 17:42   #1119  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,437
Kurt, are you sure the stats file got updated when you ran the first pass again? You'd get those errors if you'd run a first pass with --preset medium or above and a second pass with --preset fast or lower.
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order.
foxyshadis is offline   Reply With Quote
Old 29th July 2014, 19:19   #1120  |  Link
x265_Project
Registered User
 
Join Date: Jul 2013
Posts: 596
Quote:
Originally Posted by Kurtnoise View Post
Using any presets lower than "Fast" crash on the 2nd pass, here (well...not a crash but a lot of warnings/errors :




Are you able to reproduce this ?

Currently, I'm using version 1.2+349-8bab on Windows 8.1 64 bits - 8bpp.

I tried different sources with different length and different bitrates but the issue still appears...

Code:
--preset superfast --pass 1/2 --bitrate 2500 --stats "file.stats" --output "output.hevc" "input.avs"
The --bframes setting must match between the 1st and 2nd pass. So, if you want to run a faster first pass using a performance preset that uses a different --bframes value, just add --bframes to the command line for the 1st pass, and use the value associated with the setting you will use for your 2nd pass.
You can find the bframes setting for each performance preset here... http://x265.readthedocs.org/en/default/presets.html

We're also aware of an issue with 2 pass and presets faster, superfast or ultrafast. Currently you don't get any B frames in the 2nd pass, making the 2nd pass less efficient than the first. The team is working to fix this.
x265_Project is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 19:13.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.