Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 17th March 2019, 07:06   #121  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
Quote:
Originally Posted by Iron_Mike View Post
option #2 always resulted in success, #1 is the one that reports seek errors...

question is if the seek-test script is not correct, or if all plugs u mentioned above all are 3 frames off... then they would all produce the same results, which are ultimately incorrect
On the contrary, it shows that regardless of what seek errors the seek-test script is reporting for ffms2, the metric scores come out correct - i.e. they are exactly the same as obtained with LWLibavSource.

Why do you say 'or if all plugs u mentioned above all are 3 frames off' when the seek-test reports no issues with LWLibavSource ?

Also what is the conclusion when (ffms2) option #5 (seekmode=0) reports 'No seeking issues found' but (ffms2) option #1 reports seek errors - in this case the "3 frames off" thing ?
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 17th March 2019 at 07:43.
WorBry is offline   Reply With Quote
Old 17th March 2019, 07:15   #122  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
Quote:
Originally Posted by Iron_Mike View Post
Encoded a file w/ the same settings and same ffmpeg Zeranoe build as WorBry did using the 1080p50 crowdrun as ref file. Made two encodes, one with and one without keyframes/GOPsize specified.

When I encode w/ the keyframes option (specifying a GOP size) -x265-params "keyint=100:min-keyint=100:rc-lookahead=100".....

(2) the SSIM only matches the first few digits of WorBrys results...
Were they the exact same settings that I used:

Quote:
Originally Posted by WorBry View Post
So, I encoded crowd_run_1080p50.y4m to x265 CRF28 mp4:

Code:
ffmpeg -i {Path}:/crowd_run_1080p50.y4m -vcodec libx265 -preset slow -crf 28 -pix_fmt yuv420p -r 50/1 -x265-params colorprim=1:transfer=1:colormatrix=1 {Path):/crowd_run_1080p50_x265_CRF28.mp4
...or did you add "-x265-params rc-lookahead=100" ?
__________________
Nostalgia's not what it used to be
WorBry is offline   Reply With Quote
Old 17th March 2019, 09:12   #123  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by WorBry View Post
On the contrary, it shows that regardless of what seek errors the seek-test script is reporting for ffms2, the metric scores come out correct - i.e. they are exactly the same as obtained with LWLibavSource.

Why do you say 'or if all plugs u mentioned above all are 3 frames off' when the seek-test reports no issues with LWLibavSource ?
must have missed the connection of the LWLibavSource, I don't know what that is... is that the lib for LSmash-Works ? (I'm not familiar w/ many of these plugs)

Alright, so, if the seek script reports no error for using LSmash, but does report errors via ffms2, yet the GMSD/SSIM scores are the exact same when the vid files are read via ffms2 or Lsmash, then one would assume that the seek test script has either flawed logic or at least the way it uses ffms2 in the #1 test is different from the way it is used when the vid files are read in your vpy script...


What is the command syntax for reading a vid file via Lsmash/LWLibavSource ?


Quote:
Originally Posted by WorBry View Post
Also what is the conclusion when (ffms2) option #5 (seekmode=0) reports 'No seeking issues found' but (ffms2) option #1 reports seek errors - in this case the "3 frames off" thing ?
+1 on that question. Since #5 is labeled as "slow, but more safe" I assumed that this is the better verification but then CK said to test via #1 as well...


Quote:
Originally Posted by WorBry View Post
Were they the exact same settings that I used:

...or did you add "-x265-params rc-lookahead=100" ?
...
edit: I edited my original post for clarity, I hope it makes more sense now ;-) https://forum.doom9.org/showthread.p...19#post1869119
...

one encode was exactly your settings - as you can see the GMSD matches but the SSIM only matches on the first 5 decimal points... (that again is odd, considering we used same source, same ffmpeg, same settings)

then another encode w/ your settings but in addition setting the keyframes/GOPsize... the resulting file always produces different GMSD/SSIM results, meaning whatever plugs/modules/classes are utilized to obtain these metrics, at least one of them has a problem w/ that file (or the keyframes in that file)... I just have zero idea why it produces always different results - maybe the frame seek returns a random frame, hence the random metric score...

btw, I ran a verification test on that encode to check if all keyframes are in the correct position (every 100 frames, hence every 2 secs) - and they are... I doin't think it's a bad encode, it's just triggers bad logic in at least one of the involved plugs/modules/classes...

Last edited by Iron_Mike; 17th March 2019 at 09:29.
Iron_Mike is offline   Reply With Quote
Old 17th March 2019, 11:30   #124  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
Quote:
Originally Posted by Iron_Mike View Post
+1 on that question. Since #5 is labeled as "slow, but more safe" I assumed that this is the better verification but then CK said to test via #1 as well...
seekmode is a ffms2 parameter, see here https://github.com/FFMS/ffms2/blob/m...nt-seekmode--1
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 17th March 2019, 13:17   #125  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by ChaosKing View Post
seekmode is a ffms2 parameter, see here https://github.com/FFMS/ffms2/blob/m...nt-seekmode--1
ah, thanks for that.

so I just re-ran the GMSD/SSIM tests but opened both files w/ ffms and seekmode=0, and the metrics score is the same.

so I guess seekmode is not used when these tests are run (since the seek test script indicated that seek mode is 3 frames off, the scores would be different)...

Last edited by Iron_Mike; 17th March 2019 at 13:31.
Iron_Mike is offline   Reply With Quote
Old 17th March 2019, 13:48   #126  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
Can you upload a small cut of your video file that you're testing?
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 17th March 2019, 15:21   #127  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
Quote:
Originally Posted by Iron_Mike View Post
must have missed the connection of the LWLibavSource.
Quote:
Originally Posted by WorBry View Post
I've re-tested the CrowdRun x265 CRF28.mp4 encode from..

https://forum.doom9.org/showthread.p...94#post1869094

....with the original ffms2 version that came with VapourSynth Fatpack Portable and L-Smash Source (LWLibavSource) and both produce exactly the same SSIM and GMSD scores as Wolfberry's ffms2 did
Quote:
Originally Posted by Iron_Mike View Post
I don't know what that is... is that the lib for LSmash-Works ? (I'm not familiar w/ many of these plugs)
You said you tested it, but got an error.

Quote:
Originally Posted by Iron_Mike View Post
Quote:
Originally Posted by ChaosKing View Post
And have you tried lsmash, that should be ok.
just did that, it reports "[hevc @ 0000009525d3eb20] missing picture in access unit"

but the seek test was successful (no errors).
Quote:
Originally Posted by Iron_Mike View Post
What is the command syntax for reading a vid file via Lsmash/LWLibavSource ?
Code:
clip = core.lsmas.LWLibavSource(source=r'{Path}:/video')
Quote:
Originally Posted by Iron_Mike View Post
edit: I edited my original post for clarity, I hope it makes more sense now ;-) https://forum.doom9.org/showthread.p...19#post1869119
I don't know why you get inconsistent results on consecutive metric testing. I don't.

Please, no more requests for me to test this and that.

Over to you ChaosKing.

Quote:
Originally Posted by ChaosKing View Post
Can you upload a small cut of your video file that you're testing?
Edit: One last test.

I ran the SSIM:GMSD script with the 1080/50p x265 CRF28.mp4 encode imported with ffms2 and LWLibavSource - both plugins being the versions that came with VapourSynth Fatpack Portable.

I used the updated Zoptilib.py that provides the per frame metric scores:

https://forum.doom9.org/showthread.p...21#post1868821

In both tests, opened up the Previews in VSEditor and 'sent' to Frame #50. Here are the frame grabs:



Click on image to enlarge and (+) cursor to enlarge further.

Both tests exported exact same frame and per-frame SSIM/GMSD scores. Same thing stepping through the frames.

I also examined the x265 CRF26 encode in Elecard StreamEye and TMPGEnc SmartRenderer5 and they display the same frame for frame #50 i.e. frame #50 in the display sequence, but #52 in the stream sequence - it's a B frame.
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 17th March 2019 at 19:27.
WorBry is offline   Reply With Quote
Old 17th March 2019, 23:52   #128  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by WorBry View Post
You said you tested it, but got an error.
yes, using the option in the seek-test script, as CK suggested that. I couldn't run a metric test b/c I didn't have the command syntax.

Quote:
Originally Posted by WorBry View Post
Code:
clip = core.lsmas.LWLibavSource(source=r'{Path}:/video')
thank you for that.

Quote:
Originally Posted by WorBry View Post
Please, no more requests for me to test this and that.
thanks for running a couple of tests and confirming your settings.

Quote:
Originally Posted by WorBry View Post
I don't know why you get inconsistent results on consecutive metric testing. I don't.
as outlined in my other post, it's FFMS2 w/ seekmode=1 (default) that has a problem with keyframes - which will provide random, inconsistent GMSD/SSIM results.

This can be a problem for peeps running GMSD/SSIM tests on files they did not encode themselves and may not be aware of the issues w/ seekmode=1 (default).



So I ran a few more tests to confirm this.

Using ffmpeg Zeranoe 20190312-d227ed5, CK's Fatpack, and FFMS2 from Wolfberry and the crowd_run_1080p50.y4m 1080p50 source (500 frames / 10 secs), I created three encodes.


(1) encode w/o keyint, min-keyint, rc-lookahead parameters:

ffmpeg command
Code:
<ffmpeg path> -i "/path/to/crowd_run_1080p50.y4m" -c:v libx265 -preset slow -crf 28 -pix_fmt yuv420p -x265-params "colorprim=1:transfer=1:colormatrix=1" -r 50/1 "/path/to/encode.mp4"
GMSD/SSIM results - 5 consecutive metrics tests each - source filter varied

Code:
Lsmash:
stop 83.3646771904963 436.07615475501524
stop 83.3646771904963 436.07615475501524 
stop 83.3646771904963 436.07615475501524 
stop 83.36467719049627 436.0761547550152
stop 83.3646771904963 436.07615475501524
Code:
FFMS2 (Wolfberry) - seekmode=0:
stop 83.3646771904963 436.0761547550152 
stop 83.36467719049627 436.07615475501524
stop 83.3646771904963 436.07615475501524 
stop 83.3646771904963 436.07615475501524
stop 83.3646771904963 436.07615475501524
Code:
FFMS2 (Wolfberry) - seekmode=1 (default):
stop 83.36467719049627 436.07615475501524 
stop 83.3646771904963 436.07615475501524 
stop 83.3646771904963 436.0761547550152 
stop 83.3646771904963 436.07615475501524
stop 83.3646771904963 436.0761547550152
--> as you can see, w/o keyframes specified in the encode, seekmode causes no problem.


(2) encode w/ keyint, min-keyint, rc-lookahead:

ffmpeg command
Code:
<ffmpeg path> -i "/path/to/crowd_run_1080p50.y4m" -c:v libx265 -preset slow -crf 28 -pix_fmt yuv420p -x265-params "keyint=100:min-keyint=100:rc-lookahead=100:colorprim=1:transfer=1:colormatrix=1"  -r 50/1 "/path/to/encode.mp4"
Code:
Lsmash:
stop 82.89543157743563 436.641839735243 
stop 82.89543157743563 436.641839735243 
stop 82.89543157743563 436.641839735243
stop 82.89543157743563 436.641839735243
stop 82.89543157743563 436.64183973524297
Code:
FFMS2 (Wolfberry) - seekmode=0:
stop 82.89543157743563 436.641839735243 
stop 82.89543157743563 436.641839735243
stop 82.89543157743563 436.641839735243 
stop 82.89543157743563 436.641839735243
stop 82.89543157743563 436.641839735243
Code:
FFMS2 (Wolfberry) - seekmode=1 (default):
stop 123.93944837285996 227.9372376995322
stop 123.74699115224307 228.86426005799092
stop 82.89543157743563 436.641839735243 
stop 121.66415998563879 239.07092253508384
stop 122.1422683865401 236.40679167570886
--> keyframes were specified in the encode, seekmode=1 (default) provides inconsistent metrics

And as you can see ONE of the 5 metrics (w/ seekmode=1) matches the consistent metrics from Lsmash, so this can be very misleading if you only run one test and it happens to match...


(3) encode w/ keyint, min-keyint but w/o rc-lookahead:

--> wanted to see if it is specifically the rc-lookahead setting that causes the issues w/ seekmode=1

ffmpeg command
Code:
<ffmpeg path> -i "/path/to/crowd_run_1080p50.y4m" -c:v libx265 -preset slow -crf 28 -pix_fmt yuv420p -x265-params "keyint=100:min-keyint=100:colorprim=1:transfer=1:colormatrix=1" -r 50/1 "/path/to/encode.mp4"
Code:
Lsmash:
stop 83.19534706841786 435.99382191599165
stop 83.19534706841786 435.99382191599165
stop 83.19534706841786 435.99382191599165
stop 83.19534706841786 435.99382191599165
stop 83.19534706841786 435.99382191599165
Code:
FFMS2 (Wolfberry) - seekmode=0:
stop 83.19534706841786 435.99382191599165 
stop 83.19534706841786 435.99382191599165 
stop 83.19534706841786 435.99382191599165 
stop 83.19534706841786 435.99382191599165
stop 83.19534706841787 435.99382191599165
Code:
FFMS2 (Wolfberry) - seekmode=1 (default):
stop 108.17858520287257 307.76386120454794 
stop 124.03602892099701 228.48869961962274 
stop 83.19534706841786 435.99382191599165
stop 122.13314288222833 236.93921207380862 
stop 123.97733586255691 229.28604315863697
--> same issues as (2)



Notes:

(a) you need to use seekmode=0 if using FFMS2

(b) the code does provide sometimes higher precision, or very minor variations, see above decimal points highlighted in red

(c) although the same source and only the keyframes parameter as a difference in the three encodes, the consistent metrics via Lsmash or FFMS2 w/ seekmode=0 in all three tests provide different GMSD/SSIM results

83.3646771904963 vs. 82.89543157743563 vs. 83.19534706841786 # GMSD
436.07615475501524 vs. 436.641839735243 vs. 435.99382191599165 # SSIM

although these are minor variations considering one would divide the total metric by num frames, it is a another factor that affects the score, and it shouldn't IMO

Last edited by Iron_Mike; 18th March 2019 at 00:55.
Iron_Mike is offline   Reply With Quote
Old 18th March 2019, 01:15   #129  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
I recreated your tests and came to the same conclusion.

I checked it also like this in excel:
Code:
frame    gmsd                    ssim                     run2  frame                                                              run1 - run2   
255	 0.1665046186678081	 0.8689233699845679	 	255	 0.1665046186678081	 0.8689233699845679		0	        0
256	 0.30940736771850696	 0.8740746768904321	 	256	 0.16450349612365942	 0.8740746768904321		0,144903872	0
257	 0.3089268132506728	 0.8666172357253087	 	257	 0.16778360296358616	 0.8666172357253087		0,14114321	0
...
264	 0.2645300712830965	 0.8644133391203703	 	264	 0.16818244844402383	 0.8644133391203703		0,096347623	0
265	 0.2663959966212291	 0.3675497926311728	 	265	 0.1650834566291641	 0.8708016251929013		0,10131254	-0,503251833
I noticed what the first 255 frames are ok. This is also why the seek test said no seeking issues found, bcs it tests (see bat file) only the first 100 frames. Setting it to 500 showed that ffms2 has also seeking issues with the encoded x265 mp4 file.

So basically only DGDecodeNV was 100% correct, but it cuts off the last 2 frames for some reason, so I had to test with 498 frames. (maybe ffmpeg produces a "broken" stream?)

p.s. I only tested the with latest ffms2 version.
pp.s. I would be good if zopti would sort by frame before saving the log file
pppppp.s
Quote:
stop 83.36467719049627 436.0761547550152
This indicates that it could be a decoding issue. Like 1 pixel off or something. This could be a case for inspector butteraugli.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database

Last edited by ChaosKing; 18th March 2019 at 01:25.
ChaosKing is offline   Reply With Quote
Old 18th March 2019, 01:49   #130  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by ChaosKing View Post
This is also why the seek test said no seeking issues found, bcs it tests (see bat file) only the first 100 frames. Setting it to 500 showed that ffms2 has also seeking issues with the encoded x265 mp4 file.
Not sure I understand you correctly, but using Wolfberry's FFMS2 w/ this command

Code:
py seek-test.py "/path/to/crowd_run_1080p50_encode.mp4" 0 499
option #1 - seekmode=1: showed seeking issues, always 3 frames off
option #5 - seekmode=0: showed no seeking issues


Quote:
Originally Posted by ChaosKing View Post
pp.s. I would be good if zopti would sort by frame before saving the log file
+1

even better: include a header line that states the Zopti and muvsfunc version that were used in the test and specifically state which metrics are listed in the file (GMSD|SSIM|MDSI|etc) and the order in which they are listed...

otherwise nobody knows what is what unless you wrote down the metrics collection passed to the Zopti class... ;-)
Iron_Mike is offline   Reply With Quote
Old 18th March 2019, 01:52   #131  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 447
Quote:
Originally Posted by ChaosKing View Post
This indicates that it could be a decoding issue. Like 1 pixel off or something. This could be a case for inspector butteraugli.
A small difference like that can happen when summing floating point numbers in a different order. Most likely the frame order is a bit different sometimes. That could also be solved by sorting by frame numbers before summing them up.
zorr is offline   Reply With Quote
Old 18th March 2019, 01:59   #132  |  Link
zorr
Registered User
 
Join Date: Mar 2018
Posts: 447
Quote:
Originally Posted by Iron_Mike View Post
even better: include a header line that states the Zopti and muvsfunc version that were used in the test and specifically state which metrics are listed in the file (GMSD|SSIM|MDSI|etc) and the order in which they are listed...

otherwise nobody knows what is what unless you wrote down the metrics collection passed to the Zopti class... ;-)
Zoptilib was written for the Zopti optimizer and the log file was not meant for human consumption. Zopti reads the used metrics from the source so it knows. But I can see that Zoptilib can be useful without Zopti and the header would make it better. I just need to update Zopti so that it doesn't choke on the header.
zorr is offline   Reply With Quote
Old 18th March 2019, 02:19   #133  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by zorr View Post
Zoptilib was written for the Zopti optimizer and the log file was not meant for human consumption. Zopti reads the used metrics from the source so it knows. But I can see that Zoptilib can be useful without Zopti and the header would make it better. I just need to update Zopti so that it doesn't choke on the header.
Iron_Mike is offline   Reply With Quote
Old 18th March 2019, 02:42   #134  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by zorr View Post
Zoptilib was written for the Zopti optimizer and the log file was not meant for human consumption. Zopti reads the used metrics from the source so it knows. But I can see that Zoptilib can be useful without Zopti and the header would make it better. I just need to update Zopti so that it doesn't choke on the header.
btw, if it is possible for you to identify whether the metrics were calculated via YUV or RGB - since once MDSI is in the metrics list, everything goes RGB and the values change - that would also be an important parameter to state in the header...
Iron_Mike is offline   Reply With Quote
Old 18th March 2019, 06:02   #135  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
Quote:
Originally Posted by Iron_Mike View Post
Using ffmpeg Zeranoe 20190312-d227ed5, CK's Fatpack, and FFMS2 from Wolfberry and the crowd_run_1080p50.y4m 1080p50 source (500 frames / 10 secs), I created three encodes.


(1) encode w/o keyint, min-keyint, rc-lookahead parameters:

ffmpeg command
Code:
<ffmpeg path> -i "/path/to/crowd_run_1080p50.y4m" -c:v libx265 -preset slow -crf 28 -pix_fmt yuv420p -x265-params "colorprim=1:transfer=1:colormatrix=1" -r 50/1 "/path/to/encode.mp4"
GMSD/SSIM results - 5 consecutive metrics tests each - source filter varied

Code:
Lsmash:
stop 83.3646771904963 436.07615475501524
stop 83.3646771904963 436.07615475501524 
stop 83.3646771904963 436.07615475501524 
stop 83.36467719049627 436.0761547550152
stop 83.3646771904963 436.07615475501524
Code:
FFMS2 (Wolfberry) - seekmode=0:
stop 83.3646771904963 436.0761547550152 
stop 83.36467719049627 436.07615475501524
stop 83.3646771904963 436.07615475501524 
stop 83.3646771904963 436.07615475501524
stop 83.3646771904963 436.07615475501524
Code:
FFMS2 (Wolfberry) - seekmode=1 (default):
stop 83.36467719049627 436.07615475501524 
stop 83.3646771904963 436.07615475501524 
stop 83.3646771904963 436.0761547550152 
stop 83.3646771904963 436.07615475501524
stop 83.3646771904963 436.0761547550152
Well, i can only reiterate that in my tests I do not get these inconsistencies.

Same source, same x265 CRF28.mp4 command line (i.e. mine). Five consecutive runs of SSIM:GMSD (default, with downsample):

LWLibavSource:

Downsample=True (Default)

stop 475.68958616657005 45.96858528255874
stop 475.68958616657005 45.96858528255874
stop 475.68958616657005 45.96858528255874
stop 475.68958616657005 45.96858528255874
stop 475.68958616657005 45.96858528255874

Downsample=False

stop 436.07619188850276 83.3646771904963
stop 436.07619188850276 83.3646771904963
stop 436.07619188850276 83.3646771904963
stop 436.07619188850276 83.3646771904963
stop 436.07619188850276 83.3646771904963

FFMS2 (Wolfberry) -default

Downsample=True (Default)

stop 475.68958616657005 45.96858528255874
stop 475.68958616657005 45.96858528255874
stop 475.68958616657005 45.96858528255874
stop 475.68958616657005 45.96858528255874
stop 475.68958616657005 45.96858528255874

Downsample=False

stop 436.07619188850276 83.3646771904963
stop 436.07619188850276 83.3646771904963
stop 436.07619188850276 83.3646771904963
stop 436.07619188850276 83.3646771904963
stop 436.07619188850276 83.3646771904963

I assume you tested with Downsample=False. Interesting that your SSIM scores 436.0761547550152(4 ) are also marginally lower than mine 436.07619188850276, whereas the GMSD scores are the same, allowing for those inconsistencies in the last digit.

I should mention also that I ran the series' twice - first, retaining the ffindex and lwi files from the first test for the subsequent tests and then creating a fresh index file for each successive test (deleting the last one, of course). There was absolutely no difference in the results. What did you do in your tests - use the same index files or generate a fresh one for each test ?

And as a double check, I've just now imported the score data from the log files (I retained), into Excel (well, LibreOffice Calc) and, like ChaosKing, run difference calcs on the per frame scores - within each series and between the ffms2 and LWLibavSource series - and I find zero differences.

I didn't run any metric tests with ffms2 seekmode=0, but why would I ? Nor have I run any tests with x265 encoded with custom key frame and rc-lookahead settings, because I have no interest in doing so.

Quote:
Originally Posted by ChaosKing View Post
I would be good if zopti would sort by frame before saving the log file
The frames in my SSIM:GMSD test log files are all in sequence.
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 18th March 2019 at 07:16.
WorBry is offline   Reply With Quote
Old 18th March 2019, 10:17   #136  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
But have you also used the fmpeg Zeranoe 20190312-d227ed5 build?

EDIT
re-muxxing the mp4 to mkv fixes everything.

Quote:
Originally Posted by WorBry View Post

The frames in my SSIM:GMSD test log files are all in sequence.
I have a Ryzen 1700 with 16 (8 real) cores. As far as I remember vsedit makes a get_frameAsync(i) call. So a unordered list can be expected.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database

Last edited by ChaosKing; 18th March 2019 at 10:41.
ChaosKing is offline   Reply With Quote
Old 18th March 2019, 11:32   #137  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
Quote:
Originally Posted by WorBry View Post
Well, i can only reiterate that in my tests I do not get these inconsistencies.
well, did you actually read my post... ? ;-)

it is the x265 KEYFRAMES parameter that creates an encode that will throw off seekmode... all explained in detail above... simply use the provided ffmpeg commands in my post to re-create the problem

Last edited by Iron_Mike; 18th March 2019 at 11:35.
Iron_Mike is offline   Reply With Quote
Old 18th March 2019, 11:35   #138  |  Link
Iron_Mike
Registered User
 
Join Date: Jul 2010
Posts: 132
ran a couple of tests to see how preset behave for x265...

here I tested medium vs. fast... same range as WorBry CRF 0-30, used the same graph layout


Last edited by Iron_Mike; 24th March 2019 at 10:22.
Iron_Mike is offline   Reply With Quote
Old 18th March 2019, 13:39   #139  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,795
https://github.com/theChaosCoder/zoptilib
Zoptilib now writes an ordered log file.

But sadly it still shows small differences at the end sometimes. I will look for a more precise sum() function.

Code:
stop 83.3646771904963 stop 83.36467719049628
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database
ChaosKing is offline   Reply With Quote
Old 18th March 2019, 13:55   #140  |  Link
WorBry
Registered User
 
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
Quote:
Originally Posted by Iron_Mike View Post
well, did you actually read my post... ? ;-)

it is the x265 KEYFRAMES parameter that creates an encode that will throw off seekmode... all explained in detail above... simply use the provided ffmpeg commands in my post to re-create the problem
Yes, I read your post in great detail. And did you read mine ? I only referred to your first set of data with the x265 CRF26 created with the command line I gave you. "Simply use the provided ffmpeg commands in my post to re-create the problem" indeed - you've got a nerve. Yes I did, and I cannot replicate your inconsistent results - I get the exact same scores every time and a marginally higher SSIM score to you.

Read:
Quote:
Originally Posted by WorBry View Post
Well, I can only reiterate that in my tests I do not get these inconsistencies.
Quote:
Originally Posted by WorBry View Post
Interesting that your SSIM scores 436.0761547550152(4 ) are also marginally lower than mine 436.07619188850276, whereas the GMSD scores are the same, allowing for those inconsistencies in the last digit.
Yet you seem intent on asserting that your experience is somehow the norm i.e. inconsistencies are to be expected - when there is quite clearly something wrong.

And you didn't answer my question:

Quote:
Originally Posted by WorBry View Post
I should mention also that I ran the series' twice - first, retaining the ffindex and lwi files from the first test for the subsequent tests and then creating a fresh index file for each successive test (deleting the last one, of course). There was absolutely no difference in the results. What did you do in your tests - use the same index files or generate a fresh one for each test ?
Do you get the same dodgy results retaining the ffindex and lwi index files from the first test in a series vs creating a fresh index file for each ?
__________________
Nostalgia's not what it used to be

Last edited by WorBry; 18th March 2019 at 14:50.
WorBry is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 09:04.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.