Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
17th March 2019, 07:06 | #121 | Link | |
Registered User
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
|
Quote:
Why do you say 'or if all plugs u mentioned above all are 3 frames off' when the seek-test reports no issues with LWLibavSource ? Also what is the conclusion when (ffms2) option #5 (seekmode=0) reports 'No seeking issues found' but (ffms2) option #1 reports seek errors - in this case the "3 frames off" thing ?
__________________
Nostalgia's not what it used to be Last edited by WorBry; 17th March 2019 at 07:43. |
|
17th March 2019, 07:15 | #122 | Link | |
Registered User
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
|
Quote:
...or did you add "-x265-params rc-lookahead=100" ?
__________________
Nostalgia's not what it used to be |
|
17th March 2019, 09:12 | #123 | Link | |||
Registered User
Join Date: Jul 2010
Posts: 132
|
Quote:
Alright, so, if the seek script reports no error for using LSmash, but does report errors via ffms2, yet the GMSD/SSIM scores are the exact same when the vid files are read via ffms2 or Lsmash, then one would assume that the seek test script has either flawed logic or at least the way it uses ffms2 in the #1 test is different from the way it is used when the vid files are read in your vpy script... What is the command syntax for reading a vid file via Lsmash/LWLibavSource ? Quote:
Quote:
edit: I edited my original post for clarity, I hope it makes more sense now ;-) https://forum.doom9.org/showthread.p...19#post1869119 ... one encode was exactly your settings - as you can see the GMSD matches but the SSIM only matches on the first 5 decimal points... (that again is odd, considering we used same source, same ffmpeg, same settings) then another encode w/ your settings but in addition setting the keyframes/GOPsize... the resulting file always produces different GMSD/SSIM results, meaning whatever plugs/modules/classes are utilized to obtain these metrics, at least one of them has a problem w/ that file (or the keyframes in that file)... I just have zero idea why it produces always different results - maybe the frame seek returns a random frame, hence the random metric score... btw, I ran a verification test on that encode to check if all keyframes are in the correct position (every 100 frames, hence every 2 secs) - and they are... I doin't think it's a bad encode, it's just triggers bad logic in at least one of the involved plugs/modules/classes... Last edited by Iron_Mike; 17th March 2019 at 09:29. |
|||
17th March 2019, 11:30 | #124 | Link | |
Registered User
Join Date: Dec 2005
Location: Germany
Posts: 1,795
|
Quote:
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth VapourSynth Portable FATPACK || VapourSynth Database |
|
17th March 2019, 13:17 | #125 | Link | |
Registered User
Join Date: Jul 2010
Posts: 132
|
Quote:
so I just re-ran the GMSD/SSIM tests but opened both files w/ ffms and seekmode=0, and the metrics score is the same. so I guess seekmode is not used when these tests are run (since the seek test script indicated that seek mode is 3 frames off, the scores would be different)... Last edited by Iron_Mike; 17th March 2019 at 13:31. |
|
17th March 2019, 13:48 | #126 | Link |
Registered User
Join Date: Dec 2005
Location: Germany
Posts: 1,795
|
Can you upload a small cut of your video file that you're testing?
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth VapourSynth Portable FATPACK || VapourSynth Database |
17th March 2019, 15:21 | #127 | Link | ||||||
Registered User
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
|
Quote:
Quote:
Quote:
Quote:
Code:
clip = core.lsmas.LWLibavSource(source=r'{Path}:/video') Quote:
Please, no more requests for me to test this and that. Over to you ChaosKing. Quote:
I ran the SSIM:GMSD script with the 1080/50p x265 CRF28.mp4 encode imported with ffms2 and LWLibavSource - both plugins being the versions that came with VapourSynth Fatpack Portable. I used the updated Zoptilib.py that provides the per frame metric scores: https://forum.doom9.org/showthread.p...21#post1868821 In both tests, opened up the Previews in VSEditor and 'sent' to Frame #50. Here are the frame grabs: Click on image to enlarge and (+) cursor to enlarge further. Both tests exported exact same frame and per-frame SSIM/GMSD scores. Same thing stepping through the frames. I also examined the x265 CRF26 encode in Elecard StreamEye and TMPGEnc SmartRenderer5 and they display the same frame for frame #50 i.e. frame #50 in the display sequence, but #52 in the stream sequence - it's a B frame.
__________________
Nostalgia's not what it used to be Last edited by WorBry; 17th March 2019 at 19:27. |
||||||
17th March 2019, 23:52 | #128 | Link | |
Registered User
Join Date: Jul 2010
Posts: 132
|
yes, using the option in the seek-test script, as CK suggested that. I couldn't run a metric test b/c I didn't have the command syntax.
thank you for that. thanks for running a couple of tests and confirming your settings. Quote:
This can be a problem for peeps running GMSD/SSIM tests on files they did not encode themselves and may not be aware of the issues w/ seekmode=1 (default). So I ran a few more tests to confirm this. Using ffmpeg Zeranoe 20190312-d227ed5, CK's Fatpack, and FFMS2 from Wolfberry and the crowd_run_1080p50.y4m 1080p50 source (500 frames / 10 secs), I created three encodes. (1) encode w/o keyint, min-keyint, rc-lookahead parameters: ffmpeg command Code:
<ffmpeg path> -i "/path/to/crowd_run_1080p50.y4m" -c:v libx265 -preset slow -crf 28 -pix_fmt yuv420p -x265-params "colorprim=1:transfer=1:colormatrix=1" -r 50/1 "/path/to/encode.mp4" Code:
Lsmash: stop 83.3646771904963 436.07615475501524 stop 83.3646771904963 436.07615475501524 stop 83.3646771904963 436.07615475501524 stop 83.36467719049627 436.0761547550152 stop 83.3646771904963 436.07615475501524 Code:
FFMS2 (Wolfberry) - seekmode=0: stop 83.3646771904963 436.0761547550152 stop 83.36467719049627 436.07615475501524 stop 83.3646771904963 436.07615475501524 stop 83.3646771904963 436.07615475501524 stop 83.3646771904963 436.07615475501524 Code:
FFMS2 (Wolfberry) - seekmode=1 (default): stop 83.36467719049627 436.07615475501524 stop 83.3646771904963 436.07615475501524 stop 83.3646771904963 436.0761547550152 stop 83.3646771904963 436.07615475501524 stop 83.3646771904963 436.0761547550152 (2) encode w/ keyint, min-keyint, rc-lookahead: ffmpeg command Code:
<ffmpeg path> -i "/path/to/crowd_run_1080p50.y4m" -c:v libx265 -preset slow -crf 28 -pix_fmt yuv420p -x265-params "keyint=100:min-keyint=100:rc-lookahead=100:colorprim=1:transfer=1:colormatrix=1" -r 50/1 "/path/to/encode.mp4" Code:
Lsmash: stop 82.89543157743563 436.641839735243 stop 82.89543157743563 436.641839735243 stop 82.89543157743563 436.641839735243 stop 82.89543157743563 436.641839735243 stop 82.89543157743563 436.64183973524297 Code:
FFMS2 (Wolfberry) - seekmode=0: stop 82.89543157743563 436.641839735243 stop 82.89543157743563 436.641839735243 stop 82.89543157743563 436.641839735243 stop 82.89543157743563 436.641839735243 stop 82.89543157743563 436.641839735243 Code:
FFMS2 (Wolfberry) - seekmode=1 (default): stop 123.93944837285996 227.9372376995322 stop 123.74699115224307 228.86426005799092 stop 82.89543157743563 436.641839735243 stop 121.66415998563879 239.07092253508384 stop 122.1422683865401 236.40679167570886 And as you can see ONE of the 5 metrics (w/ seekmode=1) matches the consistent metrics from Lsmash, so this can be very misleading if you only run one test and it happens to match... (3) encode w/ keyint, min-keyint but w/o rc-lookahead: --> wanted to see if it is specifically the rc-lookahead setting that causes the issues w/ seekmode=1 ffmpeg command Code:
<ffmpeg path> -i "/path/to/crowd_run_1080p50.y4m" -c:v libx265 -preset slow -crf 28 -pix_fmt yuv420p -x265-params "keyint=100:min-keyint=100:colorprim=1:transfer=1:colormatrix=1" -r 50/1 "/path/to/encode.mp4" Code:
Lsmash: stop 83.19534706841786 435.99382191599165 stop 83.19534706841786 435.99382191599165 stop 83.19534706841786 435.99382191599165 stop 83.19534706841786 435.99382191599165 stop 83.19534706841786 435.99382191599165 Code:
FFMS2 (Wolfberry) - seekmode=0: stop 83.19534706841786 435.99382191599165 stop 83.19534706841786 435.99382191599165 stop 83.19534706841786 435.99382191599165 stop 83.19534706841786 435.99382191599165 stop 83.19534706841787 435.99382191599165 Code:
FFMS2 (Wolfberry) - seekmode=1 (default): stop 108.17858520287257 307.76386120454794 stop 124.03602892099701 228.48869961962274 stop 83.19534706841786 435.99382191599165 stop 122.13314288222833 236.93921207380862 stop 123.97733586255691 229.28604315863697 Notes: (a) you need to use seekmode=0 if using FFMS2 (b) the code does provide sometimes higher precision, or very minor variations, see above decimal points highlighted in red (c) although the same source and only the keyframes parameter as a difference in the three encodes, the consistent metrics via Lsmash or FFMS2 w/ seekmode=0 in all three tests provide different GMSD/SSIM results 83.3646771904963 vs. 82.89543157743563 vs. 83.19534706841786 # GMSD 436.07615475501524 vs. 436.641839735243 vs. 435.99382191599165 # SSIM although these are minor variations considering one would divide the total metric by num frames, it is a another factor that affects the score, and it shouldn't IMO Last edited by Iron_Mike; 18th March 2019 at 00:55. |
|
18th March 2019, 01:15 | #129 | Link | |
Registered User
Join Date: Dec 2005
Location: Germany
Posts: 1,795
|
I recreated your tests and came to the same conclusion.
I checked it also like this in excel: Code:
frame gmsd ssim run2 frame run1 - run2 255 0.1665046186678081 0.8689233699845679 255 0.1665046186678081 0.8689233699845679 0 0 256 0.30940736771850696 0.8740746768904321 256 0.16450349612365942 0.8740746768904321 0,144903872 0 257 0.3089268132506728 0.8666172357253087 257 0.16778360296358616 0.8666172357253087 0,14114321 0 ... 264 0.2645300712830965 0.8644133391203703 264 0.16818244844402383 0.8644133391203703 0,096347623 0 265 0.2663959966212291 0.3675497926311728 265 0.1650834566291641 0.8708016251929013 0,10131254 -0,503251833 So basically only DGDecodeNV was 100% correct, but it cuts off the last 2 frames for some reason, so I had to test with 498 frames. (maybe ffmpeg produces a "broken" stream?) p.s. I only tested the with latest ffms2 version. pp.s. I would be good if zopti would sort by frame before saving the log file pppppp.s Quote:
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth VapourSynth Portable FATPACK || VapourSynth Database Last edited by ChaosKing; 18th March 2019 at 01:25. |
|
18th March 2019, 01:49 | #130 | Link | ||
Registered User
Join Date: Jul 2010
Posts: 132
|
Quote:
Code:
py seek-test.py "/path/to/crowd_run_1080p50_encode.mp4" 0 499 option #5 - seekmode=0: showed no seeking issues Quote:
even better: include a header line that states the Zopti and muvsfunc version that were used in the test and specifically state which metrics are listed in the file (GMSD|SSIM|MDSI|etc) and the order in which they are listed... otherwise nobody knows what is what unless you wrote down the metrics collection passed to the Zopti class... ;-) |
||
18th March 2019, 01:52 | #131 | Link |
Registered User
Join Date: Mar 2018
Posts: 447
|
A small difference like that can happen when summing floating point numbers in a different order. Most likely the frame order is a bit different sometimes. That could also be solved by sorting by frame numbers before summing them up.
|
18th March 2019, 01:59 | #132 | Link | |
Registered User
Join Date: Mar 2018
Posts: 447
|
Quote:
|
|
18th March 2019, 02:19 | #133 | Link | |
Registered User
Join Date: Jul 2010
Posts: 132
|
Quote:
|
|
18th March 2019, 02:42 | #134 | Link | |
Registered User
Join Date: Jul 2010
Posts: 132
|
Quote:
|
|
18th March 2019, 06:02 | #135 | Link | |
Registered User
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
|
Quote:
Same source, same x265 CRF28.mp4 command line (i.e. mine). Five consecutive runs of SSIM:GMSD (default, with downsample): LWLibavSource: Downsample=True (Default) stop 475.68958616657005 45.96858528255874 stop 475.68958616657005 45.96858528255874 stop 475.68958616657005 45.96858528255874 stop 475.68958616657005 45.96858528255874 stop 475.68958616657005 45.96858528255874 Downsample=False stop 436.07619188850276 83.3646771904963 stop 436.07619188850276 83.3646771904963 stop 436.07619188850276 83.3646771904963 stop 436.07619188850276 83.3646771904963 stop 436.07619188850276 83.3646771904963 FFMS2 (Wolfberry) -default Downsample=True (Default) stop 475.68958616657005 45.96858528255874 stop 475.68958616657005 45.96858528255874 stop 475.68958616657005 45.96858528255874 stop 475.68958616657005 45.96858528255874 stop 475.68958616657005 45.96858528255874 Downsample=False stop 436.07619188850276 83.3646771904963 stop 436.07619188850276 83.3646771904963 stop 436.07619188850276 83.3646771904963 stop 436.07619188850276 83.3646771904963 stop 436.07619188850276 83.3646771904963 I assume you tested with Downsample=False. Interesting that your SSIM scores 436.0761547550152(4 ) are also marginally lower than mine 436.07619188850276, whereas the GMSD scores are the same, allowing for those inconsistencies in the last digit. I should mention also that I ran the series' twice - first, retaining the ffindex and lwi files from the first test for the subsequent tests and then creating a fresh index file for each successive test (deleting the last one, of course). There was absolutely no difference in the results. What did you do in your tests - use the same index files or generate a fresh one for each test ? And as a double check, I've just now imported the score data from the log files (I retained), into Excel (well, LibreOffice Calc) and, like ChaosKing, run difference calcs on the per frame scores - within each series and between the ffms2 and LWLibavSource series - and I find zero differences. I didn't run any metric tests with ffms2 seekmode=0, but why would I ? Nor have I run any tests with x265 encoded with custom key frame and rc-lookahead settings, because I have no interest in doing so. The frames in my SSIM:GMSD test log files are all in sequence.
__________________
Nostalgia's not what it used to be Last edited by WorBry; 18th March 2019 at 07:16. |
|
18th March 2019, 10:17 | #136 | Link |
Registered User
Join Date: Dec 2005
Location: Germany
Posts: 1,795
|
But have you also used the fmpeg Zeranoe 20190312-d227ed5 build?
EDIT re-muxxing the mp4 to mkv fixes everything. I have a Ryzen 1700 with 16 (8 real) cores. As far as I remember vsedit makes a get_frameAsync(i) call. So a unordered list can be expected.
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth VapourSynth Portable FATPACK || VapourSynth Database Last edited by ChaosKing; 18th March 2019 at 10:41. |
18th March 2019, 11:32 | #137 | Link | |
Registered User
Join Date: Jul 2010
Posts: 132
|
Quote:
it is the x265 KEYFRAMES parameter that creates an encode that will throw off seekmode... all explained in detail above... simply use the provided ffmpeg commands in my post to re-create the problem Last edited by Iron_Mike; 18th March 2019 at 11:35. |
|
18th March 2019, 13:39 | #139 | Link |
Registered User
Join Date: Dec 2005
Location: Germany
Posts: 1,795
|
https://github.com/theChaosCoder/zoptilib
Zoptilib now writes an ordered log file. But sadly it still shows small differences at the end sometimes. I will look for a more precise sum() function. Code:
stop 83.3646771904963 stop 83.36467719049628
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth VapourSynth Portable FATPACK || VapourSynth Database |
18th March 2019, 13:55 | #140 | Link | ||||
Registered User
Join Date: Jan 2004
Location: Here, there and everywhere
Posts: 1,197
|
Quote:
Read: Quote:
Quote:
And you didn't answer my question: Quote:
__________________
Nostalgia's not what it used to be Last edited by WorBry; 18th March 2019 at 14:50. |
||||
Thread Tools | Search this Thread |
Display Modes | |
|
|