View Single Post
Old 4th May 2018, 21:46   #8  |  Link
`Orum
Registered User
 
Join Date: Sep 2005
Posts: 178
Quote:
Originally Posted by leoenc View Post
...
I tried with ocldi=1 but getting:
"DupStep: Requested device index out of range" even though I see it listed when running DS_dumpocl():
...
Fixed by using DupStep(oclpi=-4).
Just a note on this, you should always set oclpi to a non-negative value when setting ocldi. While I mention this in the readme, I neglected to mention that it should be non-negative. I'll have an updated readme in the next release.


Specifically, in your case, it's due to the Intel CPU not supporting OpenCL atomics, but your nVidia GPU does. So just using oclpi=-4 (or -3) and not specifying ocldi is probably the best way to handle it (unless you really want it to run on your 780 instead of your 980, in which case I recommend setting both oclpi=0 and ocldi=1). The next release will be using a reduction kernel instead of atomics and should run on Intel CPUs/GPUs without issue.
Quote:
Originally Posted by leoenc View Post
@`Orum,

What is the proper way to get a list of all the duplicated frames?
After running DubStep() and getting the dupstep.dsd file I ran:
perl dsstats.pl -s dupstep.dsd >stats.txt

Not sure how to decipher the data there to know which frames were considered duplicates.
This is a bit complicated, because there's actually several ways to do this, but I'll focus on the simpler method. But before I get into that, I recommend you instead redirect the output to a ".csv" file instead of ".txt", and that you open the CSV file in a spreadsheet application, e.g. MS Excel or LibreOffice Calc. You may also want to dump only metrics (and not the priority cache) by using -m, e.g. "dsstats.pl -ms dupstep.dsd" (note: "-s dupstep.dsd" isn't necessary here as that's the default file if unspecified). Lastly, keep in mind the following assumes two things: you are using the current release, v0.03, and you are using ifmcm=0 (the default in v0.03).

Now, if you have not modified DupStep's parameters since the cache file was created, it's actually relatively simple to figure out. The first column is the frame number of the "earlier" or base frame in the comparison (i.e. the frame with the lower frame number), and the second column is the "delta" or forward distance from this frame that the metric is calculated from. If you add this delta to the first frame number, you'll get the frame number that the frame is compared to. For example (remember right now we're only looking at the first two columns, and for the sake of formatting for this forum I'll be using CSV format):
  • 0,1 - This means frame 0 is being compared to frame 1 (0+1=1). This will always be the first metric of any file, as DupStep always begins by comparing the first frame (frame 0) to the second frame (frame 1)
  • 37,8 - Frame 37 is being compared to frame 45 (37+8=45)
  • x,y - Abstracting this, frame x is being compared to frame (x+y)
So, how does this tell you what frames are being decimated? Well, in v0.03, DupStep only generates metrics as needed, which means it will keep increasing the delta until it exceeds the threshold. For example, consider the following output (again, only looking at the first two columns) from some of the test footage I run through DupStep:
Code:
frame,delta
0,1
1,1
1,2
3,1
3,2
5,1
5,2
7,1
7,2
9,1
9,2
11,1
Here, you'll see the first metric stored in the cache is 0,1, followed by 1,1. Any time the first number (the base frame number) changes, that means that the threshold was exceeded and the frame is not a duplicate. So, in this case, it means frame 1 was not a duplicate of frame 0.

However, look at the next two deltas: 1,1 followed by 1,2. This means that when comparing frame 1 to frame 2 (1+1), frame 2 was detected as a duplicate, because it then went on to calculate metrics between fame 1 and frame 3 (1+2).

Then it jumps to frame 3 as the base frame, because it discovered that frame 3 was not a duplicate of frame 1. So, in short, if you look at all the unique numbers of the base frames, you'll find almost all the frames that are non-duplicates, or in this case: frames 0, 1, 3, 5, 7, 9 and 11. The duplicate frames are almost everything else, or 2, 4, 6, 8, and 10 in this case. (Note, however, the frames returned by DupStep may not actually be those particular frame numbers if you have ptp=true (the default), as it tries to find the "highest quality" frame from a range of duplicates.)

Why almost, and what about frame 12? Well, this method will not tell you if the final frame is a duplicate or not, but it's much simpler as you don't need to do any math regarding the metrics. It does, however, require that you have not modified DupStep's parameters, as it does not delete metrics that are no longer used when you change the parameters, for performance reasons (otherwise they'd have to be recalculated if you reverted your change to the parameters).

If you're asking, "Well, how do I get the duplicates even if I change the parameters without deleting the cache, and how do I find out if the last frame is a duplicate?" the short answer is "You can using the numbers in the columns following the frame/delta columns, but it's complicated." While the math itself is relatively simple, it depends on parameters including your threshold, and if you're using an automatic threshold (i.e. negative) like I recommend, the calculation of the threshold itself is based on numerous other parameters.

In short, I recommend just waiting for a later release, as adding in display of the metrics within the video (the "show" parameter) is one of my high priority items. It probably won't be available next release, as I am focusing on getting SSIM metrics and a reduction kernel for that, but I am planning on them being in v0.05. Also, at some point in the future (probably with v0.05) I'll be adding in the capability to input your DupStep() parameters into dsstats.pl and have it spit out more useful information when you do.

Edit: You can also set show=1 to see which frame DupStep is returning from the footage you're giving it, and this method will work even if you change the parameters. However, if you want to get the range of duplicates, it's best to set ptp=false as well so it will always return the first frame of the range, and then by moving forward/backward one frame in its output, you can calculate which frames are duplicates.
__________________
My filters: DupStep | PointSize

Last edited by `Orum; 6th May 2018 at 19:32.
`Orum is offline   Reply With Quote