Reporting the presence of a color

[LStation]

http://avisynth.nl/index.php/SeeTheDifference I've slightly modified this, mine below, to screen (watch at 5fps) for a certain duplicate behavior (google drive) rarely seen when I oversample the refresh rate (rec 120 at 60hz) using OBS studio to capture (the bottom three, happened in 2 out of 50 titles). It's when there's only a chunk of the next frame but mostly a duplicate, like screen tearing but even smaller. I've simulated it for an example here(google drive)

v1 = AviSource("PATH\scuffed-snip.avi")
v2 = v1.trim(1,0)
sub = v2.subtract(v1).levels(122,1,132,0,255)
return sub

I'd like for each frame a count of middle grey color to be reported to a file, I've tried to use and modify the original

v1 = AviSource("PATH\scuffed-snip.avi").convertTOYV12()
v2 = v1.trim(1,0)
sub = v2.subtract(v1).levels(122,1,132,0,255)
FILEWRITE           = TRUE
filename            = "PATH\Bad.txt"
FileFixed = (FILEWRITE)
    \ ? WriteFileIf(sub, filename, "
    \ YDifferenceFromPrevious() / Max(YDifferenceFromPrevious( selectevery(1, -1)),0.00001) 
    \ > 0.5 && YDifferenceToNext() / Max(YDifferenceToNext(selectevery(1, 1)),0.00001)
    \ > 0.5", "current_frame", append = false) : sub
output = FileFixed
return output
partially yoinked from https://forum.doom9.org/showthread.php?p=1789580#post1789580

but I wasn't having any luck predicting ratios, my included scuffed-snip.avi was shortened for filesize
I've combed the Count... frames containing red thread but I think if I tried to look for outliers in LumaY I'd get false positives(google drive)

my top three examples in 'certain duplicate behavior' are also worst case scenarios I'm aware of, but 3/50 are good odds for a useful tool to shorten my watch time.

edit... audio sync isn't an issue, just screen capturing in bgr24 and -vf mpdecimate=hi=0,setpts=N works fine in ffmpeg with exact duplicates when I know it's not expected behavior

[StainlessS]

sub = v2.subtract(v1).levels(122,1,132,0,255)

Dont think that is quite right, subtract() produces TV levels mid point, ie Round((16 + 235) / 2.0) # Round(125.5) up to 126
So, maybe something like,

OFF = 4
sub = v2.subtract(v1).levels(126-OFF, 1.0 ,126+OFF,0,255, coring = FALSE) # EDIT: Added

Although, sub would have mid point 126, and substrong mid point of 128(I think, EDIT: YES).

EDIT: Actually, substrong = v2.subtract(v1).levels(122,1,132,0,255), does produce mid point of 128 because of the lob-sided 122,132.

[StainlessS]

Mod to SeeTheDifference on Wiki:- http://avisynth.nl/index.php/SeeTheDifference
original source

# If Videos start at different frames
frameadjust = 0

# Videos to compare: (v1 is original, v2 is encoded or whatever)
v1 = AviSource("original.avi",false).trim(frameadjust,0)
v2 = AviSource("encoded.avi",false)
sub = v1.subtract(v2)
substrong = sub.levels(122,1,132,0,255)

return StackVertical(StackHorizontal(v1.subtitle("original"),v2.subtitle("encoded")),
  \ StackHorizontal(sub.subtitle("Difference"),substrong.subtitle("Difference amplified")))

mod

/*
SeeTheDifference on Wiki:- http://avisynth.nl/index.php/SeeTheDifference

Makes differences between two videos really visible.

This can be useful to measure quality loss after encoding
*/

# If Videos start at different frames
frameadjust = 0

# Levels amplification adjust
OFF = 4

# Videos to compare: (v1 is original, v2 is encoded or whatever)
v1 = AviSource("original.avi",false).trim(frameadjust,0) #.ConvertToYV12
v2 = AviSource("encoded.avi",false) #.ConvertToYV12

# If YUV then convert YUV TV.Levels mid point 126 -> 128
# Lighter than mid grey if v2 > v1 (encoded > original)
sub = v2.subtract(v1)
sub = (sub.IsRGB()) ? sub : sub.levels(0-2,1.0,255-2, 0,255, Coring=False).MergeChroma(sub)

# Amp'ed
substrong = sub.levels(128-OFF,1.0,128+OFF, 0,255, Coring=False)

return StackVertical(StackHorizontal(v1.subtitle("original"),v2.subtitle("encoded")),
  \ StackHorizontal(sub.subtitle("Difference"),substrong.subtitle("Difference amplified")))

Moves sub Y mid point for YUV only from 126 -> 128. (mid point for RGB already 128).
Also altered order of subtract from v1-v2 to v2-v1 [Lighter than mid grey if v2 > v1 (encoded > original)]
and amp factor OFF added for substrong.
EDIT: Also added Coring=FALSE for levels.

Maybe a Wiki editor updates it.

EDIT: Changed

sub = (sub.IsRGB()) ? sub : sub.levels(0-2,1.0,255-2, 0,255, Coring=False)

to [hopefully maintain better chroma difference]

sub = (sub.IsRGB()) ? sub : sub.ColorYUV(Levels="TV->PC")

EDIT: Changed again, keep original subtract chroma, only fix luma

sub = (sub.IsRGB()) ? sub : sub.levels(0-2,1.0,255-2, 0,255, Coring=False).MergeChroma(sub)

EDIT: I'll come back after I've been t' pub and maybe look into thread further.

[LStation]

so I toyed with subtract and levels, I thought perhaps being in the YUV colorspace is the problem if avg stats don't change too much after 'seeing the difference'
I added .convertTORGB24().RGBAdjust(analyze = true) to take a look at RGB avg levels instead and with this extreme example(google drive) perhaps there are stats here I could work with instead (avg and std)

edit: looking at the scuffed example (google drive)
perhaps WriteFileIf avgr avgg avgb are > 20 (for this example)
I'd throw my other clips through this to check for false positives before relying on it

[poisondeathray]

Is there a spatial pattern ? e.g. is the duplicate always like that except for the bottom right? If location is the same, you can use a cropped detection clip

[LStation]

digging through my chat history,
I had vd2 tearing screencapture (google drive, image) with vsync on
custom ffmpeg output of obs studio spitting out 3 of them in 12 frames(google drive, video) in different places
beta tested someone else's desktop capture software (pixElv) which was later fixed (google drive, image)

[StainlessS]

For 2nd script in 1st post, I converted to this.

v1  = AviSource(".\scuffed-snip.avi").ConvertToYV12
v2  = v1.trim(1,0)

# If YUV then convert YUV TV.Levels mid point 126 -> 128
# Lighter than mid grey if v2 > v1 (encoded > original)
sub = v2.subtract(v1)
Source = (sub.IsRGB()) ? sub : sub.levels(0-2,1.0,255-2, 0,255, Coring=False).MergeChroma(sub)

METRICS             = TRUE          # TRUE will show Metrics ONLY (i.e., TRUE overrides all other selctions)
FILEWRITE           = TRUE           # TRUE will create a file which contains the frame numbers of all bad frames
filename            = ".\Bad.txt"
BadThreshold        = 0.5

Met_Script = """
    PRat  = YDifferenceFromPrevious() / Max(YDifferenceFromPrevious(selectevery(1, -1)),0.00001)
    NRat  = YDifferenceToNext()       / Max(YDifferenceToNext(selectevery(1, 1)),0.00001)
    BAD   = PRat > """ + String(BadThreshold) + """ &&  NRat > """ + String(BadThreshold) + """
    Subtitle(String(current_frame,"%.0f]") + "\nPRatio = " + String(PRat) + "\nNRatio = " + String(NRat) + "\n" + String(BAD) ,lsp=0,SIZE=48)
"""

WF_Script = """
    PRat  = YDifferenceFromPrevious() / Max(YDifferenceFromPrevious(selectevery(1, -1)),0.00001)
    NRat  = YDifferenceToNext()       / Max(YDifferenceToNext(selectevery(1, 1)),0.00001)
    BAD   = PRat > """ + String(BadThreshold) + """ &&  NRat > """ + String(BadThreshold) + """
    BAD
"""

MetClip = ScriptClip(Source,Met_Script) # Metrics clip
#return metclip

FileWritten = (FILEWRITE)
    \ ? WriteFileIf(Source, filename, WF_Script, "current_frame", append = false)
    \ : source

output = (METRICS) ? MetClip : FileWritten
return output

No real advantage other than metrics display.

I'd like for each frame a count of middle grey color to be reported to a file, I've tried to use and modify the original

I can do that. (next)

[StainlessS]

I'd like for each frame a count of middle grey color to be reported to a file, I've tried to use and modify the original

Roughly what you said you wanted,
Not really sure what else might be required, ie what logic.
NOTE, we used RT_LumaPixelsDifferent() comparing two frames, instead of subtract.
What else may be required ?

Shows metrics and writes file together.

Source  = AviSource(".\scuffed-snip.avi").ConvertToYV12

filename            = ".\Bad.txt".RT_GetFullPathName
THRESH              = 0    # Int, Default 0, Count pixels different by more than THRESH (0 = count pixels that are NOT exactly the same )

RT_FileDelete(filename)

Script = """
    TH   = %d
    Dif2Nxt = RT_LumaPixelsDifferent(Last,Last,Delta2=1,Thresh=TH)  # compare n -> n+1
    Dif2Nxt = Dif2Nxt * 100.0 / 255.0  # Convert to Percent
    SameAsNext = 100.0 - Dif2Nxt
    RT_WriteFile(FileName,"%%d] %%f",current_frame,SameAsNext,Append=True)
    Return Subtitle(String(current_frame,"%%.0f]") + "\\nDif2Nxt = " + String(Dif2Nxt) + "%%\\nSameAsNxt = " + String(SameAsNext) + "%%" ,lsp=0,SIZE=64)
"""

Script = RT_String(Script,THRESH)

Return ScriptClip(Source,Script)



__END__ # NOTHING HAPPENS AFTER THIS LINE

NOTE, Below has x,y,w,h, scan area if required.
Can compare with any frame, ie current to previous, or current next, using delta and delta2.

RT_LumaPixelsDifferent(clip c,clip c2,int "n"=current_frame,int "delta"=0,int "x"=0,int "y"=0,int "w"=0,int "h"=0, \
        int "n2"=current_frame,int "delta2"=0,int "x2"=x,int"y2"=y,bool "interlaced"=false,            \
        int "matrix"=(Width>1100||Height>600?3:2),int "Thresh"=0)

    Compares clip c frame (n+delta) at x,y,w,h, and clip c2 frame (n2+delta2) at x2,y2,w,h, and returns amount of pixels
    whose pixel luma difference is greater than Thresh (RGB converted to Luma-Y using Matrix).
    Matrix 0=Rec601, 1=Rec709, 2=PC601, 3=PC709.
    If Interlaced=True, then skips every other raster line, eg process only y, y+2, y+4 etc.
    Thresh Default 0, returns number of pixels that are not exactly the same.
    Return value is in range 0.0 (meaning none) to 255.0 meaning 100% of pixels.
    v2.0, c and c2 need not be same dimensions but BEWARE, x2 and y2 default to x and y also w and h default to 0
      which is converted to c.width-x and c.height-y, may be best to provide x2, y2, w and h where c2 not same dimensions as c.

Bad.txt [pixels % same as frame n + 1]

0] 15.278152 Vdub2 generates two frame zero's, one on load, and one on play.
0] 15.278152
1] 15.133492
2] 86.835503
3] 27.910049
4] 14.304230
5] 100.000000 # Last frame compares with itself, we can remove this if required.

EDIT: maybe a longer sample if possible. [~250MB]

ClickMe

[LStation]

while the forums were offline, I also picked up on ffmpegs blackframe filter(google drive, image), and the results of StainlessS has appear to trend in a similar direction as my results.
usually >filename.ext works, but in this instance the fix is >>filename.ext to append instead of overwrite the file for each echo

since both filters, subtract().levels() and overlay(mode="Difference"), I started with had focused on middle grey to highlight the difference I thought the next step might be to count that color, or another color that could be used to signal with.

edit2: grabbed RT_Stats plugin for avs+
ran a -benchmark test (google drive, image) against an uncompressed clip (of good footage)
then made scatter plots of their output(google drive, image)

edit3: I tried to replicate but it didn't happen (google drive, image).
context for the start of the graph, pre-fight countdown with no characters/camera moving
I've reran this test another 3 times, 90fps no drops/skips, matching settings I wrote down.. no flaws (good news? oh well, I'd probably have to rollback display driver)

[StainlessS]

usually >filename.ext works, but in this instance the fix is >>filename.ext to append instead of overwrite the file for each echo

WriteFileIf() is a filter and processes entire clip, so append=false does not append to existing log file, ie equivalent to delete existing log prior to any writing.
RT_WriteFile(), is a function, and each and every write is a separate call, so we could use Append=False on first output line instance, and thereafter
Append=True to append to file. But, is easier to just delete existing log with RT_FileDelete() and always use Append=True.

I still have no real idea about what you/we are trying to do.
[I also dont play games and am totally mystified by the fact that people like to record such 'stuff', hence, I have a problem maintaining interest in such a project, but I'll try. ]

Are the problems all similar, similar position, ie exact specific coords ?
Are we just trying to identify the frame index position of such problem, or are trying to fix something, and if so how to fix.

If I had some clue as to where we are headed, I might be able to assist better.

If similar grey blocks occur in exact same coordinate positions, then I could perhaps exactly identify them when they occur.
So, what properties of the problem are constant, so as to better be able to detect them?

EDIT: More than 5 frames sample would also be assistive.

EDIT: Also, are there ever any scene cuts/changes?

EDIT:

edit3: I tried to replicate but it didn't happen (google drive, image).
context for the start of the graph, pre-fight countdown with no characters/camera moving
I've reran this test another 3 times, 90fps no drops/skips, matching settings I wrote down.. no flaws (good news? oh well, I'd probably have to rollback display driver)

Does that mean that the problem is no longer a problem ?

[LStation]

here was the other source I was using, and it's the title I went back to try to repro the problem

custom ffmpeg output of obs studio spitting out 3 of them in 12 frames(google drive, video) in different places

this was to grab the output of pipe from ffmpeg/blackframe, not avisynth, in the screenshot I left a bad note of what I expected to work.

but in this instance the fix is >>filename.ext

I had no idea it was a problem until it was one, I'll continue to screen for it

Does that mean that the problem is no longer a problem ?

I think you nailed down a precise tool for the problem, without having the output looking like a tie-dye job (like what I came up with)
Well done, thank you.

So, what properties of the problem are constant, so as to better be able to detect them?

OBS studio/ffmpeg is crossplatform, and .. avisynth isn't oops... anyway I thought a tool to screen for problems using that/or any capture method was necessary if the goal was clean footage for science-y reasons

If I had some clue as to where we are headed, I might be able to assist better.

I don't watch sports, but 'well executed, practiced skills' as entertainment is one of the reasons people tune in
my eyes roll to the back of my head watching most FPS games though, probably because I don't play those and can't appreciate the nuances

I also dont play games and am totally mystified by the fact that people like to record such 'stuff', hence, I have a problem maintaining interest in such a project

I play and learn with avisynth first, but I think I recreated the stats tool portion with just ffmpeg now

ffmpeg -ss 0.02 -i scuffed-snip.avi -i scuffed-snip.avi -filter_complex "[0:v][1:v]blend=all_mode=difference,colorlevels=rimax=0.002:gimax=0.002:bimax=0.002,blackframe=amount=0" -f null - -nostats 2>&1|findstr pblack|for /f "tokens=4-7 delims=: " %a in ('more') do @echo %b,%d >> bad.txt
I've tweaked imax values from 0.01 to 0.002 ... 0.001 fails, but this should be enough proof(google drive, gif) for one shade difference, different blackframe proof the transformation to yuv444 shouldn't be a problem after the extreme difference is already done.

one frame ahead and current_frame, differenced, colors blown out, evaluate how much the color black (exact duplicate) was present, send report to pipe which goes to findstr which only reports lines with 'pblack' to its own pipe to a for loop that reformats that output and sends that to bad.txt

bad.txt (post-tweak)
0,4
1,4
2,85
3,18
4,3
5,100

a longer clip (2m 32s)
ffmpeg ~90fps processing that (20threads /w big utilization)
avisynth ~21fps (1thread /w big utilization)
plot results, new plot results

most plots were not as odd looking(google drive, image), which is great.

[avinewbie]

Stainless

does your Mod to SeeTheDifference work if the videos are AVC 8bit(orginal) and encoded video HEVC 10 bit?

Stack horizontal in AVSPmod will not work for me with such a pair currently unless they are in different tabs.

Thanks

[StainlessS]

if the videos are AVC 8bit(orginal) and encoded video HEVC 10 bit?

Just convert the original to 10 bit.

Maybe something like this [untested]

/*
SeeTheDifference on Wiki:- http://avisynth.nl/index.php/SeeTheDifference

Makes differences between two videos really visible.

This can be useful to measure quality loss after encoding
*/

# If Videos start at different frames
frameadjust = 0

# Levels amplification adjust
OFF = 4

# Videos to compare: (v1 is original, v2 is encoded or whatever)
v1 = AviSource("original.avi",false).trim(frameadjust,0) #.ConvertToYV12
v2 = AviSource("encoded.avi",false) #.ConvertToYV12

v1=v1.ConvertBits(10)


# If YUV then convert YUV TV.Levels mid point 126 -> 128
# Lighter than mid grey if v2 > v1 (encoded > original)
sub = v2.subtract(v1)
sub = (sub.IsRGB()) ? sub : sub.levels(0-2,1.0,255-2, 0,255, Coring=False).MergeChroma(sub)

# Amp'ed
substrong = sub.levels(128-OFF,1.0,128+OFF, 0,255, Coring=False)

return StackVertical(StackHorizontal(v1.subtitle("original"),v2.subtitle("encoded")),
  \ StackHorizontal(sub.subtitle("Difference"),substrong.subtitle("Difference amplified")))

EDIT: OOps, no that will not work.
I'm just about to go out the door, I'll come baack later.