Trailer Fingerprint: How? Is it even possible?

[Sparktank]

There's been a trend on youtube lately that explores visual representation of the color pallets in trailers.
https://www.youtube.com/user/TrailerFingerprint/featured

In this context, "fingerprint" isn't used a hash value or anything (like audio fingerprinting).

Description from the first Fingerprint trailer:
The goal of this experiment is to create a visualization of color and shape that is unique to each trailer. From this visualization, insight into the editing style, the color correct, and the audio mix for that unique trailer. The bottom bar represents the average color from each frame with a white audio volume overlay. By comparing trailers in this way we can understand the visual structure of the work.

I've been curious about this for a long time and even tried to message the youtube channel in hopes for a reply but they seem to be robots and just post trailers.
Or they want to protect their idea and not give away its methods.


I would like to know if this can be replicated at home, via AviSynth or other.
I saw the waveform avisynth plugin, but it doesn't generate the type of waveform I'm looking for.
It generates a linear line rather than the typical waveform you see when you Google image search.
(Or maybe I set it wrong in the script?)
It also doesn't support colors or some-such.

Is there anything out there that isn't mainstreamed in the forums? A VirtualDub plugin, an amalgamation of plugins and utilities (SoX, etc) to replicate this effect?

It would be nice to reproduce this effect for any video or even a movie.

Even nicer if end result can be applied to a waveform.
For aesthetic purposes, mostly to generate very large wallpapers.

[poisondeathray]

I would like to know if this can be replicated at home, via AviSynth or other.


The problem you're going to have with using standard types of plugins for this is they are reactive per frame - but you want a "snapshot" of the video "fingerprint" & audio waveform that particular time frame, not for it to change over time . You want display the summary of the past results


Not sure how to do it in realtime, but one way you can reproduce this effect is use something like after effects with a mask reveal .

For audio waveform you can use any audio editor (e.g audacity) and composite/overlay the waveform .

For the video "fingerprint" you can use various plugins or approaches. Trapcode echospace is fairly suitable for this one , which can be configured to make offset "clone" layers . Think of it as taking screenshots (resized, blurred or averaged) then combining them in a horizontal strip offset so they are beside each other . The strip is already there, but the mask "reveals" it as the video is played according to the play position . There is probably a "smarter" way to do this with expressions in AE, instead of auto generating layers.

eg
http://www.mediafire.com/?03cqek7zqxabwur



Not sure how you would do this avisynth , but one of the gurus will probably suggest a way

[Sparktank]

Those are very interesting results.
Equally aesthetic as the trailer fingerprints on youtube.
That is definitely something worth looking into.

Thanks for the sample and info.

[Keiyakusha]

this is absolutely possible with imagemagick+avisynth. maybe avisynth is enough but i'm too lazy to try.
Wall of bad english below, care! ^__^

There is multiple ways to do that, and they may look a bit different but one of them I see like so:
lets assume you want to make 1280x720 video, your source video have 3284 frames and you want fingerprint to change each pixel. means you need to prepare 1280 images.
convert video to rgb24, resize it to something small like 64x64 resolution. Actually you want 1x1 resolution but not sure if avisynth can handle that...
Now you need to make 1280 frames out of 3284. can be decimation or maybe blending, I dunno. Decimation probably better to do before resize.
Then output these frame to individual files in some folder. Using batch-script for imagemagick make it process each image by downacaling it to 1x1px (here i'm sure imagemagick can handle it), then scale it again to something like 1x100. Then take all images and append horizontally into one 1280x100 image (there is an option for that).
Now cou can append that image to the bottom of your source video (before that adjust height to what you want with any resizer). Then using masktools and using animated image with pure black and white colors you can make fingerprint reveal itself over time...
Edit: you can use photoshop instead of imagemagick I guess, if you know how to make automated and batch operations with it.
Edit2: resizing to 1x1 is to simulate average color. maybe there is a lot better ways for that in avisynth and/or imagemagick

Well didn't tried any of this and probably none of this will be done in optimal way but I THINK its quite close.

[poisondeathray]

Those are very interesting results.
Equally aesthetic as the trailer fingerprints on youtube.
That is definitely something worth looking into.

Thanks for the sample and info.



I used a coarser gradations for the "waveform" just for demonstration purposes - it doesn't represent every frame, ie. "wider" time intervals than the youtube example which used finer "gradations" - but you could choose to do it like the youtube version if you wanted to. Also I just simply resized and blurred the "small" frames, I didn't do any average color pallate analysis or anything like that . The plugin pretty much aligns and distributes everything for you




this is absolutely possible with imagemagick+avisynth. maybe avisynth is enough but i'm too lazy to try.
Wall of bad english below, care! ^__^

There is multiple ways to do that, and they may look a bit different but one of them I see like so:
lets assume you want to make 1280x720 video, your source video have 3284 frames and you want fingerprint to change each pixel. means you need to prepare 1280 images.
convert video to rgb24, resize it to something small like 64x64 resolution. Actually you want 1x1 resolution but not sure if avisynth can handle that...
Now you need to make 1280 frames out of 3284. can be decimation or maybe blending, I dunno. Decimation probably better to do before resize.
Then output these frame to individual files in some folder. Using batch-script for imagemagick make it process each image by downacaling it to 1x1px (here i'm sure imagemagick can handle it), then scale it again to something like 1x100. Then take all images and append horizontally into one 1280x100 image (there is an option for that).
Now cou can append that image to the bottom of your source video (before that adjust height to what you want with any resizer). Then using masktools and using animated image with pure black and white colors you can make fingerprint reveal itself over time...

Well didn't tried any of this and probably none of this will be done in optimal way but I THINK its quite close.



Yes - I was trying to figure out how you would do this in avisynth too, but my coding is only at a basic level. I would like some guru to come in so I can learn how it could be done in avisynth . I'm thinking some runtime functions, gscript something like that

I think you can use almost the same approach by generating an image sequence and applying the filters (resized, blurred , averaged, whatever) with ImageWriter() and overlaying it stepwise to generate the "strip" . x=0, x=2, x=4 etc... . The image reader would have to read single static images (not an image sequence), or the previous "summary" will change over time

Somthing like
overlay(base, strip, x=clp.framecount+somevariable)

The audio waveform part is simple, it's just a screenshot from audacity and used as an overlay (but since it was grey and blue, I keyed out the grey and made it greyscale and brighter) , it can easily be done in avisynth

I know there is a smarter way to do this as a template with variables (like clip width, height, framecount, fps) instead of "hardcoding" values, this way you can run the same script on different spec videos (dimension , fps) and it will automatically make adjustments, but it's beyond my knowlege



Then take all images and append horizontally into one 1280x100 image (there is an option for that).


Yes, thanks for this info about imagemagick, I didn't know it could do it



#If you want to append images horizontally, use this command
convert image1.jpg image2.jpg image3.jpg +append result.jpg

#If you want to append them vertically, change the + to a -
convert image1.jpg image2.jpg image3.jpg -append result.jpg

[Keiyakusha]

Yes - I was trying to figure out how you would do this in avisynth too, but my coding is only at a basic level. I would like some guru to come in so I can learn how it could be done in avisynth . I'm thinking some runtime functions, gscript something like that
Wait wait, runtime functions? Well maybe if only then can make it a lot faster...

Here is starting point for you. It is horrible but somewhat works.
It requires latest avisynth possible (2.6 alpha) and its very slow. No 3rdparty plugins needed. Maybe only masktools to make it reveal itself. (not part of the example)

bicubicresize(1024,576).crop(8,8,-8,-8) #not really needed, made to remove garbage from sides and stuff...
calc=FrameRate()*1280/Framecount() #if this wont return right framerate to decimate to 1280 frames, the whole thing may fail. needs proper rounding
Changefps(calc)
Trim(0,-1280) #hack to make sure we have right amount of frames. but maybe good thing to have even with nice rounding...
converttorgb32()
pointresize(1,1)
pointresize(1,64)
WeaveColumns(1280)

Making a function out of it is easy too. You'll have to specify height of the fingerprint and it will do the rest for any input...

Fate/stay night TV opening:
https://dl.dropboxusercontent.com/u/110558786/Seiga/16E9C9D8.png

F/sn UBV theatrical teaser: (lots of credits that's why a lot of black)
https://dl.dropboxusercontent.com/u/110558786/Seiga/14BF4C06.png

[poisondeathray]

^^Nice work Keiyakusha !! :goodpost:

No need for writing image sequences either!

[poisondeathray]

Here is one way to do the strip "reveal" using animate() and a black strip generated by blankclip() . No masktools used, and it's newbie code, but it one way of doing it

eg
http://www.mediafire.com/?rasd1nrt2ouexzx

#note "strip.png" was generated by Keiyakusha's code, you can combine it in one step with a=last instead of imagesource() (I saved it out as a png only for testing purposes)

I guess next is to find way to replicate an audio wavefrom like audacity in avisynth ? But I guess it's not too difficult to use audacity in a separate step and overlay()





##Load main video
main=FFVideoSource("the_dark_knight-tlr2_h480p.mov").ConvertToRGB32()

##resize width to main video's width, since it was different in this example
a=ImageSource("strip.png").converttorgb32
a.BicubicResize(main.width, a.height)
a1=last

##generate black strip the same dimensions as resized strip
b=blankclip(a1)

##join the main video and "strip"
stackvertical(main,a1)
stack=last

Animate(stack,0,main.framecount+1,"myfunc", b,0,main.height, b,main.width,main.height)

Function myfunc(clip c1, clip c2, int x, int y)
{
Overlay(c1, c2 , x , y)
}

[poisondeathray]

Or maybe the starting position for the black strip should be "x=1", because it needs to unhide the 1st frame, frame zero ? And should main.framecount+1 be just main.framecount ?



Animate(stack,0,main.framecount,"myfunc", b,1,main.height, b,main.width,main.height)

[Keiyakusha]

I guess next is to find way to replicate an audio wavefrom like audacity in avisynth ?

Oh I dunno...

Audiograph (http://avisynth.org/warpenterprises/files/audgraph_25_dll_20040318.zip)
Some crazy stuff below, better don't try at home. (｀∀´)
Also it looks ugly, but maybe some tweaking, or taking other plugin as source (there was something newer I'm sure)... But just a concept based on my previous video fingerprint.
Some parts of the waveform became black, no idea why. I'm surprised it shows something at all.


#SomeSourceWithAudio()
converttomono()
Converttoyuy2()
Audiodub(Blankclip(last),last)
normalize()
pointResize(1280,720)
Audiograph(10)
calc=FrameRate()*1280/Framecount()
Changefps(calc)
Trim(0,-1280)
converttorgb32()
bicubicresize(1281,128)
crop(640,0,-640,0)
WeaveColumns(1280)
crop(0,0,0,-64)
Greyscale()
Blur(0.8)

Random music video:
http://dl.dropboxusercontent.com/u/110558786/Seiga/EEEF2725.png

EDIT:
Tweaked. Don't ask me how it works, I have no idea ^^ Still looks nowhere near good...
Maybe something like Histogram(mode="audiolevels") can be of some use, but I got unexpected results with it.
waveform 0.2 plugin by davidhorman gives me exceptions and crashes a lot - couldn't try it.


#SomeSourceWithAudio()
converttomono()
Converttoyuy2()
Audiodub(Blankclip(last),last)
normalize()
pointResize(1280,720)
Audiograph(1000)
calc=FrameRate()*1280/Framecount()
Changefps(calc)
Trim(0,-1280)
converttorgb32()
Crop(640,262,-639,-360)
WeaveColumns(1280)
Pointresize(1280,64)
Greyscale()
blur(0.8)

F/sn UBV theatrical teaser:
http://dl.dropboxusercontent.com/u/110558786/Seiga/DCB3E14C.png

[Sparktank]

:goodpost::thanks:

My goodness! Such productivity! :D
Thank you everyone for the input!

Really loving all the samples.
Piecing together a lot right now.

Fate/stay night TV opening:
https://dl.dropboxusercontent.com/u/110558786/Seiga/16E9C9D8.png

F/sn UBV theatrical teaser: (lots of credits that's why a lot of black)
https://dl.dropboxusercontent.com/u/110558786/Seiga/14BF4C06.png

Those are extremely beautiful and rich in color! The exact sort of template I'm looking for to bring the waveforms to life.

Here is one way to do the strip "reveal" using animate() and a black strip generated by blankclip() . No masktools used, and it's newbie code, but it one way of doing it

eg
http://www.mediafire.com/?rasd1nrt2ouexzx

And this was very exciting to watch!
It was like watching the trailer for the first time (again)!

Both work flows combined can produce very interesting products.

Random music video:
http://dl.dropboxusercontent.com/u/110558786/Seiga/EEEF2725.png

F/sn UBV theatrical teaser:
http://dl.dropboxusercontent.com/u/110558786/Seiga/DCB3E14C.png

Two different audio representations, I like.
Once I get the color extraction going, I'll be able to try and apply it to various forms of audio graphs.

I'm thinking, with this Keiyakusha's script to nab the color schemes, I can set the height to much larger than I actually need.
Once I got that and a few various audio waveforms pumped out (in different styles), I could use Photoshop (or similar) to do some fancy editing to impose the colored columns over the blank waveforms.
And then use that in the video script for live feedback of the animated reveal.
And also make really fancy, desktop-sized wallpapers for just the waveforms.

You guys are terrific! I'm off to play.
Looking forward to any updates or further suggestions, if any.

[wonkey_monkey]

bicubicresize(1024,576).crop(8,8,-8,-8) #not really needed, made to remove garbage from sides and stuff...
calc=FrameRate()*1280/Framecount() #if this wont return right framerate to decimate to 1280 frames, the whole thing may fail. needs proper rounding
Changefps(calc)
Trim(0,-1280) #hack to make sure we have right amount of frames. but maybe good thing to have even with nice rounding...
converttorgb32()
pointresize(1,1)
pointresize(1,64)
WeaveColumns(1280)


Won't using pointresize result in only one pixel from the original image (or the 1024x576 resize) being considered? (e.g., all those solid blocks of colour in the examples above)

Also rather than decimating frames, why not do


weavecolumns(framecount())
bicubicresize(1280,64)


instead?

DAvid

[Keiyakusha]

Won't using pointresize result in only one pixel from the original image (or the 1024x576 resize) being considered? (e.g., all those solid blocks of colour in the examples above)
I don't know. Maybe, but that's the only algorithm in avisynth that supports 1x1 resolution. This is just a hack. As I said if there are real ways to get average they should be used instead.
Edit: maybe something like bilinearresize(2,2).pointresize(1,1) will do better.


Also rather than decimating frames, why not do

weavecolumns(framecount())
bicubicresize(1280,64)


Because I didn't tried this way and scene changes will be smoothed by scaling, which I don't like. Also this may result very huge horizontal resolution of the image.

Edit:
ok, this is pretty close to average in photoshop:

#rgb input
bilinearResize(2,2)
blur(1.58).blur(1.58).blur(1.58).blur(1.58).blur(1.58) #this needs some better alternative to achieve correct result ^__^
pointresize(1,1)
pointresize(320,200)
Source -> photoshop -> avisynth
http://dl.dropboxusercontent.com/u/110558786/Seiga/Average/source.png http://dl.dropboxusercontent.com/u/110558786/Seiga/Average/photoshop_average.png http://dl.dropboxusercontent.com/u/110558786/Seiga/Average/avisynth_average.png

Edit2:
#rgb input
bilinearresize(2,2)
a=Crop(0,0,-1,-1)
b=Crop(1,0,0,-1)
c=Crop(0,1,-1,0)
d=Crop(1,1,0,0)
average(a,0.25,b,0.25,c,0.25,d,0.25)
pointresize(640,360)
Or like that. Average (http://wilbertdijkhof.com/mg262/Average_v11.zip). Maybe some kind of built-in overlay filter will work. Not sure.

So this:
http://dl.dropboxusercontent.com/u/110558786/Seiga/14BF4C06.png
Will turn into this:
http://dl.dropboxusercontent.com/u/110558786/Seiga/1BF402ED.png
weavecolumns(framecount()).bicubicresize(1280,64) #As I expected, it just smears everything. But the difference is small really. May not work for sources longer than few minutes cause horizontal resolution will be too huge. On a half-hour 720p file after 5 minutes waiting I still didn't got the result, while my 1st approach finished in 1.5 minutes.
http://dl.dropboxusercontent.com/u/110558786/Seiga/B8C9315B.png

Edit3:
This average is so close that visually I can't tell the difference from photoshop. No plugins required. Personally I think this is good enough. We'll lose accurate colors after converting to YV12 anyway.
#rgb input
bicubicresize(4,4)
blur(0.65)
bilinearresize(2,2)
a=Crop(0,0,-1,-1)
b=Crop(1,0,0,-1)
c=Crop(0,1,-1,0)
d=Crop(1,1,0,0)
x=Overlay(a,b,mode="blend",opacity=0.5)
y=Overlay(c,d,mode="blend",opacity=0.5)
Overlay(x,y,mode="blend",opacity=0.5)
pointresize(640,360) #or whatever resolution you need

[Keiyakusha]

So here is a function that generates video fingerprint. Latest avisynth 2.6 alpha required. No 3rd party plugins needed. Audio not included as I'm not satisfied with it.
It includes reveal approach by poisondeathray.
It calculates fingerprint and attaches it at the bottom of the video.
You need to specify one parameter "fph" which stands for fingerprint height. Better specify mod16 values as there is no any kind of checks, rounding or whatever.
May fail badly if source clip have wrong resolution, if fph parameter is wrong, or whatever. Tested only on one yv12 source that had 1920x1080 resolution.
It is very-very slow at start. But once fingerprint is calculated - everything should be fine.

Example: fingerprint(fph=128)
function fingerprint(clip source, int "fph"){
w=width(source)
v=crop(source,8,8,-8,-8)
v=bicubicresize(v,640,360)
calc=FrameRate(v)*w/Framecount(v)
v=Changefps(v,calc,linear=false).Trim(0,-w)
v=converttorgb32(v).bicubicresize(4,4).blur(0.65).bilinearresize(2,2)
a=Crop(v,0,0,-1,-1)
b=Crop(v,1,0,0,-1)
c=Crop(v,0,1,-1,0)
d=Crop(v,1,1,0,0)
x=Overlay(a,b,mode="blend",opacity=0.5)
y=Overlay(c,d,mode="blend",opacity=0.5)
v=Overlay(x,y,mode="blend",opacity=0.5)
v=pointresize(v,1,fph)
v=WeaveColumns(v,w).Converttoyv12()
bg=blankclip(width=w,height=fph)
v=StackVertical(source,v)
Function ovl(clip c1, clip c2, int xx, int yy) { Overlay(c1, c2 , xx , yy) }
v=Animate(v,0,source.framecount+1,"ovl", bg,0,source.height, bg,source.width,source.height)
return v
}

[Asmodian]

Nice tricks to get a real "average" of the entire frame into one pixel. :)

[Sparktank]

We need more emotes with more emotions. ; ;

http://i.imgur.com/qL2iPWH.png

Wonderful!
The combined scripts in an .avsi function worked beautifully.

Now to put it to more experimental uses.

Thank you guys so much! XD

EDIT:
Opened in VDub (yeh, compiling the fph took some time ^__^ so it was unresponsive for a short while).
Saved the output frame to the clipboard.
Used XnView to import the image from the clipboard.
Saved it as PNG with maximum compression and lost the faint hints of red for the "S" logo part of the trailer.
Maybe I should used BMP instead, no?
Or PNG at a much lower compression level than the maximum (9).

[Keiyakusha]

I figured out it may be possible to make real waveform, but for that I need to figure out how to scale image without preserving center (if that's how it is called).
I mean i need do downscale the image in a way that upper pixels affected less than lower ones.
But don't scratch your head over that, this is just a thought it may not help me at all.

[Sparktank]

i need do downscale the image in a way that upper pixels affected less than lower ones.

I remember with waveform 0.2 plugin by davidhorman, I put the waveform underneath the video with the video scaled properly to accomodate the waveform.

http://sdrv.ms/166BB0w
https://jtyaha.blu.livefilestore.com/y2pwtqez7manSERr5PAFRiNHJjgO1GuGTnqB19kscvDeRg8zcOrPgsNbTDDlHFq6w193cdJ66to1aWZOpEZ8nSM-goidVGm4TyCghMlHS1U56U/lulz.png?psid=1

Like that?

[Keiyakusha]

I remember with waveform 0.2 plugin by davidhorman, I put the waveform underneath the video with the video scaled properly to accomodate the waveform.

Like that?

Oh, no unfortunately. I can generate the waveform, but it doesn't looks good. To make it look like the one audio editors show you, I need this unusual way of scaling... There was some ways to do that but I completely forgot everything.

[poisondeathray]

EDIT:
Opened in VDub (yeh, compiling the fph took some time ^__^ so it was unresponsive for a short while).
Saved the output frame to the clipboard.
Used XnView to import the image from the clipboard.
Saved it as PNG with maximum compression and lost the faint hints of red for the "S" logo part of the trailer.
Maybe I should used BMP instead, no?
Or PNG at a much lower compression level than the maximum (9).

PNG is lossless ; the loss of red is probably from colorspace conversions (YV12<=>RGB), lossless from rounding and chroma subsampling


I figured out it may be possible to make real waveform, but for that I need to figure out how to scale image without preserving center (if that's how it is called).
I mean i need do downscale the image in a way that upper pixels affected less than lower ones.

Can you explain what you have in mind ? Some non linear scaling ?

[Keiyakusha]

Can you explain what you have in mind ? Some non linear scaling ?

Something like this （＾_＾）

#yv12 source
Spline36resize(1280,720)
w=width()
Audiodub(blankclip(last),last)
Converttomono()
Normalize()
pointresize(1280,720)
Converttoyv24()
Histogram(mode="audiolevels")
Converttorgb32(chromaresample="point")
calc=FrameRate()*1280/Framecount()
Changefps(calc,linear=false).Trim(0,-w)
Crop(100,0,-1179,-360)
WeaveColumns(1280)
Greyscale()
Levels(0, 1.000, 1, 0, 255, coring=false)
aa=Crop(0,0,0,-height*3/4)
bb=Crop(0,height/4,0,-height/2)
cc=Crop(0,height/2,0,-height/4)
dd=Crop(0,height*3/4,0,0)
aa=pointresize(aa,width(aa), height(aa))
bb=pointresize(bb,width(bb), height(bb)/6)
cc=pointresize(cc,width(cc), height(cc)/9)
dd=pointresize(dd,width(dd), height(dd)/12)
Stackvertical(aa,bb,cc,dd)
blur(1)

F/sn UBV theatrical teaser
http://dl.dropboxusercontent.com/u/110558786/Seiga/E356B3FC.png

Edit: no blur
http://dl.dropboxusercontent.com/u/110558786/Seiga/D080F46E.png

[poisondeathray]

That looks improved :)

No blur looks better to me; or maybe add a configurable parameter

You can always add blur after the overlay/composite ; I think it's better to have sharp edges since the same image can also be used as a luma matte (or mask in overlay() )

[smok3]

my (debian) ffmpeg / imagemagick try (1 strip = 1 frame)
#!/bin/bash

# produces a color-stripe (color fingerprint) png of a video file (1 pixel = 1 frame)

while [ $# -gt 0 ]; do

# make temporary folder and trap to clean up
TMPDIR=`mktemp -d`
trap "rm -rf $TMPDIR" EXIT

# expand path, so this can be used from cli as well (on relative paths)
# file=$(readlink -f "$1")
file="$1"

ffmpeg -i "$file" -sws_flags neighbor -vf scale=2:2 -f image2 "$TMPDIR/out-%05d.png"
convert +append "$TMPDIR/out-*.png" -resize 1x1\! -resize 1x128\! "$file.png"

rm -rf $TMPDIR

shift
done

(didn't bother with audio and could not figure out how to pipe ffmpeg to convert, improvements welcome)

examples:
http://shrani.si/f/16/OU/3fUvCjIb/shakira-loca.png
http://shrani.si/f/2Z/Mh/2eV7odGN/psy.png

[Keiyakusha]

smok3
Yeah, I mentioned batch-scripted imagemagick before. But we also need to make that result match to the given video horizontally, attach to the bottom and make it reveal itself over time like a slider. And this is a bit more complicated...


ok, back to the audio. New update. Not sure If I can make adjustable height as instead of proper nonlinear scaling I just chained few usual resamplers, and simulating this "feather"...
some bicubicresize probably can be replaced to pointresize with no harm but with speedup... not sure.
Edit: or actually this is super simple? just need to set any kind of height in last bicubicresize call? Hmm...
#1280x720 yv12 input with audio. nothing else tested.
w=width()
AudioDub(BlankClip(last,pixel_type="YV24",width=1280,height=720),last)
Converttomono().Normalize()
Histogram(mode="audiolevels").Converttorgb32()
calc=FrameRate()*w/Framecount()
Changefps(calc,linear=false).Trim(0,-w)
Crop(100,0,-1179,-0188)
Greyscale().Levels(0,1.000,64,0,255,coring=false)
WeaveColumns(w).Greyscale()
e=Crop(0,0,0,-height*3/4)
f=Crop(0,height/4,0,-height/2)
g=Crop(0,height/2,0,-height/4)
h=Crop(0,height*3/4,0,0)
e=bicubicresize(e,width(e),height(e)/2)
f=bicubicresize(f,width(f),height(f)/6)
g=bicubicresize(g,width(g),height(g)/9)
h=bicubicresize(h,width(h),height(h)/12)
Stackvertical(e,f,g,h)
bicubicresize(width,48)
bicubicresize(width,6).addborders(0,1,0,0)
bicubicresize(width,48)
F/sn UBV theatrical teaser. This is constant music though, trailer for american movie probably will look more "uneven" like that thing on youtube. Or maybe not. Also I wonder how ugly it will look when overlayed...
http://dl.dropboxusercontent.com/u/110558786/Seiga/5FB8196D.png

[Keiyakusha]

ok, here is function for video AND audio combined. All standard resolutions as input should work.
Probably needs some tweaking though, to make it look more like that thing on YT.
I wanted waveform to be around 30% of the video fingerprint height. Not sure if it is right or I failed basic math, or maybe I did some other mistake.
function fingerprint(clip source, int "fph"){
w=width(source)
v=crop(source,8,8,-8,-8)
v=bicubicresize(v,640,360)
calc=FrameRate(v)*w/Framecount(v)
v=Changefps(v,calc,linear=false).Trim(0,-w)
v=converttorgb32(v).bicubicresize(4,4).blur(0.65).bilinearresize(2,2)
a=Crop(v,0,0,-1,-1)
b=Crop(v,1,0,0,-1)
c=Crop(v,0,1,-1,0)
d=Crop(v,1,1,0,0)
x=Overlay(a,b,mode="blend",opacity=0.5)
y=Overlay(c,d,mode="blend",opacity=0.5)
v=Overlay(x,y,mode="blend",opacity=0.5)
v=pointresize(v,1,fph)
v=WeaveColumns(v,w).Converttoyv12()
wf=AudioDub(BlankClip(source,pixel_type="YV24",width=1280,height=720),source)
wf=Converttomono(wf).Normalize()
wf=Histogram(wf,mode="audiolevels").Converttorgb32()
wf=Changefps(wf,calc,linear=false).Trim(0,-w)
wf=Crop(wf,100,0,-1179,-0188)
wf=Greyscale(wf).Levels(0,1.000,64,0,255,coring=false)
wf=WeaveColumns(wf,w)
e=Crop(wf,0,0,0,-height(wf)*3/4)
f=Crop(wf,0,height(wf)/4,0,-height(wf)/2)
g=Crop(wf,0,height(wf)/2,0,-height(wf)/4)
h=Crop(wf,0,height(wf)*3/4,0,0)
e=bicubicresize(e,width(e),height(e)/2)
f=bicubicresize(f,width(f),height(f)/6)
g=bicubicresize(g,width(g),height(g)/9)
h=bicubicresize(h,width(h),height(h)/12)
wf=Stackvertical(e,f,g,h)
wf=bicubicresize(wf,w,48)
wf=bicubicresize(wf,w,6).addborders(0,1,0,0)
wf=bicubicresize(wf,w,fph*30/100).addborders(0,fph-fph*30/100,0,0)
v=Overlay(v,wf,mode="Add",opacity=0.333)
bg=blankclip(width=w,height=fph)
v=StackVertical(source,v)
Function ovl(clip c1, clip c2, int xx, int yy) { Overlay(c1, c2 , xx , yy) }
v=Animate(v,0,source.framecount+1,"ovl", bg,0,source.height, bg,source.width,source.height)
return v
}
fingerprint(fph=128)

Looks like this. (http://dl.dropboxusercontent.com/u/110558786/Seiga/199B5F15.png)

[smok3]

Yeah, I mentioned batch-scripted imagemagick before. But we also need to make that result match to the given video horizontally, attach to the bottom and make it reveal itself over time like a slider. And this is a bit more complicated...
Yeah, ill skip that part, but if you have a (working) ffmpeg>convert piping solution don't be shy to post.

[wonkey_monkey]

From the Man of Steel trailer:

http://img89.imageshack.us/img89/1165/mosfp.jpg

More info and a plugin soon.

[wonkey_monkey]

Whoops, never got around to providing this.

First you'll need the plugin from here (http://horman.net/fingerprint.zip).

Then you can use this function to apply it (probably bad to give the function the same name as the filter, but whatever):

function fingerprint(clip a, int h) {
rgb=a.converttorgb32
resized=rgb.bicubicresize(a.width,h)
fingerprinted=resized.fingerprint
stack=stackvertical(rgb,fingerprinted.bicubicresize(rgb.width,fingerprinted.height))
return stack
}

You'll get gaps if you seek around in the video without playing it in sequence first, but that also means it doesn't have to preload the clip.

It forces the clip to RGB32 with the default matrix (rec601) so you'll need to bear that in mind if you want to convert it back to the original format.

David

[Sparktank]

Thanks for the plugin, davidhorman.
Interesting take on the fingerprint.

---
I've found a Windows program that can generate a favorable waveform, but it's only a matter of finding a decent way to normalize the audio.
It requires a physical WAV/MP3 file to work with and produces a PNG.

Depending what approach you take to normalize or level out the audio without it peaking, you'll get very different waveforms generated.
You can set the WxH dimensions for the PNG but if the audio peaks, the waveform will disappear off the picture.

Maybe use SoX to apply compand to each channel and then downmix to mono and normalize?
I want the lowest point of the wave form to not be at the center of the X-axis. About 1/3 to 1/2 would be nice to make it appear larger as a waveform.
And without it peaking beyond the given dimensions of the PNG frame.

I'll find the link to the Windows app. I believe there's a python extension for it or a PHP extension.
Also will post several examples of what I'm currently doing with the generated waveforms and video fingerprints.

ALSO, for full feature length films (much longer than trailers!), I got some error about nonlinear access?
I can't remember, but I just added it to the Keiyakusha/poisondeathray fingerprint script and it was able to handle full length films and allow seeking at any point in VirtualDub (quite a bit of time to load the whole project lol).

Updates and progress coming soon once I get things sorted.

[StainlessS]

David,

Using this


Avisource("D:\avs\test.avi")#.Trim(0,-2*1280)

function fingerprint(clip a, int h) {
rgb=a.converttorgb32
resized=rgb.bicubicresize(a.width,h)
fingerprinted=resized.fingerprint
stack=stackvertical(rgb,fingerprinted.bicubicresize(rgb.width,fingerprinted.height))
return stack
}

Last.fingerprint(128)



I get, "Iinvalid arguments to function fingerprint".


changing to

Avisource("D:\avs\test.avi")#.Trim(0,-2*1280)

function fingerprintB(clip a, int h) {
rgb=a.converttorgb32
resized=rgb.bicubicresize(a.width,h)
fingerprinted=resized.fingerprint
stack=stackvertical(rgb,fingerprinted.bicubicresize(rgb.width,fingerprinted.height))
return stack
}

Last.fingerprintB(128)



"There is no function named fingerprint".

How should it be called (and yes, probably a bad idea to have same name function as plugin, as user function name overrides plugin_name/builtin_Avisynth_name).

@Keiyakusha, you've made "fph" an optional argument, without a default.

EDIT: I'm also seeing a date of 11 May 2013 for the Fingerprint.dll ? EDIT: Oops, thats the day you made first post on it.

EDIT: I seem to recall a plugin called WarpResize.

[wonkey_monkey]

I've only recently moved over to Avisynth 2.6. Do you have 2.5.8? Maybe I did something wrong in compiling it...

David

[StainlessS]

Nope, I'm using 2.6a4 almost exclusively.

I know you believe you are a bit messy and dont like to reveal your source, but if you would like me to take a peek
I'll be happy to. PM me a link if you like. I've done a number of 2.6 plugs and they seem to work OK.

I had myself started out doing a Fingerprint plugin, but discarded it what I saw that you had posted one already
(had only just spent about 1/2 hour on it so was far from doing anything).

EDIT: Nice job on the fingerprint strip you posted, looks great. :)

[Sparktank]

David,

I'm using the same AVS StainlessS is using and can't get your Fingerprint to run.
It crashes instantly in VirtualDub with something about a "kernal" crash.
Hopefully you and StainlessS can find something. It would be fun to use both forms of the fingerprint.

[Sparktank]

So using an earlier Fingerprint as a function, I took a fingerprint of The Hobbit (2D version) in 1080p Blu-Ray resolutions.
I used one that doesn't work with audio just so I can get a huge color palette of the total movie.

http://i.imgur.com/kOT7XPv.png

For the waveform, I converted the DTS-HD MA (8ch) to separate WAVs with eac3to and used NicAudio to load the first 6 channels and normalize each before merging and converting to mono to run through "waveform generator". I saved the mono wav using BeHappy.

I found a Windows program that can generate a waveform in PNG formats.
It can also be used as a JavaScript or PHP function too.
It's initially a CLI app. I can't remember where I found a compiled binary package.
Here's the Github for the source code.
https://github.com/superjoe30/waveform

waveform generator can read mono wav files in the valid integer bit depths as well as IEE float (32).
It will tell you if there's a header chunk not found and will still generate waveforms saved in PNG with transparency.
Haven't tested with multichannel wavs though.

Here's the different results you can get depending on the settings you use.
There's three to toy with: inner, outer and background. The inner/outer refer to the inner part of the waveform or the outer tips.
Works in gradient color (rrggbbaa). ("a" for alpha transparency?)
To get full transparency on both the outer and inner part of the waveform on a black background, use:
--color-bg 000000ff --color-center 00000000 --color-outer 00000000

It produced these for me...
Inner: black, outer: red, background: transparent
http://i.imgur.com/M24ViZp.png

Inner: black, outer: red, background: black
http://i.imgur.com/5kyShHp.png

Inner: red, outer: red, background: black
http://i.imgur.com/XOkSOe6.png

Using transparency on both the inner/outer on black bg, I mixed it with the fingerprint in a photo editing program to get...

(open image in new tab for better viewing)
http://i.imgur.com/AhSVEFN.png
I had to brighten it a little to show more of the lower volumes.

I then split it half and saved the top/bottom halves to new files and took screenshots of the movie to make different wallpapers.
I also reduced the height of the pictures. I reduced the pictures of the colored waveforms to fit in the black bars of the screenshots from the movie.

Wallpaper 1 in 1920x1080 (open image in new tab for better viewing)
http://i.imgur.com/0o3yaFn.jpg

Wallpaper 2 in 1920x1080 (open image in new tab for better viewing)
http://i.imgur.com/82kgstG.jpg

Wonderful use!
The results are much better than I had hoped!
Now to do different resolutions for different monitors.
my laptop and desktops are completely different sizes.
That'll be fun. :)

It's interesting, I merged all layers to a single layer and then applied the brightness tweaks before splitting them in half.
After splitting them in half and applying to the screenshots, it looks like the top halves are darker than they were previously. :eek:


The only left to do is convert the whole movie with the animated function to mimic the Fingerprint trailers. :D
For the whole movie conversion, it won't be split in half like the wallpapers.
The wallpapers are separate parts of the same project.
Though for the movie conversion, I think I'll reduce it to 720p resolutions.

Thank you so much for all the help everyone!
David, hoping to see an update to your Fingerprint function for 2.6.

[wonkey_monkey]

It crashes instantly in VirtualDub with something about a "kernal" crash.


How long in frames is your clip?

[Sparktank]

How long in frames is your clip?

@23.976, 244010 frames.
2:49:37.242 (H:MM:SS.sss) total run length of The Hobbit: An Unexpected Journey.

[wonkey_monkey]

I was lazy and left resizing of the clip to AviSynth rather than doing it in-plugin. So it's probably AviSynth complaining about a) being given a 244010-pixel wide clip or b) being asked to resize said clip.

[IanB]

Probably running out of memory.

YV12 244010 x 480 clip is 167Mb per frame!

Resize works with that, but you cannot do much else.

[Sparktank]

Oh, I misread what you meant.
I thought you were asking the total runtime of the movie in frames.

The frame width is 1920x800, cropped.

Here's what the basic script looks like before adding any of the fingerprint functions.

SetMemoryMax(1280)
SetMTMode(3,4)
DirectShowSource("L:\1080p (Remux)\The Hobbit.mkv").KillAudio()
SetMTMode(2)
Crop(0,140,0,-140)
SetMTMode(1)
Distributor()
Return(last)

[IanB]

Where are people getting the crazy idea that a big SetMemoryMax is ever a good idea.

I have searched the forum numerous times looking for any fud and misinformational posts and I do not find any, all I find are the occasional post advising to turn it down. and lots of posts with people using insanely high values and having problems.

SetMemoryMax just sets the size of the Avisynth frame cache!

It does not control any other memory usage!

The frame cache only needs to be just large enough to hold the temporal requirements of the script. If a script has no temporal requirements, i.e. a frame that is required more than once in some part of the script then the cache is completely useless and may prevent some other memory usage from being able to malloc the memory that it needs to work.

A 1920x1080 YV12 video frame is ~3meg, as RGB32 it is 8meg, you can hold 64 such frames in 512meg. YV12 720x480 frames are only 0.5meg you can hold 1024 in 512 meg.

It takes some resources to manage the frame cache so doing a SetMemoryMax(2048) for a SD video and having 4000 odd buffers in play is just stupid and will actually slow the script processing down a little bit.

Advise :- SetMemoryMax, Turn it down!

[Sparktank]

Thank, I was curious about the whole SMM size.
How much to turn it down was always on my mind.

But that explanation helps understand a lot more. 512 sounds more efficient.
Though I wonder if there's a tool or function to display how much memory is being used for each thing to help further understand the SMM.
Grasping the whole scope of the math behind it isn't that easy for me.

[Reel.Deel]

Though I wonder if there's a tool or function to display how much memory is being used for each thing to help further understand the SMM.

When setting SetMemoryMax, I use AVSMeter (http://forum.doom9.org/showthread.php?t=165528) to tweak for optimal performance.

[Sparktank]

When setting SetMemoryMax, I use AVSMeter (http://forum.doom9.org/showthread.php?t=165528) to tweak for optimal performance.

Thank you! I keep forgetting about that program.

[wonkey_monkey]

Oh, I misread what you meant.
I thought you were asking the total runtime of the movie in frames.

I was. The immediate return result of the fingerprint plugin is a clip which is [number of frames] pixels wide. This then gets resized with the internal AviSynth resizer.

[Sparktank]

Would it be possible for a mod to move this to AviSynth Usage forum? I think would be more appropriate now that we've figured out a couple methods using AviSynth and not some Photoshop editors or anything.

Others may find it useful or contribute more (if any more needs contributing at all!).

[Guest]

Would it be possible for a mod to move this to AviSynth Usage forum? Done.