Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
|
2nd May 2013, 00:45 | #1 | Link | |
47.952fps@71.928Hz
Join Date: Mar 2011
Posts: 940
|
Trailer Fingerprint: How? Is it even possible?
There's been a trend on youtube lately that explores visual representation of the color pallets in trailers.
https://www.youtube.com/user/TrailerFingerprint/featured In this context, "fingerprint" isn't used a hash value or anything (like audio fingerprinting). Description from the first Fingerprint trailer: Quote:
Or they want to protect their idea and not give away its methods. I would like to know if this can be replicated at home, via AviSynth or other. I saw the waveform avisynth plugin, but it doesn't generate the type of waveform I'm looking for. It generates a linear line rather than the typical waveform you see when you Google image search. (Or maybe I set it wrong in the script?) It also doesn't support colors or some-such. Is there anything out there that isn't mainstreamed in the forums? A VirtualDub plugin, an amalgamation of plugins and utilities (SoX, etc) to replicate this effect? It would be nice to reproduce this effect for any video or even a movie. Even nicer if end result can be applied to a waveform. For aesthetic purposes, mostly to generate very large wallpapers.
__________________
Win10 (x64) build 19041 NVIDIA GeForce GTX 1060 3GB (GP106) 3071MB/GDDR5 | (r435_95-4) NTSC | DVD: R1 | BD: A AMD Ryzen 5 2600 @3.4GHz (6c/12th, I'm on AVX2 now!)
|
|
2nd May 2013, 17:12 | #2 | Link | |
Registered User
Join Date: Sep 2007
Posts: 5,340
|
Quote:
The problem you're going to have with using standard types of plugins for this is they are reactive per frame - but you want a "snapshot" of the video "fingerprint" & audio waveform that particular time frame, not for it to change over time . You want display the summary of the past results Not sure how to do it in realtime, but one way you can reproduce this effect is use something like after effects with a mask reveal . For audio waveform you can use any audio editor (e.g audacity) and composite/overlay the waveform . For the video "fingerprint" you can use various plugins or approaches. Trapcode echospace is fairly suitable for this one , which can be configured to make offset "clone" layers . Think of it as taking screenshots (resized, blurred or averaged) then combining them in a horizontal strip offset so they are beside each other . The strip is already there, but the mask "reveals" it as the video is played according to the play position . There is probably a "smarter" way to do this with expressions in AE, instead of auto generating layers. eg http://www.mediafire.com/?03cqek7zqxabwur Not sure how you would do this avisynth , but one of the gurus will probably suggest a way Last edited by poisondeathray; 2nd May 2013 at 17:28. |
|
3rd May 2013, 00:11 | #3 | Link |
47.952fps@71.928Hz
Join Date: Mar 2011
Posts: 940
|
Those are very interesting results.
Equally aesthetic as the trailer fingerprints on youtube. That is definitely something worth looking into. Thanks for the sample and info.
__________________
Win10 (x64) build 19041 NVIDIA GeForce GTX 1060 3GB (GP106) 3071MB/GDDR5 | (r435_95-4) NTSC | DVD: R1 | BD: A AMD Ryzen 5 2600 @3.4GHz (6c/12th, I'm on AVX2 now!)
|
3rd May 2013, 00:45 | #4 | Link |
契約者
Join Date: Jun 2008
Posts: 1,576
|
this is absolutely possible with imagemagick+avisynth. maybe avisynth is enough but i'm too lazy to try.
Wall of bad english below, care! ^__^ There is multiple ways to do that, and they may look a bit different but one of them I see like so: lets assume you want to make 1280x720 video, your source video have 3284 frames and you want fingerprint to change each pixel. means you need to prepare 1280 images. convert video to rgb24, resize it to something small like 64x64 resolution. Actually you want 1x1 resolution but not sure if avisynth can handle that... Now you need to make 1280 frames out of 3284. can be decimation or maybe blending, I dunno. Decimation probably better to do before resize. Then output these frame to individual files in some folder. Using batch-script for imagemagick make it process each image by downacaling it to 1x1px (here i'm sure imagemagick can handle it), then scale it again to something like 1x100. Then take all images and append horizontally into one 1280x100 image (there is an option for that). Now cou can append that image to the bottom of your source video (before that adjust height to what you want with any resizer). Then using masktools and using animated image with pure black and white colors you can make fingerprint reveal itself over time... Edit: you can use photoshop instead of imagemagick I guess, if you know how to make automated and batch operations with it. Edit2: resizing to 1x1 is to simulate average color. maybe there is a lot better ways for that in avisynth and/or imagemagick Well didn't tried any of this and probably none of this will be done in optimal way but I THINK its quite close. Last edited by Keiyakusha; 3rd May 2013 at 01:02. |
3rd May 2013, 00:51 | #5 | Link | |||
Registered User
Join Date: Sep 2007
Posts: 5,340
|
Quote:
I used a coarser gradations for the "waveform" just for demonstration purposes - it doesn't represent every frame, ie. "wider" time intervals than the youtube example which used finer "gradations" - but you could choose to do it like the youtube version if you wanted to. Also I just simply resized and blurred the "small" frames, I didn't do any average color pallate analysis or anything like that . The plugin pretty much aligns and distributes everything for you Quote:
Yes - I was trying to figure out how you would do this in avisynth too, but my coding is only at a basic level. I would like some guru to come in so I can learn how it could be done in avisynth . I'm thinking some runtime functions, gscript something like that I think you can use almost the same approach by generating an image sequence and applying the filters (resized, blurred , averaged, whatever) with ImageWriter() and overlaying it stepwise to generate the "strip" . x=0, x=2, x=4 etc... . The image reader would have to read single static images (not an image sequence), or the previous "summary" will change over time Somthing like overlay(base, strip, x=clp.framecount+somevariable) The audio waveform part is simple, it's just a screenshot from audacity and used as an overlay (but since it was grey and blue, I keyed out the grey and made it greyscale and brighter) , it can easily be done in avisynth I know there is a smarter way to do this as a template with variables (like clip width, height, framecount, fps) instead of "hardcoding" values, this way you can run the same script on different spec videos (dimension , fps) and it will automatically make adjustments, but it's beyond my knowlege Quote:
Code:
#If you want to append images horizontally, use this command convert image1.jpg image2.jpg image3.jpg +append result.jpg #If you want to append them vertically, change the + to a - convert image1.jpg image2.jpg image3.jpg -append result.jpg Last edited by poisondeathray; 3rd May 2013 at 01:09. |
|||
3rd May 2013, 01:57 | #6 | Link | |
契約者
Join Date: Jun 2008
Posts: 1,576
|
Quote:
Here is starting point for you. It is horrible but somewhat works. It requires latest avisynth possible (2.6 alpha) and its very slow. No 3rdparty plugins needed. Maybe only masktools to make it reveal itself. (not part of the example) Code:
bicubicresize(1024,576).crop(8,8,-8,-8) #not really needed, made to remove garbage from sides and stuff... calc=FrameRate()*1280/Framecount() #if this wont return right framerate to decimate to 1280 frames, the whole thing may fail. needs proper rounding Changefps(calc) Trim(0,-1280) #hack to make sure we have right amount of frames. but maybe good thing to have even with nice rounding... converttorgb32() pointresize(1,1) pointresize(1,64) WeaveColumns(1280) Fate/stay night TV opening: F/sn UBV theatrical teaser: (lots of credits that's why a lot of black) Last edited by Keiyakusha; 3rd May 2013 at 02:36. |
|
3rd May 2013, 10:15 | #7 | Link | |
Formerly davidh*****
Join Date: Jan 2004
Posts: 2,492
|
Quote:
Also rather than decimating frames, why not do Code:
weavecolumns(framecount()) bicubicresize(1280,64) DAvid |
|
3rd May 2013, 03:39 | #9 | Link |
Registered User
Join Date: Sep 2007
Posts: 5,340
|
Here is one way to do the strip "reveal" using animate() and a black strip generated by blankclip() . No masktools used, and it's newbie code, but it one way of doing it
eg http://www.mediafire.com/?rasd1nrt2ouexzx #note "strip.png" was generated by Keiyakusha's code, you can combine it in one step with a=last instead of imagesource() (I saved it out as a png only for testing purposes) I guess next is to find way to replicate an audio wavefrom like audacity in avisynth ? But I guess it's not too difficult to use audacity in a separate step and overlay() Code:
##Load main video main=FFVideoSource("the_dark_knight-tlr2_h480p.mov").ConvertToRGB32() ##resize width to main video's width, since it was different in this example a=ImageSource("strip.png").converttorgb32 a.BicubicResize(main.width, a.height) a1=last ##generate black strip the same dimensions as resized strip b=blankclip(a1) ##join the main video and "strip" stackvertical(main,a1) stack=last Animate(stack,0,main.framecount+1,"myfunc", b,0,main.height, b,main.width,main.height) Function myfunc(clip c1, clip c2, int x, int y) { Overlay(c1, c2 , x , y) } |
3rd May 2013, 05:06 | #10 | Link | |
契約者
Join Date: Jun 2008
Posts: 1,576
|
Quote:
Audiograph Some crazy stuff below, better don't try at home. (`∀´) Also it looks ugly, but maybe some tweaking, or taking other plugin as source (there was something newer I'm sure)... But just a concept based on my previous video fingerprint. Some parts of the waveform became black, no idea why. I'm surprised it shows something at all. Code:
#SomeSourceWithAudio() converttomono() Converttoyuy2() Audiodub(Blankclip(last),last) normalize() pointResize(1280,720) Audiograph(10) calc=FrameRate()*1280/Framecount() Changefps(calc) Trim(0,-1280) converttorgb32() bicubicresize(1281,128) crop(640,0,-640,0) WeaveColumns(1280) crop(0,0,0,-64) Greyscale() Blur(0.8) EDIT: Tweaked. Don't ask me how it works, I have no idea ^^ Still looks nowhere near good... Maybe something like Histogram(mode="audiolevels") can be of some use, but I got unexpected results with it. waveform 0.2 plugin by davidhorman gives me exceptions and crashes a lot - couldn't try it. Code:
#SomeSourceWithAudio() converttomono() Converttoyuy2() Audiodub(Blankclip(last),last) normalize() pointResize(1280,720) Audiograph(1000) calc=FrameRate()*1280/Framecount() Changefps(calc) Trim(0,-1280) converttorgb32() Crop(640,262,-639,-360) WeaveColumns(1280) Pointresize(1280,64) Greyscale() blur(0.8) Last edited by Keiyakusha; 3rd May 2013 at 06:27. |
|
3rd May 2013, 08:01 | #11 | Link | ||
47.952fps@71.928Hz
Join Date: Mar 2011
Posts: 940
|
My goodness! Such productivity! Thank you everyone for the input! Really loving all the samples. Piecing together a lot right now. Quote:
Quote:
It was like watching the trailer for the first time (again)! Both work flows combined can produce very interesting products. Two different audio representations, I like. Once I get the color extraction going, I'll be able to try and apply it to various forms of audio graphs. I'm thinking, with this Keiyakusha's script to nab the color schemes, I can set the height to much larger than I actually need. Once I got that and a few various audio waveforms pumped out (in different styles), I could use Photoshop (or similar) to do some fancy editing to impose the colored columns over the blank waveforms. And then use that in the video script for live feedback of the animated reveal. And also make really fancy, desktop-sized wallpapers for just the waveforms. You guys are terrific! I'm off to play. Looking forward to any updates or further suggestions, if any.
__________________
Win10 (x64) build 19041 NVIDIA GeForce GTX 1060 3GB (GP106) 3071MB/GDDR5 | (r435_95-4) NTSC | DVD: R1 | BD: A AMD Ryzen 5 2600 @3.4GHz (6c/12th, I'm on AVX2 now!)
|
||
3rd May 2013, 03:49 | #12 | Link |
Registered User
Join Date: Sep 2007
Posts: 5,340
|
Or maybe the starting position for the black strip should be "x=1", because it needs to unhide the 1st frame, frame zero ? And should main.framecount+1 be just main.framecount ?
Code:
Animate(stack,0,main.framecount,"myfunc", b,1,main.height, b,main.width,main.height) |
3rd May 2013, 15:50 | #13 | Link | ||
契約者
Join Date: Jun 2008
Posts: 1,576
|
Quote:
Edit: maybe something like bilinearresize(2,2).pointresize(1,1) will do better. Quote:
Edit: ok, this is pretty close to average in photoshop: Code:
#rgb input bilinearResize(2,2) blur(1.58).blur(1.58).blur(1.58).blur(1.58).blur(1.58) #this needs some better alternative to achieve correct result ^__^ pointresize(1,1) pointresize(320,200) Edit2: Code:
#rgb input bilinearresize(2,2) a=Crop(0,0,-1,-1) b=Crop(1,0,0,-1) c=Crop(0,1,-1,0) d=Crop(1,1,0,0) average(a,0.25,b,0.25,c,0.25,d,0.25) pointresize(640,360) So this: Will turn into this: weavecolumns(framecount()).bicubicresize(1280,64) #As I expected, it just smears everything. But the difference is small really. May not work for sources longer than few minutes cause horizontal resolution will be too huge. On a half-hour 720p file after 5 minutes waiting I still didn't got the result, while my 1st approach finished in 1.5 minutes. Edit3: This average is so close that visually I can't tell the difference from photoshop. No plugins required. Personally I think this is good enough. We'll lose accurate colors after converting to YV12 anyway. Code:
#rgb input bicubicresize(4,4) blur(0.65) bilinearresize(2,2) a=Crop(0,0,-1,-1) b=Crop(1,0,0,-1) c=Crop(0,1,-1,0) d=Crop(1,1,0,0) x=Overlay(a,b,mode="blend",opacity=0.5) y=Overlay(c,d,mode="blend",opacity=0.5) Overlay(x,y,mode="blend",opacity=0.5) pointresize(640,360) #or whatever resolution you need Last edited by Keiyakusha; 3rd May 2013 at 20:18. |
||
3rd May 2013, 21:30 | #14 | Link |
契約者
Join Date: Jun 2008
Posts: 1,576
|
So here is a function that generates video fingerprint. Latest avisynth 2.6 alpha required. No 3rd party plugins needed. Audio not included as I'm not satisfied with it.
It includes reveal approach by poisondeathray. It calculates fingerprint and attaches it at the bottom of the video. You need to specify one parameter "fph" which stands for fingerprint height. Better specify mod16 values as there is no any kind of checks, rounding or whatever. May fail badly if source clip have wrong resolution, if fph parameter is wrong, or whatever. Tested only on one yv12 source that had 1920x1080 resolution. It is very-very slow at start. But once fingerprint is calculated - everything should be fine. Example: fingerprint(fph=128) Code:
function fingerprint(clip source, int "fph"){ w=width(source) v=crop(source,8,8,-8,-8) v=bicubicresize(v,640,360) calc=FrameRate(v)*w/Framecount(v) v=Changefps(v,calc,linear=false).Trim(0,-w) v=converttorgb32(v).bicubicresize(4,4).blur(0.65).bilinearresize(2,2) a=Crop(v,0,0,-1,-1) b=Crop(v,1,0,0,-1) c=Crop(v,0,1,-1,0) d=Crop(v,1,1,0,0) x=Overlay(a,b,mode="blend",opacity=0.5) y=Overlay(c,d,mode="blend",opacity=0.5) v=Overlay(x,y,mode="blend",opacity=0.5) v=pointresize(v,1,fph) v=WeaveColumns(v,w).Converttoyv12() bg=blankclip(width=w,height=fph) v=StackVertical(source,v) Function ovl(clip c1, clip c2, int xx, int yy) { Overlay(c1, c2 , xx , yy) } v=Animate(v,0,source.framecount+1,"ovl", bg,0,source.height, bg,source.width,source.height) return v } Last edited by Keiyakusha; 3rd May 2013 at 21:46. |
3rd May 2013, 23:25 | #16 | Link |
47.952fps@71.928Hz
Join Date: Mar 2011
Posts: 940
|
We need more emotes with more emotions. ; ;
Wonderful! The combined scripts in an .avsi function worked beautifully. Now to put it to more experimental uses. Thank you guys so much! XD EDIT: Opened in VDub (yeh, compiling the fph took some time ^__^ so it was unresponsive for a short while). Saved the output frame to the clipboard. Used XnView to import the image from the clipboard. Saved it as PNG with maximum compression and lost the faint hints of red for the "S" logo part of the trailer. Maybe I should used BMP instead, no? Or PNG at a much lower compression level than the maximum (9).
__________________
Win10 (x64) build 19041 NVIDIA GeForce GTX 1060 3GB (GP106) 3071MB/GDDR5 | (r435_95-4) NTSC | DVD: R1 | BD: A AMD Ryzen 5 2600 @3.4GHz (6c/12th, I'm on AVX2 now!)
Last edited by Sparktank; 3rd May 2013 at 23:31. |
3rd May 2013, 23:29 | #17 | Link |
契約者
Join Date: Jun 2008
Posts: 1,576
|
I figured out it may be possible to make real waveform, but for that I need to figure out how to scale image without preserving center (if that's how it is called).
I mean i need do downscale the image in a way that upper pixels affected less than lower ones. But don't scratch your head over that, this is just a thought it may not help me at all. |
3rd May 2013, 23:39 | #18 | Link | |
47.952fps@71.928Hz
Join Date: Mar 2011
Posts: 940
|
Quote:
http://sdrv.ms/166BB0w Like that?
__________________
Win10 (x64) build 19041 NVIDIA GeForce GTX 1060 3GB (GP106) 3071MB/GDDR5 | (r435_95-4) NTSC | DVD: R1 | BD: A AMD Ryzen 5 2600 @3.4GHz (6c/12th, I'm on AVX2 now!)
|
|
3rd May 2013, 23:44 | #19 | Link |
契約者
Join Date: Jun 2008
Posts: 1,576
|
Oh, no unfortunately. I can generate the waveform, but it doesn't looks good. To make it look like the one audio editors show you, I need this unusual way of scaling... There was some ways to do that but I completely forgot everything.
|
4th May 2013, 00:04 | #20 | Link | |
Registered User
Join Date: Sep 2007
Posts: 5,340
|
Quote:
Can you explain what you have in mind ? Some non linear scaling ? |
|
Tags |
fingerprint, trailers, waveform |
Thread Tools | Search this Thread |
Display Modes | |
|
|