Log in

View Full Version : hand-editing anime frames


Isochroma
15th July 2005, 03:28
Since anime contains many duplicate frames, and the dup() filter is only so accurate (it tends not to remove all the duplicates at reasonable settings), and also since the dup() filter has no was to tell which frame is psychovisually better than another...

I decided some time ago to take anime source video (only Telecide() to replace the odd hard-interlaced frame), and using Vegas 6.0, split the entire movie/episode into individual frames.

Then, I manually edited frames by checking their neighbors for differences. It was easier than I thought to find frame groups (frames which differ only in quantization noise). What I found to be harder (and more time consuming) was choosing the best frame to replace the inferior ones.

Nevertheless, I proceeded to edit my first (upcoming) release: Perfect Blue. Currently, I am at frame 50,322 of just over 116,000. My record had been 6526 frames replaced in a single day of continuous work; however, a typical day sees about 4000 frames of average difficulty completed.

The results have been visually stunning: both the complete elimination of all noise, and the quality increase due to replacement of inferior frames with superior ones are immediately apparent upon inspection of the final product at full framerate.

What's more: a 48% decrease in filesize with precisely the same encoding parameters. Delta compressors output virtually no data when they parse identical frames.

By avoiding smoothing and denoising filters, I have been able to retain the original detail without in any way modifying the contents of each frame.

By combining this source technique with the MP4 format (currently created with x264), I have already achieved results that far surpass my expectations; even exceeding the quality of other releases of the same videostream size.

The combination of factors allowing for such an efficient packing of the videostream mean that there will be plenty of room in each file for the original audio stream(s), usually AC3.

Completion time for the first publicly-released video is estimated at 4-6 more days. The name is Noir, and the episode is 1.

My goal is to reach 10,000 frames edited per day, while maintaining reference quality (ie. no more than one judgement error per 300-500 output frames).

I will be establishing a website shortly to explain my techniques in still and motion graphics in an easy-to-understand way. It is my hope that others will also adopt these techniques, though they are extremely labor intensive.

Finally, it should be noted that in order to inspect frames with acceptable accuracy, the workstation's display size should be at least 21". Considering the time required, it is also very prudent to optimize the ergonomics of the system in use (refresh rate, resolution [preferably 1600x1200 or above]), as sitting in even a good position for 8-hour stretches is hard on the body.

MrTibs
28th July 2005, 17:54
Although I am impressed with your work, I have a filter in beta that may be more what your are looking for. The filter is called StaticFreeze. The idea is similar to dup() except it tries to keep the non moving parts of the frame. There is no blending just a direct copy from the last frame to the current frame. This means that you could have parts of the current frame that have been copied from 100 or more frames back. I think it may dramatically speed up your process because you clean a frame and keep the non moving parts of that frame even for non duplicate frames. The filter has demo modes so you can see the changes to the consecutive frames. At the very least, you can reduce the amount of cleaning required for each new frame.

Here is the filter...

StaticFreeze2.5 (http://www.geocities.com/mrtibsvideo/Beta.html)

Mr_Khyron
29th July 2005, 00:20
Do you have any small before and after clips to show us :)

Mug Funky
29th July 2005, 09:26
yikes!! i thought manually placing good video (from a different cut) over bad video was tedious as hell.

i really appreciate your efforts, but how on earth would you get through a box-set in a reasonable amount of time? :)

your technique would no doubt be very useful on high-budget restoration projects.

yes, a demo clip would be pretty cool - maybe 1 scene's worth in full D1 size?

mg262
29th July 2005, 11:56
@Isochroma,

that is insanely impressive (if rather masochistic)! One thought... if you have a moment, would you check if it is consistently the case that you are selecting I frames over P frames and/or P frames over B frames (or even if there's a tendency in that direction)?

You probably know this, but if you open up a video in VirtualDub, it will tell you what kind of frame it is at the bottom of the screen to the right of the frame number.

It strikes me that this is something that really needs a GUI to speed things up ... something that shows the duplicates side-by-side, or let you use the number keys to toggle between them, or something of a sort...

Edit: Mug Funky, what software do you use when you are lining up clips like that? I've hit one case (the introduction to the series I'm dubbing) where I can't just use cut detection because the (Australian) introduction is just ~six scenes from six different (French) episodes... I wrote a filter to search the DVD set to find the scenes used, but it doesn't line them up, and it's not worth writing another filter for a two-minute clip.

Isochroma
4th August 2005, 00:24
Wow! I am impressed MrTibs! I downloaded your filter, but haven't yet tried it... it will be tested on Noir ep.2...

I'd noticed the moving macroblocks and other elements in the frame in "still" areas. These I could not eliminate outside of real-motion identical framegroups. Your filter may enable that and provide a boost to achieve even higher compression and psychovisual quality. Thanks so much for the hard work; I will be your beta-tester!

mg262:

During those long hours (14 day @ ~8 hr./day) of editing, I developed an third-eye sense for which frame would be the nice one out of every group. For noir, I post the AVERAGE ratios below. Some segments different very much, so it shouldn't be used as a guide. Most frames were in groups of three, so I'll express the data in the form:

group size:frames (in order of decreasing quality)

2:2
3:3,2,1
4:3,2,1,4
>5:35% (ie. 35% from the first frame in the group was where the average best was).

I would like to maintain accurate stats but the overhead for stats logging would be at least 2x the actual work, so no go there!

Check out my page... it's up now - still writing the explanation for the DeltaChroma Process (should be up by this evening). The first episode is completed; check my website and AniDB for details:

http://209.153.193.54:8000/Chromasubs.htm

fccHandler
4th August 2005, 07:35
Wow, I thought I was the only one who did insane stuff like that! I admire your dedication.

I used to do a lot of airbrushing little specks and glitches out of movies, and manual IVTC of music videos, where the patterns shift constantly. I used Ulead Video Paint for the specks, and I wrote a program to help me IVTC manually (one field at a time...)

No doubt the results are far better than any automated process, but it does get old after a while. After spending four weeks on one particular movie, I was so sick of it that I can't even bring myself to watch that movie today.

Friendly advice: Don't burn yourself out doing this. :)

mg262
4th August 2005, 10:54
I would like to maintain accurate stats but the overhead for stats logging would be at least 2x the actual work, so no go there!It would probably be pretty straightforward to write a filter that takes the DVD source and the output from your process (before it is recompressed) and produces a log file listing which output frame(s) correspond to which input frame. I'll wait until your DeltaChroma write-up is in place before I say any more though...

Edit: I really enjoyed reading that! Your point about magnification of variation between frames is very true... I've even noticed luma flickering in the special edition Lord of the rings DVDs, and my eyes are far from being the best.

Is the frame chosen for a particular block guaranteed to come from that block? In other words, if you have e.g. a conversation where it cuts back and forth between two heads, you might find that frame 100, say, is identical to frame 110, but that frame 105 is different. In that situation, is it possible that 110 will be replaced by 100 or vice versa?

MOmonster
7th August 2005, 23:01
@Isochroma
What incredible work you do on this clip.
I like perfectionists. :)
But this is just to heavy for me.

Because my next Crestore function (to restoring fieldblended stuff) won´t have anymore the mlv/mnv feature I think about a function especially for anime, that compares the quality of very similar frames to avoid the output of bad quality frames (if possible).
I think this could be also helpful for you.
It will need some time, because less freetime, but this will be my next project.