IVTC Pattern detection on anime: impossible?

[bilu]

Hi,

Anime encoding is an unknown world to me.
My interests in video encoding lately are variable framerate encoding in Matroska and pattern detection.

Anime is the biggest challenge I've ever seen. I've downloaded some samples of problematic sources like Neo Genesis Evangelion and made some right and wrong analisys about it.

Recently I've been trying to develop a method that would help me on pattern detection for a variable framerate IVTC process.

My method so far was this: do a XVID Null encode with AVS2AVI and DBGProxy (like DebugView but command-line) and grab the output from Telecide(debug=true) to a file, then grab what matters.

In this case I wanted to grab the "in-pattern" and "out-of-pattern" statements and convert them to "1" and "0" next to the framenumber.
MOVIE.AVS
=========
Loadplugin("D:\vfr\tools\Decomb510.dll")
Loadplugin("D:\vfr\tools\MPEG2Dec3dg.dll")

#FOrder works only with MPEG-2 sources
function FOrder(clip c) {
fo= GetParity(c)? 1:0
return fo
}

mpeg2source("D:\weird_clips\eva.d2v",idct=2,cpu=0)
Telecide(FOrder(),guide=1,post=0,debug=true)

NULL.BAT
========
reg import null.reg

dbgproxy avs2avi d:\vfr\encodes\MOVIE.AVS d:\vfr\encodes\MOVIE.AVI -p 0 -c xvid 2> d:\vfr\encodes\debug1.txt
gawk -f fmatch.awk d:\vfr\encodes\debug1.txt > d:\vfr\encodes\vfr1.txt

NULL.REG
========
Windows Registry Editor Version 5.00

[HKEY_CURRENT_USER\Software\GNU\XviD]
"mode"=dword:00000006

FMATCH.AWK
==========
/Telecide/ && /using/ {gsub(/:/,"");gsub(/in-/," 1 ");gsub(/out-/," 0 ");print $3,$7}

DEBUG1.TXT OUTPUT
=================
!Telecide: frame 37: matches: 187069 209091 205427
!Telecide: frame 37: [using n] [in-pattern]
!Telecide: frame 38: matches: 205427 215163 186466
!Telecide: frame 38: [using n] [in-pattern]

VFR1.TXT OUTPUT
===============
21 1
22 1
23 0
24 0
25 1

It would help me to define ranges for VFR Matroska encodings.

Example (Star Trek TNG Hybrid output):
http://forum.doom9.org/showthread.php?s=&postid=395247#post395247

288 0
289 0
290 0
291 0
292 0
293 0
294 1
295 0
296 0
297 1
298 1
299 1
300 1
301 1
302 1
303 1
304 1

The only change that happens is this one. So it's easy.

# timecode format v1
Assume 23.976
0,297,29.97

OR

# timecode format v1
Assume 29.97
297,0,23.976 (I don't know if it accepts 0 as the end, but it's just an example)

But it's not that easy on anime. There are lots of duplicates and with problematic sources like Neo Genesis Evangelion (would like to have an Excel Saga sample myself :) ) things like this come around:

1710 0
1711 1
1712 1
1713 0
1714 0
1715 0
1716 0
1717 0
1718 0
1719 0
1720 0
1721 0
1722 0
1723 0
1724 1
1725 0
1726 1
1727 1
1728 0
1729 1
1730 1
1731 1
1732 0
1733 1
1734 1
1735 1
1736 0
1737 1
1738 1
1739 1
1740 1
1741 1
1742 1
1743 0
1744 1
1745 0
1746 0
1747 0
1748 1
1749 1
1750 1
1751 1
1752 1
1753 0
1754 1
1755 0
1756 0
1757 0
1758 0
1759 0
1760 0
1761 1
1762 1
1763 1
1764 1
1765 1
1766 1
1767 1
1768 1
1769 1
1770 1
1771 1
1772 0
1773 1
1774 1
1775 0
1776 1
1777 1
1778 1
1779 1
1780 1
1781 1
1782 1
1783 1
1784 1
1785 1
1786 1
1787 1
1788 1
1789 0
1790 0
1791 0
1792 0
1793 0

And how do you establish a pattern on that?
I was thinking about a minimum of 10 consecutive 0s or 1s to decide if a frame range belongs or not to a pattern.

Would like your opinion. Due to the nature of anime, full of duplicates and weird productions, I'm wondering about the ways of improvement (that don't require manual processing).

Sorry for the very long post.

Bilu

[bilu]

For questions related to the VFR encoding process please refer to the Star Trek thread pointed above.

[gizmotech]

Bilu;

First off, the eva clip you're using, which I also have is a VERY poor example of anime source used today. Though yes alot of anime has been butchered like the eva source, not all of it.

Too better understand the ranges of anime pattern encoding, grab the following source (ie rent, or have someone provide you a clip **I can't**)

Sailor Moon, Uncut, Season 1, any episode. (ADV)
-- Contains 6 paterns, opening, part 1, middle a,middle b, part2, ending

RahXephon, Any episode. (ADV)
-- Contains Numerous detectable paterns, all following ccnnc but at different start points, AND small 30fps sections (max 1-2 / episode if at all).

Slayers (original, next, try), any episode. (CPM)
-- Right up there with the r00t of all evil. Full of non detectable matches, full and partial frame blends, periods of in patern ccnnc followed by random foolishness.

This will give you a far better understanding of what anime source looks like these days. After looking at slayers source though you will fully understand why Anime has been called for a very long time the Bastard Child of Video Encoding.

Gizmo

[bilu]

You sure gave me a whole new perspective :)

It's difficult for me to find anime in Portugal unless I go on seeking for specialized vendors (probably not rent, just sale).
And I'm not such a great anime lover ( I do like some anime ), samples would be cool to me just by the challenge of it.

But my process looks good enough for everything that is not anime. There are some hybrids around but I don't know other weird patterns out there (besides silent movies).

(What does ADV and CPM mean?)

Your post also reminded me of this thread. Seems anime is still a bastard :)

Comment on newer Anime sources
http://forum.doom9.org/showthread.php?s=&threadid=64379


EDIT: The Neo Genesis clip seem to fit your Slayers description.


Bilu

[lamer_de]

ADV and CPM are the companies that distribute these Animes in the US.

http://www.advfilms.com/
http://www.centralparkmedia.com/

Can't help you either with the samples though.

CU,
lamer_de

[gizmotech]

No, eva is relatively tame compared to slayers.

It CAN be correctly matched 75% of the time, it's just the damn op and ed which are thoroughly butchered. (I've also since borrowed the eva source from a friend.)

And yes you are right, your methods are far more applicable to Live Action given the total amount of movement in a LA encode when combing occurs. Anime on the other hand, when movement occurs, can be matched equally well, however as it is animation (largely bulk animation in some cases) and animation remains largely still 80% of the time where problems occur. It's in these areas of relatively low motion they use combing that cause anime encodes the most trouble, as current match detection tools are unable to correctly identify the combs in the low motion (commonly on mouths), result in incorrect matching. Alternatively in these relatively low motion areas, there is so much background noise interfering in the matching that the correct match require to remove the comb on the mouth is often drowned out by the noise.

Gizmo

[bilu]

@Gizmotech

I wonder about a possibility of a less-than-perfect automated anime backup for users who choose to do so. Keeping the original framerate.
That looked at least as good as when you watch the dreaded DVDs on TV.

Something like KernelDeint+Warpsharp or something...
What would be your best option?

Just for users who would prefer a less-than-perfect automated method to a perfect manual one.

Bilu

[gizmotech]

I wouldn't recommend a less then perfect solution for anime backup's.

I've seen what turn key solutions do to anime backups, and you can see them all over the web now (or irc/bt). Shoddy, high noise, comb filed, encodes which honestly look worse then the dvd's they were created from. Unlike LA where we throw a light smoother (if even that), hit it w/ a lanczos, and encode the sucker, the same system is absolutely useless for anime.

The reason the good anime backups are good is because the encoder doesn't turn key it. They spend the time to develop a filter chain to remove the noise, while keeping detail, and defining lines. They go out of their way to remove combs, to touch up bad spots on the source, and within the near future, to maintain the frame rate. Making an anime encode highly compressible w/o noise is the ultimate objective, because noise can be seen clear as day on the flat surfaces that animation contains.

So maybe another anime encoder can recommend some settings for you, but I won't. Turn key solutions for anime is Sacriledge (sp?) and should be burned at the stake.

Gizmo.

[bilu]

Oh well... :rolleyes:

If I ever make a new R4R_ENC or something I'll put a disclaimer:

"Manual filtering of anime is advised."

;)


Bilu

[mf]

Automated? Telecide without postprocessing, then FieldDeinterlace with full processing and dthresh=0, encode and forget :D.

[bilu]

@mf

Are you kidding or serious? :devil:

You're suggesting IVTC on the Telecine parts and blending fields on the interlaced ones?

I think KernelDeint is better than FieldDeinterlace(dthresh=0), what's your oppinion?

Some users do like to blend fields: ;)
http://forum.doom9.org/showthread.php?s=&postid=317353#post317353

Bilu

[mf]

Originally posted by bilu
@mf

Are you kidding or serious? :devil:

You're suggesting IVTC on the Telecine parts and blending fields on the interlaced ones?

I think KernelDeint is better than FieldDeinterlace(dthresh=0), what's your oppinion?

Some users do like to blend fields: ;)
http://forum.doom9.org/showthread.php?s=&postid=317353#post317353

Bilu
Well, fielddeinterlace only leaves blending and blurriness as artifacts. Blending is something you have to live with because you're lazy (it's the user that chose not to use something more intelligent :D), and the blurriness is something you can clean up later with sth like ssxsh, plus it increases compressibility (noobs love compressibility). I tried kerneldeint on anime once, and it sucked bigtime. It's not made for it, it doesn't even look nice with it. My above mentioned method is imho the most visually pleasing at the least effort. I used it to make a miniraw (which don't need to be HQ) for timing/typesetting, and it was even watchable at 80MB for a 24min ep. Of course I also threw in a conv3d for processor-cheap filesize lowering.

[bilu]

Have you tried KernelDeint(sharp=false,oneway=false)?

And would you prefer FD(blend=false) or FD(blend=true)?


Bilu

[bilu]

Originally posted by mf
Automated? Telecide without postprocessing, then FieldDeinterlace with full processing and dthresh=0, encode and forget :D.

When you say "Telecide without postprocessing" you mean Telecide/Decimate over the Telecined parts, right?

"you can clean up later with sth like ssxsh"
What's sth and ssxsh? UNIX shells? :D


Bilu

[AS]

He meant telecide(post=0) and ssxsh is one of mf's scripted sharptools

[bilu]

But what's the point of using Telecide(post=0) followed by FieldDeinterlace with post-processing?

If the idea would be to IVTC the Telecined parts using Telecide/Decimate, that would make sense to me :confused:


Bilu

[mf]

sth = Something
ssxsh = Supersampled XSharpen [explanation (http://mf.onthanet.com/supersampled_xsharpen/)] [AVS function (http://mf.creations.nl/avs/functions/SharpTools-v0.3.avs)]

The point of Telecide without postprocessing is that if you use postprocessing, only certain frames will be postprocessed. (OK, you can also use Telecide(blend=true, vthresh=0, dthresh=0), I give up ;)) If only certain frames and certain parts are deinterlaced, a sharpness inconsistency occurs and it isn't pretty. Plus there's always the danger of mouth combs not being detected correctly.

Oh, and the ssxsh explanation page is very old. Using VirtualDub filtering, ssxsh is very slow, but this is much less so using AVISynth and YV12 colorspace.

[bilu]

I wonder if an automated process for variable framerate encoding, doing IVTC over the "probably telecine" parts but keeping the framerate, would end up like a Dup script, since Telecide uses gthresh between 10 and 15.

A threshold to define if a frame belongs to a pattern or not as duplicate would end up chopping the frame "probably duplicate" and low-motion would stutter a bit just like with the Dup filter.


Bilu