Judder on PAL End of Evangelion MP4 encode - PAL Anime expert required!

[pwnsweet]

Hi everyone,

I've been a long time lurker of these forums but haven't done much posting because I can usually find the answers to my questions. However in this case I've come across an interesting observation that I can't seem to find an explanation to. I've spent a lot of time trying to solve this one myself and have gone to great lengths to provide the material necessary so that someone can help me so please take the time to look at it.

I have here the PAL End of Evangelion DVD that I wish to encode into mp4. Like all PAL DVD's, it runs at 25fps. As such, when I did the mp4 encode, I set the mp4 output settings in TMPGenc to 25fps. However, the resulting mp4 is strewn with judder that is worse than that found on the DVD and is noticeably visible on panning scenes and occurs roughly every half a second.

So then I tried encoding a particular panning scene at 23.976fps and the scene now has no noticeable judder at all. I then encoded the same scene at 24fps, 29.97fps, 30fps, 50fps, 59.94fps and 60fps. Of these, only 24fps had no noticeable judder.

According to my research, apparently the Evangelion series and movies have a colourful backstory of poor transfers and bad conversions to DVD but from within the jungle of information and mis-information I was able to determine that the series was originally produced and aired in Japan. This implies that it was made in the NTSC standard (29.97fps). However, the End of Evangelion is a film that was released theatrically in Japan so I automatically assumed it was originally produced at 23.976fps.

Like I said earlier, it was only when I encoded my mp4 at 23.976/24fps that the judder disappeared. But the thing that really confuses me is why the judder was there in the first place. As we know, PAL DVD's don't have any pulldown applied to them, they merely run 4% faster. So assuming the End of Eva film was produced at 23.976fps, why does the PAL DVD have judder that is only usually found on NTSC DVD's that have had 3:2 pulldown applied to them?

I've thought about this a bit and the only explanation that I can come up with is that the End of Eva film wasn't actually produced at 23.976fps but was actually produced at 29.97fps. When it was released in PAL territories they applied inverse telecine to the image to get it to 23.976fps, thereby intruducing telecine judder, then they sped it up by 4% like they normally do for PAL DVD's. But this hypothesis leads to a dead end because it doesn't explain why my 23.976fps mp4 encode doesn't have judder.

So I'm seriously confused. If anyone can shed some light over my way it would be greatly appreciated. Basically I'd like someone to explain what I've observed. I'd also really like some hard evidence of what fps the series runs was produced at and what fps the film was produced at.

End of Eva fps encode trials (http://obeopg.blu.livefilestore.com/y1pq5hezsMPvI6D8wpENMygDzOy-S9fBHiBXGogzB2B6sI9WmeSkG3foxLt7GDGfGjTsYsCMzMpR8Gh8RlroyB3d9Ie5aqJ5clm/End%20of%20Eva%20sample.rar?download)

[Lam3rD]

As we know, PAL DVD's don't have any pulldown applied to them, they merely run 4% faster.

"What we all know" doesn't necessarily have to be the truth in all cases.

I'm no anime expert, but your source appears to be FILM->NTSC->PAL, rather than NTSC*25/24, what you assume it to be. You have:
1. Telecine (doubled every 4th field) (FILM->NTSC)
2. Every fields 5 and 6 are blended and shown as 1. (NTSC->PAL) Of course totally ignoring what has happened during stage 1.

In other words way beyond repair.

p.s: if you need to see for yourself just open your source and Bob it. Watch field by field and see how every 5th is blended and you also have duplicates from the telecine. You could fix it if you somehow "unblend" the fields affected at stage 2 into individual, good looking fields and then iVTC the result, which is more or less impossible to do, at least with live footage, but it might be doable with anime.

[nm]

Yep, it's field-blended. You can use srestore (http://avisynth.org/mediawiki/Srestore) to reverse the bad format conversion, but there's no way to recover the original video perfectly.

My srestore script and output video (x264 --preset slower --crf 23):
LoadPlugin("ffms2-2.12/ffms2.dll")
LoadCplugin("yadif\yadif.dll")
LoadPlugin ("ReduceFlicker.dll")
LoadPlugin("removegrain\RemoveGrain.dll")
LoadPlugin ("mvtools\mvtools2.dll")
import("Srestore.avsi")

FFVideoSource("eva.m2v")

d = last.bob(-0.2,0.6).reduceflicker(strength=1)
yadif(mode=1)
srestore(dclip=d)

http://www.mediafire.com/download.php?krjymue5mzy

[qyot27]

I've thought about this a bit and the only explanation that I can come up with is that the End of Eva film wasn't actually produced at 23.976fps but was actually produced at 29.97fps. When it was released in PAL territories they applied inverse telecine to the image to get it to 23.976fps, thereby intruducing telecine judder, then they sped it up by 4% like they normally do for PAL DVD's. But this hypothesis leads to a dead end because it doesn't explain why my 23.976fps mp4 encode doesn't have judder.

So I'm seriously confused. If anyone can shed some light over my way it would be greatly appreciated. Basically I'd like someone to explain what I've observed. I'd also really like some hard evidence of what fps the series runs was produced at and what fps the film was produced at.
The series and films were all produced at 23.976, although the openings/endings of the TV series might be at 29.97. The Renewal versions (Platinum in the U.S., not sure about Europe) confirm this. I have the Japanese R2 of the Renewal Films - they use pulldown flagging and can be easily Force Filmed with DGIndex, as the Film rating is something like 99%. Although apparently the nice flagging is only on the Japanese release - ADV decided to release Platinum with hardcoded telecine; I don't think there was ever a U.S. release of the films' Renewal editions, although I could be wrong.

As to why the PAL release is so messed up, it's that PAL anime distributors are notorious for screwing up on authoring discs, or taking shortcuts that are easy for them but not for those wanting to work with the footage after ripping it - or at least they were. I'm not sure if the situation has improved any over the last 4 or 5 years, but most of the advice I see coming from the European editors I know is to buy the U.S. release if you want to make things easier on yourself quality-retention-wise (the easiest, but most expensive, option, is buying the original or remastered Japanese release of a series/film, whichever applies).

[pwnsweet]

Hey guys, thanks for the responses.

I've gone ahead and obtained the NTSC release of Neon Genesis Evangelion Platinum and have set myself a new goal - to see if I can watch it Judder-free!

I've IVTC'd the first episode on the disc to restore the original 23.976 frames. But my PC monitor is a 60hz LCD so when I watch it, my monitor is going to apply 3:2 pulldown on it thereby introducing judder right? So just for the sake of testing, I want to now speed up the video by 25% (23.976 x 1.25 = 29.97) so that the framerate is an even multiple of my monitor refresh rate. I've done that using Adobe Premiere and I've run into a problem. It's still juddering.

Does anyone know why? What have I overlooked and how can I fix it? The fact that it's 25% faster doesn't bother me in this case as my only goal right now is to watch a particular panning sequence without judder.

[nm]

So just for the sake of testing, I want to now speed up the video by 25% (23.976 x 1.25 = 29.97) so that the framerate is an even multiple of my monitor refresh rate. I've done that using Adobe Premiere and I've run into a problem. It's still juddering.
Did you try watching it frame-by-frame? Are there skips or duplicated frames? Post a sample clip, if you need help with it.

You could also use some advanced framerate converter (like MVFlowFPS) to avoid speeding up the playback.

[mp3dom]

MVflowFPS will never work with such kind of footage since anime doesn't have a true motion estimation (every single frame can be completely different from its previous or next frame). Regarding juddering, you need to see if Premiere is trying to repeat frames rather than speedup. If you simply want to speedup, add "AssumeFPS(30000,1001,true)" at the end of your avisynth script.

[nm]

MVflowFPS will never work with such kind of footage since anime doesn't have a true motion estimation (every single frame can be completely different from its previous or next frame).
Well, I was only considering sequences that do contain smooth motion (pans) and therefore are also critical for framerate/displayrate mismatch. Does MVFlowFPS produce bad artifacting in the rest of the content?

[mp3dom]

If only a panning occours, well, it could work well I think. The problem with "anime" is that every frame is different since is made by "hand" and the edges are not constant in the timeline. Also the framerate is often lower than 24fps (8 or 12 is quite common) so it's also difficult to estimate the motion over time. Yes, MVFlowFPS on such contents produce visible and strong artefacts.

[qyot27]

I've IVTC'd the first episode on the disc to restore the original 23.976 frames. But my PC monitor is a 60hz LCD so when I watch it, my monitor is going to apply 3:2 pulldown on it thereby introducing judder right? So just for the sake of testing, I want to now speed up the video by 25% (23.976 x 1.25 = 29.97) so that the framerate is an even multiple of my monitor refresh rate. I've done that using Adobe Premiere and I've run into a problem. It's still juddering.

Does anyone know why? What have I overlooked and how can I fix it? The fact that it's 25% faster doesn't bother me in this case as my only goal right now is to watch a particular panning sequence without judder.
Is it at all possible to raise the refresh rate to 72 Hz (being that 72 is a multiple of 24), or even 120 Hz (as that's a multiple of both 24 and 30)? I know that I can adjust the value, but I'm also using a CRT - however I was fairly sure that my grandfather's new LCD monitor had Hz adjustment values when I was using it.

Also, it would help if you'd post the IVTC script you used.