Learning to rip/encode in a good way, specifically anime.

[nixius]

Hi all,
I now find myself with some extra time on my hands, and so I would like to begin ripping my anime DVD's once again.

Unfortunately, in the past I have found I have been trying to modify things and sort of 'hope-for-the-best'. I have followed quite a few tutorials on this site and others, so I do know the general idea of what encoding is all about.

However, I would like to understand more about it. As the title indicates, I am only really interested in encoding my anime DVD's. So basically, I was hopeing that perhaps people could share some resources on things they found particularly useful in learning to encode.

In a nut-shell, I would like to have a higher level of understanding than I do at the moment. Encoding really interests me so if you have anything you think might be related just throw it at me and I will gladly read it.

I know from googling that there are countless articles etc, but I was hopeing I could get a more refined and good quality resources from your personal experiences :D.

Thanks for reading
nixius~

[Guest]

There is so much variation in source material that it is not possible to give you an all-purpose recipe. I suggest that you start with one of your DVDs and post a link to a representative sample of the unprocessed VOB. Then we can advise you on how to properly handle it. Also tell us what your desired target format is.

[nixius]

I would intend to encode with XVid, just because I have used it before in tutorials and the like (if you would suggest something else I am open to be swayed on this matter).

I would then like to put this into an MKV with subtitles audio streams etc, which isn't that difficult and I have done before with success. Really the main part is encoding the AVI, but as I said I have always struggled to really understand what half of the stuff means and that is why I made this post.

I shall post some screenies of the VOB's I have at the moment that I am working with!

I took some pics of what I think are useful shots of the type of things that I will have to deal with. Rather than spam the pics here I made a quick web page www.jpopulous.org/dm9/test.htm

Also, I don't have very much web space... so I did a quick 80% encode of a fragment of the vob file with WMV9 ^^;

www.jpopulous.org/dm9/vob.avi

Just let me know if that is enough information for you to start with.

Also thankyou for your response, I appreciate it!

[Dark Shikari]

Might I suggest x264 for anime? You'll get much better compression than with Xvid, and if you're going to be using MKV, there's no reason to use such an aging format...

[nixius]

I just saw my edit for rule-12, I apologise if my title was too open.

I can see how it could be viewed so.

I have seen alot of use of x264 and I will certainly look into it, I am aware XVid is old but it is all I have used to any real extent.

[Guest]

Also, I don't have very much web space... so I did a quick 80% encode of a fragment of the vob file with WMV9 As I said, we need an unprocessed VOB fragment. Use a hosting site.

[nixius]

ok I have a 1m15s .vob uploading now, about 68MB. I hope that will suffice. I will post the link when it's uploaded!

All right, here is the link:

http://www.megaupload.com/?d=LUOTKYYV

I hope that the file is long enough, and that it all works. I never used anything like mega upload before ^^;.

[nixius]

Hi.

I have left it a while, so sorry for bumping my own topic.

I was wondering if perhaps the VOB file I provided wasn't good enough for some reason, if you need a longer snippet or more screenshots anything else just let me know and I will do my best to provide it!

[davexnet]

Take a look at this anime > x264 guide:
http://tankadillo.com/x264-anime

[Guest]

It's hard telecined with some 59.94 field rate fades. This appears to work reasonably:

# Use Honor Pulldown Flags in DGIndex
MPEG2Source("E:\tmp\nixius\SPLIT.d2v")
assumetff()
telecide(guide=1)
decimate()

[nixius]

The method I have always used is to create a d2v file, then open it with GK, do some modifications which will eventually give me an AVISynth file.

I have then edited this .avs file opening it with VirtualDubMod to try and get the effects I want, however, I can no longer seem to get a .d2v file that will work with GK or open with VDM.

I have changed the option to Ignore Pulldown Flags and Forced Film, (I changed it to Honor Pulldown Flags initially as you suggested). I also downloaded the newest version of DGIndex to make sure it wasn't something odd and verified I have the latest build of VDM.

Message By Virtual Dub: "Couldn't open source file, or obselete D2V file" so the problem is certainly with the .d2v (the avs also opens with MPC and says error on line X which is the mpeg2source line) which is weird, because this is the way I have always started most of my projects. I did try playing with a few more options inside DGIndex, but I figured I am probably missing something obvious.

Any ideas/comments/suggestions would be greatly appreciated!

[Guest]

You must have matching versions of DGIndex.exe and DGDecode.dll. Search the forum; this has been addressed many times.

[nixius]

You must have matching versions of DGIndex.exe and DGDecode.dll.

Thanks for the comment it worked. :thanks:

Sorry for not searching, I will make it more of a habbit in the future.

[nixius]

Hello again,

I have made a sample of what I have done so far, unfortunately I did kind of throw alot of functions at it ^^;. Although, as long as it looks good and the file size is reasonable this should be okay! (I don't care about encoding times, as long as an episode is done within a few hours that is fine).

I was wondering if anyone had any comments on the quality of the video. I have tried using x264 for encoding this, 2 pass. That's all the info I'm gonna provide for now because I don't want you to judge on the settings, just on what it looks like :devil:

Thanks for taking a look, here is the link:

http://www.megaupload.com/?d=IK8TCQ01
~6MB 41s

Also, because I have had very little experience with x264, I made an encode with XVid which I am more accustomed to. I accidently closed VirtualDub so the clips may be off by a second or two (/\).

Here is the XVid Link:

http://www.megaupload.com/?d=FCNPYTRC
~6MB 42s

XVid and x264 encodes look practically identical to me.

Finally, I think that the resolution of the video is not standard so that will need to be altered but quality wise I think it's okay (but then again I am no expert).

Vob Pic
http://www.jpopulous.org//dm9/DNA_VOB.png

x264
http://www.jpopulous.org//dm9/DNA_X264.png

xvid
http://www.jpopulous.org//dm9/DNA_XVID.png

Any comments are always appreciated!

[fazzaz31]

Hi there!

I just d/l the images as .png files and dropped the images into Photoshop CS3 for a color comparison. Here's some results:

Color picked dark area of back of hair foreground character:

VOB R:99 G:57 B:61
Xvid R:96 G:55 B:59
x264 R:96 G:55 B:59

Next a color pick on area between the eyes of the forward-facing character:

VOB R:214 G:191 B:176
Xvid R:212 G:189 B:175
x264 R:211 G:189 B:175

Both x264 and Xvid are more contrasty than the VOB, with Xvid having the most contrast.

Xvid and x254 give virtually identical color reproduction, with a tiny bit more red rendered with Xvid.

Personally speaking, the Xvid gives me a hair more colorful, slightly snappier image.

At 600% enlargement, the pixels were virtually identical in all images (including the VOB!).

I'd upload a screenshot, but I don't use hosting services, sorry.)

Cheers.

[fazzaz31]

Update:

Upon reviewing the Xvid/x264 images, I see they are not identical frames to the VOB.

Regardless, at 3200%, the x264 definitely has more gradation in pixels than the Xvid (more variation in pixel color in the same 6x6 pixel box) so it's smoother, but less saturated than Xvid. Go figure.

Still, the Xvid looks more "anime" than x264, at least to my taste.

Really, it's a tossup.

[Dark Shikari]

You're encoding at such high bitrate that there's not going to be any visual difference between the two.

This means you've reached "saturation"; you're throwing more bits at it for no real gain.

IMO, this is totally pointless, and the goal should be a reasonable compression vs. quality tradeoff. And, of course, its hard to judge how good your comparison is without knowing your settings.

Also note that "Photoshop" comparisons are totally useless, even worse than PSNR or SSIM, simply because the eye does not see chroma with nearly as much accuracy as it sees luminance. All that matters is your eyes--and if you can't see a significant difference at screen's distance from a single frame, let alone in motion, your bitrate is too high.

[fazzaz31]

"You're encoding at such high bitrate that there's not going to be any visual difference between the two.

This means you've reached "saturation"; you're throwing more bits at it for no real gain." - Shakiri

Yeah. I'd like to see a 6x6 pixel comparison of Xvid vs. x264 at a bit rate more comparable to a "mid-range" rendering (if such a thing exists).

[fazzaz31]

Hmm, seems Google has free hosting (sort of).

URL to "Xvid vs. x264," 3200% enlargement of sample area:
http://docs.google.com/View?docid=ddb5ntkj_40ghmb394n

[nixius]

First of all thanks for your comments, and thanks for all the effort in Photoshop fazzaz31 :thanks:

and if you can't see a significant difference at screen's distance from a single frame, let alone in motion, your bitrate is too high.

Hmm yes, I did use a much higher bitrate than I have seen others use, for example, in the guide earlier on. Then, I shall lower the bitrate and examine the difference. Thanks for that.

Also, I would like to know more about encoding in general (with an emphasis on x264 seeing as it seems to be the best way to go, or at least that is the feel I get from around the place), and I actually made this topic so that I could find out more. So, if you have any recommendations of websites or articles etc to read that are related to x264 or around encoding in general (so for example I can start to understand what all the settings are for in in the codecs) I'd be grateful for any assistance in that matter. It's easy to google search things, but I'd much rather have a recommended source if possible.

Another reason I didn't post my script is basically because I am not sure what I am doing and I am embaressed to post it because it is full of too many plugins (that probably conflict) and I know it isn't very ncie!

[Guest]

Another reason I didn't post my script is basically because I am not sure what I am doing and I am embaressed to post it because it is full of too many plugins (that probably conflict) and I know it isn't very ncie! But your thread is about learning to encode. How will you learn if you don't expose your work to positive criticism and then benefit from it?

[nixius]

Hmm yes, I see your point. You are correct :)

# SOURCE
mpeg2source("dnangel.d2v")

#assumetff()
telecide(guide=1,order=1,post=2,blend=true,vthresh=1)
#telecide(guide=1)
decimate()

ConvertToYUY2()
GuavaComb(mode = "NTSC", recall=95, maxVariation=25, activation=10)

DeDot()

FFT3DFilter(sigma=5,sigma2=3,sigma3=3,sigma4=.5,plane=4)

ConvertToYV12()
UnComb()

Convolution3d (0, 8, 16, 8, 8, 3, 0)

TComb(2, 5,6,6,5, false,12,false,4)

# CROPPING
crop(4,0,708,476)

# RESIZING
LanczosResize(720,544)

XVid settings I used the settings form the guide on this site, x264 settings I left everything at default except for :

Max ref Frames: 3

I encoded 2 pass, and at 1250 bitrate, although I usually do 1000.

I also added 2 VirtualDub Filters:

Smooth HQ Athlon by Klaus Post.
Diamater: 7
Threshold: 40
<all the rest default>

and HSV Adjust, moving the saturation to 108%

Edit: Oh btw, I do realise that I convert to YUY2 and then convert to YV12, when I could just move some functions to before the initial YUY2 conversion

[Dark Shikari]

x264 settings I left everything at default except for :

Max ref Frames: 3What is "default"? Default depends on the program you're using... MeGUI for example lets you paste the commandline you're using, so paste the actual settings you used ;)

[nixius]

I used VirtualDub, although the guide that was posted earlier used command line settings. Sorry I assumed the codec came with default settings.

Unfortunately I was unable to get MeGui working with command line settings, and I tried using the command line provided with x264 codec but I got some errors, so that is why I ended up using VD with x264 at least. However, I will write out what everything was:

---Rate Control
KeyFrame Boost (%): 40
B-Frames Reduction (%): 30
Bitrate Variability(%):60

Min QP: 10
Max QP: 51
Max QP Step: 4

Scene Cut Threshold: 40
Min IDR-frame interval: 25
Max... : 250

---MB & Frames
8x8 transform [X]
8x8,8x16, 16x8 P-Frame Search [X]
8x8,8x16, 16x8 B-Frame Search [X]
4x4, 4x8, 8x4 P-Frame Search [ ] <<NOT ticked>>
8x8 Intra Search [X]
4x4 Intra Search [X]

Max Consecutive B-Frames: 2
Bias: 0

Use as references [ ] <<NOT ticked>>
Bidirectional ME [ ]
Adaptive [X]
Weighted biprediction [X]

Direct Mode: Temporal

---More...

Partition Decision: 5 (High Quality)
Method: Hexagonal Search

Range:16 (this is greyed out)
Chroma ME [X]
Max.ref Frames: 3
Mixed Refs [ ] <<NOT ticked>>

Simple AR 1:1
Threads: 1
CABAC [X]
Trellis [X]
Deblocking Filter [X]
Noise Reduction: 0
Sttrength: 0
Threshold: 0 (think this and strength refer to the noise)

Log Level: Error
FourCC: H264

---Bitrate
Pass Option: (this changes betwee First Pass, single, Nth)
Bitrate: As I said 1250, but I usually used 1000
Statsfile: location (.\x264.stats) << for first pass if I remember correctly

Sorry, I should really invest some extra time in getting command line working. I was very frustrated that I couldn't get that done :devil:

[Dark Shikari]

I used VirtualDub, although the guide that was posted earlier used command line settings. Sorry I assumed the codec came with default settings.

Unfortunately I was unable to get MeGui working with command line settings, and I tried using the command line provided with x264 codec but I got some errors, so that is why I ended up using VD with x264 at least. However, I will write out what everything was:This is very bad. x264 Video for Windows is not only old and unmaintained, but H.264 in AVI as a whole is generally considered Evil (TM).

If you have problems post them and get help rather than use something that's inherently broken to begin with :p
Max Consecutive B-Frames: 2This should be 16; anime benefits greatly from many B-frames (and x264 chooses the number automatically anyways, so no reason to cap the encoder to a specific value!). In particular, lots of B-frames also help it make better use of its reference frames.
Use as references [ ] <<NOT ticked>>
Bidirectional ME [ ]These should probably be ticked.
Direct Mode: TemporalTemporal is generally not good for anime; either use Auto on both passes, or Spatial.
Partition Decision: 5 (High Quality)
Method: Hexagonal SearchYou should generally use at least subme 6 (first parameter) and probably a better motion search too.
Max.ref Frames: 3
Mixed Refs [ ] <<NOT ticked>>Adding more refs and mixed-refs drastically increase efficiency for encoding anime, albeit at the cost of encoding speed.

[Guest]

That's a LOT of filtering for what looked like a pretty clean source. Can you really justify all that, or are you suffering from filteritis?

[nixius]

Thanks for your comments, really appreciate you helping me out!

This is very bad. x264 Video for Windows is not only old and unmaintained, but H.264 in AVI as a whole is generally considered Evil (TM).

If you have problems post them and get help rather than use something that's inherently broken to begin with

I did read something about VirtualDub not being best for x264 but I didn't think all that much of it. Am I right in thinking that it is then alot better to be using MeGUI or some other command line based program and creating an .mp4 file?

I do very much appreciate your tips on the settings, however, I would really like to understand what they all mean . For example, I have no idea what a B-Frame is, or for that matter pretty much what anything you mentioned is . There probably is no easy way to learn so I think I will just google everything I can and work from there.:D

Later on I will try a few encodes with the new settings with a lower bitrate and re-post x264 vs XVid.

Am I right in think that if the bitrate is so high and they are both 'beyond' saturation point, that at a lower bitrate the differences between XViD and x264 will become apparent?

That's a LOT of filtering for what looked like a pretty clean source. Can you really justify all that, or are you suffering from filteritis?

I am pretty sure it is mostly filteritis. The main reason for the mass of filters is that some parts of the video have annoying artifacts and dot crawl.

I am not sure of the correct approach to these problems, so honestly I google'd up filters for each type of problem, read their pages for the settings and tried to understand what they meant, then guessed the best settings to use.

I will have more of a play with these filters as well... I bet I could remove some of them ^^;

[J_Darnley]

Having looked at your source, and pinching someone else's IVTC, I came up with this. Checkmate removes the rain bowing and dot crawl very well. Other than that, the source is very clean. Such strong filtering like what you have used is just going to remove detail and smooth it too much.
MPEG2Source(vid_source, cpu=6)
AssumeTFF()
Checkmate()
Telecide(guide=1)
Decimate()
Vinverse() # probably not needed if using telecide's post-processing
ToonLite(0.2) # may thin lines too much for your taste
Crop(8,2,-8,-2)

The following is a good link for explaining some command line options for x264. I'm not sure if it explains some concepts in as much detail as you might like, but Google or maybe Wikipedia should be able to help you there. And you can always post questions you have.
http://ffmpeg.x264.googlepages.com/mapping

[Deinorius]

@nixius
For removing grain DeGrainMedian(limitY=6,limitUV=8,mode=3) should be enough. I tried this script (nearly the same as J_Darnley's) and it works quite well too. With x264 and crf 20 (PS3 profile - HighProfile, subme6, RDO,...) I get around 700 kbit/s.
MPEG2Source("F:\SPLIT.d2v",idct=7)
Checkmate()
tfm(d2v="F:\SPLIT.d2v").tdecimate(mode=1)
Crop(8,0,-8,-0)
RemoveGrain(24)
DeGrainMedian(limitY=6,limitUV=8,mode=3)
#awarpsharp(depth=8,cm=1,blurlevel=1) #activate as you like (or try ToonLite)
gradfun2db(thr=1.6)

[nixius]

Hi again I am currently encoding some samples so please bear with me!

Initially I am doing a 1000 bitrate 2 pass of XVid and x264 again, but with the new x264 setting so graciousely provided by Dark Shikari - although for some reason I can't change Max Consecutive B-Frames to anything above 5.

The following is a good link for explaining some command line options for x264. I'm not sure if it explains some concepts in as much detail as you might like, but Google or maybe Wikipedia should be able to help you there. And you can always post questions you have.
http://ffmpeg.x264.googlepages.com/mapping

That as you say isn't as detailed as I would like but it is an excellent starting point. Thank you very much for the link it will surely help!

I am currently encoding the x264 file into AVI again, judging from previous comments this seems like a bad idea. What are your preferred methods of encoding x264 video?

Once I have these initial samples I will play around with the settings from Deinorius and J_Darnley posts. Thanks alot for your suggestions :D! :thanks::thanks:

[Dark Shikari]

I am currently encoding the x264 file into AVI again, judging from previous comments this seems like a bad idea. What are your preferred methods of encoding x264 video?Try MeGUI. Its pretty easy to use and has some nice presets, including anime-related ones.