Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
13th August 2008, 20:14 | #1 | Link |
Registered User
Join Date: Jan 2007
Posts: 6
|
Getting terrible results encoding anime.
It has been almost 2 years since I was encoding video and I'm trying to make a backup of an anime. I have spent a good amount of time scouring the forums for info and I'm getting awful results. So I'm looking for any suggestions of what I'm doing wrong. My goal is to create a high quality x264 encode of an anime with vfr and I'm trying to fit it to a 175MB(or 233MB) file size.
What I'm using: MeGui 0.3.0.1020, AviSynth 2.57, x264 (928 Jarod's patched build), DGIndex 1.5.0.0, nnedi 1.3.0.0, TIVTC 1.0.5.0, TDeint 1.1.0.0, TFM 1.0.4, MKVMerge 2.2.0 ......1st Pass: AviSynth Code:
LoadPlugin("C:\Apps\Encoding Tools\AviSynth 2.5\plugins\TIVTC.dll") d2vpath = "C:\DVD\Working\test01.d2v" DGDecode_mpeg2source(d2vpath,info=3) tfm(d2v=d2vpath,output="c:\DVD\Working\test01-tfm.txt") tdecimate(mode=4,output="c:\DVD\Working\test01-tdec.txt") crop( 8, 8, -8, 0) nnedi(field=1,dh=true).TurnRight().nnedi(field=1,dh=true).TurnLeft() Lanczos4Resize(640,352) Code:
program --pass 1 --bitrate 905 --stats "C:\DVD\Working\test01.stats" --bframes 5 --b-pyramid --direct auto --filter 1,1 --subme 1 --partitions none --me dia --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output NUL "input" --psy-rd 0 Avisynth Code:
LoadPlugin("C:\Apps\Encoding Tools\AviSynth 2.5\plugins\TIVTC.dll") d2vpath = "C:\DVD\Working\test01.d2v" DGDecode_mpeg2source(d2vpath,info=3) tfm(d2v=d2vpath,input="C:\DVD\Working\test01-tfm.txt") tdecimate(mode=5,hybrid=2,vfrDec=0,input="C:\DVD\Working\test01-tdec.txt",tfmIn="C:\DVD\Working\test01-tfm.txt",mkvOut="C:\DVD\Working\test01-timecodes.txt",batch=true) crop( 8, 8, -8, 0) nnedi(field=1,dh=true).TurnRight().nnedi(field=1,dh=true).TurnLeft() Lanczos4Resize(640,352) Code:
program --pass 2 --bitrate 905 --stats "C:\DVD\Working\test01.stats" --ref 16 --mixed-refs --no-fast-pskip --bframes 5 --b-pyramid --b-rdo --bime --direct auto --filter 1,1 --subme 6 --trellis 2 --partitions none --me umh --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output "output" "input" --psy-rd 0 My Source: I'm not sure if it violates rules to link the VOB of the source, I will link it(or not) as soon as I find this out.
__________________
One must be a fox in order to recognize traps, and a lion to frighten off wolves. - Niccolò Machiavelli |
13th August 2008, 20:40 | #2 | Link |
aka XaS
Join Date: Jun 2005
Location: France
Posts: 1,122
|
Any particular reason you used --partitions none ?
Using that certainly doesn't help, particularly on the second pass. Use the default setting instead. Also, why use vfr for output ? That's really looking for extra trouble since static frames barely take up any space at all (x264 marks the static parts of the picture as "skip" and doesn't even encode them). You can easily fit a clean 25min anime episode unresized (720x576 anamorphic, or 720x480 anamorphic) on 175MB and still get acceptable quality (though what's "acceptable" is very subjective). When I say "clean", I mean correctly denoised. Try adding hqdn3d(2,2,2) just before the resizing step to your script (you'll need the hqdn3d plugin), since DVD's all have noise, even anime ones (and the video you posted is full of noise). If that isn't enough, try hqdn3d() on default settings or fft3dfilter() on default settings instead (much slower). (I'll let you google the pages for those 2 filters, they're easy to find ). Also, what you posted isn't as awful as I first imagined. Also, why disable psy-rd ? Was it worse with it enabled ?
__________________
Q9300 OC @ 3.2ghz / Asus P5E3 / 4GB PC10600 / Geforce 8600 GTS Last edited by DarkZell666; 13th August 2008 at 20:44. |
13th August 2008, 21:11 | #3 | Link |
RipBot264 author
Join Date: May 2006
Location: Poland
Posts: 7,815
|
You know what? Try again with RipBot264. Enable Denoiser (ANIME profile) and everything should be ok.
__________________
Windows 7 Image Updater - SkyLake\KabyLake\CoffeLake\Ryzen Threadripper |
13th August 2008, 21:31 | #4 | Link |
The Crazy Idahoan
Join Date: Jun 2007
Location: Idaho
Posts: 249
|
Also, it looks like your settings for first pass and second pass are almost too different. For example. Im pretty sure that Psy-rdo should be enabled on both passes.
Upping the bframes to 16 should help somewhat as well (since this is anime). an really, why aren't you using all partitions? You can try the "Dup" plugin for avi-synth as well, it should offer some savings to you. |
14th August 2008, 06:27 | #6 | Link |
Registered User
Join Date: Jan 2007
Posts: 6
|
I took some of the suggestions and made a few changes, definite improvement. I don't know how I managed to miss "--partitions none", that shouldn't have happened. Anyway, here are the updated settings and sample.
......1st Pass: AviSynth Code:
LoadPlugin("C:\Apps\Encoding Tools\AviSynth 2.5\plugins\TIVTC.dll") LoadPlugin("C:\Apps\Encoding Tools\AviSynth 2.5\plugins\nnedi.dll") LoadPlugin("C:\Apps\Encoding Tools\AviSynth 2.5\plugins\hqdn3d.dll") d2vpath = "C:\DVD\Working\test01.d2v" DGDecode_mpeg2source(d2vpath,info=3) tfm(d2v=d2vpath,output="c:\DVD\Working\test01-tfm.txt") tdecimate(mode=4,output="c:\DVD\Working\test01-tdec.txt") crop( 8, 8, -8, 0) nnedi(field=1,dh=true).TurnRight().nnedi(field=1,dh=true).TurnLeft() hqdn3d(2,2,2) Lanczos4Resize(640,352) Code:
program --pass 1 --bitrate 905 --stats "C:\DVD\Working\test.stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 16 --b-pyramid --b-rdo --bime --weightb --direct auto --filter 1,1 --subme 6 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output NUL "input" ......2nd Pass: AviSynth Code:
LoadPlugin("C:\Apps\Encoding Tools\AviSynth 2.5\plugins\TIVTC.dll") LoadPlugin("C:\Apps\Encoding Tools\AviSynth 2.5\plugins\nnedi.dll") LoadPlugin("C:\Apps\Encoding Tools\AviSynth 2.5\plugins\hqdn3d.dll") d2vpath = "C:\DVD\Working\test01.d2v" DGDecode_mpeg2source(d2vpath,info=3) tfm(d2v=d2vpath,input="C:\DVD\Working\test01-tfm.txt") tdecimate(mode=5,hybrid=2,vfrDec=0,input="C:\DVD\Working\test01-tdec.txt",tfmIn="C:\DVD\Working\test01-tfm.txt",mkvOut="C:\DVD\Working\test01-timecodes.txt",batch=true) crop( 8, 8, -8, 0) nnedi(field=1,dh=true).TurnRight().nnedi(field=1,dh=true).TurnLeft() hqdn3d(2,2,2) Lanczos4Resize(640,352) Code:
program --pass 2 --bitrate 905 --stats "C:\DVD\Working\test.stats" --ref 5 --mixed-refs --no-fast-pskip --bframes 16 --b-pyramid --b-rdo --bime --weightb --direct auto --filter 1,1 --subme 6 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8 --8x8dct --me umh --threads auto --thread-input --progress --no-dct-decimate --no-psnr --no-ssim --output "output" "input" The most noticeable issue I still have is apparent in the solid color areas, primarily the dark ones. Am I getting too picky or is there a filter that would fix that?
__________________
One must be a fox in order to recognize traps, and a lion to frighten off wolves. - Niccolò Machiavelli |
14th August 2008, 12:57 | #8 | Link |
Mr. Sandman
Join Date: Sep 2003
Location: Haddonfield, IL
Posts: 11,768
|
it may be.
try disabling AQ reducing the strenght to 0. see if the situation gets better and if it does then try to find the sweetspot rising the AQ strenght until you get the desired results.
__________________
MPEG-4 ASP Custom Matrices: EQM V1(old), EQM AutoGK Sharpmatrix (aka EQM V2), EQM V3HR (updated 01/10/2004), EQM V3LR, EQM V3ULR (updated 04/02/2005), EQM V3UHR (updated 17/12/2004) and EQM V3EHR (updated 05/10/2004) Info about my ASP matrices. MPEG-4 AVC Custom Matrices: EQM AVC-HR Info about my AVC matrices My x264 builds. Mooo!!! Last edited by Sharktooth; 14th August 2008 at 15:42. |
14th August 2008, 15:43 | #10 | Link |
Mr. Sandman
Join Date: Sep 2003
Location: Haddonfield, IL
Posts: 11,768
|
infact it's not.
__________________
MPEG-4 ASP Custom Matrices: EQM V1(old), EQM AutoGK Sharpmatrix (aka EQM V2), EQM V3HR (updated 01/10/2004), EQM V3LR, EQM V3ULR (updated 04/02/2005), EQM V3UHR (updated 17/12/2004) and EQM V3EHR (updated 05/10/2004) Info about my ASP matrices. MPEG-4 AVC Custom Matrices: EQM AVC-HR Info about my AVC matrices My x264 builds. Mooo!!! |
15th August 2008, 06:07 | #14 | Link |
Angel of Night
Join Date: Nov 2004
Location: Tangled in the silks
Posts: 9,559
|
You know that with VFR, the first (mode 4) and second (mode 5) pass of the avisynth have to be run first, then the first and second encoding pass? To streamline it, you can run the first x264 pass over both avisynth passes, overwriting the first's stats, then run the second x264 pass. If you run the first pass on the first avisynth and the second on the other, you will be destroying rate control, scene-cut detection, and probably getting wildly undersized files as well.
Make sure that bitrate is also based on the 2nd pass avisynth's length, not the original's. They'll be different since you didn't change the FPS. |
15th August 2008, 15:09 | #16 | Link | |
Registered User
Join Date: Jan 2007
Posts: 6
|
Quote:
BTW, I turned off AQ and it seemed to help it out a little bit and made it less noticeable.
__________________
One must be a fox in order to recognize traps, and a lion to frighten off wolves. - Niccolò Machiavelli Last edited by Perscitus; 15th August 2008 at 15:13. |
|
15th August 2008, 16:44 | #17 | Link | |
Does it really matter?
Join Date: Jun 2004
Location: Chicago, IL
Posts: 1,542
|
Quote:
2) Take your 2nd pass script and run 2 passes of x264 like you normally would as an encode. |
|
Tags |
anime, quality, x264 |
|
|