hajj_3
1st October 2008, 22:09
This guide was given to me, it was taken to another site, doom9 doesnt really have any guides with screenshots so i thought i'd share it. Hopefully people will improve this, explain more about which settings do what and what values to use etc. All credit goes to the original creators of this guide. Moderators are welcome to change these first 3 posts and update them whenever they like.
Please could a moderator make this a sticky, it with help countless people and save loads of threads being made.
Software you will/may need:
GraphStudio............: Download Link (http://blog.monogram.sk/janos/)
Eac3to.................: Download Link (http://forum.doom9.org/showthread.php?t=125966)
x264...................: Download Link (http://x264.tk/)
AVSP...................: Download Link (http://avisynth.org/qwerpoi/Download.html)
AviSynth...............: Download Link (http://avisynth.org/mediawiki/Main_Page)
FFmpeg source..........: Download Link (http://forum.doom9.org/showthread.php?t=127037)
MeGUI..................: Download Link (http://forum.doom9.org/showthread.php?t=96032)
Haali..................: Download Link (http://haali.cs.msu.ru/mkv/)
ffdshow................: Download Link (http://ffdshow-tryout.sourceforge.net/)
SurCode DTS............:
Arcsoft DTS DLLs.......:
Arcsoft Total media....:
MKVtoolnix.............: Download Link (http://www.bunkus.org/videotools/mkvtoolnix/)
LimitedSharpenFaster...:
Soothe.................:
Notice: its possible to encode HD DVDīs with this guide and if all fails you can convert the .EVO file to mkv with eac3to and follow the normal steps
Install all programs!
In ffdshow you need to activate support for VC-1 and MPEG-2. So if you goto start under "Programs"
find ffdshow video decoder configuration" in there you see a section called "Codecs" in there you need to find "VC-1" and choose "libavcodec" as the Decoder.
http://hdimage.org/images/cwv6mtgm1dz1zgymiyb2_ffdshowpic1.jpg
Same procedure for MPEG-2 change from deactivated to "libmpeg2"
http://hdimage.org/images/7ftdlkcvwt2ygygey44x_ffdshowpic2.jpg
Graphstudio
With GraphStudio we can make a .GRF file for the avs script we will be making next in this guide.
What this program can do for us is is that we can chose which filters and decoders we want to use for our avs script.
Start up Graphstudio and press "Ctrl + F" if you look on the picture below you can see all the filters. Be sure you are under the "Directshow filter" section.
http://img236.imageshack.us/img236/3209/92399041gq1.jpg
1. Choose "Haali Media Splitter" it will prompt you for the file, choose the one you want to decode "filename0001.m2ts" (if the movie is splitted over several files learn how to mux them in the Eac3to section.)
2. Choose "ffdshow Video Decoder"
3. Join the two filters by dragging the mouse from "Video" to "In" As shown on the picture below.
http://hdimage.org/images/obruexi1k2h8rtb2deu_graphpic2.jpg
Now save your graph!
AVSP
With AVSP we can make a .avs file for MeGUI i will just cover the basics in this guide!
Open your .grf file that you just made you will have to change it from only viewing .avs files as i do on the picture below.
http://img516.imageshack.us/img516/5049/50265903xe2.jpg
Now we can begin the script
This is more or less how your script will look like when you opened the .grf file
DirectShowSource("C:\your_movie\filename.grf")
Now if you know what framerate the movie is add Fps=XX.XXX to your script, also add Audio=false as we will be focusing on the video stream. (Learn how to find the framerates in the Eac3to section).
This is how it will look
DirectShowSource("C:\your_movie\filename.grf",Audio=false,Fps=23.976)
Now add ConvertToYV12() to your script, this is the color conversion filter.
DirectShowSource("C:\your_movie\filename.grf",Audio=false,Fps=23.976)
ConvertToYV12()
Now we want to crop the movie so find a bright frame in the movie and crop all the black bars away.
If the Aspect Ratio (AR) is not correct you are allowed to undercrop to archieve the right AR.
So start of with this code "Crop(0,0,-0,-0)" Left,Top,-Right-Buttom
You always want to archieve either mod16 or mod4 undercrop if necessary there must be no remainder.
In this example i had to crop the video to 1920x800 Pixels
Code i used "Crop(0,144,-0,-136)"
How to calculate if itīs mod16 or mod4
mod16 "16 divisible With end result with no remainder"
mod4 "4 divisible with end result with no remainder"
Left+right= Pixels making the width 1920- Pixels=
Top+Buttom= Pixels making the height 1080- Pixels=
So in my example i got this result
Width 1920-0=1920 Pixels 1920/16=120
Height 1080-280=800 Pixels 800/16=50
So now our script looks something like this
DirectShowSource("C:\your_movie\filename.grf",Audio=false,Fps=23.976)
ConvertToYV12()
Crop(0,144,-0,-136)
If your video donīt need to be resized and is not interlaced you can save your avs now.
Save as "encode script" and make an extra for your samples save as "Sample script"
Now if you wanna resize the movie to 720p you can use these numbers.
Spline36resize(1280,720) Video source roughly 1920x1080
Spline36resize(1280,688) Video source roughly 1920x1040
Spline36resize(1280,544) Video source 1920x816
Spline36resize(1280,536) Video source 1920x800
Spline36resize(1280,528) video source 1920x800
Choose 536 or 528 depends on which one looks better to you.
Here is my script now
DirectShowSource("C:\your_movie\filename.grf",Audio=false,Fps=23.976)
ConvertToYV12()
Crop(0,144,-0,-136)
Spline36resize(1280,536)
OAR - Original Aspect Ratio
Now when you have resized it be sure that you have OAR or close to OAR.
Here is how you can check how to get OAR. Always go for either mod4 or mod16
Height/Width*1280=OAR
So lets say i have a source that is 1920x1040 after cropping. Height should be 693,33 Well as we canīt uptain that we are going to find the nearest spot. i did that with mod4 height = 692
Now your commandline should look something like this.
DirectShowSource("C:\your_movie\filename.grf",Audio=false,Fps=23.976)
ConvertToYV12()
Crop(0,144,-0,-136)
Spline36resize(1280,692)
LimitedSharpen(Faster)
LimitedSharpen is a avs filter that can sharpen up your source. (Not recommended if you are new to avs scripts)
Applies a strong sharpening to edges and fine details by controlling halos and minimising artifacts. All sharpening is a 3-way struggle between edge-halos,
edge-aliasing, and confetti artifacts; LimitedSharpen is an attempt to minimize each in the most visually pleasing way.
This should be used as the last filter in a chain if possible, like any sharpener. The different Smodes cause different defaults to be used in certain cases,
or limit the ranges differently. See the source for specifics.
Here is an overview of itīs commands.
Smode int (1-4, default 3)
Sharpen mode:
1 : UnsharpMask
2 : Sharpen
3 : Range sharpening.
4 : nonlinear sharpening.
Each mode has progressively less haloing in its sharpening.
ss_x float = 1.5
ss_y float = 1.5
Supersampling when > 1, which reduces aliasing on edges.
dest_x float = width
dest_y float = height
Resize directly to final size, saves a resize step.
strength int = 160
Sharpening strength:
For Smode=1, it can be 0~127 (simple sharpening), 128~255 (simple overdrive), 255~4096 (big overdrive).
For Smode=2, values 0~100 are handled over to Sharpen() as 0.0~1.0. Values >100 are mapped to 100.
For Smode=3, 0~100 is common, but 100~inf. can be used if necessary.
Default is strength=160 for Smode=1, and strength=100 otherwise.
radius int (1-10, default 1)
Radius of UnsharpMask, only used in Smode=1.
overshoot int = 1
Sharpening limit: a sharpened pixel may not exceed [min,max] of the local neighborhood by more than 'overshoot'.
(High values remove the "Limited" of LimitedSharpen, and cause excessive haloing.)
undershoot int = [overshoot]
Separate limit for pixels that get darker during sharpening. Good for cartoons with strong lines. (If not specified,
then undershoot==overshoot.)
Lmode int (0-3, default 1)
Lmode:
0 : No effect
1 : Clamp to over/undershoot.
3 : Zero over/undershoot on edges, normal over/undershoot on not-edges.
soft int (0-100, default 0)
Softens the effect of sharpening. -1 will (try to) autocalculate a good value.
edgemode int (-1 to 2, default 0)
-1 : shows edges.
0 : does nothing.
1 : restricts sharpening to edges
2 : restricts sharpening to mostly non-edges (I think?)
special bool = false
special=true tries to raise detail contrast in dark areas of a frame. (Usage not really recommended - the method is simple
and might cause artefacts.)
wide bool = false
With wide=true, the local [min,max] limits are gathered from the 5x5 neighborhood, instead just 3x3. Enables more gain in
detail contrast. Increasingly useful with increasing supersampling factors.
exborder int (0-16, default 0)
Command example of a basic script
LimitedSharpenFaster(ss_x=1.0, ss_y=1.0, Smode=3, strength=80, overshoot=0)
The final script would look something like this.
DirectShowSource("C:\your_movie\filename.grf",Audio=false,Fps=23.976)
ConvertToYV12()
Crop(0,144,-0,-136)
Spline36resize(1280,536)
LimitedSharpenFaster(ss_x=1.0, ss_y=1.0, Smode=1, strength=60, overshoot=0)
for more info about LSF read this thread LimitedSharpen (http://forum.doom9.org/showthread.php?t=84196&highlight=limitedsharpen)
For more filters visit these sites.
Link 1 (http://avisynth.org/warpenterprises/)
Link 2 (http://avisynth.org/warpenterprises/)
Now there is several of ways to encode with x264 i will show you two ways GUI encoding and Batch encoding.
MeGUI
MeGUI is the program we use as our GUI encoder. This encoder has a graphical interface if you want to encode using Batch goto the next section.
When you start MeGUI it will prompt you if there is any new updates. If you want to use the x264 builds i linked to at the top of the guide
untick the x264 update. If you want to use the x264 builds i recommended copy the x264.exe file to "c:\program files\megui\tools\x264\"
When it prompts about x264 profiles tick the following ones:
x264: DXVA-HD-HQ
x264: DXVA-HD-Insane
x264: Unrestricted (take the ones you think you need)
Now we have that in order we can continue. When you have started MeGUI choose your avisynth script (.avs)
http://hdimage.org/images/d4a7ypsl45urdbg8hh0_meguipic1.jpg
Be sure that File format is set to "MKV"
Now letīs start configuring our x264 encoding settings goto "Config"
I will be focusing on the DXVA-HD-HQ preset and tweak it a little (!When using VBV restrictions you should only be using 2-pass mode.
As 1-pass modes has some compliancy problems at the moment).
DXVA is for Gpu and Hardware support.
here is the basic commandline before i tweaked it for our guide.
program --pass 2 --bitrate XXXX --stats ".stats" --level 4.1 --ref 4 --mixed-refs --bframes 3
--b-rdo --bime --weightb --direct auto --filter -1,-1 --subme 6 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8
--8x8dct --vbv-bufsize 40000 --vbv-maxrate 50000 --me umh --threads auto --thread-input --progress
--no-psnr --no-ssim --output "output" "input"
Well the first thing i want to do is raise --subme 6 to --subme 7
You can do that under "RC and ME" There you have "subpixel Refinement" you want to set that to "7 - RDO Level 2 (Slower)"
http://hdimage.org/images/zue433dbc7d521qkog1_meguipic2.jpg
The reason for this is that Psy-rdo works very well with --subme 7 it will take some extra time, but we really want to archieve the best result.
In the same section you can change the values of "M.E. Range" Default is "16" You can raise it to 24, 32, 64 higher the better but for movies 24 should really be enough 64 can be used for Anime.
Expect a much longer encoding time when raising these numbers!
If you goto "Main" in x264 in there untick turbo mode. As we want exact same settings for pass 1 and 2. This will take twice the time but will gain you better quality.
make a sample to see if itīs worth the extra time.
http://hdimage.org/images/5zdc5x4jdlqc6db7bcc1_meguipic3.jpg
Now if you donīt care about hitting a specific filesize you can do a Single pass constant Rate Factor encode instead this will provide you with teh same quality through the hole movie.
in x264 go from 2 pass to const. Quality and choose a level between 18-24. 18 being the best.
Tweaking
Well as movies all vary how compressable, grainy and etc.
There are numerous of tweaks you can apply to your settings.
Level 4.1 Compliances:
number of reference frames.
|-----------|---------|
|Resolution | no. ref |
|-----------|---------|
| 1280x544 | 12 |
| 1280x720 | 9 |
| 1920x800 | 5 |
| 1920x816 | 5 |
| 1920x1080 | 4 |
|-----------|---------|
If none of the resolutions above match your source, use the following equation to work it out for yourself:
8388608
__________________
(width x height)
thanks to DarkShikari for the method and techouse for the math
Deblocking:
Movies: the highest you should set it is 0,0 here is some you can try out with.
-1,-1
-1,-2
-2,-1
-3,-3
Anime: With Anime you should always stay on the + side +2,+2 is considered a good setting.
Try and mess around with the settings to see which setting you get the best results from.
Adaptive Quantizers:
A new AQ algorithm can be introduced simply by introducing a new aq_mode and a corresponding if in adaptive_quant_frame.
This also allows quantizer field smoothing, since quantizers are calculated beforehand rather during encoding.
Since there is no more reason for it, aq_mode 1 is removed. The new mode 1 is in a sense a merger of the old modes 1 and 2.
WARNING: This change redefines CRF when using AQ, so output bitrate for a given CRF may be significantly different from before this change!
You can change settings with this command.
Keep in mind that you should keep the settings between 0.7 - 1.5
Adaptive Quantizers can be found under "Advanced" in x264
--aq-strength 0.8
New B-frame Decision
In the new x264 builds there is a new B-frame decision itīs much more precise than the old method. Recommended numbers of B-frames with this new B-frame decision "3-6" B-frames.
If you where using 16 B-frames before it will be considerably slower than with the old method because of the new algorithm use 4-5 B-frames instead.
Command to enable the new B-frame decision.
--b-adapt 2
Now if you donīt want to use it write this command in custom commands in the advanced section of x264
--b-adapt 1
Please could a moderator make this a sticky, it with help countless people and save loads of threads being made.
Software you will/may need:
GraphStudio............: Download Link (http://blog.monogram.sk/janos/)
Eac3to.................: Download Link (http://forum.doom9.org/showthread.php?t=125966)
x264...................: Download Link (http://x264.tk/)
AVSP...................: Download Link (http://avisynth.org/qwerpoi/Download.html)
AviSynth...............: Download Link (http://avisynth.org/mediawiki/Main_Page)
FFmpeg source..........: Download Link (http://forum.doom9.org/showthread.php?t=127037)
MeGUI..................: Download Link (http://forum.doom9.org/showthread.php?t=96032)
Haali..................: Download Link (http://haali.cs.msu.ru/mkv/)
ffdshow................: Download Link (http://ffdshow-tryout.sourceforge.net/)
SurCode DTS............:
Arcsoft DTS DLLs.......:
Arcsoft Total media....:
MKVtoolnix.............: Download Link (http://www.bunkus.org/videotools/mkvtoolnix/)
LimitedSharpenFaster...:
Soothe.................:
Notice: its possible to encode HD DVDīs with this guide and if all fails you can convert the .EVO file to mkv with eac3to and follow the normal steps
Install all programs!
In ffdshow you need to activate support for VC-1 and MPEG-2. So if you goto start under "Programs"
find ffdshow video decoder configuration" in there you see a section called "Codecs" in there you need to find "VC-1" and choose "libavcodec" as the Decoder.
http://hdimage.org/images/cwv6mtgm1dz1zgymiyb2_ffdshowpic1.jpg
Same procedure for MPEG-2 change from deactivated to "libmpeg2"
http://hdimage.org/images/7ftdlkcvwt2ygygey44x_ffdshowpic2.jpg
Graphstudio
With GraphStudio we can make a .GRF file for the avs script we will be making next in this guide.
What this program can do for us is is that we can chose which filters and decoders we want to use for our avs script.
Start up Graphstudio and press "Ctrl + F" if you look on the picture below you can see all the filters. Be sure you are under the "Directshow filter" section.
http://img236.imageshack.us/img236/3209/92399041gq1.jpg
1. Choose "Haali Media Splitter" it will prompt you for the file, choose the one you want to decode "filename0001.m2ts" (if the movie is splitted over several files learn how to mux them in the Eac3to section.)
2. Choose "ffdshow Video Decoder"
3. Join the two filters by dragging the mouse from "Video" to "In" As shown on the picture below.
http://hdimage.org/images/obruexi1k2h8rtb2deu_graphpic2.jpg
Now save your graph!
AVSP
With AVSP we can make a .avs file for MeGUI i will just cover the basics in this guide!
Open your .grf file that you just made you will have to change it from only viewing .avs files as i do on the picture below.
http://img516.imageshack.us/img516/5049/50265903xe2.jpg
Now we can begin the script
This is more or less how your script will look like when you opened the .grf file
DirectShowSource("C:\your_movie\filename.grf")
Now if you know what framerate the movie is add Fps=XX.XXX to your script, also add Audio=false as we will be focusing on the video stream. (Learn how to find the framerates in the Eac3to section).
This is how it will look
DirectShowSource("C:\your_movie\filename.grf",Audio=false,Fps=23.976)
Now add ConvertToYV12() to your script, this is the color conversion filter.
DirectShowSource("C:\your_movie\filename.grf",Audio=false,Fps=23.976)
ConvertToYV12()
Now we want to crop the movie so find a bright frame in the movie and crop all the black bars away.
If the Aspect Ratio (AR) is not correct you are allowed to undercrop to archieve the right AR.
So start of with this code "Crop(0,0,-0,-0)" Left,Top,-Right-Buttom
You always want to archieve either mod16 or mod4 undercrop if necessary there must be no remainder.
In this example i had to crop the video to 1920x800 Pixels
Code i used "Crop(0,144,-0,-136)"
How to calculate if itīs mod16 or mod4
mod16 "16 divisible With end result with no remainder"
mod4 "4 divisible with end result with no remainder"
Left+right= Pixels making the width 1920- Pixels=
Top+Buttom= Pixels making the height 1080- Pixels=
So in my example i got this result
Width 1920-0=1920 Pixels 1920/16=120
Height 1080-280=800 Pixels 800/16=50
So now our script looks something like this
DirectShowSource("C:\your_movie\filename.grf",Audio=false,Fps=23.976)
ConvertToYV12()
Crop(0,144,-0,-136)
If your video donīt need to be resized and is not interlaced you can save your avs now.
Save as "encode script" and make an extra for your samples save as "Sample script"
Now if you wanna resize the movie to 720p you can use these numbers.
Spline36resize(1280,720) Video source roughly 1920x1080
Spline36resize(1280,688) Video source roughly 1920x1040
Spline36resize(1280,544) Video source 1920x816
Spline36resize(1280,536) Video source 1920x800
Spline36resize(1280,528) video source 1920x800
Choose 536 or 528 depends on which one looks better to you.
Here is my script now
DirectShowSource("C:\your_movie\filename.grf",Audio=false,Fps=23.976)
ConvertToYV12()
Crop(0,144,-0,-136)
Spline36resize(1280,536)
OAR - Original Aspect Ratio
Now when you have resized it be sure that you have OAR or close to OAR.
Here is how you can check how to get OAR. Always go for either mod4 or mod16
Height/Width*1280=OAR
So lets say i have a source that is 1920x1040 after cropping. Height should be 693,33 Well as we canīt uptain that we are going to find the nearest spot. i did that with mod4 height = 692
Now your commandline should look something like this.
DirectShowSource("C:\your_movie\filename.grf",Audio=false,Fps=23.976)
ConvertToYV12()
Crop(0,144,-0,-136)
Spline36resize(1280,692)
LimitedSharpen(Faster)
LimitedSharpen is a avs filter that can sharpen up your source. (Not recommended if you are new to avs scripts)
Applies a strong sharpening to edges and fine details by controlling halos and minimising artifacts. All sharpening is a 3-way struggle between edge-halos,
edge-aliasing, and confetti artifacts; LimitedSharpen is an attempt to minimize each in the most visually pleasing way.
This should be used as the last filter in a chain if possible, like any sharpener. The different Smodes cause different defaults to be used in certain cases,
or limit the ranges differently. See the source for specifics.
Here is an overview of itīs commands.
Smode int (1-4, default 3)
Sharpen mode:
1 : UnsharpMask
2 : Sharpen
3 : Range sharpening.
4 : nonlinear sharpening.
Each mode has progressively less haloing in its sharpening.
ss_x float = 1.5
ss_y float = 1.5
Supersampling when > 1, which reduces aliasing on edges.
dest_x float = width
dest_y float = height
Resize directly to final size, saves a resize step.
strength int = 160
Sharpening strength:
For Smode=1, it can be 0~127 (simple sharpening), 128~255 (simple overdrive), 255~4096 (big overdrive).
For Smode=2, values 0~100 are handled over to Sharpen() as 0.0~1.0. Values >100 are mapped to 100.
For Smode=3, 0~100 is common, but 100~inf. can be used if necessary.
Default is strength=160 for Smode=1, and strength=100 otherwise.
radius int (1-10, default 1)
Radius of UnsharpMask, only used in Smode=1.
overshoot int = 1
Sharpening limit: a sharpened pixel may not exceed [min,max] of the local neighborhood by more than 'overshoot'.
(High values remove the "Limited" of LimitedSharpen, and cause excessive haloing.)
undershoot int = [overshoot]
Separate limit for pixels that get darker during sharpening. Good for cartoons with strong lines. (If not specified,
then undershoot==overshoot.)
Lmode int (0-3, default 1)
Lmode:
0 : No effect
1 : Clamp to over/undershoot.
3 : Zero over/undershoot on edges, normal over/undershoot on not-edges.
soft int (0-100, default 0)
Softens the effect of sharpening. -1 will (try to) autocalculate a good value.
edgemode int (-1 to 2, default 0)
-1 : shows edges.
0 : does nothing.
1 : restricts sharpening to edges
2 : restricts sharpening to mostly non-edges (I think?)
special bool = false
special=true tries to raise detail contrast in dark areas of a frame. (Usage not really recommended - the method is simple
and might cause artefacts.)
wide bool = false
With wide=true, the local [min,max] limits are gathered from the 5x5 neighborhood, instead just 3x3. Enables more gain in
detail contrast. Increasingly useful with increasing supersampling factors.
exborder int (0-16, default 0)
Command example of a basic script
LimitedSharpenFaster(ss_x=1.0, ss_y=1.0, Smode=3, strength=80, overshoot=0)
The final script would look something like this.
DirectShowSource("C:\your_movie\filename.grf",Audio=false,Fps=23.976)
ConvertToYV12()
Crop(0,144,-0,-136)
Spline36resize(1280,536)
LimitedSharpenFaster(ss_x=1.0, ss_y=1.0, Smode=1, strength=60, overshoot=0)
for more info about LSF read this thread LimitedSharpen (http://forum.doom9.org/showthread.php?t=84196&highlight=limitedsharpen)
For more filters visit these sites.
Link 1 (http://avisynth.org/warpenterprises/)
Link 2 (http://avisynth.org/warpenterprises/)
Now there is several of ways to encode with x264 i will show you two ways GUI encoding and Batch encoding.
MeGUI
MeGUI is the program we use as our GUI encoder. This encoder has a graphical interface if you want to encode using Batch goto the next section.
When you start MeGUI it will prompt you if there is any new updates. If you want to use the x264 builds i linked to at the top of the guide
untick the x264 update. If you want to use the x264 builds i recommended copy the x264.exe file to "c:\program files\megui\tools\x264\"
When it prompts about x264 profiles tick the following ones:
x264: DXVA-HD-HQ
x264: DXVA-HD-Insane
x264: Unrestricted (take the ones you think you need)
Now we have that in order we can continue. When you have started MeGUI choose your avisynth script (.avs)
http://hdimage.org/images/d4a7ypsl45urdbg8hh0_meguipic1.jpg
Be sure that File format is set to "MKV"
Now letīs start configuring our x264 encoding settings goto "Config"
I will be focusing on the DXVA-HD-HQ preset and tweak it a little (!When using VBV restrictions you should only be using 2-pass mode.
As 1-pass modes has some compliancy problems at the moment).
DXVA is for Gpu and Hardware support.
here is the basic commandline before i tweaked it for our guide.
program --pass 2 --bitrate XXXX --stats ".stats" --level 4.1 --ref 4 --mixed-refs --bframes 3
--b-rdo --bime --weightb --direct auto --filter -1,-1 --subme 6 --trellis 2 --partitions p8x8,b8x8,i4x4,i8x8
--8x8dct --vbv-bufsize 40000 --vbv-maxrate 50000 --me umh --threads auto --thread-input --progress
--no-psnr --no-ssim --output "output" "input"
Well the first thing i want to do is raise --subme 6 to --subme 7
You can do that under "RC and ME" There you have "subpixel Refinement" you want to set that to "7 - RDO Level 2 (Slower)"
http://hdimage.org/images/zue433dbc7d521qkog1_meguipic2.jpg
The reason for this is that Psy-rdo works very well with --subme 7 it will take some extra time, but we really want to archieve the best result.
In the same section you can change the values of "M.E. Range" Default is "16" You can raise it to 24, 32, 64 higher the better but for movies 24 should really be enough 64 can be used for Anime.
Expect a much longer encoding time when raising these numbers!
If you goto "Main" in x264 in there untick turbo mode. As we want exact same settings for pass 1 and 2. This will take twice the time but will gain you better quality.
make a sample to see if itīs worth the extra time.
http://hdimage.org/images/5zdc5x4jdlqc6db7bcc1_meguipic3.jpg
Now if you donīt care about hitting a specific filesize you can do a Single pass constant Rate Factor encode instead this will provide you with teh same quality through the hole movie.
in x264 go from 2 pass to const. Quality and choose a level between 18-24. 18 being the best.
Tweaking
Well as movies all vary how compressable, grainy and etc.
There are numerous of tweaks you can apply to your settings.
Level 4.1 Compliances:
number of reference frames.
|-----------|---------|
|Resolution | no. ref |
|-----------|---------|
| 1280x544 | 12 |
| 1280x720 | 9 |
| 1920x800 | 5 |
| 1920x816 | 5 |
| 1920x1080 | 4 |
|-----------|---------|
If none of the resolutions above match your source, use the following equation to work it out for yourself:
8388608
__________________
(width x height)
thanks to DarkShikari for the method and techouse for the math
Deblocking:
Movies: the highest you should set it is 0,0 here is some you can try out with.
-1,-1
-1,-2
-2,-1
-3,-3
Anime: With Anime you should always stay on the + side +2,+2 is considered a good setting.
Try and mess around with the settings to see which setting you get the best results from.
Adaptive Quantizers:
A new AQ algorithm can be introduced simply by introducing a new aq_mode and a corresponding if in adaptive_quant_frame.
This also allows quantizer field smoothing, since quantizers are calculated beforehand rather during encoding.
Since there is no more reason for it, aq_mode 1 is removed. The new mode 1 is in a sense a merger of the old modes 1 and 2.
WARNING: This change redefines CRF when using AQ, so output bitrate for a given CRF may be significantly different from before this change!
You can change settings with this command.
Keep in mind that you should keep the settings between 0.7 - 1.5
Adaptive Quantizers can be found under "Advanced" in x264
--aq-strength 0.8
New B-frame Decision
In the new x264 builds there is a new B-frame decision itīs much more precise than the old method. Recommended numbers of B-frames with this new B-frame decision "3-6" B-frames.
If you where using 16 B-frames before it will be considerably slower than with the old method because of the new algorithm use 4-5 B-frames instead.
Command to enable the new B-frame decision.
--b-adapt 2
Now if you donīt want to use it write this command in custom commands in the advanced section of x264
--b-adapt 1