Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Announcements and Chat > General Discussion

Reply
 
Thread Tools Search this Thread Display Modes
Old 5th November 2015, 10:26   #1  |  Link
Chyrka
Registered User
 
Join Date: Oct 2011
Location: Kyiv, Ukraine
Posts: 22
[GUIDE] Fast creation of DVD/BD rip (with chapters) of [anime] serials

Hello, folks.
I am not shure which category it belongs so I just posted it here.

I want to share my guide how to optimize process of making a high-quality rips of anime serials with adding chapters (mostly focused on DVD rips).
It is written with focus on experienced coders who can find some useful tips here, but sometimes it seems too detailed (sorry for this).
You are welcome to criticize and offer better solutions.
Sorry for my English, it is not perfect.

The programs you need:
Avisynth
encoder (x264)
avs2yuv.exe
vfr.py and Python
MKVToolNix
ChapterGrabber
AvsPmod
Notepad++
Eac3to.

Indexing
Indexing is the very first stage in rip production. It is dedicated for creation of special index-file that will be used in the script to access to source files on your HDD. There are three types of indexers for this purpose (all are created by Donald Graft):
1) DGIndex is the indexer for DVDs.
2) DGAVCIndex is the indexer for major types of Blu-ray video.
3) DGIndexNV is the NvidiaGPU-based indexer for both DVDs and Blu-Rays. (you should buy a license)
They have similar interface and usage procedure is almost the same. Before the first call in a script you must place decoder’s dll into the avisynth plugin folder.
Create a common (main) working folder for all subfolders and current files and place there all source files with video or create folders with consequent numeration e.g. DVD01, DVD02, … (BD01,BD02,…) and extract all iso-images to them or copy corresponding VIDEO_TS (BDMV) folders to them.
Run indexer’s exe. Select the biggest group of files with the same index, for example VTS_02_X.VOB,

drag-and-drop them to the indexer’s window and press “OK” button. With using mouse and Left Arrow and Right Arrow for better accuracy seek a needed frame of begin or end of episode and mark it by pressing correspond bracket “[“ or “]”. In the title of the window you also can find information from built-in data about chapters (look at the picture below). Moment, when digits here are changing, usually indicates proper boundary between episodes.

Then press F4 and save index file in the main folder with a name consists of one (1.d2v, 2.d2v, 3.d2v, etc), two (01.d2v, 02.d2v, 03.d2v, etc) or three (001.d2v, 002.d2v, 003.d2v, etc) digits when your series has (1-9); (1-99) or (1-999) episodes. Wait until indexation process is ended up. Move a cursor and specify bounds of another episode and save another index-file. Remember, that video sequence on DVD contains a few episodes, so be careful and don’t index several episodes as a one. Occasionally each episode on a DVD can have his own sequence (group of files) and unique index. So you should drag-and-drop group of files with the same index VTS_YY_X.VOB independently for each episode. And usually there is no need to specify start and finish of the episode as it was necessary in the previous case.

DGIndex also extracts audio files during indexing so these files should be placed in additional folder “Audio” after ending of indexation of all episodes.
The Blu-ray usually has different file for each episode and you should act in the same way as with DVDs, which have separate groups for each episode. But when episodes are joined into the one piece, you should index the entire file without division onto episodes.

Chapters extraction
I recommend using ChapterGrabber for chapter extraction. Run program and go to “Open file” window. Then open *.IFO file with the same index as previously indexed group of files have had, for example VTS_02_0 or one of the playlist-files in the PLAYLIST folder on Blu-Ray.

If chapter file is selected correctly you will get list of chapters in the main window. Otherwise, try to choose another file.

Then you should save chapters-file as txt to the corresponding DVD or BD folder.

If you have joined episodes on Blu-Ray you should additionally select a correct frame-rate (usually it equals 23.976 fps) at the bottom of main window

and save a special QP-file as “qpf.txt” for encoder that determines key-frames at each new scene beginning.

Also you should not do any correction in the saved chapter file and skip the next steps up to Joined m2ts exception.
Chapters correction
The main information that we are looking for in chapters is time of splitting of two parts of the episode which is usually is organized via so-called “eyecatch” (additionally we can be interested in time of opening starting and finishing if each episode has different length of Intro, for example). So, we should correct time of corresponding chapters from continuous to “local” time of each episode by taking away time of each episode beginning. The only one suitable instrument that I found is MKVmerge GUI. Go to “Chapter Editor” menu and load the saved earlier chapter file. Your chapters will be loaded. Now you should recognize a “structure” of chapters and catch a regularity in time marks. Look at an example below.

Each time of “B-part” should be corrected by extraction time of corresponding “Intro” time (episode beginning).

Select “Intro” chapter (1), select time in the field “Start time” press “Ctrl+C” (2), select “B-part” chapter (3) , press the “Adjust timecodes” button (4),

in the opened window type “-” (minus) and paste the copied time (5), press “OK” (6).

The result is here.

Keep in mind this structure, select pair of times for further episodes and extract one from another until the end of the list. Save corrected chapters into new file (xml). Repeat it for each saved file for each source disk.
Script composing
Now is a script writing stage. I won’t tell about filtering, just about serialization.
The best script editor is AvsPmod.
In order to make precise chapters you should write down exact frame numbers for each chapter. The best way is to keep this info directly in a script file as comment in such format:
Code:
#trim(0,359)+trim(360,2831)+trim(2832,17495)+trim(17496,17627)+trim(17628,34235)+trim(34236,34976)+trim(34.977,0)
(Intro+Op+A-Part+Eyecatch+B-Part+Prewiew+Ending)
For example, here 359 is a number of the last frame of the Intro and 360 is a number of the first frame of the Opening. In order to obtain this numbers you should seek it at seeking bar at the bottom of the editor’s window. The previously saved chapter-file will be quite useful here. Open it in a text files editor (Notepad++); find the first non-zero time, copy and paste it in the field with frame number, press “Enter”;

find correct frame of scene change by moving forward or backward; write down corresponding frame number into the above mentioned string. Repeat it for each chapter till the end of the episode. I recommend memorizing numbers of frames that don’t change significantly from episode to episode by clicking Right button of the mouse. Then you can get quick access to these frames by clicking corresponding small black triangles below seeking bar and don’t need to take time from the chapter file.

So, usually, for the next episodes the only needed one is a time between A and B parts that was corrected earlier for each episode. I prefer to additionally define boundaries of the eyecatch even if the only time between A-part and B-part is available. But this is not required; you can just define a single time as on the source.
New chapters creation
When we have done all scripts the next stage is coming. Select all scripts and make copies of them.

Open all copies in the Notepad++ and just make quick remove of “#trim” by “trim” in all opened files. Then create a simple text file ‘chapters.txt’ and write down chapter names. For example:
Intro
Opening
A-part
Eyecatch
B-part
Preview
Ending
For each episode with different structure (for example, final episode is usually without a Preview) you should create an additional ‘chapters_x.txt’ file.
Bat-files creation
We have to use several bat-files for different goals: for automatic new chapter creation, for encoding and for muxing. Create ‘xchap.bat’ and paste the first line:
Code:
vfr.py -f 24/1.001 -c "C:\MG\Chapters\01.txt" -n chapters.txt -q qpf01.txt "01 - Copy.avs"
-f – framerate
-c – chapter files saving path (the destination folder ‘Chapters’ must be created before running)
-n – path to file with chapter names
-q – QP-files saving path
Last one is the input file name.
Duplicate this line for each episode and correct numbers of episodes here "хх - Copy.avs"
Then in Notepad++ with holded “Alt” select by mouse column with episode numbers

And press “Ctrl+C”.
Select in the same way the column of “01”

And paste copied data by “Ctrl+V”. Repeat operation for column with names of QP-files.
‘xchap.bat’ is ready.
Copy vfr.py file to main folder and install Python if it is not installed.
Run ‘xchap.bat’ and get chapters and QP-files for encoding.

Now create ‘xcode.bat’ with similar first line where encoding parameters are defined:
Code:
avs2yuv -raw "01.avs" -o - | x264.exe - --demuxer raw --input-res 960x720 --fps 24000/1001 --input-depth 10 --crf 17 --ref 12 --mixed-refs --deblock 0:0 --bframes 16 --direct auto --min-keyint 24 --keyint 240 --scenecut 40 --rc-lookahead 60 --b-adapt 2 --trellis 2 --weightb --psy-rd 1.0:0.1 --no-mbtree --aq-mode 3 --aq-strength 0.8 --partitions p8x8,b8x8,i4x4,i8x8 --no-fast-pskip --subme 11 --me umh --merange 32 --qcomp 0.6 --vbv-maxrate 50000 --vbv-bufsize 62500 --colorprim bt709 --colormatrix bt709 --transfer bt709 --qpfile qpf01.txt -o 01.264
Repeat operations of copying string and replacing numbers of episodes:


Copy avs2yuv.exe and x264.exe to main folder and run encoding.
And, finally, ‘xmux.bat’.
Write something like this:
Code:
"C:\Program Files (x86)\MKVToolNix\mkvmerge.exe" -o "G:\\Anime\\Hana Yori Dango\\Hana Yori Dango - 01 [DVDRip 984x720 x264 AC3].mkv"  "--forced-track" "0:no" "--compression" "0:none" "-d" "0" "-A" "-S" "-T" "--no-global-tags" "--no-chapters" "F:\\Anime\\Hana Yori Dango\\01.264" "--language" "0:jpn" "--sync" "0:-144" "--forced-track" "0:no" "--compression" "0:none" "-a" "0" "-D" "-S" "-T" "--no-global-tags" "--no-chapters" "F:\\Anime\\Hana Yori Dango\\Audio\\01 T80 2_0ch 448Kbps DELAY -144ms.ac3" "--track-order" "0:0,1:0" "--chapter-language" "jpn" "--chapters" "F:\\Anime\\Hana Yori Dango\\Chapters\\01.txt"
Repeat above mentioned operations where it is possible. Don’t forget to specify correct audio delay of each audio track as it is written in the obtained earlier filename.
Install MKVToolNix if it’s absent and when encoding will be done run ‘xmux.bat’.

Last edited by Chyrka; 3rd October 2016 at 12:26.
Chyrka is offline   Reply With Quote
Old 5th November 2015, 10:27   #2  |  Link
Chyrka
Registered User
 
Join Date: Oct 2011
Location: Kyiv, Ukraine
Posts: 22
Continuation

Joined m2ts exception
If you have encoded Blu-ray with episodes joined into a single m2ts file, after indexing and chapter extraction you will have three files in each BD folder: the index-file, the file with chapters and the QP-file.
Now create a script-file (by the AvsPmod editor) and and save it in the same folder as “01.avs” for example.
After that we can move to encoding preparation. Let’s start chapters. We can copy or modify the created earlier file. I prefer to copy it and have a backup. Open it in the Notepad. It looks like this

And here we should recognize a “structure” of chapters and catch a regularity in time marks. After that just type the proper names for each of them up to the end of the file.

Now create ‘xcode.bat’ with the single line where encoding parameters are defined:
Code:
avs2yuv -raw "01.avs" -o - | x264.exe - --demuxer raw --input-res 1920x1080 --fps 24000/1001 --input-depth 10 --crf 16 --ref 4 --mixed-refs --deblock 0:0 --bframes 16 --direct auto --min-keyint 24 --keyint 240 --scenecut 40 --rc-lookahead 60 --b-adapt 2 --trellis 2 --weightb --psy-rd 1.0:0.1 --no-mbtree --aq-mode 1 --aq-strength 0.8 --partitions p8x8,b8x8,i4x4,i8x8 --no-fast-pskip --subme 11 --me umh --merange 32 --qcomp 0.6 --qpmax 40 --vbv-maxrate 50000 --vbv-bufsize 62500 --colorprim bt709 --colormatrix bt709 --transfer bt709 --qpfile qpf.txt -o 01.264
Encode it.
The next step is audio streams extraction that is described in the section Extraction of HD audio from m2ts for BD rips.
Final step is muxing. In this case I prefer to use MKVMerge GUI.
Run MKVMerge GUI and drop the encoded video file and extracted audio file here.
Now is the most important part of the muxing set-up process. Move to the tab “Global”, in the “Splitting” section set splitting by timecodes and type corresponding times of each episode beginning. Also we should specify the chapter-file in the “Chapters” section.

Now we can start muxing. After finishing the process, we obtain a set of files for each episode 01-00x.mkv. They should be renamed with proper names and episode numbers manually or by yet another bat-file.
Do the same operations for each BD.
Extraction of HD audio from m2ts for BD rips
Unlike DVD indexer that perfectly extract DVD audio, HD audio is better to extract by eac3to.
If you have joined episodes on the Blu-ray, I propose to use eac3to with GUI UsEac3To. Drag-and-drop the main *.m2ts file to the main window of the program. Wait till the scanning complete.

Now look at the number of interested stream (1), set it in the droplist (2), chose desirable format (3), press “Add” button (4) and run the command line by the button (5). Wait for the finish of encoding.
In the more usual case with separated files for each episode it is better to use command line.
Firstly, find the number of the audio stream by using GUI UsEac3To as it is described above (step (1)). After that create a file ‘xtract.bat’ with the next line (as example)
Code:
"C:\Program Files (x86)\eac3to\eac3to.exe" "G:\Anime\Chihayafuru_S2\CHIHAYAFURU_2_1\BDMV\STREAM\00003.m2ts" 3:"G:\Anime\Chihayafuru_S2\Orig Sound\01.flac"
Replicate this line for each episode and modify episode numbers and corresponding disc indexes.
Finally, run this bat-file before muxing of the rip.
Good luck.
Chyrka is offline   Reply With Quote
Reply

Tags
anime, batch encoding, chapters, dvd ripping, scripting

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 09:32.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2017, vBulletin Solutions Inc.