Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
26th April 2004, 10:57 | #1 | Link |
Registered User
Join Date: Dec 2003
Posts: 147
|
DCTune Custom Matrix Discussion (thread-split)
I have decided not to continue work on matrices at the moment, because I think I have a better strategy: calculate a new matrix for every encode by using DCTune on about 200+ frames of the source and averaging the results. Tests have been encouraging.
__________________
Stuff was here at some point. Last edited by Teegedeck; 1st May 2004 at 13:49. |
26th April 2004, 11:04 | #2 | Link | |
BeHappy/MeGUI developer
Join Date: Oct 2003
Location: Moscow, Russia
Posts: 1,727
|
Quote:
__________________
BeHappy - AviSynth-based audio transcoding tool Audio encoding via AviSynth On2 VP7 is great in quality but it is unusable for long-term video backup puposes! Sincerely Yours, MCPD/MCTS |
|
26th April 2004, 11:40 | #4 | Link | |
BeHappy/MeGUI developer
Join Date: Oct 2003
Location: Moscow, Russia
Posts: 1,727
|
Quote:
__________________
BeHappy - AviSynth-based audio transcoding tool Audio encoding via AviSynth On2 VP7 is great in quality but it is unusable for long-term video backup puposes! Sincerely Yours, MCPD/MCTS |
|
26th April 2004, 15:56 | #5 | Link |
Registered User
Join Date: Dec 2003
Posts: 147
|
It would be awesome if the process could be automated, but I don't really have a clue how to do it at the moment. It's already automated for the most part (but in different steps).
My chain of processing goes like this: 1. Use Avisynth to create frames in native DCTune format (.ppm). Example script: Code:
SelectEvery(last, 1000, 1) ConvertToRGB() ImageWriter(last, file = "C:\dctune_raw\yoursource", start = 0, end = 0, type = "ppm") 2. Use BathEncoder to call DCTune on all of the images. Simply drag and drop the files into it, set your encoding switches, and hit "go". BathEncoder creates a batchfile which calls DCTune on every image. You can get it here. Be sure to put DCTune in the same folder. The switches I normally use for DCTune are: dctune2.0 -f <infile> -p 1 -inches 8 -dpi 133 -o <outfile>_quants_p1.txt -j <outfile>_p1.jpg Note the -p value. This is basically your quality setting. -p 1 is normally perceptually lossless. For a high-bitrate encode, I normally use this to create my intra-matrix; then I run it a second time with -p 2 and change the outputs accordingly, and use those to create my inter-matrix. In general, I use the following p-values: High bitrate: 1, 2 Medium bitrat: 2, 3 Low bitrate: 3, 4 The high bitrate matrices also look good at low bitrate, but you have a higher average quantizer (so it looks a little ragged). 3. Average the results to get a good quantization matrix. I'm currently using a pretty simplistic program I wrote in Java to do this. I'm not really a programmer, so I have no clue how to export it at the moment. I *might* be able to turn it into an applet.
__________________
Stuff was here at some point. Last edited by SoonUDie; 26th April 2004 at 15:59. |
26th April 2004, 16:20 | #6 | Link | |||||
BeHappy/MeGUI developer
Join Date: Oct 2003
Location: Moscow, Russia
Posts: 1,727
|
Quote:
=================== Done! Create such directory structure: Quote:
place extract.bat into c:\dctune\ folder Quote:
Quote:
Quote:
1-st - extract frames extract.bat <source_avs_file_name_and_path> <frame_denominator> example: extract c:\dvdrip\gigli\gigli.avs 1000 2-nd - calculate qmatrix calculate.bat <intra_quality>, <inter_quality> example: calculate 2 3 3-td - got Your matrix c:\dctune\result.qmatrix
__________________
BeHappy - AviSynth-based audio transcoding tool Audio encoding via AviSynth On2 VP7 is great in quality but it is unusable for long-term video backup puposes! Sincerely Yours, MCPD/MCTS Last edited by dimzon; 27th April 2004 at 12:29. |
|||||
26th April 2004, 18:59 | #7 | Link |
Registered User
Join Date: Dec 2003
Posts: 147
|
I bow before your skills.
Another note: you should probably go through and manually check all the frames, and delete any ones that you don't want to use. You should definately delete blank frames (all black or all white), and you may want to cut out any credits as well.
__________________
Stuff was here at some point. Last edited by SoonUDie; 26th April 2004 at 19:12. |
26th April 2004, 20:22 | #8 | Link |
Registered User
Join Date: Jan 2003
Posts: 6
|
This is really cool. I am in the middle of testing it on Iron Maiden - Rock In Rio, which is the hardiest dvd to encode I have ever seen.
I did have to change a couple of lines in the calculate.vbs to get it to work. In the function ReadMatrixFromFile, where the double for loop is, your way of dividing the numbers into the array aLine did not work, some post in it where just blanks and the CLng then returned an error. instead I simply made aLine to a string and implemented the inner for loop a little different and removed the lines with replace calls in the outer for loop. So here is the loop solution that worked for me: for i=0 to 7 ' 8 lines total s = oFile.ReadLine for j=0 to 7 aLine = trim(mid(s,j*4+1,4)) aResult(i*8+j)=CLng(aLine) next next You should update your script with that. Again, thanks for this much easier automatic solution. Last edited by vaxis; 26th April 2004 at 22:05. |
27th April 2004, 00:41 | #9 | Link |
Registered User
Join Date: Dec 2003
Posts: 147
|
I hope it works out for you. In general, I've found that it seems to produce a very good amount of detail - more than HVS Best is most cases. The main plus side, in my mind, is that the matrix is totally adapted to the source. I think the only way you could improve upon this general method would be if XviD automatically calculated and used a different (optimized) quant matrix for each individual scene.
__________________
Stuff was here at some point. |
27th April 2004, 00:45 | #10 | Link |
Registered User
Join Date: Dec 2003
Posts: 147
|
Oh, I should mention that DCTune produces matrices with a lowest possible value of 2. You may want to implement a way to multiply all the other values accordingly, so that you always get 8 in the upperp left corner. If not, you can use excel and a little bit of math once you have the final matrix.
__________________
Stuff was here at some point. |
27th April 2004, 01:26 | #11 | Link |
Guest
Posts: n/a
|
Just to make sure I'm doing this right, when DCTune outputs the txt file like the one below, I use the top matrix right?
# Quantization Matrices for G:\000000.PPM # Using DCTune Version 2.0 # Quantization Table Calculated Using Perceptual Optimization # Target Perceptual Error 2.000000 # Mean Luminance: 33.532667 Candelas/meter^2 # Peak Luminance: 67.162400 Candelas/meter^2 # Number Gray Steps: 256 # Pixels/Degree X: 18.666667 # Pixels/Degree Y: 18.666667 # Size of DCT: 8 4 4 4 4 4 6 255 255 4 2 4 2 4 4 255 255 4 4 6 4 4 6 255 255 4 4 4 6 10 255 255 255 4 4 4 10 255 255 255 255 6 4 255 255 255 255 255 255 8 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 # Quantization Table Calculated Using Perceptual Optimization # Target Perceptual Error 2.000000 # Mean Luminance: 33.532667 Candelas/meter^2 # Peak Luminance: 67.162400 Candelas/meter^2 # Number Gray Steps: 256 # Pixels/Degree X: 9.333333 # Pixels/Degree Y: 9.333333 # Size of DCT: 8 18 255 255 255 255 255 255 255 24 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 # Quantization Table Calculated Using Perceptual Optimization # Target Perceptual Error 2.000000 # Mean Luminance: 33.532667 Candelas/meter^2 # Peak Luminance: 67.162400 Candelas/meter^2 # Number Gray Steps: 256 # Pixels/Degree X: 9.333333 # Pixels/Degree Y: 9.333333 # Size of DCT: 8 8 255 255 255 255 255 255 255 8 255 255 255 255 255 255 255 8 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 255 |
27th April 2004, 02:53 | #13 | Link | |
Registered User
Join Date: Dec 2003
Posts: 147
|
Quote:
__________________
Stuff was here at some point. |
|
27th April 2004, 08:10 | #15 | Link | |
Registered User
Join Date: Dec 2003
Posts: 147
|
Quote:
The difference between using a generalized matrix like HVS-best and calculating an average matrix using DCTune is that, theoretically, the averaging method will create a perceptual matrix adapted to the source's quality, and therefore better for purposes of compression, retaining the same amount of quality with less bits. Of course, this assumes that DCTune's alogrithms for creating an optimized matrix are valid. As an example of the DCTune method, here is the average matrix I got for the SeaBisucit R1 (widescreen) DVD at p=1 (~400 frames sampled, using my program to calculate the matrix): Code:
8 8 8 10 11 16 86 255 8 8 9 9 10 28 106 255 8 9 16 21 35 77 255 255 8 9 20 60 87 174 255 255 10 10 23 86 149 255 255 255 15 16 57 118 255 255 255 255 26 22 92 185 255 255 255 255 41 59 166 255 255 255 255 255
__________________
Stuff was here at some point. Last edited by SoonUDie; 4th May 2004 at 04:12. |
|
27th April 2004, 09:31 | #16 | Link | |||||
BeHappy/MeGUI developer
Join Date: Oct 2003
Location: Moscow, Russia
Posts: 1,727
|
Quote:
Quote:
Quote:
Quote:
anywhere Your solution works fine too (but i'm too lazy to analyze quantizers position into source string) and some user nickname=Shade PM-ed me such info about DCTune (i have no idea why he PM-ed to me but not to post this to this thread so I repost it): Quote:
__________________
BeHappy - AviSynth-based audio transcoding tool Audio encoding via AviSynth On2 VP7 is great in quality but it is unusable for long-term video backup puposes! Sincerely Yours, MCPD/MCTS Last edited by dimzon; 27th April 2004 at 09:40. |
|||||
27th April 2004, 10:42 | #17 | Link | ||
Registered User
Join Date: Dec 2003
Posts: 147
|
Quote:
Quote:
Also, you can stop using the Evil Matrix now - it's served its purpose as a test, and I think my current method far outstripes it in creating a quality encode. I would appreciate it if you try my new method On another point, I went through the forums using search, and DCTune has been discussed in relation to XviD a few times before (though not in this capacity or detail), and some people came to the same conclusion I have - it would be awesome if we could use use DCTune to optimize individual scenes, or even frames. For example: call DCTune on each i-frame, so that each scene gets the most appropriate matrix. I know that XviD supports (or at least, used to support) changing quantizations like this, but I think it's not mpeg4 compliant.
__________________
Stuff was here at some point. |
||
27th April 2004, 10:58 | #18 | Link | |
BeHappy/MeGUI developer
Join Date: Oct 2003
Location: Moscow, Russia
Posts: 1,727
|
Quote:
1) multiply all the other values on (8/top_left_intra_value) OR 2) increase all the other values on (8-top_left_intra_value) OR 3) set values=8 where values is < 8
__________________
BeHappy - AviSynth-based audio transcoding tool Audio encoding via AviSynth On2 VP7 is great in quality but it is unusable for long-term video backup puposes! Sincerely Yours, MCPD/MCTS |
|
27th April 2004, 12:34 | #19 | Link | |
BeHappy/MeGUI developer
Join Date: Oct 2003
Location: Moscow, Russia
Posts: 1,727
|
I have just updated calculate.vbs in my post to follow MPEG specs.
That's my quantizer matrix for "KNOCKIN' ON HEAVEN'S DOOR" movie Quote:
__________________
BeHappy - AviSynth-based audio transcoding tool Audio encoding via AviSynth On2 VP7 is great in quality but it is unusable for long-term video backup puposes! Sincerely Yours, MCPD/MCTS |
|
27th April 2004, 14:08 | #20 | Link |
Registered User
Join Date: Nov 2002
Posts: 32
|
@dimzon
How do I have to modify the bat files to make them work flawlessly under WIN98SE,'cause I get a "not valid option /Q" when I run extract.bat and calculate.bat. Moreover,due to calculate.vbs, I get a runtime error of M$ Vbscript : "activeX is not able to create the object 'MSXML2.DOMDocument.3.0'".And of course, result.qmatrix is not created. Thanks. |
|
|