Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
|
|
Thread Tools | Search this Thread | Display Modes |
17th September 2011, 01:51 | #102 | Link |
Registered User
Join Date: Aug 2007
Posts: 218
|
@mp3dom @kolak @poisondeathray
I just checked the qt format spec and found the reason. mp3dom was right and I was wrong, the offset should be 0x24 = 36. This mdat chunk has 64bit length - that is, the 4 bytes before "mdat" is "00 00 00 01", and a 8-byte length follows the tag, real data then follows the length. So file_head should be 8 bytes after mdat.
__________________
f3kdb 1.5.1 / MP_Pipeline 0.18 ffms2 builds with 10bit output hack: libav-9a60b1f / ffmpeg-1e4d049 / FFmbc-0.7.1 Built from ffms2 6e0d654 (hack a9fe004) Mirrors: http://bit.ly/19TwDD3 |
17th September 2011, 02:03 | #103 | Link |
Registered User
Join Date: Aug 2007
Posts: 218
|
v210 is an uncompressed format, so recompress the file won't change the data.
__________________
f3kdb 1.5.1 / MP_Pipeline 0.18 ffms2 builds with 10bit output hack: libav-9a60b1f / ffmpeg-1e4d049 / FFmbc-0.7.1 Built from ffms2 6e0d654 (hack a9fe004) Mirrors: http://bit.ly/19TwDD3 |
17th September 2011, 11:37 | #104 | Link |
Registered User
Join Date: Jan 2006
Posts: 1,867
|
@Sap...
That's great, I wrote the original script. Can I package it up with your credit? I wanted to know the quick way to find the file head, but I didn't want to read the huge manual.. now I know and I think I can fix this in script to be fully automatic, which is great! I think we're really making progress on some quite usable high bit depth processing, even ffmeg is going to have ProRes soon... ps what would be really useful is a type of parsing file reader that you could say fileread(int a, float b, str c ) etc. something like that, but choose word length to read only bytes, and nops to skip bytes. We need to read data files. Maybe a version of sashimi that reads to script variables instead of clips? Last edited by jmac698; 17th September 2011 at 11:40. |
17th September 2011, 12:20 | #105 | Link | |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,843
|
Quote:
btw...I compared (just quickly for now) quality and it looks very good and promising. Conversion was slow thought 1.5fps for full HD, but using only one core. Not sure how I can make it using many cores, but in theory it can run at acceptable speed. Looking good Andrew Last edited by kolak; 17th September 2011 at 12:25. |
|
17th September 2011, 12:47 | #106 | Link | |
Registered User
Join Date: Aug 2007
Posts: 218
|
Quote:
I don't understand the second part. Do you want to directly parse the file header in script? I think that will be very hard and hackish since AviSynth lacks some useful features (like loops). A better method is integrate a more powerful scripting environment into it. Actually I am thinking of creating a python plugin environment, but I don't have much time to do it recently...
__________________
f3kdb 1.5.1 / MP_Pipeline 0.18 ffms2 builds with 10bit output hack: libav-9a60b1f / ffmpeg-1e4d049 / FFmbc-0.7.1 Built from ffms2 6e0d654 (hack a9fe004) Mirrors: http://bit.ly/19TwDD3 |
|
17th September 2011, 12:51 | #107 | Link |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,843
|
Another thing- is this Floyd dithering just pure implementation?
Filter which I compared it to use noise shaping/error diffusion plus optional random noise. When noise is added, an R250 random number generator is used. The noise is injected into the quantizer, not into the signal, so the error propagated to the adjacent pixels does not include the random noise. Can we do the same- is it good idea? Andrew |
17th September 2011, 12:55 | #108 | Link | |
Registered User
Join Date: Aug 2007
Posts: 218
|
Quote:
It is hard to get speed improvement using pure script implementation. Better to integrate it into a source plugin, like Sashimi.
__________________
f3kdb 1.5.1 / MP_Pipeline 0.18 ffms2 builds with 10bit output hack: libav-9a60b1f / ffmpeg-1e4d049 / FFmbc-0.7.1 Built from ffms2 6e0d654 (hack a9fe004) Mirrors: http://bit.ly/19TwDD3 |
|
17th September 2011, 12:58 | #109 | Link | |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,843
|
Quote:
Tried 2 totally different files- without swpuv colors are wrong. Andrew Last edited by kolak; 17th September 2011 at 13:03. |
|
17th September 2011, 13:00 | #110 | Link | |
Registered User
Join Date: Aug 2007
Posts: 218
|
Quote:
__________________
f3kdb 1.5.1 / MP_Pipeline 0.18 ffms2 builds with 10bit output hack: libav-9a60b1f / ffmpeg-1e4d049 / FFmbc-0.7.1 Built from ffms2 6e0d654 (hack a9fe004) Mirrors: http://bit.ly/19TwDD3 |
|
17th September 2011, 13:02 | #111 | Link | |
Registered User
Join Date: Jul 2003
Location: Italy
Posts: 1,135
|
No, wrong colors was with wrong offset (0x1c) and in that case the SwapUV fixed it. With right offset (0x24) the colors are ok from the start.
Quote:
Is it possible (in a future release, without hurry) to implement it as an optional parameter? About f3kdb, isn't it already working on 16bit clip as source? Or are you referring to the deband part (flash3kyuu)? In that case it would be very helpful to deband a 16bit clip and then dither to 8bit. Last edited by mp3dom; 17th September 2011 at 13:13. |
|
17th September 2011, 13:03 | #112 | Link |
Registered User
Join Date: Aug 2007
Posts: 218
|
There is 2 types of mdat chunk. If the 4 bytes before mdat is 00 00 00 01, you need to add 8 to the offset after mdat. Otherwise the original offset is correct.
__________________
f3kdb 1.5.1 / MP_Pipeline 0.18 ffms2 builds with 10bit output hack: libav-9a60b1f / ffmpeg-1e4d049 / FFmbc-0.7.1 Built from ffms2 6e0d654 (hack a9fe004) Mirrors: http://bit.ly/19TwDD3 |
17th September 2011, 13:05 | #113 | Link | |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,843
|
Quote:
Yes- my files don't have 00 00 00 01, so "normal" offset works fine. Last edited by kolak; 17th September 2011 at 13:22. |
|
17th September 2011, 13:32 | #114 | Link | |
Registered User
Join Date: Aug 2007
Posts: 218
|
Quote:
__________________
f3kdb 1.5.1 / MP_Pipeline 0.18 ffms2 builds with 10bit output hack: libav-9a60b1f / ffmpeg-1e4d049 / FFmbc-0.7.1 Built from ffms2 6e0d654 (hack a9fe004) Mirrors: http://bit.ly/19TwDD3 |
|
17th September 2011, 18:40 | #115 | Link |
Registered User
Join Date: Jan 2006
Posts: 1,867
|
Yes the script can be sped up alot, that is the reason I requested some new features in sashimi. Those features were finalized just yesterday, so now it is time to rewrite readv210 with the new features, which should speed up load the files a lot.
My understanding of dither is that it can be fixed or random. I read a whole textbook on it once (long time ago), and it seems that uncorrelated noise with shaping looked best. That old floyd-steinberg is easy to understand but causes it's own banding (there's only 5 shades for 4 dots). Btw, I worked on my own algorithm for fixed dither, that always sums to the target value for the area, meaning you could perfectly reverse it back into a high bit-depth image at full resolution! It can't always work for some pixels though. As for the file parser, I could use this idea for other things, speed is not an issue, like reading the histograms from colorlike per frame, it's only 256 numbers of text, or reading the in-pattern results in dgindex, just one line per frame.. and in this case, reading 8 bytes to determine file header - such a siimple task, that I don't want to write a whole plugin just for those numbers... because ultimiately I find coding up an idea in script far, far faster to get a practical result (even though it may be slow). And yes you can do loops in avisythn, however I don't need them to parse files, the plugin will have some simple logic for that. |
17th September 2011, 19:25 | #117 | Link |
Registered User
Join Date: Jan 2006
Posts: 1,867
|
recursion and gscript are documented here
http://avisynth.org/mediawiki/Advanced_Scripting_Tips |
17th September 2011, 19:42 | #118 | Link | |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,843
|
Quote:
To be honest- going from 10 to 8bit with MS filter looks good enough- I would rather like to have more speed. What about these methods: Jarvis, Judice, and Ninke dithering diffuses the error also to pixels one step further away. The dithering is coarser, but has fewer visual artifacts. It is slower than Floyd–Steinberg dithering because it distributes errors among 12 nearby pixels instead of 4 nearby pixels for Floyd–Steinberg. Stucki dithering is based on the above, but is slightly faster. Its output tends to be clean and sharp. Burkes dithering is a simplified form of Stucki dithering that is faster, but less clean than Stucki dithering. As a goal I would see 2x faster than RT dithering for full HD on eg. 12 core PC, with quality as FS. Andrew |
|
17th September 2011, 19:47 | #119 | Link |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,843
|
Current solution can give about 12fps on 8 core machine (older Xeons- not i7)- just need to run many instances. MS filter is a bit faster.
Andrew Last edited by kolak; 17th September 2011 at 19:54. |
Tags |
avisynth, deband |
Thread Tools | Search this Thread |
Display Modes | |
|
|