Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
|
|
Thread Tools | Search this Thread | Display Modes |
28th December 2005, 03:13 | #21 | Link |
interlace this!
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
|
@ tateu:
fascinating. looks like QT/blackmagic has a dirty little secret? that chroma drift looks to have been created by multiple yuy2-rgb conversions (and it drifts to the side! that means even with 1 generation there's going to be a chroma offset... i mean come on, is it that hard to convert colourspaces without moving stuff around?) that's good though - that means us avisynthers can make as many generations as we like without loss. though i fear for 10-bit files the same losses will occur (it'd be cool to dither 10-bit to 8-bit on import, but that's for another time. and when avs 3.0 comes along it'd be much nicer to just use it's 15 bit colourspace on 10 bit files). looks like i'll be doing all my capping in 8 bit for now then thanks heaps for your import plugin - it's nice to know there's more quality to be had from quicktime than what quicktime itself actually provides.
__________________
sucking the life out of your videos since 2004 |
28th December 2005, 05:43 | #22 | Link |
Registered User
Join Date: Jan 2002
Location: Los Angeles, CA USA
Posts: 132
|
Yes, it seems strange that there would be that much of a shift. I expected lossless in, lossless out (I'm still hoping that someone proves me wrong and shows me some checkbox I forgot to select). As I delve deeper and deeper into quicktime and omf, I'm finding that I mostly prefer plain old AVIs, I seem to have fewer problems.
10bit...yes, I was looking for a way to do a raw import, but I don't know enough to write my own 10bit to 8bit function from scratch. If I can find an open source one I'll add it in. The raw import still needs some work. I need to figure out how to accurately determine the frame header size so I can skip over it. Right now it's just a guess. A call to GetMediaSample returns the raw data and the size of the data in bytes. I know the height and width of the file which allows me to determine the actual size in bytes of 1 frame. I then assume the header size is equal to (Raw data size) - (actual frame size) and skip over that amount. As an Example using 720x486, progressive, Avid AV1x codec: GetMediaSampleSize = 720,896 YUY2 FrameSize = 699,840 = 720 * 486 * 2 Assumed Header Size = 21056 = 720,896 - 699,840 This method does not work on every file I have tried and it definitely does not work for separate field interlaced files because each field has it's own header data and GetMediaSample returns the entire frame, not each field. I'm still looking for a foolproof solution. And I found a bug that shows up occasionally which sets the incorrect fps. I used the wrong values in my calculation. I used MovieTimeScale / GetMovieNextInterestingTimeDuration when it looks like it might be better to use MediaTimeScale / MediaSampleDuration. Quicktime doesn't have an fps parameter, it has timescale and sampleduration which seem to act kind of like fps.numerator and fps.denominator. The movie as a whole has a timescale and each piece of media (video track, audio track, etc.) has it's own timescale. I was calling GetMovieNextInterestingTime and using it's duration as my fps denominator but this failed on one file I tested. MovieTimeScale was 2997 and the first call to GetMovieNextInterestingTime was 64 resulting in 46.83fps. If I had used MediaTimeScale / MediaSampleDuration it would have been 600 / 20 = 30fps. I don't fully understand the concept of sampleduration, though, so even this method may not be 100% correct. One 30fps file that I have, has a MediaTimeScale of 600 so it seems that each frame should have a sample duration of 20, but the frames actually ranged in duration from 19-22. It wasn't meant to be a variable frame rate file, it was just a DV25 file rendered out of Final Cut Pro on a Mac. December is always my busiest time of year so I won't have time to work on any of these issues until the beginning of next Jan. Last edited by tateu; 28th December 2005 at 06:04. |
3rd January 2006, 06:37 | #23 | Link |
interlace this!
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
|
i wonder if it'd be possible to reverse-engineer output of some uncompressed formats?
because even though we can decode "lossless" (i'm growing more and more wary of this term when dealing with quicktime), we can't export back to it without loss. with the example of blackmagic 2Vuy, my preferred method is to export via ffdshow a uyvy avi file. the quicktime blackmagic codecs can load this without fuss, so i save it out from there. in theory there's no yuy2 to rgb conversion happening, at least in theory. now with the QTsource plugins, we can also export straight out of our scripts by just playing through. now this would seem an even safer way to preserve the colourspace throughout the chain. however, i did a little test where i made a mov with both these techniques. they both loaded fine in QT and avs, but when subtracted from the source, it was clear there were differences. the avs -> ffdshow uyvy -> mov file showed evidence of colourspace conversion (it was blurred and slightly shifted). the avs -> QToutput -> mov file showed no blurring that i could tell, but a luma change of some kind (i haven't tracked down the numbers yet). it doesn't seem to be the standard 8-235 clamping. i think if this is sorted, we might finally have a way of working with quicktime that's truly lossless. [edit] i think it's using different colour coefficients... the reds are sort of orange when i view it in QT player. that means it's almost certainly converting colourspace. bugger it.
__________________
sucking the life out of your videos since 2004 Last edited by Mug Funky; 3rd January 2006 at 06:52. |
3rd January 2006, 09:25 | #24 | Link |
the dumbest
Join Date: Oct 2002
Location: Malvinas
Posts: 494
|
one more of the hundred reasons why I insist on not using quicktime despite what Mac fans tell about about its "superior image quality"
Anyway I respect it as a container, problem is it is so closed that you can't never really tell what is going on inside it...... |
3rd January 2006, 10:55 | #25 | Link | |
Registered User
Join Date: Feb 2002
Location: Germany
Posts: 540
|
Quote:
To avoid confusion between 8-bit and 10-bit representations, the eight most-significant bits are considered to be an integer part while the two additional bits, if present, are considered to be fractional parts. So just take the eight most-significant bits and you got your 8->10 bit conversion. hanfrunz |
|
3rd January 2006, 12:34 | #26 | Link | |
interlace this!
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
|
Quote:
@ hanfrunz: true, true. i'd like to see dither added too - it might help repress blocking when encoding to mpeg-2 (or anything else for that matter). certainly a nicer kind of noise than blockbuster gives.
__________________
sucking the life out of your videos since 2004 |
|
3rd January 2006, 13:41 | #27 | Link |
Registered User
Join Date: Nov 2001
Posts: 9,770
|
tateu: i have upload a .mp4 file with h.264 video here. it has been created with x264
it plays in quicktime (jerkily) but when i try to open it via your plugin it only shows me a green picture. the funny thing is i also cant open it via the cyberlink dshow qt source filter (coming with qt alternative) any idea why it doesnt work?
__________________
Between the weak and the strong one it is the freedom which oppresses and the law that liberates (Jean Jacques Rousseau) I know, that I know nothing (Socrates) MPEG-4 ASP FAQ | AVC/H.264 FAQ | AAC FAQ | MP4 FAQ | MP4Menu stores DVD Menus in MP4 (guide) Ogg Theora | Ogg Vorbis use WM9 today and get Micro$oft controlling the A/V market tomorrow for free |
4th January 2006, 10:09 | #28 | Link |
Registered User
Join Date: Jan 2002
Location: Los Angeles, CA USA
Posts: 132
|
Mug Funky,
This document, http://developer.apple.com/quicktime...spatch019.html has a lot of info on quicktime and yuv. There are a few ImageDescription Extensions (colr & nclc) which describe color matrixes and transfer functions. I'm muddling my way through and hoping to find something useful. I've also found that mencoder using quicktime codecs in YUY2 mode does a much better job than my filter of decoding to YUY2 with the correct color output (still not perfect and some filters seem to only work in RGB). Mencoder/mplayer uses yet another quicktime function for decoding frames (ImageCodecBandDecompress) that I have not tried yet. I'll have to look into it some more. I'm definately in agreement with morsa...Quicktime is really starting to get annoying...there are way too many functions available to decompress a frame of video (ImageCodecBandDecompress, DecompressSequenceFrameWhen, MoviesTask, DecompressImage, ICMDecompressionSessionDecodeFrame, DataCodecDecompress, FDecompressImage and probably others that I haven't found yet). Bond, Sorry, I remember typing (in notepad) a reply to your question about mplayer and quicktime but I guess I never actually logged on and posted it. Mplayer has a quicktime demuxer which reads the properties of the quicktime file (FourCC, height, etc.). It then has a set of codecs that it can use to decode the video. "Codecs.conf" is used to determine which codec to use for each FourCC. If you play a Sorenson Video 3 quicktime with mplayer, the demuxer will determine that the FourCC is SVQ3 and will search through "codecs.conf" for a compatible decoder. The first one listed is ffmpeg, so mplayer will grab compressed media samples from the quicktime file and feed them to the ffmpeg SVQ3 decoder. If you remove or comment out the ffmpeg decoder entry, it will try to use the next one in the list, "Win32/QuickTime SVQ3 decoder." I have not been successful using the quicktime decoders with mplayer but they seem to work with mencoder. I added an entry for Blackmagic's 10Bit codec (v210) and mencoder successfully re-encoded the file to mpeg4. I tried "mplayer -vo directx SVQ3.mov" which crashes and "mencoder SVQ3.mov -ovc lavc -lavcopts vcodec=mpeg4 -ffourcc XVID -o SVQ3.avi" which successfully encodes an mpeg4 avi. As an example of a new mplayer/mencoder codec, here's the entry I added for the Blackmagic's 10Bit codec: videocodec qtv210 info "Win32/QuickTime Blackmagic 10Bit" status untested fourcc v210 driver qtvideo dll "BlackmagicCodec.qtx" out BGR32 About h264...There are two problems there: 1) My method of returning the frame count doesn't seem to work with your sample file and always returns a value of 1. 2) It looks like I am using older APIs that cannot handle B-Frames, the error message that quicktime gives me is "kQTMediaHasDisplayOffsetsErr = -9461, /* Returned by old (non-B-frame-aware) media APIs that cannot cope with nonzero decode to display deltas on media that contain them. */." My guess is that the cyberlink filter has a similar problem. I'll have to revert back to my first method of returning the framecount (GetMediaSampleCount) and figure out the newer quicktime 7 video APIs (ICMDecompressionSession). hanfrunz, Thanks for that info. I now have Blackmagic 10Bit (v210) raw decoding added in, using the 8 most significant bits. I should hopefully have a new version available sometime in the next few days. It will add 10Bit decoding and fix the fps error. I think I may end up needing to start a new filter from scratch to clean everything up and try to get the colorspace and h264 issues resolved correctly. Right now, I just keep dumping code into the middle of an already messy project and I'm starting to get lost in it. Last edited by tateu; 4th January 2006 at 10:12. |
4th January 2006, 21:34 | #29 | Link | |
Registered User
Join Date: Nov 2001
Posts: 9,770
|
Quote:
__________________
Between the weak and the strong one it is the freedom which oppresses and the law that liberates (Jean Jacques Rousseau) I know, that I know nothing (Socrates) MPEG-4 ASP FAQ | AVC/H.264 FAQ | AAC FAQ | MP4 FAQ | MP4Menu stores DVD Menus in MP4 (guide) Ogg Theora | Ogg Vorbis use WM9 today and get Micro$oft controlling the A/V market tomorrow for free |
|
4th January 2006, 22:54 | #30 | Link |
Registered User
Join Date: Jan 2002
Location: Los Angeles, CA USA
Posts: 132
|
Pretty much the same here, mplayer crashes and I get nothing but black output from mencoder. I've tried the MPlayer-mingw32-dev-CVS-050928 build from mplayerhq.hu and CelticDruid builds from Aug, Nov and Jan 06. It could be the same problem as my filter: the quicktime function mplayer uses (ImageCodecBandDecompress) may not be able to handle B-Frames.
I'm using this as my config: videocodec qth264 info "Win32/QuickTime h264" status untested fourcc H264,h264 fourcc avc1 driver qtvideo dll "QuickTimeH264.qtx" out YV12,I420,IYUV |
4th January 2006, 23:15 | #31 | Link |
Registered User
Join Date: Nov 2001
Posts: 9,770
|
i tried this too an a mp4 without b-frames and it didnt work either
__________________
Between the weak and the strong one it is the freedom which oppresses and the law that liberates (Jean Jacques Rousseau) I know, that I know nothing (Socrates) MPEG-4 ASP FAQ | AVC/H.264 FAQ | AAC FAQ | MP4 FAQ | MP4Menu stores DVD Menus in MP4 (guide) Ogg Theora | Ogg Vorbis use WM9 today and get Micro$oft controlling the A/V market tomorrow for free |
5th January 2006, 11:22 | #32 | Link |
Registered User
Join Date: Jan 2002
Location: Los Angeles, CA USA
Posts: 132
|
Ok, hopefully I didn't break anything with these changes...
http://www.tateu.net/software/dl.php?f=QTSource http://www.tateu.net/software/dl.php?f=QTSource_src A few bugs with fps and framecount were fixed. h264 mp4 files can now be partially decoded in mode=0 but it is still not 100% working. It returns a green frame for frame 0 and sometimes skips the decoding of a few frames. And it's really slow. Raw decoding of 10bit v210 files has been added. This is the format for the Blackmagic 10Bit codec. The last 2 bits of each component are dropped to convert to 8Bit YUY2. For QTOutput, the standard compression dialog box has been added. This allows you to set a datarate for certain codecs. If you want to use the dialog box, leave format equal to "". If you do not want to use the dialog box, set format equal to a valid FourCC. Along with the dialog box, two new parameters have been added: datarate and keyframe. h264 compression is a little buggy. It doesn't work if "keyframe every" is checked. This means you have to use the dialog box for h264 and uncheck it. |
5th January 2006, 15:40 | #33 | Link | ||
Registered User
Join Date: Nov 2001
Posts: 9,770
|
Quote:
Quote:
__________________
Between the weak and the strong one it is the freedom which oppresses and the law that liberates (Jean Jacques Rousseau) I know, that I know nothing (Socrates) MPEG-4 ASP FAQ | AVC/H.264 FAQ | AAC FAQ | MP4 FAQ | MP4Menu stores DVD Menus in MP4 (guide) Ogg Theora | Ogg Vorbis use WM9 today and get Micro$oft controlling the A/V market tomorrow for free |
||
5th January 2006, 20:46 | #34 | Link |
Registered User
Join Date: Jan 2002
Location: Los Angeles, CA USA
Posts: 132
|
bond,
It's definately something with the b-frames. I took your sample mp4 file, converted it to YUY2 avi with mencoder and then encoded it to h264 mp4 using MeGui (mencoder, mp4box, CE-Quicktime profile) with 0, 1 and 2 b-frames. The 0 b-frame version plays back with my filter at normal 25fps speed. The 1 and 2 b-frame versions behave exactly like your sample...stuttering around 2-6fps playback speed. They all play back relatively smoothly and at a decent speed in quicktime player, though. To get h264 mp4 decoding working in this latest version, I didn't actually use the new quicktime 7 API's, I just fixed an error in my old decoding method. My best guess is that my method causes quicktime to always start the decode process from the nearest ref frame, even when stepping sequentially through the file. I don't think there is a frame buffer for anything but the current frame. This means that it is decoding way more frames than it actually needs to. |
6th January 2006, 00:47 | #35 | Link |
Registered User
Join Date: Dec 2005
Location: Sweden
Posts: 703
|
DNxHD only output?
...WOW...I just need to say it...WOW!
Never thought this filter would ever come and now it is here, great work Tateu! Is it possible to import mpeg-2 material with this filter and output to QT? This would be a much faster and easier way than my present way: importing my HDV material (mpg) in VDub, add a lot of avisynth filters, output in raw .avi then import the raw avi in AfterFX to put some more filters on and finally output it in Avid's DNxHD format (1080-25P 185 10-bit). |
6th January 2006, 05:15 | #37 | Link | |
interlace this!
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
|
Quote:
one thing... would dithering be a possibility, or is there no point? i guess it would be too slow and would give a very small quality increase (if any). [edit] it just occured to me... would it be difficult to allow raw output as well? i'd like to avoid using QT's own stuff wherever possible something like the input filter where a fourcc (2Vuy for example) and pixel type (like uyvy) could be specified.
__________________
sucking the life out of your videos since 2004 Last edited by Mug Funky; 6th January 2006 at 05:47. |
|
6th January 2006, 23:04 | #38 | Link |
Registered User
Join Date: Jan 2002
Location: Los Angeles, CA USA
Posts: 132
|
Raw output into a quicktime container? I don't think that will be overly difficult. I'll look into it this weekend.
It doesn't hurt to try dithering. If there's no quality increase or it's too slow then you don't have to use it. I don't know how to do it, though. I did a quick search this morning for a formula and didn't find anything that I understood. If I figure something out, I'll add it in. One thing I did try (though I'm not sure how correct it is)...The current version completely ignores the 2 least-significant bits. I added an option which uses these bits for fractional rounding. Here's a version if you want to try it out...http://www.tateu.net/software/QTSource_10Bit_Round.zip. Add dither=1 to turn on rounding (only has an effect when using mode=1, raw="v210"). Code:
y10b = 1110 1010 10 y8b = 1110 1010 = 234 y_frac = 10 = 2 y8b_rounded = y8b + (y_frac > 1) = 235 |
7th January 2006, 05:05 | #39 | Link |
interlace this!
Join Date: Jun 2003
Location: i'm in ur transfers, addin noise
Posts: 4,555
|
hehe... thanks for that
i don't have any 10-bit stuff on my machine right now (i'm at home for the weekend thankfully), but i'll try it out when i get to work. i highly doubt there'd be a perceptible difference, but it could help to have a little bit of dither noise in there for the sake of the encoder - perfectly clean video will go blocky even at very low quants, but a little bit of noise (bonus if it's related to the signal like dither noise is) can help prevent blocking without hurting bitrate performance. and if you get raw export working, even for 2Vuy, i would kiss you. i've got a really awesome looking "my neighbor totoro" in huffy that i want to export back to didge, and it'd suck if i had to bugger up the sharpness and colours by saving it with quicktime's routines. (seriously, anyone who gets the R4 totoro in march will have the best looking release in the world . i'm real happy with how it turned out). doing a simple subtraction between a resaved QT and the original is a real eye opener... [edit] i'm just reading up on dither for both audio and video. it looks like 1/2 the LSB of noise added before rounding will be fine. only thing is you might want to use the same seed each time, otherwise we will never get the same encode twice there's also floyd-steinberg dither... i'm just looking for appropriate links [edit 2] here's a page with source for a floy-steinberg dither. i'm not sure if interlace needs to be taken into account for it though (probably) as it propagates error from 1 line onto pixels from the next (possibly unrelated) line. http://www.home.unix-ag.org/simon/gimp/fsdither.html
__________________
sucking the life out of your videos since 2004 Last edited by Mug Funky; 7th January 2006 at 05:34. |
7th January 2006, 12:07 | #40 | Link |
Registered User
Join Date: Jan 2002
Location: Los Angeles, CA USA
Posts: 132
|
I haven't looked at the dithering info yet, but here's a version with raw out to try...http://www.tateu.net/software/QTSource_RawOut.zip.
Avisynth must, of course, be outputing YUY2 and 2Vuy is the only setting currently supported (others, yuv2, etc., may be added later). You must have a codec installed that supports 2Vuy (Blackmagic) otherwise it will fail. QTOutput("file.mov", format="2Vuy", raw="uyvy") Raw out is running about 1.5 times as fast in my tests. |
|
|