Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
4th September 2011, 10:29 | #1 | Link |
Grand Fruitioner
Join Date: Mar 2004
Location: Chicago, IL
Posts: 115
|
RFC: extensions for a high bit depth pipeline
So I've finally started some code for writing high bit depth filters, allowing an entire script pipeline to make use of it without constant up/down converting between filters. Looking for comments, contributions, and probing for interest in devs of existing plugins. There's a lot of work to be done—many of the Avisynth filters would need to be rewritten, and it probably wouldn't be very useful until some of the more popular filters adopted it.
Some current design choices:
Usage would look something like this: Code:
FFVideoSource() ConvertToHQ32() ... ConvertToLQ()
__________________
Lanczos4, Spline36, what!? Don't know how to pick a resizer? Take a look at my kernel visualizations. Want a high-quality, gamma-aware resizer? Check out my ResampleHQ filter. Last edited by PhrostByte; 5th September 2011 at 04:21. |
4th September 2011, 20:47 | #3 | Link |
Grand Fruitioner
Join Date: Mar 2004
Location: Chicago, IL
Posts: 115
|
No. Think of it as a new avisynth.h for plugin developers, to provide a faster native solution.
__________________
Lanczos4, Spline36, what!? Don't know how to pick a resizer? Take a look at my kernel visualizations. Want a high-quality, gamma-aware resizer? Check out my ResampleHQ filter. |
5th September 2011, 02:25 | #5 | Link |
Grand Fruitioner
Join Date: Mar 2004
Location: Chicago, IL
Posts: 115
|
Source filters would need to be updated for it. Hopefully it would be easy to get such a change built into FFMS.
__________________
Lanczos4, Spline36, what!? Don't know how to pick a resizer? Take a look at my kernel visualizations. Want a high-quality, gamma-aware resizer? Check out my ResampleHQ filter. Last edited by PhrostByte; 5th September 2011 at 02:28. |
5th September 2011, 03:58 | #6 | Link | |
Excessively jovial fellow
Join Date: Jun 2004
Location: rude
Posts: 1,100
|
Quote:
Currently, the FFMS2 Avisynth plugin is a relatively thin wrapper around the FFMS2 API (with the exception of the pulldown flag manipulation). I don't think implementing what you want to do is particularly hard, but adding custom output mode hacks to the Avisynth plugin isn't very elegant. I will still have to admit that this is a clever and potentially useful hack, though. If you implement the FFMS2 part, you can have a branch in the main repository if you want it, like the C-plugin has. |
|
5th September 2011, 03:59 | #7 | Link |
Grand Fruitioner
Join Date: Mar 2004
Location: Chicago, IL
Posts: 115
|
Okay, I've decided to move from the mvtools2 hack to including a small header in each frame. This makes it compatible with audio filters and adds VFR support.
__________________
Lanczos4, Spline36, what!? Don't know how to pick a resizer? Take a look at my kernel visualizations. Want a high-quality, gamma-aware resizer? Check out my ResampleHQ filter. |
5th September 2011, 04:14 | #8 | Link | |
Grand Fruitioner
Join Date: Mar 2004
Location: Chicago, IL
Posts: 115
|
Quote:
I know it's a bit hacky, but I've done my best to keep the developer API as clean as possible. Should Avisynth development ever kick back into gear and get a compatible feature set, it would not be difficult to move a filter to the official API.
__________________
Lanczos4, Spline36, what!? Don't know how to pick a resizer? Take a look at my kernel visualizations. Want a high-quality, gamma-aware resizer? Check out my ResampleHQ filter. |
|
5th September 2011, 12:01 | #9 | Link |
Registered User
Join Date: Aug 2007
Posts: 374
|
Very nice ideas (especially what you are willing to write it ^_^), but my vote is for diffing it against Avisynth 2.6 code, not as another hack around. And it absolutely doesn't matter that there will be very limited support for it in internal filters. Avisynth 2.6 or even 3 development isn't going to revive itself without people willing to write something.
Another thing I don't like is names: ResampleHQ sounds reasonable, but ConvertToHQ32 is not. I suggest something like ConvertToFloatYUV444. Packed colorspaces would be nice too - I don't remember a good way of using planar images with OpenCL/OpenGL. |
5th September 2011, 19:42 | #10 | Link |
Registered User
Join Date: Jan 2006
Posts: 1,867
|
This is exactly what I've wanted Avisynth to support! The per-frame meta data is very important, we have so many hacks like stuffing the low bits with whether the frame was interlaced or not, and some information I want to access, like if the frame was originally I,B, or P, which I need for my filter. And of course, when I presented my ideas it got shot down just like you I'm all for getting something done *now*, because if you try to do it in some ideal way, it's just not going to get done. I really want to get workable high bit-depth support working, that would make it usable to professional users. If this is a plugin you can make it for both versions, of course.
To start, I'd like to see a version of ffms to support high bit-depth and also meta data (like I,B,P as I mentioned), and some functions to access these properties in script, and a way to convert to/from the usual formats, including the possibilibies to change the script convention being used (stacked MSB/LSB) into this internal format. The first filters can do levels, gamma, and dither and IVTC - that would immediately solve the most pressing needs of professionals. We also need a way to export to an encoder. |
6th September 2011, 01:20 | #11 | Link | ||
Grand Fruitioner
Join Date: Mar 2004
Location: Chicago, IL
Posts: 115
|
Quote:
I think you're right about the names too—ConvertToFloat32() and ConvertToUInt16() would work better. And OpenCL/OpenGL support is exactly the kind of reason I was looking for! I'll be sure to add support for packed colorspaces. Quote:
__________________
Lanczos4, Spline36, what!? Don't know how to pick a resizer? Take a look at my kernel visualizations. Want a high-quality, gamma-aware resizer? Check out my ResampleHQ filter. |
||
6th September 2011, 09:19 | #13 | Link | |
Registered User
Join Date: Jan 2006
Posts: 1,867
|
I've found just the kind of comments you were looking for. Actually I brought this up in 2006 and not much has changed since then - except I actually wrote Deepcolor tools
Quote:
In the last bit where he talks about standards, he didn't realize there was v210. That's what he meant about registered by Apple. So yes, v210 is an official 'hack' for high bit-depth in VFW. I see nothing wrong with this. Our new convertion of stacked 16bit as (MSB, LSB) or your planar format could become an offical 4cc as well - unless there's something existing that's similar. I had some more thoughts on this, can't find it yet - but I listed all the meta data which would actually be useful per frame. Last edited by jmac698; 6th September 2011 at 09:23. |
|
6th September 2011, 09:42 | #14 | Link | |
Registered User
Join Date: Jan 2006
Posts: 1,867
|
And here's another good thread
Quote:
The discussion was about using signed or unsigned 16bit ints. Someone from an audio background initially favored signed, but someone who had written image processing in 16bit unsigned said that there was no problem with unsigned as far as MMX. Another person then pointed out that targetting SSE2 makes sense now, and that even floating point was fast these days. A few people would definitely use Avisynth for color grading and denoising in Digital Intermediates, if we had high bit-depth. They suggested 10bit with gamma, 16bit linear, 10bit log, and 32bit float as some standards. I also discovered there is a rawsource plugin for 16bit import that should be a lot more workable now that dither, avs2yuv etc. are now available. People have access to up to 4k resolution, 16bit files. This has been requested since over 5 years ago. |
|
6th September 2011, 22:08 | #16 | Link |
Registered User
Join Date: Aug 2006
Location: Stockholm/Helsinki
Posts: 805
|
I would love to see
- ConvertTo8bit - ConvertTo16bit And while I don't fully understand the practical difference between packed and planar, - ConvertToPlanar420 - ConvertToPlanar444 - ConvertToPacked422 etc... |
6th September 2011, 23:05 | #17 | Link | ||
Grand Fruitioner
Join Date: Mar 2004
Location: Chicago, IL
Posts: 115
|
Quote:
Quote:
Windows internally uses full range unsigned integers, so that's what I'm inclined to use for now. A signed HDR format does sound interesting, though. Thanks for all your posts! It will definitely have all of these.
__________________
Lanczos4, Spline36, what!? Don't know how to pick a resizer? Take a look at my kernel visualizations. Want a high-quality, gamma-aware resizer? Check out my ResampleHQ filter. Last edited by PhrostByte; 6th September 2011 at 23:10. |
||
7th September 2011, 00:45 | #18 | Link |
Registered User
Join Date: Jan 2006
Posts: 1,867
|
Well, I can't find my big list, though I did mention this in a few threads. I just think we need an open and easy to access set of data per frame. Some things per frame: caption characters, color standard, interlaced, frame type.
dgdecode saves the per-frame color matrix as hints, the lower bits of the picture. So there is one immediate use. I saw that Fizick wanted meta data, we could ask him what his needs are. |
8th September 2011, 02:06 | #19 | Link |
Registered User
Join Date: Jan 2006
Posts: 1,867
|
Hi,
I'm still gathering requirements here, for future reference. I found this thread interesting: http://forum.doom9.org/showthread.php?t=162459 The problem was that a VOB was changing resolution mid-stream, and it was reported that ffms does handle this. It's unknown how encoders handle it. So the proposal is that, we need to store resolution per frame. And I know that audio can change from stereo to 5.1 (during commercials). Sure, some plugins that use temporal references won't be resolution change safe, perhaps they should be padded to the largest possible size? Well, just throwing it out there. |
8th September 2011, 04:01 | #20 | Link | |
Excessively jovial fellow
Join Date: Jun 2004
Location: rude
Posts: 1,100
|
Quote:
As for audio, FFMS2 doesn't support channel layout switches yet. It's in the works, though, and will probably be in 2.17. Furthermore, while high bitdepth stuff is bad enough, we're now starting to get into the sort of hacks that aren't just ugly, but really hairy as well. |
|
Thread Tools | Search this Thread |
Display Modes | |
|
|