Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Usage

Reply
 
Thread Tools Search this Thread Display Modes
Old 5th January 2014, 01:24   #1  |  Link
SEt
Registered User
 
Join Date: Aug 2007
Posts: 374
JpegSource – advanced JPEG decoder for Avisynth 2.6

As someone asked for JPEG decoder that does no colorspace conversions and provide YUV data directly, and I've written exactly such thing for some other project – here it is, JpegSource for Avisynth.

Current (2014.4.19) version: https://www.dropbox.com/s/rjnt0y3ead...ce_20140419.7z
Previous (2014.1.5) version: https://www.dropbox.com/s/qvm4kn27b4f7bb9/JpegSource.7z

Due to various chroma subsamplings of JPEG it's Avisynth 2.6 only plugin. And even current Avisynth 2.6 doesn't support all colorspaces that you can encounter in JPEG, like YUV440 and CMYK/YCCK, so to process them you'd have to use single-channel mode. To comply with other Avisynth restrictions multi-channel mode may also "un-crop" images a bit (to provide even width to YV16 etc.), while single-channel mode always returns exact dimensions.

Syntax: JpegSource(string file, int rec, int channel, int length, float fps_num, int fps_den)
  • file – source file name
  • rec – number of reconstruction passes. Default: 1.
  • channel – load only one image channel with specified index (zero based). Default: -1 (load all channels).
  • length – clip length in frames. Default: 1000.
  • fps_num and fps_den – fps numerator and denominator. Default: 24 and 1.
MTMode to use: 1. (3 and 5 are ok too; don't use 2 or 4.)

Beside simply decoding JPEG files it can try to "reconstruct" the data. Shouldn't hurt any image. On high-quality JPEGs change is insignificant, on strongly compressed can produce quite impressive results, but won't help on recompressed files where artifacts were produced by previous compressions. Reconstruction requires SSSE3 capable CPU. For example try on this file.

Can also decode PJG-compressed JPEG files and JPEG files with arithmetic compression.

Remember that JPEGs use PC levels, Rec.601 colorspace and MPEG1 chroma position most of the time. Sadly, there is no way now for source filter to provide such information and you'd have to set correct parameters manually when converting to other colorspaces. (Actually, there is something about chroma position, but it doesn't work.)

License: free for non-commercial use, closed source. (Ad-infested software is not "non-commercial".)


Usage Notes

RGB JPEGs are decoded as YV24 etc. To get proper RGB combine them by
Code:
MergeRGB(last, UToY8(), VToY8(), "RGB24").
To correctly upsample MPEG1 chroma (most JPEGs) use
Code:
YToUV(UToY8().Spline36Resize(last.width, last.height), VToY8().Spline36Resize(last.width, last.height), last)

Last edited by SEt; 19th April 2014 at 10:03.
SEt is offline   Reply With Quote
Old 5th January 2014, 13:10   #2  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,664
Thank you SEt. It's nice to finally be able to decode 4:4:4 jpegs natively in AviSynth (I was using jpeg2yuv and RawSource before but it was a hassle).
I quickly tried with the sample picture you linked, and I have to say the reconstruction pass does a very nice job of deblocking.

One last thing, I ran JpegSource.dll through Dependency Walker and I saw something odd, it lists "AVISYNTH.DLL" as a dependency. Is that normal? I've not seen that before.
Reel.Deel is offline   Reply With Quote
Old 5th January 2014, 13:24   #3  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
Quote:
Originally Posted by Reel.Deel View Post
One last thing, I ran JpegSource.dll through Dependency Walker and I saw something odd, it lists "AVISYNTH.DLL" as a dependency. Is that normal? I've not seen that before.
That happens when you statically link with avisynth.lib.
Groucho2004 is offline   Reply With Quote
Old 5th January 2014, 14:14   #4  |  Link
SEt
Registered User
 
Join Date: Aug 2007
Posts: 374
It's not just deblocking – it tries to remove all JPEG artifacts. For example text strongly compressed in JPEG should be way less painful to read after 3 passes of reconstruction. The interesting aspect of this "reconstruction" is that if you compress reconstructed image with exactly the same settings – you should get exactly the same JPEG file. So it's indeed kind of reconstruction or more accurate decoding unlike usual dumb blur-them-all postprocessings.

The funny thing is with such postprocessing JPEG can rival or even surpass JPEG2000 on low bitrates, what is even more true with more efficient final entropy coding like PJG.

You don't need to worry about missing runtimes with my builds. Yes, avisynth.dll as dependency is normal for 2.6 interfaces. (I know it's ugly, but that's how 2.6 interfaces work.)
SEt is offline   Reply With Quote
Old 5th January 2014, 14:44   #5  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
Quote:
Originally Posted by SEt View Post
Yes, avisynth.dll as dependency is normal for 2.6 interfaces. (I know it's ugly, but that's how 2.6 interfaces work.)
There is zero need for plugins to link with the avisynth.dll binary in any way, not dynamically nor statically. It is not normal for a plugin, not even in 2.6, to work like that. If for some strange reason you do want to link with it on purpose, it's fine by me, but I felt the need to post this before other plugin authors start linking with avisynth.dll by mistake.
__________________
AviSynth+
ultim is offline   Reply With Quote
Old 5th January 2014, 15:51   #6  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,364
Quote:
If for some strange reason you do want to link with it on purpose
It would be nice if it would be open source. Is there a reason that you don't want to open it?

Btw, what's YUV440? I read somewhere that it is rotated YUV422, but i have no idea what that means. Does it mean that chroma is shared between two vertical pixels (instead of horizontal)?

Last edited by Wilbert; 5th January 2014 at 15:55.
Wilbert is offline   Reply With Quote
Old 5th January 2014, 16:05   #7  |  Link
innocenat
Registered User
 
innocenat's Avatar
 
Join Date: Dec 2011
Posts: 77
Quote:
Originally Posted by Wilbert View Post
Btw, what's YUV440? I read somewhere that it is rotated YUV422, but i have no idea what that means. Does it mean that chroma is shared between two vertical pixels (instead of horizontal)?
Yes, chroma is shared between two vertical pixels.
innocenat is offline   Reply With Quote
Old 5th January 2014, 16:29   #8  |  Link
SEt
Registered User
 
Join Date: Aug 2007
Posts: 374
ultim, global variable AVS_linkage is effectively linking. Be it static like here or dynamic as assignment in AvisynthPluginInit3 doesn't change the fact. If you think that assignment is the preferred way, ok, next version will link by that.

Wilbert, closed source because I don't want commercial usage of reconstruction code.
Yes, in YUV440 chroma is shared between two vertical pixels but not horizontal. It often appears as result of lossless rotation by 90 degrees of YUV422 data. Avisynth has enough flags to express such subsampling, but refuses to create frames of this type.
Btw, does anyone know how rotated YUV411 is called? I can't think of a way to express such subsampling in standard notation.
SEt is offline   Reply With Quote
Old 7th January 2014, 03:43   #9  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,556
Can you add a native VapourSynth interface? (Or have you?) The colorspace options there are more versatile.

You could also possibly take advantage of decoding into high bit depth - I've always wondered if using a floating-point iDCT directly into a high bit depth prior to transforming to RGB would solve some banding problems, even if it obviously wasn't created with more than 8 bits.
foxyshadis is offline   Reply With Quote
Old 7th January 2014, 06:36   #10  |  Link
wOxxOm
Oz of the zOo
 
Join Date: May 2005
Posts: 208
Quote:
Originally Posted by SEt View Post
Btw, does anyone know how rotated YUV411 is called? I can't think of a way to express such subsampling in standard notation.
The standard notation describes two rows, so the only way is to use a non-standard 4-row 4-column description 4:4:0:0:0, also as an example: the non-rotated 420 would be 42020 (to convert any standard notation simply repeat the chroma numbers twice).

Last edited by wOxxOm; 7th January 2014 at 07:15.
wOxxOm is offline   Reply With Quote
Old 7th January 2014, 15:09   #11  |  Link
SEt
Registered User
 
Join Date: Aug 2007
Posts: 374
Quote:
Originally Posted by foxyshadis View Post
Can you add a native VapourSynth interface? (Or have you?) The colorspace options there are more versatile.
I have no interest in VapourSynth.

Quote:
Originally Posted by foxyshadis View Post
You could also possibly take advantage of decoding into high bit depth - I've always wondered if using a floating-point iDCT directly into a high bit depth prior to transforming to RGB would solve some banding problems, even if it obviously wasn't created with more than 8 bits.
Maybe. With standard decoding blocking is far more serious issue, but with reconstruction it might make sense to try to combat banding too. Can you post examples of JPEGs with banding problem?
SEt is offline   Reply With Quote
Old 8th January 2014, 01:59   #12  |  Link
ajp_anton
Registered User
 
ajp_anton's Avatar
 
Join Date: Aug 2006
Location: Stockholm/Helsinki
Posts: 805
So with an unsupported colorspace, will it fail to open it, or silently convert it to a compatible format?
ajp_anton is offline   Reply With Quote
Old 8th January 2014, 08:45   #13  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
Quote:
Originally Posted by SEt View Post
ultim, global variable AVS_linkage is effectively linking. Be it static like here or dynamic as assignment in AvisynthPluginInit3 doesn't change the fact. If you think that assignment is the preferred way, ok, next version will link by that.
From the Avisynth's core view, I don't care. But it is your interest that you don't link with avisynth.dll directly, because if you do, Windows will only load your plugin if it can find avisynth.dll in the standard system paths. It will be in the system path for most installations, but not all.
__________________
AviSynth+

Last edited by ultim; 8th January 2014 at 09:09.
ultim is offline   Reply With Quote
Old 8th January 2014, 10:57   #14  |  Link
SEt
Registered User
 
Join Date: Aug 2007
Posts: 374
Quote:
Originally Posted by ajp_anton View Post
So with an unsupported colorspace, will it fail to open it, or silently convert it to a compatible format?
No silent conversions here – it will fail. This plugin always provides raw data.

Quote:
Originally Posted by ultim View Post
But it is your interest that you don't link with avisynth.dll directly, because if you do, Windows will only load your plugin if it can find avisynth.dll in the standard system paths. It will be in the system path for most installations, but not all.
Not a problem if it's all you have against symbol importing. As it's plugin to Avisynth and will be loaded by Avisynth that means that avisynth.dll will already be loaded in memory. In that case import will be resolved by avisynth.dll in memory regardless of where the file is placed.

Last edited by SEt; 8th January 2014 at 10:59.
SEt is offline   Reply With Quote
Old 8th January 2014, 18:46   #15  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
Quote:
Originally Posted by SEt View Post
Not a problem if it's all you have against symbol importing. As it's plugin to Avisynth and will be loaded by Avisynth that means that avisynth.dll will already be loaded in memory. In that case import will be resolved by avisynth.dll in memory regardless of where the file is placed.
To my knowledge, Windows uses the basename, and optionally the full path if supplied, when searching for already loaded libraries in memory. That means it wouldn't work if avisynth.dll is named differently, or if the symbols are supplied by a different library, like VapourSynth (as VS is able to load AVS plugins). I know you don't care about VS, but it is a real-life example of the problem I am trying to demonstrate.

Anyway, if not supporting those scenarios is fine by you, fine by me. I only brought up the issue because others might care about those few cases that you don't care about, and because of those few cases it is generally better not to link with avisynth.dll directly. With my original post I'm not trying to get you to change your code, it was just to let other developers know that there is an even better approach than linking to the core.
__________________
AviSynth+

Last edited by ultim; 8th January 2014 at 18:51.
ultim is offline   Reply With Quote
Old 8th January 2014, 19:28   #16  |  Link
SEt
Registered User
 
Join Date: Aug 2007
Posts: 374
Indeed, this time you are correct, though the case of renamed avisynth.dll is kind of artificial. It was explicitly stated that it's Avisynth 2.6 plugin.
SEt is offline   Reply With Quote
Old 9th January 2014, 05:08   #17  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,664
Quote:
Originally Posted by SEt View Post
Can you post examples of JPEGs with banding problem?
I took a picture of the sun and sky a while back and it made a good example of JPEG banding. Source is a raw photo taken with my Canon 7D and processed with Adobe Lightroom 5.

- Banding.7z - 4:2:0 JPEGs ranging from quality 0 - 100 (>54 are 4:4:4)
- Original.7z - 16-bit TIFF

Let me know if you want something with more detail.

Last edited by Reel.Deel; 9th January 2014 at 05:20.
Reel.Deel is offline   Reply With Quote
Old 12th April 2014, 17:57   #18  |  Link
Nevilne
Registered User
 
Join Date: Aug 2010
Posts: 134
Thanks for the filter, the reconstruction is very nice. What would be the safe number for good quality pictures? I've tested some with 3 and it didn't seem to do any harm.

Some bugs to report:
RGB jpegs open as YV24:
http://i1.someimage.com/kFS0P4R.jpg

YUV 4:1:1 chroma gets displaced (sane shamanism with avisynth chromaplacements is not enough, needs an offset)
http://i1.someimage.com/KXOlJZB.jpg
(compare red leaves of jpegsource with imagesource)

When you open the image with jpegsource, it blocks the source file. Other image source filters like ImageSource, FFImageSource don't block the file and even allow seeking after file deletion with end=999 repetition.

Last edited by Nevilne; 12th April 2014 at 18:04.
Nevilne is offline   Reply With Quote
Old 13th April 2014, 11:23   #19  |  Link
SEt
Registered User
 
Join Date: Aug 2007
Posts: 374
For good quality pictures doesn't really matter – filter limits itself to variation introduced by compression data loss, so change will be small anyway. I usually use 1 for realtime postprocessing in my image viewer to save time.
For highly compressed photos I prefer 1 level, as 2+ produce too soft image.
For highly compressed text 3-4 are nice for salvaging as much as possible.

RGB JPEGs are quite rare, most Avisynth filters now strongly prefer planar formats and there is no planar RGB in Avisynth yet. So that behavior is somewhat "by design". You can easily get correct RGB from that by
Code:
MergeRGB(last, UToY8(), VToY8(), "RGB24")
For correctly handling MPEG1 YUV chroma in conversions you can explicitly upsample chroma before it. For example to get correctly aligned YV24 you can use:
Code:
YToUV(UToY8().Spline36Resize(last.width, last.height), VToY8().Spline36Resize(last.width, last.height), last)
Also do note that current resizing functions always process chroma as MPEG1, while conversions process it as MPEG2 by default.

Indeed, it's possible to close file earlier – will do it in next version.


My new version of decoder can also support JPEGs with arithmetic compression – not sure if there is any interest here for working with such images.

Last edited by SEt; 13th April 2014 at 11:36.
SEt is offline   Reply With Quote
Old 14th April 2014, 19:23   #20  |  Link
wOxxOm
Oz of the zOo
 
Join Date: May 2005
Posts: 208
SEt, if it's possible at all, can you adapt your pp algorithm to make an MPEG-2 post-processor?
wOxxOm is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 04:41.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.