Video Super Resolution for everyone [Archive] - Page 2

scharfis_brain

30th July 2006, 14:36

I tried this super resolution program on a small video.
It did not achieved any comparable results to my avisynth test scripts I made for that purpose. Meaning it wasn't able to enhance the resolution significantly.

It seems like this application does motion compensation and merges the compensated frames into the current frame using some sort of kernel (highpass filter).
I came to this conclusion because MoComp mismatches are leaving similar artifacts like v-t-deinterlacing kernels do with fast motion. (Kernel Ghosts)
Is this true?

DmitryPopov

31st July 2006, 11:10

Partly true.
It really does motion compensation but the next step is different. Looks more like simplified Kalman Filter.

The difference between slow and fast modes is in motion estimation. Slow mode is almost full search while fast mode is kind of hexagonal search with candidates.

DmitryPopov

12th August 2006, 00:34

Version 1.2 has come out.
Now all filters, coders and decoders work simultaneously in separate threads. This makes Video Enhancer a lot faster on machines with several logical CPUs (multiprocessors, dual cores, hyperthreading).

A simple test with processing DV video with 3 filters and compressing it with XviD showed 1.9 times better speed than VirtualDub on Dual Core AMD Opteron.
Details here:
http://www.thedeemon.com/VideoEnhancer/speed.html

frodeste

13th August 2006, 13:10

DmitryPopov

11th November 2006, 22:30

Version 1.3 released.
Improved support for different file formats and added audio recompression. Enjoy!
http://www.thedeemon.com/VideoEnhancer/

Fizick

19th November 2006, 23:12

We will enjoy when you distribute it under GNU GPL or remove all violations (in MY opinion - I am not a lawyer) of GPL license:
1. Probably you use some part of VirtualDub source code to implement plugins support.
2. Probably dynamicaly linkage with GPL plugins violate copyright of their authors.
3. Certainly distribution of GPL plugins binary without source code (from the same place) violates it too.

Suggestion: read GNU GPL license (also consult GNU faq), remove VirtualDub plugins support and create great VideoEnhancer.

DmitryPopov

20th November 2006, 07:00

1. From VirtualDub I only use Filter.h which comes with following conditions:

// As a special exemption to the GPL in order to permit creation of
// filters that work with multiple programs as well as VirtualDub,
// compiling with this header file shall not be considered creation
// of a derived work; that is, the act of compiling with this header
// file does not require your source code or the resulting module
// to be released in source code form or under a GPL-compatible
// license according to parts (2) and (3) of the GPL. A filter built
// using this header file may thus be licensed or dual-licensed so
// that it may be used with VirtualDub as well as an alternative
// product whose license is incompatible with the GPL.

2. And probably not. Or stop using Xvid and x264 under Windows. ;) Anyway, I do not force anybody to use GPLed filters. You are always free to use only MSU filters that are not GPLed.

3. Video Enhancer installation does not include any. Separate part of the website contains links to the filters, as well as links to their pages with source code.
VD Filter Pack also contains links to the pages with sources.

So, no violations.

Suggestion: stop being ignorant and accusing me of imagined violations when real filters authors don't.

Chainmax

20th November 2006, 07:15

That script can be sped up considerably, with just a small change. (Mates, do set those darn idx vars...)
However, this script is a very raw proof-of-concept, and is fully unusable in practice. Tests with re-upsizing a 50% downsized movie source have shown it puts out a softer & less detailed result than a plain lanczos or spline resize, d'oh. And heaps of artefacts. It surely does show the potential on some certain selected frames, but that's just it. Perhaps it's good for such a maximum-digital-crap source like that of Chainmax. But surely not for your "average" source, whatever that might be.
...

There's no way to turn it into a general-purpose superresolution function then? That's too bad :(. I'd love to see a sped-up version without the framerate conversion part and with some EEDI2 thrown in (maybe it would improve results?).

Fizick

20th November 2006, 19:34

Imagine? Not quite so. :)
1. Even if you do not use other virtualdub source code files:
The exception "permit creation of filters that work with multiple programs as well as VirtualDub...".
"A filter built using this header file may thus be licensed..."
But your VideoEnhancer is not a Vitrualdub filter, but host application for filters.
2. read GNU GPL faq about it. It permits to use OS.
3. Sources must be at the same place (your server)
[Quote]
If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code[/qoute]

DmitryPopov

21st November 2006, 07:02

1. I see the words "...it may be used with VirtualDub as well as an alternative product whose license is incompatible with the GPL." So I guess Avery meant there could be other non GPLed products that can use the filters.

Technically, usage of Filter.h and dynamic linking with .VDFs is done in a DirectShow filter ;) called VDFilter, not in Video Enhancer itself. Video Enhancer works with VDFilter via DirectShow API just as many videoplayers work with GPLed video codecs via DirectShow and VfW API. I think this way of cooperation is not considered dynamic linking. VDFilter is a separate product:
http://www.thedeemon.com/VDFilter/

2. Yeah, I know OS is different case. But videoplayers and videocodecs are a good example. The way of linking is the same - DirectShow or VfW API. Does just the ability of Windows Media Player to use XviD require WMP to be GPLed? Don't think so.
Does the ability of Video Enhancer to use GPLed plugins require it to be GPLed? Don't think so.

3. Ok, I should put direct links to the sources on the page with filters. Right? Then I'll do it soon.

Fizick

21st November 2006, 18:32

1. IMO, the exception covers other cases:
Some VirtualDub filter uses (call, link) non-GPL program (library, DLL or EXE).
Or: Filter is simultaneously plugin for VirtualDub and other program. (rare example: WNR plugin - it is also plugin for Avisynth).

Thanks for technical info.
In this case Video Enhancer itself is probably OK.
VDFilter is PROBABLY not.
I suggest you to release your VDFilter under GPL.
Generally, it is not only legal (and wording) issue, but respect to Avery and plugins writers.

2. Yes, it is hard to get a clear threshold.
But if you release VDFilter under GPL, it will be probably OK for VideoEnhancer according to your argumentation.

3. No, not links. I say about VirtulaDub Filter pack only (your other links is OK). If you distribute binaries (GPL) without sources from your web site, you must also distribute its (GPL) sources from your web site (the same place), for example as a second pack.

By the way, are you still in MSU?

DmitryPopov

21st November 2006, 18:49

1,2. I'll think about making VDFilter GPLed. By this moment I haven't received any complaints from Avery and plugins writers..
Well, the main thing will be Avery's view on using Filter.h. As for dynamic linking with filters, since I do not force anyone to load GPLed plugins, the ability of linking itself does not imply anything, I guess.

3. Yeah, I should put the sources of filters used in the pack. I'll do it in a few days.

4. Graduated 2 years ago, now working closely with MSU VideoGroup in some projects, but not an official member.

Fizick

21st November 2006, 22:17

forget to add:
"As a special exception, the copyright holders of XviD give you
permission to link XviD with independent modules that communicate with XviD solely through the VFW1.1 and DShow interfaces ....."

satmonk

30th November 2006, 07:30

Hi Didée,

what is Sharp TSR? Your Results looking good, but never heard of Sharp TSR, search-engines give 0 results.
Can you clearify?

thanks
satmonk

fenomeno83

2nd December 2006, 01:38

Can I get a better resolution(HD) from dvd(720x576)?For example frame from 720x576 to 1280x720 (HD)!

Chainmax

2nd December 2006, 02:56

Yes, but probably not with this. Take a look at IIP (http://forum.doom9.org/showthread.php?t=70916).

DmitryPopov

2nd December 2006, 11:38

Can I get a better resolution(HD) from dvd(720x576)?
Sure. That's what Video Enhancer for.

fenomeno83

2nd December 2006, 11:46

but will I have a better quality(high definition)?

DmitryPopov

2nd December 2006, 12:54

If the source wasn't compressed too much then yes, quality shall be enhanced.

SealTooGreat

29th December 2006, 06:50

Nice work!...Indeed, the difference between bicubic,Lanczos3 methods and Video Enhancer on your site is obvious.
Do Video Enhancer support frameservering like virtualdub?!

DmitryPopov

29th December 2006, 07:24

No, frameserving is not supported now.

SealTooGreat

4th January 2007, 17:48

Is it possible to manage Video Enhancer to work with CLI encoders like xvid_encraw.exe or x264.exe?!

Fizick

4th January 2007, 21:08

QE SuperResolution
http://www.soft32.com/download_120675.html

DmitryPopov

28th May 2007, 16:25

I'm back with version 1.5.
Changes:
* Super-resolution quality improved a lot.
* Finally supports VBR audio (like VBR MP3).
* High quality mode is 1,5 times faster now.

http://www.thedeemon.com/VideoEnhancer/

Objective comparison of different upsize methods and solutions here:
http://www.thedeemon.com/articles/video_upsize_methods_comparison.html

P.S. QE SuperResolution is not for resizing video, it's supposed to get single frames. And I couldn't get it to work with uncompressed RGB24 AVIs. :(

MfA

29th May 2007, 22:18

The edges for topaz enhance don't really seem cartoon like to me, it's just that the lack of other detail makes them stand out (plus they are aliased like hell).

foxyshadis

30th May 2007, 02:15

Topaz just released a new version of Moment that supposedly looks a little more natural, less aliased and flat. You could probably use a simple avisynth resize script to shift the image over slightly, so that the PSNR measure is more useful, like I do to get EEDI2 to line up with a normal resize.

DmitryPopov

30th May 2007, 05:44

Yeah, I've tried Topaz Moment 3.2, it gives different picture with additional noise and different artifacts. The problem is that I can't resize video in Moment, only get some stills. And in case of my RGB24 sources it saves BMP files flipped vertically.
http://stuff.thedeemon.com/smiths09-org.jpg
http://stuff.thedeemon.com/smiths09-topenh.jpg
http://stuff.thedeemon.com/smiths09-topmom.jpg
http://stuff.thedeemon.com/smiths09-videoenhancer.jpg
Shifting image with resize script changes the image itself, I don't know how valid will be PSNR values in that case because they will strongly depend on algorithm of shifting. I agree that current PSNR of original shifted video is also not so meaningful, but at least all upsize methods in comparison were in the same conditions.

I tried to contact Topaz asking about the shift and how to avoid it but they never answered.

If anyone here finds a correct way to measure Topaz's PSNR, I'll be very glad.

If anyone have doubts in PSNR values in my comparison, I can provide hi-res originals to measure PSNR with, try it for yourself.

*.mp4 guy

30th May 2007, 07:25

shifting with spline36 will have a very minimal impact on psnr if you are performing the shift on an already upsampled image.

MfA

2nd June 2007, 05:45

How much history does videoenhancer take into account for resizing? I want to construct a couple of short sequences for objective testing of resizers from some HD footage (using gaussian filtering in "linear" colorspace to create the high&low resolution pair of sequences, rather than box/bilinear filtering in gamma corrected colorspace which is not realistic).

foxyshadis

2nd June 2007, 08:26

Well, aside from still looking much too crisp, in the new version it applies automatic levels correction, apparently, so measuring it against anything is probably a lost cause. Oh well.

spline36resize(width,height,-1,-1,-1,-1)
appears to do the trick for Enhance, though I don't know the exact location of pixels in their formula and didn't try to optimize the values for psnr, obviously. As a guess, I'd say the optimal values might be just a little less than that.

DmitryPopov

2nd June 2007, 10:05

> How much history does videoenhancer take into account for resizing?

In a sense, all of it. The method is iterative, previous upsized frame is used to upsize next frame. Generally, 20-100 frames is long enough. In my comparison I used 45-100 frames long videos.

Recently I managed to run another "super-resolution" solution - Resizer plugin for After Effects from Digital Anarchy. This is what I had using its two best methods with default parameters:
http://stuff.thedeemon.com/smiths09-da-sharp.jpg
http://stuff.thedeemon.com/smiths09-da-smooth.jpg

If anyone is familiar with this tool, I'd be happy to hear how to get best quality of it.

Didée

4th June 2007, 11:36

I downloaded the new trial, gave it a shot, aaand ...

It does not work for me. The output is corrupted.

Video Enhancer v1.5 processes only a couple of frames (varying, from ~50 to ~90 frames), and after a short time it says "done".

The very start of the processed output is OK, and after a few frames it starts to look like this:

http://img524.imageshack.us/img524/3901/vsrproblemhx7.th.jpg (http://img524.imageshack.us/my.php?image=vsrproblemhx7.jpg)

Sample of processed output: >link< (http://www.megaupload.com/?d=X9WYWC9O) (5 MB)

The AVS script fed into Video Enhancer was just this:

mpeg2source("F:\Video\Samples\snip4.d2v")
crop(8,80,-8,-80)
BicubicResize( 352,208, 0.2,0.4 )

Is this the expected behaviour?

DmitryPopov

4th June 2007, 11:57

Avisynth support is not garanteed nor even tested.
Save your low res material in a real file (using a lossless codec to prevent distortions) and then process it in VE.

Didée

4th June 2007, 12:11

Ah okay. Then by now, it's tested and approved to not work.
Cool stuff.

Didée

4th June 2007, 12:30

Follow-up: Nada. It still does not work for me.

I saved the above posted script to an AVI file, compressed with Huffyuv (ffdshow, YV12). This AVI file I loaded into Video Enhancer.

The result is basically the same:

- no more "Done!" message. After some frames, Video Enhancer just sits there and does not proceed any more, while "Remaining time" keeps climbing & climbing.

The result is still corrupted. Only thing that has changed is the look of the corruption:

http://img411.imageshack.us/img411/2416/vsrproblemavisourceqj4.th.jpg (http://img411.imageshack.us/my.php?image=vsrproblemavisourceqj4.jpg)

What next? Only uncompressed RGB32 sources? Or must the pixels be translated into ASCII art?

Only thing I can see is that "it does not work", sorry. Your job to tell how to make it work. After all, you're expecting to make money with that thing, aren't you.

Edit:

Got it finally working, with input=uncompressed RGB / output=uncompressed RGB. Very convenient.

The app seems to be quite picky & particular about codecs (i.e., couldn't use mjpeg for output ["cannot connect"] ... also, ffdshow hooked in somewhere in the chain where it shouldn't.)
No mood/time to track down all possible do's and dont's. At least it works ... somehow.

*.mp4 guy

4th June 2007, 15:57

FFdshow has notoriously buggy video encoding frontends, I've had the errors you ran into pop up from time to time whenever I use FFdshows huffyuv encoder, If you used something known to be very stable, like lagarith (old huffyuv doesn't count, to many things hijack its FOURCC) and still had problems then it would be safe to say that its a problem with video enhancer.

Didée

4th June 2007, 16:58

Well, it's not really an issue to me. Knowing how to play the Avisynth piano, there's no need to worry about other's instruments being out-of-tune. ;)

foxyshadis

4th June 2007, 22:29

Unfortunately, that does look like a huffyuv error. :( ffdshow's had one or two huffyuv fixes lately, but I'll see if I can track down any more bugs.

DmitryPopov

5th June 2007, 06:08

Thanks for reporting!
Since all processing is done in RGB32 (to easily build chains of super-resolution and VirtualDub filters) Video Enhancer tries to feed RGB32 data to encoder (and asks from decoder). Some codecs support only YUV, that could be the problem.

I'll try to make VE friendlier to different codecs and avisynth.

SealTooGreat

20th June 2007, 11:40

@DmitryPopov
Can I use Super Resolution without upsizing input material, keeping output res. same as input? And how SR behaves when video is downsized?

DmitryPopov

20th June 2007, 12:01

SR is a technique for getting bigger images/video from several smaller ones. There's no sense of using it without upsizing because upsizing is what it's intended for.
If you have some blocking artifacts or mosaic there are some ways to deal with them (see http://www.thedeemon.com/articles/ ).
When you downsize video in Video Enhancer it just uses Lanczos3 resize. So it's sharper than bilinear or bicubic but nothing special.

chros

13th July 2007, 10:23

Can I use SR in realtime (eg. through ffdshow-avisynth) ?
(I saw the SR.ax in the install dir, but I can't find the External filters section of MPC.)

DmitryPopov

13th July 2007, 10:52

Not yet.
First, this filter is still quite slow (RGB24 320x240 -> 640x480 at 20 fps on a dual AMD Opteron). A real time version may come out in autumn.
Second, you can't use SR.ax outside Video Enhancer, because only VE knows how to set its parameters.

chros

13th July 2007, 16:21

davidhalopoff

21st August 2009, 23:08

This program does work but it has its limitations and inconsistencies but probably hand in hand with the multitude of input material. Thin material like small screen captures can only improve so much.

Uncompressed mode will usually garble the sound but then recompression with any one one of the auto processing programs- even MS Movie Maker will sort it back out, reduce the file size and get you the cleaned up product you worked for.

It will only improve so much. I have experimented with all sorts of filter chains, input formats and out chop codecs and you sometimes have to run it again with different variables for optimum clean up.

Try to use one of the analysis tools on your input and start taking notes.

I have gotten to where I can guesstimate based on material density and playback appearance how well it will clean up.

You don't have to run DivX Pro in unconstrained mode. If it doesn't hang in preview then it will work.

Best subjective improvement on some war clips was 40%