Video Super Resolution for everyone [Archive]

DmitryPopov

10th July 2006, 12:22

Super Resolution is a technique of increasing images or video resolution with superb quality. For each processed frame it uses information not only from this frame but also from other frames. If picture in your video doesn't change too fast, then information from several frames can be added to create a picture of a bigger size. No algorithms of ordinary image resize/resample can provide this quality just because there's not enough information in one frame.

Video Enhancer is a tool which uses Super Resolution technique to increase resolution of your video.

Also, it can use a hundred of VirtualDub filters to process your video: deinterlacing, denoising, deshaking, color correction etc.

Main page:
http://www.thedeemon.com/VideoEnhancer/

Compare Super Resolution to bicubic and Lanczos3:
http://www.thedeemon.com/VideoEnhancer/compare.html

Edit:
Version 1.0 didn't work with files containing mp3 audio stream. Now this is fixed, so don't be scared by error reports in the beginning of this thread.

neuron2

10th July 2006, 14:01

Is this free software?

DmitryPopov

10th July 2006, 14:55

It's a shareware.
Registration will be accessible in a couple of days.

VDFilter (DirectShow wrapper for VirtualDub filters) is freeware (for non-commercial use).

Didée

10th July 2006, 15:08

Sweet application. Conversion is done in zero time, and the output file consists of 1 frame. :)
Tested both operation modes with both AVI and AVS input, on a Celeron 2.6, XP SP1, without dotNET.

DmitryPopov

10th July 2006, 15:16

What was the input file?
What codec used for output compression?
Any errors?

check

10th July 2006, 15:20

Also tested, same error.
o 640x480 AVI input decoded through ffdshow encoded to uncompressed 2048x1536,
o 640x368 AVS encoded to uncompressed 2048x1152
both of these gave the same error.

As a side note, a shortcut to the readme in the start menu would be nice.

DmitryPopov

10th July 2006, 15:32

Thanks! I'll check that out.
What kind of video decoded by ffdshow?

I've found a file that gives this kind of error. It has VBR audio stream, it can be the cause...

check

10th July 2006, 15:35

the video is xvid-mp3.vbr. When I click "go", it begins to draw the progress screen (I assume that's what it is), writes the first frame to disk and displays the success message. I'll take a video of it in action (or not :P). What a software launch hey? :(

check

10th July 2006, 15:51

Here is a video of the error in action: http://project357.com/ul/upload/output.wmv
As you can see, it instantly ends. The *interesting* part is that when I clicked start, it loads up a second ffdshow video filter for the processing and requests the first frame only.

I also just somehow got it to display the first frame of the encoding, but it still crashes at the same point, the GUI just updates correctly now. EDIT: hmm, seems that the GUI will update if the destination image is smaller than a certain level. 1280x960 will update the GUI to display the first frame, 2048x1536 will not.

EDIT: vbr mp3 audio is not the cause. cbr mp3 avi files also produce the same error.

foxyshadis

10th July 2006, 20:15

Since I work mostly with still images for my resizer, I hardly included any temporal information, but I suspect it wouldn't be too hard to cajole mvtools into providing some backup. I'm always in the market for more ways to make it the slowest filter around. =D

Revgen

10th July 2006, 21:33

Super Resolution is a technique of increasing images or video resolution with superb quality. For each processed frame it uses information not only from this frame but also from other frames. If picture in your video doesn't change too fast, then information from several frames can be added to create a picture of a bigger size. No algorithms of ordinary image resize/resample can provide this quality just because there's not enough information in one frame.

Video Enhancer is a tool which uses Super Resolution technique to increase resolution of your video.

Also, it can use a hundred of VirtualDub filters to process your video: deinterlacing, denoising, deshaking, color correction etc.

Main page:
http://www.thedeemon.com/VideoEnhancer/

Compare Super Resolution to bicubic and Lanczos3:
http://www.thedeemon.com/VideoEnhancer/compare.html

Not too bad. However I don't think it's good enough to pay for yet. EEDI2 does a pretty good job and Foxyshadis' EEDI2 resizer function works even better.

The only thing that would tempt me to buy this is if you provided good support for multithreading since I have a Dual-Core CPU. MVTools doesn't really work with multithreading that well so it would give your tool an advantage over any MVTools-EEDI2 based script.

DmitryPopov

10th July 2006, 23:49

Thank you all guys for valuable info!

The problem appears when an mp3 audio stream is present. DirectShow says sample has no time stamp. I'll fix this soon. Till that moment you can still work with videos without mp3 streams inside.

Multithreading support is the next thing planned to be implemented. My algorithm can be paralleled very good.

MrTroy

11th July 2006, 15:04

I have mixed feelings about this one.

In your Star Wars example:
- the archway looks really good
but
- there are all kinds of resizing artefacts around Yoda's head

DmitryPopov

11th July 2006, 23:43

Yes, but you can see his eyes. Does any other upsample method show them? Hardly.

Overall quality is going to be better soon, that was just a version 1.0, just the beginning.

BTW, can anyone post here a frame from the example resized with his favourite upsample method? Like EEDI2?

*.mp4 guy

12th July 2006, 14:13

ImageSource("m4x.jpg").Converttoyv12(matrix="pc.601").Pointresize(100, 60)

MultiSWAR(400, 240, warping=1, StepRatio=2, sharper=true, Sharpen=6, Smoothing=true, Sthresh=128, Sstrength=.8, EDetail=true, Esharp=.33, FDetail=1, Sthresh2=1.1)
image (http://img231.imageshack.us/img231/7664/multiswar8rj.png)

The reason that there is color bleading is becaue MultiSWAR only works in the yv12 colorspace, which in practical usage isn't an issue.

Didée

12th July 2006, 15:04

That's a spot-on confirmation of Dimitry's point, mp4guy. MultiSWAR shows straightened lines, faces blurred and warped-away into posterization. In a word, this shot looks pretty poor.
Temporal superresolution is not about straightening lines in the first place. It is about digging-out detail out of the swamp below the pixel "quantums". And this you will never achieve with spatial filtering.

---

Dimitry - haven't had much time for testing, up to now. The application/GUI definetly has its buggies that should be fixed, you probably know. For example, my above error report was false: the application actually does the conversion; however, directly after pressing "start", the message window "Done" comes up immediately (always). But the conversion in fact is running in the background, then ... one just must not confirm that "Done" message, in order to get a result.

However, bugs aside:

You know that in the past, mostly I haven't been all that gracious with MSU filters ...
... But this time, I give a definite commendation. This is very good and very promising work! Especially when considering it's only a first release.

Well done. Applause.

.
.
.

Downside is that it's proprietary ... no surprise I've a corresponding framework for Avisynth already, for quite some time ...
but getting *those* results is not *that* easy. Really not.

*.mp4 guy

12th July 2006, 16:12

I realize that the results on that frame aren't great, I only posted it becuae DmitryPopov asked for examples of other filters. That said It still performed better then any of the other spacial only filters. The detail restoration that DmitryPopov's filter managed was impressive, and obviously no spacial filter could manage those results because the detail is quite simply not in the source frame.

Revgen

12th July 2006, 16:29

Dimitry - haven't had much time for testing, up to now. The application/GUI definetly has its buggies that should be fixed, you probably know. For example, my above error report was false: the application actually does the conversion; however, directly after pressing "start", the message window "Done" comes up immediately (always). But the conversion in fact is running in the background, then ... one just must not confirm that "Done" message, in order to get a result.

I've only had this issue using .avs files. AVI files work fine. Try using FFDShow's MakeAVI's and it should work.

Revgen

12th July 2006, 16:47

@Dmitry

Does your filter work directly in YV12? Because whenver I load a YV12 video, your app lists "uncompressed RGB" as the first option. Also, when I load an avs file using FFDshow, FFDshow processes the video in "RGB Flipped" colorspace instead of YV12 like it would in WMP.

Revgen

12th July 2006, 21:48

Okay I did comparisons between Foxyshadis' EEDI2 resizer and VRS and EEDI2 looks a lot better to my eyes. I'll post pics later.

DmitryPopov

12th July 2006, 21:55

Thanks again for valuable info and testing!
I haven't got much experience with AviSynth, and I guess most of ordinal users haven't got it either.

Thanks for MultiSWAR example!

Now Super Resolution is done in RGB, no YV12 at this moment.
That's because all the VirtualDub filters work in RGB32 and you can make any chains of VD filters and Super Resolutions in Video Enhancer.
But if there's really a need for YV12 support, that can be done for sure.

Sirber

12th July 2006, 22:03

Most people here use avisynth. VirtualDub is going down as well as VFW.

Revgen

12th July 2006, 22:42

Most people here use avisynth. VirtualDub is going down as well as VFW.

Sirber, although I agree, I don't think we should be starting a VFW flame war just this minute.

Chainmax

12th July 2006, 22:48

Revgen

12th July 2006, 23:31

Okay here is the test I've done. I used a 720p basketball game that I capped awhile back and used PointResize to resize it down to 720x480. PointResize makes the clip look uglier than Lanczos so it should be a challenge each resizer. I used both SuperResolution as well as FoxyShadis' EEDI2 Resizer (http://forum.doom9.org/showthread.php?t=112249&highlight). Here is the package (http://www.filefactory.com/?02e555) I created that has all functions and plugins setup. Just import the ekg.avs into your script.

Here are PNG's (http://www.filefactory.com/?24da86). It has the original HD video, PointResize vid, SuperResolution, and EEDI2.

Here is the setting I use for EEDI2 Resizer.

ekg(grain=0,iter=1,bp=1,sharp=0)

I don't use grain or shapening since the clip is free of grain and is already quite sharp.

IMHO the EEDI2 Resizer more closely resembles the original clip although it does cause slight oversharping in some areas. Keep in mind that I've not tried any other material except sports material since it's the only HD material that I usually cap.

EDIT: EEDI2 Resize package updated.

Revgen

12th July 2006, 23:40

foxyshadis

12th July 2006, 23:46

I like the results with:

ImageSource("sw4x.jpg",end=0)
ox=width
oy=height
PointResize(width/4,height/4)
makemod(8,crop=true)
converttoyv12(matrix="PC.601")
LimitedSharpenFaster(SMode=4,LMode=3,Strength=80,special=true,wide=true,soft=25)
EKG(grain=10,iter=1,bp=1,sharp=2,wsharp=0,gthresh=.5)
EKG(grain=20,iter=1,bp=1,sharp=2,wsharp=.5,gthresh=.5)
spline36resize(ox,oy)
ConvertToRGB32(matrix="PC.601")

http://foxyshadis.slightlydark.com/random/yoda1.jpg (http://foxyshadis.slightlydark.com/random/yoda1.png)
My function's been updated quite a bit (including today!), though I didn't post about it, just updated the first post, so you might want to update the package. ^^;

Since I have no context I can't test the new motion-adaptive part, and it's a bit annoying to start from a jpeg, but that's the way it goes.

A single run with extra sharpening also looks good:

ImageSource("sw4x.jpg",end=0)
ox=width
oy=height
PointResize(width/4,height/4)
makemod(8,crop=true)
converttoyv12(matrix="PC.601")
LimitedSharpenFaster(SMode=4,LMode=3,Strength=80,special=true,wide=true,soft=25)
EKG(grain=20,iter=1,bp=1,sharp=2,wsharp=0,gthresh=.8)
spline36resize(ox,oy)
LimitedSharpenFaster(SMode=4,LMode=3,Strength=80,special=true,wide=true,soft=25)
ConvertToRGB32(matrix="PC.601")

http://foxyshadis.slightlydark.com/random/yoda2.jpg (http://foxyshadis.slightlydark.com/random/yoda2.png)

That grain is simply my taste, not a necessary part of the script. It's a bit hard to balance between the "flowy" look two iterations leads to and a detailed look. Motion might help with that, as you guys've shown, but I have yet to do more than tinker with it.

Revgen

12th July 2006, 23:52

My function's been updated quite a bit (including today!), though I didn't post about it, just updated the first post, so you might want to update the package. ^^;

You managed to update it 3 minutes after I uploaded the package.:D

I'll update it right now.

Chainmax

13th July 2006, 02:00

Yeah, I forgot about that one. It's slow since it uses MVtools and there is no way to speed it up with SetMTmode since MVtools doesn't work well with MultiThreading.

Some people prefer speed, some people prefer image quality. Both deserve to see methods that fit their specific needs. Also, you can't argue with such an amazing result as scharfis_brain got on the 4x resize in my thread :).

Didée

13th July 2006, 02:54

scharfis_brain made a superresolution script for a clip of mine here (http://forum.doom9.org/showthread.php?t=109422). He also made a thread about it here (http://forum.doom9.org/showthread.php?t=81868).

Yeah, I forgot about that one. It's slow since it uses MVtools and there is no way to speed it up with SetMTmode since MVtools doesn't work well with MultiThreading.

That script can be sped up considerably, with just a small change. (Mates, do set those darn idx vars...)
However, this script is a very raw proof-of-concept, and is fully unusable in practice. Tests with re-upsizing a 50% downsized movie source have shown it puts out a softer & less detailed result than a plain lanczos or spline resize, d'oh. And heaps of artefacts. It surely does show the potential on some certain selected frames, but that's just it. Perhaps it's good for such a maximum-digital-crap source like that of Chainmax. But surely not for your "average" source, whatever that might be.

I used a 720p basketball game ... and used PointResize to resize it down to 720x480. PointResize makes the clip look uglier than Lanczos so it should be a challenge each resizer
Invalid test, or at least extremely suboptimal. Period.
With this method, you play the ball directly into eedi2's arms, and tie VSR to a pole, to hinder it from participating at the game. (Too tired for an explanation right know ... if you know what you're doing, you know that's right.)

BTW, are you sure you posted the right screenshot for VSR? There is virtually no difference between that VSR screenshot, and a plain Lanczos-upsize of the pointsample-destroyed version...

Of course, EDI algo's do have a big strength in spatial interpolation. And spatial interpolations you need even in temporal supersampling. That was my point in the "proprietary" comment - the user can not interfere somewhere in the middle of the process, only use post-processing afterwards. But at that time, it's too late for applying EDI ... So you're left at the developper's mercy to have done the job as good as possible.

That's why I love Avisynth. Everything under user's control. :)

Source: zoomed by 2x -vs.- sharp TSR with Avisynth
http://img155.imageshack.us/img155/3205/baseframezoom2x7xo.th.png (http://img155.imageshack.us/my.php?image=baseframezoom2x7xo.png) http://img126.imageshack.us/img126/8049/baseframetsrupsize3sx.th.png (http://img126.imageshack.us/my.php?image=baseframetsrupsize3sx.png)

I like the results with ...
Foxyshadis, everything okay with you? Can we help, is there hope? :D
(Waaay too artificial for my taste ... in this case, I'd prefer the beautifully smooth bicubicresize'd version.)

---

What happened -- me standing up for MSU against the masses? The world is out of order. :D

CruNcher

13th July 2006, 03:16

Dmitry Nice any HDTV Manufacture gonna use that in HDTVs (Hardware implementation) for upscaleing SD input to HD resolutions maybe Samsung ;) hmm btw is it possible to get a list of products in wich your algorithms are used :P

Revgen

13th July 2006, 03:32

Invalid test, or at least extremely suboptimal. Period.
With this method, you play the ball directly into eedi2's arms, and tie VSR to a pole, to hinder it from participating at the game. (Too tired for an explanation right know ... if you know what you're doing, you know that's right.)

Oooh... I must be an EEDI2 conspiracist.:D

I'm interested in an explanation anytime you're ready. BTW is there another resizer you would like me to use like Lanczos?

BTW, are you sure you posted the right screenshot for VSR? There is virtually no difference between that VSR screenshot, and a plain Lanczos-upsize of the pointsample-destroyed version...

It is from VSR. The differences are extremely subtle compared to EEDI2 resizer. I can create some .ebmp files that aren't converted to RGB and send them if you're interested.

foxyshadis

13th July 2006, 03:59

EEDI2 was developed specifically to solve the "every other field missing" problem, which is what you're asking of it, whereas MSU was made to solve the much more common (for its problem set) bilinear/bicubic downsized video problem.

Sorry, Didée, I think a plain resize just looks as if there's a clouded glass window between the screen and me. Looking again, I like the second better, though; I don't think the problem of eedi2 smoothing everything out after a couple of iterations is really possible to workaround in script, so I think combining it with regular resize and a crazy sharpening (SeeSaw would be better than LS here, but LS needs very little tweaking ;)) will give a more pleasant result.

Revgen

13th July 2006, 04:35

Okay I used LanczosResize instead of PointResize and differences are not as big now. SuperResolution does a good job of increasing resolution while keeping details. EEDI2 tends to smooth details a little more. This can be fixed somewhat by using more sharpening. However, it still can't beat EEDI2 when it comes to getting rid of aliasing. If you look at the yellow frame supporting the backboard you can notice a lot more aliasing in the SuperResolution screenshot.

Keep in mind that most sources won't have the diagonal lines, arches, circles, and other tricky stuff that a basketball court would have. SuperResolution should do fine with most movies.

Here's the PNG screen shots (http://www.sendspace.com/file/yhoejc)

Quick Summary:

heavily aliased content=EEDI2

blury low-rez content=SuperResolution

EDIT: Link updated. Using Sendspace now.

foxyshadis

13th July 2006, 05:11

Your upload is bad. eedi and sr pngs fail crc after multiple downloads. (One of the reasons I dislike filefactory, if it isn't corrupt on upload it'll probably corrupt randomly on download.)

Revgen

13th July 2006, 05:18

PNG's have been uploaded to sendpace. Hopefully it works better.

CruNcher

13th July 2006, 05:39

@Didée
yeah the RTL upsize looks amazing too, when we will be able to test it ourselves ? :D

@foxyshadis
your 2nd one reminds me more of the MSU Cartoonizer but that's not bad we really need some fancy Video Effect Scripts for Avisynth :D

and btw lets do a Motion Compensated Logo Remover :D

DmitryPopov

13th July 2006, 07:25

Since I have no context I can't test the new motion-adaptive part, and it's a bit annoying to start from a jpeg, but that's the way it goes.

Just to remind, there's a video clip (starwars, 50 frames) used for testing:
http://www.thedeemon.com/VideoEnhancer/ve-test.avi (2 MB, msu-ls-codec)
If you need to compare temporal algos.

*.mp4 guy

13th July 2006, 12:57

Revgen

13th July 2006, 16:47

MultiSWAR (http://www.yousendit.com/transfer.php?action=download&ufid=AF3144BA29AFC48C), for comparisons sake.

Hey not too bad. It's similar to EEDI2. There is, however, still aliasing on the yellow frame holding the backboard, but not nearly as bad as SuperResolution.

What settings were used?

MfA

13th July 2006, 17:03

Just to remind, there's a video clip (starwars, 50 frames) used for testing:
http://www.thedeemon.com/VideoEnhancer/ve-test.avi (2 MB, msu-ls-codec)
If you need to compare temporal algos.
It would be much better to have the high resolution version and a script to create the downsized version (my preferred resize would be 1-2-1 gaussian blur with 2x decimation in each dimension). That way you can also do objective testing.

*.mp4 guy

13th July 2006, 17:22

Hey not too bad. It's similar to EEDI2. There is, however, still aliasing on the yellow frame holding the backboard, but not nearly as bad as SuperResolution.

What settings were used?

I think I used something very close to this, I can't be sure however becuase I deleted the script.
MultiSWAR(1440, 960, warping=1, StepRatio=4, sharper=true, Sharpen=8, Smoothing=true, Sthresh=128, Sstrength=.8, EDetail=true, Esharp=.25, FDetail=1, Sthresh2=1.1)

If you wan't all of the aliasing removed MultiSWAR can do that to, you just loose some detail. These settings remove more or less all aliasing from that frame.
MultiSWAR(1440, 960, warping=1, StepRatio=4, sharper=true, Sharpen=8, Smoothing=true, Sthresh=128, Sstrength=1, EDetail=true, Esharp=.25, FDetail=1, Sthresh2=1.1)

foxyshadis

13th July 2006, 19:27

EEDI-based algorithms tend to turn details (ie, faces) into random squigglies, and lower-detail areas into very sharp curves. Whereas at least normal resizing methods look rather fuzzy (and/or oversharpened) across everything. I'll play with mine once I have some more free time.

Dayvon

15th July 2006, 01:43

Super Resolution is a technique of increasing images or video resolution with superb quality. For each processed frame it uses information not only from this frame but also from other frames. If picture in your video doesn't change too fast, then information from several frames can be added to create a picture of a bigger size. No algorithms of ordinary image resize/resample can provide this quality just because there's not enough information in one frame.

Video Enhancer is a tool which uses Super Resolution technique to increase resolution of your video.

Also, it can use a hundred of VirtualDub filters to process your video: deinterlacing, denoising, deshaking, color correction etc.

Main page:
http://www.thedeemon.com/VideoEnhancer/

Compare Super Resolution to bicubic and Lanczos3:
http://www.thedeemon.com/VideoEnhancer/compare.html

If this was AVISynth compatible and also added little to no slowdown.... I'd definitely look at picking it up. I like the results and have always been intrigued with upsizing.

BTW, It was said earlier that this filter is fast? Like TIVCT+TDeint fast?

unmei

19th July 2006, 03:19

with the version i downloaded today, it seems i have to press stop when the preview doesn't advance anymore. It doesn't look like it would complete otherwise (xvid avi input, uncompressed avi output).

billou2000

24th July 2006, 19:02

is there a support for interlaced content?

DmitryPopov

24th July 2006, 19:39

Current version of Super Resolution algorithm works with progressive video. In Video Enhancer you can use deinterlacing video filters before SR. Native support for interlaced content is planned for future versions.

DmitryPopov

28th July 2006, 14:59

Today version 1.1 released.
Now works with files containing mp3 audio stream. Previous version stopped processing on such files right at the start. Please check it out.
http://www.thedeemon.com/VideoEnhancer/

Doobie

30th July 2006, 03:08

I only now read this thread. It took so long becaue the title sounded so much like spam.

The last place I want to fudge details is in the encoding. There it would dramatically increases the size of the encoded file while providing only minimal quality improvement. Enlargment of the video should be done on playback.

laserfan

30th July 2006, 03:21

Is Video Enhancer completely self-contained, or are helper apps needed? I tried a couple of "other" filters in a chain, and their Preview buttons didn't seem to do anything i.e. I expected some sort of ability to see the impact of the filter as I adjust its settings?

DmitryPopov

30th July 2006, 10:04

No other apps needed.

Preview function is just not implemented. But you can see what filter does while processing, do not have to process all the file.

laserfan

30th July 2006, 15:29

...you can see what filter does while processing, do not have to process all the file.Thanks, I will work with it some more.

scharfis_brain

30th July 2006, 15:36

I tried this super resolution program on a small video.
It did not achieved any comparable results to my avisynth test scripts I made for that purpose. Meaning it wasn't able to enhance the resolution significantly.

It seems like this application does motion compensation and merges the compensated frames into the current frame using some sort of kernel (highpass filter).
I came to this conclusion because MoComp mismatches are leaving similar artifacts like v-t-deinterlacing kernels do with fast motion. (Kernel Ghosts)
Is this true?

DmitryPopov

31st July 2006, 12:10

Partly true.
It really does motion compensation but the next step is different. Looks more like simplified Kalman Filter.

The difference between slow and fast modes is in motion estimation. Slow mode is almost full search while fast mode is kind of hexagonal search with candidates.

DmitryPopov

12th August 2006, 01:34

Version 1.2 has come out.
Now all filters, coders and decoders work simultaneously in separate threads. This makes Video Enhancer a lot faster on machines with several logical CPUs (multiprocessors, dual cores, hyperthreading).

A simple test with processing DV video with 3 filters and compressing it with XviD showed 1.9 times better speed than VirtualDub on Dual Core AMD Opteron.
Details here:
http://www.thedeemon.com/VideoEnhancer/speed.html

frodeste

13th August 2006, 14:10

DmitryPopov

11th November 2006, 22:30

Version 1.3 released.
Improved support for different file formats and added audio recompression. Enjoy!
http://www.thedeemon.com/VideoEnhancer/

Fizick

19th November 2006, 23:12

We will enjoy when you distribute it under GNU GPL or remove all violations (in MY opinion - I am not a lawyer) of GPL license:
1. Probably you use some part of VirtualDub source code to implement plugins support.
2. Probably dynamicaly linkage with GPL plugins violate copyright of their authors.
3. Certainly distribution of GPL plugins binary without source code (from the same place) violates it too.

Suggestion: read GNU GPL license (also consult GNU faq), remove VirtualDub plugins support and create great VideoEnhancer.

DmitryPopov

20th November 2006, 07:00

1. From VirtualDub I only use Filter.h which comes with following conditions:

// As a special exemption to the GPL in order to permit creation of
// filters that work with multiple programs as well as VirtualDub,
// compiling with this header file shall not be considered creation
// of a derived work; that is, the act of compiling with this header
// file does not require your source code or the resulting module
// to be released in source code form or under a GPL-compatible
// license according to parts (2) and (3) of the GPL. A filter built
// using this header file may thus be licensed or dual-licensed so
// that it may be used with VirtualDub as well as an alternative
// product whose license is incompatible with the GPL.

2. And probably not. Or stop using Xvid and x264 under Windows. ;) Anyway, I do not force anybody to use GPLed filters. You are always free to use only MSU filters that are not GPLed.

3. Video Enhancer installation does not include any. Separate part of the website contains links to the filters, as well as links to their pages with source code.
VD Filter Pack also contains links to the pages with sources.

So, no violations.

Suggestion: stop being ignorant and accusing me of imagined violations when real filters authors don't.

Chainmax

20th November 2006, 07:15

That script can be sped up considerably, with just a small change. (Mates, do set those darn idx vars...)
However, this script is a very raw proof-of-concept, and is fully unusable in practice. Tests with re-upsizing a 50% downsized movie source have shown it puts out a softer & less detailed result than a plain lanczos or spline resize, d'oh. And heaps of artefacts. It surely does show the potential on some certain selected frames, but that's just it. Perhaps it's good for such a maximum-digital-crap source like that of Chainmax. But surely not for your "average" source, whatever that might be.
...

There's no way to turn it into a general-purpose superresolution function then? That's too bad :(. I'd love to see a sped-up version without the framerate conversion part and with some EEDI2 thrown in (maybe it would improve results?).

Fizick

20th November 2006, 19:34

Imagine? Not quite so. :)
1. Even if you do not use other virtualdub source code files:
The exception "permit creation of filters that work with multiple programs as well as VirtualDub...".
"A filter built using this header file may thus be licensed..."
But your VideoEnhancer is not a Vitrualdub filter, but host application for filters.
2. read GNU GPL faq about it. It permits to use OS.
3. Sources must be at the same place (your server)
[Quote]
If distribution of executable or object code is made by offering
access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code[/qoute]

DmitryPopov

21st November 2006, 07:02

1. I see the words "...it may be used with VirtualDub as well as an alternative product whose license is incompatible with the GPL." So I guess Avery meant there could be other non GPLed products that can use the filters.

Technically, usage of Filter.h and dynamic linking with .VDFs is done in a DirectShow filter ;) called VDFilter, not in Video Enhancer itself. Video Enhancer works with VDFilter via DirectShow API just as many videoplayers work with GPLed video codecs via DirectShow and VfW API. I think this way of cooperation is not considered dynamic linking. VDFilter is a separate product:
http://www.thedeemon.com/VDFilter/

2. Yeah, I know OS is different case. But videoplayers and videocodecs are a good example. The way of linking is the same - DirectShow or VfW API. Does just the ability of Windows Media Player to use XviD require WMP to be GPLed? Don't think so.
Does the ability of Video Enhancer to use GPLed plugins require it to be GPLed? Don't think so.

3. Ok, I should put direct links to the sources on the page with filters. Right? Then I'll do it soon.

Fizick

21st November 2006, 18:32

1. IMO, the exception covers other cases:
Some VirtualDub filter uses (call, link) non-GPL program (library, DLL or EXE).
Or: Filter is simultaneously plugin for VirtualDub and other program. (rare example: WNR plugin - it is also plugin for Avisynth).

Thanks for technical info.
In this case Video Enhancer itself is probably OK.
VDFilter is PROBABLY not.
I suggest you to release your VDFilter under GPL.
Generally, it is not only legal (and wording) issue, but respect to Avery and plugins writers.

2. Yes, it is hard to get a clear threshold.
But if you release VDFilter under GPL, it will be probably OK for VideoEnhancer according to your argumentation.

3. No, not links. I say about VirtulaDub Filter pack only (your other links is OK). If you distribute binaries (GPL) without sources from your web site, you must also distribute its (GPL) sources from your web site (the same place), for example as a second pack.

By the way, are you still in MSU?

DmitryPopov

21st November 2006, 18:49

1,2. I'll think about making VDFilter GPLed. By this moment I haven't received any complaints from Avery and plugins writers..
Well, the main thing will be Avery's view on using Filter.h. As for dynamic linking with filters, since I do not force anyone to load GPLed plugins, the ability of linking itself does not imply anything, I guess.

3. Yeah, I should put the sources of filters used in the pack. I'll do it in a few days.

4. Graduated 2 years ago, now working closely with MSU VideoGroup in some projects, but not an official member.

Fizick

21st November 2006, 22:17

forget to add:
"As a special exception, the copyright holders of XviD give you
permission to link XviD with independent modules that communicate with XviD solely through the VFW1.1 and DShow interfaces ....."

satmonk

30th November 2006, 07:30

Hi Didée,

what is Sharp TSR? Your Results looking good, but never heard of Sharp TSR, search-engines give 0 results.
Can you clearify?

thanks
satmonk

fenomeno83

2nd December 2006, 01:38

Can I get a better resolution(HD) from dvd(720x576)?For example frame from 720x576 to 1280x720 (HD)!

Chainmax

2nd December 2006, 02:56

Yes, but probably not with this. Take a look at IIP (http://forum.doom9.org/showthread.php?t=70916).

DmitryPopov

2nd December 2006, 11:38

Can I get a better resolution(HD) from dvd(720x576)?
Sure. That's what Video Enhancer for.

fenomeno83

2nd December 2006, 11:46

but will I have a better quality(high definition)?

DmitryPopov

2nd December 2006, 12:54

If the source wasn't compressed too much then yes, quality shall be enhanced.

SealTooGreat

29th December 2006, 06:50

Nice work!...Indeed, the difference between bicubic,Lanczos3 methods and Video Enhancer on your site is obvious.
Do Video Enhancer support frameservering like virtualdub?!

DmitryPopov

29th December 2006, 07:24

No, frameserving is not supported now.

SealTooGreat

4th January 2007, 17:48

Is it possible to manage Video Enhancer to work with CLI encoders like xvid_encraw.exe or x264.exe?!

Fizick

4th January 2007, 21:08

QE SuperResolution
http://www.soft32.com/download_120675.html

DmitryPopov

28th May 2007, 17:25

I'm back with version 1.5.
Changes:
* Super-resolution quality improved a lot.
* Finally supports VBR audio (like VBR MP3).
* High quality mode is 1,5 times faster now.

http://www.thedeemon.com/VideoEnhancer/

Objective comparison of different upsize methods and solutions here:
http://www.thedeemon.com/articles/video_upsize_methods_comparison.html

P.S. QE SuperResolution is not for resizing video, it's supposed to get single frames. And I couldn't get it to work with uncompressed RGB24 AVIs. :(

MfA

29th May 2007, 23:18

The edges for topaz enhance don't really seem cartoon like to me, it's just that the lack of other detail makes them stand out (plus they are aliased like hell).

foxyshadis

30th May 2007, 03:15

Topaz just released a new version of Moment that supposedly looks a little more natural, less aliased and flat. You could probably use a simple avisynth resize script to shift the image over slightly, so that the PSNR measure is more useful, like I do to get EEDI2 to line up with a normal resize.

DmitryPopov

30th May 2007, 06:44

Yeah, I've tried Topaz Moment 3.2, it gives different picture with additional noise and different artifacts. The problem is that I can't resize video in Moment, only get some stills. And in case of my RGB24 sources it saves BMP files flipped vertically.
http://stuff.thedeemon.com/smiths09-org.jpg
http://stuff.thedeemon.com/smiths09-topenh.jpg
http://stuff.thedeemon.com/smiths09-topmom.jpg
http://stuff.thedeemon.com/smiths09-videoenhancer.jpg
Shifting image with resize script changes the image itself, I don't know how valid will be PSNR values in that case because they will strongly depend on algorithm of shifting. I agree that current PSNR of original shifted video is also not so meaningful, but at least all upsize methods in comparison were in the same conditions.

I tried to contact Topaz asking about the shift and how to avoid it but they never answered.

If anyone here finds a correct way to measure Topaz's PSNR, I'll be very glad.

If anyone have doubts in PSNR values in my comparison, I can provide hi-res originals to measure PSNR with, try it for yourself.

*.mp4 guy

30th May 2007, 08:25

shifting with spline36 will have a very minimal impact on psnr if you are performing the shift on an already upsampled image.

MfA

2nd June 2007, 06:45

How much history does videoenhancer take into account for resizing? I want to construct a couple of short sequences for objective testing of resizers from some HD footage (using gaussian filtering in "linear" colorspace to create the high&low resolution pair of sequences, rather than box/bilinear filtering in gamma corrected colorspace which is not realistic).

foxyshadis

2nd June 2007, 09:26

Well, aside from still looking much too crisp, in the new version it applies automatic levels correction, apparently, so measuring it against anything is probably a lost cause. Oh well.

spline36resize(width,height,-1,-1,-1,-1)
appears to do the trick for Enhance, though I don't know the exact location of pixels in their formula and didn't try to optimize the values for psnr, obviously. As a guess, I'd say the optimal values might be just a little less than that.

DmitryPopov

2nd June 2007, 11:05

> How much history does videoenhancer take into account for resizing?

In a sense, all of it. The method is iterative, previous upsized frame is used to upsize next frame. Generally, 20-100 frames is long enough. In my comparison I used 45-100 frames long videos.

Recently I managed to run another "super-resolution" solution - Resizer plugin for After Effects from Digital Anarchy. This is what I had using its two best methods with default parameters:
http://stuff.thedeemon.com/smiths09-da-sharp.jpg
http://stuff.thedeemon.com/smiths09-da-smooth.jpg

If anyone is familiar with this tool, I'd be happy to hear how to get best quality of it.

Didée

4th June 2007, 12:36

I downloaded the new trial, gave it a shot, aaand ...

It does not work for me. The output is corrupted.

Video Enhancer v1.5 processes only a couple of frames (varying, from ~50 to ~90 frames), and after a short time it says "done".

The very start of the processed output is OK, and after a few frames it starts to look like this:

http://img524.imageshack.us/img524/3901/vsrproblemhx7.th.jpg (http://img524.imageshack.us/my.php?image=vsrproblemhx7.jpg)

Sample of processed output: >link< (http://www.megaupload.com/?d=X9WYWC9O) (5 MB)

The AVS script fed into Video Enhancer was just this:

mpeg2source("F:\Video\Samples\snip4.d2v")
crop(8,80,-8,-80)
BicubicResize( 352,208, 0.2,0.4 )

Is this the expected behaviour?

DmitryPopov

4th June 2007, 12:57

Avisynth support is not garanteed nor even tested.
Save your low res material in a real file (using a lossless codec to prevent distortions) and then process it in VE.

Didée

4th June 2007, 13:11

Ah okay. Then by now, it's tested and approved to not work.
Cool stuff.

Didée

4th June 2007, 13:30

Follow-up: Nada. It still does not work for me.

I saved the above posted script to an AVI file, compressed with Huffyuv (ffdshow, YV12). This AVI file I loaded into Video Enhancer.

The result is basically the same:

- no more "Done!" message. After some frames, Video Enhancer just sits there and does not proceed any more, while "Remaining time" keeps climbing & climbing.

The result is still corrupted. Only thing that has changed is the look of the corruption:

http://img411.imageshack.us/img411/2416/vsrproblemavisourceqj4.th.jpg (http://img411.imageshack.us/my.php?image=vsrproblemavisourceqj4.jpg)

What next? Only uncompressed RGB32 sources? Or must the pixels be translated into ASCII art?

Only thing I can see is that "it does not work", sorry. Your job to tell how to make it work. After all, you're expecting to make money with that thing, aren't you.

Edit:

Got it finally working, with input=uncompressed RGB / output=uncompressed RGB. Very convenient.

The app seems to be quite picky & particular about codecs (i.e., couldn't use mjpeg for output ["cannot connect"] ... also, ffdshow hooked in somewhere in the chain where it shouldn't.)
No mood/time to track down all possible do's and dont's. At least it works ... somehow.

*.mp4 guy

4th June 2007, 16:57

FFdshow has notoriously buggy video encoding frontends, I've had the errors you ran into pop up from time to time whenever I use FFdshows huffyuv encoder, If you used something known to be very stable, like lagarith (old huffyuv doesn't count, to many things hijack its FOURCC) and still had problems then it would be safe to say that its a problem with video enhancer.

Didée

4th June 2007, 17:58

Well, it's not really an issue to me. Knowing how to play the Avisynth piano, there's no need to worry about other's instruments being out-of-tune. ;)

foxyshadis

4th June 2007, 23:29

Unfortunately, that does look like a huffyuv error. :( ffdshow's had one or two huffyuv fixes lately, but I'll see if I can track down any more bugs.

DmitryPopov

5th June 2007, 07:08

Thanks for reporting!
Since all processing is done in RGB32 (to easily build chains of super-resolution and VirtualDub filters) Video Enhancer tries to feed RGB32 data to encoder (and asks from decoder). Some codecs support only YUV, that could be the problem.

I'll try to make VE friendlier to different codecs and avisynth.

SealTooGreat

20th June 2007, 12:40

@DmitryPopov
Can I use Super Resolution without upsizing input material, keeping output res. same as input? And how SR behaves when video is downsized?

DmitryPopov

20th June 2007, 13:01

SR is a technique for getting bigger images/video from several smaller ones. There's no sense of using it without upsizing because upsizing is what it's intended for.
If you have some blocking artifacts or mosaic there are some ways to deal with them (see http://www.thedeemon.com/articles/ ).
When you downsize video in Video Enhancer it just uses Lanczos3 resize. So it's sharper than bilinear or bicubic but nothing special.

chros

13th July 2007, 11:23

Can I use SR in realtime (eg. through ffdshow-avisynth) ?
(I saw the SR.ax in the install dir, but I can't find the External filters section of MPC.)

DmitryPopov

13th July 2007, 11:52

Not yet.
First, this filter is still quite slow (RGB24 320x240 -> 640x480 at 20 fps on a dual AMD Opteron). A real time version may come out in autumn.
Second, you can't use SR.ax outside Video Enhancer, because only VE knows how to set its parameters.

chros

13th July 2007, 17:21

davidhalopoff

22nd August 2009, 00:08

This program does work but it has its limitations and inconsistencies but probably hand in hand with the multitude of input material. Thin material like small screen captures can only improve so much.

Uncompressed mode will usually garble the sound but then recompression with any one one of the auto processing programs- even MS Movie Maker will sort it back out, reduce the file size and get you the cleaned up product you worked for.

It will only improve so much. I have experimented with all sorts of filter chains, input formats and out chop codecs and you sometimes have to run it again with different variables for optimum clean up.

Try to use one of the analysis tools on your input and start taking notes.

I have gotten to where I can guesstimate based on material density and playback appearance how well it will clean up.

You don't have to run DivX Pro in unconstrained mode. If it doesn't hang in preview then it will work.

Best subjective improvement on some war clips was 40%