nnedi3 vs NeuronDoubler

[Reel.Deel]

A couple of weeks ago I was looking for something and I stumbled across NeuronDoubler, a program that upscales images similar to nnedi3.
NeuronDoubler is not an Avisynth plugin and it's super slow compared to nnedi3 but the output (IMO) can sometimes be better than nnedi3.
It has a sharper output than nnedi3 but it's also a bit more prone to aliasing (go figure). Anyways, I compared some images between the two
and I'd like to share the results. If you guys have better examples showing the good and the bad please share. Also, any feedback is welcomed. :)

You can download NeuronDoubler here: http://loggialogic.blogspot.com/2014/03/neurondoubler-v321.html (http://loggialogic.blogspot.com/2014/03/neurondoubler-v321.html)
That page is in Japanese but the actual program is in English. I don't think NeuronDoubler is open source but the author said it's "free software".

https://dl.dropboxusercontent.com/s/m18f4nbyaoroozq/NeuronDoubler.png

Also, here's a quick video from Wirewriggle (the author) illustrating how NeuronDoubler works: http://www.youtube.com/watch?v=aMXLpFz_Tt4 (http://www.youtube.com/watch?v=aMXLpFz_Tt4)
In the comments section the author lists the recommended requirements and additional info in English.

-------

All of the following images were upscaled by 4x and I used the following settings for nnedi3:
nnedi3_rpow2(rfactor=4, cshift="spline36resize")


Car Show:
http://bengal.missouri.edu/~kes25c/nnedi4/car_show.png
[nnedi3 (https://dl.dropboxusercontent.com/s/8qtmwazqfknk3o7/car_show_nnedi3.png)]
[NeuronDoubler (https://dl.dropboxusercontent.com/s/tplk26blvyo4zp6/car_show_NeuronDoubler.png)]


Castle:
http://bengal.missouri.edu/~kes25c/nnedi4/castle.png
[nnedi3 (https://dl.dropboxusercontent.com/s/r1rvz6wqtj64t9u/Castle_nnedi3.png)]
[NeuronDoubler (https://dl.dropboxusercontent.com/s/7l1qio6ok3p9xvc/castle_NeuronDoubler.png)]


Cat:
http://bengal.missouri.edu/~kes25c/nnedi4/cat.png
[nnedi3 (https://dl.dropboxusercontent.com/s/7uvkolak6hm6i9w/cat_nnedi3.png)]
[NeuronDoubler (https://dl.dropboxusercontent.com/s/171k9rd8oz483ir/cat_NeuronDoubler.png)]


Clown:
http://bengal.missouri.edu/~kes25c/nnedi4/clown.png
[nnedi3 (https://dl.dropboxusercontent.com/s/mhwdf5oy658tjrj/clown_nnedi3.png)]
[NeuronDoubler (https://dl.dropboxusercontent.com/s/zksnsgmqv3g9cmx/clown_NeuronDoubler.png)]


Flowers:
http://bengal.missouri.edu/~kes25c/nnedi4/flowers.png
[nnedi3 (https://dl.dropboxusercontent.com/s/xcoi7hkrcm1h6je/flowers_nnedi3.png)]
[NeuronDoubler (https://dl.dropboxusercontent.com/s/zuljcus1whubbzc/flowers_NeuronDoubler.png)]


Lighthouse:
http://bengal.missouri.edu/~kes25c/nnedi4/lighthouse.png
[nnedi3 (https://dl.dropboxusercontent.com/s/smgqvqds9hfeh08/lighthouse_nnedi3.png)]
[NeuronDoubler (https://dl.dropboxusercontent.com/s/8l0rybu8s2rz6b1/lighthouse_NeuronDoubler.png)]


Meter:
http://bengal.missouri.edu/~kes25c/nnedi4/meter.png
[nnedi3 (https://dl.dropboxusercontent.com/s/ju83ok3ai1978ui/meter_nnedi3.png)]
[NeuronDoubler (https://dl.dropboxusercontent.com/s/a00joar6ayeyd9d/meter_NeuronDoubler.png)]


"wc":
http://bengal.missouri.edu/~kes25c/nnedi4/wc.png
[nnedi3 (https://dl.dropboxusercontent.com/s/czqldcvdx8atfpd/wc_nnedi3.png)]
[NeuronDoubler (https://dl.dropboxusercontent.com/s/rdkpfnt45hil8jj/wc_NeuronDoubler.png)]

-----

Just for fun, some pixel art (here I like nnedi3 more but both have their pros and cons, obviously):

Gundam Wing:
https://dl.dropbox.com/s/k2jl2iepyhuotbx/gundam_wing.png
[nnedi3 (https://dl.dropboxusercontent.com/s/y8z4yzfzuho7udr/gundam_wing_nnedi3.png)]
[NeuronDoubler (https://dl.dropboxusercontent.com/s/tali0pijqi9ne13/gundam_wing_NeuronDoubler.png)]


Street Fighter:
https://dl.dropboxusercontent.com/s/f1xisjkvu0jzkve/street_fighter.png
[nnedi3 (https://dl.dropboxusercontent.com/s/uh0il9mphedeasg/street_fighter_nnedi3.png)]
[NeuronDoubler (https://dl.dropboxusercontent.com/s/0h1pm6kgmr7ojoe/street_fighter_NeuronDoubler.png)]

[feisty2]

I guess u can get a similar result with nnedi3+awarpsharp

[SEt]

To me it looks like NeuronDoubler just have way more ringing.

[TheSkiller]

NeuronDoubler looks slightly artificial to me. It has a tad of "oil painting" look.


Btw, old video games look best if resized by PointResize in powers of 2 imo...

[foxyshadis]

The output looks nearly identical to nnedi3+edge enhancement (like unsharp mask), frankly. That's a feat on its own, but nnedi3 is much faster and lets you use better sharpening.

I was hoping the NeuronDoubler_Algorithm.png document would have details about the training or sharpening, but it mostly describes an abstract of the process. (What the problem is, what neural networks are, that ND has relatively few neurons (64) but sufficient for correct recognition, then about the non-neural logic: that it does basic pattern matching first, that it decides on flat areas that skip neural processing, that it pre-analyzes the angles to pass to neural, and that it will correct pixels with low confidence (with basic resizing?).) There is one interesting tidbit: It always analyses at maximum size (super-resolution, I'm assuming 32x, but maybe 4x repeatedly?), then shrinks as necessary to fit your desired zoom level. I haven't had time to hunt through the site for more.

One line did amuse me: "Each partition processing is performed by professionals," pointing to the neurons. :D

[Reel.Deel]

To me it looks like NeuronDoubler just have way more ringing.

Indeed, with sharp images it creates very noticeable ringing, it seems it works best on soft images. For example, NeuronDoubler IMO worked well with the flowers image. I wonder what makes nnedi3 so well balanced? Much more robust neural network/training data?

------

@foxyshadis

Interesting information, kinda sad that ND is only using 64 neurons. nnedi3_rpow2 by default uses 128. Judging by the speed I assumed ND was processing everything at it's highest quality possible, I guess that's not the case.

Edit: Regarding nnedi3+edge enhancement, tritical was working on nnedi4 and nnsharpen (http://forum.doom9.org/showthread.php?p=1577586#post1577586). The results looked great, I wonder why tritical never released it? For those interested you can find some examples of nnedi4+nnsharpen here (http://bengal.missouri.edu/~kes25c/nnedi4/).

[Bloax]

Btw, old video games look best if resized by PointResize in powers of 2 imo...
Well there's also XBR (http://www.libretro.com/forums/viewtopic.php?p=7762&sid=17ac3b048510d9210637a0495af4fea1#p7762), which doesn't look too shabby either (https://dl.dropboxusercontent.com/u/63152810/streets.png). (In addition to being very fast.)

[creaothceann]

Btw, old video games look best if resized by PointResize in powers of 2 imo...

http://board.byuu.org/viewtopic.php?f=10&t=4719

[foxyshadis]

Well there's also XBR (http://www.libretro.com/forums/viewtopic.php?p=7762&sid=17ac3b048510d9210637a0495af4fea1#p7762), which doesn't look too shabby either (https://dl.dropboxusercontent.com/u/63152810/streets.png). (In addition to being very fast.)

Cool!

http://board.byuu.org/viewtopic.php?f=10&t=4719

Also cool!

There's a heck of a wealth of cool shaders out there these days. I'm so glad to live in a time when each individual player doesn't have to build-in (or have a plugin for) everything, plus they're all open source by design.

Maybe someday, set-top boxes and game consoles will support custom shaders.

[luquinhas0021]

How can I download this software? I get this site https://docs.google.com/file/d/0ByGff51gVCuSXy1udFpodXg4ZFU/edit?pli=1, but there is no way of downloading, because the sub-links takes to nowhere.

[wonkey_monkey]

Click the download icon (downward arrow above horizontal bar) on the top middle of the page.

[luquinhas0021]

NeuronDoubler is better than NNEDI3 and Spline 100 (5 taps)?

[luquinhas0021]

The Windows doesn't open the .bat file included on NeuronDoubler folder.

[Groucho2004]

The Windows doesn't open the .bat file included on NeuronDoubler folder.
Do you have Java installed?

[luquinhas0021]

Yes, i have Java 7 Update 45. Why?

[Groucho2004]

Yes, i have Java 7 Update 45. Why?
Because Java is required. Since you just posted a vague "Batch file does not open", this was my first guess.

Use google translate on this (http://loggialogic.blogspot.jp/2012/06/neurondoubler.html) or this (http://loggialogic.blogspot.jp/2013/10/neurondoublerbatjvm.html) page.

[luquinhas0021]

You already tested this algorithm? It is good (Better than NNEDI3 and Spline 100)?

[Groucho2004]

You already tested this algorithm?
No, just looked at the batch files.

[luquinhas0021]

Have you ever tested some upscaling algorithm? If yes, which? What do you guess the best?

[Groucho2004]

Have you ever tested some upscaling algorithm? If yes, which? What do you guess the best?
All I upscale nowadays are my PAL DVDs using just a horizontal Spline36Resize, 720x576 -> 1024x576 and then re-encode with x264 for playback with a media player.

I did some upscaling experiments in the past and I found nnedi3_rpow2 quite pleasing.

[luquinhas0021]

I guess Spline 100 (5 taps) is better than nnedi3_rpow2, because is more sharper and faster than nnedi3.

[Groucho2004]

I guess Spline 100 (5 taps) is better than nnedi3_rpow2, because is more sharper and faster than nnedi3.
You can't have your cake and eat it too:
Sharper algorithms -> more artefacts

Ultimately it's your decision what you can live with.

[luquinhas0021]

It's not like this! Algorithms that use Edge detection doesn't have artifacts, sometimes, and are sharpers.

[colours]

It's not like this! Algorithms that use Edge detection doesn't have artifacts, sometimes, and are sharpers.

That's only true on the assumption that the edge detection is perfect.

[luquinhas0021]

Colours, explain me what oy wannted say when you said perfect Edge Detection. Perfect in what sense: all edges detection or none artifact detection?

[colours]

Perfect in the sense that it detects all the edges and only the edges.

This is obviously not a well-posed problem because (in the context of photography) the projection from three dimensions to two is lossy, not to mention that things sometimes don't even have hard, defined edges. Also, Moiré patterns tend to mess with edge detectors.

It's simply not possible for an upscaling algorithm to be free of artifacts in every case. (Unless the input is bandlimited and sampling theorem etc.)

Not that this is all too relevant to the thread; if I'm not mistaken (though I very well might be), using a neural network approach to upscaling is more like solving a maximum likelihood problem than doing explicitly edge-directed interpolation. It's probably not entirely accurate to call nnedi3 an "EDI" type of algorithm, for example. The edge-directedness is an emergent property, not something inherent to how it works.