MPEG2DEC + PostProc => Need Beta testers

[Marc FD]

i've just made a Mod for MPEG2DEC : this version uses PostProcessing.
(big thx for Nic because the PostProcessing is from his XviD Dshow filter)

It seems stable and very fast (40 fps on my config with cpu=6 :devil: )

I didn't made real encodes with it yet, i only tested the playback ability.

I would like some beta-testers to test it.
i attached the zipped .dll
to use it after, simply use

MPEG2source("your_dvd2avi_project_file.d2v",cpu=number between 0-6)

0 = normal use (just a bit faster than current MPEG2Source)
1 = [0] + Luma Horizontal Deblock
2 = [1] + Luma Vertical Deblock
3 = [2] + Chroma Horizontal Deblock
4 = [3] + Chroma Vertical Deblock
5 = [4] + Luma Dering
6 = [5] + Chroma Dering

they are other goodies in (like DEBLOCK_HORIZ_USEDC_THR and DEBLOCK_VERT_USEDC_THR but i would add temoral strength normalization before, to avoid unstable results (due to MPEG-2 GOPs structures))

[Damn, when i use it, i feel like in a dream...all blocks are gone :) ]

[Koepi]

...testing now with matrix, let's see the result :)

(well, it's sure a bit slower than before with PP=6...)

Regards,
koepi

[kyousuke]

good mod :)

i'll try this on my anime "soul taker opening"

[kyousuke]

no doc with the dll ! :scared:

marc part pitié, fait des docs, je sais cé chiant mais à chq fois suis obligé de faire des c/c du forum ds un petit txt en attendant tes doc quand yen a pas...

in english, makes doc(s) like donald ^^

(nan pa frapper :p )

[vlad59]

Good work Marc,

It seems to work without any problem with my test vob.

What version of mpeg2dec.dll have you modified, Dividee's one, Tom's one ???

How was school today ;) ;) (sorry for that, I could not retain me ;) )

[ACClarke]

I test it on anime and it appears to be ok
no bug, but i don't have 40 fps with cpu=6 :D

[pandv]

One suggestion, Marc FD.

Can you separate the postprocessing from the Mpeg2 decoding (making a postprocessing only filter available)? Or, is it not possible?

You know, i want to test it with mpeg1 films, and I think this postprocessing filter can be very useful for all the transcoding tasks.

Good work.

Unava.

[MoonWalker]

Hmm..Sound promising..I wonder if it will remove the mosquito noise when you use the MPEG quant with cpu=6..Lets try :)

MoonWalker

[soulfx]

I ran some tests at cpu=6 and found it to do a nice job at smoothing out MPEG2 blocks while not killing the details for the non-blocky parts of the video. Under a compression test it added about 2% without causing any noticable detail loss.

The tests were done on animation, but from comparrisons made between original and cpu=6 it looks as if it should also work very well for "real life" (non anime) video.

The cool thing about applying these filters directly to the source is you can come out with more details and greater compressibilty then applying them after the video has been encoded.

BTW: could the function be called something different besides MPEG2source because I still use the TemporalSmoother from the non post-pro MPEG2DEC.DLL. The same function names causes Plugin Autoloading not to work and I have to call each MPEG2DEC from a LoadPlugin. Maybe rename the function to MPEG2postSource or something like that?

I haven't gotten any errors or anything, so all seems okay.

Peace,
SoulFX

[iago]

@Marc FD,

I tested it on a vob (normal movie) at cpu=6, using MPEG/MPEG quantization, and it improved compressibility by ~4%. I took some screenshots, unfortunately in jpeg format due to band-width limitation :(, but I suppose they still give some idea about the result. They are attached below.

regards,
iago

[Marc FD]

no doc with the dll !

marc part pitié, fait des docs, je sais cé chiant mais à chq fois suis obligé de faire des c/c du forum ds un petit txt en attendant tes doc quand yen a pas...

in english, makes doc(s) like donald ^^

(nan pa frapper )

it's beta, and it's so easy to use... do you really need doc for this one ?

donald release very few filters and test them alot before (it's why i say it's a "reference" for good & easy-to-use filters)

i release alot of experimental filters, i'm not a english-speaker like him, and i don't have his knowledge/wisdow/experience.

but i try to put more and more efforts in my docs !!


Good work Marc,

It seems to work without any problem with my test vob.

What version of mpeg2dec.dll have you modified, Dividee's one, Tom's one ???

How was school today (sorry for that, I could not retain me )

:) ... I used the original source of MPEG2Dec. i know dividee's one, but what did Tom with Mpeg2dec ????

je suis en TS1 :) mais bon c clair que j'ai bien moins de temps pour coder.. mais bon la c le WE alors je vais me defouler :devil:

One suggestion, Marc FD.

Can you separate the postprocessing from the Mpeg2 decoding (making a postprocessing only filter available)? Or, is it not possible?

You know, i want to test it with mpeg1 films, and I think this postprocessing filter can be very useful for all the transcoding tasks.

Good work.

Unava.

sorry, it's impossible :
1) i need some information from the decoder
2) it works in YUV12 space, so i apply post-processing BEFORE yuv12->yuy2 conversion :cool:

@Marc FD,

I tested it on a vob (normal movie) at cpu=6, using MPEG/MPEG quantization, and it improved compressibility by ~4%. I took some screenshots, unfortunately in jpeg format due to band-width limitation , but I suppose they still give some idea about the result. They are attached below.

regards,
iago

thx for the test. I don't need snapshots because i tested it alot with VDub (frame by frame, 4x, ect...)

but i would like tests on INTERLACED MPEG-2 content.
to see if it's blending fields or not.
because i use image-based PostProcessing but i could add field-based post-processing support with 100% accurate auto-detection.
test i've made on interlaced streams seemed ok, but more tests are welcome.

PS : there is a mode in the postprocessing you could use to blur a lot more the picture. Less sharp, but would increase compressibility
i would add it in the next release. but the results are not "static" enough (jumping blured block artifacts). need to ficxit.

BTW, if you made encode tests, please use low quants (2-6) and try PSNR test between unfilitered/filtered source and encoded unfiltered/filtered (4 tests)

[Koepi]

Uh, sorry, but one encoding run takes >6hours now here (4h30 with Nic's decoder *g*), so I won't run PSNR tests on that, my computer is simply too slow for that.

Sorry,

regards,
Koepi

[Marc FD]

We are not testing curve-based compression ;)

10'000 frames would suffise (choose various scenes : walls, actions, zooms, travellings, flashing, ect...)

but the best is in fact the _visual_ impression.
(but it's too subjective :D )

[soulfx]

I was wondering about Interlaced MPEG2 support. I noticed a strange situation once when I was testing out cpu=6 against other settings such as cpu=4 and cpu=2. I was comparing frames when all of a sudden I relized cpu=6 was off by 1 frame (playing it too early). cpu=5 also could be seen as being off by 1 frame compared to the rest, including the source (cpu=0). It had to do something with enabling the DeRinging options in the filter. All the other settings compared frame-per frame.

I was in the middle of trouble shooting to see if IVTC was having some trouble, but then it stopped being off by 1 frame. Definately some weird stuff. I'll try and get this to happen again and get some IVTC frame order to see where it's messing up.

Just thought I would mention this situation since you mentioned Interlaced MPEG2s.

Peace,
SoulFX

[Koepi]

Btw.... when I tell it just "6" instead of "cpu=6", postprocessing still is in effect, right?

[soulfx]

Yep, 6 is the same as cpu=6. At least that's what I figure since it filter's the same at both settings.

I've tried to get the weird frame off by 1 thing and I can't get it to do it anymore. That sorta bugs me, but like I said earlier I think it was just IVTC picking the wrong frames.

Peace,
SoulFX

[Didée]

Since I´m no programmer, I don´t get into the very inner secrets of all this stuff. So excuse me if I have a dumb question:

- Now, we have PP in mpeg2dec
- We have PP in Nic´s DS filter
- We have (extensive) PP in ffdshow

Why is it so difficult to do the "usual" PP stuff (deblocking, deringing) in a standalone AviSynth filter ???

It´d be a big step to be able to run ALL sources through PP, prior to encoding!
Where lies the big problem?

[Nic]

The deblocking filter needs to know about what quant what used on a particular macroblock to aid in the deblocking, you cant get that information as a simple filter in a chain....

Cheers,
-Nic

[Guest]

Please explain how knowing the quant aids in the deblocking process. We are not all familiar with the intricacies. Thank you.

[Nic]

Sorry neuron2 :) That explanation was written in quite a rush :)
The code uses the quantizer of a MB as a rough guide to the level of "deblocking" needed for each block. I must admit I'd have to look at the code again to give more detail. :(

However, a pre-defined figure could be given to give a certain level of deblocking, a threshold can also be changed for the horizontal & vertical axis which decides whether to use a stronger or weaker form of the deblocking filter. The deblocking only occurs around the edges of each macroblock (smoothing the joins, so to speak).

The deringing does not rely on such information, but, like the deblocking filter, does at present work only in YV12 rather than AviSynth's normal YUY2 (although, from memory, the code could be ported without too much bother)

That is about all I can remember at present (England played Portugal today & the alcohol ive intaken is effecting my memory)

Sorry for my original short reply, hope that helps :)

Take Care,
-Nic

[Marc FD]

i think i can write an algo who will estimate the quant used for a block.
i don't know if it would gives good results.

anyways, i need 2 things for that :
- Potential users (i have only one yet ;) )
- YUV12 support in avisynth (maybe in next avisynth 2.x)

@Nic

The PP is simply great when used on bad masterized DVDs. Thx a lot :)

about the DEBLOCK_HORIZ_USEDC_THR/DEBLOCK_VERT_USEDC_THR it's used to moderate the postprocessing, right ? i'm using 20,40 by default in my MPEG2dec+PP, like in your Dshow filter, and it gives good results. i saw that 0,0 was doing a very bug blur, and because of quantizers jumping, was doing horrible artifacts. but if i manage to normalize the quantizer values in time, i think it would do a good compressibility-improver filter, no ??

@All

there is a hidden function in the .dll you've downloaded :devil:
you can set DEBLOCK_HORIZ_USEDC_THR/DEBLOCK_VERT_USEDC_THR, by using :
MPEG2source("src.d2v",cpu=1..6,moderate_h=(int),moderate_v=(int))
notice that low values will blur more and higher values would gives less efficient PP. 20,40 is the default and gives very good results for my on any type of material.

PS : take a look on unfiliter fires/action scenes in a 2h/4Gbytes Hong-Kong anime DVD, then try PP, and you'll be as happy as me.
(i'm not using this for encoding, but for DVD playback on my HDD, using avisynth frameserving to BSplayer (my favourite) and it's better than WinDVD/PowerDVD (and it's free ;) )

[bill_baroud]

*bill_baroud shake his hand


hmm i don't encode so much of dvd, most often it's some ****** avi with fancy codec like divx5 :p


so i would like try it with Avisource() ;)

[Marc FD]

okay.
users who want a Standalone avisynth PP filter (based on Nic's PP) : 2

(with 4 i experiment, with 8 i release a beta ;) )
(no, it a joke)

[Didée]

Shall we make a poll?

:D

[Guest]

Marc,

Look, I count for 10, so get going. :)

Seriously, isn't that one of the mini Holy Grails we've had for a long time.

As we say here in America, "Let's Roll".

Nic, thank you for the explanation. It was highly cogent. Were you talking about cricket?

[Sigmatador]

very very good results with "Ghost in the shell"
--> my new default Anime mpeg2 decoder :D

[Sigmatador]

Originally posted by bill_baroud
*bill_baroud shake his hand


hmm i don't encode so much of dvd, most often it's some ****** avi with fancy codec like divx5 :p


so i would like try it with Avisource() ;)

And what about mpeg1+Postproc, for (S)VCD-rip ...

[ACClarke]

Originally posted by Sigmatador
very very good results with "Ghost in the shell"
--> my new default Anime mpeg2 decoder :D
and very cheap (10 euros if I remember well)

[Sigmatador]

arf! is there a free adware version? ;)

:D :D :D

[ACClarke]

Originally posted by Sigmatador
arf! is there a free adware version? ;)

:D :D :D
Ghost in the shell dispo dans toute les bonnes librairie :D

[Sigmatador]

5€95 :cool: tu croi ki vient dou le dvd ki ma servi de source :D

[Marc FD]

tiens, ca vaut peut être le coup d' acquerir un DVD !
(je n'en possède aucun :( )

5e ca fait combien en francs ??

bon, c pas tout, j'était en train de faire joujou avec XviD et je viens de découvrir un putain de truc bien puissant (comment faire du CopySame/Dup a la puissance 10 dans XviD :devil: )

French powaaaaaaaaaaa

[Sigmatador]

en gros 40fr mais eu y son un peu en rupture de stock un peu partout je croi ^^, et pi y sont passer a blood pour 10€ (genre 9€95 koi)

--> eu c koi ton histoire de supra-copysame stp ? :D :D (truc pa interressant du tou deja :cool: )

(sa mérite un ptit translate dailleur)
--> Marc have found a way to drop duplicated frames with xvid better than copysame/dup

[Marc FD]

Hmmm, je suis pas sur que ca va sortir de chez moi :)
LOL,non en fait je dois beta-tester comme un malade pour éviter d'arriver à de la merde. le principe serait d'ameliorer le MB skip du XviD.

Mais il y a une _grosse_ dificultée.
-> elle m'a fait arrêter tout devellopement sur CopySame (même si j'arrivais a faire un tout petit peu mieux que dup)
-> elle fait que j'utiliserait jamais CopySame (d'ailleurs je vais le virer de mon site)

Elle tient en un seul mot : nuage.

En gros CopySame/Dup sont completement incapables de detecter un mouvement de degradé de quelques pixels (~<4)

Resultat : encode des nuages qui bougent avec dup/copysame et ca va saccader a mort. comme le smearing du DivX 3.11 en bien plus degeulasse :(

Bon exemple : les 300 premières frames de "Pincess Mononoké" (DVD R2) (après le logo ghibli)

[Sigmatador]

bon bah bon courage :) si ta besoin de betatester... ;)

avec copysame sur du NTSC on ne ma jamais raporté de sacade, par contre sur du pal... à tou les coup

[ACClarke]

[QUOTE]Originally posted by Marc FD

bon, c pas tout, j'était en train de faire joujou avec XviD et je viens de découvrir un putain de truc bien puissant (comment faire du CopySame/Dup a la puissance 10 dans XviD :devil: )
[/QUOTE)
very interesting :cool:
so don't speak just work :D

ps: ça marche bien le lycée en codant en meme temps ?

[Marc FD]

Ahhhhh :)

CA MARCHE !!!!

1ère fonctionnalitée du Pre Processing de XviD (sur mon ordi)

results :

XviD : XviD "HEAD" 12/09/2002
Source : Princess Mononoké DVD R2 (3000 first frames)

1st Test : Const Q2 / All Defaults
2nd Test : Const Q2 / All Defaults / Movement-Based Adaptive Skipping
3rd Test : Const Q2 / All Defaults / Dup 1.0b1 (3%)

Size / Avg PSNR
1) 19'036 Kbytes / 45.4 Db
2) 16'570 Kbytes / 45.2 Db
3) 15'958 Kbytes / 44.9 Db


En plus, le test 3) a un debut horriblement saccadé (-2 Db de PSNR sur 100 frames !!)

Alors ? qu'est-ce que tu en dis ?
je crois que vais avoir besoin de beta - testeurs ;)

EDIT : Ah ouais, autre chose : avec 2), il y a 0 frames droppées. je crois que ca vaut le coup de le dire ;)

[ACClarke]

j'ai quelques anime bien pourrite (bien qu'en dvd)
tu me dis ce que tu veut qu'on te renvoi comme info et je marche

c'est vrai qu'a premiere vue dup se fait enterrer......

[MaTTeR]

Why did this thread switch to French all of the sudden:confused:

[Marc FD]

je vais rajouter d'autre trucs de pre proc et je ferais un thread sur le XviD forum. le gros probleme va être que le pre-proc va complètement niquer les stats en 2pass (fichiers trop petits, quantizeurs trop bas, qualitée trop grande ;) )

@Matter

Because we are speaking of a ****** good stuff ;)
French powaaaaaaaaa :D

[ACClarke]

Originally posted by Marc FD
je vais rajouter d'autre trucs de pre proc et je ferais un thread sur le XviD forum. le gros probleme va être que le pre-proc va complètement niquer les stats en 2pass (fichiers trop petits, quantizeurs trop bas, qualitée trop grande ;)
donc pas de 2-pass avec se filtre en gros (ou alors j'ai rien compris)
mais de toute façon pour les anime le 2-pass je m'en sert quasiement jamais.

sorry for this french part...

[Guest]

Gentlepersons,

This is an English language forum. I will tolerate occasional off-the-cuff remarks in other languages, but the main content is to be in English. This not only makes the content accessible to the majority of readers, but it also makes it possible for me to properly exercise my role as moderator.

Also, I have struck Marc FD for profanity. He should know better by now. I am closing this thread.