[TEST] Visual, PSNR, SSIM, VQM tests with RV9, XviD ,DivX, WMV9 ... - Page 12

Sirber · 30th August 2003, 01:38

I have 2 partitions, one for OS and one for Datas. I don,t like to have 2 OS and I'm french

SeeMoreDigital · 30th August 2003, 10:14

It was just a thought!

All our work PC's (all 6 of them) we have 4no 8GB approx WinXP Pro 'boot' partitions, on an independant 40GB hard drive.
One boot partition for geneneral/office use.
One boot partition for video editing. And Mpeg1,2 & 4, WMV9 (.avi and wmv) and QuickTime encoding
One boot partition for video editing. And RealMedia encoding.
One boot partition to be used as a spare, should one of the other 'boots' crash or become unstable.

All four of the 'Hire Out' PC's have WinXP Pro installed on them 3no times - and are formatted at least 6 times a year.

At home my 'Media PC' has 3 boot partitions. XP Home x2 and Linux. And my main PC. And my main PC has XP Home x3.

I take the view, that with hard disc space being so cheap and M$ operating systems being so expensive. It's more economical to install the Windows O/S's more than once! And the whole exercise saves time (and money) in the long term!

Cheers

Sirber · 30th August 2003, 17:03

hum... as a linux addict, I like to have a long and hard uptime

Also, I don't want to waste my precious disk space on multiple Windows installation

C0mPr355 · 3rd September 2003, 19:42

can you tell me what trailer you used ?

I would like to run some tests as well.

ty.

deXtoRious · 3rd September 2003, 19:55

2 C0mPr355

Whom did you ask?

2 all

Sorry for the delays, I'm currently experienced a lack of time because of the disturbing phenomena called school

I will try to finish the test tommorrow, but unfortunately I can't promise anything.

deXtoRious · 12th September 2003, 16:34

I've finally got some free time, for two days

I've done almost all the encodes and tests, but my psnr script (which was kindly given to me by a helpful member of this forum) for rv9 doesn't work.

Code:

n=3635
name="trailer rv9 550kbps.rmvb"
source=avisource("trailer.avi").converttoyuy2().deleteframe(0).deleteframe(n-2)
test=directshowsource(name,fps=25).converttoyuy2().deleteframe(n-1)
compare(test,source,"",name+"psnr.txt",false)

DirectShowSource just shows that RV9 is an unsupported format. Could anyone help me on this?

C0mPr355 · 12th September 2003, 16:36

i meant to direct that question at sirber. Which LOTR trailer did he use.

karl_lillevold · 12th September 2003, 17:10

Quote:

Originally posted by deXtoRious
DirectShowSource just shows that RV9 is an unsupported format. Could anyone help me on this?

You need Gabest's RealMediaSplitter:
http://forum.doom9.org/showthread.php?s=&threadid=56940

You can also enable PSNR in RV9 encoder with the calcPSNR codecProperty [d9 search for calcPSNR], to double-check the Avisynth compare numbers.

deXtoRious · 12th September 2003, 22:06

Thanks!
I thought I was using it, but it turned out I had removed it some time ago. Another stupid mistake from my part

deXtoRious · 13th September 2003, 22:38

When I try to use this script:

Code:

n=3635
name="trailer rv9 550kbps.rmvb"
source=avisource("trailer.avi").converttoyuy2().deleteframe(0).deleteframe(n-2)
test=directshowsource(name,fps=25).converttoyuy2().deleteframe(n-1)

with ssim or vqm, they start yelling that they need yv12 instead of yuv2. So I tried to replace the convert with a converttoyv12. However, after that the compare functions just kept yelling as if nothing had happened...

So, how do I convert this to YV12?

deXtoRious · 14th September 2003, 16:19

Well, I hate to beg for help, but I have to, otherwise my test will never be finished

PLEASE HELP!

14th September 2003, 17:54

dextorious:

Code:

n=3635
name="trailer rv9 550kbps.rmvb"
source=avisource("trailer.avi").converttoyv12().deleteframe(0).deleteframe(n-2)
test=directshowsource(name,fps=25).converttoyv12().deleteframe(n-1)

That should work just fine. Although, iirc, RV9 uses YV12 anyway so you shouldn't need the conversion for that one and if your source avi is in VBLE or YV12 huffyuv or YV12 FFV1 or some other lossless YV12 format then that shouldn't need it either.

deXtoRious · 14th September 2003, 19:03

It still doesn't work, because due to RV9's VFR, the clips are of different length

14th September 2003, 23:13

Well that's RV9's own damn fault isn't it?

Maybe it would be reasonable when comparing a VFR and a CFR video stream to compare the CFR stream's frames to the currently decoded image in the VFR stream? I don't know if that would be possible though. Ask the avisynth gurus.

deXtoRious · 15th September 2003, 04:20

The question is simple: how the hell do I get SSIM and VQM out of RV9???
The test in the first page of this thread clearly shows it's possible.

15th September 2003, 09:40

OK, firstly get the following and load the relevant one for each test using "LoadPlugin("c:\blah\blah\blah.dll")"

VQMCalc DLL

SSIM DLL

They should then be used by scripting your source file, or better, using VBLE and making an AVI as your source file. Then script the following to test VQM:

Code:

LoadPlugin("C:\blah\vqmcalc.dll")
source=AVIsource("C:\Video\Source.avs")
test=AVIsource("C:\Video\XviD.avi")
return VqmCalc(source,test,"results.log")

And this for SSIM:

Code:

LoadPlugin("C:\blah\SSIM.dll")
source=AVIsource("C:\Video\Source.avs")
test=AVIsource("C:\Video\XviD.avi")
return ssim(source,test,"log.csv","score.txt",lumimask=true,scaled=true)

And just change the function name to SSIM and add .Converttoyv12() and the delete frames as necessary.

Also, did you get the latest CVS build of avisynth? (google for 'CVS Avisynth', it should be the first result) As this provides an Overall PSNR value when running a compare. The average PSNR value that is stated can be biased as it just sums the PSNR of each frame and takes and average. The overall PSNR does something like sum all the MSEs (mean-squared-errors) of each frame, takes an average and then puts that into the PSNR formula giving an accurate output for the PSNR.

I hope this helps.

EDIT: Updates and changes to functions.

15th September 2003, 09:59

Oh, and the conditions of the VQMCalc plugin is that both dimensions of your video have to be an integer multiple of 8 (mod 8) and, as you know, they have to be in YV12 format for both SSIM and VQMCalc.

EDIT: Errors.

lilhobo · 15th September 2003, 20:00

for a newbie into thread will means nothing, where can i go to get explanation for the diffrent tests ( encoding methods i guess, dll and such)

so which codec has the best results?

CQ · 10th September 2004, 19:04

hey
1. after writing the script what do you do with it? run in VDub or what? and... please be nice to me

2. how do you obtain those graphs? with excel? how? plz, someone, i really need to know. (to do a subj. test its easy but when it comes to obj i`m stuck, its not my fault that i`m almost a director (stil studying) not a video engineer

please superdump make some time and write a full (or almost) guide (with writing the avs, how to run the avs

, how to make graphs from the cvs data....). a stupid, i mean very stupid codec tester guide

11th September 2004, 09:58

The method hasn't changed in the last year, apart from possibly using the CompareYV12 dll where appropriate. If you use either CompareYV12 or the internal Compare function you can use the Java XviD Stats Viewer to create some very nice antialiased graphs but I don't know how Sagittaire makes his SSIM graphs, he may have written a program for it. And yes, you do just play the resulting script in Vdub or mplayerc or whatever until it's played all the way through and then close it.

For XviD with b-frames in AVI you have to remove the first frame (unless using packed bitstream). Even then it's sometimes a real pain to get it to work. Thankfully everwicked is working on optimising his VQS objective program so that we can use that.

30th August 2003, 01:38	#221 \| Link
Sirber retired developer Join Date: Oct 2002 Location: Canada Posts: 8,978	I have 2 partitions, one for OS and one for Datas. I don,t like to have 2 OS and I'm french __________________ Detritus Software

30th August 2003, 10:14	#222 \| Link
SeeMoreDigital Life's clearer in 4K UHD Join Date: Jun 2003 Location: Notts, UK Posts: 12,227	It was just a thought! All our work PC's (all 6 of them) we have 4no 8GB approx WinXP Pro 'boot' partitions, on an independant 40GB hard drive. One boot partition for geneneral/office use. One boot partition for video editing. And Mpeg1,2 & 4, WMV9 (.avi and wmv) and QuickTime encoding One boot partition for video editing. And RealMedia encoding. One boot partition to be used as a spare, should one of the other 'boots' crash or become unstable. All four of the 'Hire Out' PC's have WinXP Pro installed on them 3no times - and are formatted at least 6 times a year. At home my 'Media PC' has 3 boot partitions. XP Home x2 and Linux. And my main PC. And my main PC has XP Home x3. I take the view, that with hard disc space being so cheap and M$ operating systems being so expensive. It's more economical to install the Windows O/S's more than once! And the whole exercise saves time (and money) in the long term! Cheers __________________ \| I've been testing hardware media playback devices and software A/V encoders and decoders since 2001 \| My Network Layout & A/V Gear \|

30th August 2003, 17:03	#223 \| Link
Sirber retired developer Join Date: Oct 2002 Location: Canada Posts: 8,978	hum... as a linux addict, I like to have a long and hard uptime Also, I don't want to waste my precious disk space on multiple Windows installation __________________ Detritus Software

3rd September 2003, 19:42	#224 \| Link
C0mPr355 On2 dungeon crawler Join Date: Oct 2001 Location: NY Posts: 78	can you tell me what trailer you used ? I would like to run some tests as well. ty. __________________ I know nuhtzing!

12th September 2003, 16:34	#226 \| Link
deXtoRious Registered User Join Date: Feb 2003 Location: Riga, Latvia Posts: 192	I've finally got some free time, for two days I've done almost all the encodes and tests, but my psnr script (which was kindly given to me by a helpful member of this forum) for rv9 doesn't work. Code: n=3635 name="trailer rv9 550kbps.rmvb" source=avisource("trailer.avi").converttoyuy2().deleteframe(0).deleteframe(n-2) test=directshowsource(name,fps=25).converttoyuy2().deleteframe(n-1) compare(test,source,"",name+"psnr.txt",false) DirectShowSource just shows that RV9 is an unsupported format. Could anyone help me on this?

3rd September 2003, 19:55	#225 \| Link
deXtoRious Registered User Join Date: Feb 2003 Location: Riga, Latvia Posts: 192	2 C0mPr355 Whom did you ask? 2 all Sorry for the delays, I'm currently experienced a lack of time because of the disturbing phenomena called school I will try to finish the test tommorrow, but unfortunately I can't promise anything.

12th September 2003, 16:36	#227 \| Link
C0mPr355 On2 dungeon crawler Join Date: Oct 2001 Location: NY Posts: 78	i meant to direct that question at sirber. Which LOTR trailer did he use. __________________ I know nuhtzing!

12th September 2003, 22:06	#229 \| Link
deXtoRious Registered User Join Date: Feb 2003 Location: Riga, Latvia Posts: 192	Thanks! I thought I was using it, but it turned out I had removed it some time ago. Another stupid mistake from my part

13th September 2003, 22:38	#230 \| Link
deXtoRious Registered User Join Date: Feb 2003 Location: Riga, Latvia Posts: 192	When I try to use this script: Code: n=3635 name="trailer rv9 550kbps.rmvb" source=avisource("trailer.avi").converttoyuy2().deleteframe(0).deleteframe(n-2) test=directshowsource(name,fps=25).converttoyuy2().deleteframe(n-1) with ssim or vqm, they start yelling that they need yv12 instead of yuv2. So I tried to replace the convert with a converttoyv12. However, after that the compare functions just kept yelling as if nothing had happened... So, how do I convert this to YV12?

14th September 2003, 16:19	#231 \| Link
deXtoRious Registered User Join Date: Feb 2003 Location: Riga, Latvia Posts: 192	Well, I hate to beg for help, but I have to, otherwise my test will never be finished PLEASE HELP!

14th September 2003, 17:54	#232 \| Link
superdump Guest Posts: n/a	dextorious: Code: n=3635 name="trailer rv9 550kbps.rmvb" source=avisource("trailer.avi").converttoyv12().deleteframe(0).deleteframe(n-2) test=directshowsource(name,fps=25).converttoyv12().deleteframe(n-1) That should work just fine. Although, iirc, RV9 uses YV12 anyway so you shouldn't need the conversion for that one and if your source avi is in VBLE or YV12 huffyuv or YV12 FFV1 or some other lossless YV12 format then that shouldn't need it either.

14th September 2003, 19:03	#233 \| Link
deXtoRious Registered User Join Date: Feb 2003 Location: Riga, Latvia Posts: 192	It still doesn't work, because due to RV9's VFR, the clips are of different length

14th September 2003, 23:13	#234 \| Link
superdump Guest Posts: n/a	Well that's RV9's own damn fault isn't it? Maybe it would be reasonable when comparing a VFR and a CFR video stream to compare the CFR stream's frames to the currently decoded image in the VFR stream? I don't know if that would be possible though. Ask the avisynth gurus.

15th September 2003, 04:20	#235 \| Link
deXtoRious Registered User Join Date: Feb 2003 Location: Riga, Latvia Posts: 192	The question is simple: how the hell do I get SSIM and VQM out of RV9??? The test in the first page of this thread clearly shows it's possible.

15th September 2003, 09:40	#236 \| Link
superdump Guest Posts: n/a	OK, firstly get the following and load the relevant one for each test using "LoadPlugin("c:\blah\blah\blah.dll")" VQMCalc DLL SSIM DLL They should then be used by scripting your source file, or better, using VBLE and making an AVI as your source file. Then script the following to test VQM: Code: LoadPlugin("C:\blah\vqmcalc.dll") source=AVIsource("C:\Video\Source.avs") test=AVIsource("C:\Video\XviD.avi") return VqmCalc(source,test,"results.log") And this for SSIM: Code: LoadPlugin("C:\blah\SSIM.dll") source=AVIsource("C:\Video\Source.avs") test=AVIsource("C:\Video\XviD.avi") return ssim(source,test,"log.csv","score.txt",lumimask=true,scaled=true) And just change the function name to SSIM and add .Converttoyv12() and the delete frames as necessary. Also, did you get the latest CVS build of avisynth? (google for 'CVS Avisynth', it should be the first result) As this provides an Overall PSNR value when running a compare. The average PSNR value that is stated can be biased as it just sums the PSNR of each frame and takes and average. The overall PSNR does something like sum all the MSEs (mean-squared-errors) of each frame, takes an average and then puts that into the PSNR formula giving an accurate output for the PSNR. I hope this helps. EDIT: Updates and changes to functions. Last edited by superdump; 15th September 2003 at 11:29.

15th September 2003, 09:59	#237 \| Link
superdump Guest Posts: n/a	Oh, and the conditions of the VQMCalc plugin is that both dimensions of your video have to be an integer multiple of 8 (mod 8) and, as you know, they have to be in YV12 format for both SSIM and VQMCalc. EDIT: Errors. Last edited by superdump; 15th September 2003 at 11:21.

15th September 2003, 20:00	#238 \| Link
lilhobo Registered User Join Date: Mar 2002 Posts: 410	for a newbie into thread will means nothing, where can i go to get explanation for the diffrent tests ( encoding methods i guess, dll and such) so which codec has the best results?

10th September 2004, 19:04	#239 \| Link
CQ MPEG1 FAN ;-) Join Date: Mar 2004 Location: Galaþi, România Posts: 75	heeeeeeeeeeeeeelp!!!!!!!!!!!!!!!!! hey 1. after writing the script what do you do with it? run in VDub or what? and... please be nice to me 2. how do you obtain those graphs? with excel? how? plz, someone, i really need to know. (to do a subj. test its easy but when it comes to obj i`m stuck, its not my fault that i`m almost a director (stil studying) not a video engineer please superdump make some time and write a full (or almost) guide (with writing the avs, how to run the avs , how to make graphs from the cvs data....). a stupid, i mean very stupid codec tester guide

11th September 2004, 09:58	#240 \| Link
superdump Guest Posts: n/a	The method hasn't changed in the last year, apart from possibly using the CompareYV12 dll where appropriate. If you use either CompareYV12 or the internal Compare function you can use the Java XviD Stats Viewer to create some very nice antialiased graphs but I don't know how Sagittaire makes his SSIM graphs, he may have written a program for it. And yes, you do just play the resulting script in Vdub or mplayerc or whatever until it's played all the way through and then close it. For XviD with b-frames in AVI you have to remove the first frame (unless using packed bitstream). Even then it's sometimes a real pain to get it to work. Thankfully everwicked is working on optimising his VQS objective program so that we can use that.