Doom9's Forum - View Single Post

DTL · 1st April 2023, 16:55

"do you have literature about this approach (covariance for texture/sharpness metrics)? What variables would covariance employ in this case?"

Covariance is major part of most 'complex' metrics (more complex in compare with 'simple' SAD). SSIM and VIF widely uses covariance component. SSIM have 'structure' part based on covariance and VIF also uses covariance in output computing formula. You can look into SSIM and VIF computing functions in mvtools to see how it calculated from blocks arrays samples:

SSIM https://github.com/DTL2020/mvtools/b...MFunctions.cpp

Full SSIM is SSIM_Light * SSIM_Contrast * SSIM_Structure . SSIM_Light is sort of simple average of samples. SSIM_Contrast is some function of (variance_a, variance_b) where variance_a is fsX and variance_b is fsY at https://github.com/DTL2020/mvtools/b...ctions.cpp#L57

And SSIM_Structure is function of covariance(a,b) (fsXY) - https://github.com/DTL2020/mvtools/b...ctions.cpp#L66 . And after getting all 3 components of SSIM output is simple multiplication.

But for some specific analisys each part of SSIM may be used separately. Most interesting for textures may be SSIM_Structure calculated from covariance.

VIF is https://github.com/DTL2020/mvtools/b...FFunctions.cpp

It uses covariance of DWT - https://github.com/DTL2020/mvtools/b...ctions.cpp#L66 (DWT_a component) for VIF_A part and also some covariance for VIF_E part - https://github.com/DTL2020/mvtools/b...ctions.cpp#L98

It may be possible to convert C-computing to AVS scripting computing and you can make full or structure-only SSIM in AVS function.

The VIF DWT-based require DWT (it is not complex but more slow). The most diffrerence between SSIM and VIF for structure part is SSIM compute from direct input blocks samples covariance and VIF compute covariance from DWT-transformed blocks (may be sort of low-pass filtering or twice reducing ?).

1st April 2023, 16:55	#2285 \| Link
DTL Registered User Join Date: Jul 2018 Posts: 1,075	"do you have literature about this approach (covariance for texture/sharpness metrics)? What variables would covariance employ in this case?" Covariance is major part of most 'complex' metrics (more complex in compare with 'simple' SAD). SSIM and VIF widely uses covariance component. SSIM have 'structure' part based on covariance and VIF also uses covariance in output computing formula. You can look into SSIM and VIF computing functions in mvtools to see how it calculated from blocks arrays samples: SSIM https://github.com/DTL2020/mvtools/b...MFunctions.cpp Full SSIM is SSIM_Light * SSIM_Contrast * SSIM_Structure . SSIM_Light is sort of simple average of samples. SSIM_Contrast is some function of (variance_a, variance_b) where variance_a is fsX and variance_b is fsY at https://github.com/DTL2020/mvtools/b...ctions.cpp#L57 And SSIM_Structure is function of covariance(a,b) (fsXY) - https://github.com/DTL2020/mvtools/b...ctions.cpp#L66 . And after getting all 3 components of SSIM output is simple multiplication. But for some specific analisys each part of SSIM may be used separately. Most interesting for textures may be SSIM_Structure calculated from covariance. VIF is https://github.com/DTL2020/mvtools/b...FFunctions.cpp It uses covariance of DWT - https://github.com/DTL2020/mvtools/b...ctions.cpp#L66 (DWT_a component) for VIF_A part and also some covariance for VIF_E part - https://github.com/DTL2020/mvtools/b...ctions.cpp#L98 It may be possible to convert C-computing to AVS scripting computing and you can make full or structure-only SSIM in AVS function. The VIF DWT-based require DWT (it is not complex but more slow). The most diffrerence between SSIM and VIF for structure part is SSIM compute from direct input blocks samples covariance and VIF compute covariance from DWT-transformed blocks (may be sort of low-pass filtering or twice reducing ?). Last edited by DTL; 1st April 2023 at 17:05.