MPEG1 upscale?

[asarian]

Hello,

I'm trying to find a good filter to smoothen out an old MPEG1 VCD I want to upscale. Like a blurry jpeg, it has these typical low-res blotches on it.

I was wondering whether someone can recommend a good filter for that kind of stuff. Most of the filters I have usually are meant for material that is already HD(-ish).

Thanks.

[vid.user]

By up-scaling you will only loose, never gain, why would you want that?

[asarian]

By up-scaling you will only loose, never gain, why would you want that?

Properly upscaling SD material can often do wonders for how it looks. :) Anyway, the upscaling (Lanczos4Resize) is only the post-process step: it's about getting rid of low-res blotches.

EDIT: I found filters like Blockbuster, but that seems to be doing something else (also useful, btw).

[StainlessS]

Might want to try nnedi3_rpow2 for upscale, also I would prefer Spline36Resize or Spline64Resize.


nnedi3_rpow2(int rfactor, int nsize, int nns, int qual, bool pscrn, string cshift, int fwidth,
int fheight, float ep0, float ep1, int threads, int opt, int fapprox)


No particular suggestion for the blotches, although temporal / spatio temporal can help if blotches vary frame to frame.
My favorite spacio/temp filter is FF3DFilter. (Favorite spacial only is VagueDenoise).
ExBlend() intended for (NTSC) 3/2 blend removal of eg SD converted for VCD via ReduceBy2.

EDIT: Other deblock filters I have on disk are FunkyDeblock, DeBlock_QED, and SmoothDeblock.
Only used SmoothDeblock which was very good for a particular clip (although slow I think).

[creaothceann]

Dither might help too.

[asarian]

Might want to try nnedi3_rpow2 for upscale, also I would prefer Spline36Resize or Spline64Resize.


nnedi3_rpow2(int rfactor, int nsize, int nns, int qual, bool pscrn, string cshift, int fwidth,
int fheight, float ep0, float ep1, int threads, int opt, int fapprox)


No particular suggestion for the blotches, although temporal / spatio temporal can help if blotches vary frame to frame.
My favorite spacio/temp filter is FF3DFilter. (Favorite spacial only is VagueDenoise).
ExBlend() intended for (NTSC) 3/2 blend removal of eg SD converted for VCD via ReduceBy2.

EDIT: Other deblock filters I have on disk are FunkyDeblock, DeBlock_QED, and SmoothDeblock.
Only used SmoothDeblock which was very good for a particular clip (although slow I think).


Thanks! :) I also found some good info on the process at:

How to upscale with Nnedi3? (http://forum.videohelp.com/threads/353535-How-to-upscale-with-Nnedi3)

I always thought Lanczos4Resize was the best, LOL.

Speaking of VCD, 352x288, that doesn't really look like true 4:3 to me (more like 4:3.2727). Should I just upscale to 1440x1080 (4:3) and call it the day?

[feisty2]

eedi3 is a smooth and high quality upscaler
nnedi3 is way too sharp for crappy clips

[feisty2]

use this script
http://pastebin.com/Zj4GMDxG

dither_convert_8_to_16 ()
ediresize16 (x,y,kernel_u="bicubic",noring=true,cplace="mpeg1") #x,y is your target resolution
ditherpost (mode=6)

the result is soft and artifacts free

[asarian]

use this script
http://pastebin.com/Zj4GMDxG

dither_convert_8_to_16 ()
ediresize16 (x,y,kernel_u="bicubic",noring=true,cplace="mpeg1") #x,y is your target resolution
ditherpost (mode=6)

the result is soft and artifacts free

Thank you kindly! :)

[asarian]

Btw, just to give you guys an idea of how very bad it really is:

https://www.dropbox.com/s/2u3gx0ffv8o7yw9/verybad.jpg?dl=0

LOL.

EDIT: Hmm, direct images to dropbox apparently don't work here.

[asarian]

use this script
http://pastebin.com/Zj4GMDxG

dither_convert_8_to_16 ()
ediresize16 (x,y,kernel_u="bicubic",noring=true,cplace="mpeg1") #x,y is your target resolution
ditherpost (mode=6)

the result is soft and artifacts free


Odd. I'm getting the following error all the time:

"There is no function named dither_convert_8_to_16"

[feisty2]

Odd. I'm getting the following error all the time:

"There is no function named dither_convert_8_to_16"

you need dither package

[asarian]

you need dither package

Doh. :o

Just my luck, though, Dither 1.26.1 download site seems gone now. :(

http://ldesoras.free.fr/src/avs/dither-1.26.1.zip

[StainlessS]

EDIT: Hmm, direct images to dropbox apparently don't work here.

Try PhotoBucket.com next time, images appear direct on forum post. (EDIT: Immediately)

[asarian]

you need dither package

Nearly there, I think :) Now I get:

"No function named UnDefined()" in your script, at:

sclip = Default (sclip, UnDefined())

[feisty2]

Nearly there, I think :) Now I get:

"No function named UnDefined()" in your script, at:

sclip = Default (sclip, UnDefined())

whats your avisynth version, the script requires avs 2.6 or avs+

[asarian]

whats your avisynth version, the script requires avs 2.6 or avs+

Oops, I'm still on 2.5.8. Is 2.6 downwards compatible? Don't want to ruin my entire setup.

[feisty2]

Oops, I'm still on 2.5.8. Is 2.6 downwards compatible? Don't want to ruin my entire setup.

2.6 is completely compatible with 2.5.8 scripts and plugins, and there are some new filters can only be called on 2.6 platform

[asarian]

2.6 is completely compatible with 2.5.8 scripts and plugins, and there are some new filters can only be called on 2.6 platform

Okay. :) Thanks for all your help!

[feisty2]

Okay. :) Thanks for all your help!

and you need eedi3mod here instead of eedi3
its much faster (at least 4x faster) and can be even faster if set mclip=true in ediresize16
https://www.doom9.org/showthread.php?p=1654880#476

[asarian]

2.6 is completely compatible with 2.5.8 scripts and plugins, and there are some new filters can only be called on 2.6 platform

Hhm, 2.6 may not be entirely as stable as my good old 2.5.8. After running for a few seconds:

Avisynth error at frame 5:
GetFrameBuffer: Returned a VFB with a 0 data pointer!
size=9596992, max=805306368, used=446743904
I think we have run out of memory folks!

This is not good. :(

[feisty2]

Hhm, 2.6 may not be entirely as stable as my good old 2.5.8. After running for a few seconds:

Avisynth error at frame 5:
GetFrameBuffer: Returned a VFB with a 0 data pointer!
size=9596992, max=805306368, used=446743904
I think we have run out of memory folks!

This is not good. :(

how large is your ram? I guess the script you are encoding is wicked complex so the ram got french fried
you can add "setmemorymax (512)" at the top of the script

[Groucho2004]

you can add "setmemorymax (512)" at the top of the script
That's already the default. No need to set that value.

[asarian]

how large is your ram? I guess the script you are encoding is wicked complex so the ram got french fried
you can add "setmemorymax (512)" at the top of the script

Nah. I have 12G of ram, and the script is very easy:

LoadPlugin("C:\Program Files (x86)\AviSynth 2.5\extra\avss.dll")

SetMemoryMax(768)

SetMTMode(5,4)

DSS2("f:\jobs\video.mpeg").ConvertToYV12()

SetMTMode(3)

QTGMC(InputType=1, Preset="Slow", EdiThreads=2, TR2=3)
MCTemporalDenoise(settings="low", stabilize=true)

crop(0, 2, -2, -2)
Dither_Convert_8_to_16()
ediresize16 (1444, 1080, kernel_u="bicubic", noring=true, cplace="mpeg1")
ditherpost (mode=6)

GradFun3 ()

AddBorders(238, 0, 238, 0)

QTGMC causes the horrible crash. And it's the ediresize16 line that makes the output 100x as slow! (ETA 300 hours). Something is very very wrong.

[feisty2]

oh my lord, you are calling 3 FREAKING SLOW filters (qtgmc,mctemporaldenoise,ediresize16) altogether within one script, I guess the script exceeds the 4GB ram limit of 32bit program limit, my advice would be, doing them step by step, first qtgmc, encode it to a fast lossless format, then load the encoded file then call mctemporaldenoise, encode again, then ediresize16

[Groucho2004]

You're running out of memory. Reduce the number of threads.
Check how much memory your script needs with this (http://forum.doom9.org/showthread.php?t=165528) tool.

If you're on a 64 Bit OS, maximize available memory for the frame serving and encoding by using a pipe tool that has the "LARGEADDRESSAWARE" linker flag set and pipe the output to the encoder. This way the frame server can allocate almost 4 GB.

[Groucho2004]

I guess the script exceeds the 4GB ram limit of 32bit program limit
Sorry to bark again, the user process memory limit for a 32 Bit process is 2 GB. ;)
However, this can be increased to about 4 GB in a 64 Bit OS with a "hack".

[asarian]

oh my lord, you are calling 3 FREAKING SLOW filters (qtgmc,mctemporaldenoise,ediresize16) altogether within one script, I guess the script exceeds the 4GB ram limit of 32bit program limit, my advice would be, doing them step by step, first qtgmc, encode it to a fast lossless format, the load the encoded file then mctemporaldenoise, encode again, then ediresize16

Keep in mind that I have done *many* movies like that on 2.5 (except for ediresize). And QTGMC never crashed like that before.

And even with QTGMC and MCTemporalDenoise commented out, the ediresize16 still causes a 300 hours ETA to finish!

[asarian]

You're running out of memory. Reduce the number of threads.
Check how much memory your script needs with this (http://forum.doom9.org/showthread.php?t=165528) tool.

If you're on a 64 Bit OS, maximize available memory for the frame serving and encoding by using a pipe tool that has the "LARGEADDRESSAWARE" linker flag set and pipe the output to the encoder. This way the frame server can allocate almost 4 GB.

I have done entire Blu-Rays like this with MCTemporalDenoise and QTGMC. Never an issue. I install AviSyn 2.6, an now it runs horribly out of memory on a 352x288 lowly VDC source!? That don't make much sense.

[Groucho2004]

I have done entire Blu-Rays like this with MCTemporalDenoise and QTGMC. Never an issue. I install AviSyn 2.6, an now it runs horribly out of memory on a 352x288 lowly VDC source!? That don't make much sense.
Did you use the MT version of 2.58?

[feisty2]

Sorry to bark again, the user process memory limit for a 32 Bit process is 2 GB. ;)
However, this can be increased to about 4 GB in a 64 Bit OS with a "hack".

thx, I didn't know windows limits the max ram 2GB to 32bit programs before

[asarian]

Did you use the MT version of 2.58?

No, I got this MT 2.6 version today:

http://forum.doom9.org/showthread.php?t=148782

[feisty2]

I have done entire Blu-Rays like this with MCTemporalDenoise and QTGMC. Never an issue. I install AviSyn 2.6, an now it runs horribly out of memory on a 352x288 lowly VDC source!? That don't make much sense.

upscaling 352x288 to 1444x1080 would call eedi3 (dh=true) 9 times inside the ediresize16 script, and we all know how slow eedi3 is, even with the moded version
EDIT: its actually 9 times, not 20 times, I was out of my mind

[Groucho2004]

No, I got this MT 2.6 version today:

http://forum.doom9.org/showthread.php?t=148782
You're using a multi-threaded version of Avisynth (and not correctly, I may add). The additional threads use a lot more memory than your old, single-threaded 2.5.8. Remove all "setmtmode()" calls and try again.

[asarian]

You're using a multi-threaded version of Avisynth (and not correctly, I may add). The additional threads use a lot more memory than your old, single-threaded 2.5.8. Remove all "setmtmode()" calls and try again.

What do you mean, not correctly?! Ony DSS2 needs to be in MTMODE 5, the rest can run safely in 3.

And I *always* encode my blu-rays like that. Without the setmt_mode calls, MCTemporalDenoise et. all fails almost instantly.

[asarian]

And now I can't go back to 2.5.8 any more either, as SEt has removed 2.5.8 MT. :(

[feisty2]

What do you mean, not correctly?! Ony DSS2 needs to be in MTMODE 5, the rest can run safely in 3.

And I *always* encode my blu-rays like that. Without the setmt_mode calls, MCTemporalDenoise et. all fails almost instantly.

my apologies for all the inconveniences, but like I said, vcd to 1080p is a LARGE factor of upscaling, it might cause memory issues along with other slow plugins, please just split your script into 2 parts, first part with qtgmc+mctd, second part with single ediresize16, the problem should be gone

[StainlessS]

If you do decide and manage to go back to v2.58, rt_stats has a rt_undefined() func.

[asarian]

my apologies for all the inconveniences, but like I said, vcd to 1080p is a LARGE factor of upscaling, it might cause memory issues along with other slow plugins, please just split your script into 2 parts, first part with qtgmc+mctd, second part with single ediresize16, the problem should be gone

Upon closer examination, it's really ediresize16 that causes the crash (together with the rest). Both QTGMC and TemporalDenoise run fine when ediresize16 is not running. It's the combination that makes it crash.

At least that means my AVS install isn't totally hosed. :) I shall try a multi-pass. Thanks yet again.

[asarian]

If you do decide and manage to go back to v2.58, rt_stats has a rt_undefined() func.

Nah, I decided to stick with 2.6 for a while. :) If anything, feisty2's script has taught me that I can no longer really postpone the upgrade; eventually I had to anyway.

[StainlessS]

V2.6 is generally more stable but I am not totally happy with the cache mods that have been implemented since v2.58.
Edit: since v2.6a3.

[Groucho2004]

What do you mean, not correctly?! Ony DSS2 needs to be in MTMODE 5, the rest can run safely in 3.
Mode 3 or 5 are modes for source filters. For the following filters, you should use mode 2 for optimum efficiency.

[Groucho2004]

Without the setmt_mode calls, MCTemporalDenoise et. all fails almost instantly.
That makes no sense. Without the calls, Avisynth MT will use much less memory and will be more stable, although slower.

[asarian]

That makes no sense. Without the calls, Avisynth MT will use much less memory and will be more stable, although slower.

The problem is DSS2. It absolutely needs to be threaded the way I do, surrounded my SetMTMode calls. If I don't, frame seeking on MCTemporalDenoise fails horribly (and will soon crash).

[Groucho2004]

The problem is DSS2. It absolutely needs to be threaded the way I do, surrounded my SetMTMode calls. If I don't, frame seeking on MCTemporalDenoise fails horribly (and will soon crash).
Use FFVideoSource or LWLibavVideoSource.

[asarian]

That makes no sense. Without the calls, Avisynth MT will use much less memory and will be more stable, although slower.

Doing it like this now, for the second pass:


SetMemoryMax(768)

SetMTMode(5,4)

FFVideoSource("f:\jobs\zenon1.mkv")

SetMTMode(3)

Dither_Convert_8_to_16()
ediresize16 (1444, 1080, kernel_u="bicubic", noring=true, cplace="mpeg1")
ditherpost (mode=6)
# GradFun3 ()

AddBorders(238, 0, 238, 0)



SetMTMode(2), btw, immediately makes eedi3 crash ("malloc error").

On my i7 980X (6 cores), at 0.55fps, this is going to take me ~72 hours! Ediresize16 really better be a lot better than Lanczos4Resize! :p (I'll soon find out, once I got a partial clip done).

EDIT: zenon1.mkv, btw, is the lossless output of first pass, using QTGMC and MCTemporalDenoise.

[feisty2]

Doing it like this now, for the

remove all setmtmode stuff, malloc error of eedi3 means you ran out of memory, yet again

[feisty2]

if you are looking for a magic upscale filter, there's no such thing yet, ediresize16 won't offer you HD level delicate details, it just produces less artifacts than other resizers, a quick comparison.
lanczos4 2x upscale
http://thumbnails110.imagebam.com/34782/d555ff347813381.jpg (http://www.imagebam.com/image/d555ff347813381)
ediresize16 2x upscale
http://thumbnails110.imagebam.com/34782/cce2ee347813349.jpg (http://www.imagebam.com/image/cce2ee347813349)
ediresize16 2x upscale with aggressive edi kernel
http://thumbnails109.imagebam.com/34782/837402347813316.jpg (http://www.imagebam.com/image/837402347813316)

[asarian]

remove all setmtmode stuff, malloc error of eedi3 means you ran out of memory, yet again

Didn't run out of memory with SetMTMode(3), though. :)

Just for the heck of it, I removed all SetMTMode() calls. Surprisingly, it seems to make no discernable difference at all: about an equal amount of memory is being used, and CPU usage is at ca 100% too. The latter is actually what surprises me the most, as I don't get how AviSynth still uses all my cores in what is supposed to be a single-threaded process now.

[asarian]

if you are looking for a magic upscale filter, there's no such thing yet, ediresize16 won't offer you HD level delicate details, it just produces less artifacts than other resizers, a quick comparison.
lanczos4 2x upscale
[

No magic is required, or expected. :) As long as it's better than Lanczos4Resize(). :)

[feisty2]

Didn't run out of memory with SetMTMode(3), though. :)

Just for the heck of it, I removed all SetMTMode() calls. Surprisingly, it seems to make no discernable difference at all: about an equal amount of memory is being used, and CPU usage is at ca 100% too. The latter is actually what surprises me the most, as I don't get how AviSynth still uses all my cores in what is supposed to be a single-threaded process now.

because dither, eedi3, nnedi3, everything called in ediresize16 are INTERNAL multi threaded already, you don't need to use the mt feature of avisynth at all.

[asarian]

because dither, eedi3, nnedi3, everything called in ediresize16 are INTERNAL multi threaded already, you don't need to use the mt feature of avisynth at all.

Ah. :) Cool. Thx. That really *is* a step up from 2.5.8!

[asarian]

On my i7 980X (6 cores), at 0.55fps, this is going to take me ~72 hours! Ediresize16 really better be a lot better than Lanczos4Resize! :p (I'll soon find out, once I got a partial clip done).

At still 64 hours left to go, so far it would seem Ediresize16 really *is* a lot better! :)

There's still quite a bit of smudging left from the low-quality MPEG1 source, of course. But I think that's a lot harder to repair than noise.

[feisty2]

At still 64 hours left to go, so far it would seem Ediresize16 really *is* a lot better! :)

There's still quite a bit of smudging left from the low-quality MPEG1 source, of course. But I think that's a lot harder to repair than noise.

you might want an extra strong motion compensated denoise filter like mdegrainn (tr=12,thsad=2000) along with mmask to kill high motion mpeg blocks, and the strongly denoised clip can also be used as a mpeg ringing remover (mosquito noise) along with a special expanded mask near the very edge

[asarian]

you might want an extra strong motion compensated denoise filter like mdegrainn (tr=12,thsad=2000) along with mmask to kill high motion mpeg blocks, and the strongly denoised clip can also be used as a mpeg ringing remover (mosquito noise) along with a special expanded mask near the very edge

Thx. :) I'll be looking into those for the next pass.

[feisty2]

Thx. :) I'll be looking into those for the next pass.

nah, you can't denoise an upscaled clip, these approaches gotta be performed before upscale

[asarian]

you might want an extra strong motion compensated denoise filter like mdegrainn (tr=12,thsad=2000) along with mmask to kill high motion mpeg blocks, and the strongly denoised clip can also be used as a mpeg ringing remover (mosquito noise) along with a special expanded mask near the very edge

My, that's a pretty hefty thsad value. :)

Currently, I'm doing it after this:

QTGMC(InputType=1, Preset="Slow", EdiThreads=2, TR2=3)
MCTemporalDenoise(settings="low", stabilize=true)

SMDegrain (tr=12, thsad=2000)

Or should I now remove one of the other filters?

[feisty2]

My, that's a pretty hefty thsad value. :)

Currently, I'm doing it after this:

QTGMC(InputType=1, Preset="Slow", EdiThreads=2, TR2=3)
MCTemporalDenoise(settings="low", stabilize=true)

SMDegrain (tr=12, thsad=2000)

Or should I now remove one of the other filters?

uh, holy lord, you can't use mdegrainn (tr=12,thsad=2000) on the whole video, it will seriously blur the clip and make it look shitty, like I said, use it with mmask, only apply it on high motion parts, and keep using mctd or whatever sane denoise method for overall denoise
and thsad=2000 can give you tons of motion compensate artifacts with no sweat, so you should use mdegrainn (thsad=2000).repair (last,mode=13) to cleanse artifacts

[asarian]

uh, holy lord, you can't use mdegrainn (tr=12,thsad=2000) on the whole video, it will seriously blur the clip and make it look shitty, like I said, use it with mmask, only apply it on high motion parts, and keep using mctd or whatever sane denoise method for overall denoise


Haha! Shows how much I know, eh?! LOL.

I was still trying to figure out how to create an mmask syntax, with the vectors and all, which seems pretty complicated; so I hadn't used that yet.

[feisty2]

mvtools2 is not that hard to use like many people overrate it,
all mvtools2 filters follow the same following calling format

super=msuper (xxxx)
vectors=super.manalyse (xxxx)
last.mxx (super,vectors,xxxx) or last.mxx (vectors)

I'll try to write a simple script that shows you how to use mmask
blocky=last
deblock=last.smdegrain (tr=12,thsad=2000,blksize=4,search=3,prefilter=3,mode=-1).repair (blocky,mode=13)
super=blocky.msuper (pel=2)
vectors=super.manalyse (search=4,searchparam=2,pelsearch=4)
motionmask=blocky.mmask (vectors,kind=0,gamma=2,ml=5)
mt_merge (blocky,deblock,motionmask)
EDIT: I forgot repair, its important and necessary for such a high thsad value

[asarian]

mvtools2 is not that hard to use like many people overrate it,
all mvtools2 filters follow the same following calling format

super=msuper (xxxx)
vectors=super.manalyse (xxxx)
last.mxx (super,vectors,xxxx) or last.mxx (vectors)

I'll try to write a simple script that shows you how to use mmask
blocky=last
deblock=last.smdegrain (tr=12,thsad=2000,blksize=4,search=3,prefilter=3,mode=-1).repair (blocky,mode=13)
super=blocky.msuper (pel=2)
vectors=super.manalyse (search=4,searchparam=2,pelsearch=4)
motionmask=blocky.mmask (vectors,kind=0,gamma=2,ml=5)
mt_merge (blocky,deblock,motionmask)
EDIT: I forgot repair, its important and necessary for such a high thsad value

Thx a lot! :) Still looks a bit daunting, but I'm going to experiment with it rightaway!

Honestly, you've been a great help! :goodpost:

[feisty2]

Thx a lot! :) Still looks a bit daunting, but I'm going to experiment with it rightaway!

Honestly, you've been a great help! :goodpost:

you can tweak "ml" in mmask func, larger ml = stronger motion detection

[asarian]

mvtools2 is not that hard to use like many people overrate it,
all mvtools2 filters follow the same following calling format

super=msuper (xxxx)
vectors=super.manalyse (xxxx)
last.mxx (super,vectors,xxxx) or last.mxx (vectors)

I'll try to write a simple script that shows you how to use mmask
blocky=last
deblock=last.smdegrain (tr=12,thsad=2000,blksize=4,search=3,prefilter=3,mode=-1).repair (blocky,mode=13)
super=blocky.msuper (pel=2)
vectors=super.manalyse (search=4,searchparam=2,pelsearch=4)
motionmask=blocky.mmask (vectors,kind=0,gamma=2,ml=5)
mt_merge (blocky,deblock,motionmask)
EDIT: I forgot repair, its important and necessary for such a high thsad value


Still experiencing some pretty hefty macroblocking:

https://www.dropbox.com/s/2ark4uwkpfxutut/macro.jpg?dl=0

Despite all the deblocking already done on it. :) The blocks appear to be 8x8 pixels (I enlarged the image a bit, just for viewability).

Is there anything else I can do to get rid of that? I know, it's an awefully crappy VCD source, but still, I had expected the most severe blockiness to be gone by now.

[ChiDragon]

Having just looked at your screenshot, I have some fantastic news for you.

https://itunes.apple.com/us/tv-season/zenon-girl-of-the-21st-century/id873191357

All three of them are 16:9. They've also aired on Disney's HD channels in the US and Europe, but they are just upscaled from SD sources. And Disney USA's deinterlacing for their 720p feed sucks; I think they inherited the hardware bought for ABC in 1998 or something.

[asarian]

Having just looked at your screenshot, I have some fantastic news for you.

https://itunes.apple.com/us/tv-season/zenon-girl-of-the-21st-century/id873191357

All three of them are 16:9. They've also aired on Disney's HD channels in the US and Europe, but they are just upscaled from SD sources. And Disney USA's deinterlacing for their 720p feed sucks; I think they inherited the hardware bought for ABC in 1998 or something.

Thx. :) Yeah, I saw that, prior to starting this project.

The source I have came from a very low quality dual VCD I bought, way back when (only format available). Not a huge fan of iTunes, though: if I buy something, I'd like to keep it on my own computer (and not on an iPad cloud somewhere), so I can post-process it.

I may have to go the iTunes route, though, should this prove to be undoable.

[asarian]

Having just looked at your screenshot, I have some fantastic news for you.

https://itunes.apple.com/us/tv-season/zenon-girl-of-the-21st-century/id873191357

All three of them are 16:9. They've also aired on Disney's HD channels in the US and Europe, but they are just upscaled from SD sources. And Disney USA's deinterlacing for their 720p feed sucks; I think they inherited the hardware bought for ABC in 1998 or something.

Btw, how did you arrive at 16:9? All I'm seeing is "SD Version" on that page. Good chance I'd be buying the same version I already have. :P

[feisty2]

Still experiencing some pretty hefty macroblocking:

https://www.dropbox.com/s/2ark4uwkpfxutut/macro.jpg?dl=0

Despite all the deblocking already done on it. :) The blocks appear to be 8x8 pixels (I enlarged the image a bit, just for viewability).

Is there anything else I can do to get rid of that? I know, it's an awefully crappy VCD source, but still, I had expected the most severe blockiness to be gone by now.

you can use smdegrain (tr=6,thsad=650,thsadc=650,prefilter=3) for overall denoise after deblock
prefilter=3 (dfttest) is chill at deblocking

[ChiDragon]

Not a huge fan of iTunes, though: if I buy something, I'd like to keep it on my own computer (and not on an iPad cloud somewhere), so I can post-process it.
Duuude. Use Requiem to un-DRM the purchase. There's no reason to torture yourself with VCD for these movies.

Btw, how did you arrive at 16:9? All I'm seeing is "SD Version" on that page. Good chance I'd be buying the same version I already have. :P
See here (http://screenshotcomparison.com/comparison/80320). You should be able to view a 20-second preview in the iTunes application too.

The logo shot is cropped on the widescreen master for some reason, but the rest of the movie keeps the centre and adds side info.

[feisty2]

Duuude. Use Requiem to un-DRM the purchase. There's no reason to torture yourself with VCD for these movies.


See here (http://screenshotcomparison.com/comparison/80320). You should be able to view a 20-second preview in the iTunes application too.

The logo shot is cropped on the widescreen master for some reason, but the rest of the movie keeps the centre and adds side info.

not like tv episodes or music videos, movies are at least shot and mastered in 2k resolution, which is a little bit larger than 1080p (actually 2048x1080), so why put yourself in misery of sd upscaled stuff or shitty vcd crap, if it was me, I would just wait for a bluray release

[asarian]

Duuude. Use Requiem to un-DRM the purchase. There's no reason to torture yourself with VCD for these movies.


See here (http://screenshotcomparison.com/comparison/80320). You should be able to view a 20-second preview in the iTunes application too.

The logo shot is cropped on the widescreen master for some reason, but the rest of the movie keeps the centre and adds side info.


This is absolutely amazing! The best one seems to be this:

Best so far (http://screenshotcomparison.com/comparison/42868/picture:1) (and mousover on '720p-FiRE'). Definitely an order of magnitude better than what I have! It looks to be from the Disney-channel, and appears a true remastered version, as it adds border info, like you say, not seen on the 4:3 VCD version I have. Which, btw is, as crappy as this, even *after* post-processing and upscaling!

OMG! It can't be this bad, can it!? (https://www.dropbox.com/s/dnqu934guii8t8l/crapvcd.jpg?dl=0)

Unfortunely, they don't say where you can buy it. :( (I do get 2 "This webpage cannot be found" errors underneath, though).

I got word back from iTunes today too, but they said (translated by me from Dutch):

"I found 3 versions of Zenon in the American iTunes Store:

- Zenon, Girl of the 21st Century
- Zenon, Z3
- Zenon the Sequel

All 3 are in SD, and only available in the American iTunes Store."

Definitely, definitely want that Disney Channel version though!

P.S. And thanks for your life-saving help! :)

[asarian]

not like tv episodes or music videos, movies are at least shot and mastered in 2k resolution, which is a little bit larger than 1080p (actually 2048x1080), so why put yourself in misery of sd upscaled stuff or shitty vcd crap, if it was me, I would just wait for a bluray release

Obviously you're not that familiar with Disney. :P They have a habit of easily letting ppl wait 30+ years ere releasing some of their classics.

Would immediately buy the Blu-Ray, of course, should it ever be released!

[feisty2]

Obviously you're not that familiar with Disney. :P They have a habit of easily letting ppl wait 30+ years ere releasing some of their classics.

Would immediately buy the Blu-Ray, of course, should it ever be released!

2030 is not that far far away, right? :D
I never was a Disney fan, I dislike cheesy happy ending fairytales, guess what, I prefer to choose Once Upon a Time (ABC TV Show) if I gotta watch fairytale stories
back to the point, I took a peek at Canada and the states iTunes store, both only provide sd download, no hd version
Canadian iTunes Store
http://thumbnails109.imagebam.com/34823/ec102d348222529.jpg (http://www.imagebam.com/image/ec102d348222529)
US iTunes Store
http://thumbnails109.imagebam.com/34823/3efc74348222541.jpg (http://www.imagebam.com/image/3efc74348222541)
the movie doesn't seem popular tho, I was only 3 when this movie was premiered, so don't know much about it :o

[asarian]

2030 is not that far far away, right? :D
I never was a Disney fan, I dislike cheesy happy ending fairytales, guess what, I prefer to choose Once Upon a Time (ABC TV Show) if I gotta watch fairytale stories
back to the point, I took a peak at Canada and states iTunes store, both only provide sd download, no hd version
Canadian iTunes Store
http://thumbnails109.imagebam.com/34823/ec102d348222529.jpg (http://www.imagebam.com/image/ec102d348222529)
US iTunes Store
http://thumbnails109.imagebam.com/34823/3efc74348222541.jpg (http://www.imagebam.com/image/3efc74348222541)
the movie doesn't seem popular tho, I was only 3 when this movie was premiered, so don't know much about it :o


When Zenon first aired I was, realistically, probably a bit too old for it. :D But it's just one of those classic feel-good movies I just can't let go of. :)

Back to the topic at hand, thanks for the screenshots. I get to see the same thing: 'SD Only.'

[feisty2]

sd is 720x486 for NTSC broadcast system, that's already way better than vcd clips (352x288), well, only if iTunes version is not vcd upscaled crap, but the chance of that tragedy is so small, simply almost impossible, no director will shoot anything at a vcd resolution

[asarian]

sd is 720x486 for NTSC broadcast system, that's already way better than vcd clips (352x288), well, only if iTunes version is not vcd upscaled crap, but the chance of that tragedy is so small, simply almost impossible, no director will shoot anything at a vcd resolution

Yes, you're right! I assumed SD would just be the upscaled VCD, but I will ask them specifically about it.

Thanks!

[asarian]

you can use smdegrain (tr=6,thsad=650,thsadc=650,prefilter=3) for overall denoise after deblock
prefilter=3 (dfttest) is chill at deblocking

Before I forget, do I add that line directly after the deblock line?

blocky=last
deblock=last.smdegrain (tr=12,thsad=2000,blksize=4,search=3,prefilter=3,mode=-1).repair (blocky,mode=13)
smdegrain (tr=6,thsad=650,thsadc=650,prefilter=3)
super=blocky.msuper (pel=2)
vectors=super.manalyse (search=4,searchparam=2,pelsearch=4)
motionmask=blocky.mmask (vectors,kind=0,gamma=2,ml=5)
mt_merge (blocky,deblock,motionmask)

Or after mt_merge (when the entire initial deblock operation has finished).

[feisty2]

Before I forget, do I add that line directly after the deblock line?

blocky=last
deblock=last.smdegrain (tr=12,thsad=2000,blksize=4,search=3,prefilter=3,mode=-1).repair (blocky,mode=13)
smdegrain (tr=6,thsad=650,thsadc=650,prefilter=3)
super=blocky.msuper (pel=2)
vectors=super.manalyse (search=4,searchparam=2,pelsearch=4)
motionmask=blocky.mmask (vectors,kind=0,gamma=2,ml=5)
mt_merge (blocky,deblock,motionmask)

Or after mt_merge (when the entire initial deblock operation has finished).

I'd say, replace "blocky=last" to "blocky=smdegrain (tr=6,thsad=650,thsadc=650,prefilter=3)"

[asarian]

I'd say, replace "blocky=last" to "blocky=smdegrain (tr=6,thsad=650,thsadc=650,prefilter=3)"

Thanks. :) I know, lotsa noob questions on my end, but these processes take ages, so I figured I better ask, after all.

[feisty2]

Thanks. :) I know, lotsa noob questions on my end, but these processes take ages, so I figured I better ask, after all.

smdegrain can get you 16bpc output, which is better for the latter upscale process,
if you want 16bpc precision, here's the script

blocky=smdegrain (tr=6,thsad=650,thsadc=650,prefilter=3,lsb_out=true)
deblock=last.smdegrain (tr=12,thsad=2000,blksize=4,search=3,prefilter=3,mode=-1,lsb_out=true).dither_repair16 (blocky,mode=13)
super=blocky.ditherpost (mode=-1).msuper (pel=2)
vectors=super.manalyse (search=4,searchparam=2,pelsearch=4)
motionmask=blocky.ditherpost (mode=-1).mmask (vectors,kind=0,gamma=2,ml=5)
Dither_merge16_8 (blocky,deblock,motionmask)

the output will be double heighted, but don't worry, there's nothing wrong about it, when upscaling, remove "dither_convert_8_to_16 ()" cuz your clip is already in stacked 16bpc format

[asarian]

smdegrain can get you 16bpc output, which is better for the latter upscale process,
if you want 16bpc precision, here's the script

blocky=smdegrain (tr=6,thsad=650,thsadc=650,prefilter=3,lsb_out=true)
deblock=last.smdegrain (tr=12,thsad=2000,blksize=4,search=3,prefilter=3,mode=-1,lsb_out=true).dither_repair16 (blocky,mode=13)
super=blocky.ditherpost (mode=-1).msuper (pel=2)
vectors=super.manalyse (search=4,searchparam=2,pelsearch=4)
motionmask=blocky.ditherpost (mode=-1).mmask (vectors,kind=0,gamma=2,ml=5)
Dither_merge16_8 (blocky,deblock,motionmask)

the output will be double heighted, but don't worry, there's nothing wrong about it, when upscaling, remove "dither_convert_8_to_16 ()" cuz your clip is already in stacked 16bpc format


This is getting more brilliant by the minute! :) You're a true asset to the community, far as I'm concerned!

[asarian]

don't worry, there's nothing wrong about it, when upscaling, remove "dither_convert_8_to_16 ()" cuz your clip is already in stacked 16bpc format

I take it you can't do any post-processing on it, after it's in 'doubled' state, right? (like a second QTGMC pass or something). Guess that would horrible mess up the LSB field. :p

[feisty2]

This is getting more brilliant by the minute! :) You're a true asset to the community, far as I'm concerned!

no sweat, but, nah, I'm still a newbie, just happen to dig things about image retouch, so searched around Google and read some documents about digital videos ;)

[feisty2]

I take it you can't do any post-processing on it, after it's in 'doubled' state, right? (like a second QTGMC pass or something). Guess that would horrible mess up the LSB field. :p

if the filter is highbitdepth aware like gotta function name like "xxx16" or got "lsb_in,lsb,lsb_out" kinda stuff in its parameters, you can use it directly on your stacked 16 bpc clip, otherwise, you're gonna have to downconvert the bitdepth back to 8bpc then apply common 8bpc filters on it

[feisty2]

and there're several alternative approaches to apply 8bpc filters without losing the lsb part
you may read the document of "dither" for further details
I'll show you 2 common tricks to apply 8bpc filters while keeping the lsb part

#16bpc stacked input#
source=last
filtered=last.ditherpost (mode=-1).#8bpc filters here#.dither_convert_8_to_16 ()
source.Dither_limit_dif16 (filtered, thr=0.5, elast=3.0, y=3, u=3, v=3)

or this

#16bpc stacked input#
source=last
filtered=last.ditherpost (mode=-1).dither2pre (flt="8bpc filter")
source.Dither_limit_dif16 (filtered, thr=1.0, elast=2.0, y=3, u=3, v=3)

[asarian]

if the filter is highbitdepth aware like gotta function name like "xxx16" or got "lsb_in,lsb,lsb_out" kinda stuff in its parameters, you can use it directly on your stacked 16 bpc clip, otherwise, you're gonna have to downconvert the bitdepth back to 8bpc then apply common 8bpc filters on it

It doesn't look like QTGMC is 16-bit aware. No biggie, as it turns out I was doing it wrong anyway, as you can't make multiple calls to the same filter (which I actually knew, but had forgotten). Apparently the parameters of the last call override everything, and I needed to use PrevGlobals for that.

[feisty2]

It doesn't look like QTGMC is 16-bit aware. No biggie, as it turns out I was doing it wrong anyway, as you can't make multiple calls to the same filter (which I actually knew, but had forgotten). Apparently the parameters of the last call override everything, and I needed to use PrevGlobals for that.

why are you using qtgmc twice? I don't get it, what's that for?
and yes, you can make multiple calls for most filters like blur (1).blur (1).blur (1).blur (1).blur (1) which sometimes gives you better result than single call

[asarian]

why are you using qtgmc twice? I don't get it, what's that for?

It was just a quick experiment. :) I wanted to see whether a last QTGMC call, after all other filtering (but before dithering), would repair the result even more.

[feisty2]

It was just a quick experiment. :) I wanted to see whether a last QTGMC call, after all other filtering (but before dithering), would repair the result even more.

aah, you might wanna take a glance at this thread
http://forum.doom9.org/showthread.php?t=171096

[asarian]

why are you using qtgmc twice? I don't get it, what's that for?
and yes, you can make multiple calls for most filters like blur (1).blur (1).blur (1).blur (1).blur (1) which sometimes gives you better result than single call

I was talking about something like this (just an example):

QTGMC( Preset="Slower", InputType=2)
QTGMC( Preset="Slower", InputType=3)

Apparently, AviSynth will then just use the parameters of the last call globally (and override it for previous calls). Instead, the wiki suggests:

t = QTGMC( Preset="Slower", InputType=2)
b = QTGMC( Preset="Slower", InputType=3, PrevGlobals="Reuse" )
Repair( t, b, 1 )

[asarian]

aah, you might wanna take a glance at this thread
http://forum.doom9.org/showthread.php?t=171096

Already saw it. :) (Good thread!) Which is why I removed the second QTGMC call.

[feisty2]

I was talking about something like this (just an example):

QTGMC( Preset="Slower", InputType=2)
QTGMC( Preset="Slower", InputType=3)

Apparently, AviSynth will then just use the parameters of the last call globally (and override it for previous calls). Instead, the wiki suggests:

t = QTGMC( Preset="Slower", InputType=2)
b = QTGMC( Preset="Slower", InputType=3, PrevGlobals="Reuse" )
Repair( t, b, 1 )

QTGMC( Preset="Slower", InputType=2)
QTGMC( Preset="Slower", InputType=3)
This will do qtgmc filter twice on the original clip, similar to something like "blur (1).blur (1)" I mentioned before
t = QTGMC( Preset="Slower", InputType=2)
b = QTGMC( Preset="Slower", InputType=3, PrevGlobals="Reuse" )
Repair( t, b, 1 )
This is wicked different from the upper script
this script will separately apply qtgmc filter on the original clip with different parameters and return you 2 different clips
how you're gonna do with these 2 clips, that's another story, in this case, 2 clips are connected by "repair" function
as for "prevglobals" parameter, set it "reuse" or just leave it empty won't affect the output clip, set it "reuse", qtgmc will use the vectors already being generated from the previous qtgmc call, so qtgmc won't do the vector search this time, so will save your time, nothing else

[asarian]

Having just looked at your screenshot, I have some fantastic news for you.

https://itunes.apple.com/us/tv-season/zenon-girl-of-the-21st-century/id873191357

All three of them are 16:9. They've also aired on Disney's HD channels in the US and Europe, but they are just upscaled from SD sources. And Disney USA's deinterlacing for their 720p feed sucks; I think they inherited the hardware bought for ABC in 1998 or something.


Sigh. Just got word again from iTunes. Sales rep told me they have it at 720x486, but they're not selling it to me: only via the American iTunes store, and you need an American method of payment. Like an American iTunes Card or creditcard. :(

I'm inclined to pull a Godwin on iTunes now, but that would likely cause me to violate the forum rules. Needless to say, you can imagine how I feel about Apple right now.

[feisty2]

Sigh. Just got word again from iTunes. Sales rep told me they have it at 720x486, but they're not selling it to me: only via the American iTunes store, and you need an American method of payment. Like an American iTunes Card or creditcard. :(

I'm inclined to pull a Godwin on iTunes now, but that would likely cause me to violate the forum rules. Needless to say, you can imagine how I feel about Apple right now.

you can register an American account and purchase the iTunes store version, it's not hard anyway.
I did things like this for like a thousand times, I'm an American, but I registered a Japanese account so I can download exclusive bonus tracks from Japanese version of album releases. I don't know why, but it happens all the time, there're often some exclusive bonus tracks in Japanese releases of albums of American singers. so I have to register a Japanese account.

[asarian]

you can register an American account and purchase the iTunes store version, it's not hard anyway.
I did things like this for like a thousand times, I'm an American, but I registered a Japanese account so I can download exclusive bonus tracks from Japanese version of album releases. I don't know why, but it happens all the time, there're often some exclusive bonus tracks in Japanese releases of albums of American singers. so I have to register a Japanese account.

The sales rep said, literally:

"As a customer that is purchasing in the Dutch iTunes Store it is not possible to purchase American items. You can however change the country of your iTunes Store, but this is not recommendable because you would be unable to install updates or redownload older purchases you have done in the Dutch iTunes Store. Also you will need an American method of payment. Like an American iTunes Card or creditcard."

So, basically, he's telling me I can register an American account just fine, but buy nothing from the American store with it, as I don't have an American method of payment (I'm Dutch, and live in the Netherlands, so why would I?!) Maybe this requirement/restriction does not exist for the Japanese store?!

[feisty2]

that's bullcrap
when you're signing up a new account in US iTunes Store, after filling in basic information like email address and password, you'll go to this payment choosing page
http://thumbnails109.imagebam.com/34829/1e96a4348287578.jpg (http://www.imagebam.com/image/1e96a4348287578)
as you can see, there're various payment methods you can choose
VISA MasterCard PayPal, they are clearly international online banking systems.
you can get a visa credit card from any bank with the time of a blink
EDIT: when you're done getting everything you need in US iTunes Store, you can switch back to your old account in iTunes Store, there's no problem of messing your old stuff

[asarian]

that's bullcrap
when you're signing up a new account in US iTunes Store, after filling in basic information like email address and password, you'll go to this payment choosing page
http://thumbnails109.imagebam.com/34829/1e96a4348287578.jpg (http://www.imagebam.com/image/1e96a4348287578)
as you can see, there're various payment methods you can choose
VISA MasterCard PayPal, they are clearly international online banking systems.
you can get a visa credit card from any bank with the time of a blink
EDIT: when you're done getting everything you need in US iTunes Store, you can switch back to your old account in iTunes Store, there's no problem of messing your old stuff


Well, I have a Mastercard; but it's issued from a Dutch bank, or course. What with Mastercard being American and all, maybe using my Mastercard credit card constitutes as 'an American method of payment,' after all. :)

Back to writing the rep again!

And thx for all your trouble!

[feisty2]

This is the payment choosing page of Japanese iTunes Store
http://thumbnails110.imagebam.com/34830/b43f1b348294431.jpg (http://www.imagebam.com/image/b43f1b348294431)
VISA Amex MasterCard are still available, I gotta visa credit card, and a mastercard too, so I registered the American account with visa and Japanese account with mastercard

[ChiDragon]

It's adorable that you're talking to iTunes tech support about how to bypass their deliberate regional restrictions. :D

Usually people buy a code from a gift card online to buy from an iTunes store from outside their own country. There are entire websites dedicated to this (as well as for PSN, Xbox Marketplace, etc.) and they are sold on eBay too -- all of these services are paid for by a markup on the card's actual value of course.

BTW since the earlier links showed the Canadian store has it for $2 instead of $6, you should be buying it there instead IMO... Same file for cheaper. If you want I could buy a Canadian gift card from the store for you and you could PayPal me the money. I'm not sure the smallest one that can be bought though.

not like tv episodes or music videos, movies are at least shot and mastered in 2k resolution
It's a kids TV movie from 1999, shot on 16mm and mastered on digital tape with interlaced CGI. It's a miracle they were even protecting for 16:9 at the time.

[feisty2]

shot on 16mm film and mastered on digital tape? that's toxic, but will they ever rescan the movie someday? I guess 16 mm film is roughly at the detail level of 2k, while common 35mm at 4k

[asarian]

It's adorable that you're talking to iTunes tech support about how to bypass their deliberate regional restrictions. :D

Hey, it's the iTunes guy himself who suggested it. :p

BTW since the earlier links showed the Canadian store has it for $2 instead of $6, you should be buying it there instead IMO... Same file for cheaper. If you want I could buy a Canadian gift card from the store for you and you could PayPal me the money. I'm not sure the smallest one that can be bought though.

That's a very kind offer! Wouldn't mind taking you up on that! :) If you're still willing, PM me, please! And thank you!

It's a kids TV movie from 1999, shot on 16mm and mastered on digital tape with interlaced CGI. It's a miracle they were even protecting for 16:9 at the time.

The VCD I got is progressive PAL (no doubt just deinterlaced from the crappy source); but yeah, a widescreen version in higher res *does* exist somewhere! :)

[feisty2]

The VCD I got is progressive PAL (no doubt just deinterlaced from the crappy source); but yeah, a widescreen version in higher res *does* exist somewhere! :)

you will be able to get a 1080p version if the director hasn't thrown the 16mm movie material away as garbage and would love to rescan and remaster the movie someday

[asarian]

that's bullcrap
when you're signing up a new account in US iTunes Store, after filling in basic information like email address and password, you'll go to this payment choosing page
http://thumbnails109.imagebam.com/34829/1e96a4348287578.jpg (http://www.imagebam.com/image/1e96a4348287578)
as you can see, there're various payment methods you can choose
VISA MasterCard PayPal, they are clearly international online banking systems.
you can get a visa credit card from any bank with the time of a blink
EDIT: when you're done getting everything you need in US iTunes Store, you can switch back to your old account in iTunes Store, there's no problem of messing your old stuff


Well, 'bullcrap''notwithstanding, it can't be done. :( I should have known better. They (very aggressively) require a USA billing address and USA funding source. Even using paypal as funding source, they (equally aggressively) require that paypal account has a USA billing address.

In simple terms, they won't let me use a USA store, ever; and I just (*&$%^&*( wasted $12 bucks on an US iTunes gift card.

EDIT: P.S. Gift card must be redeemed from USA iTunes store too, btw. So I'm effed all the way.

[feisty2]

Did you try this
http://thumbnails109.imagebam.com/34847/0d83aa348461154.jpg (http://www.imagebam.com/image/0d83aa348461154)
they must have a backup plan for Americans that don't live in the states, I think this is the plan, when I registered the jp account, I clicked this

[asarian]

Did you try this
http://thumbnails109.imagebam.com/34847/0d83aa348461154.jpg (http://www.imagebam.com/image/0d83aa348461154)
they must have a backup plan for Americans that don't live in the states, I think this is the plan, when I registered the jp account, I clicked this

That option does not exist for me. In fact, they're so aggressive about it, when I start up, thet even says: "This Apple ID is only valid for purchases in the Dutch iTunes store."

[feisty2]

That option does not exist for me. In fact, they're so aggressive about it, when I start up, thet even says: "This Apple ID is only valid for purchases in the Dutch iTunes store."

you gotta change your country to US before you sign up a new account, did you forget that?
EDIT: and sign out your old account before everything!

[ChiDragon]

Guess you don't want that Canadian card anymore.

Remember, you're not the first person to have to deal with Apple's nonsense. There are guides on how to sign up and purchase things from other countries' iTunes stores. I'm not sure which are up-to-date.

http://lifehacker.com/5524520/create-a-dummy-account-to-get-around-app-store-restrictions

You'll also need to be careful which version you use to purchase it, as I don't know the newest version that Requiem supports.

The VCD I got is progressive PAL (no doubt just deinterlaced from the crappy source)
VCDs can't contain interlacing. Typically they just drop one of the fields, which leads to obvious aliasing of sharp onscreen titles.

[feisty2]

iTunes crap has been solved now, I guess the reason why iTunes sets all those outrageous region restrictions is for the price differences, like the screenshots I showed earlier, if there was no region restriction, all Americans would download the movie from Canadian iTunes store, which greedy Apple assumes that sucks!
edit: autocorrect sucks! it gives me more typos than I do it myself

[Reel.Deel]

You'll also need to be careful which version you use to purchase it, as I don't know the newest version that Requiem supports.


Requiem 4.1 supports iTunes 10.7 (http://support.apple.com/kb/DL1576). (works very well, Requiem is the only reason I buy (hard to find) content from iTunes)

[asarian]

iTunes crap has been solved now, I guess the reason why iTunes sets all that outrageous region restrictions is for the price differences, like the screenshots I showed earlier, if there was no region restriction, all Americans would download the movie from Canadian iTunes store, which greedy Apple assumes that sucks!
edit: autocorrect sucks! it gives me more typos than I do it myself

Yup, iTunes certainly be annoying. :) Thanks again for all your help, guys!

Meanwhile, back at the ranch, on a test clip of my other source, this code suddenly makes my Avisynth process hang:

blocky=smdegrain (tr=6,thsad=650,thsadc=650,prefilter=3,lsb_out=true)
deblock=last.smdegrain (tr=12,thsad=2000,blksize=4,search=3,prefilter=3,mode=-1,lsb_out=true).dither_repair16 (blocky,mode=13)
super=blocky.ditherpost (mode=-1).msuper (pel=2)
vectors=super.manalyse (search=4,searchparam=2,pelsearch=4)
motionmask=blocky.ditherpost (mode=-1).mmask (vectors,kind=0,gamma=2,ml=5)
Dither_merge16_8 (blocky,deblock,motionmask)


It does the "blocky=smdegrain" line, but anything else afterwards, and the process just permanently drops to 3% CPU. I guess maybe because it exhaustively tries to look for blocks that are not there?! :P

Is there any way to do just do the dither_repair16() on blocky, sans the rest? And still keep the 16bit output for the next upscaling pass? Cuz that dithering alone seems better than GradFunDB().

^^ EDIT: Would just "smdegrain (tr=6,thsad=650,thsadc=650,prefilter=3,lsb_out=true).dither_repair16 (blocky,mode=13)" do it?

[feisty2]

if your new clip is not blocky, a simple smdegrain call would be fine
and you can't use ediresize16 with those old parameters cuz that would be too soft for not so crappy clips
im on the cellphone at the moment, talk to you later

[asarian]

if your new clip is not blocky, a simple smdegrain call would be fine

Cool. But won't I need the dithered 16-bit output for the second-pass upscaling?

[feisty2]

set lsb_out=true, you will get 16bpc output

[asarian]

set lsb_out=true, you will get 16bpc output

Most peculiar. After this:

smdegrain (tr=6,thsad=650,thsadc=650,prefilter=3,lsb_out=true)

The process comes to a virtual halt, exactly after frame 62. But when I do this:

DeBlock_QED()
smdegrain (tr=6,thsad=650,thsadc=650,prefilter=3,lsb_out=true)

Then the process keeps running. Hmm, I need to examine this some more. :)

[feisty2]

how's your another clip you're working on right now
is the quality crappy, passable, good or ultra high

[feisty2]

for general high quality filtering, follow these steps
dither_convert_8_to_16 ()
ediresize16 (output="rgb48y")
#16 bpc denoise, sharpen, blah blah#
ediresize16 (x,y,curve="709",tv_range=false)
dither_convert_rgb_to_yuv (selectevery (3,0),selectevery (3,1),selectevery(3,2),lsb=true,output="yv24",mode=-1)
#you gotta separate r g b channels to single independent clips first if you wanna apply some temporal filters on it#

[asarian]

how's your another clip you're working on right now
is the quality crappy, passable, good or ultra high

The new source is quite a step up from the old one. :) Thanks for asking. It still has an overall bitrate of only ~1,673 Kbps (@ 640x480, with a 16:9 DAR), but it's MPEG-4, and thus, for that reason alone, far superiorly compressed than that lowly MPEG-1 source. And it appears to be severely deblocked already. The latter wasn't done entirely properly, I think, as there is quite some banding in several spaces.

It's quite illuminiating, btw, what info iTunes put inside it, like:

Encoded date : UTC 2014-05-15 23:55:32
Color primaries : BT.601 NTSC
Transfer characteristics : BT.709
Matrix coefficients : BT.601

If I'm reading the "BT.709" part right, it was remastered from an original HD source (so, not a crappy VCD upscale). So I'm still holding out hope for a Blu-Ray, one day. :)

It also says,

Muxing mode : Final Cut

Which is, of course, *not* all that encouraging. Or... it could mean nothing at all. :p

[feisty2]

no, that's not correct, transfer characteristics is actually the gamma compress curve, not matrix
almost all videos, sd or hd use bt 709 for the gamma compress curve

[asarian]

for general high quality filtering, follow these steps
dither_convert_8_to_16 ()
ediresize16 (output="rgb48y")
#16 bpc denoise, sharpen, blah blah#
ediresize16 (x,y,curve="709",tv_range=false)
dither_convert_rgb_to_yuv (selectevery (3,0),selectevery (3,1),selectevery(3,2),lsb=true,output="yv24",mode=-1)
#you gotta separate r g b channels to single independent clips first if you wanna apply some temporal filters on it#

Ah, where would I be without you?! :) Thanks yet again!

Based on what I saw in the other thread, I had already prepared something (for the second-pass upscale phase):

Dither_convert_yuv_to_rgb (matrix="601", output="rgb48y", noring=true, lsb_in=true)
Dither_convert_rgb_to_yuv (SelectEvery (3, 0), SelectEvery (3, 1), SelectEvery (3, 2), matrix="709", noring=true, lsb=true, mode=0)

ediresize16 (1920, 1080, kernel_u="bicubic", noring=true)
ditherpost (mode=6)

Guess I did that wrong. :) If I'm reading you correctly, I think it should be more something like:

ediresize16 (1920, 1080, kernel_u="bicubic", noring=true, curve="601", tv_range=false, output="rgb48y")
ditherpost (mode=6)
Dither_convert_rgb_to_yuv (SelectEvery (3, 0), SelectEvery (3, 1), SelectEvery (3, 2), lsb=true, output="yv24", mode=-1)

Right?

[feisty2]

srgb is not a linear display system, there's something called gamma compress, widely used in low precision images, it enhances the precision of dark part and sacrifices precision of bright part, because human eyes are more sensitive to dark part, this would minimize the banding problem
edit: stupid typos, i'm gonna throw this smart?phone away soon

[asarian]

srgb is not a linear display system, there's something called gamma compress, widely used in low precision images, it enhances the precision of dark part and sacrifices precision of bright part, because human eyes are more sensitive to dark part, so this would minorize the banding problem

Ah. Learn something every day. :) Thx.

[feisty2]

that's wrong
you gotta convert your clip to rgb BEFORE upscale, but what you did, it converts your clip to rgb after upscale, that's pointless. and don't use noring, it gives you troubles like aliasing and bluring, its useless for high quality source upscale, the edi core is already ringing free

[feisty2]

and dont use bicubic kernel, that's too soft for high quality clips, just leave most parameters to its default, like i wrote earlier

[feisty2]

fine, i give up, i will make typos many as i can with or without autocorrect anyway
maybe off the topic, but any advice?

[asarian]

and dont use bicubic kernel, that's too soft for high quality clips, just leave most parameters to its default, like i wrote earlier

Ah. The "kernel_u="bicubic", noring=true" I had taken from an earlier ediresize16 example by you in this thread. :)

So, this gives me a grand total of:

ediresize16 (output="rgb48y")
ediresize16 (1920, 1080, curve="709", tv_range=false)
ditherpost (mode=6)
Dither_convert_rgb_to_yuv (SelectEvery (3, 0), SelectEvery (3, 1), SelectEvery (3, 2), lsb=true, output="yv24", mode=-1)

[feisty2]

remove that ditherpost stuff, why would you convert rgb48 to rgb24 before convert it to yuv colorspace, that's an avoidable and unnecessary precision loss

[asarian]

remove that ditherpost stuff, why would you convert rgb48 to rgb24 before convert it to yuv colorspace, that's an avoidable and unnecessary precision loss

Hehe. I had just copied these parameters over from the other source examples. :) Thx again, ditherpost() is gone!

[feisty2]

and the script returns the processed clip in yuv444p16 format, you have to set the colorspace "i444" when you encode it with x264

[asarian]

and the script returns the processed clip in yuv444p16 format, you have to set the colorspace "i444" when you encode it with x264

That was actually a copy-error on my part: should have been 'lsb=false.'

From the dither manual, I got this code:

MPEG2Source ("video_sd.d2v")
Dither_convert_8_to_16 ()
Dither_resize16 (1280, 720)
Dither_convert_yuv_to_rgb (matrix="601", output="rgb48y", lsb_in=true)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
Dither_convert_rgb_to_yuv (r, g, b, matrix="709", lsb=false, mode=0)

So, maybe I just shouldn't set the output colorspace at all, as I'm rather confused now what x264 wants. :)

[feisty2]

thats a basic colormatrix convert example, the goal of that script is different from yours, you didn't do anything wrong, just keep the previous script and lsb=true is correct, you are doing different things, keep that

[asarian]

and dont use bicubic kernel, that's too soft for high quality clips, just leave most parameters to its default, like i wrote earlier

What about Spline36, btw? Cuz yeah, that bicubic thingy makes it look a wee too soft.

[feisty2]

the default kernel is 32taps spline, way much better than spline36 when resizing in linear light like what you're doing

[asarian]

the default kernel is 32taps spline, way much better than spline36 when resizing in linear light like what you're doing

LOL. See?! I should just have listened to you and keep the defaults, like you said! :P

and the script returns the processed clip in yuv444p16 format, you have to set the colorspace "i444" when you encode it with x264

So, this won't output it in 8-bit then?

Dither_convert_rgb_to_yuv (SelectEvery (3, 0), SelectEvery (3, 1), SelectEvery (3, 2), lsb=true, output="yv24", mode=-1)


Cuz that will have been the final stage, before compressing it with x264.

[feisty2]

32 taps spline = spline4096resize
everything inside ediresize16 is set to placebo precision by default
you don't need to tweak much if you're a quality freak

[asarian]

32 taps spline = spline4096resize
everything inside ediresize16 is set to placebo precision by default
you don't need to tweak much if you're a quality freak

And that's why I love your EDIResize16 script. :) It's wickedly slow, but I only want the best! And besides, that's what I bought my i7 980X for to begin with. :)

[feisty2]

i suggest you should encode at 10bpc to avoid banding pickles
how to encode at 10bpc is written in dither document

[feisty2]

actually ediresize16 is just a supporting function I wrote for the experimental function package I'm working on, for superclip generating in msuper, but can be used as a high quality resizer also, I wrote the package trying to restore the common dvd video back to ntsc broadcast quality, the package is not useful tho, it's extremely slow, so just for tests

[feisty2]

https://www.sendspace.com/file/oz3e28
the package is here, test it if you want, you can apply "last.edgehd16 (warp=last.warp16 (str=8,edi="nnedi",tv_range=false),tv_range=false)"
right after ediresize16 upscale if you want HD look thin and sharp edge

[asarian]

i suggest you should encode at 10bpc to avoid banding pickles
how to encode at 10bpc is written in dither document

Hmm, never even considered that. :)

That may require a 10-bit x264 (naturally). And I probably have to do this differently too:

"c:\x264\avs2yuv.exe" "f:\jobs\%1.avs" -o - | "c:\x264\x264.exe" - --demuxer y4m ....

Which is how I normally call x264 (from a cmd file). If I recall correctly, that was a trick by Lord_Mulder, so as to maximize available memory to the x264 process. But it will likely ruin the 10-bit color-space.

[asarian]

thats a basic colormatrix convert example, the goal of that script is different from yours, you didn't do anything wrong, just keep the previous script and lsb=true is correct, you are doing different things, keep that

Hmm, obviously I'm doing something wrong, after all. :)

EDIResize16 (output="rgb48y")
EDIResize16 (1920, 1080, curve="709", tv_range=false)
Dither_convert_rgb_to_yuv (SelectEvery (3, 0), SelectEvery (3, 1), SelectEvery (3, 2), lsb=true, output="yv24", mode=-1)

That gives me a 16-bit output (which is good, I reckon), but I'm getting this (ediresize16?) warning:

resize [warning]: converting from yuv444p16le to yuv420p16le

[feisty2]

the output clip is yuv 4:4:4 (full sampling), the format got the best quality of all yuv formats, common videos are yuv 4:2:2 or 4:2:0
you have to set both input and output colorspaces "i444" and set the input depth 16 and add dither_out at the end of the script

[asarian]

the output clip is yuv 4:4:4 (full sampling), the format got the best quality of all yuv formats, common videos are yuv 4:2:2 or 4:2:0
you have to set both input and output colorspaces "i444" and set the input depth 16 and add dither_out at the end of the script

Thx. :) I found out about the correct piping for x264-10b now too (I just needed to add a few extra parameters, like '-raw' and one for input depth and resolution). Final encoding is on its way. :)

Thx again for all your help!

[asarian]

Well, can you believe this %^&*()(*&^$# crap?! I had decoded original stream with DGIndexNV, which pegged the file at 25fps. So, 50+ hours and two lossless passes later, when I'm ready to mix in the audio, and it appears out of sync, turns out movie was 23.976fps, after all!

Well, that's the very last time I'll ever use DGIndexNV again!

[feisty2]

you can force a new framerate within container properties

[asarian]

you can force a new framerate within container properties

Wait, you mean I don't have to recode everything again? :) :goodpost: That would be awesome! Gonna try it rightaway!

[asarian]

you can force a new framerate within container properties

So, is I use tsMuxeR to just set it to 23.976, and remix, would that work? Or would I lose frames like that?

[feisty2]

that would work, the container framerate got the higher priority than stream framerate, both only control the play speed, harmless to video quality

[asarian]

that would work, the container framerate got the higher priority than stream framerate, both only control the play speed, harmless to video quality

Brilliant!! Looks like I won't have to redo everything, after all! Thx! :)