What I'm assuming are simple AVS questions... [Archive]

View Full Version : What I'm assuming are simple AVS questions...

Typhoon859

8th February 2009, 10:03

1) Basically, here's the code I'm currently using for general quality enhancing purposes:
Import("C:\Program Files (x86)\AviSynth 2.5\plugins\LimitedSharpen.avs")
crop(2,2,-2,-2,true)
UnDot().hqdn3D(0,1,1,4.5).LimitedSharpen(ss_x=1.0,ss_y=1.0,S mode=3,strength=25)
Lanczos4Resize(960,540)
Now, I bet you're wondering why I have a resize to 960x540. Well, I've tested it and for some reason, a resize to 1280 by 720 changes the colors, unlike in the resolution I put. It's not only the 720 resolution that does it and I'm not sure why that happens exactly - makes no sense to me. It sorta makes the colors orangish instead of pink. My question is, why does that happen? I went on to check for what the problem might be and that's when I came across ColorMatrix. When I used it, the colors became EVEN MORE orange. Then I tested ColorMatrix without the resize and it basically does exactly what the resize does at 1280 by 720. Is there like something that does the opposite of ColorMatrix? It's important to note that it's not just the Lanczos4Resize that changes the colors but it's any kind that does it.

2) LimitedSharpen is good but it does one thing that bothers me - It makes the lines very jittery. Is there something I can do about that or is that just how it is? If there's nothing I can do, from your experience, maybe you know of a better sharpener? (I believe that the PS3's Mosquito Noise Reduction Filter does exactly that - it evens out jittery lines. If not that, I don't know what it is but something the PS3 does resolves the issue.)

3) LimitedSharpen doesn't work for any of my 720p videos and I have the Q9450 (Yorkfield Quad-Core 2.66GHz overclocked to 3.2GHz) processor, the GeForce 8800GTS 512 GPU, 8GB of DDR2 1066MHz RAM (4 2GB sticks), and Windows Vista Home Premium 64-bit. It seems retarded to me that the video lags so far behind the audio with this kind of power. Speaking of Quad-Cores, will FFDShow ever be able to do multi-core processing?

4) Is simply resizing the video using Lanczos4 the same as a Blu-Ray player's upscale like the PS3's? How is resizing it using AVS scripts any different than the player resizing it? Better algorithm?

*Can anyone link me to the latest LimitedSharpen AVS code? I think there might be a newer version of it than what I'm using. I copied and pasted it from here: http://forum.doom9.org/showthread.php?s=&threadid=84196&perpage=20&pagenumber=1 - third post. If there isn't any kind of revision, then nevermind.

-Even if you don't reply and have just taken the time to read this, sincerely, I greatly appreciate it. Thank you.

Typhoon859

8th February 2009, 20:36

I'm guessing that they're not as easy as I thought? If anybody knows the answer to any of the questions, that's good too...

J_Darnley

8th February 2009, 21:22

Typhoon859

8th February 2009, 21:30

1 - Nothing in the script changes colours. How are you previewing the script? Does that make some assumption on the matrix to be used based on the dimensions? To ensure that the conversion is constant, use ConvertToRGB(matrix="PC.601") at the end. Replace the matrix with another if you want to, RTM for options.
3 - You could try LimitedSharpenFaster (see here ( See: http://avisynth.org/mediawiki/LimitedSharpen)) and/or the multi-threaded build of Avisynth. You could also try ffdshow-mt but I don't know if it has threaded the post-processing filters.
4 - Probably not. Most hardware scalers use a bilinear or bicubic scaler. Most people prefer anything over bilinear because it is sharper.

1- That's the problem. Nothing in the script is supposed to change any colors but as I described, when I resize to 720p and other resolutions greater than that of what I put (960,540), it lowers the redness in the picture. As I said, ColorMatrix does EXACTLY the same effect. I can post a picture comparison if you want.

3- Does LimitedSharpenFaster work just as well?

4- Well, I did see a difference between using Lanczos4Resize and the player resizing. Lanczos4 worked much better. And uh, so, you basically said that the PS3 is worse at upscaling? Not too clear on what you said...

EDIT: I just tested the ConvertToRGB and it didn't work as I didn't think it would. It remained the same. Why resizing changes the colors makes no sense to me.

poisondeathray

8th February 2009, 22:02

J_Darnley

9th February 2009, 00:29

1- That's the problem. Nothing in the script is supposed to change any colors but as I described, when I resize to 720p and other resolutions greater than that of what I put (960,540), it lowers the redness in the picture. As I said, ColorMatrix does EXACTLY the same effect. I can post a picture comparison if you want.

3- Does LimitedSharpenFaster work just as well?

4- Well, I did see a difference between using Lanczos4Resize and the player resizing. Lanczos4 worked much better. And uh, so, you basically said that the PS3 is worse at upscaling? Not too clear on what you said...

EDIT: I just tested the ConvertToRGB and it didn't work as I didn't think it would. It remained the same. Why resizing changes the colors makes no sense to me.

1 - Well if you always manually convert to RGB and you still get changing colours then I can't help you much. I have no idea what could be causing it. If you want to provide some images, I suggest you use Avisynth's features for capturing the stills to prevent unwanted or incorrect colour conversions.

Use something like:
Import("C:\Program Files (x86)\AviSynth 2.5\plugins\LimitedSharpen.avs")
crop(2,2,-2,-2,true)
UnDot()
hqdn3D(0,1,1,4.5)
LimitedSharpen(ss_x=1.0,ss_y=1.0,S mode=3,strength=25)
ImageWriter("X:\path\to\before")
Lanczos4Resize(960,540)
ImageWriter("X:\path\to\after")
Seek to a few frames where you notice the colour change so that Avisynth can write a couple of images. Zip together the files it produces along with the script that created them so that we can try to figure out what's going on.

3 - LSF should produce exactly the same output with the same settings. I think they have different defaults.

4 - Okay, using lanczos is better. I didn't specifically mention the PS3 because I have no idea how it scales videos. Usually hardware scalers, like a graphics card, use a bilinear scaler which produces a soft image. Lanczos produces a sharp image. Some software can make a graphics card use bicubic which lies somewhere between those. If you have no idea what I mean by soft and sharp, I suggest you start reading some guides. Perhaps this (http://en.wikipedia.org/wiki/Sharpness_(visual))

Typhoon859

9th February 2009, 02:57

Regarding the color changes, how are you determining this? Are you using a media player? What decoder & renderer? Is this viewed through a script editor (e.g. avsp)? or the end product encoded video? What format are the source video, and ending output video?

Post examples if you can, and describe the process by which you are taking screenshots

I'm enabling AVISynth in FFDShow and I put the code in there. This is real-time. The player I'm using is Media Player Classic. The way I'm determining it the differences is by taking a snapshot of the frame without the resize and then another with it. I just do a Print Screen considering I'm not using Overlay, making it possible. It's the same video in both. I'm not encoding - only running the filters. The comparison captures are in the next post...

Typhoon859

9th February 2009, 03:18

1 - Well if you always manually convert to RGB and you still get changing colours then I can't help you much. I have no idea what could be causing it. If you want to provide some images, I suggest you use Avisynth's features for capturing the stills to prevent unwanted or incorrect colour conversions.

Use something like:
Import("C:\Program Files (x86)\AviSynth 2.5\plugins\LimitedSharpen.avs")
crop(2,2,-2,-2,true)
UnDot()
hqdn3D(0,1,1,4.5)
LimitedSharpen(ss_x=1.0,ss_y=1.0,S mode=3,strength=25)
ImageWriter("X:\path\to\before")
Lanczos4Resize(960,540)
ImageWriter("X:\path\to\after")
Seek to a few frames where you notice the colour change so that Avisynth can write a couple of images. Zip together the files it produces along with the script that created them so that we can try to figure out what's going on.

3 - LSF should produce exactly the same output with the same settings. I think they have different defaults.

4 - Okay, using lanczos is better. I didn't specifically mention the PS3 because I have no idea how it scales videos. Usually hardware scalers, like a graphics card, use a bilinear scaler which produces a soft image. Lanczos produces a sharp image. Some software can make a graphics card use bicubic which lies somewhere between those. If you have no idea what I mean by soft and sharp, I suggest you start reading some guides. Perhaps this (http://en.wikipedia.org/wiki/Sharpness_(visual))

4- Hmmm... I always felt that something more advanced was involved in upscaling like the PS3 does. The quality gets quite a boost. I myself see the difference that Lanczos4Resize is sharper than simly the media player's resize - I see it. I don't know what's involved though so as you mentioned, maybe I'll read up on it later.

3- LSF is just better coded or something? For what reason is it faster then if it gives the same results? Why would anybody use the regular one?

-In terms of the captures, I just print the screen at the same frame with the different settings and save it as a PNG. The following will be the frames, resembling the code/description above them.

This is the original, with no AVISynth code.
http://i7.photobucket.com/albums/y274/Typhoon859/OriginalNoAVISynth.png

crop(2,2,-2,-2,true)
http://i7.photobucket.com/albums/y274/Typhoon859/Crop.png

Import("C:\Program Files (x86)\AviSynth 2.5\plugins\LimitedSharpen.avs")

Import("C:\Program Files (x86)\AviSynth 2.5\plugins\LimitedSharpen.avs")
crop(2,2,-2,-2,true)
UnDot().hqdn3D(0,1,1,4.5).LimitedSharpen(ss_x=1.0,ss_y=1.0,Smode=3,strength=32)
http://i7.photobucket.com/albums/y274/Typhoon859/CropFilters.png

Import("C:\Program Files (x86)\AviSynth 2.5\plugins\LimitedSharpen.avs")
crop(2,2,-2,-2,true)
UnDot().hqdn3D(0,1,1,4.5).LimitedSharpen(ss_x=1.0,ss_y=1.0,Smode=3,strength=32)
Lanczos4Resize(960,540)
http://i7.photobucket.com/albums/y274/Typhoon859/CropFiltersWorkingResize.png

Import("C:\Program Files (x86)\AviSynth 2.5\plugins\LimitedSharpen.avs")
crop(2,2,-2,-2,true)
UnDot().hqdn3D(0,1,1,4.5).LimitedSharpen(ss_x=1.0,ss_y=1.0,Smode=3,strength=32)
Lanczos4Resize(1280,720)
http://i7.photobucket.com/albums/y274/Typhoon859/CropFiltersProblemCausingResize.png
It is important to note that any kind of resize used with certain resolutions such as this one causes exactly this color change, even the resize that I tried to do within the LimitedSharpen...

Import("C:\Program Files (x86)\AviSynth 2.5\plugins\LimitedSharpen.avs")
crop(2,2,-2,-2,true)
UnDot().hqdn3D(0,1,1,4.5).LimitedSharpen(ss_x=1.0,ss_y=1.0,Smode=3,strength=32)
ColorMatrix()
http://i7.photobucket.com/albums/y274/Typhoon859/CropFiltersColorMatrix.png

Import("C:\Program Files (x86)\AviSynth 2.5\plugins\LimitedSharpen.avs")
crop(2,2,-2,-2,true)
UnDot().hqdn3D(0,1,1,4.5).LimitedSharpen(ss_x=1.0,ss_y=1.0,Smode=3,strength=32)
Lanczos4Resize(1280,720)
ColorMatrix()
http://i7.photobucket.com/albums/y274/Typhoon859/CropFiltersProblemCausingResizeColo.png

poisondeathray

9th February 2009, 03:32

Typhoon859

9th February 2009, 04:03

I'm a bit "fuzzy" on this, but I recall a discussion in the software/playback subforum when width is >1024 with some renderers switch to a different color matrix is used. HD uses rec.709, SD material uses BT.601 or something along those lines. I'm hoping someone will give the "real" answer, but I'm pretty sure it's related...

You could try different ColorMatrix settings e.g ColorMatrix(mode="Rec.709->Rec.601") , but I think rec.709 is the "proper" one for HD material

Well, I dunno... I just tried 1184x666 which has a bigger width than what you said the point is at which the Color Matrix changes and the colors were the same actually. At the next 16:9 resolution, 1216x694, that's when the colors changed. Yeah, then I tried ColorMatrix(mode="Rec.709->Rec.601") but it didn't change anything.

poisondeathray

9th February 2009, 04:09

Try Rec.709...

It also might be a mod16 issue, but I really think it's the colormatrix and renderer behaviour with certain graphics card drivers. I'll try to dig up that old thread

Typhoon859

9th February 2009, 04:21

Try Rec.709...

It also might be a mod16 issue, but I really think it's the colormatrix and renderer behaviour with certain graphics card drivers. I'll try to dig up that old thread

Oh. Wow! It actually worked. Thanks! But... sadly, there's another problem now. It lags now. Sometimes it freezes for a bit and sometimes there are these weird flashes of random colors around edges of lines. Maybe I can restructure the code somehow to make it more efficient?

EDIT: Well, this did the trick-
Import("C:\Program Files (x86)\AviSynth 2.5\plugins\LimitedSharpen.avs")

ConvertToYV12()
crop(2,2,-2,-2,true)
UnDot().hqdn3D(0,1,1,4.5).LimitedSharpen(ss_x=1.0,ss_y=1.0,Smode=3,strength=32)
ColorMatrix(mode="Rec.601->Rec.709")
Lanczos4Resize(1280,720)

-Thank you very much. I knew that something could go into the parameters of ColorMatrix to get this to work. This is interesting what you said. I'd like to know more about this because honestly, I don't really understand this ColorMatrix stuff. How does the resolution effect anything?...

EDIT 2: With further watching, I found there to be lag, though less frequently than before, it still happened. What I don't understand is why. My CPU is only working at around 30% with one of the cores higher than the rest, topping at 75%-80%. So I don't see a reason for it to lag. I have the Yorkfield Q9450 Intel Core 2 Quad-Core, now running at 2.66GHz. Don't know what to do. I will test more to see whether it's the Color Matrix conversion or the resolution itself.

EDIT 3: Yeah, it only seems to lag when it's with the ColorMatrix line. Can't think of a solution unfortunately... Any ideas?

EDIT 4: Okay, I can 100% say that this did it...
Import("C:\Program Files (x86)\AviSynth 2.5\plugins\LimitedSharpen.avs")

ConvertToYV12()
ColorMatrix(mode="Rec.601->Rec.709")
crop(2,2,-2,-2,true)
UnDot().hqdn3D(0,1,1,4.5).LimitedSharpen(ss_x=1.0,ss_y=1.0,Smode=3,strength=32)
Lanczos4Resize(1280,720)
NOTE: ConvertToYV12() is crucial for efficiency in this script. Without it, the video lags when ColorMatrix is in place.

TinTime

9th February 2009, 05:08

I also use an Nvidia card and I've noticed this too. I've got an 8600GT, Win XP, VMR9 video renderer. It appears (as poisondeathray says) to assume BT.709 for HD resolutions and BT.601 for SD and I think the cut-off is at 1280*720. Not sure what it does with resolutions like 1024 * 768.

So if you upscale SD to 720p or greater then you'll need to convert BT.601 to BT.709 as you've discovered. Also if you ever downscale from HD to SD for any reason then you'd need to convert the other way.

BT.601/709 define the functions used to convert your YV12 source to RGB for display. Each one produces different RGB levels. Resolution matters here because your video renderer decides which function to use based on the resolution of the video you pass to it.

The other thing you could try is Haali's video renderer. This will let you switch between 601 and 709 on the fly.

Edit: Just noticed - you said the cut-off was between 1184x666 and 1216x694, not 1280x720 as I thought. But you get the idea.

Typhoon859

9th February 2009, 05:13

I also use an Nvidia card and I've noticed this too. I've got an 8600GT, Win XP, VMR9 video renderer. It appears (as poisondeathray says) to assume BT.709 for HD resolutions and BT.601 for SD and I think the cut-off is at 1280*720. Not sure what it does with resolutions like 1024 * 768.

So if you upscale SD to 720p or greater then you'll need to convert BT.601 to BT.709 as you've discovered. Also if you ever downscale from HD to SD for any reason then you'd need to convert the other way.

BT.601/709 define the functions used to convert your YV12 source to RGB for display. Each one produces different RGB levels. Resolution matters here because your video renderer decides which function to use based on the resolution of the video you pass to it.

The other thing you could try is Haali's video renderer. This will let you switch between 601 and 709 on the fly.

Yeah well. I don;t like the Haali Video Renderer and now it's okay. I'll never need to downscale so this is fine. Also, as I mentioned the cutoff is actually at any resolution greater than 1184x666. When I tried the next largest 16:9 resolution (1216x694), the colors changed. That problem is resolved now. The other ones are left.

Typhoon859

9th February 2009, 05:17

Modified a little - "2) LimitedSharpen is good but it does one thing that bothers me - It makes the lines very jittery. Is there something I can do about that or is that just how it is? If there's nothing I can do, from your experience, maybe you know of a better sharpener? (I believe that the PS3's Mosquito Noise Reduction Filter does exactly that - it evens out jittery lines. If not that, I don't know what it is but something the PS3 does resolves the jitter issue which some videos have.)"

TinTime

9th February 2009, 05:17

Also, as I mentioned the cutoff is actually at any resolution greater than 1184x666.

Beat me to my post edit :)

Typhoon859

9th February 2009, 05:19

Beat me to my post edit :)

Well, what exactly is BT.601/709? Why does it even need to exist? Why is that YV12 to RGB conversion is different with higher resolutions?

Typhoon859

9th February 2009, 06:06

Bad news... well, for me mainly and for anybody else who this might interest. I tried a video with a higher bitrate. I used this code to upscale a same sized h264 encode but with a larger bitrate and it started to lag. I guess this might be for a different thread but, maybe somebody can think of a more efficient way to do what I'm doing in my code.

poisondeathray

9th February 2009, 06:19

LimitedSharpenFaster (guess why it's called faster :) )

Are you using ffdshow-mt branch? quite a bit faster for h264 decoding

Typhoon859

9th February 2009, 06:25

LimitedSharpenFaster (guess why it's called faster :) )

Are you using ffdshow-mt branch? quite a bit faster for h264 decoding

Not sure. How can I check? From what I remember, I just searched Google and downloaded the latest version. I didn't know there were different versions of it. And if I don't have the mt branch and replace the one I have now with that one. Is there any downside to it? If not, why does the other one exist? (Yeah, I know I'm full of questions. I'm just very interested in this stuff and would like to know these things.)