when sharpening, encoding is very blocky [Archive]

shpitz

5th September 2005, 14:16

hey everyone, i got a problem, maybe you guys can help.

i'm trying to encode a soccer match, the source is 544x480 dvb, and it is very soft.

i've played with many sharpening filters, and compared the output in vdub (had several vdubs open on the same frame) and finally got the right sharpening filter and settings i like.

it looks great on the screen compared to the original non-sharpened source.

the problem i have is this: when i go to encode it to xvid, there are tons of artifacts all over, the picture looks aweful, and it doesn't matter which xvid settings i tried to use to eliminate all those artifacts (large blocks all over, and a 'bubbly' feel to the picture).

as soon as in the same script i comment the sharpening, the clip encodes great with the same exact xvid settings, but again, it looks really soft.

SetMemoryMax(256)

LoadPlugin("D:\TBS\Filters\DGDecode.dll")
LoadPlugin("D:\TBS\Filters\aSharp.dll")
LoadPlugin("D:\TBS\Filters\TDeint.dll")

# SOURCE
mpeg2source("F:\Brazil.vs.Chile.d2v",idct=7,iPP=true,cpu=6,moderate_h=45,moderate_v=60)

tdeint(mode=1, type=2, sharp=true)

Crop(8,4,-14,-2)

asharp(2,12)

Lanczos4Resize(512,384)

i also tried to put the asharp in other places (before crop, before deint), and also tried other resizing filters.

if needed i can provide a sample.

thanks for your help.

Guest

5th September 2005, 14:42

Sharpening makes the video less compressible, so you'll have to increase the bitrate of your encode.

Didée

5th September 2005, 15:22

Are you sure that cpu=6 is really needed? Many users are not using the deringing routines, because these make the source very very soft. Check if cpu=4 is enough.

If some deringing is indeed necessary, there were other solutions posted on this forum that perhaps are more suited: HQDering (http://forum.doom9.org/showthread.php?t=67532&highlight=HQDering), aWarpSharpDering (http://www.leoagricola.net:8888/~kp/AviSynth/AWarpSharpDering/), BlindDehalo3 (http://forum.doom9.org/showthread.php?p=622289#post622289) ...

And don't forget the general rule of thumb regarding sharpness:

"The more you like the sharpness of a source, the more the encoder will hate it." ;)

shpitz

5th September 2005, 15:33

haha, thanks guys, that makes a lot of sense.

i also forgot to mention i tried many matrices as well.

i tried to encode @ 1400kbps.

i use tdeint to make the 30fps cap into 60fps for smooth movement of the players and ball.

do you think i can get higher bitrate possible with the same amount of filesize if i convert it back to 30fps?

if so, how would i go about doing that?

thanks again

Guest

5th September 2005, 15:36

do you think i can get higher bitrate possible with the same amount of filesize if i convert it back to 30fps? You can get better quality per frame for the same filesize, but it would probably be better to start again and just deinterlace the original at single rate.

if so, how would i go about doing that? To convert back, just use SelectEven() to discard every other frame.

mg262

5th September 2005, 15:52

You will get more bits per frame by doing this, which will reduce the blocking; the bit rate (1400kbps) can't change without changing the file size. (I'm not trying to be pedantic... I just mean that you shouldn't try to change the bit rate if you want to keep the file size the same.)

shpitz

5th September 2005, 16:13

i've changed to cpu=4 and the image looks exactly the same as cpu=6

i even tried cpu=0.

i'm going to set it to cpu=0 and then encode with asharp and see what comes out.

thanks

shpitz

5th September 2005, 17:39

you are killing me guys... LOL

it looks like crap, it seems that if i try to keep it the same bitrate and change back to 30fps, it will lose all details (the grass is just a big chunk of green) by trying to eliminate the blockyness.

i changed my mpeg2source command to this:

mpeg2source("F:\Brazil.vs.Chile.d2v",idct=3,iPP=true,cpu=6,moderate_h=10,moderate_v=10)

and reduced asharp to this:

asharp(1,16)

there are no blocks at all, just mosquito noise around the players and the grass is 1 big green chunk, no details what so ever.

so what you're saying is that it's impossible to keep the details while keeping the filesize to be 2 cds?

what if i do not deinterlace? will that help in any way?

thanks for all your suggestions

mic

5th September 2005, 21:36

When a picture's soft, running a sharpen filter tries to re-create data that's missing from the original (or creates that visual impression anyway :)). Very generally... Many codecs look for the sort of uniform color patches sharpening creates, and tries to save file space, treating it the same way it would a plain blue wall. Things that I've found *sometimes* help avoiding blockiness with hi compression codecs in general:

WHen possible feed the encoder the original frame size, & let it do the resizing. This gives the encoder the max amount of data. When you do reduce frame size, larger isn't always better -- going small averages data, & lets graphics hardware enlarge on playback.

When possible, increase minimum bitrate -- reduce top bitrate if you have to to maintain file size.

Add a slight amount of noise. :D The idea is the encoder will see the small moving dots as motion and retain more data per frame. One method I've used when going to intermediate avi is to use something like Picvid at less then top quality settings if that helps to describe the desired effect before final encode.

shpitz

5th September 2005, 23:45

i'm at a loss when you say min & max bitrate... where do i set that?

also, the original cap is 544x480 and i only crop the black edges and the noise at the top. after crop the AR is not 4:3 anymore that's why i resize to 512x384.

i guess there's nothing i can do and the video will have to remain very soft in order to retain filesize.

thanks

Chainmax

6th September 2005, 00:23

mic makes a very good point, I never thought about the issue that way. Try using something like AddGrain(x,0,0) (http://home.comcast.net/~trbarry/AddGrain.zip), with x being 2 or 3, right after the sharpening (which in my opinion should go right after the resizing). Also, I never used asharp, but I can tell you that LimitedSharpen() (http://forum.doom9.org/showthread.php?t=84196&highlight=limitedsharpen) is an awesome sharpener.

rfmmars

6th September 2005, 02:34

shpitz

6th September 2005, 02:39

thanks guys for the tips, i will give them a shot.

i tried limitedsharpen and many others, but when compared to asharp using 2 vdubs open with the same script but with the diff sharpener, nothing could beat asharp. the picture looked really sharp with it, but the encoding messed it all up.

i also played with the placement of the sharpen filter in the script, the results in vdub preview where the same.

i will try the addgrain idea and see what comes up.

i will also give smartbrightness a shot as well.

thanks guys

mic

6th September 2005, 02:42

@Chainmax
Usually I get blasted for that one, but it has helped a lot. :D

i'm at a loss when you say min & max bitrate... where do i set that?

also, the original cap is 544x480 and i only crop the black edges and the noise at the top. after crop the AR is not 4:3 anymore that's why i resize to 512x384.

i guess there's nothing i can do and the video will have to remain very soft in order to retain filesize.

thanks
Sorry -- I was just making general points about this type of compression, not just xvid, that have worked for me from time to time over the years.

When doing vbr video, some encoders ask for min, ave, max values. If the min is too low, the majority of the video can have extreme compression causing blockiness, just so (often short) high motion segments can use the least compression possible. But it's just a general statement, not something you're doing wrong. :)

I explained a bit better (I hope :)) regarding why to let the encoder resize, but again only when you have that option. Cropping borders itself is good, removing an extra chance for errors before the encoder tries to average in black borders.

As far as downsizing, if you're only going for PC playback, then the amount of image data you do have will or should be averaged by the resize itself and possibly get you a cleaner encoded frame, plus you might get away with less compression for a set file size.

All of these are easily tested by a short clip cut from your source video, so you can see how much or if anything helps. :)

Mug Funky

6th September 2005, 04:30

btw, is your sharpened encode coming out at a lower bitrate than you expected? i noticed a while ago that particularly noisy sources seem to throw xvid's normally perfect ratecontrol off a tad. it'll hit the noise too hard, giving a swimming effect (this makes sense to the encoder - a patch of noise looks like a similar patch of noise in the last frame, so it codes it as motion data instead of texture data. it's actually quite clever, but unfortunately xvid's comparison metrics don't work the same way as human eyes).

it might be better to encode it with very conservative sharpening (ie not much at all, just enough to offset what encoding will remove) and apply harder sharpening on playback. if you're watching on a relatively fast PC you should be able to get away with it quite well using ffdshow's sharpening, or it's avisynth filter.

[edit]
When possible, increase minimum bitrate -- reduce top bitrate if you have to to maintain file size.
not sure about that... max bitrate should always be the highest you can get away with without causing playback problems - it's one of the strengths of VBR that it can be a very high bitrate for a very short time and still maintain a low average. i agree about raising min rate though - some sources on some encoders just require a certain minimum to avoid spazzing out (standards-converted PAL video on an MPX3000 needs at least 4500 kbps in my experience... but xvid doesn't allow a min rate so this is irrelevant...)

kassandro

6th September 2005, 08:12

mpeg2source("F:\Brazil.vs.Chile.d2v",idct=7,iPP=true,cpu=6,moderate_h=45,moderate_v=60)

Before you sharpen, you should stop blurring. "cpu=6" is simply a desaster especially in the interlaced case. In mpeg2dec3 the deringing code is simply broken resulting in excessive blurring. While it can be easily fixed, the underlying algorithm is so poor, that it is not worth to be fixed. The deringing code is due to trbarry and I did send him an email describing the problem months ago. The deblocking code is much better, but the default deblocking routine contains a bug as well, which can be easily fixed. While it is a very good idea to exploit quantiser information for deblocking and deringing, I would not recommend to use postprocessing currently. I don't know about DGDecode, but it probably has the same postprocessing routines.

shpitz

6th September 2005, 14:05

i'm not using mpeg2dec3, using dgdecode.

i changed it to cpu=4 and it gotten a little better.

as for adding noise with addgrain, it didn't seem to make much difference, i tried addgrain(2,0,0), 3,0,0 , and even 20,0.3,0.4 just to play with the settings.

the thing is that even when i tried a very small sharpening asharp(1,5) it was still looking blocky, so it looks like the video cannot be sharpened and keep the same filesize.

the suggestion to use the smartbrightness seems to work well, since the filter improves the shadows and highlights and gives me the sense of more detail while there is no sharpening taking place.

kassandro, what would you suggest for the mpeg2source parameters for me to use on my source, which is a 544x480 DVB with bitrates of around 2000kbps on average, and the recording looks always soft, some recordings are more, some are less, but it's always too soft to my eyes...

would you think i should use unfilter or undot? i've never really saw them provide any benefit on my captures.

i'm also trying many other matrices and different xvid options, but i'm not really sure what i'm doing. is there a mathematical way by calculating or running some kind of a program or an encode session that will help determine the optimal settings to retain detail as close to the original cap?

i'm still confused as to when i should use qpel, adaptice quant, b-vop, and all those other settings...

my goal is to always get the best quality (closest to the original) for the least amount of space, i don't really care for encoding speed.

my dekstop res is 1600x1200 and pretty much all my encodes look like crap to me, but i learned to live with it... many will look very sharp in their original size but when viewed in full screen i will see all the blockyness and the 'swimming effect'.

and finally, i got a good result on the encode without sharpening at all, but there was still mosquito noise around the players, those small dots surrounding them, and also around the station logo. is there anything i can apply to help reduce those artifacts?

thanks again for all the help.

mg262

6th September 2005, 14:11

You may simply be expecting too much... I don't personally think you're going to get something you're really happy with without increasing the bit rate. That said, you might find it worth looking at the previous codec comparison and trying codecs other than xvid. It might also be worth looking at postprocessing options in whatever you are using for playback... for example FFdshow has deblocking options.

shpitz

6th September 2005, 14:26

yeah, you're exactly right mg262, i'm really expecting too much.

after seeing hdtv captures sdtv really looks like vhs... hehe

but if for example the capture filesize is around 450mb for a 30min show (w/o commercials), i would like to reduce the size by 1/2 and still keep the quality.

i do the same with photos, edit them in photoshop and adjust levels and reduce noise etc, i would like to do the same with my recordings.

mic

6th September 2005, 22:58

i do the same with photos, edit them in photoshop and adjust levels and reduce noise etc, i would like to do the same with my recordings.
But I'd wager that those photos have a lot more resolution to them. :)

Is it possible to increase your resolution from the start, on the next project? Are there any other settings for recording, or maybe it would be worth it for you to look at PC capture hardware?

If you look at other codecs besides xvid as mg262 suggested, some of them seem to try and blur the background a bit to avoid the prob. you're having now, but if I remember correctly that's in the reviews too.

mic

6th September 2005, 23:05

not sure about that... max bitrate should always be the highest you can get away with without causing playback problems - it's one of the strengths of VBR that it can be a very high bitrate for a very short time and still maintain a low average. i agree about raising min rate though - some sources on some encoders just require a certain minimum to avoid spazzing out (standards-converted PAL video on an MPX3000 needs at least 4500 kbps in my experience... but xvid doesn't allow a min rate so this is irrelevant...)
Well.... I think it depends on type of video. I've had some with a few scenes where motion was very fast, but the rest was very average. Closing the brackets a bit the high motion suffers, but can be over so fast most people never notice. But it's all tradeoffs, where it might help sometimes, and with xvid as you've said, irrelevant anyway. :D

shpitz

7th September 2005, 00:39

thanks mic for trying...

the capture is purely digital, DVB, no settings in choice of res or anything. i get w/e the satellite company broadcasts, it's not analog.

the problem with dvb is that the mpeg2 they broadcast looks good on tv but looks like crap on a pc, it is very soft and lacks of detail.

all i want is sharpen it a little, nothing more.

the whole process doesn't add any noise so the video is suppose to be very clean...

and since i'm a quality freak i'm trying to improve the capture the best i can...

i'm trying to keep the file at a certain size for archiving purposes, but if i have to bump up the bitrate to achieve the sharpness i seek then so be it...

but i would still love to learn new tricks and new filters that will give good results of the type of source i have.

i've got 2 clips you can play with if you have time:

avisynth.org/shpitz/xgames_sample.mpg

avisynth.org/shpitz/soccer_clip.vob

i haven't had much success with IIP as well which i really would like to get good results with...

thanks

Shinigami-Sama

7th September 2005, 03:22

just to be on the safe side...
are you sure it isn't just your playback de-interlacer making it look smooth on your pc?

mic

8th September 2005, 00:22

the capture is purely digital, DVB, no settings in choice of res or anything. i get w/e the satellite company broadcasts, it's not analog.
Sorry -- thought *maybe* you were using a DVR with qual settings, like record top qual for a couple of hours worth, reduced qual for twice as long, that sort of thing.
the problem with dvb is that the mpeg2 they broadcast looks good on tv but looks like crap on a pc, it is very soft and lacks of detail.
Grabbed your 1st sample... Seems like standard analog converted to mpg2 and sent over sat/cable. Lots of noise -- no need to add :) -- and a lot of detail missing at the individual frame level, with objects showing some pretty big gaps between fields.

Tried a few of methods tageting around 700 k bitrate, about 5.5 meg file size. Real seemed to do best -- the only one where I could get away from blockiness or pixelation in the scene transitions. Ironically, the sharpest picture came from feeding the encoder an avs file with just 2 lines: the path to dgdecode & the d2v file. When you don't have a lot of data, everything counts and every step loses some. Also got [I think much] better results going to 320 x 240, & then doubling size in player. Resize and noise were done in encoder. Even deinterlace lost noticable detail. And that got the sharpest picture without pixelation.

I did try a couple of filters in V/Dub where I could see previews. I ran deinterlace, crop/resize to full frame, smart smoother, & unsharp mask, all at conservative settings. This got a pretty decent picture, but it was on the soft side, which was one of your don'ts. I tried without the resize, but it didn't make a lot of difference. I wanted to try Vegas' supersampling out of curiosity, so I fed it the filtered V/Dub output. This created not only the extra frames (not dupes) to go to 60 fps, but calculated a bunch in between existing frames and the new one. Nice picture, almost like film, but slower then watching paint dry, and in the end if I had to choose it or direct to Real from avs, I think I'd choose the direct route.

Otherwise, nothing will ever look as good on a PC as it does on [crt] TV. :) Yeah, I know that's wrong, but different animals almost entirely. :)

shpitz

8th September 2005, 02:34

really appreciate your effort mic,

but what do you refer to 'real'?

do you have any experience with IIP?

thanks

mg262

8th September 2005, 02:42

Real is a company that release number of video-related things, like RealPlayer (now I think renamed to RealOne). The last time I checked, their products tended to be slow, full of advertising, and hijacked every association in sight without asking, so they didn't exactly endear themselves to me...

They also produced a codec and file format (extension .rm for RealMedia, I think) -- search for RealMedia codec.

Shinigami-Sama

8th September 2005, 03:00

it;d down in the mpeg-4 section actualy..
the only thing I liked real for back in the day was he dl manager for the days when 33.6k was new
but it's not doinf to bad now
it;s good for ulrta-low bitrate stuff now

mic

8th September 2005, 18:33

but what do you refer to 'real'?
Real is actually the folks who started the high compression video game. They have a free encoder (called producer) that is limited a bit, but you can still do most things by editing what they call audience profiles (xml text files). They also have the Helix projects and web sites, which are open source, and include a cli encoder -- there are a few gui front ends that you can use. There is a decent amount of work being done keeping up with (and in some cases exceeding IMO) other formats.

The RealOne player can seem obnoxious until you simply tell it to just open the player part -- like wmplayer with all the bells and whistles open versus the minimal window/interface. [Preferences -> General -> On start-up = Player Only]

See the codec comparrisons here at doom9.org for info re: speed etc... The advantage Real has (again IMO) with video like your sample is that it tries to focus on the moving object, and blends the background a bit, giving you more detail where you need it. I came close with other methods, but at scene changes they all showed very noticable artifacts.

"... hijacked every association in sight without asking"

No more, no less then q/time, wmplayer, nero.... :)

"...search for RealMedia codec."
http://forum.doom9.org/forumdisplay.php?f=76

"it;s good for ulrta-low bitrate stuff now"
They're doing quite a bit with downloadable movies, handle DVDs etc...