odd mencoder distortion regardless of settings used with 1080i source

[Thasp]

edited: new issue, similar though

http://www.ingraldi.us/thasp/thasp/testforbadivtc12.mkv

Using filmdint with scale always makes it look like that. The problem goes away if I use filmdint after scale, but then uncombing is bad. The problem goes away if I don't scale, but then the video is too big for the bitrate. The problem goes away if I use pullup, or ivtc, but those filters suck, leave too many dupe frames in(only 2 is enough to throw the audio 60 ms out of sync) regardless of what settings I use. pp=lb before filmdint fixes the issue, but I don't need to deinterlace.. and there's no reason that without deinterlacing before filmdint I should have green pixelation at the top of the screen.

I'd like filmdint to work with scale.

I've tried many different filters, many different settings, over the past few months.

[morph166955]

im going to assume that this is from an HD transport stream you recorded, probably from over the air? If so, your source is bad. Ive seen this in my own things. sometimes when pulling HD over the air your signal strength drops for that few seconds of time and your record does not have enough to finish the frame. in this case you loose an I frame and things go all whacky for a while. does it just magically get better at some point?

[Thasp]

The source is fine.

I know for three reasons.

a) I can encode it to xvid, mpeg2, or x264 in windows without issue.

b) 0 errors in videoredo

c) It does this to every 1080i show at the same time.

Also, if I cut the show up into 10 pieces and encode them all seperate, 2pass(no point to this since it messes up bitrate distribution), the nonsense at the end of the video isn't present. :(

[morph166955]

ok then...thats just plain weird! whats the exact commands/method your using to create your files?

[Thasp]

mencoder test.mpg -sws 9 -vf pullup,softskip,crop=1904:1064:8:8,scale=1280:720:0:0:4, -ovc x264 -x264encopts bitrate=3000:threads=auto:pass=1:turbo=2:subq=6:partitions=p8x8,b8x8,i4x4:frameref=3:bframes=3:b_pyramid:weight_b:brdo:bime:cabac:no_fast_pskip:mixed_refs:trellis=1 -nosound -ofps 24000/1001 -of rawvideo -o test.264
mencoder test.mpg -sws 9 -vf pullup,softskip,crop=1904:1064:8:8,scale=1280:720:0:0:4, -ovc x264 -x264encopts bitrate=3000:threads=auto:pass=2:subq=6:partitions=p8x8,b8x8,i4x4:frameref=3:bframes=3:b_pyramid:weight_b:brdo:bime:cabac:no_fast_pskip:mixed_refs:trellis=1 -nosound -ofps 24000/1001 -of rawvideo -o test.264
MP4Box -fps 23.976025 -add test.264 test.mp4
mkvmerge -o test.mkv test.mp4 test.ac3 --default-duration 0:24000/1001fps


That was the latest garbage. I tried subq=6:partitions=all:8x8dct:me=umh:frameref=5:bframes=3:b_pyramid:weight_b for x264 settings as well - and it was still borked.

Something else I notice is that this only happens in 2pass encoding.

[morph166955]

im going to assume that you can do this based on your last statement but can you play it successfully in mplayer using the sws & vf settings your using above?

what version of x264 are you running? what patches have you applied to it?

[Thasp]

I can't play with mplayer. That machine doesn't have X installed, or a monitor, or a keyboard. It sits in my closet and quietly does my bidding.

I run x264 0.54.648

I didn't put any patches on it. I compiled it with --enable-pthread. I deleted every possible instance of mencoder and old x264 before compiling it again recently to see if new stuff would fix the issue. Lately I haven't had the time to recompile different versions or with different settings, but it is something I hope to get around to soon. :)

[Thasp]

I did some tests yesterday. It wound up being caused by hardware failure. It happened at the same time because the test clips I used were the same length, same general settings, and it failed at the same time each encode.

Now all I have to find is a seperate pullup/decomb filter and I'm done.. filmdint is great for decimate=5, and pullup is great for uncomb() or telecide, but pullup sucks as a mencoder version of decimate=5 in avisynth. And I can't tell it to not do IVTC.. and I can't find a seperate decomb filter :(

[morph166955]

could you be more specific as to the hardware failure? im kinda curious how you came to that conclusion and why your relating the hardware failure to the pullup/decomb filter (you very well could be dead on im just curious as to your thought process thats all)

[Thasp]

I tested at 800 MHz/DDR333 with a few test 44 minute clips, including those, which is way below stock speed, and it works.

It's a long story about PSU failure and cpu/mobo issues, but I'm sure now it's hardware failure and not a fault of mencoder.

as far as decomb/pullup, pullup still doesn't seem as accurate as straight up filmdint. at any CPU speed it has a few dupe frames when I check the mencoder window. The audio is in sync, but not as in sync as I think it could be. I'll test on the windows machine later to make sure I'm right, and ABX blind compare the two different encodes to check sync, and see if it's in my head or if it's actually there, but for now I think it is there.

I'm trying vf pp=fd,filmdint=io=30001/1001:24000/1001=dint_thres=256 now. pp=fd seems to do uncomb()'s job - I tested it on a 10 second clip that makes it very obvious since the camera moves slowly.. without uncombing it looks like a stuttery camera. I did my test without specifying io, so I am curious how it will do the encode now, on a full clip.

[Thasp]

Now there's a new fun kind of distortion, but with any hardware.

If I use pullup and scale, there are no problems. Scale, no problems. Filmdint, no problems. filmdint plus scale.. now there's a problem. I get this funky green video. Here's a ten second long, low bitrate example.

http://www.ingraldi.us/thasp/thasp/testforbadivtc12.mkv

With long samples, the pullup filter goes more out of sync as time goes on. ivtc, ivtc=1, and ivtc=2 have maybe 2-3 dupe frames. The video looks 60-100ms out of sync depending on where I am in the file.

So I have a choice between not scaling my video or having out of sync video with mencoder.. neither of which are acceptable. I have a log of all the encodes I have tried today on clips going through the scale, pullup, and filmdint settings and the results I got and I have 0 that work like uncomb(),decimate=5 do with avisynth in windows. :(

I don't understand why filmdint does that green pixelation at the top when used with scale. I don't expect anyone to answer at this point, but I'm still curious. heh

Here's the best I have so far. I'm using bad settings so I can do encodes as quickly as I can, with a higher bitrate so it's easy on the eyes.


mencoder testforbadivtc.mpg -sws 9 -vf filmdint=dint_thres=256=comb_thres=8,crop=1912:1072:4:4,scale=1280:720:0:0:4 -ovc x264 -x264encopts bitrate=5000:threads=3:subq=1:me=dia:bframes=2:b_pyramid:weight_b:keyint=240:keyint_min=24:frameref=1:nodeblock -nosound -noskip -fps 30000/1001 -ofps 24000/1001 -of rawvideo -o pleasework.264
MP4Box -fps 23.976025 -add pleasework.264 pleasework.mp4
mkvmerge -o testforbadivtc18222.mkv pleasework.mp4 testforbadivtc.ac3 --default-duration 0:24000/1001fps


harddup and noskip, taking out -fps and -ofps do nothing for the green pixelation. Taking out scale fixes it, or taking out filmdint. :(

[Thasp]

..Something else interesting.

If I add pp=lb before filmdint, the green goes away.

pp=lb is a deinterlacer. I merely want it to match fields, not deinterlace. :(

If I put crop/scale before filmdint, the green also goes away, but I know this is just bad practice... or is that how one is supposed to do it in mencoder?

Now I am trying different pullups one last time to see how they work. I used this command line

mencoder pleasework.mpg -sws 9 -vf pullup=sb1,softskip,crop=1912:1072:4:4,scale=1280:720:0:0:4,harddup -ovc x264 -x264encopts pass=1:turbo=2:bitrate=3200:threads=auto:subq=6:partitions=all:8x8dct:me=hex:frameref=3:bframes=3:b_pyramid:weight_b:brdo:bime:cabac:no_fast_pskip:mixed_refs:trellis=11:keyint=240:keyint_min=24 -nosound -noskip -o /dev/null
mencoder pleasework.mpg -sws 9 -vf pullup=sb1,softskip,crop=1912:1072:4:4,scale=1280:720:0:0:4,harddup -ovc x264 -x264encopts pass=2:bitrate=3200:threads=auto:subq=6:partitions=all:8x8dct:me=hex:frameref=3:bframes=3:b_pyramid:weight_b:brdo:bime:cabac:no_fast_pskip:mixed_refs:trellis=11:keyint=240:keyint_min=24 -nosound -noskip -of rawvideo -o pleasework.264
MP4Box -fps 23.976025 -add pleasework.264 pleasework.mp4
mkvmerge -o pleasework.mkv pleasework.mp4 pleasework.ac3 --default-duration 0:24000/1001fps



and got this

Opening video filter: [harddup]
Opening video filter: [scale w=1280 h=720 interlaced=0 chr-drop=0 param=4]
Opening video filter: [crop w=1912 h=1072 x=4 y=4]
Crop: 1912 x 1072, 4 ; 4
Opening video filter: [softskip]
Opening video filter: [pullup=sb1]


Is it opening the filters in the reverse order how I put them in? This would explain that green distortion nonsense a little bit.

[nm]

Is it opening the filters in the reverse order how I put them in? This would explain that green distortion nonsense a little bit.
It may open the filters in reverse order, but it still applies them in the order you gave them to the -vf parameter. If you want to try scaling before filmdint, make sure that you use the scale filter in interlaced mode (-vf scale=1280:720:1:0:4,...).

I tried filmdint with MPlayer on a progressive 1080p24 source and it caused the same green/purple blocking you had, even without scaling or any other filters applied. Lower resolution sources (720p, ...) didn't have this problem. That also explains why scaling before filmdint works. This is clearly a bug, so it should be reported on the MEncoder or MPlayer users mailing list (accompanied with a source that exhibits the problem).

The sync problem with pullup and other ivtc filters may be caused by -nosound. Finding a way to encode with -oac copy is preferred. Test with XviD first so that AVC-in-AVI problems won't bite you.

Btw, testing the filter chain in cases like this filmdint problem is much easier with MPlayer, so I'd advice you to install it on Windows if you don't have Linux systems with X installed.

[Thasp]

I didn't want to report it as a bug until I could confirm someone else had the same problem with that filter. It's good to know filmdint doesn't have a personal vendetta against me. :)

I didn't put scale in interlaced mode. I've been told scaling before uncombing, in any mode, is a very bad thing to do - does this hold true even if I scale with 1:0:4?

I'll try with xvid once I install xvid, and compile mplayer to work with it.

Thanks again. You're always helpful with these issues!

[nm]

I didn't put scale in interlaced mode. I've been told scaling before uncombing, in any mode, is a very bad thing to do - does this hold true even if I scale with 1:0:4?
It is definitely a bad thing when using the default mode, but I think it could work in interlaced mode. Of course, doing IVTC first would be preferred if it wasn't for the filmdint bug.

I'll try with xvid once I install xvid, and compile mplayer to work with it.
On second thought, lavc would work as well and is integrated to MEncoder, so try this for fast encoding (or use vbitrate instead of vqscale if you prefer specifying a bitrate):
-ovc lavc -lavcopts vcodec=mpeg4:vqscale=4

[Thasp]

http://bugzilla.mplayerhq.hu/show_bug.cgi?id=798

I'll append some more substance to it in the morning, hopefully this gets somewhere. :)

[Thasp]

http://www.ingraldi.us/thasp/thasp/sample.mkv

I think this is good. I used the following command line.

mencoder sample.mpg -sws 9 -vf crop=1912:1072:4:4,scale=1280:720:1:0:4,filmdint=dint_thres=256 -ovc x264 -x264encopts pass=1:turbo=2:bitrate=3200:threads=auto:subq=6:partitions=all:8x8dct:me=hex:frameref=3:bframes=3:b_pyramid:weight_b:brdo:bime:cabac:no_fast_pskip:mixed_refs:trellis=1:keyint=240:keyint_min=24 -ofps 24000/1001 -nosound -of rawvideo -o sample.264
mencoder sample.mpg -sws 9 -vf crop=1912:1072:4:4,scale=1280:720:1:0:4,filmdint=dint_thres=256 -ovc x264 -x264encopts pass=2:bitrate=3200:threads=auto:subq=6:partitions=all:8x8dct:me=hex:frameref=3:bframes=3:b_pyramid:weight_b:brdo:bime:cabac:no_fast_pskip:mixed_refs:trellis=1:keyint=240:keyint_min=24 -ofps 24000/1001 -nosound -of rawvideo -o sample.264
MP4Box -fps 23.976025 -add sample.264 sample.mp4
mkvmerge -o sample.mkv sample.mp4 sample.ac3 --default-duration 0:24000/1001fps


I think this looks good, as far as artifacting, out of sync, and uncombing goes. Someone commented that it looked like it had been deinterlaced - but I don't see it. You seem like you already know know to get this all to work, but if not, maybe some snippets from the above line could help. :)

It was originally 4-5 minutes, I cut it up into 10 second segments and posted one of the last ones for space/copyright reasons, but the sync was maintained for the entire clip.

[nm]

Thanks, I'm sure other MEncoder users will find that useful! Luckily I don't usually need IVTC filters since I live here on the PAL side of the world :)

[Thasp]

And they say Americans have things easier. :p

Here's the same thing encoded on windows, using settings I am 100% sure do correct ivtc, uncomb, and resize.

http://www.ingraldi.us/thasp/thasp/sample2.mkv

To me, they look the same, which signifies success. I'd like to thank you for all the help you've provided since I first decided to jump into linux video in early February, I'd surely be nowhere by now if not for your assistance. :) and, with no offense intended, I hope I never have to check back at this forum or man mencoder ever again. :devil: