H.264 playback nightmare

[flyvholm]

I have a stubborn H.264 playback problem. A friend and I have spent weeks troubleshooting without finding out what's wrong, so it's time to ask the experts for help.

I've produced a film that I'll be offering as digital download. On all devices I've tested things work as expected: Film looks fine, and higher bitrate gives better visual quality.

My friend is in the US, I'm in Denmark. He has been testing the same video files and is seeing the opposite: Higher bitrate gives heavy artifacts. In particular, the noise becomes so ugly that the film isn't presentable at all. This is consistent across a number of HDTVs (he went to a store to try a bunch), either playing back from a USB stick, portable media players or a laptop.

I've tried using Adobe Media Encoder + MainConcept w. various settings (incl. stock settings) and also VirtualDub + x264 + mp4box w. various settings. The above observations have applied regardless what I've tried. The only way we've been able to make the film look decent on HDTVs on his end is by using heavy compression (i.e. 2.5 mbps for 1080p). Apparently this reduces the problem to an acceptable level by simply smearing out the noise. But that certainly doesn't look better on my end, so it's not a solution I'm happy with.

It's quite perplexing that he cannot find any HDTV setup where the film doesn't look horrible at normal bitrates, while I've also tested various HDTV setups and not seen the problem anywhere, using the same video files. So I would very much like to know what results other people are getting. I've compiled two samples for testing, using x264:
High bitrate sample (8mbps) (http://thetimelapsephotographer.com/shared/forums/Title-high-bitrate.mp4)
Low bitrate sample (2.5mbps) (http://thetimelapsephotographer.com/shared/forums/Title-low-bitrate.mp4)

Test results or comments regarding this issue highly appreciated.

[Dust Signs]

Both samples look o.k. for me. What setup(s) is your friend using?

Dust Signs

[foxyshadis]

I'd have him CRC test his download to be sure it isn't corrupted. The file looks fine, and there's no setting that any player should balk on, especially his laptop.

[flyvholm]

Thanks a lot for your replies and for checking the files.

My friend has been testing playback from 2 Dell laptops (he mentioned they had small, low-quality panels, so probably cheap), two portable players (Patriot Box Office and Micca Speck G2) and directly from USB stick with capable HDTVs. He has tried two Seiki, a Vizio, a large LG, maybe 7 different Samsung HDTVs of varying size/quality, and finally a 4K TV (didn't remember brand). Not all combinations were tested, but suffice it to say, a lot of testing has been done. In all cases the higher bitrate versions show what he calls "crazy speckled noise", seemingly some interplay between camera noise and TV sharpening. When playing back the trailer for the film (https://www.youtube.com/watch?v=OEZtMZ7pwFM) from Youtube (i.e. recompressed by Youtube, noise smeared out) it looks great, while the original that was uploaded to Youtube shows the awful noise.

File corruption seems unlikely. He has downloaded/tested many samples, and the results are consistent: Everything high bitrate shows the artifacts while the low bitrate samples are much better. He did send a checksum that matched mine, though it was of a low bitrate file.

It's quite frustrating that I can't see the artifacts myself. He is going to see if it's possible to capture them on video. I'm also uploading a Blu-ray image for him to burn and test, which will be interesting.

[ChiDragon]

In all cases the higher bitrate versions show what he calls "crazy speckled noise", seemingly some interplay between camera noise and TV sharpening.
So then has he tried a TV with sharpness, motion interpolation, etc. etc. "Torch Mode" turned off?

[flyvholm]

Yes. Exaggerated saturation/sharpness/contrast will destroy aurora footage in various ways. But the speckled noise ruins the footage even at neutral settings and is particularly sensitive to the sharpness setting. He says he could get rid of it by reducing sharpness to minimum, but then it's quite soft, defeating the purpose of HD (playback from DVD looked better). However, heavy compression smears the noise sufficiently that reasonable sharpness can be maintained without the noise ruining the footage.

He has now tried playback on a 1080p projector (Mitsubishi HC4000) as well. Here the noise wasn't really objectionable unless he cranked the sharpness considerably up from the default neutral setting. He still preferred the heavily compressed version, but now it's more of a subjective preference.

It's hard to believe that none of the variety of HDTVs he has tested can handle the noise you see in the high bitrate sample. Especially because I have also played back aurora footage on a variety of HDTVs without ever encountering the issue. Something's just wrong about this...

[Stereodude]

Both playback fine on my laptop. The higher bitrate file looks a little better. I can try them when I get home on my HDTV.

What x264 settings are you using? Both streams have artifacts that I don't typically see in H.264 video. Considering how little movement there is in the clip I would expect a 8Mbit/sec encode to look better than yours does.

[flyvholm]

Test on HDTV would be great since that's where we really run into trouble. Would be good to know brand/model of HDTV. Here are the x264 + mp4box command lines used to create the videos in VirtualDub:
x264 --bitrate 8000 --pass 1 --stats "%(hostdir)\x264stats.log" --demuxer raw --input-csp i420 --input-res %(width)x%(height) --fps %(fpsnum)/%(fpsden) -o NUL -
x264 --bitrate 8000 --pass 2 --stats "%(hostdir)\x264stats.log" --demuxer raw --input-csp i420 --input-res %(width)x%(height) --fps %(fpsnum)/%(fpsden) -o "%(tempvideofile)" -
mp4box -add "%(outputname).264" -add "%(outputname).aac" -fps %(fps) "%(outputname)"
The input file is a 1080p Lagarith (lossless) compressed AVI file, no processing in VirtualDub. What artifacts are you seeing?

[Stereodude]

Why aren't you calling x264 from the command line and feeding the AVI directly?

You're missing a lot of the useful tuning encoder parameters.

Assuming you have a 1920x1080 4:2:0 Lagarith source for blu-ray it would be something like this:
x264 --bitrate XXXXX --preset veryslow --tune film --bluray-compat --vbv-maxrate 40000 --vbv-bufsize 30000 --level 4.1 --keyint 24 --open-gop --slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 --pass 1 -o out.264 input.avi
x264 --bitrate XXXXX --preset veryslow --tune film --bluray-compat --vbv-maxrate 40000 --vbv-bufsize 30000 --level 4.1 --keyint 24 --open-gop --slices 4 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 --pass 2 -o out.264 input.avi

For a very high quality digital download I'd do something like this:
x264 --crf 18 --preset veryslow --tune film --vbv-maxrate 50000 --vbv-bufsize 62500 --level 4.1 --keyint 120 --open-gop --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 -o out.264 input.avi

As for the artifacts, there are some sort of blocking artifacts around the letters when they fade in. The clip is also pretty noisy. Have you considering using an AVIsynth filter like MCTD (http://forum.doom9.org/showthread.php?t=139766)? It works wonders for removing grain and noise from footage and improving compressibility.

[Stereodude]

I tried both clips on my Samsung HDTV via a USB stick. I didn't notice any unusual problems with the high bitrate clip. The noise is a little less smoothed over in the high bitrate clip though.

[ChiDragon]

He still preferred the heavily compressed version, but now it's more of a subjective preference.

It's hard to believe that none of the variety of HDTVs he has tested can handle the noise you see in the high bitrate sample. Especially because I have also played back aurora footage on a variety of HDTVs without ever encountering the issue.
Perhaps he just hates noise?

[flyvholm]

Why aren't you calling x264 from the command line and feeding the AVI directly?
VirtualDub makes a quite effective GUI for this. IMO tidier and decreases risk of errors, but it's a matter of preference.

You're missing a lot of the useful tuning encoder parameters.
Yes. I do use the exact settings you list for my Blu-ray encodes. I can (and will) use better settings for encoding the digital download as you suggest. However, the big question is what bitrate I should go with for the default digital download, considering that my friend cannot even find an HDTV setup that the high bitrate version looks good on (and most end users will be less tech savvy than him).

As for the artifacts, there are some sort of blocking artifacts around the letters when they fade in.
I do see that too, much worse for the low bitrate version.

The clip is also pretty noisy. Have you considering using an AVIsynth filter like MCTD (http://forum.doom9.org/showthread.php?t=139766)?
It is noisy. I posted a clip without noise removal in order to make the HDTV artifacts obvious in case they occur. I did do some noise removal with frame blending in After Effects. Works quite well, but blurs motion a bit (problematic for stars). I wasn't aware of MCTD and will do some testing. One problem is that the noise removal really should be performed on all the original image sequences (as I do in After Effects). More than 50 timelapse sequences in the film need noise removal, so that's a lot of work.

I tried both clips on my Samsung HDTV via a USB stick. I didn't notice any unusual problems with the high bitrate clip. The noise is a little less smoothed over in the high bitrate clip though.
Thanks a lot, much appreciated. Same result as I've been getting on the setups I've tested.

Perhaps he just hates noise?
If he didn't before, he does now! But something is definitely going wrong with the HDTV setups, verified by the fact that the noise was ok to him on a 1080p projector. As I write he's trying to see if it's possible to capture the artifacts on video.

[flyvholm]

So my friend came up with a way to approximately reproduce the HDTV artifacts: The sharpness setting on the VLC player. Crank that up to the 2nd bar and similar artifacts appear. Not only does the noise become much harsher, it also arranges in patchy blocks that change quite artificially. Below are links to a VLC screenshot and a video showing the artifacts up close on an HDTV.

Screenshot of artifacts simulated with VLC (http://thetimelapsephotographer.com/shared/forums/VLC-sharpness-screenshot.jpg)
Video showing artifacts on HDTV (http://thetimelapsephotographer.com/shared/forums/HDTV-artifacts.MOV)

If enlarging the screenshot you can see how that noise becomes really ugly on a large screen as the video shows. Like on VLC, minimizing sharpening on the HDTVs can reduce the noise to an acceptable level. Unfortunately, unlike VLC it will make the image very soft. There's no reasonable compromise.

It just seems beyond bad luck that I (or anybody else I've asked to test) have not encountered the issue while it's present on every one of the 10+ HDTVs he has tested. But given the variety of setups he has tested for playback - laptops, USB stick, portable media players, different cables - I really don't know what to blame. :confused:

[vivan]

Tbh, for "High bitrate sample (8mbps)" bitrate is way too low, that's why grain looks so bad. Try crf 16..18, and/or proper denoise...

Sharpening is bad and wrong, producing such terrible artifacts is one of the reasons. Disabling it doesn't make image softer, it was soft to begin with. I highly doubt that TV's apply blurring when you turn sharpness down, so it should stay at zero.
If you, as director, believe that your video is soft and you don't like it, then you should apply mild sharpening (after denoise) to your footage.

[Stereodude]

So my friend came up with a way to approximately reproduce the HDTV artifacts: The sharpness setting on the VLC player. Crank that up to the 2nd bar and similar artifacts appear. Not only does the noise become much harsher, it also arranges in patchy blocks that change quite artificially. Below are links to a VLC screenshot and a video showing the artifacts up close on an HDTV.

Screenshot of artifacts simulated with VLC (http://thetimelapsephotographer.com/shared/forums/VLC-sharpness-screenshot.jpg)
Video showing artifacts on HDTV (http://thetimelapsephotographer.com/shared/forums/HDTV-artifacts.MOV)

If enlarging the screenshot you can see how that noise becomes really ugly on a large screen as the video shows. Like on VLC, minimizing sharpening on the HDTVs can reduce the noise to an acceptable level. Unfortunately, unlike VLC it will make the image very soft. There's no reasonable compromise.

It just seems beyond bad luck that I (or anybody else I've asked to test) have not encountered the issue while it's present on every one of the 10+ HDTVs he has tested. But given the variety of setups he has tested for playback - laptops, USB stick, portable media players, different cables - I really don't know what to blame. :confused:
Your VLC screenshot generates a 403.

If sharpening shows the issue, it's not surprising. TVs in stores in the US aren't going to be set up properly, so whatever settings they default to out of the box are used. Generally, the default sharpness settings are much too high.

I have the sharpness on my TV turned down as it should be. Maybe the TVs you've tried have also been setup better with the sharpness turned down.

[Ghitulescu]

TVs in stores in the US aren't going to be set up properly, so whatever settings they default to out of the box are used. Generally, the default sharpness settings are much too high.

I have the sharpness on my TV turned down as it should be. Maybe the TVs you've tried have also been setup better with the sharpness turned down.

The point would be valid if the file was designed for a single person.
But since the file was conceived as a "digital download" and the bilk of the population has anyway "tweaked" TV sets (also in Europe, not only in the US :) ), the problem remains.

Besides, a good source also suffers from "image improvers".

[flyvholm]

Your VLC screenshot generates a 403.
Oops, sorry. Should be fixed now.

No doubt it's a common problem that default HDTV settings are too aggressive. It's just hard to believe that all the TVs he has tested are not even capable of showing source footage as-is, without applying destructive levels of either sharpening or softening. And particularly hard to believe that a number of different Samsungs would be that poor while Stereodude's happens to be fine.

@Stereodude: What model is your Samsung TV, and are you in the US? Could be interesting in case my friend gets a chance to try the same model.

Are there any differences in specs between HDTVs in EU and US that could be relevant?

[foxyshadis]

If people see sparkly artifacts because they've got their sharpness maxed out, or some other picture processing, I guarantee that they're seeing the same artifacts across everything else they watch anyway, so they'd be used to it. It's not worth worrying about. Most people will make some attempt to fix the factory settings, even if it's from a friend or hired help if they can't.

It's annoying that not only manufacturers, but even retailers like to boost the sharpening, saturation, and motion interpolation to unrealistic levels when they unbox display models.

[Stereodude]

@Stereodude: What model is your Samsung TV, and are you in the US? Could be interesting in case my friend gets a chance to try the same model.
PN64F8500 & yes.

Are there any differences in specs between HDTVs in EU and US that could be relevant?
Not that I'm aware of.

I knew enough to turn the sharpness way down.

[Stereodude]

It is noisy. I posted a clip without noise removal in order to make the HDTV artifacts obvious in case they occur. I did do some noise removal with frame blending in After Effects. Works quite well, but blurs motion a bit (problematic for stars). I wasn't aware of MCTD and will do some testing. One problem is that the noise removal really should be performed on all the original image sequences (as I do in After Effects). More than 50 timelapse sequences in the film need noise removal, so that's a lot of work.
MCTD doesn't blur motion. You should give it a shot. It's very powerful.

[flyvholm]

If people see sparkly artifacts because they've got their sharpness maxed out, or some other picture processing, I guarantee that they're seeing the same artifacts across everything else they watch anyway, so they'd be used to it. It's not worth worrying about.
My film is mostly aurora footage, meaning high ISO, low light, subtle gradients across the sky. By it's nature it's unusually susceptible to artifacts, so I'm afraid it's quite possible that poorly adjusted TVs can show most content ok while pushing this footage over the edge. However, I can't shed the feeling that the issues my friend is seeing are not just down to poor TVs. I used to live same place he does and showed my aurora footage on misc HDTVs without issues. In any case, I do want to find out what's at fault and how widespread the problem is, reason being that I must try to make my digital download look acceptable to as many users as possible.

MCTD doesn't blur motion. You should give it a shot. It's very powerful.
It does look promising, and I'm looking forward to seeing results. Unfortunately my first attempts resulted in VirtualDub crashing or running out of memory, but that will be a subject for another thread if I can't solve it.

[foxyshadis]

One other thing I just thought of -- LCD overdrive naturally tends to sparkle, and if there's a little shimmer in the video, that'll probably lead to a lot of sparkle with overdrive. Most TVs don't give you any way to change overdrive, but a few nice units do. Overdrive basically creates a frame (or two) of ringing when a pixel changes, because the overdrive pushes it beyond where it needs to go to make it change faster.

It's not nearly as much of a problem now that response times are much faster across the board than in the 2000's, but it still crops up as a problem from time to time. That's something you can't do much about.

[huhn]

the samples are very noise and I would say the quality is pretty poor. tested on my 3D LUT calibrated TV and that is not sharp or denoise at all.

do you have a lossless sample part? and this has nothing to do with how motion is handled on my TV is is easily visible on a still standing picture.

[Taurus]

Yeah, huhn is right.
Give us a sample lagarith lossless file to toy with.
I watched your "high bitrate":confused::mad: file
on two computer monitors and a standard flatscreen tv.
It looks like crap even on an old CRT monitor.
Noise all over the place.
Try more bitrate and/or different x264 settings.
Or try Stereodude' suggestion and calm the grain/dirt down
before encoding.

[flyvholm]

Huhn, Taurus: Thank you very much for testing, it is much appreciated! Interestingly your opinions appear to be somewhat different from that of previous testers, so now the big question is if there's an actual difference or if you are just more critical.

Take a look at the following two screenshots:
VLC without sharpening (http://thetimelapsephotographer.com/shared/forums/VLC-unaltered-screenshot.jpg)
VLC with sharpening (http://thetimelapsephotographer.com/shared/forums/VLC-sharpness-screenshot.jpg)
This illustrates the difference between what I see on my end and what my friend sees on misc HDTVs. Both images have issues, but the question is: Does the noise you're seeing resemble the non-sharpened screenshot (softer, smoother) or the sharpened one (harsh w. artifacts)? And what make/model are your TVs?

Here's the Lagarith lossless AVI, which is the source of the MP4-clips:
Lossless AVI file (http://thetimelapsephotographer.com/shared/forums/Title-uncompressed.avi) (483MB)
Bear in mind that this is before noise reduction - purposely, I'm trying to establish why the noise looks so different depending on playback setup - and that shooting aurora footage is a compromise. Noise and softness simply cannot be avoided if you want fluid footage.

[Taurus]

Downloading your clip right now.
Tomorrow I have access to a calibrated 27" monitor.
Will report later on.
And yes, it looks more like the sharpened image on the TV flatscreen.
It's a Phillips 6000 series 32" (I think). Specs unknown.
All settings on factory defaults for the test.
Way too much oversharpening this way.
Without color enhancement/sharpening much better,
but still not pro level.
Will report on tuesday if you are still interested.

[Stereodude]

Here's the Lagarith lossless AVI, which is the source of the MP4-clips:
Lossless AVI file (http://thetimelapsephotographer.com/shared/forums/Title-uncompressed.avi) (483MB)
So I made two attempts at compressing your clip. The first basically compressing it as is, the second with some MCTD NR. Here's a zip of both.

http://stereodude.net/title_x264.zip

[flyvholm]

@Taurus: Thanks a lot for the detailed feedback, and I'm quite interested in what else you find. On the other monitors you tried and when disabling color enhancement/sharpening on the TV, do you get an image similar to my screenshot without sharpening? Or is it still looking worse, one way or another?

The source is soft/noisy, but it doesn't actually bother most people as long as the noise looks "natural". However, the sharpened noise is highly disturbing, so all the HDTVs with default, excessive sharpening make a real problem. The noise reduction I've done isn't sufficient to deal with that, so I have a lot of work ahead of me trying to minimize noise while also avoiding too obvious artifacts.

@Stereodude: Thanks a lot. You've used higher bitrate, and as should be the case it looks better - when displayed properly. Unfortunately, the much more detailed noise goes all nuts on those TVs with excessive sharpening. My friend suggests that in terms of the noise the poor HDTVs look like VLC w. sharpening cranked up to the second bar. That's just torture, but it's what I'll have to deal with, because it seems like that is what a lot of people are going to see. You'll see that your MTCD-treated version doesn't even cut it. I made a version that seems to have the noise sufficiently controlled via MCTD("high") and high compression on top:
Denoised with MCTD (high) (http://thetimelapsephotographer.com/shared/forums/Title-MCTD-high.mp4)
I prefer the look of your sensibly denoised/compressed version, but I'm afraid I'll have to go harder with noise suppression. Still experimenting...

[Stereodude]

@Stereodude: Thanks a lot. You've used higher bitrate, and as should be the case it looks better - when displayed properly. Unfortunately, the much more detailed noise goes all nuts on those TVs with excessive sharpening. My friend suggests that in terms of the noise the poor HDTVs look like VLC w. sharpening cranked up to the second bar. That's just torture, but it's what I'll have to deal with, because it seems like that is what a lot of people are going to see. You'll see that your MTCD-treated version doesn't even cut it. I made a version that seems to have the noise sufficiently controlled via MCTD("high") and high compression on top:
Denoised with MCTD (high) (http://thetimelapsephotographer.com/shared/forums/Title-MCTD-high.mp4)
I prefer the look of your sensibly denoised/compressed version, but I'm afraid I'll have to go harder with noise suppression. Still experimenting...
So I tried all the clips on both of my HDTVs (1 LCD / 1 Plasma) with the Sharpness set to the default of 50 (should be 0). I agree the noise / grain is accentuated by the increased sharpness, but it isn't as bad as your VLC screenshot.

I also played all the clips back with the sharpness properly set. When set correctly I liked the title_crf18_nr.mkv the best. The noise is significantly reduced, but not completely wiped. title_crf18_nr.mkv looked acceptable with the sharpness at 50. There are plenty of commercial blu-ray discs with more noise / grain than it that will have similar problems with noise / grain accentuation by the cranked sharpness. Those sorts of people probably just think there's lots of "detail" in their HD.

I should probably also mention that I sit closer to my TVs than most people. My head is 8.5' from the 64.5" plasma.

[Taurus]

Hi flyvholm,
yesterday evening i had access to a calibrated monitor.
The good thing is: Your files look better than expected.
Much less visible noise and distortions.
The bad thing is: I guess about 90 % of all TV's are misaligned.
Same to PC monitors.
The only thing you can do is, find your sweetspot between sharpness and blurring.
Your material is quite demanding.
I have tried some (avisynth) degrainers/denoisers on it.
Nothing really impressed me.
I saw your post in the avisynth thread about MCTD.
I dont think this will do the trick.
I'm not so familiar with MCTD.
The last time I used it is maybe 4-5 years ago.
It needs a high learning curve (Haha, I read about your crashes).
One suggestion:
Try the neatvideo plugin for virtualdub (payware).
Simple to use and in my opinion better than all avisynth plugins
that I have tested on your lagarith clip.
But maybe some of the avisynth gurus will drop by and give a helping hand.
I wish you good luck on your project.
If you have further questions: just go on

[flyvholm]

@Stereodude: I'm getting the feeling HDTVs are highly variable in this respect. The Sony Bravia I have here is extremely well behaved with the footage looking ok even at max sharpness. I would guess what my friend has been seeing is on the other end of the spectrum, and he hardly cares about what looks best on good displays because all he sees are bad displays. So based on his experience he wants to denoise/compress as hard as needed for the footage to look acceptable on those because he believes that's what most people will see. And maybe he's right. I can offer two versions, of course, but the question is what I should go with as the default choice for John Doe.

@Taurus: Thanks a lot for your feedback, I pretty much agree with everything. Indeed, Neat Video looks like a very good proposition, especially because I can get it as an After Effects plugin so I can easily integrate it with the rest of my post processing - would save me a ton of work. The color space conversions required for the Avisynth filters aren't doing the footage any favors either. Even if Neat Video looks like a winner, I owe the good guys here at Doom 9 to investigate the Avisynth options too, crash as I may. :cool:

[DarkSpace]

So based on his experience he wants to denoise/compress as hard as needed for the footage to look acceptable on those because he believes that's what most people will see. And maybe he's right. I can offer two versions, of course, but the question is what I should go with as the default choice for John Doe.
I'm not Stereodude (obviously), but if you're going to offer two versions (and getting the alt. version has no additional fee attached), then I'd say you should make the heavily compressed version the default:
John Doe has no idea about video compression, artefacts, or others (in fact, some people even once told me that they couldn't see a significant difference between the VHS recording and the BD version of a movie [I don't remember which], though I'd estimate that this was rather an extreme case as well), but you should then clearly make the higher-quality version noticeably available (perhaps with a caveat that it might look worse on badly set-up screens).

[huhn]

Huhn, Taurus: Thank you very much for testing, it is much appreciated! Interestingly your opinions appear to be somewhat different from that of previous testers, so now the big question is if there's an actual difference or if you are just more critical.

Take a look at the following two screenshots:
VLC without sharpening (http://thetimelapsephotographer.com/shared/forums/VLC-unaltered-screenshot.jpg)
VLC with sharpening (http://thetimelapsephotographer.com/shared/forums/VLC-sharpness-screenshot.jpg)
This illustrates the difference between what I see on my end and what my friend sees on misc HDTVs. Both images have issues, but the question is: Does the noise you're seeing resemble the non-sharpened screenshot (softer, smoother) or the sharpened one (harsh w. artifacts)? And what make/model are your TVs?

Here's the Lagarith lossless AVI, which is the source of the MP4-clips:
Lossless AVI file (http://thetimelapsephotographer.com/shared/forums/Title-uncompressed.avi) (483MB)
Bear in mind that this is before noise reduction - purposely, I'm trying to establish why the noise looks so different depending on playback setup - and that shooting aurora footage is a compromise. Noise and softness simply cannot be avoided if you want fluid footage.

it looks like the VLC without sharpening picture.
the problem is already in the source.

the source is pretty blocky and noise. the blocks are easy to see in the top right just go 1 by 1 frame looks like "noise" blocks.the picture is so noisy it looks like it is dithered to 5-6 bit. I guess this just a limitation of the used camera or even camera in general.

the title_crf18.mkv encode from Stereodude looks decent while the title_crf18_nr.mkv looks fake. the uneven noise is kind of weird.

not sure about the Title-MCTD-high.mp4 it looks really blocky.

I guess I prefer title_crf18.mkv. I wonder this is just a problem in these dark scenes.

[flyvholm]

@DarkSpace: That is likely what I'll end up doing. I'm just hoping that I can do well enough with noise reduction that I don't have to squish the content too much for poor HDTVs.

@huhn: I also prefer natural looking noise over footage that looks processed and artificial. Unfortunately I think a processed version with much lower noise is going to look better to the average user with a poorly calibrated display. The heavy noise is a result of using ISO 3200+ in night scenes (this clip is ISO 8000 IIRC). May not have been clear, but the film is timelapse produced from DSLR still images. Higher ISO -> shorter exposures -> better time resolution in timelapse -> much smoother aurora footage. Daytime footage with low ISO is no problem.

[Stereodude]

the source is pretty blocky and noise. the blocks are easy to see in the top right just go 1 by 1 frame looks like "noise" blocks.the picture is so noisy it looks like it is dithered to 5-6 bit. I guess this just a limitation of the used camera or even camera in general.IMHO you shouldn't look at lossy compressed video frame by frame. You need to watch it in motion played back at full speed.

[huhn]

IMHO you shouldn't look at lossy compressed video frame by frame. You need to watch it in motion played back at full speed.

first of all the source is lossless and you can easily see it in motion.

[Lyris]

Are there any differences in specs between HDTVs in EU and US that could be relevant?

No. HDTV worldwide is defined in ITU Rec709 but manufacturers ignore it.

You're probably seeing image processing crap turned on in the TV. Is it in the Movie mode? You mentioned Sharpness is off, what about Noise Reduction (I think Samsung calls it Digital Clean View). All the stuff turned off?

[flyvholm]

You're probably seeing image processing crap turned on in the TV. Is it in the Movie mode? You mentioned Sharpness is off, what about Noise Reduction (I think Samsung calls it Digital Clean View). All the stuff turned off?
Problem is, I don't see any noise artifacts on the TV I have available for testing. But by now I think it's clear that the problem is indeed poor internal processing in the TVs. My friend who tested a bunch of TVs couldn't even make the footage display properly if neutralizing all settings. It was a choice between ugly, sharpened noise or an image significantly softer than the source. I'm sure that for some TVs neutralizing settings will work. Regardless, the problem remains. Even if I sell the film with an advisory about TVs, I can't expect the average user to dig into the TV settings. So I must do what I can to make the footage look decent on common, default HDTV setups - bad as they are.

[flyvholm]

So I jumped in and bought the Neat Video plugin, which appears to be a good decision. I'm able to get rid of most of the noise and make the remainder static so it is less distracting. The downside with the sample I chose is that it looks completely "dead" now (could have been a still image). Can't have it both ways. Most importantly, I don't see objectionable processing artifacts, but some may disagree. ;) You can see the noise-reduced sample here:

Sample processed with Neat Video (http://thetimelapsephotographer.com/shared/forums/Title-neat-crf20.mp4)

[Stereodude]

first of all the source is lossless and you can easily see it in motion.
Yes, I know the source is lossless. Some of the artifacts you talked about in the the compressed versions I don't see in motion, only with slowly stepping through frames of them.

I agree that title_crf18.mkv isn't perfect. It could be better. Some of the noise / artifacts are static and don't change frame to frame so MCTD won't clean that up since that's not the type of noise it works on. However, in general my preference is to remove the fine dancing grain/noise that rides on top of video footage since it doesn't add anything to the content and hinders compressibility.

[Lyris]

Problem is, I don't see any noise artifacts on the TV I have available for testing.

Which TV is that? (Apologies if you said already).

And, is it accurate in terms of video processing? How do you know your own TV doesn't have undefeatable NR that stops you from seeing a true representation of the source?

(The Samsung F8500 plasma does not have undefeatable NR, BTW - at least not in the US or European versions.)

[Stereodude]

And, is it accurate in terms of video processing? How do you know your own TV doesn't have undefeatable NR that stops you from seeing a true representation of the source?
Like something from LG? ;)

I think as long as we don't mention those organic LED displays from Lucky Goldstar Ken won't start posting here to defend them. Hopefully this post goes undetected and doesn't trigger the bat signal. :p

[flyvholm]

The TV is a Sony Bravia KDL-32V5500. The footage plays back on my laptop (Asus G73JH) with a 17.3" 1080p display, connected to the TV via HDMI. The two displays essentially show the same (except for color calibration).

[huhn]

not sure how old that sony is but you can go into gaming/graphic or pc mode that should disable all unnecessary progressing and may give you 4:4:4. if it is not calibrated the colors are wrong any way and who knows which preset is more accurate.

[foxyshadis]

The TV is a Sony Bravia KDL-32V5500. The footage plays back on my laptop (Asus G73JH) with a 17.3" 1080p display, connected to the TV via HDMI. The two displays essentially show the same (except for color calibration).

Bravias are some of the finest TVs money can buy, so it's no surprise it looks fantastic compared to a random Visio or Westinghouse at the store. Glad you found some decent settings to help, though.

[Stereodude]

So I jumped in and bought the Neat Video plugin, which appears to be a good decision. I'm able to get rid of most of the noise and make the remainder static so it is less distracting. The downside with the sample I chose is that it looks completely "dead" now (could have been a still image). Can't have it both ways. Most importantly, I don't see objectionable processing artifacts, but some may disagree. ;) You can see the noise-reduced sample here:

Sample processed with Neat Video (http://thetimelapsephotographer.com/shared/forums/Title-neat-crf20.mp4)
That still has some artifacts that look like JPEG compression artifacts that are in the "static" image. IMHO it looks over-processed.

Bravias are some of the finest TVs money can buy, so it's no surprise it looks fantastic compared to a random Visio or Westinghouse at the store. Glad you found some decent settings to help, though.
Did you suddenly find yourself transported to the early to mid 90's making a statement like that? :p

[Stereodude]

Alright, so I decided to make a serious attempt at cleaning up the footage and came up with this (http://stereodude.net/title_crf18_alt_nr3.zip).

[Lyris]

Like something from LG? ;)

I think as long as we don't mention those organic LED displays from Lucky Goldstar Ken won't start posting here to defend them. Hopefully this post goes undetected and doesn't trigger the bat signal. :p
Oh my. The internet is a small place!

Fortunately LG took the feedback on board for the newer models!

Sony LCDs aren't my specialty, so no idea what the KDL-32V5500 is doing.

[Stereodude]

Oh my. The internet is a small place!
It indeed is. I've even run into some people on car forums that I knew years earlier on a few different A/V related forums.

Fortunately LG took the feedback on board for the newer models!

Yes, that's a good thing. I wonder if LG fixed their LCDs also. I recall reading they had the same issue with forced DNR.

[flyvholm]

Alright, so I decided to make a serious attempt at cleaning up the footage and came up with this (http://stereodude.net/title_crf18_alt_nr3.zip).
That does look better. What did it take to get there?