FFMpeg - Good Settings for SD?

[ENunn]

I'm encoding one of my lossless Betacam rips to x265 just for fun, and I'm trying to find some good settings that preserve as much detail as possible. I'm playing around with it but it seems like no matter what I do, there's smoothing. It might not be a bad thing though.

Source is 720x480 MagicYUV, deinterlaced with QTGMC in Avisynth.

Script:
setmemorymax(63448)
v = AVISource("i:\virtualdub\tape transfers\betacam\sp\boxing adventure work 2.avi")
a = lwlibavaudiosource("i:\recordings\boxing adventure work 2.wav")
audiodub(v,a)
assumetff()
converttoyuv444(matrix="rec601", interlaced=true).convertbits(10)
Levels(55, 1, 915, 0, 1020, coring=false,dither=true).tweak(bright=0, cont=1.00, hue=-0, sat=1, coring=false, dither=true).convertbits(8)
#turnRight().Histogram().TurnLeft()
Trim(186, 5815)
Trim(0, 1371)
qtgmc(preset="faster", ezdenoise=0.0, sharpness=0, sourcematch=3, lossless=2)
prefetch(6)

Original
https://i.imgur.com/LgnrXUW.png


-pix_fmt yuv420p10le -c:v libx265 -crf 16 -b:v 0K -preset slow
https://i.imgur.com/IuqCPOX.png


-pix_fmt yuv420p10le -c:v libx265 -crf 16 -x265-params "no-deblock=1:no-sao=1" -b:v 0K -preset slow
https://i.imgur.com/vIKwBQk.png


-pix_fmt yuv420p10le -c:v libx265 -crf 16 -x265-params "no-deblock=1:no-sao=1:tskip=1:tskip-fast=1:subme=7:no-strong-intra-smoothing=1:hme=1:amp=1:rect=1" -b:v 0K -preset slow
https://i.imgur.com/ratfWg2.png

And here's a raw clip. (https://mega.nz/file/D58XSKZS#2LC8r5RHS8cb6Kk8Jz-TVzAczxKjrIo1GXHz_sDACkc)

The results aren't bad by any means, but I think the smoothing is a bit too much. I know x265 isn't really meant for SD, and I usually encode to interlaced x264, but I waned to give it a shot. Anyone have any ideas?

[benwaggoner]

Leave deblocking on; turning it off won't help much. Increasing --psy-rd and --psy-rdoq will help more.

I wouldn't use anything below --preset slower for SD. Even veryslow should have good performance at those low resolutions.

(remembering back when 320x240p15 encoding took 80:1 real time to encode to Cinepak...)

[ENunn]

Gotcha. I did another test encode with new settings. Unfortunately the encode speed dipped from 12-15fps to 3-5fps. It gets slower over time for some reason.

The results are really good though.

-pix_fmt yuv420p10le -c:v libx265 -x265-params "no-sao=1:tskip=1:tskip-fast=1:subme=7:no-strong-intra-smoothing=1:hme=1:amp=1:rect=1:psy-rd=3.0:psy-rdoq=2.0" -crf 16 -b:v 0K -preset slower
https://i.imgur.com/RuZ8qk7.png

I wish it was faster though :/

[Boulder]

subme=7 and hme are probably the biggest reasons for the encoder being slow. There's absolutely no reason for using HME at SD resolutions, to be honest, I've not seen any benefits even at 1440p.

Also, why not encode at 10 bits? It will also help 8-bit content avoid banding.

[ENunn]

subme=7 and hme are probably the biggest reasons for the encoder being slow. There's absolutely no reason for using HME at SD resolutions, to be honest, I've not seen any benefits even at 1440p.

Took hme out, around 4.8-7fps. Better, but not great. Does subme=7 have any benefits?

The result still looks good though. Although I noticed that there was more grain kept when I had hme, but then again, it's probably not a big deal.

-pix_fmt yuv420p10le -c:v libx265 -x265-params "no-sao=1:tskip=1:tskip-fast=1:subme=7:no-strong-intra-smoothing=1:amp=1:rect=1:psy-rd=3.0:psy-rdoq=2.0" -crf 16 -b:v 0K
https://i.imgur.com/UcjWjsS.png

Also, why not encode at 10 bits? It will also help 8-bit content avoid banding.

I am encoding at 10 bits. I have -pix_fmt yuv420p10le in my command.

[Boulder]

Took hme out, around 4.8-7fps. Better, but not great. Does subme=7 have any benefits?
Subme 5 is already considered placebo. I've used 4 in my encodes and I'm happy with that setting.


I am encoding at 10 bits. I have -pix_fmt yuv420p10le in my command.
I would then deinterlace first, then do all the processing in higher bitdepth and feed 16- or 10-bit data to x265 with --dither enabled. I think that matrix change could be done in a more optimal way with avsresize.

[benwaggoner]

Took hme out, around 4.8-7fps. Better, but not great. Does subme=7 have any benefits?

The result still looks good though. Although I noticed that there was more grain kept when I had hme, but then again, it's probably not a big deal.
I've never seen subme 7 do anything material that subme 5 didn't. But if it would be anywhere, it would be SD and below resolutions, where every pixel does count visually.

-pix_fmt yuv420p10le -c:v libx265 -x265-params "no-sao=1:tskip=1:tskip-fast=1:subme=7:no-strong-intra-smoothing=1:amp=1:rect=1:psy-rd=3.0:psy-rdoq=2.0" -crf 16 -b:v 0K
Just try --preset placebo with -F 1, --skip, and your --psy* settigns and see if that improves anything. If so, we can figure out why.

Adjusting the --psy* values could help. If placebo doesn't look better than slower, you can keep tuning from slower. Switching to --rskip 2 with a --rskip-edge-threshold of 2 or 3 can give you better detail than the default --rskip 1.

--ctu 32 --qg-size 16 could also help, and with slower and above you have the --tu-int* of 3 so full recursion down to the smallest TU size.

--no-strong-intra-smoothing doesn't do anything beneficial. Really, --no-sao is about the only in-loop filter that has positive benefit on detail retention, and then only at quite high bits-per-pixel.


I am encoding at 10 bits. I have -pix_fmt yuv420p10le in my command.
I don't see you setting --profile main10. That's required for true 10-bit encoding.

Also, crop 8 pixels left and right to get rid of the horizontal blanking area. The clean 4:3 area in a NTSC SD source is the center 704x480.

Doing a good bob deinterlace from 30i to 60p will retain all the source motion, which can really help the quality of experience at SD. You already don't have enough pixels, so keeping all the temporal data can make a really big difference.

In the end, based on your "original" frame below the source isn't that great. I doubt many of the above tweaks will make a difference when the video is playing at full speed.

[ENunn]

Subme 5 is already considered placebo. I've used 4 in my encodes and I'm happy with that setting.
I've never seen subme 7 do anything material that subme 5 didn't. But if it would be anywhere, it would be SD and below resolutions, where every pixel does count visually.
Changed it to 4 then.

I don't see you setting --profile main10. That's required for true 10-bit encoding.
Mediainfo says the profile is Main10. Main 10@L3.1@Main. So I think ffmpeg applied the profile automatically.

I would then deinterlace first, then do all the processing in higher bitdepth and feed 16- or 10-bit data to x265 with --dither enabled. I think that matrix change could be done in a more optimal way with avsresize.
Doing a good bob deinterlace from 30i to 60p will retain all the source motion, which can really help the quality of experience at SD. You already don't have enough pixels, so keeping all the temporal data can make a really big difference.
Already taken care of, QTGMC is in my Avisynth script.

Also, crop 8 pixels left and right to get rid of the horizontal blanking area. The clean 4:3 area in a NTSC SD source is the center 704x480.
Not a fan of cropping any visible information, only the black lines and head switch noise, so I set an aspect ratio flag to 15:11 via -aspect 652:480 and I'll get the same result without cropping.

In the end, based on your "original" frame below the source isn't that great. I doubt many of the above tweaks will make a difference when the video is playing at full speed.
I have a clip linked in the first post if you want to play around with it.

Played around with my command and added Ben's suggestions. Encoding speed was a little better but I didn't see the framerate because I was away from my computer. My 8700K isn't great though, that pretty much explains the slow speed.

Results look good, maybe a little better than my last command.

-pix_fmt yuv420p10le -c:v libx265 -x265-params "no-sao=1:tskip=1:tskip-fast=1:subme=4:rskip=2:rskip-edge-threshold=3:ctu=32:qg-size=16:amp=1:rect=1:psy-rd=3.0:psy-rdoq=2.0" -crf 16 -b:v 0K -preset slower
https://i.imgur.com/6Cxqdzm.png

Just try --preset placebo with -F 1, --skip, and your --psy* settigns and see if that improves anything. If so, we can figure out why.

In terms of speed or quality? I'm encoding right now and I'm getting 1-2fps.

[Boulder]

How about --psy-rd 1.8 --psy-rdoq 5.0 --rdoq-level 1 for the psy settings, adjust psy-rd some fractions up or down to tune. I remember someone was quite happy with those baseline parameters.

[benwaggoner]

Changed it to 4 then.
Well, --subme 5 we do know improves quality a bit over 4. It's only 6 & 7 that seem to have almost no measurable improvement.

Not a fan of cropping any visible information, only the black lines and head switch noise, so I set an aspect ratio flag to 15:11 via -aspect 652:480 and I'll get the same result without cropping.
Sure, but you clearly have horizontal blanking in your example frame that should be removed.

I also stay away from fractional SAR ratios, as some pretty popular SoCs a few years back shipped with drivers that didn't support them, and defaulted to square pixel. It's best to stick to one of the standard 16, which are universally supported these days (wasn't true 20+ years ago). There is one for 704x480 4:3.

Played around with my command and added Ben's suggestions. Encoding speed was a little better but I didn't see the framerate because I was away from my computer. My 8700K isn't great though, that pretty much explains the slow speed.

Results look good, maybe a little better than my last command.

-pix_fmt yuv420p10le -c:v libx265 -x265-params "no-sao=1:tskip=1:tskip-fast=1:subme=4:rskip=2:rskip-edge-threshold=3:ctu=32:qg-size=16:amp=1:rect=1:psy-rd=3.0:psy-rdoq=2.0" -crf 16 -b:v 0K -preset slower
https://i.imgur.com/6Cxqdzm.png

In terms of speed or quality? I'm encoding right now and I'm getting 1-2fps.
Six cores with AVX2 isn't too bad for standard def encoding.

For this kind of analog source, --tskip isn't going to provide any benefit. Dropping that is one way to get some speed back.

Did you try leaving --sao on? since Beta SP source is going to be pretty soft anyway, SAO might not hurt detail significantly and could drop your bitrate some, or allow for a slightly higher CRF.