Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
20th April 2021, 16:32 | #1 | Link |
Registered User
Join Date: Dec 2008
Posts: 99
|
Encoding 1080i clips with x264 incredibly slow
For years I've been capturing video from my DirecTV box using a Hauppauge box at 720p60. I recently decided that I would try to capture at 1080i30 since I recently upgraded my computer which should be able to handle such material.
This is the script I normally use for 720p60: Code:
v=LWLibavVideoSource("J:\Clip.TS") a=LWLibavAudioSource("J:\Clip.TS").Normalize() AudioDub(v,a) ColorMatrix() UnDot() ConvertToYV12() Trim(373,108160) When I used the 1080i clips in that same script, the result was a very jumpy video, which I somewhat expected. After some research, I decided upon this script: Code:
v=LWLibavVideoSource("J:\Clip.TS",fpsnum=30000, fpsden=1001) a=LWLibavAudioSource("J:Clip.TS").Normalize() AudioDub(v,a) ColorMatrix() UnDot() ConvertToYV12() Trim(700,218080) AssumeTFF() AssumeFrameBased() SeparateFields() Spline36Resize(1920,1080) SelectEven() However, when encoding in MeGUI using the same x264 settings, I was getting ~10 fps on the first pass! This a major dropoff and a dealbreaker (the CPU is running at 99% here too). Is this to be expected? I realize I'm asking Avisynth to do more, but didn't think it would slow down this much. Is there anything I can do to speed this up? |
20th April 2021, 18:57 | #3 | Link |
Registered User
Join Date: Dec 2008
Posts: 99
|
AVSMeter screencap attached. The ColorMatrix, UnDot and ConvertToYV12 are holdovers that I picked up awhile back. I haven't revisited them to see if they are still valid. Last edited by abyss616; 20th April 2021 at 19:05. |
20th April 2021, 22:30 | #4 | Link | |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
|
You're saying you record in 1080i which basically means that you're already on 1920x1080 29.970 interlaced, so why are you doing this:
Code:
AssumeTFF() AssumeFrameBased() SeparateFields() Spline36Resize(1920,1080) SelectEven() Quote:
This is what you can do to encode in 1080i NTSC using/refactoring your own script: Code:
#Indexing v=LWLibavVideoSource("J:\Clip.TS",fpsnum=30000, fpsden=1001) a=LWLibavAudioSource("J:Clip.TS").Normalize() AudioDub(v,a) #trim Trim(700,218080) #Let's go to 4:2:0 planar 8bit Converttoyv12(matrix="Rec.601", interlaced=true) #From BT601 to BT709 ColorMatrix(mode="Rec.601->Rec.709", interlaced=true, threads=0, thrdmthd=0) #Bobbing to 60p to remove dots Bob() UnDot() #Re-interlace to 30i assumeTFF() separatefields() selectevery(4,0,3) weave() If the final result is choppy, it means that the fields were somehow not read properly. Sometimes you need to specify AssumeTFF or AssumeBFF to get them correctly. Anyway, let me know... Last edited by FranceBB; 20th April 2021 at 22:33. |
|
21st April 2021, 02:08 | #5 | Link |
Registered User
Join Date: Dec 2008
Posts: 99
|
First: thank you for responding. I appreciate the help.
Second: why did I do what I did? Because I don't know what I'm doing! This is my first time working with 1080i material - I looked up a few threads on deinterlacing and that's what was recommended. ::shrug:: Using your edits gives me this in AVSMeter: ... and the resulting video is chaos - tries to give me a seizure, with or without the --tff switch. MediaInfo says my source media is: Scan type: Interlaced Scan type, store method: Separated fields Scan order: Top Field First |
21st April 2021, 13:11 | #6 | Link | |||
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
|
Quote:
If it's really interlaced and the field parity is correct, then I guess there's something wrong in the metadata of the source file and the way they're passed to Avisynth by LWLibav. For instance, if the source file says progressive but it's actually interlaced you would need an AssumeTFF() or AssumeBFF() before everything in Avisynth, otherwise bad things are gonna happen. But in the meantime, try with this: Code:
#Indexing FFMpegSource2("J:\Clip.TS", atrack=-1) AssumeTFF() #Loudness Correction to -24LUFS Normalize(0.24) #trim Trim(700,218080) #Let's go to 4:2:0 planar 8bit Converttoyv12(matrix="Rec.601", interlaced=true) #From BT601 to BT709 ColorMatrix(mode="Rec.601->Rec.709", interlaced=true, threads=0, thrdmthd=0) #Bobbing to 60p to remove dots Bob() UnDot() #Re-interlace to 30i assumeTFF() separatefields() selectevery(4,0,3) weave() if it's still a mess, then try with: Code:
#Indexing FFMpegSource2("J:\Clip.TS", atrack=-1) AssumeBFF() #Loudness Correction to -24LUFS Normalize(0.24) #trim Trim(700,218080) #Let's go to 4:2:0 planar 8bit Converttoyv12(matrix="Rec.601", interlaced=true) #From BT601 to BT709 ColorMatrix(mode="Rec.601->Rec.709", interlaced=true, threads=0, thrdmthd=0) #Bobbing to 60p to remove dots Bob() UnDot() #Re-interlace to 30i assumeTFF() separatefields() selectevery(4,0,3) weave() If once again is a mess, then I gotta take a look at the the sample... There are many things that can go wrong with interlaced stuff, but there's no way for us to find out without a sample, unfortunately... Quote:
Quote:
What I tried to do with the script is to deliver an interlaced result, but you could as well go for a progressive output. Honestly it's up to you. In my case, working in broadcast (although in a PAL country) I have to go to interlaced, but if you're a home user, you can go to progressive as well. A few things to keep in mind: - if you keep the encode as interlaced, you're gonna need less space as interlaced encodes take less space - it's gonna be up to the decoder to deinterlace or bob-deinterlace for display according to the metadata. Pretty much any decent decoder can easily do that, like any players on Windows (MPV, MPC-HC, PotPlayer etc) or TVs etc. Anyway, I'm gonna be here waiting for a sample. Oh and in case you wanna go to progressive, we're gonna use something better than bob() of course xD Side note about the "Loudness Correction": you have put a "Normalize()" there in your script without anything, so I assumed you wanted to use it to perform some kind of correction, so I put the values there, but if it's not what you wanted to do, then feel free to remove it. Last edited by FranceBB; 21st April 2021 at 13:15. |
|||
21st April 2021, 15:24 | #7 | Link |
Registered User
Join Date: Dec 2008
Posts: 99
|
1) I've changed so many things around it's hard for me to keep track what I've tried, what I haven't, what worked, what didn't, and what the results look like.
2) It seems that demuxing the original TS file into separate audio/video streams with LWLibavVideoSource makes all the difference in the world in terms of encoding speed. The original script you posted encodes at ~8 fps using TS file as video input and ~76 fps using a demuxed h.264 video stream in an MKV container for video input. 3) What does it tell you that I took out everything from the script after ColorMatrix (so, Bob, UnDot, AssumeTFF, etc...) and just used the --tff switch in x264, and the result doesn't look all that bad? No combing effects, no jumping or anything. It's hard to get a great comparison when you're having to go back and forth between the original and output, but I don't think I can tell much of a difference. |
21st April 2021, 18:07 | #9 | Link | ||
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
|
Quote:
I can't speculate over what could have gone wrong... Quote:
But honestly, man, post a sample... If you think that it's private and you wanna keep it confidential, can't you trim a part where there's only a landscape or nature or a part in which there's nobody and just objects? Anything will do... |
||
22nd April 2021, 00:38 | #13 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Mediafire is popular.
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? |
22nd April 2021, 01:01 | #14 | Link |
Registered User
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
|
__________________
See My Avisynth Stuff |
22nd April 2021, 11:27 | #16 | Link |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
|
Gotcha!
Oh! But this is the NCAA! We air this! hahaha When I first came to Sky Sports in 2017 they were like "and then there's the NCAA" and I was like "what the heck is that?" and they were like "Oh, college stuff" xD Anyway, I can totally say that it's not your fault: the file is a recording from a CBS live feed, so it's truly interlaced and can be easily bobbed to 59.940fps progressive. Now the problem is that both FFMpegSource2 and LWLibavVideoSource are broken due to the container which confuses them (they've never been working great with .ts as container) which is why you saw improvements once you remuxed. Anyway, this is a simple H.264 truly interlaced top field first 29.970i and guess what indexed it correctly? The latest and greatest ffms2? Nope! LSMASH.dll? Nope... The 12 years old indexer DGAVCDecode made by Donald Graft (by the way, he's here, it's "videoh" and he's not a noob, the "noob" thing in his description is a joke, he knows more than you and me combined xD). So, once again, with kudos to Donald Graft for getting it right when others got it wrong, here is your script, using the 12 years old DGAVCDecode for video and the 15 years old NicAudio for audio (boys, we're getting old): Code:
#Indexing video=DGAVCDecode_AVCSource(dga="Z:\2021 One Shining Moment.dga", i420=yes, deblock=yes) audio=NicAC3Source("Z:\2021 One Shining Moment PID 1100 2_0ch 48KHz 384Kbps DELAY -68ms.ac3") AudioDub(video, audio) #Trimming ads trim(161, 5578) #Bob-deinterlacing AssumeTFF() QTGMC(Preset="Medium") #Clipping Limiter(min_luma=16, max_luma=235, min_chroma=16, max_chroma=240) #Loudness Correction Normalize(0.22, show=false) Note: the last two parts (Clipping + Loudness) are optional as you don't have to air anything and you can skip those if you want. Then encode with x264 like you've always done for progressive materials with no flags. Oh well since we're here, here is the x264 crf 18 encode with the script: https://we.tl/t-rUxYcxDEys Cheers, Frank |
22nd April 2021, 12:13 | #17 | Link |
Useful n00b
Join Date: Jul 2014
Posts: 1,667
|
Thank you, FranceBB, for your kindness. Of course DGDecNV (DGIndexNV + DGDecodeNV) is preferred when an nVidia adapter is available, because one can use the PureVideo double-rate deinterlacer, which is quite effective and fast. The OP was concerned about the speed of his script so perhaps QTGMC may be an issue for him.
Always wondered why the alternative source filters cannot handle a relatively simple transport format. The specs are readily available and the authors are very smart people. Kinda baffling. A pathology of open-source development; multiple developers but no-one in particular wants to step up and fix things? Last edited by videoh; 22nd April 2021 at 12:23. |
22nd April 2021, 12:30 | #18 | Link |
HeartlessS Usurer
Join Date: Dec 2009
Location: Over the rainbow
Posts: 10,980
|
Perhaps it requires someone who knows how to do it [subtle hint]
__________________
I sometimes post sober. StainlessS@MediaFire ::: AND/OR ::: StainlessS@SendSpace "Some infinities are bigger than other infinities", but how many of them are infinitely bigger ??? |
22nd April 2021, 13:14 | #19 | Link | |
Broadcast Encoder
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
|
Quote:
Well, if he has an NVIDIA GPU, then yes, DGDecNV is great. @abyss616... If you have a modern NVIDIA GPU and your concern is speed, you can definitely use it. I've been using it with my NVIDIA Quadro P4000 for some years now and it works like a charm. It ain't free, though, but you get lots of activation in a licence for a very small fee. Anyway, if you wanna stick with the free stuff and the script I posted above is too slow, you can replace QTGMC with: Code:
tdeint(mode=1, order=-1, field=-1, mthreshL=6, mthreshC=6, map=0, type=2, debug=false, mtnmode=1, sharp=true, cthresh=6, blockx=16, blocky=16, chroma=true, MI=64, tryWeave=true, link=1, denoise=true) +1 |
|
22nd April 2021, 13:51 | #20 | Link |
Registered User
Join Date: Dec 2008
Posts: 99
|
Thanks everyone -- especially FranceBB -- for the help. My main concern, in regards to the speed, was why it was so slow (exponentially so) when compared to the 720p60 source material I had been using. Plus, the only interlacing/deinterlacing I had done before was on VHS captures that needed to have QTGMC applied to it to get rid of combing. But from what I'm seeing, this doesn't need QTGMC - just AssumeTFF in the script and the -tff flag in x264 and I'm good to go. Sometimes, simpler is better (and faster).
While I appreciate Donald's efforts in the way of his indexer (I am a capitalist, after all), I'll stick with SMASH since its free and demuxing into an MKV container isn't hard to do. My encodes are getting ~113 fps on the first pass, which isn't close to the ~300 fps for 720p60, but it's much better than the ~10 I was getting before. I do have an NVIDIA GeForce GTX 1650 SUPER, so maybe it's something to look into. |
Thread Tools | Search this Thread |
Display Modes | |
|
|