Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
![]() |
#1 | Link |
Apprentice
Join Date: Apr 2012
Location: Poland
Posts: 6
|
Need help with encoding DVB-T PAL signal
Hello dear friends.
I know there are lots of questions like this, but I'm stuck. I'm recording some anime from the DVB-T broadcasted in Poland (which means 576i50). So - the first thing is my decoder. It's Ferguson FT-8100 which records .pvr files (WHICH turn out to be AVC in MPEG2-TS) with ~3000kb/s bitrate. I don't know any program that successfully cuts M2TS (so I can get rid of commercials, etc) - that's why I didn't bother with it too much. I've figured out I can mux it to the MKV with MKVmerge. Output file is then ~100-150MB smaller. How come? For example - 25 minutes long record is ~700MB. After muxing it weighs ~500MBs. But then - I can use Avidemux or VirtualDub to operate on the file. Output files look nicely in the VLC AFTER performing Yadif 2x deinterlace and little sharpening. But I'd like it to be independent from the player - don't need interlaced files. Getting to the point... I'd like to deinterlace the file, sharpen it, reduce the size (I've seen files half the size that look the same if not better) and possibly denoise. You can find RAW sample attached below (already muxed into MKV). VirtualDub always makes it problematic for me so I've been trying to use Avidemux 2.6.0 to recompress it. The thing is - it crashes just after first pass. Speaking of which - I'd need some help with trascoding - not to make the quality worse or the file bigger in size. SAMPLE |
![]() |
![]() |
![]() |
#2 | Link | |||
Registered User
Join Date: Mar 2005
Location: Finland
Posts: 2,641
|
Quote:
Quote:
Why 2-pass? CRF would probably suit you better. Quote:
There might have been some truly interlaced scenes (usually camera pans) too in the original, but I didn't spot any in that sample. They can't be fully restored, and bob-deinterlacing might be the easiest option to handle those parts. Try QTGMC for that. I don't know if there's any tool that can reliably detect such sequences within field-blended video. Last edited by nm; 15th April 2012 at 01:13. |
|||
![]() |
![]() |
![]() |
#3 | Link |
Apprentice
Join Date: Apr 2012
Location: Poland
Posts: 6
|
Thanks for some answers.
ad.1 - it only crashes with 2-pass encoding. You've mentioned I should use CRF - with it I get the error File too short which probably is a side effect of my poor cutting abilities (the sample probably doesn't start with key frame; couldn't find proper indicator in Avidemux - it shows only P and B frames on the scale). BTW - I always thought 2-pass encoding offered the best quality. Any article I can read about it? ad.2 - I'll check this Avisynth script later. Meanwhile I've tried following filters in avidemux (in this order): - Yadif (Bob, temporal & spatial check) - asharp (threshold/strength/block adaptive = 2.0/4.0/2.0) - MPlayer Denoise 3D HQ - and it crashes on this one - only with these muxed MKVs. Now - without denoise filter, after launching it in VLC and performing MPlayer Denoise 3D HQ post processing filter there, with maximum PP strength, the video looks quite nice. The only problem is the size doubles with quantizer value=22. Anyway - when I'll have some free time I'll definitely check this and other AVS scripts. Still got some free space (=time) on the HDD ![]() |
![]() |
![]() |
![]() |
#4 | Link | ||
Registered User
Join Date: Mar 2005
Location: Finland
Posts: 2,641
|
Quote:
In practice CRF is usually the better choice if you aren't targeting a specific file size. You don't need to guess a suitable bitrate to reach the quality you want. CRF is also faster since it only needs one pass. http://www.avidemux.org/admWiki/doku...:h.264#general Quote:
Here's my first try: sm_mdegrain_lsfmod.mkv - Yadifmod with nnedi3 to bob the video for SRestore - SRestore to recover the 24p (24000/1001 fps) video - Stabilise with Depan - Sharpen with LSFMod - Denoise with MDegrain2 (Some lower frequency moving noise is left over though. Would need something heavier.) Encoding at CRF 20 reduced video size by 26 % compared to the original sample (copied here for direct downloads). Another, completely denoised version with reconstructed lines: sm_restored.mkv - Used this script after srestore & depan Encoding at CRF 20 reduced video size by 41 % compared to the original sample. My encodes have a couple bad frames in the beginning. I had to use ffmpeg to make a lossless intermediate since ffms2 has difficulties with interlaced H.264 and I can't use other AviSynth source filters on Linux. Ffmpeg had some problems with the original sample too, although MPlayer and VLC seem to play it fine. Last edited by nm; 16th April 2012 at 17:16. |
||
![]() |
![]() |
![]() |
#5 | Link |
Apprentice
Join Date: Apr 2012
Location: Poland
Posts: 6
|
Hello again. Sorry for such a delay with a reply.
I'd like to say it first - you clip looks ASTONISHING! I didn't know it's possible to achieve such quality with a original source like this. I wanted to follow your guideline but it seems I'm stuck. This is my script: Code:
LoadPlugin("E:\Programy\Multimedia\MeGUI\tools\avisynth_plugin\aWarpSharp.dll") LoadPlugin("E:\Programy\Multimedia\MeGUI\tools\avisynth_plugin\Deen.dll") LoadPlugin("E:\Programy\Multimedia\MeGUI\tools\avisynth_plugin\Depan.dll") LoadPlugin("E:\Programy\Multimedia\MeGUI\tools\avisynth_plugin\MaskTools-v1.5.5.dll") LoadPlugin("E:\Programy\Multimedia\MeGUI\tools\avisynth_plugin\mt_masktools-26.dll") LoadPlugin("E:\Programy\Multimedia\MeGUI\tools\avisynth_plugin\Mvtools2.dll") LoadPlugin("E:\Programy\Multimedia\MeGUI\tools\avisynth_plugin\dfttest.dll") LoadPlugin("E:\Programy\Multimedia\MeGUI\tools\avisynth_plugin\deblock.dll") LoadPlugin("E:\Programy\Multimedia\MeGUI\tools\avisynth_plugin\nnedi3.dll") LoadPlugin("E:\Programy\Multimedia\MeGUI\tools\avisynth_plugin\yadifmod.dll") LoadPlugin("E:\Programy\Multimedia\MeGUI\tools\ffms\ffms2.dll") Import("E:\Programy\Multimedia\MeGUI\tools\avisynth_plugin\FastLineDarken.avs") Import("E:\Programy\Multimedia\MeGUI\tools\avisynth_plugin\DeHalo_alpha.avsi") FFVideoSource("C:\Users\Markus\Desktop\SM_sample.mkv", threads=1) yadifmod(edeint=nnedi3(field=-1)) Import("E:\Programy\Multimedia\MeGUI\tools\avisynth_plugin\SRestore.avsi") srestore() Depan() super = MSuper(pel=2, sharp=1) backward_vec2 = MAnalyse(super, isb = true, delta = 2, overlap=4) backward_vec1 = MAnalyse(super, isb = true, delta = 1, overlap=4) forward_vec1 = MAnalyse(super, isb = false, delta = 1, overlap=4) forward_vec2 = MAnalyse(super, isb = false, delta = 2, overlap=4) Deblock() dfttest() FastLineDarken() deen("a3d",4,8,9) Spline36Resize(last.width*2,last.height*2) AddBorders(4, 0, 4, 0) aWarpSharp(depth=12,blurlevel=4,thresh=0.2,cm=1) FastLineDarken() aWarpSharp(depth=6,blurlevel=4,thresh=0.7,cm=1) Crop(4,0,-4,0) FastLineDarken() DeHalo_Alpha() Spline36Resize(last.width/2,last.height/2) Import("E:\Programy\Multimedia\MeGUI\tools\avisynth_plugin\LSFmod.v1.9.avsi") MDegrain2(super, backward_vec1,forward_vec1,backward_vec2,forward_vec2,thSAD=400) Besides - am I doing it right (loading scripts, DLLs, order of things and parameters)? I've encoded the video without the Depan() function and it looks like it isn't deinterlaced :/ No to mention the 0.5 fps on my C2D E6300... Last edited by marek1712; 1st May 2012 at 20:31. |
![]() |
![]() |
![]() |
#6 | Link |
RipBot264 author
Join Date: May 2006
Location: Poland
Posts: 7,915
|
sometimes less is more.
you are using awarpsharp twice. fastlinedarken three times. resizer two times. first you add borders then you remove them. you should use filters which are really really really needed. no wonder you get only 0.5 fps
__________________
Windows 7 Image Updater - SkyLake\KabyLake\CoffeLake\Ryzen Threadripper Last edited by Atak_Snajpera; 1st May 2012 at 21:22. |
![]() |
![]() |
![]() |
#8 | Link |
Derek Prestegard IRL
![]() Join Date: Nov 2003
Location: Los Angeles
Posts: 5,998
|
Regarding your original question - MPEG-TS almost always contains a good deal of padding / nulls. This is because the final muxes usually need to be strictly CBR, and the video is basically never strictly CBR. Even pseudo CBR usually has a lot of variation on a frame by frame basis (which is a very good thing for quality). MPEG-TS itself also has inherent overhead. Transmitting things like the PCR, PMT, descriptors etc does use some bitrate.
For example, if you were shooting for a 5625kbps mux (commonly used in IPTV applications), you might encode the video at 4700kbps pseudo-CBR, and the audio at 384kbps pure CBR. This leaves an average of 541 kbps, which is about 9.6% of the full mux. The overhead of MPEG-TS will get you close to the full mux rate, and the nulls will pad you up to the full rate. I'd say this example is fairly aggressive with usage of the available bitrate. Your example is obviously from a much more conservative deployment. There may be additional bitrate consumed by error correction with things like DVB-T, but I'm really not sure. It's ALWAYS better to err on the side of caution than it is to accidentally cause broadast issues! Derek
__________________
These are all my personal statements, not those of my employer :) |
![]() |
![]() |
![]() |
#9 | Link |
Registered User
Join Date: Mar 2009
Location: Germany
Posts: 5,773
|
Not all MPEG-PS streams carry a "good load" of NULLs.
DVB-T has in Europe typically 2-3Mbps per video PID. Which is at most half of what a DVB-S stream usually carries (apart from teleshopping channels, but who records teleshopping?). A full movie (90min) can be broadcast in around or less than 2GB. Because of this lower bitrate post-processing must be performed, in particular deblocking.
__________________
Born in the USB (not USA) |
![]() |
![]() |
![]() |
#10 | Link | ||
Registered User
Join Date: Mar 2005
Location: Finland
Posts: 2,641
|
Quote:
Quote:
It's also possible that direct input of interlaced H.264 video doesn't work properly with FFMS2. It didn't for me, and I had to encode the video losslessly to FFV1 with ffmpeg and then open it in AviSynth with FFMS2. Yep, that reconstruction script by Dark Shikari is pretty slow. I used this script. Just a quick try, not optimal: Code:
LoadPlugin("ffms2.dll") LoadPlugin("mvtools2.dll") LoadPlugin("removegrainbeta\RemoveGrainSSE2.dll") LoadPlugin("mt_masktools-25.dll") LoadPlugin("yadifmod.dll") LoadPlugin("nnedi3.dll") LoadPlugin("RepairSSE2.dll") LoadPlugin("deblock.dll") LoadPlugin("dctfilter.dll") LoadPlugin("dfttest.dll") LoadPlugin("DePan.dll") LoadPlugin("DePanEstimate.dll") LoadPlugin("Deen.dll") LoadPlugin("aWarpSharp.dll") LoadPlugin("fft3dfilter.dll") import("Srestore.avsi") import("deblock_qed.avsi") import("dehalo_alpha_mt.avsi") import("fastlinedarkenmod.avsi") setmemorymax(1024) # FFVideoSource("input.mkv",seekmode=0,threads=1,fpsnum=25,fpsden=1) FFVideoSource("input.avi") AssumeFPS(25) AssumeTFF() # Remember to crop vertically by mod 4 amounts when dealing with interlaced video crop(12,4,696,572) # Bob and deblend yadifmod(mode=1,edeint=nnedi3(field=-2)) srestore() # Stabilize clp=last temp = clp.TemporalSoften(7,255,255,25,2) inter = Interleave(temp.Repair(clp.TemporalSoften(1,255,255,25,2)),clp) mdata = DePanEstimate(inter,range=1,trust=0,dxmax=16,dymax=16) DePan(inter,data=mdata,offset=-1,mirror=15) SelectEvery(2,0) # Remove dust and noise o=last osup = o.MSuper(pel=2,sharp=2) bv2 = MAnalyse(osup, isb=true, delta=2, blksize=8, overlap=4, search=4) bv1 = MAnalyse(osup, isb=true, delta=1, blksize=8, overlap=4, search=4) fv1 = MAnalyse(osup, isb=false, delta=1, blksize=8, overlap=4, search=4) fv2 = MAnalyse(osup, isb=false, delta=2, blksize=8, overlap=4, search=4) bc1 = o.MCompensate(osup, bv1) fc1 = o.MCompensate(osup, fv1) Interleave(fc1,o,bc1) Clense(reduceflicker=false) SelectEvery(3,1) MDegrain2(osup, bv1, fv1, bv2, fv2, thSAD=400) # Dark Shikari's anime reconstruction script dfttest() FastLineDarkenMod() deen("a3d",4,8,9) Spline36Resize(last.width*2,last.height*2) AddBorders(4, 0, 4, 0) aWarpSharp(depth=12,blurlevel=4,thresh=0.2,cm=1) FastLineDarkenMod() aWarpSharp(depth=6,blurlevel=4,thresh=0.7,cm=1) Crop(4,0,-4,0) FastLineDarkenMod() dehalo_alpha_mt() fft3dfilter(bt=3,sigma=8,sharpen=1.3) Spline36Resize(last.width/2,last.height/2) |
||
![]() |
![]() |
![]() |
#12 | Link |
Derek Prestegard IRL
![]() Join Date: Nov 2003
Location: Los Angeles
Posts: 5,998
|
That makes good sense! It's funny, I was just reading ATSC specs the other day for fun, and came to the same exact conclusion regarding where error correction happens.
__________________
These are all my personal statements, not those of my employer :) |
![]() |
![]() |
![]() |
#13 | Link | |
Registered User
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 3,113
|
Quote:
I usually reencode my DVB-T captures to remove the station logo. So far I have never used any deblocker mainly because I still watch my movies on a CRT TV which is very forgiving when it comes to blocking. Recently I watched one of my reencoded captures at a friend's house on an LCD, and I was quite shocked about the very visible blocking. So it might be a good idea to add deblocking routinely. But since I cannot judge the result on my CRT I need some recommendations. The settings should be a good compromise between enough deblocking and not destroying all the details. And it should be fast. Deblock_QED would probably be too slow for me. MPEG2Source ("", cpu=4) or Deblock(30) sound reasonable. Some people recommend the MSU Smart Deblocker, but I think this would also be too slow. Thanks manolito |
|
![]() |
![]() |
![]() |
#14 | Link |
Registered User
Join Date: Mar 2005
Location: Finland
Posts: 2,641
|
Shouldn't be too slow for reencoding. With a decent CPU (2.4 GHz Core 2 or better) it runs easily in real time for SD videos, in a single thread.
It's the only deblocking filter I've found satisfactory for MPEG-2 DVB sources. |
![]() |
![]() |
![]() |
#15 | Link |
Apprentice
Join Date: Apr 2012
Location: Poland
Posts: 6
|
@nm - I'd like to ask you another question. You've mentioned that ffms2 doesn't like interlaced H.264 (it's even mentioned in the readme). Is the symptom similar to video going back like 0.2s every second or two?
I've made simple recompression to FFV1 with ffmpeg (with vcodec ffv1 parameter). Is it enough or should I add something else? BTW - nonOCed i7-2600k reaches as much as 1.8fps with this heavy script (and CPU utilization is 40% max) ![]() |
![]() |
![]() |
![]() |
#16 | Link | ||
Registered User
Join Date: Mar 2005
Location: Finland
Posts: 2,641
|
Quote:
Quote:
|
||
![]() |
![]() |
![]() |
#18 | Link |
Apprentice
Join Date: Apr 2012
Location: Poland
Posts: 6
|
Sorry for dragging it for so long but there's another problem.
After some time the resolution of the source material has changed to something strange... 1047x576 (something which isn't even a 16:9 aspect ratio). Do I have to crop in the H and V part (and get rid of the vertical part of the image) or can I remove the black borders to make it 4/3 somehow. I've tried to calculate the aspect ratio of 1.333333333333 and came with something like 768x576. Sample: ![]() |
![]() |
![]() |
![]() |
#19 | Link | |
Registered User
Join Date: Mar 2005
Location: Finland
Posts: 2,641
|
Quote:
I'd keep the SAR and crop the black borders of 4:3 content that was encoded anamorphically. If the resulting non-standard frame size is a problem, resize to 720x576 and change the SAR to either 16:11 or 64:45. |
|
![]() |
![]() |
![]() |
Thread Tools | Search this Thread |
Display Modes | |
|
|