DTV tools -- .TS -> AVISynth?

[jrmann1999]

No offense, but your first Questions screams that you don't know DVD or HD specs.

1. DVDs are encoded at a maximum resolution of 720x480p for NTSC(720x576p for PAL). They are also capped at a max bitrate right aroud 10mbit. HDTV on the other hand can range from 1280x720p to 1920x1080i resolution, and up to 19mbit bitrate. This is what accounts for the large size of the files.
2. Not true, using avisynth and either CCE or QuEnc you can encode HDTV directly to mpeg2 compliant dvd streams.
3. No, your pc is having to decode each frame you move into in order to display it on your screen, very intensive so it will not speed up. There's no real magic bullet on this one.
4. You quite simply do a 1-2s cut of an existing HD program that has the audio in the format you want. You then put that in your list as the first file loaded.

[Karl Beem]

Using the techniques described in this thread, I've processed the 720p broadcast of the Duke - NCS game on ESPNHD with good results. The .ts file was captured with CAPDVHS thru the 1394 from my Comcast(Motorola) 6208 DVR. The Duke - Miami game was also on ESPNHD, but was not HDTV - it had those ugly vertical bars. The actual game looked pretty good, not HDTV but better than analog, so I recorded it and am processing it.

1) Is this what they call SD programming?

2) With the bars cropped out, it is 932x720. Should I resize it to 720x480?

3) would you recommend that I use SelectEven()?

[nnigam]

It depends on what you want to do with it. If you want to burn to dvd to watch on your dvd player, you need to have it at 720x480 resolution at 29.97 fps. If you want to create an avi, use whatever resolution you want to watch it at. I generally resize mine to 768x432 which works very well for me.

As for select even, this is only for reducing the frame rate on video whose frame rate has been increased from 29.97 to 59.94. If you open the file in virtualdub, and scroll through the videw and you see that each frame is duplicated, then you can use this. However, I use the decomb filter decimate(cycle=2) which I think is better to reduce 59.94 to 29.97fps. Better still is the fdecimate filter which will remove all duplicated frames reducing the size of the video considerably.

As for the SD programming, I think this means the regular analog broadcast. You are most likely getting HD broadcast. You are getting this if either you get an almost perfect picture, or no picture. If you can get weak signals, snow etc, then you have SD programming. HD can be either 3:4 or 9:16 aspect ratio, and black bars are used to fit one aspect ration video so that it is visible in a tv in the other aspect ratio.

Hope this helps. I do not consider myself an expert, but I have been playing around with this for the last month or so trying to get everything working perfectly.

[FreQi]

@nnigam
Your answer for 2) and 3) are great, but your definition of SD is a bit off, at least as I understand it.

@Karl Beem
The term "SD" is loosely used, at least in here, to describe "Standard Definition" television, and like "HD", it refers to the broadcast format, not necessarily the material that is being broadcast. So by nature, SD is a 4:3 broadcast and a relatively low resolution (think 704 interlaced vertical lines by 480 columns). Then "HD" as you certainly know is "High Definition" and it comes in a variety of flavors (most notably 1080i and 720p), but they are all 16:9 wide-screen broadcasts. If it's not native 16:9 is it not HD.

There is another acronym that people like us use, "PD" which is sort of a hybrid of the terminology. It simply means "Pure Digital" and is by no means an industry standard term. To understand what "PD" refers to lends itself to understanding how a station broadcasts it's programming, and there are a couple different ways.

(note: I am pulling this all completely from memory based on stuff I've read and my own observations and "interviews" with my network affiliates, so if I am mistaken, please feel free to chime in)

- HDTV programming is sent out from the network headquarters (meaning ABC, NBC, CBS, etc) to some satellites in space and is then beamed back down to the earth on encrypted C-Band. The network affiliates (the local tv stations in your and my town) then pick up that satellite broadcast then re-broadcast it over their dTV equipment in real time with the network HQ's broadcast. They broadcast "dead air" or blackness at commercial breaks which the affiliates overlay with their pre-recorded tapes with commercials already inserted (see SDTV)

- SDTV programming is traditionally also sent out from the network HQ's to some satellites C-Band for the affiliates to pick up. However, this programming is -not- re-broadcasted in real time. Instead, the networks transmit the programming well before the scheduled airing of the show, sometimes more than 24 hours before. (google "Wildfeed" for more). This gives the affiliates opportunity to record the show, typically on VHS/Beta tapes, then insert their commercials (typically by re-dubbing the show to another tape) and then simply play the cassette when the shows time slot comes up. This process of recording and copying makes most SDTV look like garbage, but most people had crappy reception anyway, so who knew.

- PDTV comes in somewhere in the middle, and has several variants that can define it. Live broadcasts, like sports events and the local news are sent straight from the cameras through the cloud of equipment and over the antennas to your receiver. This lack of post processing tends to deliver a far superior picture and you can just see it. The main idea is the picture never flips to analog.

Digital Satellite TV is usually considered PDTV, HBO on Digital Cable/Satellite is PD, as are most of the subscription only channels like SciFi, Discovery and History channel.

Another example of PDTV is Scrubs on NBC. For whatever reason, they don't film that series in HD, but NBC still sends a live feed from their HQ to the affiliates who rebroadcast it just like the HD programming. The result is a very clear picture with sidebars.

Most SD programming of sporting events are actually cropped versions of the HD broadcast (remember, it's not PD because it was broadcasted in analog form, even on your "Digital Cable"). At the next football game, watch the same network on your SD and HD tv's. You'll see the same picture, only the HD one has more on the sides. This explains why on HD the score and time banners at the top are sort of scrunched in the middle. They just lop off the sides, down sample it and send it off as the 4:3 signal.

For some real specific descriptions, you might want to check out the online dictionary at AudioVideo101 (http://www.audiovideo101.com/dictionary/). They don't have anything on PDTV, but remember that's nothing official. Just a made up term to describe a phenomenon.

[venutolo]

I'm currently working on creating a DVD from 720p source. I ran the stream through DGIndex, giving me a .d2v project file. Then I wrote an avs script, and opened it in CCE and created a 720x480 23.97fps video file, which was then used to author a DVD. When I play the DVD the video isn't smooth like the source. I imagine this has something to do with going the 59.940fps source to the 23.97fps final video, and that it is a problem with my avs script. Is there something I'm missing, or doing in the wrong order?

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\DGDecode.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Decomb521.dll")
MPEG2Source("K:\HDTV\24S04E01\24S04E01.d2v")
SelectEven()
FDecimate()
LanczosResize(720,480)

Any help is appreciated.

[Guest]

Take out the SelectEven().

[venutolo]

Originally posted by neuron2
Take out the SelectEven().
That was my first guess. Much thanks.

[FreQi]

If you're going to make an NTSC DVDr, you should also run pulldown.exe on the 23.976fps .mpv you made. It makes the video think it runs at 29.97fps through mpeg flag trickery. You'll also want to tell it to flag it as progressive scan and enable drop frame (keeps the clock in sync).

I recommend you use bbMpeg to mux the ac3 and the m2v into an mpg, if that's what you plan on ending up with

[venutolo]

Originally posted by FreQi
If you're going to make an NTSC DVDr, you should also run pulldown.exe on the 23.976fps .mpv you made. It makes the video think it runs at 29.97fps through mpeg flag trickery. You'll also want to tell it to flag it as progressive scan and enable drop frame (keeps the clock in sync).
As far as I can tell, CCE takes care of pulldown. As for flagging it as progressive scan, does checking the box next to "Progressive Frame" take care of it, or is it something in authoring the DVD? Finally, I have no idea how to enable drop frame, so could you please point me in the right direction? I have had some experience working with CCE, but never with a 720p source, and this is the first time I'm authoring a DVD from scratch, so all of this is a learning experience. Thanks for the help.

Below are screenshots of my CCE template's settings.
http://img140.exs.cx/img140/3326/24cce11dv.th.png (http://img140.exs.cx/my.php?loc=img140&image=24cce11dv.png)http://img140.exs.cx/img140/4308/24cce28eq.th.png (http://img140.exs.cx/my.php?loc=img140&image=24cce28eq.png)http://img140.exs.cx/img140/5186/24cce33hy.th.png (http://img140.exs.cx/my.php?loc=img140&image=24cce33hy.png)

[nnigam]

Originally posted by FreQi
If you're going to make an NTSC DVDr, you should also run pulldown.exe on the 23.976fps .mpv you made. It makes the video think it runs at 29.97fps through mpeg flag trickery. You'll also want to tell it to flag it as progressive scan and enable drop frame (keeps the clock in sync).

I recommend you use bbMpeg to mux the ac3 and the m2v into an mpg, if that's what you plan on ending up with

----------------------------------------------------------

I just convert my 59.94 fps progressive to 29.97 by decimate(cycle=2), and leave the extra frame there. How is bbMpeg, I use Quenc, but will certainly give both pulldown and bbmpeg a try.

Would the avisynth cmd changefps be similar to pulldown since it just slows the video down.

[FreQi]

venutolo writes
As far as I can tell, CCE takes care of pulldown.
Yeah, CCE does have an option for pulldown, but I've read about issues with it enough to decide not to use it. I don't really have any examples, so take my advise for what it's worth, but PULLDOWN by HardCode (http://www.inwards.com/inwards/?id=36) does more than just set some 2:3 flags, so even if you did use CCE to set pulldown, I'd still recommend you run it through pulldown.exe for these two features:

1) Enabling Drop Frame
This has to do with the way the mpeg counts time in relation to frames. When you encode the video at 23.96 and make it play back as if it were 29.97, the mpeg's "natural seconds" appear to play longer than a "real" second. This results in the video's duration appear to be shorter than it actually is... Now, when I first read about this, I thought the flag would cause the video to actually skip playing a frame or something, but that's not the case. This guys dissertation (http://www.philrees.co.uk/articles/timecode.htm#drop) explains it rather well: "It is not so much frames which are dropped as frame numbers which are skipped." In the end, if you do not use the Drop Frame Flag in your .m2v, you could have a real hard time setting chapters accurately or getting subtitles to sync right.

2) Setting all Frames to Progressive
Now, I don't really know how beneficial this is, but here's my line of thought... The video -is- encoded with progressive frames. The 2:3 pulldown flags makes it appear to be 29.97fps, which is typically an Interlaced video. So in truth, a 29.97fps Progressive scan video would actually contain duplicate frames. By setting this flag, you're telling the players not to deinterlace when decoding and just show the frame. Progressive scan enabled dvd players should love that. Now, I've tried using the other prog_seq switch that marks the entire stream as progressive but found playback to be...problematic. Xbox Media Center hated it anyway, so marking the frames progressive and not the stream seemed to be the way to go.

So, how do you apply pulldown? I made a .bat file (see below) and made a shortcut to it on my desktop. When CCE has finished making it's .mpv I just drag and drop it on the shortcut and this .bat makes an .m2v for me. So for example, I encode lost-s1e14.mpv and drop it on the shortcut and it spits out lost-s1e14_PULLED.m2v for me to mux with the ac3.

@echo off
set INFILE=%1
set OUTFILE=%INFILE:~0,-4%_PULLED.m2v
PULLDOWN.EXE %INFILE% %OUTFILE% -drop_frame true -prog_frames p
pause

WoofSoft built a GUI (http://www.woofsoft.com/Downloads/53.aspx) out of the source code from this command line app and called it DoPulldown v1.0.0.0, but I prefer to drag and drop my .mpv on the shortcut. Old habits die hard, but it'd be easier if DoPulldown supported Drag-n-Drop (@WoofSoft: hint hint)

venutolo writes
As for flagging it as progressive scan, does checking the box next to "Progressive Frame" take care of it
Checking the "Progressive Frame" box tells CCE to encode the stream as progressive, and yes I suspect it does the same thing as pulldown.exe's -prog_frames p switch, but I'm just guessing on that...

nnigam writes:
I just convert my 59.94 fps progressive to 29.97 by decimate(cycle=2), and leave the extra frame there.
But just to be clear on why using 2:3 pulldown is a good idea... Remember how Telecine works? 24fps (what tv shows are actually filmed at) gets to 30fps by using duplicates of previous frames. The Pulldown flags tells it to simply show the previous frame, rather than having an actual copy of it in the stream. This flag trick allows you to allocate the bits that would be needed to show approximately 6 frames and spread it around the other 24.

And incidentally, decimate(cycle=2) is probably a lot slower than just using SelectEven() or SelectOdd() which would yield (probably) the same result.

nnigam writes:
Would the avisynth cmd changefps be similar to pulldown since it just slows the video down.
Absolutely not the same thing. ChangeFPS() actually changes the framerate by simply duplicating or deleting frames. Pulldown tells the software the decodes the stream to show a frame twice, with ChangeFPS, the duplicate frame would actually be there. This is also not the same thing as the ever popular Decimate() function from the Decomb plugin. Where ChangeFPS just drops or copies any frame as it needs to, Decimate actually compares frames to each other to discover which is a duplicate and drops one of those.

nnigam writes:
How is bbMpeg
Honestly, it does a lot more than I need, and it's a little convoluted to use just for muxing, but it's done a great job so I stick with it.

You can pick it up here (http://members.cox.net/beyeler/bbmpeg.html). You have to open avi2mpg2.exe which opens this interface for...honestly, I have no idea. But you have to click the "Load" button as if you wanted to load a project file and click Cancel (apparently there is a known bug that this fixes, *shrug*). Then you click the "Start Encoding" button, even though you are not encoding anything, and this opens the bbMPEG window. At this point you can click the "Settings" button and go to the "Program Stream Settings" tab and tick the dot for ~DVD. The defaults should be fine, then go to the "Input and Output files" tab to select the target .mpg and your source .m2v and .ac3 files. Click OK, then Start and it'll mux it up for you.

What can you do with the final .mpg? I've found InterVideo WinDVD to be a great app for playing the video, that or Xbox Media Center. You can also use the .mpg to author a DVD using TMPGEnc DVD Author.

[nnigam]

Thanks FreQi. Very informatinve. My project for the weekend is set. I am going to try all of this on my videos.

As for changefps, you are right. Thats what happens when I try to write from memory. I meant assumefps. I stopped using it because it changed the video rate so that audio and video no longer were in synch. I guess that changefps was also incorrect.

[MarkGrigsby]

I am using this method to convert 720p material to DVD, and am not sure why I need to use 'Telecide()' for 720p material? Surely I could just use FDecimate to get the frame rate down to 24, since there are no interlaced frames present?

Or am I missing something?!

[Guest]

That's right. If there are no combed frames, Telecide is not required.

[FreQi]

You're right, Telecide() is not used on 720p (it's an option for 1080i) but I use Decimate(cycle=5) because I first use SelectEven() which takes it down to a progressive 29.97fps. My scripts for 720p all look something like this:

LoadPlugin("C:\Program Files\theeo\DGMpegDec\DGDecode.dll")
LoadPlugin("C:\Program Files\theeo\AVISynth\Decomb.dll")
LoadPlugin("C:\Program Files\theeo\AVISynth\BT709ToBT601.dll")
MPEG2Source("F:\According.to.Jim.s4e23\atj-s4e23.d2v")

# correct the slight color error SDTV->HDTV introduces
BT709ToBT601()

theeoAddRange(302,9931)++theeoAddRange(18822,22801)++theeoAddRange(53890,21667)

# InverseTelecine and Decimation
SelectEven().Decimate(cycle=5)

Crop(4,4,-4,-4) # take 4 pixels of left, top, right, bottom

LanczosResize(720,480).ConvertToYUY2().AudioDub(BlankClip())

function theeoAddRange(clip c, int s, int r) { return c.Trim(s,s+r-1) }

[Wilbert]

@FreQi,

Not all HDTV streams use Rec.709, contrary to what the readme of BT709ToBT601 claims.

[FreQi]

How do you determine when it needs correction? Is there some way to detect it in software, or is it just visually apparent? Perhaps there is a way to determine it logically.

For example, when something is transferred from Film to HD, it would need correction...? I believe that would be the case with just about all Drama's and Sitcoms.

When something is filmed with an HD Camera, then it would not...? The only thing I can think of here would be things like the specials you'd see on PBS, NASCAR races, the Superbowl and other live broadcast events. Incidentilly, these types of HD streams would probably not take to well to being IVTC'd either.

[Wilbert]

Just open the stream in GSpot and see what coefficients are being used. If it indicates nothing then Rec.709 coefficients are being used since that's the default. See also readme of ColorMatrix.

[nnigam]

OK. I have learned a lot here. A brief delay while I had to remove a very persistent virus on my pc that just refused to go away, but now I am back in business.

I process my 720p using dgindex, convert to 23.976 using fdecimate since they happen to be 3:2 format. I convert to avi using vdubmod and everything is in perfect synch. I convert to m2v using quenc and then run dgpulldown to convert to 29.97, and covert to dvd using muxman. Again everything is in perfect synch. I then open the puldown m2v file and the ac3 audio in Mpeg2Schnitt to cut the commercials. First problem is that there is no place to put the audio delay. Even in Mpeg2Schnitt I can see audio synch errors greater than would be caused by the delay alone. I tried the original 23.976 m2v, and I still have audio errors. I then tried delaycuter utility on the audio with both the original and pulldown m2v and I still have audio synch errors. Pulling the audio into gspot, It says the audio frame rate is 31.5fps. Somehow this synchs with the 23.976 or 29.97 pulldown audio and sitll maintain perfect audio synch while cutting causes audio errors.

I read somewhere of a utility that will take the cut list from Mpeg2Schnitt and perform the same cuts in the audio seperately. I will look for that and try again.

[FreQi]

Cut the commercials -before- you encode. If for no other reason, it's the only way to accuratly target any final file size.

[nnigam]

I would love to. How to cut an HD transport stream. I process the hd stream to a d2v project. Open the d2v in an avsynth script. Cutter tools need an m2v which I get only after processing with quenc. I tried cutting in vdubmod, but it takes forever.

In your sample, I saw

theeoAddRange(302,9931)++theeoAddRange(18822,22801)++theeoAddRange(53890,21667)

How did you get these ranges? Any limit to the number of ranges I can add here? How is the range affected by the fps changes needed to convert the 3:2 720p to 23.976?

Thanks

[FreQi]

Check out my second post on page 2 (http://forum.doom9.org/showthread.php?s=&threadid=49349&perpage=20&pagenumber=2) of this thread. It's explains pretty much what I still do today, except I wrote an app to automate the whole process (called theeo, which still needs a little tweaking before I publicly release it).

Keep in mind that in order to cut the AC3 with VirtualDubMod, you have to use v1.4.13 (http://sourceforge.net/project/showfiles.php?group_id=65889&package_id=63653&release_id=148604). The newer 1.5x doesn't cut the ac3 stream when you demux it with trim's. Not sure why.

Also, rather than using BeSplit, I'd suggest you use DelayCut (http://www.videohelp.com/~jsoto/audiotools.htm) to Fix the trimmed ac3. It seems to do a better job of maintaining sync.

Any limit to the number of ranges I can add here?
nope.

How is the range affected by the fps changes needed to convert the 3:2 720p to 23.976?

If you do your trims -before- you IVTC, you're unaffeted. You can try any sort of IVTC method and never have to cut commercials again. Just do it in the order I have my last example in. Load the .dll's, load the .d2v, trim the commercials, then process the video.

[squid_80]

I read somewhere of a utility that will take the cut list from Mpeg2Schnitt and perform the same cuts in the audio seperately. I will look for that and try again.

Mpeg2Schnitt is designed to cut the audio as well. If the audio file has the same name as the video (except for extension) it should be loaded automatically. Doesn't it do this for you?

[Bruce Willis]

Interesting thread..

Is it true that converting the framerate from 60 to 24 fps, either by SelectEven().Decimate or by FDecimate, slows down the encoding 4 times or even more? Or am I doing something wrong?

[nnigam]

I never timed it. It takes a long time regardless. I am processing 1 hour hd programmming. Taking out commercials, it comes to about 45 minutes. Encoding this takes about 8 hours. Three hours for the first pass and 5 for the second. I would assume FDecimate would be slower because it checks all frames to find best frames to drop, while SelectEven.Decimats() does not have to be too intelligent, but I could be wrong. I just process overnight and so do not care how long it takes.

[Guest]

Decimate() also compares frames.

If you want a fast but dumb decimation, use ChangeFPS(). You can compare that to FDecimate() to get an idea of the time difference. I doubt it is as bad as you suggest.

The advantage of FDecimate() is that it preferentially drops duplicates where possible.

[FreQi]

Yeah ChangeFPS would probably be faster but I really wouldn't trust it to make a good looking video (it would probably jitter pretty regularly). Decimation is the way to go, dispite the potential speed hit.

I've got a P4 2.8 and I find encoding times to be about 4x the runtime. So if you have a 30min show, it takes me about 2hrs to encode. That's a really rough estimate... I don't know the real times off hand as I also set up my encodes and go to sleep. They're all done bu the time I wake up.

[Leehro]

learned some great stuff here, thanks for all the info. I'm still having some sync issues but I think i'm on the right track.

Most of my concern though is for converting interlaced HD content (like leno/conan/letterman and live music on pbs) down to DVD.

does anyone have a good script for those? i end up playing with the settings a lot. this was one of my scripts for an Austin City Limits performance (which was upconverted and, broadcast 1080i widescreen)

LoadPlugin("C:\progra~1\gordia~1\MPEG2Dec3.dll")
mpeg2source("P:\HDTV\pixies.d2v")
crop(16,0,-16,-8)
SeparateFields
SeparateFields
Lanczos4Resize(720,240)
SelectEvery(4,0,2)
doubleWeave
selectodd



just seems like i'm missing something and going overboard. but it worked and looked fine on TV.

I then encoded that with CCE and authored a 16x9 DVD, with original AC3 audio. Didn't have any sync issues because I didn't cut out any commercials. But I tried a lot of the info here on SNL from a couple weeks ago, and best I got was bad field ordering and a constant sync offset (improvement over a sliding one).

[FreQi]

The live programming that you're talking about is filmed natively in NTSC, which means you do not want to decimate it. If you do you'll get jitter and general baddness. I'd try something like this:


LoadPlugin("C:\progra~1\gordia~1\MPEG2Dec3.dll")
mpeg2source("P:\HDTV\pixies.d2v")
SeparateFields()
SelectEven()
Crop(16,0,-16,-8)
Lanczos4Resize(720,240)


This should give you a progressive scan 29.97fps video at 720x480, so you won't need to use pulldown.exe for 2:3 pulldown or drop-fram flags, but you'll probably still want to set the pregressive flag.

You could use Decomb.dll to run Telecide() in order to deinterlace it (do not decimate), but you'd be spending more processing time for something that's going to be less than what you get from simply SeparateFileds().SelectEven() anyway. It might look better tho. Try both, see what you prefer. LMK.

[Leehro]

yeah it's NTSC stuff like letterman, SNL. stuff that is natively 480i (probably slightly higher than 480 but still standard def, interlaced video)

i don't want progressive video. I want to keep it interlaced. Sometimes i want to convert to divx/xvid, and in that case yeah i want progressive.

but if it's originally standard def, interlaced video at 29.97fps, that's what I want to be able to put on a DVD.

[dvgeek]

This has been a very enlightening thread. I currently use the modified ProjectX for 59.94fps, which seemed to fix my sync issues. (My caps are mostly ABC-Alias and Fox-24).

I demux in ProjectX, encode in WMV9 single pass (for a faster processing - 2hrs HD in about 6 hours, XVID - 2pass takes about 24 hours for the same avs script, which is just a one-liner for loading the .d2v).

I use VirtualDubMod for encoding with Fast Recompress - no audio.

I addin the ac3 file after encoding - edit out the commercials and save the avi with a direct stream copy. I always have audio interleave at 32ms.

Does anyone know if the latest ProjectX releases have got the 59.94fps mod incorporated ? Thanks.

[FreQi]

Seriously, you should not be cutting commercials after you encode. You're easily encoding 17 to 20 minuets of material that you are going to throw away. Cutting commercials first will shave 30% or more off of your current encode times.

Also, for shows like 24 and Alias, you should also be Decimating it to 23.96fps. The show is origiannly filmed in 24fps, so you are not loosing anything. You only gain more bits to allocate to the frames and ending up with a higher quality encode.

[dvgeek]

Thanks for the suggestion. I've had so many audio sync problems when I edited out commercials with HDTV2MPEG that now I've given up on that tool. It's a great tool - fast and all - but sadly for me, I could never get audio sync - no matter what method I tried.

So, now I clean up the transport stream by demuxing with ProjectX, then use DGIndex to create the D2V file and use the latest version of AVISynth/VirtualDub to encode into WMV9 HD. Then I use VirtualDubMod to mux in the AC3 audio and cut out the commercials - I do not mind the extra time as it gives me audio in sync.

Also - if I use fdecimate() or any other form to decimate to 23.976fps, my video gets choppy as if it suddenly decides to switch field priority. So what if my video is 10% more - I get smooth motion by keeping the original frame rate. I've all but given up on decimating to 23.976fps or do edits before encoding. I'll keep experimenting though. Thanks again.

[Leehro]

actually your decimate problem could be because of commercials. if you're encoding with commercials (often interlaced content, or at least a different field/frame offset) that can screw with decimate.

i had some sync problems too, but i eventually used VDubMod to make all the edits, save the frame ranges into avisynth, and fix the chopped up ac3. after that i think it was just a constant audio offset. i'm not using hd2mpeg anymore at all. just dgindex, vdumbod, avisynth, and whatever the latest greatest ac3 cut fixer is.

edit: still looking for anyone with a good script for 1080i->480i

[nnigam]

Also - if I use fdecimate() or any other form to decimate to 23.976fps, my video gets choppy as if it suddenly decides to switch field priority..

Is your original source at 23.97fps. Before using Fdecimate, use Fdecimate(metrics=true) in the avisynth script. Then curser through your video. You will see a large change in the number when a new frame is found instead of a duplicate frame. You can use the largest of the small numbers as the threshold for fdecimate, and see how many actual distinct frames you have to get what the actual frame rate should be. Most of my hd stuff is 23.976 fps which has been converted to 29.97 by adding duplicating one frame, and then every other frame is duplicated. I also have some which are at 29.97 and only every other frame is duplicated. If this is converted to 23.976 it will be choppy.

I also agree with you that a little larger video to get better audio synch and smoother picture is worth it. Many times

[Ac3Dc3]

This has got to be one of the best threads on converting HDTV out there, thanks for all the info!

does anyone know where i can get FreQi's tool theeo? and is it up to date with the current version of DGIndex?

[FreQi]

Interesting that you ask about theeo. I was working on it last night for the first time in maybe 6 months. It is true that theeo exists, and it does work, pretty well. However, when DGIndex stopped putting AC3 in the name of the files it demuxed, it threw a wrench in there that I've been just working around ever since. I finally got tired of it and decided to fix it, but instead of fixing it, I redesigned the interface, and pretty much started over. I am currently trying to decide if I want to actually start completly over and use Visual Studio 2005 and .NET2.

If you're interested, I could get you up and running with what I'm calling "theeo-v0.00" as I rework the tool. In any case, I would like to get your feedback on it. Send me a PM.

[FreQi]

I've just posted the first release of theeo. I'd be interested in getting your feedback in that thread (http://forum.doom9.org/showthread.php?p=789248)

[coolyhu2]

Thx


~~

[G_M_C]

Hmmm, reading throught this thread becaus this week im gonna try my first .TS to XviD.

As I understand it it's only needed to use HDTV2Mpeg2, and than DVD2AVI to make it possible to open the stream in avisynth through the Mpeg2source() command ?

And if true does it really need to be (the special version of) DVD2AVI ? Or can DGindex (DGMPGDec) do the same thing ? This because this thread (http://forum.doom9.org/showthread.php?t=87809) says DVD2AVI has to be considerated "obsolete" (because it makes errors). And I've allready got it installed (and having several different versions of MPEG2DECx.dll is a problem waiting to happen). And if DGIndex can work, does the "hints" switch work too (for use with Colormatrix) ?

If I cant use my allready installed DGIndex/DGMPGDec, does the "hints-switch"(for the use with Colormatrix) work with that special version of DVD2AVI ?

I want to have this info, so i cam make my own AviSynth script, so the only thing I do need to be shure of is the bvest way to open the .TS in the script :)

[squid_80]

DGIndex can handle unprocessed .TS streams ok, as long as there's no corruption (common with DVB-T, I don't have experience with anything else).

[FreQi]

There are of course several ways to get your .TS's into an .AVS script, but the "best" way is purely subjective. Some people use HDTVtoMPEG2 to do their trimming, others use ProjectX. In either case, I would suggest that you use DGIndex to generate the .D2V that you load in your .AVS since it is true that DVD2AVI has been pretty much abandonded in favor of DGIndex.

The way I process the .ts's has changed in the recent past. I used to use just DGIndex then cut the video in AVISynth and the Audio with VDMod. This was perfect in most cases, but would ocassionaly result in desync'd audio, especially if there were problems with the recording (drops, etc). It seems the more predictable method that I have been using is to use ProjectX to trim the unwanted segments then demux the .TS to it's MPV and .AC3 elements. Because ProjectX "fixes" drops and maintains A/V sync, you never have to worry about things like AC3 Delay anymore because it's allways 0ms. Then I use DGIndex to generate a .D2V to open then .MPV with AVISynth for trans-encoding since DGDecode is faster than other mpeg2source() plugins. The main downside to this process is the near doubling of hard drive space required to do the work, but it's a small trade-off for the reliability of the end product.

[G_M_C]

Well thx guys :thanks:

All the answers I needed. So hope to be able to give it a try this weekend :)

[jmac698]

I record the .ts, then use mpg2cut2 to take out commercials. This app is very fast as it simply does a binary copy of the remaining parts of the file, there is no processing at all. The only thing fancy about this file splitter is that it understands mpeg2 enough to find the exact byte to cut the file on. This byte is called a "GOP boundary", roughly speaking. A GOP is a set of, say, 15 pictures that can be independently decoded from any other part of the file, so you can call it one "atom" of film.
This should tell you the disadvantage of this method - you can only cut to the nearest 15 frames or 1/4 of a second, or so. This is usually good enough to cut perfectly.
The advantage is that there is no change in sync, because nothing in the file has changed, in a sense. It's also as fast as your hard drives. (hint - save to a different hard drive).
After this, just load in StreamClip and Convert to AVI, and your done! For full quality you can also Convert to MPEG2, this saves a bit of space but makes it a more compatible file, an mpeg2 (instead of transport stream).

[FreQi]

I can't offer perspective on a speed comparison, but it sounds like the same thing that HDTVtoMPEG2 (http://www-personal.umich.edu/~balazer/HDTVtoMPEG2/) (forum (http://www.avsforum.com/avs-vb/showthread.php?s=&threadid=395744)) would do. ProjectX (http://www.videohelp.com/tools?tool=ProjectX) also cuts on GOP, but it also corrects errors in the recorded stream, so I count it as a bonus. It'll also generate a .D2V for you, but it doesn't use the fancy format that DGIndex will make, so you have some options there. You can also use pjx to process to a new .ts instead of demuxing if you really wanted to.

[FreQi]

So after a lot of encoding, trying different apps and writing scripts and apps like theeo (http://freqi.net/theeo/) and tons of trial and error, I've managed to come up with what I consider to be a really simple and straight forward way of pre-processing HDTV Transport Streams to the point where you'll have an AVISynth script that you can feed into whatever encoder you want.

How To get Transport Streams into VirtualDubMod via AVISynth using ProjectX and DGIndex

What I have done is created a system that uses ProjectX as the starting point, then uses DGIndex to generate the .d2v project and the AVISynth script. Then, just to take it one step further, I made it open the .avs in VirtualDubMod so I could easily set up the encode jobs. The funny thing is, it's all bundled together by a windows batch script. Here's how...

First, I made a directory structure like this:

C:\Program Files\Encoding
C:\Program Files\Encoding\DGIndex
C:\Program Files\Encoding\VirtualDubMod
C:\Program Files\Encoding\ProjectX

In the DGIndex directory, I put DGindex 1.4.8 (found here (http://neuron2.net/dgmpgdec/dgmpgdec.html)), and two AVISynth Templates which I've listed below (modify them to your needs, but these are what I use and the resolutions I like to encode my XviD's to).

720p-to-FILM.avs

LoadPlugin("C:\Program Files\Encoding\DGIndex\DGDecode.dll")
MPEG2Source("__vid__")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Decomb.dll")
SelectEven().Decimate(cycle=5).Crop(4,4,-4,-4).Lanczos4Resize(960,528)


1080i-to-FILM.avs

LoadPlugin("C:\Program Files\Encoding\DGIndex\DGDecode.dll")
MPEG2Source("__vid__")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Decomb.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Uncomb.dll")
Uncomb().Decimate(cycle=5).Crop(4,4,-4,-8).Lanczos4Resize(960,528)


In the VirtualDubMod directory, I put VirtualDubMod 1.5.10.2 build 2542 (found here (http://sourceforge.net/projects/virtualdubmod)).

In the ProjectX directory, I have a couple things. First, I extracted the "launch.exe" program dvd.matt linked me to (found here (http://forum.dvbtechnics.info/showpost.php?p=15262&postcount=3)). I also created two batch files that reside there.

MakeD2V-720p.cmd and
MakeD2V-1080i.cmd

@echo off
echo.

IF [%1]==[] GOTO NoArgs
set STRIPTIN=%~dp0

:: this is the template DGIndex will use to generate your AVISynth Script.
set AVSTEMPLATE=720p-to-FILM.avs

::the only thing you can pass is a filename, so if there are
::more "args" then assume it's a filename with spaces.
set INFILE=%1
:getname
shift
IF "%1"=="" GOTO gotname
set INFILE=%INFILE% %1
GOTO getname
:gotname
::launch.exe seems to put quotes around the filename
::so trim the double quotes off both ends
set INFILE=%INFILE:~1,-1%
echo. Source File: %INFILE%

:: trim the file extension off the end
set OUTFILE=%INFILE:~0,-4%
echo. Output Base: %OUTFILE%

set DGINDEX="%STRIPTIN%..\DGIndex\DGIndex.exe"
set ARGS=-IF=[%INFILE%] -YR=1 -FO=0 -OM=2 -AT=[%STRIPTIN%..\DGIndex\%AVSTEMPLATE%]-OF=[%OUTFILE%] -exit

echo. Launching DGIndex to make .d2v project file...
echo %DGINDEX% %ARGS%
echo.
%DGINDEX% %ARGS%
echo.

IF NOT EXIST "%OUTFILE%.avs" GOTO NoAVS
IF NOT EXIST "%STRIPTIN%..\VirtualDubMod\VirtualDubMod.exe" GOTO NoVDMod
echo.
echo. AVISynth Script found, opening in VirtualDubMod...
IF EXIST "%OUTFILE%.avs" "%STRIPTIN%..\VirtualDubMod\VirtualDubMod.exe" "%OUTFILE%.avs"
GOTO end

:NoArgs
echo.
echo This script is meant to be run by launch.exe demuxing with ProjectX. To use
echo it, open the PreSettings and on the postprocess tab and put something like...
echo.
echo. "C:\<pathTo>\launch.exe" open "C:\<pathToThisScript>\MakeD2V.cmd" ?1
echo.
echo in the demux line. Also, on the ProcessWindow (click "prepare>>"),
echo make sure you have "enable post processing" checked under the Edit menu.
echo.
pause
GOTO END

:NoAVS
:NoVDMod
:end
::exit


The only difference between the two scripts is the value of AVSTEMPLATE. 720p uses the 720p template, 1080i uses the 1080i template (duh?). The ProjectX directory also contains two copies of ProjectX 0.90.04.00 (found here (http://www.oozoon.de/main_en.html)) that are in their own subdirectories cleverly named \720p and \1080i (see footnote1).

So, I extracted one copy of ProjectX to ProjectX\720p and set it up the way I like (see the "First Time Configuration" section of my guide (http://freqi.net/tutorials/projectx/) for details). In addition to that, I put the following command in the "demux" box under the postprocess tab of the PreSettings:

"C:\Program Files\Encoding\ProjectX\launch.exe" open "C:\Program Files\Encoding\ProjectX\MakeD2V-720p.cmd" ?1

When I was done, I copied the 720p directory to the 1080i directory, but I opened up ProjectX and changed the demux line under the postprocess tab to use the MakeD2V-1080i.cmd script instead (make sense?).

To finish the setup, I just created shortcuts to the ProjectX binary and put them in my quicklaunch for easy access.

So, now when I have a show I want to encode that's been recorded from a 720p source, I just open that flavor of ProjectX, drag-n-drop the source files on it, cut the commercials (as outlined in my guide (http://freqi.net/tutorials/projectx/)) and click the "Quickstart" button. After a couple minutes of processing, DGIndex opens up, makes the d2v and avs script then opens it up in VirtualDubMod for me to double check and setup the encoding job (see footnote2).

Done. Wha'd'ya think? :cool:



footnote1
Why two copies of ProjectX? Because Interlaced video needs to be processed differently than Progressive video, and to make that happen, i needed ProjectX to lauch a different batch script when it's done demuxing. The only way to do that is to either
(a) change the Post Processing command every time I switch from I to P or back,
(b) create different projectX .ini files and load them from command line or in the target properties of a shortcut to the jar (ie: C:\WINDOWS\system32\java.exe -jar "C:\Program Files\Encoding\ProjectX\ProjectX-v0.90.04.00-20060330.jar" -ini "C:\Program Files\Encoding\ProjectX\720p.ini" -gui)
or
(c) you can just make two copies, one with the settings for 720p material and one for 1080i.
It seemed like less work, so I opt'd for c.

footnote2
Instead of opening VirtualDubMod, there is a ton of other things you could make the cmd script do. You could launch x264.exe to encode it, or feed the avs to avs2avi instead. Go nuts.

[Livesms]

I have HDTV (1080p) video (Gladiator) with 2 audio streams (eng and rus) - 21Gb. PID 011 fo video, 014 for rus audio and 015 for eng.
I want to reencode it with x264 to fit 2DVD.
I tried to put it to Avisynth using d2v generated by DGMPGDec 1.4.9 Final but it always says smth like filed order was incorect (generating d2v) and then eng audio stream is shorter then rus (2:34:58 vs. 2:33:51). d2v files generated by DGMPGDec 1.4.9 Final is differs when i generate it using Streams 011, 014 and when i do the same task with 011, 015.

How can I reencode correctly?

[FreQi]

I don't think you need to worry about the runtime for the audio being different by one minute. There's probably just no audio at the end durring the credits or something.

I also think you might want to take your question to a thread about DGIndex. This thread is more about ATSC HDTV Broacast Transport Streams, not HDDVD/Bluray (although you should be able to use much of the info here).

[Livesms]

I don't think you need to worry about the runtime for the audio being different by one minute. There's probably just no audio at the end durring the credits or something.

I also think you might want to take your question to a thread about DGIndex. This thread is more about ATSC HDTV Broacast Transport Streams, not HDDVD/Bluray (although you should be able to use much of the info here).
I checked audio in original ts for both PID - both in eng and rus audio run till the end of the video and ts has length of 2:34:58

And seems like AC3 audio generated be DGindex is not the same audio i hear with original TS file.
I just put the slider into random position (for ex. 0:24:40) and TS audio is far differ (by time I mean) from AC3.

When I use HDTVtoMPEG for converting TS to MPEG i have the same problem - video is trancating to 2:33:51 (from 2:34:58).

[FreQi]

Well, like I said, you'll probably want to take this to a DGIndex thread...