DTV tools -- .TS -> AVISynth?

[FreQi]

Edit: I've written a more current guide later in this thread (link (http://forum.doom9.org/showthread.php?p=976687#post976687))

I finally managed to upgrade my computer, put an antenna on my roof and install my HiPix card. I am currently getting really good signals for ABC, NBC and CBS (strengths exceeding 80) and am able to successfully "record" the transport streams. I've been poking through the forums a bit, but haven't really come up with any straight how-to's for getting a .ts into avisynth so I can make some sexy xvid's. All I know is I need to use some or all of the following:

* HDTV2Mpeg2 v1.07
* DVD2AVI (no idea what version)
* AVISynth 2.5
* MPEG2DEC.DLL (unknown what version)
* Decomb.dll (one that supports avisynth 2.5)

Could some one type up, or point me to, a basic set up guide? I am familiar with doing tv captures, so all I need to know is how to get a .ts into avisynth. I should be able to figure out the deinterlacing and IVTC'ing.

Note: I did find SMC's post on inmatrix (http://www.inmatrix.com/articles/atsc.shtml) but the steps there seem a bit out-dated, particularly since the use of YU12...

[trbarry]

FreQi -

I'll be making up a mini-guide that follows how I've been handling it. I'll post it here but it is still only in the planning stages right now.

But I can summarize a 3 step process:

1) Use HDTV2Mpeg2 (v1.09 or v1.10) just to verify which PID numbers you want, that is, which sub-channel you want from your input stream. You don't actually have to save any files here, just write down the 2 numbers. Optionally you can also use this to cut commercials though I prefer to do it later in VirtualDubMod.

2) Use the save-oe HDTV supporting version of DVD2AVI to make a project file, demux the ac3, and also to decide how you will process the file (which VirtualdubMod template). A test binary of the save-oe version is on my site at www.trbarry.com/DVD2AVIT3.zip. Note you still have to set the PID numbers in the DVD2AVI.ini file for HDTV in this version. Also, you usually can't use Force Film on HDTV and will need Decomb(), TomsMoComp(), or YV12InterlacedReduceBy2() later.

Since the PID numbers are almost always one of the pairs <x11,x14>, <x21,x24>, or <x31,x34> I personally just have 3 folders with 3 copies of DVD2AVI and the ini file and then just start the right one. It's a small program.

3) Then (with Avisynth 2.5 and MPEG2DEC3) use VirtualdubMod to open an HDTV template with the Open Using Avisynth function. I'll post some templates (shell Avisynth scripts) for HDTV with my mini-guide. Or just make and use an Avisynth script. I keep the ac3 so I also load the ac3 file into VdubM here, set the audio delay, cut out commercials if needed, set audio to direct stream copy, and encode. You can also edit the script (Tools->Script Editor) at this point to uncomment some extra filters in the template.

I mostly encode with Xvid and find HDTV needs at least a bit rate of about .25 bits per total pixels encoded. I usually reduce the dimensions to either 1280x720 or 960x544, cropped to remove black bars. For 1280x720 @ 30 fps this comes out to about 7 mbps and for only 950x544 @ 24 it's only about 3.1 mbps, though I may use a bit more.

The real limitation tends not to be the space required but the needed CPU to smoothly play Xvid HDTV clips which rises with higher bit rates, resolutions, and frame rates. For this reason I avoid the newer fancier compression options like b-frames, 1/4 pel, and GMC. This is not just because they are experimental and maybe subject to change but because all of them can tend to need a more powerful CPU to play at higher resolutions. So use them with care but be aware a bunch of folks on the Xvid forum will know a lot more about this than I do, so maybe seek advice there.

And for 60 FPS progressive source material (like 720p or 480p) I'll reduce the frame rate using SelectEven(), possibly followed by Decimate(cycle=5) to get either 30 or 24 FPS.

Most of the rest of the stuff you will need can be found in the Avisynth 2.5 (YV12) FAQ (http://forum.doom9.org/showthread.php?s=&threadid=37276). After the DVD2AVI step it's basically just like processing VOB's except with bigger pictures.

To get you started, here is the HDTV_1080_To_Qtr_Res.avst template. This is is a fast simple one that will deinterlace/ivtc most any 1080i stream to a 960x544 output. Just cut and paste it to a file by that name in your Virtualdubmod/template folder, then say Open With Avisynth, choose this template (first!) then choose your .d2v project file as the input file.

#ASYNTHER HDTV_1080_QTR_RES (To 960x544)
[MPEG2Source("%f",cpu=0)]
#crop(8,64,0,-64) # Uncomment to crop if needed
#UnDot() # Uncomment to remove mosquito noise
YV12InterlacedReduceBy2()
#Decimate(cycle=5) # UnComment for movies to get 24 fps
return last

Feel free to ask here again if you have more specific questions on this. My post here was just a brain dump of what I'd been planning.

BTW, kudos & props to the Virtualdubmod & Avisynth teams for making all this a bunch easier than it used to be. :)

- Tom

edit: Since I've got this far I've made this thread sticky until I get a real mini-guide out.

[FreQi]

Does using DVD2AVIT3 eliminate the need for converting the .TS's into VOBs (or aka mpeg2). That sounds like a nice time saver, but I also read somewehre that it's really easy to make these Transport Stream -> VOBs into DVD's (I have access to a burner). However, that should probably be a totally different thread...

I continued to poke around trying to figure out how to get my d2v into avisynth, and I managed to find out DVD2AVI v1.76 fixed me up. The .d2v file that DVD2AVI 1.77 was making didn't want to open in AVISynth 2.08 (I switched back to 2.0x when I couldn't get 2.5 to work with MPEG2DEC.DLL from Dividee). The problem I ran into was AC3 Audio. I would really like to keep ac3 audio with my encodes, however I do not know how to use avisynth to make my Trim()'s cut the audio as well. Is MarcFD's AC3Source (http://ziquash.chez.tiscali.fr/) what I am looking for? (it's an avs 2.5 filter, so I'll have to switchup to 2.5 and MPEG2DEC3.dll). In the past I have always worked with huffyuv avi's with wav audio. What I know I can do is use DVD2AVI to demux the audio to a 44.1 wav, then use AVISource().AudioDub() and just stick with vbr mp3. But when those 5.1 audio feeds come of the air, I'll really want to know how to keep AC3 (won't I?).

BTW, The reason I want to make avisynth do the video and audio cuts (instead of vdub) is because I bounce between divx 3.11a and xvid, and I want to make sure the material each is encoding is exactly the same. Trying to set the exact same frame cuts in vdub and then in nandub is a waste of time when I can just open the same .avs in the different programs.

[trbarry]

Yep, you don't need to convert it to a vob first. A few of the DVD2AVI versions just demux and do that on the fly.

And if the only thing you are worrying about is getting the exact same deleted sections then you can just save & load the processing settings in Vdub. I always save them anyway in case something decides to crash or I have to do it over for some other reason.

I keep meaning to learn more about how Avisynth handles 5.1 sound but I've pretty much gotten used to this way now.

- Tom

[FreQi]

It seems I am unable to use YV12. Here's what brings me to that conclusion (I'll write these out in case other people have any of these symptoms, maybe they suffer from the same problem I am):

* When I use DVD2AVIT3 to open a set of .TS files, I get an error saying "You video card cannot handle YUV display. Conversion will work, but you will not get any display in YUV mode". I click OK and it crashes. Note: The error message does say "You video card..." instead of "Your..." in case you feel like correcting grammar ;]

* Then I tried using AVISynth 2.51 and VirtualDub 1.5.1 to open a script (below) but got the error message "Couldn't locate decompressor for format 'YV12' (unknown)" and it goes on to say I need a Video for Windows (VFW) codec. So I tried installing MarcFD's vble video codec (beta version).zip (http://forum.doom9.org/attachment.php?s=&postid=263985) (linked here (http://forum.doom9.org/showthread.php?s=&threadid=38389&perpage=20&pagenumber=5#post263985)) thinking "well, that's a YV12 codec" but that didn't seem to help.

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\ac3source.dll")
MPEG2Source("ss-e12.d2v")
AC3source("ss-e12 AC3 T01 3_2ch 384Kbps DELAY -371ms.ac3")
BicubicResize(512,384,0,0.5, 170,4,940,716)

* I thought the problem might be that I was using VirtualDub 1.5.1, so I installed VirtualDubMod v1.4.13_2 with needed_DLLs_251102 and AviSynthLexer_0_1.zip, but when I first open vdmod, I get the License agreement, acept it and it crashes.

* Lastly, I decided to simply add ConvertToRGB32() to the (above) avisynth script, and VirtualDub 1.5.1 was able to open it just fine.

So I can't seem to work with YV12, and I don't really know why, except it seems to be the video card, or driver. I'm currently using an old nVidia RIVA TNT2 64mb card that I intend to replace in a week or so. Is there anything I should look for in a card to avoid this problem? Or is there something simple I am over looking?

[Wilbert]

Then I tried using AVISynth 2.51 and VirtualDub 1.5.1 to open a script (below) but got the error message "Couldn't locate decompressor for format 'YV12' (unknown)" and it goes on to say I need a Video for Windows (VFW) codec. So I tried installing MarcFD's vble video codec (beta version).zip (linked here) thinking "well, that's a YV12 codec" but that didn't seem to help.
Install DivX5 or one of the recent builds of Koepi (a stated in the AviSynth v2.5 FAQ).

AC3source("ss-e12 AC3 T01 3_2ch 384Kbps DELAY -371ms.ac3")
I should warn you that this plugin downmixes to a two channel audio. Of course you can also mux the AC3 with VirtualdubMod.

I thought the problem might be that I was using VirtualDub 1.5.1, so I installed VirtualDubMod v1.4.13_2 with needed_DLLs_251102 and AviSynthLexer_0_1.zip, but when I first open vdmod, I get the License agreement, acept it and it crashes.
Empty the plugin dir of AviSynth and try again. If that doesn't help install VirtualDubMod v1.4.13_1.

Lastly, I decided to simply add ConvertToRGB32() to the (above) avisynth script, and VirtualDub 1.5.1 was able to open it just fine.
That's because Virtualdub 1.5.1 can't handle YV12 clips, while VirtualdubMod (in fast recompress mode) can. This is all stated in the AviSynth v2.5 FAQ.

[trbarry]

* When I use DVD2AVIT3 to open a set of .TS files, I get an error saying "You video card cannot handle YUV display. Conversion will work, but you will not get any display in YUV mode". I click OK and it crashes. Note: The error message does say "You video card..." instead of "Your..." in case you feel like correcting grammar ;]

It's a limitation of your video card but it doesn't really matter if you are using MPEG2DECx. Try the following:

1) Delete your DVD2AVI.ini file.

2) Bring up DVD2AVI but change to RGB. This is ignored by MPEG2DEC3 anyway but affects the preview display. Exit again without opening any files. This will save a new default DVD2AVI.ini file.

3) Put the correct PID's in the DVD2AVI.ini file and try opening your files again.

That message spelling has been there since the beginning but I'll try again to remember to fix it. ;)

- Tom

[Entropy512]

Originally posted by trbarry
It's a limitation of your video card but it doesn't really matter if you are using MPEG2DECx. Try the following:

1) Delete your DVD2AVI.ini file.

2) Bring up DVD2AVI but change to RGB. This is ignored by MPEG2DEC3 anyway but affects the preview display. Exit again without opening any files. This will save a new default DVD2AVI.ini file.

3) Put the correct PID's in the DVD2AVI.ini file and try opening your files again.

That message spelling has been there since the beginning but I'll try again to remember to fix it. ;)

- Tom
Sometimes a partially corrupted input file can also cause this.

I've received this error message, even though 90% of the time I don't get it, on certain files I will get it 100% of the time. Usually those files are ones that I give up on transcoding eventually. (When decoding from VOBs created with HDTVtoMPEG2, the AC3 audio stops prematurely, when directly importing the transport stream, the decoder will crash at some point in the transcode process, even after using the GraphEdit cleanup trick used in the AVSForum WM9 encoding guides.)

[trbarry]

I've tried a number of tricks for problem .tp files but usually these days I'll just delete the darn things unless I really want it.

But if HDTV2mpeg runs into any problems at all I recommend not bothering to try the output but instead immediately switch to a different method. It is disheartening to get all the way through encoding and then find the audio is not fixable for some reason.

I've gotten so paranoid about break-ups that I mostly record only in the background and not use the computer for anything else until it's over.

Another useful trick in DVD2AVI is to skip the first minute or so if it's a commercial. Do this by hitting the "[" key at the right location. Some (all) HDTV cards cause a couple more break-ups at the beginning as they start recording.

Back when I was using a Hipix card I often would start the recording a minute early just so I could have something to throw away.

- Tom

[Entropy512]

Originally posted by trbarry
I've tried a number of tricks for problem .tp files but usually these days I'll just delete the darn things unless I really want it.

But if HDTV2mpeg runs into any problems at all I recommend not bothering to try the output but instead immediately switch to a different method. It is disheartening to get all the way through encoding and then find the audio is not fixable for some reason.

I've gotten so paranoid about break-ups that I mostly record only in the background and not use the computer for anything else until it's over.

Another useful trick in DVD2AVI is to skip the first minute or so if it's a commercial. Do this by hitting the "[" key at the right location. Some (all) HDTV cards cause a couple more break-ups at the beginning as they start recording.

Back when I was using a Hipix card I often would start the recording a minute early just so I could have something to throw away.

- Tom
Thanks for the tips. I typically delete the files after 2-3 failures, but I've been getting more aggressive about it as time goes by and I know more about what almost always works and what never seems to work. :)

In my case, I think one of the only solutions will be a bigger antenna. Something I need to work on anyway. :)

[FreQi]

It seems my problems with YV12 were fixed by installing a new version of XviD, and the new version of VirtualDubMod. I no longer get any error messages or program crashes, so I've been working on getting this under way for the last couple hours. I have most of a step-by-step done, however I haven't actually encoded anything yet because I kind of got stuck, and it's bed time. But here is what I have done so far (pls forgive spelling and grammar, this was a "quick" write up).


How To get HiPix Transport Streams into VirtualDubMod via AVISynth

Apps used in this guide:
- HiPix DTV-200 v3.5.2 Release Candidate 1 (AVS Forum Version (http://www.midwinter.com/hipix/))
- HDTV2Mpeg2 v1.09 (http://www.avsforum.com/avs-vb/attachment.php?s=&postid=1408610)
- DVD2AVIT3 v1.83 (http://www.trbarry.com/DVD2AVIT3.zip) (modified ver of DVD2AVI v1.76 for Transport Streams)
- AC3Filter v0.66b (http://ac3filter.sourceforge.net)
- Nic's 2003-Mar-16 build of XviD (http://nic.dnsalias.com)
- VirtualDubMod (http://sourceforge.net/projects/virtualdubmod) v1.4.13.2v2 (http://prdownloads.sourceforge.net/virtualdubmod/VirtualDubMod_1_4_13_2v2.zip?download) +vd-dll's (http://prdownloads.sourceforge.net/virtualdubmod/VirtualDubMod_needed_DLLs_from_VirtualDub_1_4_13.zip?download), Lexer 0.1 (http://prdownloads.sourceforge.net/virtualdubmod/AviSynthLexer_0_1.zip?download), DLLs_260303 (http://prdownloads.sourceforge.net/virtualdubmod/VirtualDubMod_needed_DLLs_260303.zip?download), modified AviSynth 2.5.1 (http://prdownloads.sourceforge.net/virtualdubmod/Avisynth-2.5.1-VirtualDubMod.zip?download) (all extracted to the same dir)
- Donald Graft's (http://sauron.mordor.net/dgraft/) Decomb.dll v4.06 beta 6 (http://sauron.mordor.net/dgraft/decomb/decombnew.html)
- AviSynth (http://avisynth.org) v2.5.1 (http://prdownloads.sourceforge.net/avisynth2/AviSynth_251.exe?download)
- MarcFD (http://ziquash.chez.tiscali.fr/)'s MPEG2Dec3 v1.00 (http://ziquash.chez.tiscali.fr/MPEG2Dec3%20v1.00.zip)


Step 1: Record some DTV

- Use the HiPix software from AVS Forum to tune/record some tv

- Take note of the Resolution and Frame Rate it records in. You'll need to know this later on.

- The HiPix is the only card I have, but I believe anything that makes .TS files will work with the rest of this guide as well.


Step 2: Find your PID numbers

- Start HDTV2Mpeg2 and open your recorded .TS files

- Select the Channel you want from the drop down (This really only matters if the source you recorded from multicats content)

- See what the Video PID and Audio PID's are

- Close HDTV2Mpeg2 (do not do any converting)


Step 3: Create a D2V project file and Demux the AC3 audio

- Before you start DVD2AVIT3, make sure your dvd2avi.ini file has the video and audio PID's set to the same thing you found in HDTV2Mpeg2. It should look like one of the following lines:
MPEG2_Transport_PID=11,14
MPEG2_Transport_PID=21,24
MPEG2_Transport_PID=31,34

- Start DVD2AVIT3 and open your .TS files

- Set the following options in the menus (some may be the defaults)
* Video -> Field Operation -> None
* Audio -> Track Number -> Don't process Audio
* Audio -> Channel Format -> Auto Select
* Audio -> Dolby Digital -> Demux All Tracks
* Audio -> MPEG Audio -> Demux

- Save the project file by pressing F4 (File -> Save Project)

- Watch the Statistics Window until it finishes.
Note: If DVD2AVIT3 looses focus, it seems to halt processing


Before you do the next step, make sure you have a version of XviD installed that supports YV12 (I used Nic's from 2003-Mar-16). To hear the AC3 you demuxed, you'll want to install AC3Filter so you can play it in media player. Also make sure you have VirtualDubMod and all the DLL packs/etc to get that working. Make sure you have AVISynth 2.5.1 (or newer) installed with the YV12 version of Decomb.dll and MPEG2Dec3.dll in it's plugin dir. You'll also want to put the following in with VDMod's Templates dir:

----- Begin: hdtv-1280x720-59.940fps.avst -----
#ASYNTHER HDTV 1280x720 59.940fps

# This template will take a source with a res of 1280x720 at a
# frame rate of 59.940fps and perform IVTC to get 23.976fps and
# resize it to 640x352. Other Resize options are below.

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MPEG2Dec3.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Decomb.dll")

[MPEG2Source("%f")]

Telecide()
SelectEven()
Decimate(cycle=5)

# When Source AR is 1.778:1 -- 16:9 Wide-Ssreen
Crop(4,4,-4,-4)
#Resized AR 1.800:1 with 170,240 pixels
#BicubicResize(576,320, 0, 0.5)
#Resized AR 1.818:1 with 225,280 pixels
BicubicResize(640,352, 0, 0.5)

# When Source AR is 2.4:1 (Theatrical/Panavision Wide-Screen)
#Crop(?,?,?,?)
#Resized AR 2.437:1 with 159,744 pixels
#BicubicResize(624,256, 0, 0.5)
#Resized AR 2.350:1 with 174,080 pixels
#BicubicResize(640,272, 0, 0.5)
#Resized AR 2.5:1 with 163,840 pixels
#BicubicResize(640,256, 0, 0.5)

# When Source AR is 1.333:1 -- 4:3 Full-Screen
#Crop(?,?,?,?)
#Resized AR 1.333:1 with 163,840 pixels
#BicubicResize(512,384,0,0.5)
----- End: hdtv-1280x720-59.940fps.avst -----


Step 4: Trim unwanted footage

- Start VirtualDubMod

- File -> Open video file via AviSynth...

- Select the Template that is appropriate for the material you recorded. For me, ABC broadcasts have a resolution of 1280x720 at 59.940fps, so I'd select that template (the one listed above).

- Scroll through the video to see if you like the cropping/resizing.

- Make any changes you may need by using the Script editor (under the Tools menu or just hit Ctrl-E)

- Once you're happy with the Crop/Resize, it's time to add your audio.
* Go to AVI -> Audio -> Interleaving. Under "Audio skew correction" enter the value that is in the name of the AC3 File that DVD2AVIT3 demuxed. It's the number between DELAY and ms.ac3 (ex: for "mwnk AC3 T01 3_2ch 384Kbps DELAY -327ms.ac3" you'd put in -327)
* Go to AVI -> Audio -> AC3 Audio. Select the AC3 file. Also make sure Direct Stream Copy is selected.

- Now use the slider and "Selection Start/End" buttons to delete the frames you do not want (Commercials/etc).

- SAFTEY FIRST: Now Save your processing settings, in case something breaks. Just go to File -> Save Processing Settings


Step 5: Audio Treatment

- There are so many things you can do with the audio, but for now, I'm just going to convert it to a VBR MP3. Possibly a better alternative would be to downsample the AC3 to make the file smaller, but I don't know enough about AC3 yet. Mabe in an updated guide...

- Now save your "edited" audio by going to AVI -> Demux audio...


Right about there is where I got stuck. I have very little know-how when it comes to dealing with AC3 audio at this point. I tried opening up BeSweetGUI but I'm in no condition to try figuring out what all thoe options are right now. I'd like to know how to do a few things at this point:

1) How can I make my AC3 a WAV so I can open in Audacity and amplify/clean it, then encode with RazorLAME.

2) How do I go from AC3 -> Normalized -> VBR MP3 using LAME's --r3mix switch(es)? I admit I haven't looked much, but I didn't see a way to add a profile to BeSweetGUI (so tired...)

3) How can I go from AC3 -> Normalized -> Downsampled AC3?

Anyway, I'm interested in your take on the steps I've taken so far.

[jrmann1999]

You forgot one important tool.
BeSplitv0.82.zip (http://dspguru.notrace.dk/BeSplitv0.82.zip)
This utility allows you to "fix" the AC3 stream if you have CRC or sync errors.

[trbarry]

FreQi -

Great write-up.

I can't help you much with the audio since I just keep the ac3 anyway. But in your template, for 720p source I think you can remove the Telecide() line since it probably doesn't do anything for 720p except take time.

- Tom

[FreQi]

Ak, you're right. I didn't even think about the fact it was progressive scan. I've posted an updated template below. Being new to VDMod templates, is my used of "LoadPlugin" appropriate? I was looking at the sample templates and the plugins seem to be used there, but are not specified. I looked in the VDMod FAQ (http://forum.doom9.org/showthread.php?s=&threadid=44244) and the AVISynthesizer home page (http://tangentsoft.net/video/asynther/) but didn't see anything about it.

----- Begin: hdtv-1280x720p-59.940fps.avst -----
#ASYNTHER HDTV 1280x720p @ 59.940fps
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MPEG2Dec3.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Decomb.dll")

[MPEG2Source("%f")]

SelectEven() # reduce frame rate to ~30fps
Decimate(cycle=5) # reduce framerate to ~24fps

# When Source AR is 1.778:1 -- 16:9 Wide-Ssreen
Crop(4,4,-4,-4) # take 4 pixels of left, top, right, bottom
#Resized AR 1.800:1 with 170,240 pixels
#BicubicResize(576,320, 0, 0.5)
#Resized AR 1.818:1 with 225,280 pixels
BicubicResize(640,352, 0, 0.5)

# When Source AR is 2.4:1 (Theatrical/Panavision Wide-Screen)
#Crop(?,?,?,?)
#Resized AR 2.437:1 with 159,744 pixels
#BicubicResize(624,256, 0, 0.5)
#Resized AR 2.350:1 with 174,080 pixels
#BicubicResize(640,272, 0, 0.5)
#Resized AR 2.5:1 with 163,840 pixels
#BicubicResize(640,256, 0, 0.5)

# When Source AR is 1.333:1 -- 4:3 Full-Screen
#Crop(?,?,?,?)
#Resized AR 1.333:1 with 163,840 pixels
#BicubicResize(512,384,0,0.5)
----- End: hdtv-1280x720-59.940fps.avst -----

I also started writing one for 1920x1080i, but until I actually play with it, I'll keep it to myself to avoid confusion.

There's already a software update to the above guide. It looks like Graft updated Decomb.dll (http://sauron.mordor.net/dgraft/decomb/decombnew.html) last night, so I'm now using 4.06 Beta 7 (http://sauron.mordor.net/dgraft/decomb/decomb406b7.zip).

@trbarry
There's one thing about DVD2AVI that has been bothering me. Maybe you have the tools to do something about it. I noticed that when I open files, the "File List" window is not resizeable. It doesn't even add scroll bars. And when I click ADD I can not select multiple files (like you can with HDTV2Mpeg2). When my files are in 2 sub directories and have names like "[2003-03-22 11.35pm] She Spies.0000.ts" it is impossible to tell what files are being loaded and in what order. The only way around it is to rename the dirs and files. Can this be changed?

[trbarry]

Being new to VDMod templates, is my used of "LoadPlugin" appropriate? I was looking at the sample templates and the plugins seem to be used there, but are not specified.

You can specify path names for testing but if they are in that default plugin folder they will be found anyway, starting with Avisynth 2.x (?). So as long as you have run the install then you don't need those lines. And that isn't anything specific to Virtualdubmod, it's true for all filters.

There's one thing about DVD2AVI that has been bothering me. Maybe you have the tools to do something about it. I noticed that when I open files, the "File List" window is not resizeable. It doesn't even add scroll bars. And when I click ADD I can not select multiple files (like you can with HDTV2Mpeg2). When my files are in 2 sub directories and have names like "[2003-03-22 11.35pm] She Spies.0000.ts" it is impossible to tell what files are being loaded and in what order. The only way around it is to rename the dirs and files. Can this be changed?

I've been lazy about that, but probably won't fix it that way. The trick is that DVD2AVI (all of them) will get up to 10 sequential file names if you just select one of them. But there is a bug that it doesn't do it properly for files that don't have a 3 letter file type. So I've been renaming mine to "trp". I'll fix that part shortly, along with coming up with a default of mpeg2 & HDTV files.

I don't really consider that DVD2AVI pgm mine and don't work on it much but occasionally someone sends me a fix or something irratates me enough that I go change it. But it's all in Sourceforge (and source in my zip) if anyone wants to have at it. ;)

- Tom

[trbarry]

See http://forum.doom9.org/showthread.php?s=&threadid=49866

I corrected that spelling error. ;)

Also made the file list box bigger, corrected handling of 2 char file types, and the occasionally huge audio delay.

www.trbarry.com/DVD2AVIT3.zip

- Tom

[jrmann1999]

I can thank you wholeheartedly. For once I didn't have a -12349812ms audio delay and I muxed within 1 minute of finishing a transcode. Thanks for all your help.

[FreQi]

I haven't had much of a chance to play with the new version, but I did get to use it on two encodes, and I have to say it's nice being able to load the .ts files now without renaming all 40 files (although I do still need to rename the directory to fit the full name in the box).

Thank you!

[Mikename]

to clean the audio I use a command line utility called ac3fix to get rid of the silent errors that seem to seep into all my feeds. Then I just use Headac3he to dump it to a wave, do whatever you want in audacity, then razor lame it to mp3.

[Enrico Ng]

thanks for the guide.
I just started with HDTV so I don't know much.
Everything worked great except for the audio sync. It seemed alittle off when I viewed the .tp file, but after typing in the delay that DVD2AVI told me, it was even more off.

[FreQi]

You might need to note the value of the audio delay. I remember being confused about the value being negative and I wasn't sure if I should specify the - or not because at the time, the thought of a "positive delay" was just really confusing. Basically, if the AC3 that DVD2AVI gives you says it's -324 ms, then that's what you put in VirtualDub. If it say's it's a 634ms delay, then you use that. I never did finish this guide once I got the audio part down, and I feel horrible about that, but the tools have all been updated since then, so I'd want to completly re-write it with the updated tools...

If you have questions, I might be able to help out, so don't hesitate to ask.

[jrmann1999]

You might also want to look into ac3delay corrector. Some of the ac3's that vdubmod produces are technically correct(insomuch as the delay was auto-detected) but the ac3 doesn't have the right delay in it's headers. ac3delay corrector fixes that.

[FreQi]

Personally, I do not like to have an audio delay in my final encodes (meaning I like to have a delay value of 0 in the finished product, be it xvid or dvd compliant mpeg2). I just think it safer and possibly more compatible with different players if there is no delay. So the method I use for the audio treatment eliminates any delay, and I don't know if ac3 delay corrector (http://home.t-online.de/home/520072193568-0001/) would be necessary.

In brief...

After I use DVD2AVI to demux the ac3 and generate a d2v, I use VirtualdubMod and an avs template I wrote to open the d2v. Then I set the audio delay to whatever the delay value is for the ac3, then I select the ac3 file and put it in Direct Stream Copy mode. Then I use the slider and range markers to delete the commercials out, and save my processing settings. I then use VDMod to demux the ac3.

This gives me the audio file for just the portion of video I want, and because the delay has alredy been applied, the effective delay on this newly cut and demuxed ac3 is now 0 (zero). But because AC3 is written in blocks, you can't exactly cut the audio stream in any arbitrary place without cutting a block in half. So this trimmed ac3 you've just demuxed has errors in the audio stream. To fix this, I just open up BeSliced (a GUI for BeSplit (http://dspguru.doom9.net/)) then drag-n-drop the trimed ac3 on it and select "fix". This scans the ac3 and fixes the cuts.

Once you have the fixed ac3, you can decide a couple paths, depending on what your final encode will be for. I ususally pick one of three options.
1: Use BeSweet to encode the AC3 to an MP3. This is usually the case when making an XviD.
2: Leave the AC3 alone. This would be if I was making a particularly large XviD, but typically would be if I am encoding for a DVD.
3: Use AC3Machine to transcode the AC3. I'd only do this if the source AC3 is 5.1, but I know the audio within that is only 2ch stereo (my ABC affiliate broadcasts in 5.1, but anything non-hd on the dTV feed is only stereo, so transcoding to a lower bitrate 2ch AC3 makes sense).

Now that I have the audio cut commercial free, you might be wondering how I get the video to be cut to match. This is the reason I save the vdmod processing settings. What I do is open the .vcf in ultraedit (or notepad) and I copy the AddRange lines to my .avs script, and use a function I wrote to convert these virtualdub Ranges to avisynth Trim()'s. This perfectly matches the cuts in the audio to the cuts in the video. So let's say the .vcf has these Ranges in it:

VirtualDub.subset.AddRange(1269,2708);
VirtualDub.subset.AddRange(15603,33680);
VirtualDub.subset.AddRange(62108,23270);
VirtualDub.subset.AddRange(94955,17768);

I just put that in my avs like this:

AddRange(1269,2708)++AddRange(15603,33680)++AddRange(62108,23270)++AddRange(94955,17768)

The function I wrote is simply named AddRange and looks like this:


# to easily translate VirtualDub's version of "Trim()"
# c = video clip
# s = frame number to start on
# r = number of frames to take after, and including, s (the range)
function AddRange(clip c, int s, int r)
{ return c.Trim(s,s+r-1)
}

Once I have these cuts in my avs, all that's left is to encode the video. If I'm going to make an xvid, I simply set virtualdub to Fast Recompress and the Audio to None and encode it. If I'm making a DVD, I drop the avs into CCE. When it's all done I just mux the audio and video together without setting any delay values because it was applied before doing any cuts (I still use nandub to mux xvid and vbr mp3's).

I think that just about covers it...

[Enrico Ng]

Originally posted by FreQi
You might need to note the value of the audio delay. I remember being confused about the value being negative and I wasn't sure if I should specify the - or not because at the time, the thought of a "positive delay" was just really confusing. Basically, if the AC3 that DVD2AVI gives you says it's -324 ms, then that's what you put in VirtualDub. If it say's it's a 634ms delay, then you use that. I never did finish this guide once I got the audio part down, and I feel horrible about that, but the tools have all been updated since then, so I'd want to completly re-write it with the updated tools...

If you have questions, I might be able to help out, so don't hesitate to ask.

I played around with this and tried the ac3 fix thing, it said it saw some errors
I used beslice to fix it, but the audio was still out of sync.

for me, DVD2AVI says "-421ms"
I tried typing 421ms POSITIVE into virtualdub, and it seems insync now.
I guess I had it backwards.

Everything seems to work well now, though I tried capturing with a different channel today and when I tried opening it in HDTV2MPEG2 and the video and audio pid's were ZERO. I'm assuming my clip was corrupt

[FreQi]

Are you using the latest version of HDTV2Mpeg2 ? Just a few days ago I found version 1.10b (http://www.derangedkiwi.net/software/) and it's a bit more rhobust than the v1.09 I was using. I haven't run into a time when the PID's were not 11/14 or 21/24, so you could always just take a stab in the dark and see which ones work for that particular recording.

If I remember right, the newer version of VirtualDubMod actually auto-compensates for the audio delay by reading the name of the audio file you import. Now, I don't know if this just sets the delay for you, or if it uses some behind the scenes voodoo, but I'm affraid I am not qualified to comment on that since I am still using vdmod v1.4.13, and 1.5.10.1 has been out for a couple weeks now. So I am a little behind the times on that software.

[trbarry]

for me, DVD2AVI says "-421ms"
I tried typing 421ms POSITIVE into virtualdub, and it seems insync now.
I guess I had it backwards.

That's pretty scary. It used to be that you would type in -421 for a delay, including the sign. Has Vdubmod changed?

- Tom

[Enrico Ng]

Originally posted by trbarry
That's pretty scary. It used to be that you would type in -421 for a delay, including the sign. Has Vdubmod changed?

- Tom

Its strange, I don't know which way it should go. Sometimes including the - works, sometimes not including the - works. At least its either one or the other.

Anyways, this dvd2avi/Avisynth/virtualdubmod technique works great.

I also would like to try making a DVD.
Should I just do the same thing (cut out commercials and synch audio), and save it to uncompressed AVI, then use TMPGEnc to re-encode it for DVD?

[FreQi]

Just feed the .avs to your dvd encoder. If it won't open it, rename the .avs to .avi and it should read it just fine. The only advantage to putting it to an uncompressed avi would be if you're doing multiple passes on your dvd encode (think more than 2). If you're doing 6 passes, an uncompressed avi as a source might get it done quicker than if you have avisynth process the source every time.

[bdraw]

I am just getting started and I would like to edit the show at the beggining of the process using HDTV2MPEG but on my first attempt the audio gets off sync at one of the edits despite running the ac3 file through both ac3fix and beslice.

Do most of you edit using Virtualdubmod?
If so I know there has to be a link to a site that explains those cryptic buttons on the bottom of Virtualdub better, that I asume are used for editing. I guess I need it spelled out.

I have already read "VirtualDubmod procedures"(http://www.doom9.org/vdubmod-procedures.htm) and am still not clear how I would use virutaldubmod to edit. In the past I have used HDTV2MPEG and Vidomi to edit. I have also used TMPGENC but I hope this is not anything like it, becuase I find that method awkward.

[trbarry]

I always edit HD commercials with vdmod because of the audio sync problems you mentioned in HDTV2MPEG. Editing involves selecting the start and end of the range with the check buttons and then hitting the delete key to delete the section marked. Since you are now working with uncompressed video there is no need to try to cut on key frames.

Vdubmod seems good at cutting the ac3 in legal places, so there is probably no problem there.

But since editing can be a time consuming process you should "Save Processing Settings" once in awhile (like any ap) since I think MPEG2DEC3 can sometimes crash when using the left arrow key or button to go back on frame at a time, at least with some filter combinations.

There is bubble help over the buttons if you hover the mouse and there are also shortcut keys shown on the edit menu and I think in the help. Clicking on the seek bar with uncompressed data seems to move 50 frames at a time, IIRC.

If you have a range of frames selected when you Save then only that selection is saved/encoded.

More complex questions & answers can be found by searching and/or posting in the Virtualdub forum here.

- Tom

[bdraw]

Thanks, Tom I was missing the "Delete" key, I kept hitting the mark in and mark out and it would only let me mark one part at a time. Now I know what I was missing.

[bdraw]

I am still having trouble.

For some reason vdubmod does not recognise that the ac3 stream is the correct legnth. It detects 34mins when it is really 57mins. I have not edited clip. I can open the ac3 file in WMP and elecard the time is detected properly. The sound is fine, there is not dropouts.

I can drag the ac3 file into graphedit and playback is fine.
Threatmatrix.ac3-->AC3 Parser Filter-->Intervideo Audio Decoder-->ReClock

I have tried both ac3fix and beslice to try to check for errors in the ac3 file. There are none reported.

When I mux in the audio using vdubmod it doesn't get passed 0:34:37
Anyone have any idea why vdubmod(1.5.10) would detect the incorrect legnth?

[FreQi]

What did you do to get this AC3? Did you get it from DVD2AVI? HDTV2Mpeg2? Did you cut any ranges out of the sources you demuxed from? What version(s)?

[bdraw]

Originally posted by FreQi
What did you do to get this AC3? Did you get it from DVD2AVI? HDTV2Mpeg2? Did you cut any ranges out of the sources you demuxed from? What version(s)?

I cliped the begginning and the end of the clip(using HDTV2MPEG v1.10b), but no splicing anywhere else. I did not use HDTV2MPEG to convert it to .mpg. I opened the transport stream with DVD2AVIT3(v1.83) from Trbarry. Besweet has no problem at all converting the ac3 file to mp3. If I insert the mp3 stream using vdubmod it detects the correct legnth.

[NorcoO1]

Has anybody else experiment in VdubMod with using WMV9 as the compression? I was messing around with it and wondering what kind of quality xvid offers against WMV9.

[jrmann1999]

The true hinderance to most people using WMV9 is being restricted(mostly) to windows only machines for playback. Other than that I think it's all "in the eye of the beholder" as to what you like.

Personally I've found that WMV9 is a bit tougher to achieve good quality encodes. Probably only because I'm so used to xvid though.

[NorcoO1]

I'm having issues with audio and video not being in sync. The only cause of this I can assume is that the clips that have sync issues are the ones that multicast. So CBS has no problems encoding, but NBC gives me out of sync issues. I do all the same settings in VDubMod with both stations, making sure to put the correct delay on and such. Another thing that makes me wonder is when I'm making a project file in DVD2AVI, the timecode shows a negative when I do NBC multicast HD, and its positive when I do CBS shows. Anybody know what the deal is? Thanks.

[jrmann1999]

Hopefully someone has told you by now that vdubmod has issues demuxing AC3(the delay values are wrong) so if you interleave and demux assuming that you now have 0 delay, you do..but the headers in the AC3 file don't reflect that so any player that reads them will delay it. There's a tool called AC3 Delay corrector that lets you patch up all the headers to the correct delay, fixed 99% of my problems with audio. My issue now is that vdubmod is 50/50 on importing and recognizing a delay(fixing it itself) or not, no real big deal just more of a pain to have to demux audio twice.

[crahak]

Just a quick note

It seems like everyone is doing

SelectEven()
Decimate(cycle=5)

But I find it's too choppy, and it looks like most people end up not using the decimate because of that and encoding at 30fps instead of 24. I like to use 24 personally (more bits/(pixel*frame) and less cpu needed to decode, making the use of B frames easier too...)

If you use those 2 lines in that order here's what happens:

take 15 consecutive video frames, which we'll name A to O

Source: ABCDEFGHIJKLMNO
After SelectEven: ACEGIKMO
After Decimate: ACEGKMO

See where it jumps from G to K? instead of dropping only a single frame, where it was decimated, we dropped 3 consecutive frames, which makes it look choppy.

If you Decimate first, then SelectEven after, then the max consecutive frames dropped is 2, which is significantly better. It makes it watcheable at 24fps. :D

Sorry if my explanation sucks... Nothing fancy but it seems most people don't think of it. Hope it helps.

[FreQi]

You shouldn't really think of SelectEven() as being used to drop frames. In the conext of this "guide" it's really being used to drop one of the fields of the video.

Remeber that with an interlaced source at 29.97 you use SeperateFields() first, which doubles the frame rate and makes each field it's own progressive frame. THEN you'd use SelectEven() to essentially drop the odd fields.

The only time you'd use that method with a source that's already progressive scan is when it's frame rate was already doubled (to 59.940fps). If you frame-by-frame through an hdtv source like that you'll see that almost every two frames are the same and sometimes you'll get a third. This is pretty much the same effect that results from separatefields() on an interlaced hd source, except the aspect ratios are wonky lookin there.

But yes, using Telecide().Decimate() is a better way to inverse telecine a 29.97fps Interlaced HDTV source. It's just slower.

Sometimes a choppy video results on progressive scan sources, and there you don't really have the option to use Telecide(), but I've found that to be rarely the case (thank god). In these rare but infurriating circumstances MultiDecimate has done the trick for me.

Multidecimate took a lot of poking around to figure out, but this is what I used (I've included my other attempts to fix it more or less just as examples of failure really...)


LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MPEG2Dec3.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Decomb.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MultiDecimate.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\BT709ToBT601.dll")

MPEG2Source("F:\hd.streams\someshow\series-s1e01.d2v")

# correct the slight color error SDTV->HDTV introduces
BT709ToBT601()

# cut commercials
AddRange(12110,7074)++AddRange(28389,26206)++AddRange(66398,40041)++AddRange(117499,2121)
# 75442 (0:20:58.599) <-- frames and length at 59.540fps

#ConvertToYUY2().MultiDecimate(pass=1)
ConvertToYUY2.MultiDecimate(pass=2,quality=2)
# 30176 (0:20:58.592) <-- Multidecimate settings c:120, r:72, t:0.2, l:3.
# Results in smooth playback, synched audio and good runtime

#assumefps(23.976)
# 30907 (0:21:29.069) <-- increases time too much and would desync audio

#changefps(23.976)
# 30177 (0:20:58.622) <-- good time code but drops frames and become jittery again

#convertfps(23.976)
# 30177 (0:20:58.622) <-- introduces a lot of motion blur and blended scene changes

LanczosResize(720,480)

function AddRange(clip c, int s, int r)
{ return c.Trim(s,s+r-1)
}

[crahak]

I'm not dealing with interlaced video but 720p.
And I'm discarding video frames, not fields...

(no deinterlacing/ivtc of any kind involved)

The framerate is 59.940 but every single frame (usually) is different. If the source is a 23.976fps film movie upscaled and with framerate brought up, yes, it would be the case...

Anyways. I know the explanation sucked... You seem to be using other methods and I'm sure there's even more ways to it, but so far this works and looks great @ 24fps, xvid/unlimited profile & trellis...

The only problem I ever run across is stream erros which un-sync A/V which sucks. Oh well. No miracle cures for that.

[FreQi]

I was having problems with a/v desync until I started doing a double fix on the trimmed & demuxed AC3. What I mean by that is after I use VDMod to demux the commercial free AC3, I use AC3Fix.exe to "fix" the cut points. After that I run the fixed AC3 through BeSliced which typically fixes another one or two points. I used to just use BeSliced, but it wasn't uncommon for the AC3 to drift a bit. AC3Fix doesn't suffer this but it seems to leave errors in the stream (at least according to BeSliced). I haven't had a problem with sync since.

[Lobuz]

There's a newer FDecimate() (http://neuron2.net/fdecimate/fdecimate.html) filter which should help to get rid of duplicated frames in one pass.

Regards
Lobuz

[FreQi]

Oh, good call Lobuz. I didn't know such a thing existed. I'll have to do a couple trials with FDecimate() to get a feel for performance.

On a side note, I've been working sorta half-assed on a tool that I think most people will find very useful for encoding HDTV sources to DVD compatible MPEG2's. I call it "theeo", "The Encode Orchestrator".

It's still in it's infancy, but here's what it does and bit of how it does it:

- Drag 'n Drop HD sources onto theeo
- Pick a template (defines your source type and destination settings)
- Uses a special build DGIndex to make .d2v and .ac3 (big thanks to neuron2 for that)
- Generates an AVISynth script and processing settings for VirtualDubMod
- Opens that AVS in VDMod and waits for you to cut commercials
- You save the processing settings and return to theeo
- theeo then uses your new processing settings to generate a final .avs that excludes the commercials, resizes and IVTC's the video and can be used for encoding

Those last couple steps were finished only last night, so there's a lot of tweaking and changes to be made to the way it works still. I plan on adding the stuff below before I make a release:

- demux the commercial free audio
- double fix the demux'd audio using AC3Fix and BeSplit
- chapter point management based on the Commercial Trim's
- generate a CCE .ecl

And there are a couple more features I want to implement down the road, such as...

- apply pulldown to the .mpv that comes out of CCE, making the .m2v
- multiplex the .m2v and the double fixed .ac3 to make an .mpg

I started working on this several months ago and managed to let it slip away from me, until last night. I figure the new season of TV is starting up (Las Vegas season 2 premieres tonight!) so I've regained some motivation to get this done. However, I am not too opposed to the idea of some collaborative effort on this. If you're interested in working on this with me, send me a PM. It's written in .Net (almost entirely VB.Net), which is something I am pretty new at, so I could even use someone that's just interested in reading over the code to make suggestions / tweaks for me.

[Van the man]

when I try to load the .wmv file into dvd2avit3 the program just close down and gives me that microsoft error(if you wanna report the error to microsoft).
what am I doing wrong? doesnt the dvd2avi version handle .wmv files?
the .wmv file is from microsofts hdtv samples.

any answers would be great

thanks

[FreQi]

WMV = Windows Media Video. It might be at HDTV resolutions, but it's not a Transport Stream (.ts) or another flavor of MPEG2, which is what DVD2AVI is used for.

Try VirtualDubMod or something

[eb]

My way of procesing HDTV .ts streams:

demux in ProjectX to video + audios + subtitles,
from ProjectX all files are synchronised and corrected for dicontinuinities,audios are normalised to 98%

to get final product as .avi or .mkv i am processing video with DVD2AVIT3 {big thanks to trbarry for this}, downscaling to the size where my computer can play it without any problems, as common 720x408interlaced
muxing video+mp2 audios with VirtualDubMod to .avi /yes .avi with mp2 audio/
or if audios are already as AC3 and subtitles are srt or ssa then AVIMux_GUI is used to mux all this to .mkv , playback in VLCPlayer.

to get final as DVD, the resulted files from ProjectX are processed by Nero Express or by ReJig program

and that is all

eb

edited: here my very kind ask to author of AVIMUX_GUI
is it possible that .mp2 audios can be accepted and muxed to .avi like it is done in VirtualDubMod

[jrmann1999]

eb:

Since you're using projectX first, you might want to try dgindex instead of dvd2avit3. It's a bit more accurate and it's associated dll is faster for avisynth stuff....

[rs008f]

VirtualdubMod 1.4/1.5 froze when I import the avs script. Below is my script. My MPEG2Dec3.dll is v1.0. The d2v made from dvd2avi 1.76 works fine however but the video and audio are out of synch. Video is shorter than the audio by 2 seconds.
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\MPEG2Dec3.dll")
Mpeg2Source("j:\abc.d2v")

[nnigam]

So Many questions, so few answers. I have read posts here, at inmatrix.com and a few other places without getting all the answers. I think a new and updated guides are needed. I am ready to help if anyone has something in the works. What I would like to do is as below

1. Record HDTV, my dvd's, VHS, and camcorder movies to my hard drive using my MYHD card.
2. Cut/Edit Advertisements, re-sequence personal films etc.
3. Save as AVI that can play on any pc
4. Burn to DVD/CD that can play on any dvd player
4. Archive someplace so when I finally get a HD tv, I can watch them there as well.

Questions I have include
1. DVD's have 2/3 hours of video. HDTV2Mpeg2 and other utilities convert 1 hour of HDTV to 8 gigs so that only 30 minutes gets on a DVD. What gives.
2. To convert HDTV to DVD it seems that I have to convert HDTV to AVI and then convert AVI to DVD. Is'nt there a direct process.
3. Cutting commercials using virtualdubmod is very tedious for finding the commercials to cut. Would project X and MPEG2Scnitt help. 1 hour of hdtv still creates files multiple files for a single audio stream. How do you load both. to MPeg2Schnitt.
4. I am still unable to get proper synching in some 720p hd recordings. Just read about different audio rates for advertisement and program. How do you get a sample of the program to put at the beginning to get correct audio sampling.

I have no desire to re-invent the wheel. If someone has a manual in progress, I am ready to offer testing and other tasks.

[jrmann1999]

No offense, but your first Questions screams that you don't know DVD or HD specs.

1. DVDs are encoded at a maximum resolution of 720x480p for NTSC(720x576p for PAL). They are also capped at a max bitrate right aroud 10mbit. HDTV on the other hand can range from 1280x720p to 1920x1080i resolution, and up to 19mbit bitrate. This is what accounts for the large size of the files.
2. Not true, using avisynth and either CCE or QuEnc you can encode HDTV directly to mpeg2 compliant dvd streams.
3. No, your pc is having to decode each frame you move into in order to display it on your screen, very intensive so it will not speed up. There's no real magic bullet on this one.
4. You quite simply do a 1-2s cut of an existing HD program that has the audio in the format you want. You then put that in your list as the first file loaded.

[Karl Beem]

Using the techniques described in this thread, I've processed the 720p broadcast of the Duke - NCS game on ESPNHD with good results. The .ts file was captured with CAPDVHS thru the 1394 from my Comcast(Motorola) 6208 DVR. The Duke - Miami game was also on ESPNHD, but was not HDTV - it had those ugly vertical bars. The actual game looked pretty good, not HDTV but better than analog, so I recorded it and am processing it.

1) Is this what they call SD programming?

2) With the bars cropped out, it is 932x720. Should I resize it to 720x480?

3) would you recommend that I use SelectEven()?

[nnigam]

It depends on what you want to do with it. If you want to burn to dvd to watch on your dvd player, you need to have it at 720x480 resolution at 29.97 fps. If you want to create an avi, use whatever resolution you want to watch it at. I generally resize mine to 768x432 which works very well for me.

As for select even, this is only for reducing the frame rate on video whose frame rate has been increased from 29.97 to 59.94. If you open the file in virtualdub, and scroll through the videw and you see that each frame is duplicated, then you can use this. However, I use the decomb filter decimate(cycle=2) which I think is better to reduce 59.94 to 29.97fps. Better still is the fdecimate filter which will remove all duplicated frames reducing the size of the video considerably.

As for the SD programming, I think this means the regular analog broadcast. You are most likely getting HD broadcast. You are getting this if either you get an almost perfect picture, or no picture. If you can get weak signals, snow etc, then you have SD programming. HD can be either 3:4 or 9:16 aspect ratio, and black bars are used to fit one aspect ration video so that it is visible in a tv in the other aspect ratio.

Hope this helps. I do not consider myself an expert, but I have been playing around with this for the last month or so trying to get everything working perfectly.

[FreQi]

@nnigam
Your answer for 2) and 3) are great, but your definition of SD is a bit off, at least as I understand it.

@Karl Beem
The term "SD" is loosely used, at least in here, to describe "Standard Definition" television, and like "HD", it refers to the broadcast format, not necessarily the material that is being broadcast. So by nature, SD is a 4:3 broadcast and a relatively low resolution (think 704 interlaced vertical lines by 480 columns). Then "HD" as you certainly know is "High Definition" and it comes in a variety of flavors (most notably 1080i and 720p), but they are all 16:9 wide-screen broadcasts. If it's not native 16:9 is it not HD.

There is another acronym that people like us use, "PD" which is sort of a hybrid of the terminology. It simply means "Pure Digital" and is by no means an industry standard term. To understand what "PD" refers to lends itself to understanding how a station broadcasts it's programming, and there are a couple different ways.

(note: I am pulling this all completely from memory based on stuff I've read and my own observations and "interviews" with my network affiliates, so if I am mistaken, please feel free to chime in)

- HDTV programming is sent out from the network headquarters (meaning ABC, NBC, CBS, etc) to some satellites in space and is then beamed back down to the earth on encrypted C-Band. The network affiliates (the local tv stations in your and my town) then pick up that satellite broadcast then re-broadcast it over their dTV equipment in real time with the network HQ's broadcast. They broadcast "dead air" or blackness at commercial breaks which the affiliates overlay with their pre-recorded tapes with commercials already inserted (see SDTV)

- SDTV programming is traditionally also sent out from the network HQ's to some satellites C-Band for the affiliates to pick up. However, this programming is -not- re-broadcasted in real time. Instead, the networks transmit the programming well before the scheduled airing of the show, sometimes more than 24 hours before. (google "Wildfeed" for more). This gives the affiliates opportunity to record the show, typically on VHS/Beta tapes, then insert their commercials (typically by re-dubbing the show to another tape) and then simply play the cassette when the shows time slot comes up. This process of recording and copying makes most SDTV look like garbage, but most people had crappy reception anyway, so who knew.

- PDTV comes in somewhere in the middle, and has several variants that can define it. Live broadcasts, like sports events and the local news are sent straight from the cameras through the cloud of equipment and over the antennas to your receiver. This lack of post processing tends to deliver a far superior picture and you can just see it. The main idea is the picture never flips to analog.

Digital Satellite TV is usually considered PDTV, HBO on Digital Cable/Satellite is PD, as are most of the subscription only channels like SciFi, Discovery and History channel.

Another example of PDTV is Scrubs on NBC. For whatever reason, they don't film that series in HD, but NBC still sends a live feed from their HQ to the affiliates who rebroadcast it just like the HD programming. The result is a very clear picture with sidebars.

Most SD programming of sporting events are actually cropped versions of the HD broadcast (remember, it's not PD because it was broadcasted in analog form, even on your "Digital Cable"). At the next football game, watch the same network on your SD and HD tv's. You'll see the same picture, only the HD one has more on the sides. This explains why on HD the score and time banners at the top are sort of scrunched in the middle. They just lop off the sides, down sample it and send it off as the 4:3 signal.

For some real specific descriptions, you might want to check out the online dictionary at AudioVideo101 (http://www.audiovideo101.com/dictionary/). They don't have anything on PDTV, but remember that's nothing official. Just a made up term to describe a phenomenon.

[venutolo]

I'm currently working on creating a DVD from 720p source. I ran the stream through DGIndex, giving me a .d2v project file. Then I wrote an avs script, and opened it in CCE and created a 720x480 23.97fps video file, which was then used to author a DVD. When I play the DVD the video isn't smooth like the source. I imagine this has something to do with going the 59.940fps source to the 23.97fps final video, and that it is a problem with my avs script. Is there something I'm missing, or doing in the wrong order?

LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\DGDecode.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Decomb521.dll")
MPEG2Source("K:\HDTV\24S04E01\24S04E01.d2v")
SelectEven()
FDecimate()
LanczosResize(720,480)

Any help is appreciated.

[neuron2]

Take out the SelectEven().

[venutolo]

Originally posted by neuron2
Take out the SelectEven().
That was my first guess. Much thanks.

[FreQi]

If you're going to make an NTSC DVDr, you should also run pulldown.exe on the 23.976fps .mpv you made. It makes the video think it runs at 29.97fps through mpeg flag trickery. You'll also want to tell it to flag it as progressive scan and enable drop frame (keeps the clock in sync).

I recommend you use bbMpeg to mux the ac3 and the m2v into an mpg, if that's what you plan on ending up with

[venutolo]

Originally posted by FreQi
If you're going to make an NTSC DVDr, you should also run pulldown.exe on the 23.976fps .mpv you made. It makes the video think it runs at 29.97fps through mpeg flag trickery. You'll also want to tell it to flag it as progressive scan and enable drop frame (keeps the clock in sync).
As far as I can tell, CCE takes care of pulldown. As for flagging it as progressive scan, does checking the box next to "Progressive Frame" take care of it, or is it something in authoring the DVD? Finally, I have no idea how to enable drop frame, so could you please point me in the right direction? I have had some experience working with CCE, but never with a 720p source, and this is the first time I'm authoring a DVD from scratch, so all of this is a learning experience. Thanks for the help.

Below are screenshots of my CCE template's settings.
http://img140.exs.cx/img140/3326/24cce11dv.th.png (http://img140.exs.cx/my.php?loc=img140&image=24cce11dv.png)http://img140.exs.cx/img140/4308/24cce28eq.th.png (http://img140.exs.cx/my.php?loc=img140&image=24cce28eq.png)http://img140.exs.cx/img140/5186/24cce33hy.th.png (http://img140.exs.cx/my.php?loc=img140&image=24cce33hy.png)

[nnigam]

Originally posted by FreQi
If you're going to make an NTSC DVDr, you should also run pulldown.exe on the 23.976fps .mpv you made. It makes the video think it runs at 29.97fps through mpeg flag trickery. You'll also want to tell it to flag it as progressive scan and enable drop frame (keeps the clock in sync).

I recommend you use bbMpeg to mux the ac3 and the m2v into an mpg, if that's what you plan on ending up with

----------------------------------------------------------

I just convert my 59.94 fps progressive to 29.97 by decimate(cycle=2), and leave the extra frame there. How is bbMpeg, I use Quenc, but will certainly give both pulldown and bbmpeg a try.

Would the avisynth cmd changefps be similar to pulldown since it just slows the video down.

[FreQi]

venutolo writes
As far as I can tell, CCE takes care of pulldown.
Yeah, CCE does have an option for pulldown, but I've read about issues with it enough to decide not to use it. I don't really have any examples, so take my advise for what it's worth, but PULLDOWN by HardCode (http://www.inwards.com/inwards/?id=36) does more than just set some 2:3 flags, so even if you did use CCE to set pulldown, I'd still recommend you run it through pulldown.exe for these two features:

1) Enabling Drop Frame
This has to do with the way the mpeg counts time in relation to frames. When you encode the video at 23.96 and make it play back as if it were 29.97, the mpeg's "natural seconds" appear to play longer than a "real" second. This results in the video's duration appear to be shorter than it actually is... Now, when I first read about this, I thought the flag would cause the video to actually skip playing a frame or something, but that's not the case. This guys dissertation (http://www.philrees.co.uk/articles/timecode.htm#drop) explains it rather well: "It is not so much frames which are dropped as frame numbers which are skipped." In the end, if you do not use the Drop Frame Flag in your .m2v, you could have a real hard time setting chapters accurately or getting subtitles to sync right.

2) Setting all Frames to Progressive
Now, I don't really know how beneficial this is, but here's my line of thought... The video -is- encoded with progressive frames. The 2:3 pulldown flags makes it appear to be 29.97fps, which is typically an Interlaced video. So in truth, a 29.97fps Progressive scan video would actually contain duplicate frames. By setting this flag, you're telling the players not to deinterlace when decoding and just show the frame. Progressive scan enabled dvd players should love that. Now, I've tried using the other prog_seq switch that marks the entire stream as progressive but found playback to be...problematic. Xbox Media Center hated it anyway, so marking the frames progressive and not the stream seemed to be the way to go.

So, how do you apply pulldown? I made a .bat file (see below) and made a shortcut to it on my desktop. When CCE has finished making it's .mpv I just drag and drop it on the shortcut and this .bat makes an .m2v for me. So for example, I encode lost-s1e14.mpv and drop it on the shortcut and it spits out lost-s1e14_PULLED.m2v for me to mux with the ac3.

@echo off
set INFILE=%1
set OUTFILE=%INFILE:~0,-4%_PULLED.m2v
PULLDOWN.EXE %INFILE% %OUTFILE% -drop_frame true -prog_frames p
pause

WoofSoft built a GUI (http://www.woofsoft.com/Downloads/53.aspx) out of the source code from this command line app and called it DoPulldown v1.0.0.0, but I prefer to drag and drop my .mpv on the shortcut. Old habits die hard, but it'd be easier if DoPulldown supported Drag-n-Drop (@WoofSoft: hint hint)

venutolo writes
As for flagging it as progressive scan, does checking the box next to "Progressive Frame" take care of it
Checking the "Progressive Frame" box tells CCE to encode the stream as progressive, and yes I suspect it does the same thing as pulldown.exe's -prog_frames p switch, but I'm just guessing on that...

nnigam writes:
I just convert my 59.94 fps progressive to 29.97 by decimate(cycle=2), and leave the extra frame there.
But just to be clear on why using 2:3 pulldown is a good idea... Remember how Telecine works? 24fps (what tv shows are actually filmed at) gets to 30fps by using duplicates of previous frames. The Pulldown flags tells it to simply show the previous frame, rather than having an actual copy of it in the stream. This flag trick allows you to allocate the bits that would be needed to show approximately 6 frames and spread it around the other 24.

And incidentally, decimate(cycle=2) is probably a lot slower than just using SelectEven() or SelectOdd() which would yield (probably) the same result.

nnigam writes:
Would the avisynth cmd changefps be similar to pulldown since it just slows the video down.
Absolutely not the same thing. ChangeFPS() actually changes the framerate by simply duplicating or deleting frames. Pulldown tells the software the decodes the stream to show a frame twice, with ChangeFPS, the duplicate frame would actually be there. This is also not the same thing as the ever popular Decimate() function from the Decomb plugin. Where ChangeFPS just drops or copies any frame as it needs to, Decimate actually compares frames to each other to discover which is a duplicate and drops one of those.

nnigam writes:
How is bbMpeg
Honestly, it does a lot more than I need, and it's a little convoluted to use just for muxing, but it's done a great job so I stick with it.

You can pick it up here (http://members.cox.net/beyeler/bbmpeg.html). You have to open avi2mpg2.exe which opens this interface for...honestly, I have no idea. But you have to click the "Load" button as if you wanted to load a project file and click Cancel (apparently there is a known bug that this fixes, *shrug*). Then you click the "Start Encoding" button, even though you are not encoding anything, and this opens the bbMPEG window. At this point you can click the "Settings" button and go to the "Program Stream Settings" tab and tick the dot for ~DVD. The defaults should be fine, then go to the "Input and Output files" tab to select the target .mpg and your source .m2v and .ac3 files. Click OK, then Start and it'll mux it up for you.

What can you do with the final .mpg? I've found InterVideo WinDVD to be a great app for playing the video, that or Xbox Media Center. You can also use the .mpg to author a DVD using TMPGEnc DVD Author.

[nnigam]

Thanks FreQi. Very informatinve. My project for the weekend is set. I am going to try all of this on my videos.

As for changefps, you are right. Thats what happens when I try to write from memory. I meant assumefps. I stopped using it because it changed the video rate so that audio and video no longer were in synch. I guess that changefps was also incorrect.

[MarkGrigsby]

I am using this method to convert 720p material to DVD, and am not sure why I need to use 'Telecide()' for 720p material? Surely I could just use FDecimate to get the frame rate down to 24, since there are no interlaced frames present?

Or am I missing something?!

[neuron2]

That's right. If there are no combed frames, Telecide is not required.

[FreQi]

You're right, Telecide() is not used on 720p (it's an option for 1080i) but I use Decimate(cycle=5) because I first use SelectEven() which takes it down to a progressive 29.97fps. My scripts for 720p all look something like this:

LoadPlugin("C:\Program Files\theeo\DGMpegDec\DGDecode.dll")
LoadPlugin("C:\Program Files\theeo\AVISynth\Decomb.dll")
LoadPlugin("C:\Program Files\theeo\AVISynth\BT709ToBT601.dll")
MPEG2Source("F:\According.to.Jim.s4e23\atj-s4e23.d2v")

# correct the slight color error SDTV->HDTV introduces
BT709ToBT601()

theeoAddRange(302,9931)++theeoAddRange(18822,22801)++theeoAddRange(53890,21667)

# InverseTelecine and Decimation
SelectEven().Decimate(cycle=5)

Crop(4,4,-4,-4) # take 4 pixels of left, top, right, bottom

LanczosResize(720,480).ConvertToYUY2().AudioDub(BlankClip())

function theeoAddRange(clip c, int s, int r) { return c.Trim(s,s+r-1) }

[Wilbert]

@FreQi,

Not all HDTV streams use Rec.709, contrary to what the readme of BT709ToBT601 claims.

[FreQi]

How do you determine when it needs correction? Is there some way to detect it in software, or is it just visually apparent? Perhaps there is a way to determine it logically.

For example, when something is transferred from Film to HD, it would need correction...? I believe that would be the case with just about all Drama's and Sitcoms.

When something is filmed with an HD Camera, then it would not...? The only thing I can think of here would be things like the specials you'd see on PBS, NASCAR races, the Superbowl and other live broadcast events. Incidentilly, these types of HD streams would probably not take to well to being IVTC'd either.

[Wilbert]

Just open the stream in GSpot and see what coefficients are being used. If it indicates nothing then Rec.709 coefficients are being used since that's the default. See also readme of ColorMatrix.

[nnigam]

OK. I have learned a lot here. A brief delay while I had to remove a very persistent virus on my pc that just refused to go away, but now I am back in business.

I process my 720p using dgindex, convert to 23.976 using fdecimate since they happen to be 3:2 format. I convert to avi using vdubmod and everything is in perfect synch. I convert to m2v using quenc and then run dgpulldown to convert to 29.97, and covert to dvd using muxman. Again everything is in perfect synch. I then open the puldown m2v file and the ac3 audio in Mpeg2Schnitt to cut the commercials. First problem is that there is no place to put the audio delay. Even in Mpeg2Schnitt I can see audio synch errors greater than would be caused by the delay alone. I tried the original 23.976 m2v, and I still have audio errors. I then tried delaycuter utility on the audio with both the original and pulldown m2v and I still have audio synch errors. Pulling the audio into gspot, It says the audio frame rate is 31.5fps. Somehow this synchs with the 23.976 or 29.97 pulldown audio and sitll maintain perfect audio synch while cutting causes audio errors.

I read somewhere of a utility that will take the cut list from Mpeg2Schnitt and perform the same cuts in the audio seperately. I will look for that and try again.

[FreQi]

Cut the commercials -before- you encode. If for no other reason, it's the only way to accuratly target any final file size.

[nnigam]

I would love to. How to cut an HD transport stream. I process the hd stream to a d2v project. Open the d2v in an avsynth script. Cutter tools need an m2v which I get only after processing with quenc. I tried cutting in vdubmod, but it takes forever.

In your sample, I saw

theeoAddRange(302,9931)++theeoAddRange(18822,22801)++theeoAddRange(53890,21667)

How did you get these ranges? Any limit to the number of ranges I can add here? How is the range affected by the fps changes needed to convert the 3:2 720p to 23.976?

Thanks

[FreQi]

Check out my second post on page 2 (http://forum.doom9.org/showthread.php?s=&threadid=49349&perpage=20&pagenumber=2) of this thread. It's explains pretty much what I still do today, except I wrote an app to automate the whole process (called theeo, which still needs a little tweaking before I publicly release it).

Keep in mind that in order to cut the AC3 with VirtualDubMod, you have to use v1.4.13 (http://sourceforge.net/project/showfiles.php?group_id=65889&package_id=63653&release_id=148604). The newer 1.5x doesn't cut the ac3 stream when you demux it with trim's. Not sure why.

Also, rather than using BeSplit, I'd suggest you use DelayCut (http://www.videohelp.com/~jsoto/audiotools.htm) to Fix the trimmed ac3. It seems to do a better job of maintaining sync.

Any limit to the number of ranges I can add here?
nope.

How is the range affected by the fps changes needed to convert the 3:2 720p to 23.976?

If you do your trims -before- you IVTC, you're unaffeted. You can try any sort of IVTC method and never have to cut commercials again. Just do it in the order I have my last example in. Load the .dll's, load the .d2v, trim the commercials, then process the video.

[squid_80]

I read somewhere of a utility that will take the cut list from Mpeg2Schnitt and perform the same cuts in the audio seperately. I will look for that and try again.

Mpeg2Schnitt is designed to cut the audio as well. If the audio file has the same name as the video (except for extension) it should be loaded automatically. Doesn't it do this for you?

[Bruce Willis]

Interesting thread..

Is it true that converting the framerate from 60 to 24 fps, either by SelectEven().Decimate or by FDecimate, slows down the encoding 4 times or even more? Or am I doing something wrong?

[nnigam]

I never timed it. It takes a long time regardless. I am processing 1 hour hd programmming. Taking out commercials, it comes to about 45 minutes. Encoding this takes about 8 hours. Three hours for the first pass and 5 for the second. I would assume FDecimate would be slower because it checks all frames to find best frames to drop, while SelectEven.Decimats() does not have to be too intelligent, but I could be wrong. I just process overnight and so do not care how long it takes.

[neuron2]

Decimate() also compares frames.

If you want a fast but dumb decimation, use ChangeFPS(). You can compare that to FDecimate() to get an idea of the time difference. I doubt it is as bad as you suggest.

The advantage of FDecimate() is that it preferentially drops duplicates where possible.

[FreQi]

Yeah ChangeFPS would probably be faster but I really wouldn't trust it to make a good looking video (it would probably jitter pretty regularly). Decimation is the way to go, dispite the potential speed hit.

I've got a P4 2.8 and I find encoding times to be about 4x the runtime. So if you have a 30min show, it takes me about 2hrs to encode. That's a really rough estimate... I don't know the real times off hand as I also set up my encodes and go to sleep. They're all done bu the time I wake up.

[Leehro]

learned some great stuff here, thanks for all the info. I'm still having some sync issues but I think i'm on the right track.

Most of my concern though is for converting interlaced HD content (like leno/conan/letterman and live music on pbs) down to DVD.

does anyone have a good script for those? i end up playing with the settings a lot. this was one of my scripts for an Austin City Limits performance (which was upconverted and, broadcast 1080i widescreen)

LoadPlugin("C:\progra~1\gordia~1\MPEG2Dec3.dll")
mpeg2source("P:\HDTV\pixies.d2v")
crop(16,0,-16,-8)
SeparateFields
SeparateFields
Lanczos4Resize(720,240)
SelectEvery(4,0,2)
doubleWeave
selectodd



just seems like i'm missing something and going overboard. but it worked and looked fine on TV.

I then encoded that with CCE and authored a 16x9 DVD, with original AC3 audio. Didn't have any sync issues because I didn't cut out any commercials. But I tried a lot of the info here on SNL from a couple weeks ago, and best I got was bad field ordering and a constant sync offset (improvement over a sliding one).

[FreQi]

The live programming that you're talking about is filmed natively in NTSC, which means you do not want to decimate it. If you do you'll get jitter and general baddness. I'd try something like this:


LoadPlugin("C:\progra~1\gordia~1\MPEG2Dec3.dll")
mpeg2source("P:\HDTV\pixies.d2v")
SeparateFields()
SelectEven()
Crop(16,0,-16,-8)
Lanczos4Resize(720,240)


This should give you a progressive scan 29.97fps video at 720x480, so you won't need to use pulldown.exe for 2:3 pulldown or drop-fram flags, but you'll probably still want to set the pregressive flag.

You could use Decomb.dll to run Telecide() in order to deinterlace it (do not decimate), but you'd be spending more processing time for something that's going to be less than what you get from simply SeparateFileds().SelectEven() anyway. It might look better tho. Try both, see what you prefer. LMK.

[Leehro]

yeah it's NTSC stuff like letterman, SNL. stuff that is natively 480i (probably slightly higher than 480 but still standard def, interlaced video)

i don't want progressive video. I want to keep it interlaced. Sometimes i want to convert to divx/xvid, and in that case yeah i want progressive.

but if it's originally standard def, interlaced video at 29.97fps, that's what I want to be able to put on a DVD.

[dvgeek]

This has been a very enlightening thread. I currently use the modified ProjectX for 59.94fps, which seemed to fix my sync issues. (My caps are mostly ABC-Alias and Fox-24).

I demux in ProjectX, encode in WMV9 single pass (for a faster processing - 2hrs HD in about 6 hours, XVID - 2pass takes about 24 hours for the same avs script, which is just a one-liner for loading the .d2v).

I use VirtualDubMod for encoding with Fast Recompress - no audio.

I addin the ac3 file after encoding - edit out the commercials and save the avi with a direct stream copy. I always have audio interleave at 32ms.

Does anyone know if the latest ProjectX releases have got the 59.94fps mod incorporated ? Thanks.

[FreQi]

Seriously, you should not be cutting commercials after you encode. You're easily encoding 17 to 20 minuets of material that you are going to throw away. Cutting commercials first will shave 30% or more off of your current encode times.

Also, for shows like 24 and Alias, you should also be Decimating it to 23.96fps. The show is origiannly filmed in 24fps, so you are not loosing anything. You only gain more bits to allocate to the frames and ending up with a higher quality encode.

[dvgeek]

Thanks for the suggestion. I've had so many audio sync problems when I edited out commercials with HDTV2MPEG that now I've given up on that tool. It's a great tool - fast and all - but sadly for me, I could never get audio sync - no matter what method I tried.

So, now I clean up the transport stream by demuxing with ProjectX, then use DGIndex to create the D2V file and use the latest version of AVISynth/VirtualDub to encode into WMV9 HD. Then I use VirtualDubMod to mux in the AC3 audio and cut out the commercials - I do not mind the extra time as it gives me audio in sync.

Also - if I use fdecimate() or any other form to decimate to 23.976fps, my video gets choppy as if it suddenly decides to switch field priority. So what if my video is 10% more - I get smooth motion by keeping the original frame rate. I've all but given up on decimating to 23.976fps or do edits before encoding. I'll keep experimenting though. Thanks again.

[Leehro]

actually your decimate problem could be because of commercials. if you're encoding with commercials (often interlaced content, or at least a different field/frame offset) that can screw with decimate.

i had some sync problems too, but i eventually used VDubMod to make all the edits, save the frame ranges into avisynth, and fix the chopped up ac3. after that i think it was just a constant audio offset. i'm not using hd2mpeg anymore at all. just dgindex, vdumbod, avisynth, and whatever the latest greatest ac3 cut fixer is.

edit: still looking for anyone with a good script for 1080i->480i

[nnigam]

Also - if I use fdecimate() or any other form to decimate to 23.976fps, my video gets choppy as if it suddenly decides to switch field priority..

Is your original source at 23.97fps. Before using Fdecimate, use Fdecimate(metrics=true) in the avisynth script. Then curser through your video. You will see a large change in the number when a new frame is found instead of a duplicate frame. You can use the largest of the small numbers as the threshold for fdecimate, and see how many actual distinct frames you have to get what the actual frame rate should be. Most of my hd stuff is 23.976 fps which has been converted to 29.97 by adding duplicating one frame, and then every other frame is duplicated. I also have some which are at 29.97 and only every other frame is duplicated. If this is converted to 23.976 it will be choppy.

I also agree with you that a little larger video to get better audio synch and smoother picture is worth it. Many times

[Ac3Dc3]

This has got to be one of the best threads on converting HDTV out there, thanks for all the info!

does anyone know where i can get FreQi's tool theeo? and is it up to date with the current version of DGIndex?

[FreQi]

Interesting that you ask about theeo. I was working on it last night for the first time in maybe 6 months. It is true that theeo exists, and it does work, pretty well. However, when DGIndex stopped putting AC3 in the name of the files it demuxed, it threw a wrench in there that I've been just working around ever since. I finally got tired of it and decided to fix it, but instead of fixing it, I redesigned the interface, and pretty much started over. I am currently trying to decide if I want to actually start completly over and use Visual Studio 2005 and .NET2.

If you're interested, I could get you up and running with what I'm calling "theeo-v0.00" as I rework the tool. In any case, I would like to get your feedback on it. Send me a PM.

[FreQi]

I've just posted the first release of theeo. I'd be interested in getting your feedback in that thread (http://forum.doom9.org/showthread.php?p=789248)

[coolyhu2]

Thx


~~

[G_M_C]

Hmmm, reading throught this thread becaus this week im gonna try my first .TS to XviD.

As I understand it it's only needed to use HDTV2Mpeg2, and than DVD2AVI to make it possible to open the stream in avisynth through the Mpeg2source() command ?

And if true does it really need to be (the special version of) DVD2AVI ? Or can DGindex (DGMPGDec) do the same thing ? This because this thread (http://forum.doom9.org/showthread.php?t=87809) says DVD2AVI has to be considerated "obsolete" (because it makes errors). And I've allready got it installed (and having several different versions of MPEG2DECx.dll is a problem waiting to happen). And if DGIndex can work, does the "hints" switch work too (for use with Colormatrix) ?

If I cant use my allready installed DGIndex/DGMPGDec, does the "hints-switch"(for the use with Colormatrix) work with that special version of DVD2AVI ?

I want to have this info, so i cam make my own AviSynth script, so the only thing I do need to be shure of is the bvest way to open the .TS in the script :)

[squid_80]

DGIndex can handle unprocessed .TS streams ok, as long as there's no corruption (common with DVB-T, I don't have experience with anything else).

[FreQi]

There are of course several ways to get your .TS's into an .AVS script, but the "best" way is purely subjective. Some people use HDTVtoMPEG2 to do their trimming, others use ProjectX. In either case, I would suggest that you use DGIndex to generate the .D2V that you load in your .AVS since it is true that DVD2AVI has been pretty much abandonded in favor of DGIndex.

The way I process the .ts's has changed in the recent past. I used to use just DGIndex then cut the video in AVISynth and the Audio with VDMod. This was perfect in most cases, but would ocassionaly result in desync'd audio, especially if there were problems with the recording (drops, etc). It seems the more predictable method that I have been using is to use ProjectX to trim the unwanted segments then demux the .TS to it's MPV and .AC3 elements. Because ProjectX "fixes" drops and maintains A/V sync, you never have to worry about things like AC3 Delay anymore because it's allways 0ms. Then I use DGIndex to generate a .D2V to open then .MPV with AVISynth for trans-encoding since DGDecode is faster than other mpeg2source() plugins. The main downside to this process is the near doubling of hard drive space required to do the work, but it's a small trade-off for the reliability of the end product.

[G_M_C]

Well thx guys :thanks:

All the answers I needed. So hope to be able to give it a try this weekend :)

[jmac698]

I record the .ts, then use mpg2cut2 to take out commercials. This app is very fast as it simply does a binary copy of the remaining parts of the file, there is no processing at all. The only thing fancy about this file splitter is that it understands mpeg2 enough to find the exact byte to cut the file on. This byte is called a "GOP boundary", roughly speaking. A GOP is a set of, say, 15 pictures that can be independently decoded from any other part of the file, so you can call it one "atom" of film.
This should tell you the disadvantage of this method - you can only cut to the nearest 15 frames or 1/4 of a second, or so. This is usually good enough to cut perfectly.
The advantage is that there is no change in sync, because nothing in the file has changed, in a sense. It's also as fast as your hard drives. (hint - save to a different hard drive).
After this, just load in StreamClip and Convert to AVI, and your done! For full quality you can also Convert to MPEG2, this saves a bit of space but makes it a more compatible file, an mpeg2 (instead of transport stream).

[FreQi]

I can't offer perspective on a speed comparison, but it sounds like the same thing that HDTVtoMPEG2 (http://www-personal.umich.edu/~balazer/HDTVtoMPEG2/) (forum (http://www.avsforum.com/avs-vb/showthread.php?s=&threadid=395744)) would do. ProjectX (http://www.videohelp.com/tools?tool=ProjectX) also cuts on GOP, but it also corrects errors in the recorded stream, so I count it as a bonus. It'll also generate a .D2V for you, but it doesn't use the fancy format that DGIndex will make, so you have some options there. You can also use pjx to process to a new .ts instead of demuxing if you really wanted to.

[FreQi]

So after a lot of encoding, trying different apps and writing scripts and apps like theeo (http://freqi.net/theeo/) and tons of trial and error, I've managed to come up with what I consider to be a really simple and straight forward way of pre-processing HDTV Transport Streams to the point where you'll have an AVISynth script that you can feed into whatever encoder you want.

How To get Transport Streams into VirtualDubMod via AVISynth using ProjectX and DGIndex

What I have done is created a system that uses ProjectX as the starting point, then uses DGIndex to generate the .d2v project and the AVISynth script. Then, just to take it one step further, I made it open the .avs in VirtualDubMod so I could easily set up the encode jobs. The funny thing is, it's all bundled together by a windows batch script. Here's how...

First, I made a directory structure like this:

C:\Program Files\Encoding
C:\Program Files\Encoding\DGIndex
C:\Program Files\Encoding\VirtualDubMod
C:\Program Files\Encoding\ProjectX

In the DGIndex directory, I put DGindex 1.4.8 (found here (http://neuron2.net/dgmpgdec/dgmpgdec.html)), and two AVISynth Templates which I've listed below (modify them to your needs, but these are what I use and the resolutions I like to encode my XviD's to).

720p-to-FILM.avs

LoadPlugin("C:\Program Files\Encoding\DGIndex\DGDecode.dll")
MPEG2Source("__vid__")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Decomb.dll")
SelectEven().Decimate(cycle=5).Crop(4,4,-4,-4).Lanczos4Resize(960,528)


1080i-to-FILM.avs

LoadPlugin("C:\Program Files\Encoding\DGIndex\DGDecode.dll")
MPEG2Source("__vid__")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Decomb.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Uncomb.dll")
Uncomb().Decimate(cycle=5).Crop(4,4,-4,-8).Lanczos4Resize(960,528)


In the VirtualDubMod directory, I put VirtualDubMod 1.5.10.2 build 2542 (found here (http://sourceforge.net/projects/virtualdubmod)).

In the ProjectX directory, I have a couple things. First, I extracted the "launch.exe" program dvd.matt linked me to (found here (http://forum.dvbtechnics.info/showpost.php?p=15262&postcount=3)). I also created two batch files that reside there.

MakeD2V-720p.cmd and
MakeD2V-1080i.cmd

@echo off
echo.

IF [%1]==[] GOTO NoArgs
set STRIPTIN=%~dp0

:: this is the template DGIndex will use to generate your AVISynth Script.
set AVSTEMPLATE=720p-to-FILM.avs

::the only thing you can pass is a filename, so if there are
::more "args" then assume it's a filename with spaces.
set INFILE=%1
:getname
shift
IF "%1"=="" GOTO gotname
set INFILE=%INFILE% %1
GOTO getname
:gotname
::launch.exe seems to put quotes around the filename
::so trim the double quotes off both ends
set INFILE=%INFILE:~1,-1%
echo. Source File: %INFILE%

:: trim the file extension off the end
set OUTFILE=%INFILE:~0,-4%
echo. Output Base: %OUTFILE%

set DGINDEX="%STRIPTIN%..\DGIndex\DGIndex.exe"
set ARGS=-IF=[%INFILE%] -YR=1 -FO=0 -OM=2 -AT=[%STRIPTIN%..\DGIndex\%AVSTEMPLATE%]-OF=[%OUTFILE%] -exit

echo. Launching DGIndex to make .d2v project file...
echo %DGINDEX% %ARGS%
echo.
%DGINDEX% %ARGS%
echo.

IF NOT EXIST "%OUTFILE%.avs" GOTO NoAVS
IF NOT EXIST "%STRIPTIN%..\VirtualDubMod\VirtualDubMod.exe" GOTO NoVDMod
echo.
echo. AVISynth Script found, opening in VirtualDubMod...
IF EXIST "%OUTFILE%.avs" "%STRIPTIN%..\VirtualDubMod\VirtualDubMod.exe" "%OUTFILE%.avs"
GOTO end

:NoArgs
echo.
echo This script is meant to be run by launch.exe demuxing with ProjectX. To use
echo it, open the PreSettings and on the postprocess tab and put something like...
echo.
echo. "C:\<pathTo>\launch.exe" open "C:\<pathToThisScript>\MakeD2V.cmd" ?1
echo.
echo in the demux line. Also, on the ProcessWindow (click "prepare>>"),
echo make sure you have "enable post processing" checked under the Edit menu.
echo.
pause
GOTO END

:NoAVS
:NoVDMod
:end
::exit


The only difference between the two scripts is the value of AVSTEMPLATE. 720p uses the 720p template, 1080i uses the 1080i template (duh?). The ProjectX directory also contains two copies of ProjectX 0.90.04.00 (found here (http://www.oozoon.de/main_en.html)) that are in their own subdirectories cleverly named \720p and \1080i (see footnote1).

So, I extracted one copy of ProjectX to ProjectX\720p and set it up the way I like (see the "First Time Configuration" section of my guide (http://freqi.net/tutorials/projectx/) for details). In addition to that, I put the following command in the "demux" box under the postprocess tab of the PreSettings:

"C:\Program Files\Encoding\ProjectX\launch.exe" open "C:\Program Files\Encoding\ProjectX\MakeD2V-720p.cmd" ?1

When I was done, I copied the 720p directory to the 1080i directory, but I opened up ProjectX and changed the demux line under the postprocess tab to use the MakeD2V-1080i.cmd script instead (make sense?).

To finish the setup, I just created shortcuts to the ProjectX binary and put them in my quicklaunch for easy access.

So, now when I have a show I want to encode that's been recorded from a 720p source, I just open that flavor of ProjectX, drag-n-drop the source files on it, cut the commercials (as outlined in my guide (http://freqi.net/tutorials/projectx/)) and click the "Quickstart" button. After a couple minutes of processing, DGIndex opens up, makes the d2v and avs script then opens it up in VirtualDubMod for me to double check and setup the encoding job (see footnote2).

Done. Wha'd'ya think? :cool:



footnote1
Why two copies of ProjectX? Because Interlaced video needs to be processed differently than Progressive video, and to make that happen, i needed ProjectX to lauch a different batch script when it's done demuxing. The only way to do that is to either
(a) change the Post Processing command every time I switch from I to P or back,
(b) create different projectX .ini files and load them from command line or in the target properties of a shortcut to the jar (ie: C:\WINDOWS\system32\java.exe -jar "C:\Program Files\Encoding\ProjectX\ProjectX-v0.90.04.00-20060330.jar" -ini "C:\Program Files\Encoding\ProjectX\720p.ini" -gui)
or
(c) you can just make two copies, one with the settings for 720p material and one for 1080i.
It seemed like less work, so I opt'd for c.

footnote2
Instead of opening VirtualDubMod, there is a ton of other things you could make the cmd script do. You could launch x264.exe to encode it, or feed the avs to avs2avi instead. Go nuts.

[Livesms]

I have HDTV (1080p) video (Gladiator) with 2 audio streams (eng and rus) - 21Gb. PID 011 fo video, 014 for rus audio and 015 for eng.
I want to reencode it with x264 to fit 2DVD.
I tried to put it to Avisynth using d2v generated by DGMPGDec 1.4.9 Final but it always says smth like filed order was incorect (generating d2v) and then eng audio stream is shorter then rus (2:34:58 vs. 2:33:51). d2v files generated by DGMPGDec 1.4.9 Final is differs when i generate it using Streams 011, 014 and when i do the same task with 011, 015.

How can I reencode correctly?

[FreQi]

I don't think you need to worry about the runtime for the audio being different by one minute. There's probably just no audio at the end durring the credits or something.

I also think you might want to take your question to a thread about DGIndex. This thread is more about ATSC HDTV Broacast Transport Streams, not HDDVD/Bluray (although you should be able to use much of the info here).

[Livesms]

I don't think you need to worry about the runtime for the audio being different by one minute. There's probably just no audio at the end durring the credits or something.

I also think you might want to take your question to a thread about DGIndex. This thread is more about ATSC HDTV Broacast Transport Streams, not HDDVD/Bluray (although you should be able to use much of the info here).
I checked audio in original ts for both PID - both in eng and rus audio run till the end of the video and ts has length of 2:34:58

And seems like AC3 audio generated be DGindex is not the same audio i hear with original TS file.
I just put the slider into random position (for ex. 0:24:40) and TS audio is far differ (by time I mean) from AC3.

When I use HDTVtoMPEG for converting TS to MPEG i have the same problem - video is trancating to 2:33:51 (from 2:34:58).

[FreQi]

Well, like I said, you'll probably want to take this to a DGIndex thread...

[Randall]

Does anybody here know the best way to convert 1080i30 ATSC to 24fps progressive using MEncoder? Is this even possible with avisynth? I know it's possible to get 720p60 down to 24 fps progressive, but I have a feeling that this is close to impossible with 1080i. Thanks.

[jobe228]

hey guys, i need a little help

I am trying to convert a .ts source (1920x1080i, 29.97fps) to NTSC DVD format (720x480). The Audio is AC3 5.1 so I would like to keep that unchanged unless someone recommends switching to PCM. I've tried procoder but it keeps de-interlacing the file. Shouldn't I keep it interlaced or should I make it progressive?

Aviscript? Virtualdubmod?

Thanks for any replies

[Adub]

Well, you can use DGIndex to create an index file that Avisynth can use, and then you can feed it directly into HCenc, Quenc, or whatever your favorite Mpeg-2 encoder is.

You should be able to keep the ac3 file, just make sure it is up to spec with the DVD specifications. You can always use Aften do re-encode it if you need to.

[Toti]

Actually is quite simple, use DGIndex to make a .d2v then use avisynth to do inverse telecine your video back to its original 24fps.

NOTE: This is only good if the video was originally shot at 24fps then used pulldown to make it 29.97/59.96fps If your video was shot at 29.97fps and true interlaced (like football games) then simply deinterlace but stay at 29.97. TV shows like CSI, Heroes, 24 etc... then its ok. Take out the commercials first since almost all commercials at native 29.97fps.

Anyway, use this for 29.97 -> 23.976

LoadPlugin("C:\Program Files\AviSynth 2.5.7\plugins\DGDecode.dll")

LoadPlugin("C:\Program Files\AviSynth 2.5.7\plugins\Decomb521.dll")

mpeg2source("C:\Movie\MainMovie.d2v", upConv=1)

Telecide(1,guide=1,vthresh=0)

Decimate(cycle=5)

FieldDeinterlace(Full=False,threshold=15,dthreshold=9)

Crop(4,4,-4,-4)

BilinearResize(1920,1080)

or from 59.96 -> 23.976


LoadPlugin("C:\Program Files\AviSynth 2.57\plugins\DGDecode.dll")

LoadPlugin("C:\Program Files\AviSynth 2.57\plugins\Decomb521.dll")

mpeg2source("C:\Transporter 2\Main Movie\MainMovie.d2v", upConv=1)

Telecide(1,guide=1,vthresh=0)

Decimate(cycle=2)
Decimate(cycle=5)

FieldDeinterlace(Full=False,threshold=15,dthreshold=9)

Crop(4,4,-4,-4)

BilinearResize(1920,1080)

The last two lines you don't need but I have found that sources from DirecTV - DishNetwork - Over The Air always need to cut some black lines on sides which lines and how much depends on how bad they mess them up.

[hkazemi]

720p-to-FILM.avs

LoadPlugin("C:\Program Files\Encoding\DGIndex\DGDecode.dll")
MPEG2Source("__vid__")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Decomb.dll")
SelectEven().Decimate(cycle=5).Crop(4,4,-4,-4).Lanczos4Resize(960,528)


1080i-to-FILM.avs

LoadPlugin("C:\Program Files\Encoding\DGIndex\DGDecode.dll")
MPEG2Source("__vid__")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Decomb.dll")
LoadPlugin("C:\Program Files\AviSynth 2.5\plugins\Uncomb.dll")
Uncomb().Decimate(cycle=5).Crop(4,4,-4,-8).Lanczos4Resize(960,528)



You are resizing from HD (height >=720) to SD (height < 720 lines) resolutions...a good portion of codecs, players, software, and renderers appear to use different color matrixes for HD and SD. The effect is seen as a color shift, usually most noticeable in greens and reds. (HD resolutions nearly always use the BT709 color matrix, while SD resolutions stick to BT601. MPEG streams are supposed to specify this information, but sometimes they don't, so most devices assume what I've just described in the absence of specific information.)

The solution is to use a color matrix changer like Avisynth plugins Colormatrix() or BT709toBT601() when resizing from HD to SD. If you use Colormatrix, it is supposed to be used early in the script, before deinterlacing. It also needs to explicitly be told whether the clip is interlaced or not.

In case you're interested in more, I have posted a thorough summary article here on this topic:
http://forum.doom9.org/showthread.php?t=133982