Field order problems with xvid/divx capturing in V-dub (Swan?)

[begu]

Hi, I'm having problems in capturing directly using xvid / divx codec.
The problem is that field order depends on the time I start recording.

I have tested latest Nic build of xvid and divx502 (503 to try today). Let me describe the problem in detail.

I can get perfect captures when using those codecs. I capture full PAL resolution (720 x 576) 25fps. If I get perfect capture (the field order is correct) the result viewed trough TV is almost the same as live. But when I get bad capture (wrong field order), the playback 'stutters' (I don't know proper word).

I'm using sigma designs x-card for playback. It has great image quality, but drivers/software are not best in world (it is an mpeg1/2/4 harware decoder card). So it outputs just proper resolution, so there is not any conversions done.

When I get good capture, I believe captured fields in frames go like this:
frame1: 1,2 / frame2: 3,4 / frame3: 5,6
But in bad capture they go like this:
frame1: 2,1 / frame2: 4,3 / frame3: 6,5
So You can imagine how that looks.

I get nice interlaced capture if everything goes well. I don't want to deinterlace, because it destroys the video motion smoothness and crispness. I'm using latest v-dub, but not latest WDM drivers for my capture card (gf4 myvivo). I try newest drivers today.

I think taht the problem can lie in the following locations:
1 - WDM drivers: field order is lost or something
2 - v-dub: it feeds somehow fields in wrong order
3 - video codec: the fields are not encoded properly
4 - playback: assumes top or bottom field first

Has anyone else experienced this kind of problem? The success is depending only in luck: sometimes it is good, sometimes not. It all depends on the time I start capturing and what field is currently present in video.

There should be somekind of dynamic correction of field order. I found filter for vdub, that does just it, but after capturing, and it didn't fix the avi's that were captured using xvid/divx. I believe it only works for mjpeg or something.

There was a theread here about exactly the same problem:
http://forum.doom9.org/showthread.php?s=&threadid=31812

Found just the same thing in virtual dub forums:
http://virtualdub.everwicked.com/index.php?act=ST&f=5&t=1339&hl=field,and,order&s=50f95b934586dfa40e7ffc00821f3cc8

I'll try with newest drivers and with divx503. Is there any other suggestions?

[Swan]

I don't capture in DivX or Xvid, I've only captured using Mpeg-2 compression and now, Canopus DV compression via the ADVC-100 I am playing around with.

I cannot tell you what the problem is, but have you taken into account that it could be a playback problem?
If your file is Top Field First and the decoder or graphics card sends it to the TV Bottom Field First, then you will see strange "shaking" or "stutters" like you say, as soon as something moves. On stills, it looks OK. But movement looks weird.

When one captures, at least in Mpeg-2, the field order is written into the file. Some software let you choose the field order, others assume it's Top Field First. This is because almost all capture devices, except DV devices, captures Top Field First.

When the playback device (the decoder) gets the Mpeg-2 file and is going to read, it reads "hmm, Top Field First, OK I'll output the video that way" or "I see a Bottom Field First flag here, so I'll output it like that".

One gets terrible problems if this gets screwed up. If the capture device captures Top Field First and the encoder writes into the file that it is Bottom Field First, then you have a playback mess!
Or, like I just discovered, the graphics card, which takes care of TV-Out, can also be the culprit.

Nowadays, I capture with a Canopus ADVC-100 and it captures Bottom Field First, like all DV devices. My graphics card is a Matrox G450 eTV. It can apparently only ouput Top Field First (!). All videos that I have captured with the ADVC-100 and then output from the computer to the TV "stutter". I have to load a special Directshow filter called GDivX 400, which shifts the lines one row, to get a workaround for the Matrox outputting Bottom Field First material as Top Field First!

If I were you, I'd experiment with other codecs. Try capturing with Mjpeg compression, see if you still get the field order problem (if that's what it is). Also, try Ulead VideoStudio or DVD Workshop, where you can set the Field Order before you start the capture. This is just for testing purposes, I am not trying to sell you Mpeg-2 capturing. :)

I don't know if DivX and Xvid has support for writing into the file which field should come first. Maybe someone else can fill that info in here?

You are not using TVTool, I guess? I've heard it can cause some problems with interlaced video. ;)

Another idea: If you have a DVD player, and it can handle SVCD, take one of the problematic clips and create an SVCD out of it.
In the encoding software (TMPGEnc for example, don't use CCE in this case, it will only add confusion), make sure you set the Field flag to Top Field first ( I believe your capture device captures Top Field First). If this stutters on the TV, when played on your standalone DVD player, encode it Bottom Field first.

Does the video stutter when you play it in Windows too? Or just when viewed via the TV-out?

/Swan

[begu]

Thanks for reply.

Nice to have that ADVC-card to work for You. But it is quite expensive, but hey, Isn't it so that "if You are poor You should only buy expensive things" ;)

I'm using Sigmadesigns x-card for playback. It's good HW and has amazing image quality, but SW and drivers aren't so 'fine' as hey should be. That card is the only way to test proper interlaced behaviour, because I have gf4 with philips SA7108E TV-chip. Philips chip is not Yet supported by TV-tool to have special resolutions like true PAL. And if I launch TV tool to view avi from TV, I think it is first resized to match 800x600 and then back to proper TV resolution.
But x-card outputs avi without any conversions and quality is almost same as compared to real live broadcast. This is when the divx/xvid is encoding in interlaced mode properly. Motion is screwed badly, if the captue is 'bad'.

Well I'm not sure, but it seems that somehow it is the xvid codec or WDMVFD-wrapper causign trouble. I found out if I don't use vdub's capture, and use virtual vcr instead, and divx503 codec I got better results. Things are somehow screwing up if I use xvid or vdub for capturing. hmmm.. Seems that I should reinstall XP soon :(

Well I need more time to test. Divx takes bit more resources than xvid, and it seems to dynamically control if it uses interlaced encoding or not. Maybe it depends on bitrate. (I assume that interlaced encoding takes more bitrate to maintain same 'quality' compared to progressive encoding) Sad, that there is no 'force mode' for interlacing like in xvid.

And after analyzing more the werid movement of those 'bad' clips, maybe it is not the field order wrong, there is something very weird. It looks like bad interlacing (blended fields) and wrong field order are mixed together or something. Don't know what causes that. Very odd is the fact that if I got good capture using VVCR and divx503, and tried xvid in VVCR i got bad result. And then switching back to divx503 there is no video at all in captured avi! And I needed to reinstall divx503 to get it working. Considering the above, maybe the bad captures aren't caused by WDMVFD-wrapper. Maybe the xvid and divx503 can't live together, or something like that.

Well I must try mjpeg and huffyuv, but I'd really like to have this divx/xvid thing to working. I have the Ulead Videostudio 6 bundled software, that came with my gf4 card. But I can't use it, because the x-card software makes it to crash before it opens. This is bad issue, and it is caused because Ulead tries to get proper dll-entries through registry rather that directly from disk. (x-card software does something to mpeg2 information in registry)

My point in using xvid/divx is to have digital VCR like this:
record -> watch -> throw away. So those codecs suit very well for this kind of use. I'm not intrested to archive these kind of recordings. I could use mpeg2, because it seems to work everytime, but the ulead doesn't work. I tired mpeg2 with windvr, but it doesn't support interlaced encoding. I might buy new HD and install ulead there but not x-card software. And then make manual switch for my HD's so that I can switch betweed slave and master to load proper OS (for recording mpeg2 and wathing) but it is quite a hassle.

Well I must test more, and maybe reinstall OS, uh :(

[Swan]

Hey begu,

Your post is packed with info and details.
Let's try to sort this out.I wouldn't reinstall Windows over a thing like this. In my view it's better to first establish what the problem is, what causes it and if it is likely that a reformat will solve it, before getting drastic.

Let's keep this simple (I get confused easily :))
So.. First, we must establish whether the files are corrupt or not. If indeed the fields are messed up *on the actual recordings* or not.
If you can upload a small portion of a problematic file on the web somewhere, I'm sure there are plenty of people on this Forum who are willing to inspect your files (me included) to see if they are abnormal in any way.

Depending on the findings there, we can determine if it's a capturing problem or a TV-Out problem. If we can verify that the files are A-OK, then the next step is verifying that the Sigma card you have isn't doing anything it shouldn't. This is something only you can do. You should convert your DivX/Xvid files to SVCD and try in a standalone DVD player, experimenting with Top Field First and Bottom Field First flags. That's the easiest way. If you don't have a standalone DVD player, maybe a friend does?

But I can't use it, because the x-card software makes it to crash before it opens.
Yes, this happened for me also, and a friend of mine, with the 6.0 version. Download the 6.02 patch from Ulead's site and see of that won't solve the issue. It will also greatly affect quality, as the Mpeg encoding is vastly improved in 6.02. The crash at startup problem is related to a file called uvavi.vio. Once that is replaced with the 6.02 version, uvavi.vio no longer caused VideoStudio to crash at startup here or on my friend's computer.
http://www.ulead.com/tech/vs/vs_ftp.htm

If you can get that to work, then that would be great. The DVD preset in both VideoStudio and MediaStudio gives awesome quality. Some people who've never tried it of course doubt it, but even on-the-fly recordings looks stunning. ;)
WinDVR is a piece of cr**, since it deinterlaces, you are absolutely right about that. I've not used it for that very same reason.

Is it a capturing problem?
Or a TV-Out problem with interlaced video?
Let's establish what the problem is!

/Swan

[begu]

Great answer.

However I don't have place where to send the clip. Maybe only way is email or something. I suppose that only few frames will be enough. I did some tests more. And it seems that the problem is not the divx/xvid codec. I got bad captures using virtual vcr too, so the problem is not in WDMVFW-wrapper.

When I examined bad clip in virtual dub with "separate fields" filter, it is very clear, that field order is wrong. I describe it here somehow:

Good capture:
- using separate field filter
- detecting the direction of movement by wathing few frames
- bottom field comes first, this is seen very clearly

Bad capture:
- same procedures in filters and watching
- upper field comes first, very clearly seen too
- in first frame there is no difference between fields !
- so here we have irrational movement when viewed with TV

So I tried to fix that with swap fields internal filter. It did fix the movement so that in bad capture bottom field comes first. But the lines are 'shifted' oddly, so image is not good in edges of objects. I found new term in smart deinterlace filter called "phase shift". That makes sense,the actual problem might just be not "wrong field order" but rather "pahse shift problem". I must find pahse shift filter for virtual dub.

This makes sense, baceuse if the phase is shifted by one field (I don't know in what direction, back or forth), I think that swap fileds doesn't correct problem, as I had found out before. It corrects the movement beahviour, but edges are not correctly shown.

Good capture:
frame: A B C
fields: 12 34 56

Bad capture:
frame: A B C
fields: 23 45 67 (here the fields 2 and 3 are the same)

Bad capture using 'swap fields' internal filter in vdub:
frame: A B C
fields: 32 54 76 (motion corrected, but bad combing on object edges)

Bad capture propably fixed with some kind of 'phase shift' filter:
frame: A B C
fields: -2 34 56 (here the field "-" is blank (black) for example)

or like this (with 'phase shift' filter):
frame: A B C
fields: 34 56 78 (here the field "2" is lost)

If I'm correct about this, the phase shifter will help. But I might be wrong too :(. But I'll try to find proper filter for v-dub. Anyoe knows such one? I think using avisynth this would be easy, but I haven't used it yet. Seems complicated, but I might learn how to use it.

I'll post my findings tomorrow. But I'll also try to make one 'bad' clip with few frames to send it by email, if You are intrested.

I tried to install that 6.02 update, but still it crashes. Maybe I'll have to buy new HD and isntall there ulead with no xcard soft.

Edit: I found something:
http://www.stud.uni-karlsruhe.de/~usng/virtualdub-filter/FieldDelay_Doc/index.html

I'll try that. It looks promising :)

[begu]

Ah, found fix for this field phase problem.
It really was wrong field phase in my bad captures.

I used the "Field delay" filter in virtual dub and it perfectly corrected motion issues. I used it to delay bottom field by one frame. And after filtering the result looks like there never was anything wrong :) Only downside is that I must recompress my bad captures and that decreases quality and takes time too. I hope there should be something other solutions in future. Here is my suggestions:

1 - make somekind of field phase detect in capture program (virtual dub) so that encoder (compression codec) starts always in correct phase.

2 - make samekind of field phase detect to compression codec

3 - implement field delay filter in video decoder, for example in ffdshow (I think it would be quite easy to do). It could also have somekind of dynamic detection whether to delay which field or do nothing.

4 - hope for driver fix from nvidia -cough- I really think that this is not real suggestion ;)

5 - hope for special drivers like (btwincap for bt-chips) from somewhere / someone

I think that the easiest way is to do field delay in decoder (ffdshow) and second easiest is to implement fix in capture program (vdub or other). Well I might to write to ffdshow and vdub developers about this. And I know that it's not worth of contacting nvidia. I guess if I get response it is something like "get new drivers or update Your OS". Unfortunately their WDM drivers lack even support for PAL ! Yes, I had to manually edit .inf-file in driver dir to make them install as PAL-drivers. All of the drivers downloaded were defaulted to NTSC. At least in Leadtek retail-package there were PAL drivers.

So I'm quite happy now. I know the problem and also have an fix for it. But it would nice to see some of my suggestion in reality in future. Or I might just be purchasing bt-chip based card. But it is nice to have capture-capability in display adapter. Thanks for Swan for info.

Swan: may I ask You couple of questions?
I guess You had same field problem in the thread what I included in my first post, or am I correct? You had also somekind of weird movement on TV, when tired to capture interlaced video. What capture device did You use by that time?

You said Your DV-capture is working very good. Have You had problems after win XP SP1 installation? If I remember correct there were issues with some DV-capture cards with SP1. Can You use virtual dub for editing or converting DV-captures to mpeg2/4?

[Swan]

Begu, so it is a driver issue then?
You can fix it afterwards, but it is caused by Nvidia's drivers for your Geforce capture card?


Swan: may I ask You couple of questions?
Surte thing! Ask all you want. :D

I guess You had same field problem in the thread what I included in my first post, or am I correct? You had also somekind of weird movement on TV, when tired to capture interlaced video. What capture device did You use by that time
I read my original question in that thread (thanks for adding the URL) and no, I didn't have the same problem as you. I've never had any problems with field order/field phase being screwed up on my captures. The thread was about how I could convert Mpeg-2 videos (720 x 576) to a format that compresses better than Mpeg-2, yet can fit as much video on a CD as DivX and keeps the interlace structure.

I was wondering if DivX and Xvid had support for interlacing at all, bascially. After that, I did a test that suggested the formats did not support interlace. But I was an idiot and accidentally resized the video I encoded to a resolution that had less than 576 in vertical resolution, so naturally, interlace was not preserved! :)
Since then I have encoded clips (for test purposes) with 480 x 576 and 720 x 576 resolution to both DivX and Xvid and yes, both keep interlace structure on the files and play 100% correct on my TV-Out. My mistake, and what caused my confusion as to whether or not interlace was supported in Xvid and DivX, was based on my stupid attempt when I resized the video to less than 576. The interlace structure was messed up because of resizing, that's why it looked so weird on the TV. I felt like an moron, but I'm over it now. :D
I had a Pinnacle PCTVPro then, I think (still have it) or maybe an AverTV Studio (both BT, both capture Top Field First).

You said Your DV-capture is working very good. Have You had problems after win XP SP1 installation? If I remember correct there were issues with some DV-capture cards with SP1.
I am on Windows 2000 SP3, but yes, as I understand it from the many posts on Canopus's ADVC forum, there is an issue with Service Pack 1 for XP.Can You use virtual dub for editing or converting DV-captures to mpeg2/4?
Yes!! It's so easy, even a 3- year old could do it. Well, not really, but.. :) I edit with VirtualDub and have tested both CCE and TMPGEnc to create DVD-ready Mpeg-2 (hopefully, as I don't have a DVD writer to test the compliance with yet) and also converting a DV Avi compressed movie to DivX with GordianKnot.

And the sync, aah, the sync.. It's always perfect! This was actually why I bought the ADVC. I got so tired of bad sync, trying various workarounds, always ending up having to go back to Mpeg-2 direct capturing (it always kept the sync) that I felt sick to my stomach. The ADVC is expensive as you wrote earlier and has its drawbacks, but you'll never find anything that's easier to use, I'm sure. I mean, even the washing machine I have is more complicated. :)

All that is needed to be able to edit the files in Virtual Dub is a DV VFW codec. Vdub cannot use the "DV Video Renderer" which is Microsoft's DV playback codec. And finding a free DV VFW codedc was real easy. :-) I read the rundown on codecs on the DV forum and went with Panasonic's codec.

I am going to write a review as soon as I feel that I know the ADVC-100 a little better. I have for one thing not tested it with VHS material yet. I capture from my satellite digital receiver and for that it works like a charm. But the audio is a bit of a problem. I would recommend anyone who buys an ADVC and is as picky as I am, to also buy a DJ mixer, with many level indicators on. So one can set the recording levels and adjust them before capturing. I get about -12 DB on my captures from the "digibox" and it's way too low for my taste. I want as much of the dynamic range I can get!
I will buy a mixer soon.:)

If there's anything else you want to know, feel free to ask.

/Swan

[begu]

Yes, it is driver issue. I can fix it afterwards. But is is pain in the *ss to re-encode just because of the wrong field phase.

I discussed by e-mail with Avery Lee (vdub) and he also stated it seems to be driver issue. And also he stated that vdub cannot access the field combing order, that the driver creates. So there might be no way vdub to detect correct filed phase, and adjust proper start momonet for capturing.

I also discussed with other person who also had gf4 and just the same problem. He bought bt-based capture card, and with btwincap drivers everything was correct in vdub capturing. (but some problems occur with TV-viewing programs using btwincap drivers) He also pointed out that using mjpeg codec those line artifacts were bad to compress if the filed phase was wrong. But I'm using xvid and it separates frames to fields and those compress properly. But I think that there is option in mjpeg codec to compress "two fileds if more than xx lines".

So it seems that I'm stuck. I have to wait for driver fix, but that might take too long. Best solution would be post-capture fix -> ffdshow. I' going next to write them. Only thing to do is field delay filter in ffdshow. Then I could delay bottom field when needed.

One thing to ask about avisynth:
Is it possible to easily delay bottom field with that?
I guess something similar can be done but differently.
Maybe to first separate frames to fileds and then cut the
first off. Then pack them again to frames. But that would need decode and encode of the video so it is out of the question.

Maybe someday I'll buy that ADVC :)

[begu]

All Ok now.
I noticed that if I use x-cards internal hardware decoder it takes filed phase in just opposite order compared to using Joveplayer. Joveplayer uses ffdshow and special re-encoder and then sends video to hardware decoder.

Now I can just change between pure hardware decoding and software->hardware decoding. (when HW is 'bad' then use SW->HW to get it good and if SW->HW is bad, then use HW) This is very easily done in Joveplayer configuration. So I'm very satisfied and suprised about the quality.
If I let the bitrate be over 6000 kbps for xvid, it's very hard to tell if it's live or recorded video :)

One thing still is wrong with Ulead Videostudio 6 SE.
I tried that before for mpeg2 recording. Have anyone here got problems to capture full PAL resolution??

There is option in capture settings called "frame size" and there is no option for 576 lines, only 480. It seems to be designed in NTSC world indeed. There is PAL of course in template options, where I create PAL 720x576 template with audio and video codecs and settings. And I can set the source to 720x576 lines, and it shows correctly. But now if I start capturing, it captures in awful 160 lines resolution! This is because I can't find 576 lines for "frame size". I tried last update, but no solution found.

All this is done to fresh platform, installed win XP and needed codecs and software for capturing. Well if anyone knows any tricks for Ulead V-studio, then please let me know. However I seem to get better video quality by using mpeg4. But properly working V-studio would be nice.

[Swan]

There is option in capture settings called "frame size" and there is no option for 576 lines, only 480.
I don't know the difference between the SE and the full-blown version, but your problems sound odd!

When you start the program, do you enter an almost full-screen window first called "Start"?
If you do, do you select "new project"?
A window with pre-defined settings will pop up. Select "PAL DVD" and click OK.
Videostudio now goes to the "Capture" window.
From there, you can start capturing right away.
Are your Mpeg-2 files, captured with VideoStudio not 720 x 576 when viewed in Mediaplayer 6.4 (for example)?

Begu, please descibe the steps you take when capturing in VideoStudio more precisely and I may be able to help.

/Swan

[begu]

Thanks, I'll try to test that out today.
As far as I remember, I did things like You described.
But the result is 160 lines.. very odd indeed.
Well I'll report, when I check UVS again.