Change the pulldown flag of a VC-1 ES

[AEtH]

As I wrote, is there any way to change the pulldown flag of a HDDVD compliant VC-1 elementary stream?? I know that the video is 1080p24 with a post-processing pulldown flag, and I want to know if it is possible to change it to an elementary stream without this flag.
I don't want to mux it in a mkv, so the timecodes file does not work for me.

Thanks in advance

[crypto]

I think that is possible. Its just a bit in the sequence header.

I already thought about that for the vc12avi tool. Currently the avi results in 29.96 fps. Removing the pulldown it would result in 23.976 fps.

[zambelli]

I think that is possible. Its just a bit in the sequence header.

I already thought about that for the vc12avi tool. Currently the avi results in 29.96 fps. Removing the pulldown it would result in 23.976 fps.
It's more than just in the sequence header - there are also TFF/RFF flags in the bitstream that need to be stripped out, otherwise you might confuse the decoder.

I would guess the most knowledgable person on the subject would be Neuron2 because he wrote DGPulldown for MPEG-2 - and VC-1 pretty much functions exactly the same way.

[crypto]

It's more than just in the sequence header - there are also TFF/RFF flags in the bitstream that need to be stripped out, otherwise you might confuse the decoder.

Oh, thanks for the info. I see changing patterns of the TFF/RFF flags.

It would be nice, if Neuron2 could clear up how to set these flags to remove the pulldown.

[Golgot13]

As I wrote, is there any way to change the pulldown flag of a HDDVD compliant VC-1 elementary stream??

Yes, I know two little exe which can do this: one from Inlet and another one from MS.
This two software are used to convert HDDVD compliant stream to BD compliant stream
(thank Warner about MS version for BD title).
BD format don't support 3:2 pulldown, it support directly 23.976fps and 24fps :D

[AEtH]

I think that is possible. Its just a bit in the sequence header.

I already thought about that for the vc12avi tool. Currently the avi results in 29.96 fps. Removing the pulldown it would result in 23.976 fps.

The idea came out from vc12avi xD. I did yesterday some research about VC-1 headers, and as I could find it seems that the problem is "frameratenr: 3". Changing it to 1 seems to change the framerate but as zambelli pointed the flags in the bitstream should be stripped out. (I did not know that).

Well... any ideas? It would be a nice mod for vc12avi.

@Golgot13
I suppose that those little exe are not public, aren't they?

[Trahald]

i was thinking of adding vc1 pulldown support to my h264_info app (which probably should be called h264_pulldown). it wouldnt be quick tho probably cause 1. im a slow coder. and 2. im really busy. i havent been able to find any free apps that are able to do it yet.

[AEtH]

It's not a matter of time, of course, it's just being capable of doing it. It would be so great if anybody can implement that.

[zambelli]

I hate to pull the "it's easy but I don't have the time to do it" routine too, but somebody with knowledge of the VC-1 specification ought to be able to do it by looking at Neuron2's MPEG-2 pulldown implementation in DGPulldown: http://neuron2.net/dgpulldown/dgpulldown.html. The logic should be the same - one just needs to apply it to VC-1 format.

[drmpeg]

I had most of the code already written in xport, so it didn't take very much to modify it for this application.

http://www.w6rz.net/vc1conv.zip

Usage:

vc1conv <infile> <outfile>

For example:

vc1conv original_clip.mpv converted_clip.mpv

Just to be clear, the tool removes pulldown (not add it like dgpulldown).

For this first release, the tool does not attempt to make the bitstream progressive. That requires a lot more work. The tool just changes the frame rate from 29.97 to 23.976 in every sequence header, sets the TFF bit in every picture header and clears the RFF bit in every picture header.

Ron

[AEtH]

This is just what I was looking for! Awesome!
I'll start now testing the output of this little exe, let's see what I can obtain.


EDIT
After applying vc1conv to a test file,
vc1conv test.vc1 test_23.976.vc1
Converted video plays smoothly, framerate is 23.976... PERFECT
Awesome work drmpeg!!

To test the files I used vc12avi to multiplex them into an avi file

[Guest]

Just to be clear, the tool removes pulldown (not add it like dgpulldown). DGPulldown can remove pulldown too. :)

(But not for VC1.)

[vsv]

DGPulldown can remove pulldown too. :)

(But not for VC1.)

How about one tool for manipulation with flags of header for mpeg, h264, vc1 ? ;)

[crypto]

Hi Ron,

very cool tool. And thanks for sharing the details on how to set the flags.

[yesgrey]

For this first release, the tool does not attempt to make the bitstream progressive. That requires a lot more work.Ron

Are you considering adding it? or does it not matter?

[drmpeg]

Are you considering adding it? or does it not matter?
For what the OP wanted to do, it doesn't matter. However, if you want to be Blu-ray compliant, then it does.

It's fairly painful to implement, since you need to delete single bit syntax elements from the bitstream. Worse yet, my parser is pretty much byte orientated, so bit manipulation is a stretch for it.

BTW, I do have a small update to the original release. It just fixes a formatting bug with the bitrate meter (if the bitrate drops below 10 Mbps, it looks like it's saying 90 Mbits). I also made it a little faster and if you inadvertently feed it a Blu-ray VC-1 ES, it does the right thing (nothing).

Version 0.1

http://www.w6rz.net/vc1conv.zip

Ron

[AEtH]

For what the OP wanted to do, it doesn't matter. However, if you want to be Blu-ray compliant, then it does.

I'm not gonna feed any authoring program with the converted stream, so this program works so well for me.
But, anyway, what's the difference between an original 24p coded stream and this converted stream and why is not Blu-ray compliant??

I'm just curious :)

PS. And, of course... are you planning on implement this?? For me it doesn't care so much, but it would be a great improvement (but not an easy one).

[Trahald]

Blu-ray should be ok as it takes 23.976 fps feeds directly (at least afaik) hddvd being closer related to dvd still requires streams have flags to fake 29.97fps. it only takes 29.97 and 59.94 fps for ntsc input.

as far as a vc1 pulldown... The way i handle h264 bitstreams is to completely parse them on a bit level saving each variable, then reconstitute them (except for the actual video data).. so changing a bit at that point was no big deal. i haven't even looked at a vc1 stream yet (on a bit level) so i am just starting there. once i can create a 1:1 copy of a stream with full parsing, and reconstituting, then i'll add the flag in.

one question .. with mpeg2.. time codes were important with pulldown. is it an issue with vc1?

[drmpeg]

I'm not gonna feed any authoring program with the converted stream, so this program works so well for me.
But, anyway, what's the difference between an original 24p coded stream and this converted stream and why is not Blu-ray compliant??

I'm just curious :)
In Blu-ray, you can't have INTERLACE = 1 with a frame rate of 23.976 fps. Because of that, there would be four additional things to do to make the converted stream Blu-ray compliant.

1) In the sequence headers, INTERLACE should be changed from 1 to 0.

2) If INTERLACE is 0, then the FCM (Frame Coding Mode) field must be removed from the picture headers. FCM is always 0 (progressive) in VC-1 HD-DVD streams. 1 bit must be deleted.

3) If INTERLACE is 0, then the UVSAMP (UV Sampling Format) field must be removed from the picture headers. UVSAMP is always 1 (progressive) in HD-DVD VC-1 streams. 1 bit must be deleted.

4) If INTERLACE is 0, then the TFF and RFF flags become the RPTFRM flag. Instead of forcing TFF = 1 and RFF = 0 like I'm doing now, RPTFRM would be set to 0 (no repeated frames). Since the RPTFRM field is 2 bits, there are no bits to be added or deleted.

Ron

[drmpeg]

Blu-ray should be ok as it takes 23.976 fps feeds directly (at least afaik) hddvd being closer related to dvd still requires streams have flags to fake 29.97fps. it only takes 29.97 and 59.94 fps for ntsc input.

as far as a vc1 pulldown... The way i handle h264 bitstreams is to completely parse them on a bit level saving each variable, then reconstitute them (except for the actual video data).. so changing a bit at that point was no big deal. i haven't even looked at a vc1 stream yet (on a bit level) so i am just starting there. once i can create a 1:1 copy of a stream with full parsing, and reconstituting, then i'll add the flag in.

one question .. with mpeg2.. time codes were important with pulldown. is it an issue with vc1?
No timecodes in VC-1.

Ron

[AEtH]

1) In the sequence headers, INTERLACE should be changed from 1 to 0.

And what is the PULLDOWN flag for in the sequence headers?? Does it need to be changed from 1 to 0 (as we have undone the pulldown) or doesn't matter?



2) If INTERLACE is 0, then the FCM (Frame Coding Mode) field must be removed from the picture headers. FCM is always 0 (progressive) in VC-1 HD-DVD streams. 1 bit must be deleted.

3) If INTERLACE is 0, then the UVSAMP (UV Sampling Format) field must be removed from the picture headers. UVSAMP is always 1 (progressive) in HD-DVD VC-1 streams. 1 bit must be deleted.

What are the consequences of deleting one bit form picture headers? Do you have to remap the headers?
I don't know if I'm explaining well... if you delete that bit I suposse that you have to "tell" the decoder not to read it.


4) If INTERLACE is 0, then the TFF and RFF flags become the RPTFRM flag. Instead of forcing TFF = 1 and RFF = 0 like I'm doing now, RPTFRM would be set to 0 (no repeated frames). Since the RPTFRM field is 2 bits, there are no bits to be added or deleted.

So, in the original stream (INTERLACE=1) there are two flags (bits) and let's suppose that first is TFF and second is RFF. The coding for the modified stream would be 10 (as you have forced), but to completely change the stream the coding should be 00, is that ok??

[crypto]

And what is the PULLDOWN flag for in the sequence headers?? Does it need to be changed from 1 to 0 (as we have undone the pulldown) or doesn't matter?

I also wondered about the pulldown flag, as Ron leaves it on.

What are the consequences of deleting one bit form picture headers? Do you have to remap the headers?
I don't know if I'm explaining well... if you delete that bit I suposse that you have to "tell" the decoder not to read it.

Deleting means removing. As the VC-1 stream is bit orientated, there are no byte boundaries. The headers have to be reassembled/recompiled/reencoded or whatever you call it. Steve had the some problem with his asf2vc1 tool. He parsed the headers, made the necessary changes like adding display extensions and recompiled the headers.

[AEtH]

The headers have to be reassembled/recompiled/reencoded or whatever you call it.

Second question answered, maybe drmpeg can answer the first one.

Now, about reencode the headers... any documentation about VC-1 headers?? I took a look at the source of vc1_conf and the code is really clear... but I'm missing information about what bytes need to be parsed. And, of course, if the parsing that has to be done is bit by bit... hard job without header info. Any chance on having it?

[yesgrey]

However, if you want to be Blu-ray compliant, then it does.

No, not needed.

Thanks.

[AEtH]

@drmpeg
Could you clarify the function of the PULLDOWN flag??

@crypto
Are you working on changing the framerate? Are you including it in vc12avi?

Thanks

[crypto]

Regarding the pulldown flag, it needs to be left on after the tff/rff flag manipulation or those flags wouldn't be read anymore.


Are you working on changing the framerate? Are you including it in vc12avi?


No, not in vc12avi version 1.0.1

[ACrowley]

i always get a cannot open Bitstream Error ?

EDIT:
Must be caused by Evodemux. With TSRemux or Grapephedit it works nicely

great Work !

[hollydoc]

I have demuxed a HD-DVD with evodemux in VC-1 ES and EAC3, changed eac3 in ac3 and create a Bluray with Scenarist.
The Bluray plays fine on PS3 in 1080i/60hz with 2:3 Puldown. My projector is 24P compatibel. When i manipulate VC-1 ES with vc1conv scenarist means, it is 23,976i and the errormessage says 23,976i is not BR compatibel.Is tehre a way to change the interlaceflag?

Hollydoc

[hollydoc]

have demuxed a HD-DVD with evodemux in VC-1 ES and EAC3, changed eac3 in ac3 and create a Bluray with Scenarist.
The Bluray plays fine on PS3 in 1080i/60hz with 2:3 Puldown. My projector is 24P compatibel. When i manipulate VC-1 ES with vc1conv scenarist means, it is 23,976i and the errormessage says 23,976i is not BR compatibel.Is tehre a way to change the interlaceflag?

Hollydoc

[Kumo]

i'm trying to convert a vc1 stream file from 29,970fps to 23,976fps, removing the pulldown with vc1conv.exe .vc1conv says "cannot open bitstream file" every time i try. in a previous post i read maybe it was due to evodemux, but i tried to demux the vc1 file with 3 different tools(evodemux,tsremux and hddvd demuxer) and i always get the same error message. anybody can help plz??

[drmpeg]

i'm trying to convert a vc1 stream file from 29,970fps to 23,976fps, removing the pulldown with vc1conv.exe .vc1conv says "cannot open bitstream file" every time i try. in a previous post i read maybe it was due to evodemux, but i tried to demux the vc1 file with 3 different tools(evodemux,tsremux and hddvd demuxer) and i always get the same error message. anybody can help plz??
That error message usually means that some other program has the file still open.

Ron

[drmpeg]

have demuxed a HD-DVD with evodemux in VC-1 ES and EAC3, changed eac3 in ac3 and create a Bluray with Scenarist.
The Bluray plays fine on PS3 in 1080i/60hz with 2:3 Puldown. My projector is 24P compatibel. When i manipulate VC-1 ES with vc1conv scenarist means, it is 23,976i and the errormessage says 23,976i is not BR compatibel.Is tehre a way to change the interlaceflag?

Hollydoc
It's possible, but not with the current scheme I'm using in vc1conv. To change the interlace flag, I'll have to pretty much rewrite vc1conv in a more flexible fashion.

Also, be aware that some HD-DVD VC-1 streams may have problems with the IMODE syntax element. I don't think it's possible to convert those streams without re-encoding.

Ron

[hollydoc]

Hi drmpeg,

tank you for answer and for your great work, i hope, you can give us a tool to remove the polldown flag and change a HD- DVD to Bluray in 24P.
There are a lot of HD only movies , i like to convert to Bluray. Reencoding is not the thing, i want.
The BD+ let us locking more to HD-DVD, but ther is no HD-burner availible.

Hollydoc

[dburckh]

I'm working on something to reset the flags and bitshift the packet. I have something that is somewhat working, but I have stream corruption problems. It may just be my bit shifting code or parsing code.

Here's what I'm doing:
Reseting the interlaced flag to 0
Reseting the pulldown flag to 0
Reseting the frame rate to 24000/1001
Removing the FCM
Removing TFF/RFF (aka RPTFRM)
Removing UVSAMP
Removing that last byte if it is 0x00.

Should this work?

[drmpeg]

I'm working on something to reset the flags and bitshift the packet. I have something that is somewhat working, but I have stream corruption problems. It may just be my bit shifting code or parsing code.

Here's what I'm doing:
Reseting the interlaced flag to 0
Reseting the pulldown flag to 0
Reseting the frame rate to 24000/1001
Removing the FCM
Removing TFF/RFF (aka RPTFRM)
Removing UVSAMP
Removing that last byte if it is 0x00.

Should this work?
Sounds good, although I'm not sure about the last 0x00 byte. I've sent you a PM.

Ron

[dburckh]

Sounds good, although I'm not sure about the last 0x00 byte. I've sent you a PM.

Ron

Thanks for the help. I'll try your suggestion.

I thought I had posted what I did next, but it looks like I didn't. To make my PM make sense, I'll post it here for posterity.

By doing the changes above the stream ends up shifted one nibble (have used that term for a while). I spot checked it with a hex editor and it looked clean, but still had corruption. My guess is that something is bit aligned in picture itself (B frames?). I was going to set the post processing flag and keep the pulldown flag set. This would get me a net picture bit shift of 0. Unfortunately, not all the picture types have the post processing structure, so that won't work.

[fbd]

I'm working on something to reset the flags and bitshift the packet. I have something that is somewhat working, but I have stream corruption problems. It may just be my bit shifting code or parsing code.

Here's what I'm doing:
Reseting the interlaced flag to 0
Reseting the pulldown flag to 0
Reseting the frame rate to 24000/1001
Removing the FCM
Removing TFF/RFF (aka RPTFRM)
Removing UVSAMP
Removing that last byte if it is 0x00.

Should this work?

Thank you for your job dburckh, I am in a hurry to test your code on my vc-1 files:)

Regards
Franck

[zeroprobe]

Any updates? :)

[kurt]

yeah, I would like to change the interlaced flag of some vc-1 files too. Then there should be proper playback possible in 24hz on my popcorn hour :)

[glen8]

let me know if you need any beta testers :cool:

[DonCarlos]

Hi guys.
Is it really that much work to the stuff you supposed?I really would like to have your tool because this is the last peace missing to be able to convert an HDDVD to a Blu ray. I don't want to going on your nerves, but are there plans to release a small tool which can do this:(HDDVD) VC-1 1080i 29.97----->(Blu ray) VC-1 1080p 23.973?

Or could you explain what you mean by this:
Reseting the interlaced flag to 0
Reseting the pulldown flag to 0
Reseting the frame rate to 24000/1001
Removing the FCM
Removing TFF/RFF (aka RPTFRM)
Removing UVSAMP
Removing that last byte if it is 0x00.

and how we could do this?

Thanks

[kurt]

yeah, I would like to change the interlaced flag of some vc-1 files too. Then there should be proper playback possible in 24hz on my popcorn hour :)
problem solved, vc1conf does work! completely smooth playback with popcorn hour :)

[drmpeg]

The new version (0.2) of vc1conv is ready. It does the complete conversion from 29.97 fps interlaced with pulldown flags to 23.976 fps progressive. It is fully Blu-ray compatible as long as the POSTPROCFLAG field is zero (it will fail with an error message if POSTPROCFLAG is set) and there are no IMODE changes from Raw mode to anything else (not sure if any tool looks for this).

http://www.w6rz.net/vc1conv.zip

Ron

[fbd]

The new version (0.2) of vc1conv is ready. It does the complete conversion from 29.97 fps interlaced with pulldown flags to 23.976 fps progressive. It is fully Blu-ray compatible as long as the POSTPROCFLAG field is zero (it will fail with an error message if POSTPROCFLAG is set) and there are no IMODE changes from Raw mode to anything else (not sure if any tool looks for this).


Ron

Thank you drmpeg,

I will test it ASAP.

regards
Franck

[PHD_1976]

I've tested a piece of vc1 file from HD DVD (1gb) with vc1conv v.0.2 . I muxed this file into Blu-Ray with Scenarist HDMV ver.4.3. Everything went fine, I didn't notice any glitches or stuttering on the created BD.
Thank you, drmpeg for your efforts and results -))

[stevensf]

I have tested it too. works fine :)

Tx for this great job :))))))

[menlvd]

roman76 in next version of smartLabs ts muxer try to add pulldown patch for vc1 stream. thats can be useful in mkv(vc1)->ts(vc1)

[PHD_1976]

drmpeg,
I khow that madshi's already asked you a question about avc from HD DVD to Blu-Ray modification.
Just like you explained in this thread about what should be changed in 29.97 vc1 from HD to make it fully compatible with BD, could you please list parameters that should be changed/deleted/added in AVC stream to make it BD compatible if you have an idea?

About vc1conv 0.2:
I've tested it again with large file (full movie) Liar Liar - so converted file has one less frame than the original. Don't khow whether it's a bug or not and will it show itself somehow after muxing with Scenarist - i'm just reporting my notes, maybe you would like to check it.

[drmpeg]

drmpeg,
I khow that madshi's already asked you a question about avc from HD DVD to Blu-Ray modification.
Just like you explained in this thread about what should be changed in 29.97 vc1 from HD to make it fully compatible with BD, could you please list parameters that should be changed/deleted/added in AVC stream to make it BD compatible if you have an idea?

I'll have to get back to you on that one.

About vc1conv 0.2:
I've tested it again with large file (full movie) Liar Liar - so converted file has one less frame than the original. Don't khow whether it's a bug or not and will it show itself somehow after muxing with Scenarist - i'm just reporting my notes, maybe you would like to check it.
I know what's wrong. It's cutting off the very last frame. I'll have a fix shortly.

Ron

[PHD_1976]

I'll have to get back to you on that one.

Ron

Thank you, will be waiting -)