Looking for coding help combining ffmpeg and mpeg2enc

[FredThompson]

Would anyone be interested in helping improve the quality of mpeg2enc so mosquito noise is reduced?

For a number of technical reasons (flipping field dominance, flags, etc.), mpeg2enc is the best option for TyTool, a TiVo-oriented MPEG2 cut editor.

However, mpeg2enc creates some horrible mosquito noise where QuEnc does not.

I'm searching for someone who can help tweak the two code bases so the quality of compression of QuEnc/ffmpeg could be migrated into mpeg2enc.

Or, I suppose, it could go the other way, expand QuEnc to accept input of YUV4MPEG and more flexible structures.

Either option would work.

[Nic]

Expanding QuEnc is by far your easiest option. Trying to improve MPEG2Enc's quality would be far more difficult.

Could you explain more why QuEnc/FFMPEG is less suitable. I.e. what flags? What more flexible structures? etc :)

-Nic

[jdiner]

Originally posted by Nic
Could you explain more why QuEnc/FFMPEG is less suitable. I.e. what flags? What more flexible structures? etc :)
Nic as the author of the tool FredThompson was talking about perhaps I am the best one to explain what is going on.

Right now I use the mjpegtools mpeg2enc to re-encode small sections of a longer MPEG-2 in order to provide accurate editing of MPEG-2 PS. The nature of these streams is a bit odd mostly because of their source being different models of Tivo's.

In this process I decode the oddly structured GOPs and rebuild the proper YUV data for each frame, I then pass this on to an encoding program (currently mjpegtools mpeg2enc) and in turn take the resulting GOP and use it to rebuild the source stream in an edited fashion.

The problem with QuEnc and ffmpeg focuses around the frame type ordering in the result. Especially when dealing with things at the resolutions found in certain HiDef source materials.

Source material from a tivo can have extremely long GOPs. I have seen 100+ picture headers in a single GOP, and when combined with the odd use of RFF and TFF flags it has resulted in more than 164 display frames from that 1 GOP. Like I said, odd. With mpeg2enc the size and nature of the GOP format is controllable. I can put it all back into 1 destination GOP and stitch things back together for the final use on a DVD or just plain PS file. With QuEnc and ffmpeg I have been unable to control this in the same way.

Can you or someone else tell me how to control the frame pattern of the ffmpeg/quenc output. So that it was in proper format IPBBPBBPBB...

From my testing it looks like something is kicking in and saying something to the effect of "things have changed enough to warrant and I-Frame here". I can provide the pattern I get if it helps anyone but I get IPBBPIIPIIIPI from one of my test GOPs.

This in it's own way is even worse than I am seeing from the Tivo. :)

I am not married to any particular tool for the re-encoding I just need better visual quality than what I am getting and a properly encoded result.

Hopefully the background and nature of the problem will help a bit in moving things forward.

--jdiner

[jdiner]

I suppose I should add that I can whatever other details I know about the source materials that will help, just ask as I don't know what anyone will need to know.

--jdiner

[Nic]

@jdiner: Do you need to control the exact GOP contents (i.e the type exact pattern) or just the length? With scene detection off you shouldn't get weird I Frames added all over the place, so it should adhere to the GOP size you entered.

I'm still not sure exactly what you'd need...So let me know the specifics and i'll see what I can do.

Cheers,
-Nic

[video]

Originally posted by Nic
@jdiner: Do you need to control the exact GOP contents (i.e the type exact pattern) or just the length? With scene detection off you shouldn't get weird I Frames added all over the place, so it should adhere to the GOP size you entered.

I'm still not sure exactly what you'd need...So let me know the specifics and i'll see what I can do.

Cheers,
-Nic

I feel the problem is that dvd authoring sw makes complaints about gops having more than a specific number of frames/fields.

So I think jinder needs a following:
-count 30 fields played
-rewind to the nearest I frame
-insert a GOP header just before that I frame.

Time code within the newly added GOP header can be tricky, in case of NTSC drop frame mode, but salvageable...


if there's really a need to edit TiVos werid mpeg, just a question wether can it be DGIndex-ed and DGDecode-d via avisynth? If so, then We have a 29.97fps or 25fps interlaced material - what can be edited after that in premiere/avid xpress dv w/o any problem.

[FredThompson]

Short answer: no

Explanation: Those ideas would make sense if the goal were to re-encode captured source to meet DVD spec requirements. That's not the case or the goal. The vast majority of DVD players, hardware or software, do not require strict DVD spec source.

TyTool stays as close to the original transport stream as possible to ensure the highest quality and for a number of technical reasons. TyTool is used to remove sections of video during which it stitches the 2 remaining "partial" GOPs together. Video in the other GOPs is unmodified.

mpeg2enc gives enough low-level control of the encoding process so the resultant streams play properly but it makes a huge amount of mosquito noise. mpeg2enc is also used to make individual I-frames used to generate DVD menus.

QuEnc does a far, far, far better job of encoding high-contrast edges than does mpeg2enc. The goal is to create a mix of QuEnc quality and mpeg2enc low-level control.

[jdiner]

Originally posted by Nic
@jdiner: Do you need to control the exact GOP contents (i.e the type exact pattern) or just the length? With scene detection off you shouldn't get weird I Frames added all over the place, so it should adhere to the GOP size you entered.
Hummm. I had thought it was a scene detection type of system, but I can find nowhere in the various docs where it can be turned off or otherwise altered. Care to point me in the correct direction?

--jdiner

[jdiner]

Originally posted by video
I feel the problem is that dvd authoring sw makes complaints about gops having more than a specific number of frames/fields.
Ummm. Not quite. The DVD creation and burning process was addressed and solved years ago. Many of the users of TyTool have burned literally thousands of disks from tivo source material. We are simply trying to get better results with what little of the data "must" be re-encoded.

TyTool does NOT re-encode and entire stream. It makes DVD Menu images and re-encodes around the frame structure as the base upon edits to produce the necessary data to have a seemless edit take place.

The long GOp sizes are not an issue on playback at this time. They were background into what has to be done to feed the re-encoding program.

if there's really a need to edit TiVos werid mpeg, just a question wether can it be DGIndex-ed and DGDecode-d via avisynth? If so, then We have a 29.97fps or 25fps interlaced material - what can be edited after that in premiere/avid xpress dv w/o any problem.
No. It has been tried and tried and tried. That is why several years ago I built my own. There issues are both numerous and well documented. So suffice it to say there are reasons why I am doing what I am doing.

--jdiner

[dragongodz]

I had thought it was a scene detection type of system, but I can find nowhere in the various docs where it can be turned off or otherwise altered.
from the CLI use -
QuEnc.exe -?
to pop up a window listing all the CLI commands and what they do.

to turn off scene detection is -noscene

[video]

FredThompson & jdiner, so it's about getting rid of commercial breaks? i'm really lost about the purpose of re-encoding small portions.

[FredThompson]

The only way to do accurate editing of open GOPs is to create a new GOP out of the remnants of the remaining 2 partial GOPs where a section of source was removed. Re-encoding just that portion and stitiching it into place, so to speak...er...type, maintains the highest quality in the unedited GOPs.

[video]

Originally posted by FredThompson
The only way to do accurate editing of open GOPs is to create a new GOP out of the remnants of the remaining 2 partial GOPs where a section of source was removed. Re-encoding just that portion and stitiching it into place, so to speak...er...type, maintains the highest quality in the unedited GOPs.
that's okay but you have mentionaed field operations a drop frames signalled, hmm? if you just stich you may break motion fluidity or encounter temporal a-v sync lost...

However you may look at GopChop (http://sourceforge.net/projects/gopchop/)

[FredThompson]

Dropped frames isn't a problem. That was just an attempt to explain simply. Suppose you have this:
I1....I2....I3....I4....I5....I6....I7
you mark a section for removal, based on fields and have this:
I1... ....I5....I6....I7
Thus, you'd need to create a new GOP which looks like:I1.......This gets joined to the I0 and I5 GOPs. That's the concept.

Throw in the possibility of audio holes, flipped field dominance, open GOPs, etc. and it gets tricky. jdiner has worked out solutions for most of those issues.

What we're discussing here is how to use the visual quality of QuEnc's video with the data packaging control of mpeg2enc so the resultant video looks better.

[video]

Originally posted by FredThompson
Dropped frames isn't a problem. That was just an attempt to explain simply. Suppose you have this:
I1....I2....I3....I4....I5....I6....I7
you mark a section for removal, based on fields and have this:
I1... ....I5....I6....I7
Thus, you'd need to create a new GOP which looks like:I1.......This gets joined to the I0 and I5 GOPs. That's the concept.

Throw in the possibility of audio holes, flipped field dominance, open GOPs, etc. and it gets tricky. jdiner has worked out solutions for most of those issues.

What we're discussing here is how to use the visual quality of QuEnc's video with the data packaging control of mpeg2enc so the resultant video looks better.

FredThompson look at gopchop, it does exacly what you want, even take car the audio protions as well.

[FredThompson]

No, it won't. GOPChop simply removes GOPs. That has nothing to do with seamless editing at the field level.

The diagram above is supposed to illustrate I-frames and fields. It shows 2 partial GOPs which need to be joined. The data structure, the byte sequencing, is "solved." The question is how to get better quality video data. mpeg2enc is a comparatively horrible encoder but it allows the structural control.

[video]

Originally posted by FredThompson
No, it won't. GOPChop simply removes GOPs. That has nothing to do with seamless editing at the field level.

The diagram above is supposed to illustrate I-frames and fields. It shows 2 partial GOPs which need to be joined. The data structure, the byte sequencing, is "solved." The question is how to get better quality video data. mpeg2enc is a comparatively horrible encoder but it allows the structural control.

okay. your picture and the text misleaded me, i interpreted I1...I2...I3 as 3 GOPs andsoforth. afaik mn comes into play on P and B images with low stream bitrate while the content has somewhat sharp edges. during the stich you may have to re-encode max 4 frames (max 2 orpaned 'B' at the end of reel left and max 2 'B' or one 'P' of reel right) i feel. does mpeg2enc hurt even if you emit 4 'I' frames, with Q=1?
if so, you may try the following (idiot idea of mine). dump out the stich point pictures (that 4) into bmp. use avisynth to compose a 4-picture clip. fire up tmpegencplus 2.5, specify project attributes, resolution, framereate, quality, bitrate, etc. load the avs at video source, press button settings (right bottom corner), review the contents of the video and advanced tabs, then go to tab gop sturcture, tick 'force picture type setting' press settings, click on the picture, and at the leftdown side input box force desired picture type using right button mouse menu. then go to quant matrix tab, load your original gop matrices, choose fp DCT. probably you want to play with soften block noise as well. tmpegenc will output you a tiny gop with the encoded frames of the stich point. you have to get the frames out and put into the desired salvageable GOP.
i've atached some images what might be seen after modreator approval.
if this answer is inadequate for you, too, then i'm sorry i've got no better idea. if you give me one month (i'm terribly overloaded now), may figure out how can i enhance mpeg2enc's encoding quality - but i cannot touch it in april

[FredThompson]

The diagram does represent I, B and P frames. The periods represent B and P frames.

You're creative and inquisitive, good qualities to have, but keep missing the level of discussion.

The topic is combining aspects of ffmpeg and mpeg2enc and operation at the data level. It's about internal construction of the encoders, period.

[dragongodz]

i would love to see someone add mpeg2enc's rate control as a second selectable option(being GPL it would need to be only selectable when compiled GPL and not LGPL). :D

[jdiner]

Originally posted by video
FredThompson & jdiner, so it's about getting rid of commercial breaks? i'm really lost about the purpose of re-encoding small portions.
The short answer is yes. It is usable in many other ways but cutting commercials in the most "accurate" way is the most common use. There is also the lead-in and -out to movies, creating shorter segments, etc... Remember the nature of the source and purpose of the tool when considering the capabilities.

--jdiner

[jdiner]

Originally posted by video
FredThompson look at gopchop, it does exacly what you want, even take car the audio protions as well.
No it doesn't. And come on dude, we aren't asking how to edit. We did already did all of that in what is generall a better way. We are asking how to TUNE the recompression tools to get just what we need from them in the best possible way. That's it.

I appriciate the replies but we seem to be getting a bit off track here.

--jdiner

[jdiner]

Originally posted by video
during the stich you may have to re-encode max 4 frames (max 2 orpaned 'B' at the end of reel left and max 2 'B' or one 'P' of reel right) i feel. does mpeg2enc hurt even if you emit 4 'I' frames, with Q=1?
That would be true if the DTV source material was "standard" mpeg-2. It isn't. Trust me. It just plain isn't. That is the reason the entire gop structure is re-encoded to fix/correct the nature of the source GOPs themselves.

if this answer is inadequate for you, too, then i'm sorry i've got no better idea. if you give me one month (i'm terribly overloaded now), may figure out how can i enhance mpeg2enc's encoding quality - but i cannot touch it in april
I for one appriciate the interest and the help.

--jdiner

[video]

Originally posted by jdiner
The short answer is yes. It is usable in many other ways but cutting commercials in the most "accurate" way is the most common use. There is also the lead-in and -out to movies, creating shorter segments, etc... Remember the nature of the source and purpose of the tool when considering the capabilities.

--jdiner

Okay, so something like this (http://www.pegasys-inc.com/en/product/tme_spec.html) but jinder what behaviour should be improved in libmjpeg? overall, intra frame generation accuracy, qunatization or what? since i dunno how would you like to use it. You have told that redo stich point as Intra frames is not possible...

I mean:

reel1: GOP[ippbpbpibipbbpbppppipib|bbpppi]
^ cut v
reel2: GOP [iiiipippibpibipbpbppiiibbibi|bibbbbbiip]

joined: GOP[ippbpbpibipbbpbppppipiiiibbbbbiip]

isn't it possible?

[jdiner]

Originally posted by video
...but jinder what behaviour should be improved in libmjpeg? overall, intra frame generation accuracy, qunatization or what? since i dunno how would you like to use it. You have told that redo stich point as Intra frames is not possible...
The problem is the nature of the material. The biggest issue present is that the DTV, the source for the material on a DirecTivo, put things into place before the MPEG-2 spec was finished. So while it is "close" enough for a forgiving decoder it isn't really right. The GOP length is only 1 of these issues.

The other, and by far the biggest one for this question, is that they use the mechanisms for 3:2 pulldown in a non-standard custom way. This is seen most often as field order precednec flipping. Basically they appear to be using the RFF and TFF flags in each of the picture headers as a space saving mechanism for transmission purposes not a true frame decimation.

The "normal" pattern of these flags is basically never found. It is not uncommon to have a 1 GOP that is 60+ pictures long that has only 1 RFF=1 in it. Giving a GOP with an odd number of fields and starting the next I-frame with a different field order. Then in the same stream you will hit a GOP wherein basically every picture header has RFF settings in it.

So while what you suggest is I suppose theorectically possible, in practical application it has proven to be necessary to reprocess everything to maintain proper field order and post stitching to have proper playback. Especially given the control option for the re-encoding programs being used.

Sorry. Wasn't trying to be obscure on any of this stuff, in the TyTool forum there are just very few people that even care about this kind of information so I didn't think about adding it.

--jdiner

[video]

Originally posted by jdiner
So while what you suggest is I suppose theorectically possible, in practical application it has proven to be necessary to reprocess everything to maintain proper field order and post stitching to have proper playback. Especially given the control option for the re-encoding programs being used.

Sorry. Wasn't trying to be obscure on any of this stuff, in the TyTool forum there are just very few people that even care about this kind of information so I didn't think about adding it.

--jdiner

So does it mean, that you re-encode practically tha stiched GOP - or - resolve field ops with inserting frames into the stream instead of field ops hand having a plain interlacaed movie w/o filed ops at the end of the game?

what i can imagine as algorythm:
1) count fileds, simulate filed ops, and log picture numbers where you have exactly reached 59.94 fileds played (you should consider drop frame issues as well) and log them. so you have the sync points. do it for booth reels
2) do cut and stich, and don't care about field ops.
3) you know where were the sinch point in reel1 and you know where will you reach the synch point in reel2. between the two synch points achieve the proper frame rate with filed op redesign

[FredThompson]

Inserted fields would be incredibly nice. The problem is B P frames could be pointing to data which has been removed, so to speak...er...type. This means he does, essentially, re-encode the entire "new" GOP. IIRC, it's done on each "end" of the "new" GOP individually then they're stitched together.

Those are the structural issues and why a custom encoder use is needed.

The quality aspect is illustrated by these 2 single frames (http://home.mindspring.com/~fredthompson/menu-tests.zip) generated to make DVD menus. The noisy one is mpeg2enc, the "clean" one is QuEnc.

[video]

Originally posted by FredThompson
Inserted fields would be incredibly nice. The problem is B P frames could be pointing to data which has been removed, so to speak...er...type. This means he does, essentially, re-encode the entire "new" GOP. IIRC, it's done on each "end" of the "new" GOP individually then they're stitched together.

Those are the structural issues and why a custom encoder use is needed.

The quality aspect is illustrated by these 2 single frames (http://home.mindspring.com/~fredthompson/menu-tests.zip) generated to make DVD menus. The noisy one is mpeg2enc, the "clean" one is QuEnc.

I am sorry - could you tell me please how can i see mpeg stills under linux?

[FredThompson]

They should load into a DVD player or MPEG2 editor. There must be something odd about them, though. Maybe they're too short.

Here they are as PNGs:

lossless (http://home.mindspring.com/~utils/Town Haul test background PNG.png)

mpeg2enc (http://home.mindspring.com/~utils/mpeg2enc.png)

QuEnc (http://home.mindspring.com/~utils/QuEnc.png)

mpeg2enc makes a similar level of distortion with "natural" backgrounds, too. Setting a high bitrate didn't help the mpeg2enc version. QuEnc looked worse with 2 passes instead of 1. That was interesting.

This image has a good mix of natural images, gradients, hard edges and contrasting colors. Besides, Genevieve is cute.

[video]

ooops, mpg2enc is really hurt....

[guada 2]

FredThompson,

Would you think have to create an intermediate codec, as within the framework of the importation of files HDV?

It is true that this hybrid format ranging between Mpeg4 and the mpeg2 is not thin to make.

But I am grateful to know that that interests you.
I would stain with next to provide you a little more information.

Mario

[FredThompson]

There is no MPEG4. The source and destination are MPEG2.

[guada 2]

Sorry FredThompson, it is not of Mpeg4 but a hybrid format.
I do not manage to have infos, but it specified me that it was not this format.
As soon as I am informed, I would say to you a little more.

So long

[FredThompson]

Perhaps you don't understand. We are only working with a special form of MPEG2.

[guada 2]

Technically speaking, I am not there.
Nevertheless, this Mpeg2 form which you speak, which would be then its characteristics with respect to current the Mpeg2 Format?

Which is the interest for you to improve this format?
Lastly, would you be tried to establish Hd-mpeg2, by the means of Hr-mpeg2 in prospect?

[FredThompson]

The answers are earlier in this thread.

[guada 2]

I hope well.
With very well soon "FREDTHOMPSON" :)

[guada 2]

Hello "Fred.."

I wondered by chance if your step did not rest on a system to control by a VBR in real time and a technique calling upon the modulation of the visibility.

It is undoubtedly dared, but that will be oh how much brilliant. ;)

[video]

Originally posted by FredThompson
Perhaps you don't understand. We are only working with a special form of MPEG2.

guada 2 Fred's mpeg2 is not out of spec. It's simply a streaming optimzed mpeg2. On DVDs you will find 'navigation optimized' mpeg2. That1s all. Streaming optimized mpeg2 is not suitable for editing/putting on a DVD. Streaming mpeg2s are using frame/field repeats and long gops to reduce format overhead loss. A sequence/gop header is almost 2kbytes - if you send sequence/gop header every 2 seconds instead of half a second (as dvd does) you are preserving 6kbytes/sec for the effective content. for the same reason using field repeats instead of b-frame calculations you can preserve bandwith as well. If the hardware smart enough then it can produce nice output even at rate 3-4Mbps. The huge proble with that this is a pain in the ass to edit this kind of mpeg2s. Fred and jinder would like to do it anyway, probably after editing they would like to put it on a DVD.

[guada 2]

Video, thank you for this clarification.

[guada 2]

Hello "Fred.." :)

No the news, good new.
Should I understand that my quotation is checked.
If it is true, let me know it.

Thank you.