"Reverse ReJig" possible?

[bb]

I wonder if it were possible to reverse ReJig's functionality for the following purpose:

Overcompressed MPEG streams often suffer from pumping and exaggerated quantization. I thought it would be possible to create an application which would
- keep I-frames and motion vectors
- recreate P- and B-frames using lower quantizers

I think only the DCT had to be recalculated with lower quantizers for each macro block, and you'd get a higher bitrate file with better quality.

What do you think?

bb

[Didée]

Now, what should that proggy actually do?

The pumping and such means that the information that IS IN the stream is drifting away from what originally was, and gets refreshed minorly on P's and majorly on I's.

So, you'd need to get the information in the frames from their "drifted-away" content closer back to the original content ... which is NOT in the stream: a DCT component that was DCT'ed to zero, but wouldn't have been with lower quantization, will stay zero whatever you do. The information is just not there.
Shortly, you'd have to invent information out of thin air. The old game ...

But perhaps I'm simply not thinking far enough out of the box (ever played Sokoman/Sokowahn? :D)

I'm the first one crying "Hooray!" if it should be possible somehow.

{edit}
This came just to my mind:
If at all, this would be a job for -h -- but it seems we lost him to his GF :( ;)
{/edit}

- Didée

[TCmullet]

Boy oh boy oh boy!! I've been thinking of something like this for over 6 months. I was looking at my 2002 Olympic figure skating gala, capped at a relatively low bitrate and remorsing, "Why couldn't I have captured at a higher bitrate?" (Using the Dazzle DVCII.) Then I remembered a recent interaction with Paul Currie the author of the "coring" VirtualDub filter. His tool had helped me realize that a region of solid color does NOT get encoded to a solid color with mpeg technology no matter how high the bitrate. But in the course of this, he informed me that we don't lower the bitrate by reducing the amount of detail encoded; rather we simply drop DCT coefficients. This was startling to me. (Of course I know very little of mpeg internals, barely that the further down the list a coefficient occurs the smaller its value and the lesser it's impact if dropped.) As I was observing (with irritating regret) the occasional mpeg artifacts in my otherwise pretty clean skating gala, I started to remember Paul's comments. I ALSO started thinking of my recent search for appropriate noise reduction Vdub filters. As I mulled over my newbie awarenesses of the different techniques described by those filter authors, it suddenly dawned on me... Might it be possible to syntheticly derive some of those missing DCT coefficients??? Not to reclaim missing detail, of course. But simply to clear up that pixelation caused by the missing coefficients? I don't know enough about mpeg workings and math to believe it's possible. But I've been meaning to (fearfully and privately) write a group email to Paul C., "fccHandler", "Phaeron" the Vdub author, and a few others, to see if any of them think this is even possible. I haven't gotten around to it. But now that someone else has raised the matter, I feel it's time to express myself. Maybe ReJig is the program to do this; maybe it should be a separate program.

While most are trying to reduce the size of an mpeg2 stream, some of us have been capturing kazillions of files at low-bitrates (to save scarce CD-R space), and would like to improve the quality now that DVD-R storage is feasible. Mind you, I'm not asking to restore lost detail, but rather to somehow synthesize a facsimile of the DCT coefficients previously dropped by the original encoder, so as to make the picture cleaner. I'm not sure if this is exactly what the starter of this thread had in mind, but it's what's been in the back of my mind for months.

Thanks for listening! :)

[Joe Fenton]

What you want sounds similar to an edge-preserving interpolation... use the info encoded in the DCT coefficients to determine if an edge is present, then use that to select a filter that preserves the edge when you scale and filter (normal interpolation blurs edges).

In this case, you aren't expanding the image, but smoothing what you already have to remove artifacts. Use the same technique to smooth without losing the edges. Pick a smoothing filter based on the edge info encoded in the Ch=(1,0) and Cv=(0,1) coefficients.

The article I was reading on this used five different gaussian filters based on the relative weights of the two coefficients. The cases were:

|Cv|<T1 and |Ch|<T1 = no edge
|Ch| - |Cv| > T2 = -
|Cv| - |Ch| > T2 = |
Cv x Ch > 0 = /
otherwise = \

Just doing that much greatly increases the clarity of the resultant image.

Edit: it occurs to me that this same method of locating edges might be used by ReJig when requantizing frames to change the level of requantizing on the fly inside the frame... a larger scale when no edge is detected and a smaller scale when an edge is indicated. This would help the picture retain some quality even at higher scale factors.

[fccHandler]

Originally posted by TCmullet
But I've been meaning to (fearfully and privately) write a group email to Paul C., "fccHandler", "Phaeron" the Vdub author, and a few others, to see if any of them think this is even possible.
For the record, I don't think it is.

[dragongodz]

is it possible to transcode and lower the quant values thus adding size to output. yes actually it is but why would you ? i mean you would not actually gain anything except size.

"it occurs to me that this same method of locating edges might be used by ReJig when requantizing frames to change the level of requantizing"
Joe - the original encoder should have already done this when the film was done. since requant is done evenly with rejig this should not be needed.

[TCmullet]

Originally posted by dragongodz
is it possible to transcode and lower the quant values thus adding size to output. yes actually it is but why would you ? i mean you would not actually gain anything except size.I believe we are thinking that perhaps at the expense of a bigger file, we could reduce artifacting caused by an excessively low encoding bitrate. (Please see my 1st post this thread.)

TC :)

[duartix]

What you people are asking is to pass a filter on the source and then recompress it at a higher bitrate. Of course this can remove some blocking or mosquito noise but, other than that you can't get the original detail once it is lost, and believe me it is lost.
For this you can use VDub.

What I don't get is what this process has to do with Rejig.

[TCmullet]

Originally posted by duartix
What you people are asking is to pass a filter on the source and then recompress it at a higher bitrate. Of course this can remove some blocking or mosquito noise but, other than that you can't get the original detail once it is lost, and believe me it is lost.
For this you can use VDub.

What I don't get is what this process has to do with Rejig.
I think we realize that the original detail is lost. But what I'm hoping is that the dropped coefficients could be somehow intelligently synthesized in a way merely to reduce mpeg "pixelation", not restore any lost detail. (Actually, losing detail makes the picture cleaner anyway.)

What does this have to do with ReJig?? Everything.. Rejig is doing some changes to the mpeg encoding WITHOUT totally reencoding the picture from scratch. Rejig only modifies the existing mpeg data without doing an mpeg2 total reencoding. This thread is speculating that maybe software could be written to synthesize some of the lost DCT coefficient data simply to reduce artifacting. Both Rejig and the unwritten software DO NOT scrap all the valuable mpeg data by totally decoding it to raw video (as reencoding would do).

A further thought. Reencoding requires a good high-bitrate capture in order for reencoding to not introduce a whole lot more artifacts. What I'm talking about is the scenario where the original capture was done at a bitrate TOO LOW for reencoding. Also, I don't want to lose what good quality the original file has (which any reencoding will do); I just want to augment it with additional data which hopefully can reduce the artifacts.

[duartix]

Hi TCmullet!

OK. I get the point. But think about it, if those missing coefficients could be synthesized in an inteligent way, that would mean a new breaktrough in compression too.

Unless of course you just want them to be synthesized in a way just to loose artifacts. The problem with that is that (I believe) the DCT has little to do with 2D image structure otherwise mosquito wouldn't be such a problem, in which case so you would be better of with a filter (at least for anime).

This is somehow what ffdshow and all those post processing filters are doing: deblocking out the 2D image assuming(?) the source was less blocky and also cleaner.

Do you mean a deblock/dering filter for Rejig?

[bb]

For my purposes ReJig has been interesting, because it's the only tool which would allow me to compress an elementary stream "a little more" at high speed. This is a great feature e.g. if your single-pass encoding has turned out a little too big; using ReJig I don't have to do the full encoding again.

If your encoding has turned out too small, you could increase the size and get better quality, if the "reverse ReJig" I described above was available. To reconstruct the P- and B-frames you'd need the original source, though. I doubt that some sort of interpolation or the like would help much with quality. But having the original source you could speed up the process by much if you need to increase the file size (and get better quality, of course, else it doesn't make sense).

bb

[Joe Fenton]

Originally posted by dragongodz
"it occurs to me that this same method of locating edges might be used by ReJig when requantizing frames to change the level of requantizing"
Joe - the original encoder should have already done this when the film was done. since requant is done evenly with rejig this should not be needed.

Rejig degrades the quality evenly to shrink the size of the mpeg2 data stream. If the source is really high-quality, you can degrade it quite a ways and it will still look good.

What I was thinking was degrading the edges less than the "flat" areas. You are degrading the flat areas even more, but since they have less detail in the first place, it shouldn't be as noticeable.

E.G., instead of shrinking everything 75%, shrink the edges 90% and shrink the flat areas 60% (or some similar percentages that makes the whole thing work out to about 75%). I'll bet it looks better for a variety of sources.

If you made two compress factors, the user could just set them to the same value for operation identical to the current version.

[TCmullet]

Originally posted by duartix
Unless of course you just want them to be synthesized in a way just to lose artifacts.Thats exactly what I'm talking about.
Do you mean a deblock/dering filter for Rejig?Not sure exactly. Not sure what deblocking/deringing is. And I'm not saying that Rejig should be the program to "EnhancePoorMpeg" files. I'm just saying that maybe the folks that are smart enough to tamper with the mpeg data to do what Rejig does, are also smart enough to think of a way to approximately synthesize the lost DCT coefficients (or some of them).

Here's a VERY basic, and very high-level definition of what might be done:

1. Develop a way to measure (quantify) the degree of artifacting (blocking, pixelation, whatever you wanna call it).
2. For each DCT that has a coefficient dropped (which I guess is most DCTs), generate several random values (maybe 8) as test coefficients.
3. For each value, call the artifact-measure function to see if the artifacting is raised or lowered by incorporating the test coefficient into the DCT.
4. Use the value giving the lowest function value as the newly synthesized coefficient.

If all this is even possible, I'm sure it would take a LOT of CPU time. Perhaps even longer than a normal soft-encode. But to take a 1.5GB file containing 2 hours of poor video, and turn it into a 4GB file clean with no or much less artifacting, would be WORTH 20+ hours to generate it. And as I've said before, there's TONS of footage waiting for this kind of expansion.

[TCmullet]

Originally posted by dragongodz
is it possible to transcode and lower the quant values thus adding size to output. yes actually it is but why would you ? i mean you would not actually gain anything except size.Would artifacting be reduced?

[dragongodz]

artifacting probably not, blocking maybe a little.

from the sound of what you want you i can suggest a current way.

load mpeg
deblocking filter
denoiseing filter
sharpening filter
output new mpeg with large bitrate(so as not to introduce new artifacts etc)

that should roughly do want you want.

[TCmullet]

Well, yes, sort of. Very roughly--only in the maximum case. That is, where we don't mind expanding the file size greatly. But your still doing a total reencode, thereby largely "losing" what good mpeg encoding has already been done.

As I said, only in the maximum case. Realisticly, we'd want to be able to do it in degrees, like Rejig. With Rejig, I do a little reduction specifying, say, 97%. Then if the file isn't small enough and the picture can tolerate more deteriation, try 96%. And so on, until file is barely small enough, or picture has degraded so bad that other options have to be explored besides rejig.

Likewise, a coefficient generator should accept some kind of parameter specifying "how many" coefficients should be generated. And we'd want to start out small (105%?) and try working our way up, seeing how good the picture could improve vs. file size.

I hope you get my point(s). And I hope all mpeg-savvy brains will keep thinking about this matter. Thank you!

[TCmullet]

Originally posted by dragongodz
artifacting probably not, blocking maybe a little.
I thought blockiness WAS a form of artifact. What distinction do you mean? And when you see a layer of pixelation surrounding a moving object, what do you call that?


load mpeg
deblocking filter
denoiseing filter
sharpening filter
output new mpeg with large bitrate(so as not to introduce new artifacts etc)
Thanks for the tip. I may do it some time. But I gather you're referring to VirtualDub filters, right? I am sometimes using Vdub-mpeg2 with frameserving to use Tmpgenc to reencode. Can you please tell me exactly which "deblocking", "denoising", and "sharpening" filters you use and recommend?

But again, I greatly hope this thread proposing an mpeg expander (dct coefficient synthesizer) will not die.

[dragongodz]

"layer of pixelation surrounding a moving object"
this is generally refered to as ringing or ghosting.

"I thought blockiness WAS a form of artifact"
well yes you could call it a kind of artifacting but thats a lose term in that case. i was more talking about artifacts such as stray pixels and noise which is why i mentioned both.

"I gather you're referring to VirtualDub filters, right?"
actually no. i meant load the mpeg in dvd2avi and save a project. write an avisynth script to load/decode it and apply the filters. then load the script in your favorite mpeg encoder, such as tmpgenc since you mention it. :)

"please tell me exactly which "deblocking", "denoising", and "sharpening" filters you use and recommend?"
i tend not to use any filters because i rarly encode mpegs etc. if you ask in the avisynth forum i am sure people will give you plenty of suggestions though. also you can do short encodes and see how the different filters work and which you prefer.

"largely "losing" what good mpeg encoding has already been done."
well thats what the filters are for. to try and actually improve the frames before encoding. thus if you use a larger bitrate you should not introduce any new(or very little) new artifacting etc. end result should be a better quality video. but of course the filters can only do so much so dont expect miracles. :D