Cross-codec conversion question

[Saiyr]

I don't know if this is true, but my friend suggested that doing a full decompress before recompressing in a different codec would increase the quality, as opposed to just going straight to one codec to another (e.g., DivX->XviD). Is this true in any way? I just got a new computer, so I think I'm pretty capable of doing it, I just don't know if it's a good idea.

[Cyberman]

I don´t think that there´s much difference. Actually, I´d say that quality will get worse if you save the uncompressed video just to recompress it.
It may depend on the program you´re using, but in the ideal case the two codecs should "talk" to each other finding the best way of transmitting the data(conversion of color space, YUV to RGB and such), while the program doing the decompression/recompression will have to decompress it to a format it can use to save the file.
VirtualDub, for example, can only work with RGB, not YUV data, so any codec storing in YUV colorspace would have to convert to RGB first, just to let VDub save the file.
If the other codec then uses YUV as well(quite likely), it´ll mean another conversion from RGB to YUV.

Besides, there´s not much use in converting DivX to XviD - they´re both Mpeg4, so you´ll only lose quality anyway(both work by eliminating small detail).

Any reason for the conversion?

[DAvenger]

DivX to XviD? A waste of time ...

[Saiyr]

Heh, it was my idea to leave the codec the way it was, but someone suggested that converting it to XviD would be better. I figured it was redundant, but the guy is supposed to know more about video than I do. What about converting from WMV9 in an AVI container? Is that redundant also? XviD is always the preferred codec to me, but if there's loss of quality in between, I'd just leave it as it is. I'm told that DivX has more problems and isn't very fault tolerant, etc.

[stephanV]

Originally posted by Saiyr
Heh, it was my idea to leave the codec the way it was, but someone suggested that converting it to XviD would be better. I figured it was redundant, but the guy is supposed to know more about video than I do.
Use your own judgment, it seems to be pretty good. :)

What about converting from WMV9 in an AVI container? Is that redundant also?

It depends, if you want to watch the video on a standalone player then right now you don't have any other choice. For watching the video on your comp, re-encoding is quite useless.

I'm told that DivX has more problems and isn't very fault tolerant, etc.
Are you experiencing problems with that file then? Seems the people who told you this are greatly overestimating the quality of XviD (or they just don't know what they are talking about). :rolleyes:

[Saiyr]

Originally posted by stephanV
It depends, if you want to watch the video on a standalone player then right now you don't have any other choice. For watching the video on your comp, re-encoding is quite useless.

Ok, just to clarify. Re-encoding isn't the only thing I'm doing, I'm also adding in sub-titles. Does it still matter whether or not I'm remaining in the native codec, quality-wise? I use VirtualDub with the subtitler filter. If re-encoding no matter what means quality loss, then I'll be sticking with what's given, but I'd like to have some consistency with choice of codec.

By the way, I've never really had problems with any DivX AVI, or any AVI that I recall. :o

[stephanV]

re-encoding with a lossy codec will alwyas give quality loss.

[Cyberman]

Are the subtitles for you, or for someone else(distribution)?
Because it´d be better to use a container(AVI is a container) that can hold subtitle streams(not sure about AVI), so you don´t have to re-encode.

My suggestion would be Matroska. It´s new, thus not too well supported by programs, but it´s better than AVI.

Anyway, soft subs don´t require a re-encode, so you won´t lose quality.
The downside is that you either need a player that supports them internally, or another program to display the subs.
Not that much of a downside, IMO.

--

OT:
StephanV: Where can I go without finding out you´re there as well? ;-)
(just kidding, of course)

[ntojzan]

I've been reading this thread, and found some false information within. So here are the corrections:

1) Cyberman stated: Actually, I´d say that quality will get worse if you save the uncompressed video just to recompress it.

That is false. The quality will be exactly the same.


2) stephanV stated: re-encoding with a lossy codec will alwyas give quality loss.

That is almost true. However it doesn't have to be. You may have the same visual quality with less datarate in some rare cases. However in most cases it will give quality loss.


Also I want to ask Saiyr a question: Why do you need to render the subtitles to the video file? Standalone subtitle files can be also used, it would save you the time of recompressing the file. (or putting the stream into another container)

[Saiyr]

I use hard subtitles right now because I'm not up to date with other methods of subtitling. True, I could use Matroska, but I'm not the only one working on this, so I'm not the only one that gives input. Also, the subs are for distro, so using MKV may not be a good idea, right? Once again, no real experience with MKV (besides playing it), but one would think that soft subtitles would be easy to extract, etc. I don't really want that.

ntojzan, what do you mean standalone subtitle files can be used? Like an SRT file plugged into BSPlayer, or what? The only format I'm really used to working with is SSA, in the old SSA program. Is the SSA format playable as such? And even so, it's about the same as doing soft subtitles, I'd think.

[Cyberman]

Originally posted by ntojzan
1) Cyberman stated: Actually, I´d say that quality will get worse if you save the uncompressed video just to recompress it.

That is false. The quality will be exactly the same.
The codec will try to compress the uncompressed result - so it´s going to remove detail again. The conversion of color-space, if necessary, will create loss of data too. The last one might not be noticeable, though.

2) stephanV stated: re-encoding with a lossy codec will alwyas give quality loss.

That is almost true. However it doesn't have to be. You may have the same visual quality with less datarate in some rare cases. However in most cases it will give quality loss.
Lossy codecs lose detail - that´s a definition of them. You might not see it, but there´s loss of detail.

[stephanV]

Originally posted by ntojzan
I've been reading this thread, and found some false information within. So here are the corrections:

1) Cyberman stated: Actually, I´d say that quality will get worse if you save the uncompressed video just to recompress it.

That is false. The quality will be exactly the same.

Cyberman is right, there may be an extra colorspace conversion and those are not perfect.


2) stephanV stated: re-encoding with a lossy codec will alwyas give quality loss.

That is almost true. However it doesn't have to be. You may have the same visual quality with less datarate in some rare cases. However in most cases it will give quality loss.

Whether it the visual quality is the same is hard to determine on beforehand. Also visual quality is as subjective as you can get. The fact is that there will be quality loss. So what i said is not false.

[manono]

Hi-

Is the SSA format playable as such?

If I'm understanding you correctly, then the answer's yes. If you have VobSub installed, and if the SSA file name is the same as the movie name (Movie.avi and Movie.ssa), then you can keep the subtitles separate, you don't have to reencode to burn them into the video, and they'll play just fine through DirectVobSub.

[ntojzan]

To both Cyberman and StephanV:

a) you can have both uncompressed RGB and YUV files. If the source is compressed YUV and you decompress it to YUV and then recompress using another lossy compression technique, the result will be the same as if you were using straight recompress to another lossy compression from the same source.
b) Saiyr clearly stated that he needs to recompress the video file in order to render the subtitles on it. As far as I know (I might be wrong, please correct me if I am) the process of rendering subtitles over a video file is usually done in RGB colorspace, so even decompressing to RGB uncompressed video file will give the exact same result as straight recompress in this specific case.
c) Lossy compression does not loose quality _ALWAYS_. When you have a MPEG-4 source compressed using quantization X, and you decompress it, and later compress it back to another MPEG-4 variant using the same quantizer, no additional loss will occur because of the nature of quantization.

To Saiyr:

Manono already gave you the solution for SSA, and for SRT and SUB files I use DIVXG400 in order to load the subtitles automaticly. It also works if you have the subtitle file using the same name but different extension as the video file.

[stephanV]

Originally posted by ntojzan
To both Cyberman and StephanV:

b) Saiyr clearly stated that he needs to recompress the video file in order to render the subtitles on it. As far as I know (I might be wrong, please correct me if I am) the process of rendering subtitles over a video file is usually done in RGB colorspace, so even decompressing to RGB uncompressed video file will give the exact same result as straight recompress in this specific case.

He clearly stated it AFTER our comments. Anyway, saying you can add text to a video, re-encode it and then have no qualtiy loss is a bit weird.

c) Lossy compression does not loose quality _ALWAYS_. When you have a MPEG-4 source compressed using quantization X, and you decompress it, and later compress it back to another MPEG-4 variant using the same quantizer, no additional loss will occur because of the nature of quantization.


this is complete nonsense.

1. Different MPEG4-varaints dont compress exactly in the same way, even if they use the same quants. otherwise the whole the discussion about them would be useless.

2. I just tested it with recompressing the same file with 3ivx quant = 3, VDM 1.5.4, fast recompress.

file one was 3,692 kB
file two was 3,612 kB
file three was 3,566 kB

and you would argue these files all have the same quality? or maybe theres a bug in 3ivx? i will try XviD and DivX later on.

[ntojzan]

<quote>
Anyway, saying you can add text to a video, re-encode it and then have no qualtiy loss is a bit weird.
</quote>

I'm not saying that. What I'm saying is that cases a) and b) will have the EXACT same output:

case: a)
- decompress the video file
- write the uncompressed video file
- read the uncompressed video file
- render the subtitles
- compress the video file

case: b)
- decompress the video file
- render the subtitles
- compress the video file

I'm saying THAT right from the beginning, because someone stated that if you first decompress the file, and then compress it back using another compression will give worse quality than if you recompress the file. btw: I know that a) variant is nonsense, just wanted to correct a statement which is not true.

<quote>
this is complete nonsense.

1. Different MPEG4-varaints dont compress exactly in the same way, even if they use the same quants. otherwise the whole the discussion about them would be useless.

2. I just tested it with recompressing the same file with 3ivx quant = 3, VDM 1.5.4, fast recompress.
</quote>

It is not complete nonsense. I'm happend to doing a research on a new video codec right now, and I know that my compression sheme will not lose quality on already quantized video. However I believe you that 3ivx does. They maybe quantize it differently, I dont know. btw: If you read my first post CAREFULY you'll find that I said that there are some RARE cases when further quality loss will not occur. So I'm not saying that it will NEVER occur. I'm just stating that lossy compression does not ALWAYS lose quality.

[stephanV]

well, that might be true in your compression scheme, but i dont think it holds for MPEG4. even not in rare cases. well, if your input is a black screen i dont expect the quality to get worse. but thats not really practical example.

[ntojzan]

I wasn't speaking about practice. I just stated that LOSSY COMPRESSION DOES NOT ALWAYS LOSE QUALITY. And it is proven even by your own black frame example. :)

[stephanV]

:)

well, still i think that the general rule should be to always avoid recompression and that compression with a lossy codec will usually result in quality loss. theres no reason to needlessly confuse beginners, like you might be doing now.

remember how this thread started off:

I don't know if this is true, but my friend suggested that doing a full decompress before recompressing in a different codec would increase the quality, as opposed to just going straight to one codec to another (e.g., DivX->XviD).
the guy was actually wondering if it would INCREASE the quality.

in this context it would have been better if you left your comments for a more "advanced" thread where you would explain your compression scheme, dont you think?

im actually quite interested in it. im not an expert in compression though, but im still wondering how you would avoid lossy compression.

[ntojzan]

<quote>
well, still i think that the general rule should be to always avoid recompression and that compression with a lossy codec will usually result in quality loss.
</quote>

Agree in 100%.


<quote>
in this context it would have been better if you left your comments for a more "advanced" thread where you would explain your compression scheme, dont you think?
</quote>

It was not about my compression sheme, I just dont like when people are using false theorems (like: lossy compression ALWAYS lose quality) - even if those theorems are ALMOST ALWAYS aplyable. The correct theorem would be: lossy compression MOSTLY lose quality. Hope I'm not too ennoying? :)


<quote>
im actually quite interested in it. im not an expert in compression though, but im still wondering how you would avoid lossy compression.
</quote>

Well, it's quite simple, that's my opsession... And I care, the video files dont - I win. :) (btw: Ford Prefect did put it right)

Seriously: I didnt avoid lossy compression, I just avoided lossy recompression of similairly compressed sources. If you compress an uncompressed video file, or a video file compressed with another type of compression, you'll have loss. Otherwise you'd have loss only at the parts you've changed. (example: the parts of the frames where the subtitles are rendered)

[stephanV]

let me rephrase again:

what you are saying is:

an uncompressed frame X, compressed with your scheme will result in a frame Y with lower quality. but recompressing frame Y with your scheme would result in frame Y again?

again i ask, how would you avoid frame Y not turning into frame Z with even lower quality assuming that frame Y is presented to your codec as an image and not as combination of intra-blocks and predicted blocks (if your scheme works that way?)

[ntojzan]

We are a bit off the topic here, so I suggest if you have further question to do it in the New video formats -> Codecs -> VDWL codec thread.

However my compression sheme works on a bit different way, it does the motion compensation differently, viewing the arraw of frames as one 3D image, and that way it is possible to recompress the video without furter loss.

[Cyberman]

Originally posted by ntojzan
What I'm saying is that cases a) and b) will have the EXACT same output:

case: a)
- decompress the video file
- write the uncompressed video file
- read the uncompressed video file
- render the subtitles
- compress the video file

case: b)
- decompress the video file
- render the subtitles
- compress the video file

I'm saying THAT right from the beginning, because someone stated that if you first decompress the file, and then compress it back using another compression will give worse quality than if you recompress the file. btw: I know that a) variant is nonsense, just wanted to correct a statement which is not true.

In this case, I agree.
What I´ve been talking about, though, was this:

case a)
- decompress using Codec X
- write uncompressed
- read uncompressed
- compress using Codec Y

case b)
- decompress using Codec X
- compress using CodeC Y

If we assume that Codec X and Codec Y are both using the YUV color space, and the program used to decompress/save uses RGB(as does VirtualDub), then it should be fairly obvious that case a makes conversion from YUV to RGB to YUV necessary, while in the latter case that needn´t be.
AFAIK, a conversion from YUV to RGB does lose information. Even though you won´t notice.

[Asmodian]

Vdub doesn't need to use RGB, just use the fast recompression mode.

There is no difference between using codec X or Y right? It wouldn't change anything if it was codec X to codec X (if they use the same color space)?

[Saiyr]

Fast recompression for me isn't an option. Subtitler filter automatically makes you use full processing mode.

[Stux]

Originally posted by ntojzan
<quote>
this is complete nonsense.

1. Different MPEG4-varaints dont compress exactly in the same way, even if they use the same quants. otherwise the whole the discussion about them would be useless.

2. I just tested it with recompressing the same file with 3ivx quant = 3, VDM 1.5.4, fast recompress.
</quote>

It is not complete nonsense. I'm happend to doing a research on a new video codec right now, and I know that my compression sheme will not lose quality on already quantized video. However I believe you that 3ivx does. They maybe quantize it differently, I dont know. btw: If you read my first post CAREFULY you'll find that I said that there are some RARE cases when further quality loss will not occur. So I'm not saying that it will NEVER occur. I'm just stating that lossy compression does not ALWAYS lose quality.

What you say is only really true for INTRA frame only codecs.

Basically, once a coefficient is quantized, then it can be dequantized and requantized (with the same quantizer) with no loss.

The reason this does not apply to INTER frames is because the reference picture used for the motion search is substantially different the second time around (the lossy version), there by resulting in slightly different motion vectors, which results in a different error signal, which results in new values which results in extra quantizer loss, which dooms the cycle to repeat (since the next generations reference pictures will again be slightly different (more loss).

This is why the files are approximately the same when re-encoded again and again, but still suffering more loss (quantization losses)

INTRA frame only codecs include Motion-JPEG and the DV codecs.

[ntojzan]

The reason this does not apply to INTER frames is because the reference picture used for the motion search is substantially different the second time around (the lossy version), there by resulting in slightly different motion vectors...


This is interesting. I always tought that the decompressed reference picture will give the same motion vectors as the original... Well, anyway you surely can't exclude the possibility that SOMETIMES it will find the same motion vectors, can you?

[Asmodian]

@Saiyr
Have you looked into avisynth? It has the ability to overlay subtitles in yv12 (using textsub – part of vsfilter from Gabest) if you really want to do burnt in subs. It is faster too.

As others have suggested, overlaying the subs onto the video at playback is preferred. Even if you are not recompressing the video (multiple lossy compressions) the quality is better without burnt in subtitles because text is hard to compress due to all those hard edges. The subs also look better because they can be rendered at a higher resolution yielding less aliasing and they don't have ringing artifacts from the compression. Plus if others want them smaller/larger, moved up/down, or not there at all they can do that (if they know how :p ).

[Saiyr]

Nope, I'll look into it, hopefully it won't take too much work. I started using SubtitleWorkshop and it doesn't have full compatibility with SSA anyway. Nerr.

[Stux]

Originally posted by ntojzan
This is interesting. I always tought that the decompressed reference picture will give the same motion vectors as the original... Well, anyway you surely can't exclude the possibility that SOMETIMES it will find the same motion vectors, can you?

Sure, but the point is, sometimes they're not the same... and that's what's important when talking about whether something is lossy or not lossy

[ntojzan]

[QUOTE
...the point is, sometimes they're not the same... and that's what's important when talking about whether something is lossy or not lossy[/QUOTE]

Please reread the previous posts. It was not about whether something is lossy or not lossy. My statement was (repeating for the 5th time):

Lossy compression does not ALWAYS lose quality during recompression.

So, generally I was saying that it MOSTLY do lose quality, but that there are some RARE cases when it does not. (for example recompressing black frames) The reason of all that was the fact that people (mostly) accept the FALSE theorem that:

Lossy compression ALWAYS lose quality.

[killingspree]

ok, i'Ve watched this long enough now,

@ntojzan: i think we all get your point here, now i'm going to try to make one :)

in short, in my eyes this discussion is totally irrelevant, so out of 120000 frames, you may be able to recompress a couple without quality loss, still the overall quality won't improve a lot...

anyway, be it as you said, or not, this discussion does not fit into this forum (the newbies section), if you want to lead this discussion further (and find someone who's willing to argue with you about this) - you'll most likely have to provide a couple of stronger arguments, along with some hard evidence (sample clips anyone?) to bring some sense into this discussion. And if you continue it or not, i don't care, but don't do it here! go to the new codecs forum, or maybe the avisynth forum. there you might find some people that will lead this discussion further down a constructive road!

from now on I ask you, and to the same extent all the others to only reply to this thread, if you have some constructive piece of information to give to Saiyr. anything that is not closely connected to a resolution for his problem will be immediately deleted and you may face a rule 3 strike!

@Saiyr: please, post whatever questions you still have regarding your problem, we're glad to help you!

kr
steVe

[Stux]

Originally posted by ntojzan

...the point is, sometimes they're not the same... and that's what's important when talking about whether something is lossy or not lossy

Please reread the previous posts. It was not about whether something is lossy or not lossy. My statement was (repeating for the 5th time):

Lossy compression does not ALWAYS lose quality during recompression.


Your statement was also:


When you have a MPEG-4 source compressed using quantization X, and you decompress it, and later compress it back to another MPEG-4 variant using the same quantizer, no additional loss will occur because of the nature of quantization.


Basically, I'm telling you this is false, and that is *my* point (and probably everyone else's)

Further, StephenV empirically proved your statement wrong

I just tested it with recompressing the same file with 3ivx quant = 3, VDM 1.5.4, fast recompress.

file one was 3,692 kB
file two was 3,612 kB
file three was 3,566 kB


Of course, you implied this was because he (or 3ivx) used a different quantizer?

No, its because your statement was wrong that when recompressing MPEG-4 to MPEG-4 with the same quantizer "no additional loss will occur because of the nature of quantization."

And I provided at least one theoretical reason why, its because of the motion search. (Which is not a different quantizer btw)

Now, THAT is what I'm calling you up on.

This will probably be my last post in this thread, I believe I have made my point ;)

[Stux]

Answering the original question

theoretically, if you must re-encode from one lossy format to another (for instance, to burn-in the subtitles with virtual dub then going

lossy1->lossy2

to

lossy1->lossless->lossy2 will make no difference

the lossless step is lossless, so its as if it never happened.

Of course, there maybe be other conversions you don't know about for instance YUV->RGB and RGB->YUV

YUV and RGB conversions can also be thought of as lossy processes (we are talking integer video here) which have their own quantization errors.

Where it makes a difference is if you have the following process

lossy1->lossy2->lossy3

for example, capture into lossy1, then edit out the ads and remove analog video noise and re-encode into lossy2, then subtitle and re-encode into lossy3

If you were to replace lossy2 with a lossless video (such as HuffYUV) then you will suffer no extra loss over what you *have* to suffer.

The ideal situation is to capture into a lossless format too... then only the final encode is lossy

[ammck55]

Yes, it will be your last post to this thread. (Well, next to last, ya snuck another one in on me before I could close this, you rascal!) Killingspree specifically asked you guys to stop this discussion, or barring that outcome, take it up in a more appropriate setting, but apparently that wasn't quite clear enough. I do commend you gentlemen on not turning this into a "heat wave", but this is enough.

The last post that directly addressed Saiyr's problems was published by Manono.....back on the first page of this thread.

Saiyr--If you're still having issues with this, crank up a new thread and post all of your relevant testing results and further research and findings, as....

This thread is closed.

ammck55