AVI T-compressor

[Marc FD]

hat i mean :
A compressor who will take an AVI file (video only),
find the frames who are very similar (eg. FlashBacks)
hold one and replace the other by an alias
(eg : this frames is same as frame xxx)
It could be usefull for archiving.
I think it could reduce significantly the size of
files with no visible loss in quality when
used on a special content (I think of series
of animes like "NGE" (eva) or "Noir"
who have a big redudancy on the Time Axis,
if you see what i mean
(tons of flashbacks, ect... )

My Answers :
- Does this already exist ??
(if s.o. coded it for fun and never released
it because he founded it was not usefull )
(Or if everyone -but me- is using it already )

- What do you think of it (great or worthless) ?

[Koepi]

Hu, dunno if it could work, you'd need a bubble-sort like algorithm there. So it's _very_ slow to encode that way. And you'd need to construct a DSF ("uncompressing" such content should be done within very few CPU cycles).

I like the idea, but I don't think you can really make it that way.

Just look at the noise such a video has:
If it really would be possible, all mpeg-files would be MUCH smaller, as there are only very few MBs moving. But due to the noise/flicker the algorithms don't find those similiarities, as from the pure data-point-of-view they are different.

So if you'd somehow "clean" all the colours in a cartoon, it _might_ be possible to do.

Just my first ideas,

regards,
Koepi

[UGAthecat]

this would probably be doable with a shape based encoder, as the shapes from one section to the next would be more easily recognized, but of course were probably still years away from a working shape recognition based encoder.

[Marc FD]

I think i was not explicit enough but i'm
happy to see there is interest in the thing
I don't want to create a codec nor AVI files !
A better explanation of what T-Compression should do :
1) Take an AVI file. (video only)
(compressed with XviD/DivX 3/4/5/or whatever else)
2) Find similar frames packs (i.e. similar when highly resized)
3) Cutting at I-frames (like VirtualDub can do), shrink the
AVI file in small parts.
4) Check that you have parts who match in size/length/appearance
5) Choose the better (with less noise)
6) Keep it. Throw the bad one.
7) Make a file with your avi packs and the replace the delete
ones by aliases
8) To play the files, Expand the compressed file to an AVI
( I thing realtime playback should be possible )
Since the redundancy on the Time-axis is already used by all
delta-frames video codecs, but only on 1 (P-Frames) or 2 (B-Frames)
frames (about 1/10 of a second max). The T-Compressor should work on a bigger scale (i.e. scene-changes, who are easy to cut with no loss of quality)
I know every content would not be compressible but for animes (see before), i'm sure we could shrink alot , specially because the exactly sames scene are often played many times (costs less to reutilize it )

@Koepi : Does it seems possible to do ? It could be implement using the MPEG-7 standard if possible (i'm not well informed on MPEG-7 but load and .mp7 file in BSPlayer (my favourite video player since i'm on XP , M$ player XP really s*cks) could be a cool experience

@UGA : Do you see what i think now ?

[Selur]

The idea sound promessing, but I don't think it'll work,..
Point number 2&5 sounds a bit too lossy for my taste,... (especially for non anime movies) sounds like some sort of fractal compression where u would take the whole movie as one pic,...

just the thoughts that popped up my mind,...

Cu Selur

Originally posted by Marc FD 8) To play the files, Expand the compressed file to an AVI
( I thing realtime playback should be possible )

.. as Koepi was already pointing out, the problem here is to find an algo that will really find the similar picture for you. In case this would ever be possible, you cant do it with AVI. Maybe a new container, like Ogg or MCF, could be prepared for this, by allowing to tag some frames that way that they are repeated in the parser for a defined number of times ( 2nd tag ). As these frames have to be key frames i doubt you will really get any savings in the end ... unless you have still pictures for lets say a couple of seconds .... and even then you have to understand that there is not much delta information between consecutive frames, so the P frames are quite small and dont take much space ...

[Marc FD]

Is my english so worse ??

@Christian
what i propose is DOABLE, the only thing is to automate it...
i don't want to re-encode frames,nor even mofify them
(but creating I-frames with XviD could be cool..)

I will make an exemple you can try yourself (it would just be very,very,very,very.......,very long (it why we need to automate it)
Load an AVI file twice with 2 VDubs (at the same time)
Look at the 1st Scene (between the 2 firsts KFs)
- compare it with all the other scenes of the file
IF you see 2 scenes who are the same
(Warning : if I was not clear enough, this is impossible with normal movie/video content. ONLY with cartoons (who artificialy created). The thing is that when the creators of a cartoon re-use a scene, you would re-use it too ! cool, isn't it ?)

THEN you cut the scene (1) and destroy the other (2) using VDub/AVS
when you have cutted all the scenes who where the same, you create a little AVS script : (it is only a sample)

your original file was 30 min long. you splited it in 5 parts
(00-10,10-15,15-20,20-25,25-30) the 10-15 scene was the same
as the 20-25 scene. Ok ?

# So here we go
b = AVIsource("00-10min.avi")
m = AVIsource("15-20min.avi")
e = AVIsource("25-30min.avi")

s = AVIsource("10-15min.avi")

result = b + s + m + s + e
# Got It ?? (only to play the file, not to encode it)

I think you have never seen the 10 firsts Noir : You get a 1 min long flashback 3 times per episode !!! when i think to the size reduction we can get WITHOUT QUALITY REDUCTION, (it's already encoded)
I thinked there was something to do : T-compression
(i think the size reduction can be 5% to 25%, from what i've calculated)

It's a bit like cutting a movie
you don't lose quality but you cut the superficial things and make your movie lighter (a little "metaphore" (dunno the word in english))

I thing u've got it now. sorry if i didn't say it before so, but it is not easy to explain.

@Selur : you see, it more, more simple than "fractal compression" (who would take more than forever.. may be in decades )
but you are right (i thing you've got it) : It's ONLY for anime/cartoons : it's based on the time-redundacy of these artificaly made video-content (if we created a format, we could call it .amc )
(i would explain it..)

i will answer you :
for 2), the difficulty is to find scenes who are the sames.
I think using progressive matching, highly shrinked resizes, heavily filtered sources, we can make a very fast and accurate detector.
for 5), when 2 sames scenes are detected, i think the prog should simply show the scenes to the user who will select it. It could say which is the smaller too.

well, if you haven't understood what in my mind, keep answering...
I'm here for that... )

[gldblade]

>5% to 25%, from what i've calculated

Repeats take up to 1/4 of the movie?! That's a bit extreme, isn't it?

>Find similar frames packs

That in itself could take quite a bit of processing time. Let's assume, there are 100 "packs". That would essentially mean 4950 comparisons already. Let's up it to something a little more realistic. At 2,500 packs, that would be 3,123,750 compares. At 5,000 packs, 12,497,500. Imagine repeating an algorithm to compare two packs that many times...

You could possibly cut some of these comparisons off. You could look for packs of the same length and similar size and then compare. That would make it quite a bit faster.

>You get a 1 min long flashback 3 times per episode !!!

You would be getting rid of about 2 minutes. How does 2 minutes compare to the rest of the episode? I don't think it would be very much.

[Sven Bent]

it would be like kind of alpha frames but with a longer compression window.

in anime cartons many times the bagground consit of 3-5 still pictures repete in a cyklus.

a T-compressor here or an alpha frame which kan have an source longer away than just one frame woulde be great



N.B.
please corret me if I'm wrong that normal alpha frames can only save similarities between previus or next frame.

[UGAthecat]

I had a similar thought once, except I was thinking if, whenever the codec detected it should insert a new keyframe, it would look at all previous keyframes (or a certain number of previous keyframes to reduce stress on the system), and if a matching frame was found (or a frame that could be used to make the current frame a p frame instead of an i frame) it would use that frame.
This would help a lot in sequences where there are strobing lights where every other frame is made into a keyframe because one is dark, the next has a picture, the next is dark, repeat, repeat, repeat.

so if codec says frame 5998, 5999, and 6000 should all be a keyframe (I have quite a few videos that have done this), but frame 5998 is exactly the same as 6000, it just reloads 5998 again, and goes on to 6001. If frame 5998 was only a partial match, it would reload (but not render on screen) 5998, and render 6000 as a P frame based on 5998.
I frames that get re-used could be specifically marked so that when the decoding is being done they stay in memory until the last frame that reuses them gets processed to prevent skipping back and forth in the file.
end result, more memory used during encoding and decoding, and more comparisons being done for each frame being detected as I frame during encode, but ultimately a great space saver. Even the amount of additional memory required to support this could be reduced by only allowing 5 or so previous I frames to be kept.

[Marc FD]

That was exacltly my 2nd idea to improve compression
(the better ideas at last ....)
It think there are 3 problems but i think it's a good idea (because i had it too )

1) this will slow down the encoding and require a lot of memory
BUT it could be done using a 1,5 pass (this is not a real problem)

2) I'm 95% sure that it is impossible in the MPEG-4 specs so XviD could not make it (the original version i mean)

3) I'm 95% sure too that vfw feed the codec with one frame by one, so
you will need to keep your frames in memory (but it is still doable, it's a bit like B-frames from DivX 5)

So the idea is really good (see before) but it would be so cool if it could be doable in the MPEG-4 specs (i'm a XviD unconditianal fan)

If it can't be done using the MPEG-4 specs, i think we need a new standart for Alpha-Frames,Background-frames and Copy-frames

[-h]

a T-compressor here or an alpha frame which kan have an source longer away than just one frame woulde be great

This is possible in "newer" MPEG derivatives such as JVT - a frame can reference not only immediately neighbouring, but any designated frame back to the previous I-frame. Of course it is extremely CPU-intensive.

If it can't be done using the MPEG-4 specs, i think we need a new standart for Alpha-Frames,Background-frames and Copy-frames

It would be possible to encode an entire frame or common sub-frame image as an MPEG-4 sprite, then repeatedly display it as needed over the subsequent frames. Of course this is enormously complex as well, and XviD does not yet support such functionality.

Cartoon/anime really deserves its own dedicated codec.

-h

[Atamido]

Encoding in the way originaly mentioned by Marc FD would undoubtedly be far to complex to be handled by reencoding an existing video because of the difficulty comparing so many frames and determining if differences in a series of frames are static or intentional.

But it might be possible when using computer animation. A program could certainly determine if a sequence was completely identical, and it could, if possible, tell the codec to duplicate a previous video stream at this location in this space.

But I don't see anyone actually going through the trouble of trying to implement this in a rendering program as space saving usually isn't a very big concern.

[bb]

Originally posted by Pamel
But I don't see anyone actually going through the trouble of trying to implement this in a rendering program as space saving usually isn't a very big concern.

Space saving isn't a very big concern? So what's all the video compression technology about?

bb

[Marc FD]

@-h

Cartoon/anime really deserves its own dedicated codec.

i agree. I'd like so much XviD could be it (i'm crazy about XviD)
something like XviD-AnimeMediaCompressor
I'll learn C/C++ because i want to help to code a so cool codec
I think XviD could have an Anime-mode who would be CPU-Intensive and who would try everything to give the best compressed content
i'd like to be good enough in coding to do it but i have many many things to learn before (hey i'm sure i'm the youngest on the entire Doom9 forum , and guys who have my age are not crazy enough to code )

How about my third and last idea ?? (the _BEST_ )
a simple way to improve anime compression should be Variable FrameRate like RealVideo do (i think it's why RV is so good on anime content)
but it is impossible with AVI
You will say me i can reduce the framerate by decimation but the problem is that almost anime i have have 24 fps scene (action),
12 fps scenes (pan,zoom) and 8 fps scenes (freezes) so if i decimate such content i will end with horrible action scenes but i makes me
sick when i think about the quantity of data who is spend for nothing in 12 and 8 fps scenes :
Make a try : use an avisyth script when you freeze frames to get 1 fps and then you encode it at 24 fps with any codec like XviD / DivX. you will get 50% of the data used after the new frames in delta frames who just save noise and nothing else

I've an idea who (i think) it's simple to implement :
>> An automatic frame-dropper for XviD
Generaly you avoid to drop frames but for these particular contents, when two consecutives frames are pratically the same (no movement, just noise), the codec will drop the second frame and the third if it is still the same and so on ...
i know it could seems odd but i'm sure with this feature correctly used on anime content you should have an amazing compression improvement and better achived quality because the first frame of a PD
(inter+dropped) section will get the entire bitrate !!
The thing is to create an algo who can determine when 2 consecutives frames are 2 copy of the same. a first version could use constant values to determine when 2 consecutives frames are the same :
- below a defined amount of movement
- below a defined amount of difference (noise only)
- if predecessing frames where dropped
(because it should make IPDPDPDPDPDPDPDPDPDPDPDPD blocks
( here 12 fps))

you can think i'm crazy but i'm very used to anime so i think the biggest reason RV is so incredible on anime content when you compare it with DivX / XviD is that he use and abuse of veriable frame rates.
i know too that a very good way to improve quality when you use very low bitrates is to reduce the frame rate but we need (i think we = the anime encoders community ) something more flexible because it can vary very fast (ie a panning just after a fast action scene)
I think XviD+framedropping is the best way to make the XviD codec (the better on HQ movie content) an alternative for RV9 on low bitrate / Anime content because RV9 is clearly the best now on these content

if an XviD programmer could hear my prayer i think i would never have to say to me : <<shit RV9 is better than XviD but i want to use XviD anyway because i'm totaly crazy about it and i cannot use another codec for my encodes>> (i make tests only with DivX/RV)

Thanks !

[Marc FD]

@gldblade
excuse me i havn't answer to your previous post

Repeats take up to 1/4 of the movie?! That's a bit extreme, isn't it?

yes i know but you are right: what i think is 25% of repeat = 12.5% of reduction (more,more much if you count the op/end )

At 5,000 packs, 12,497,500. Imagine repeating an algorithm to compare two packs that many times...

(12,495,001 i've calculated)

it could seems extreme but you only need to detect the frames so you make a 1,5 pass (after first before second ) where you use ie 64*48 resolution. (100x less data than your source) it's far enough to see if two frames are the sames or not (you can do better verification after when you have found the frames)
then you don't compare massively these frames : you compare only some pixels and the order them... then they are ordered by averaged pel YUV value differences. when you have tested 10 or 20 pels you have only to compare a frame with the 10 before and the 10 after.
with a 1-2 Ghz fast computer i think this can be done very very fast ( when you have an exponantial complex pb you use exponential algos and it is doable )
And there are more much optimisations who can be done as you said
(5'000 packs ??? there are about 500 KF per hour in a movie and about 3 scenes per minute = a 27h45 long movie ??? seems huge )

You would be getting rid of about 2 minutes. How does 2 minutes compare to the rest of the episode? I don't think it would be very much.

2 minutes of a 20 minutes-long anime (like 99% of animes) = 10% !
if you are not interessed by 10% file reduction without quality reduction you should simply use MPEG-2 and make 10-CD long encodes
(i'm just kidding )

[gldblade]

>2 minutes of a 20 minutes-long anime (like 99% of animes) = 10% !
if you are not interessed by 10% file reduction without quality reduction you should simply use MPEG-2 and make 10-CD long encodes

Sorry, I've never ever watched an anime. As such, 20 minutes seems to be extremely short to me.

>(5'000 packs ??? there are about 500 KF per hour in a movie and about 3 scenes per minute = a 27h45 long movie ??? seems huge )

Heh, I didn't really know what number to pick and I didn't bother to calculate. 5,000 just seemed like a good number to use. BTW, I pretty sure it's quite a bit higher than 3 scenes per minute. Only 60 scenes per anime?

Ok, it's more doable than I thought.

[Marc FD]

the thing with anime is that in a 26 episode season the drawers reuse the scenes so you've got a lot of redundancy
3 scenes per minute is a bit extreme ...
it depends... i'll say with a average duration of 10 sec = 6 scenes per minute. It depends on the quantity of action
(it' why it's very hard to use the same size for 26 epos : they have not the same compressibility so you need to make const quant tests to see the compressibility and then big calculations to fit everything on the number of CD you want (2 to 8 )

Anime encode is the harder thing to do very good because when something is too compressed, you see it immediatly
(it's why DivX 3.12 SBC is very used with anime, and why i do reencodes : too much low-level encoders who can't make something unreprochable (for VCR records especially) : the fools don't use XviD , nor avs/vdub filters to improve compressibility)
but the content itself is potentially very compressible

[-h]

i agree. I'd like so much XviD could be it (i'm crazy about XviD)
something like XviD-AnimeMediaCompressor

gruel (an XviD developer) has mentioned quite a few times that he wants XviD to be as good at encoding anime/cartoons as is possible inside MPEG-4. Also, if he believes he can do better by forking the codec to a non-MPEG-4 "anime codec", he intends to do so eventually.

a simple way to improve anime compression should be Variable FrameRate like RealVideo do (i think it's why RV is so good on anime content)

It's possible - just drop the frame. MPEG-4 supports this with a "not-coded" flag in the frame info, such that dropping a frame should only consume 24 bytes or so. suxen_drol meant to code this feature a while back, but hasn't had the time to do so. It's not all that complicated, and yes it'll give significant benefits for (clean) anime sources.

It could be a tad complex to achieve with a high-quality encode (like quant=2 or 3), as there will be quantization noise encoded which will trigger blocks to be coded instead of skipped. But this can be handled also.

-h

[Marc FD]

i hope gruel will join us in this thread then
i'm learning C++ (i'll try to make an avs or a Vdub filter)
then i will be able to help you on the XviD project )
(i think i can learn C++ pretty fast because i'm already not to bad in coding (Delphi is maybe easier to learn than C++, but it is as powefull

I'll try to make a filter who will fix entire frames when they seems to be very close, so the codec would have perfectly same frames to handle it would be easier to code