MagicYUV - a new fast lossless video codec

[nekosama]

Fixed the problem by swapping avisource with ffmpegsource2, found out that avisource can bug out with big files.

[zerowalker]

How big?

[nekosama]

5.2 GB, 704x450, not 'that' big though it seems like the only logical cause till something new pops up.

[zerowalker]

Indeed not that big.
I can easily have 300gb of Video without issue.
Something must be wrong on your end, probably some complications, mismatching etc.

[the_weirdo]

Originally Posted by nekosama

Fixed the problem by swapping avisource with ffmpegsource2, found out that avisource can bug out with big files.

Wait, I don't think FFMS2 can decode MagicYUV sources, at least by now. Your avi may not be a MagicYUV encode then.

[Sparktank]

I've used SD resolution with MagicYUV as an intermediate without problems.
>30GB for lossless intermediates and no issues.

[nekosama]

Yes my bad the_weirdo, made an x264 lossless in the meantime and mistook it for magicyuv. Though it's still the same problem with trying to filter a magic yuv source with avisource input.

[foxyshadis]

Originally Posted by zerowalker

Indeed not that big.
I can easily have 300gb of Video without issue.
Something must be wrong on your end, probably some complications, mismatching etc.

He's got to be using Avisynth 2.5. 2.6 has lots of fixes for 4+ GB AVI files.

nekosama, try updating to 2.6a5.

[De-M-oN]

Originally Posted by foxyshadis

He's got to be using Avisynth 2.5. 2.6 has lots of fixes for 4+ GB AVI files.

I could load with 2.5 avis with similar sizes too though.

[zerowalker]

Have you been able to test something with GPU acceleration, HSA etc?
We talked about it before and know you were interested in it, so wondered if you have had any chance to test it, or some ideas?

Thanks

[Ignus2]

Originally Posted by zerowalker

Have you been able to test something with GPU acceleration, HSA etc?
We talked about it before and know you were interested in it, so wondered if you have had any chance to test it, or some ideas?

Thanks

Hehe, well, I'd love to, but time doesn't permit :-/
Also, I don't yet have hardware to develop on, so that's also an obstacle.
It's still something for the future to try out.

Greets,
I.

[zerowalker]

Ah, well was expecting that. Not by any means saying you Should have done it by now, but asking doesn't hurt;P

Just that you have the interest in it is a good thing, as i myself find it an interesting approach, as it holds some hope.

Well will have to see what happens in the future when you got the time,will and hardware to do any sort of testing!

Thanks

[Mick]

Hi there,
i downloaded RC5 and 1.0 for my Win2K Pro System and the Configuration Tool for the Codecs simply Reports the Error Message "Function not supported" and quits. The MagicYUV Codec in Versions RC5/1.0 is about 20 Frames slower (!) with a higher CPU (28-35 % more) load than the RC4 that caused no trouble with Win2K Pro (SP4). Sure, i could step up to another OS but i don't want that for several reasons, so let's leave that Discussion out right now.

Typical CPU load in Virtualub (Encoding 4K YUY2) with W2K/XP:
HuffYuv -> between 15 to 35 %
Lagarith -> between 40 to 65 %
UtVideo -> between 48 to 70 %
MagicYUV -> between 28 to 40 % (RC4), 62 to 88 % (RC5/1.0)

Resolution: 4K PAL at 25 Fps with PCM Audio, 1:1 Interleaved

Note: The CPU load is about the same on a Win7 64 Bit System.

Decoding/Playback 4K PAL Windowed/Full Screen:
HuffYuv -> fluent and stable Frame rate (+)
Lagarith -> stutters, drops Frame rate down (-)
UtVideo -> almost fluent, drops down to about 20 Fps (-)
MagicYUV -> fluent, drops occasionally down few frames (+)

Note: The Playback was the same on W2K/XP/Win7 (32/64 Bit) with Intel, ATI, NVidia and Matrox Boards

Legend:
(+) -> Audio in Sync with Video
(-) -> Audio out of Sync with Video

Further, "VideoPad" from NCH rejects to load AVI's encoded with MagicYUV, regardless of the settings and FourCC's used.

You can find the Software here and test it yourself:
http://www.nchsoftware.com/videopad/index.html

Further, if the "UtVideo" Codec is also installed on W2K (SP4)/XP (SP3), then MagicYUV seem to "stutter" at Resolutions over 4K encoding/decoding Video. I noticed this with the UtVideo Codec Version 13.3.1 and believe it's the included DMO Module from UtVideo that causes Problems. When UtVideo is removed from the System, everything is back to normal with MagicYUV. Plus, MagicYUV co-exists fine with Lagarith and HuffYuv without any Problems.

Speaking of UtVideo/VirtualDub and "v210" 10 Bit: Both produce Videos that can not (!) be played on OptiBase, AJA/Kona or DVS I/O Boards. The only Codec so far is the Drastic Codec that fully conforms to the Industrial standard and Definition for 10 Bit. So, if you plan to develop in this "10 Bit Playground", chances are that it won't work with professional Equipment.

Example:
A Colleague of mine encoded 3 TB of Material with UtVideo's v210 Codec to save space and has to start all over again now because the Files can not be used on such Systems he made them for, even if the UtVideo Codec is installed on those Systems. On a DVS HD Station, only the first 3 Frames can be loaded, the rest is green without Sound and takes awfully long to load.

In VirtualDub, no Problems with Intel, ATI, NVidia and Matrox Boards and the "v210" Codec from UtVideo. also, the UtVideo v210 Codec is painfully slow for encoding with a super high CPU load on all Cores, nothing I can recommend at this point.

You can find "UtVideo" here:
http://umezawa.dyndns.info/archive/utvideo/

Still not solved:
MagicYUV RC4 still reports "RGB24" to other Codecs with VirtualDub (Version 1.6.19 - 1.10.4) no matter how the Settings are with the "Fast Re-compress" and "Normal Re-compress" Modes. HuffYuv and Lagarith use the Color-space they encoded (YV12/YUY2) the Video with.

Interlaced Mode with MagicYUV only takes the TFF (Top Field First) which is the same for all HD Modes, but in SD Resolutions for NTSC with BFF (Bottom Field First) the Image is distorted.

I work a lot with PAL, SECAM and NTSC Material from all over the World, only HuffYuv took the Fields correctly while UtVideo suffers the same Problem like MagicYUV with the TFF only and Lagarith does not handle Interlaced as well as HuffYuv.

To give you a brief Overview about Field Orders:
PAL SD/D1/HD -> Top Field First / Upper Field First
PAL SD DV -> Bottom Field First / Lower Field first
NTSC D1/HD -> Top Field First / Upper Field First
NTSC SD DV -> Bottom Field First / Lower Field First
SECAM -> same as PAL SD/D1/HD

I hope this was helpful to you. Take care

Cheers

Mick

[De-M-oN]

MagicYUV RC4 still reports "RGB24" to other Codecs

Sorry cant agree you with this

[Mick]

Hi De-M-oN,
well, MagicYUV certainly behaves with "Top Field First" with my Typhoon Board switched to NTSC. When I select "BFF" everything is fine but MagicYUV encodes to "TFF". Your Screenshot shows "Parity: Bottom Field First", makes me wonder because the Filter for my Typhoon let's me choose the Field Order and sets it according to the Video standard, never failed me so far.

Most Material i get is the NTSC 4.43 BFF Format. Which one was yours ? NTSC M ? PAL N (PAL NTSC Playback) ?

Your Screenshot shows "YUY2", but why is RGB24 reported from MagicYUV to XviD and x264 in VirtualDub while HuffYuv reports with the same Settings YUY2 ? I really tried all the Settings in MagicYUV but still the Output is RGB24. And yes, under "Color Depth" I chosen for Output "Same as Input" which is YUY2. What are your Settings in VirtualDub ? Maybe I've overseen something.

Cheers

Mick

[De-M-oN]

My video is progressive. The information is given at 6th line : FieldBased (Seperated) Video: NO

My video is a game recording. I recorded with DXTory.

It is a simple progressive 30fps video.

[Ignus2]

Originally Posted by Mick

Hi there,
i downloaded RC5 and 1.0 for my Win2K Pro System and the Configuration Tool for the Codecs simply Reports the Error Message "Function not supported" and quits.

That's possible. The minimum supported OS is Windows XP, the fact that it worked with anything below that was pure luck. Sorry.

Originally Posted by Mick

The MagicYUV Codec in Versions RC5/1.0 is about 20 Frames slower (!) with a higher CPU (28-35 % more) load than the RC4 that caused no trouble with Win2K Pro (SP4). Sure, i could step up to another OS but i don't want that for several reasons, so let's leave that Discussion out right now.

There has been no change algorithm-wise between RC4 and RC5/1.0, so the cause is something else here.

Most of the time the encoded data is not what people think it is. So for example people think it's YUY2, but actually it is RGB. That could explain why it is slower (or why the file is bigger).
The next release (in the coming days) will include an icon in the notification area, which will show exactly what is being decompressed into what format and resolution etc.

Until then, to be sure that you really encoded YUY2, select YUV 4:2:2 for "Accepted colorspace" in the codec config dialog before encoding. That way the codec will reject everything that is not YUV 4:2:2 when encoding.

Originally Posted by Mick

Further, "VideoPad" from NCH rejects to load AVI's encoded with MagicYUV, regardless of the settings and FourCC's used.

You can find the Software here and test it yourself:
http://www.nchsoftware.com/videopad/index.html

I tried the software, and it loads MagicYUV encoded files perfectly for me.

Originally Posted by Mick

Further, if the "UtVideo" Codec is also installed on W2K (SP4)/XP (SP3), then MagicYUV seem to "stutter" at Resolutions over 4K encoding/decoding Video. I noticed this with the UtVideo Codec Version 13.3.1 and believe it's the included DMO Module from UtVideo that causes Problems. When UtVideo is removed from the System, everything is back to normal with MagicYUV. Plus, MagicYUV co-exists fine with Lagarith and HuffYuv without any Problems.

That can happen. I also noticed, that the UtVideo DMO sometimes gets inserted to do needless colorspace conversion or even encode to Ut and then back (!) for no reason.

Originally Posted by Mick

Speaking of UtVideo/VirtualDub and "v210" 10 Bit: Both produce Videos that can not (!) be played on OptiBase, AJA/Kona or DVS I/O Boards. The only Codec so far is the Drastic Codec that fully conforms to the Industrial standard and Definition for 10 Bit. So, if you plan to develop in this "10 Bit Playground", chances are that it won't work with professional Equipment.

The raw v210 AVI produced by the latest (!) VirtualDub conforms to the v210 spec (it was buggy with earlier VDub though!).
I also had someone test my codec with v210, and it played back fine.
But we can discuss/test more about this if you want.

Originally Posted by Mick

Still not solved:
MagicYUV RC4 still reports "RGB24" to other Codecs with VirtualDub (Version 1.6.19 - 1.10.4) no matter how the Settings are with the "Fast Re-compress" and "Normal Re-compress" Modes. HuffYuv and Lagarith use the Color-space they encoded (YV12/YUY2) the Video with.

MagicYUV does the same, it reports the colorspace that was encoded. If it always reports RGB (despite all conversion options disabled in the "Decompression Settings"), then the encoded data has to be RGB.

Originally Posted by Mick

Interlaced Mode with MagicYUV only takes the TFF (Top Field First) which is the same for all HD Modes, but in SD Resolutions for NTSC with BFF (Bottom Field First) the Image is distorted.

Yes, the codec assumes TFF for now.
There is actually no way of knowing if the input is TFF or BFF, but I can include an option for the encoder settings.
What I would like to know is what to do on the decoder side? So for example if I encode BFF, should I also output BFF when decoding? Or always TFF, regardless of the what was encoded (so to swap the fields)? I would need some help with this.

Thanks for the comments. The next release will have the notification icon, so you can check the colorspace, until then use the "Accepted Colorspace" option. Also, if we can get to it, I can get the TFF/BFF be included as well.

Greets,
I.

[zerowalker]

I would like the Encoder to have that Option of choosing TFF or BFF (it's kinda a must, if you use that feature to play interlaced video).
How to do it, well either swap fields like you said or do it the normal way. Do what's most efficient as the end result is the same

Thanks

[ChiDragon]

Alright, here's a stupid question: why does an intra-only lossless codec care whether the material is TFF or BFF?

[zerowalker]

To be able to decode and deinterlace on playback, i think?