Ut Video Codec Suite - a new lossless video codec for Windows!

[kolak]

Can we have 10bit version with v210 pin output and 8bit pin with optional dithering (ordered+noise). Thank you - hehe :)

Latest ffmbc opened bitrate/quantizer in ProRes encoder and with q=1 it goes almost lossless- don't think it's 100% lossless, but PSNR is around 90dB (450Mbit average bitrate, HD, 24p)

[Midzuki]

Version 10.2.4 has been released

Changelog:

(Windows) Installer now displays "before-install" notice.

(Mac) Report more apropreate codec name to QuickTime.

http://umezawa.dyndns.info/archive/utvideo/

OR

http://www.videohelp.com/tools/Ut-Video-Codec-Suite

[smok3]

still waiting for that magic few words on where to drop stuff on osx....

edit: p.s. is anybody working on ffmpeg encoder as well? (the decoding seems to be working fine)

[mp3dom]

+1 for 10bit (it would be fantastic!). ProRes is lossy and uncompressed is extremely huge in filesize. A lossless 10bit codec would be great for original backup/archive.

[SubOne]

I have a newbie question... If the source is 12-bit then is 8-bit UT really lossless? If 12-bit is 0-4096 and 8-bit is 0-256, then 16 in 8-bit = 256 in 12-bit, and 17 in 8-bit = 272 in 12-bit. So what happens if a channel is at level 260 in the 12-bit source? It should be 16.2 in 8-bit but since that is not possible does it get approximated to 16 in 8-bit and thus loses accuracy? Or is there some other mojo going on that ensures UT is actually lossless even at 8-bit? Hope I am making sense at all...

By the way, X264 10-bit works great, but it's not as fast.

[kolak]

UT is lossless only at 8bit.

[kolak]

+1 for 10bit (it would be fantastic!). ProRes is lossy and uncompressed is extremely huge in filesize. A lossless 10bit codec would be great for original backup/archive.

ffmbc has open ProRes, so you can have 99.9% lossless encoding at around 500Mbit.

[yup]

Hi all!
11.0 released
http://umezawa.dyndns.info/archive/utvideo/
yup.

[_gl]

Been using UT for a while and it rocks.

But I would kill (well maybe not kill) for a 4:4:4 YUV flavour (8bit or 10 bit).

I sometimes need to do an intermediate render in Premiere. I only use YUV effects so the entire Premiere pipe is YUV up to the output codec. So if I want it to be truly lossless, 4:4:4 YUV is the only option. Rendering to 4:2:2 (or lower) drops the chroma resolution, and rendering to RGB 8bit causes gradient damage due to the unwanted expansion from studio to PC levels (as well as the YUV->RGB conversion).

I tried using the 4:4:4 mode of HNxHD via Quicktime, but that introduces an unwanted gamma and colour shift (maybe due to Premiere incorrectly using BT601 instead of BT709 for my HD footage). So right now I'm stuck using RGB 8bit lossless (UT or Lagarith).

[SubOne]

Gamma shift is a "feature" of Quicktime/MOV. As with everything Apple, there are no bugs, only features... Plus DNx is lossy.

We really need a 10-bit lossless codec though, yes.

[_gl]

Gamma shift is a "feature" of Quicktime/MOV. As with everything Apple, there are no bugs, only features...

Well, useless then. But the extra colour shift seems to be Premiere using the wrong matrix - or is that also a known Quicktime issue?

Plus DNx is lossy.

Sure, but studio 4:4:4 is sometimes more important to me.

We really need a 10-bit lossless codec though, yes.

Ideally yeah. In Premiere you at least have the option of v210 (uncompressed 10bit 4:2:2), but no 4:4:4 YUV options at all.

[qyot27]

*cough* FFV1 already supports it *cough* (http://git.videolan.org/?p=ffmpeg.git;a=commit;h=1eabd71c2bed13648643433249386f4b965b5c14) (it's had YUV420P10 and YUV422P10 support since May 2011)


If the underlying video is 4:2:0, then 4:4:4 rendering isn't lossless to start with. And unless Premiere can handle 4:4:4 exporting internally (and especially at higher bit depths), it'll always be doing some sort of lossy conversion before the final output render. There's a lot more involved in the situation than just making sure the export or import codecs support 10-bit 4:4:4.

[_gl]

If the underlying video is 4:2:0, then 4:4:4 rendering isn't lossless to start with.

Combinations of processing (grading, sharpening etc) and/or overlaying graphics (logos, gradients etc) can create 4:4:4 detail from 4:2:0 etc. source footage.

And unless Premiere can handle 4:4:4 exporting internally

I'm pretty sure it can, it didn't loose chroma when I tested rendering to DNxHD 4:4:4, and Premiere's internal YUV pipe is actually 4:4:4. Of course it has to be tested for AVI specifically, but there's a good chance it's supported.

[SubOne]

The underlying acquisition footage is RGB 4:4:4, hence the requirement for a 4:4:4 lossless codec. Premiere can definitely export 4:4:4, as evidenced from testing each channel from likes of Uncompressed, DPX or DNx 444.

[Mr_Khyron]

Version 11.1.0

read me (http://umezawa.dyndns.info/archive/utvideo/utvideo-11.1.0-readme.en.html) / installer for Windows (http://umezawa.dyndns.info/archive/utvideo/utvideo-11.1.0-win.exe) / zip for MacOSX (http://umezawa.dyndns.info/archive/utvideo/utvideo-11.1.0-macosx.zip) / source (http://umezawa.dyndns.info/archive/utvideo/utvideo-11.1.0-src.zip)


Performance Improvements

(Mac) Add assembly language version of colorspace conversion routine from RGB to ULY2.
Common: Speed up decoding a little.
Common: Speed up encoding on x64 about 10%

Others

(Mac) encoded size becomes a little smaller by omitting unnecessary media sample flags.

[johnsonlam]

Version 11.1.0
(Mac) Add assembly language version of colorspace conversion routine from RGB to ULY2.
Common: Speed up decoding a little.
Common: Speed up encoding on x64 about 10%
Others
(Mac) encoded size becomes a little smaller by omitting unnecessary media sample flags.

I fail to find the Win 64bit binary, can somebody kindly tell me where?

[the_weirdo]

I fail to find the Win 64bit binary, can somebody kindly tell me where?

The installer will now install both 32-bit and 64-bit version of UtVideo on Windows 64-bit.

[infinityloop]

The installer will now install both 32-bit and 64-bit version of UtVideo on Windows 64-bit.
I'd like to add that 32-bit versions are also required under W7-64 bit, because a 32bit editing or player application can not use the 64-bit codec. :)

[johnsonlam]

I'd like to add that 32-bit versions are also required under W7-64 bit, because a 32bit editing or player application can not use the 64-bit codec. :)

Thanks.
If Umezawa can write down (32 and 64bit) should avoid confusion.

[kypec]

Thanks.
If Umezawa can write down (32 and 64bit) should avoid confusion.
He did write about it (http://umezawa.dyndns.info/archive/utvideo/utvideo-10.2.1-readme.en.html) and you can always look it up in the changelog.

[zerowalker]

Can Null Frames be added?
As that is one of the Main reasons i love Lagarith, as it´s just a waste to recompress identical frames.

Or maybe it´s a reason it can´t be used, something in the encoder or how it works?

[Keiyakusha]

Can you show any real example of where can you find identical frames in video? Most of the time even if they look the same, they are different mathematically and cant be replaced by selecting one of them or their average since this is lossless format.
edit: something like 2-3 100% black frames after my just created CG clip ends and before credits start can be it, but i doubt you'll actually save something with this...

[zerowalker]

There is many times where this can be found.
One is Games, let´s say.

You play 60 fps, it records 30 fps, all i fine.
Then you enter some are, and it goes down to 20 fps, then it will have to repeat some frames to make it 30 fps.

Or let´s say you go to a desk and pick up some paper with text, then it will probably be a still frame with the text on the paper.

Get what i mean:)?

Not really a big realistic use, but it´s very useful for not wasting space on digital recordings(mostly games).

For archival stuff, like analogue, it won´t matter at all, as no screen is identical, as long as it doesn´t lag or something.

[Keiyakusha]

You play 60 fps, it records 30 fps, all i fine.
Then you enter some are, and it goes down to 20 fps, then it will have to repeat some frames to make it 30 fps.

Or let´s say you go to a desk and pick up some paper with text, then it will probably be a still frame with the text on the paper.
I'm not convinced. This is may be true with games but then this indicates some problem. In this case you usually want to go for lower graphic settings or upgrading hardware, not compressing badly recorded footage... As for the desk its not true. You probably will be shooting this with camera, means frames wont be the same. They will have different noise pattern and if at least 1 pixel between 2 frames is different, one of them can't be replaced by "duplicate" mark.
Also not sure how UTvideo works, but if not every frame is keyframe I wonder how it will handle absence of the actual frame in the middle of the group of picture. If group will be split, probably it will only hurt compression.
And if compression algorithm is advanced enough, it almost doesn't uses extra bits for compressing almost the same frames. This is what should be improved as much as possible.

[zerowalker]

Don´t get what you mean with lower settings and compressing badly recorded footage.
With recording, i am talking about Dxtory for example.
And yes it´s true that there often is movement, or if it´s some grain effect.

But in some games, it will be a completely still frame, for example a Menu or when you press start etc.
It´s now very common maybe, but it can be very useful, think about 2D games, there it´s very useful, as there isn´t really stuff moving all the time (especially old games).

Now i understand that this isn´t something to strive for as it´s not a miracle to save some frames here and there.
But i was thinking, if the UT system allows it, i can´t see that it´s hard to add (May be wrong).

And if it can be added with ease, then i don´t see any problems. It will be the best of 2 worlds if you get what i mean.

And of course if it is as you say with the keyframes, and there is really no use, then it´s not worth anything.

But i have a Digital movie i made from a gameplay which has some identical frames (it´s from N64, so it´s 20 fps at 30 fps pretty much all the time in reality, if i didn´t speed upp or add stuff).

And there it´s quite a difference between Lagarith with Null Frames and UT, i think there is about 30% size difference.
And in the end of the video there is like 3 frames which are still for 1-2 sec or more, and there the null feature is really showing it´s usefulness.

Okay it was much more:

Lagarith: 961 MB
UT Video Codec RGB: 2.01GB

I compared them in Avisynth and they are identical, so no color conversion was made.

[Keiyakusha]

Don´t get what you mean with lower settings and compressing badly recorded footage.
I probably got wrong the reason why fps drops. I just remembered something like skyrim where you have 60 fps in one location then you go into some house or out of it and woohoo, its 24 fps now!


Lagarith: 961 MB
UT Video Codec RGB: 2.01GB


I'm sure most of that difference is because difference in algorythm, disregarding duplicates. UTvideo doesnt aims to be best compression-wise, decoding speed is important too. That's why lagarith is not so popular now. Even h264 lossless looks more attractive. I havent tried lagarith for a long time but it was really slow compared to haffman's implementations, they was even saying this on front page. I dont doubth that artificially generated footage indeed may have identical frames. I don't mind anything that may improve compression without affecting speed. I just think this particular feature not worth even slightest effort.

Edit: lets better make Umezawa-san add 10bit support. There is no good solutions right now. x264 vfw can be potential solution, but developers (or should i say maintainers?) are not motivated enough. Yes I know that x264 vfw is bad and everyone hate it but for example... i really want to see some working alternative to all those commercial plugins for 1K$ each.

[zerowalker]

Oh, well then you did get a bit wrong impression there. But that is also a case.

Hmm maybe, but when i use it for normal stuff, they are pretty much the same, let´s say Lagarith is 100gb, then UT may be like 110gb or something like that.
And yeah i know many say it´s slow, i myself hasn´t really run into any problems, even on 1920x1200, even less if i use LAV Filter for decoding.
But i guess the normal cpu doesn´t handle the stress to well, as i got 4ghz i5 and it uses about 100% of 2 cores when it´s on high resolutions, and i think it´s only optimized for 2 cores.

[qyot27]

The most recent versions of Lagarith are a lot more speed optimized, although I don't have any definitive numbers. It's better than it was years ago, though. It was enough for me to actually notice it on a computer that's 11 years old, so if it had to do with assembly it was in the SSE or below realm. Most (all?) of Ut's optimizations are in SSE2, as I understand it, and that's why on newer computers it can run circles around HuffYUV.

Maximizing the impact of the null frame filters for normal footage usually means pulling some AviSynth trickery beforehand, as it can level the sources enough to make it worthwhile (for instance, using Dupped (http://forum.doom9.org/showthread.php?t=134930) on it, especially if a smoother's been run on it first). It also probably has more of an effect on animation.

Feasibly, one could test the impact of null frames on a source by encoding it in Lagarith+nulls first, and then transcoding to Ut whilst preserving the nulls (so that only the real frames are being compared). And then compare the null-containing Ut file to a regular Ut encode of the same source. VirtualDub has a 'Preserve empty frames' option that might let you do a transcode from Lagarith to Ut while keeping nulls intact. If not, it can be forced to work by using avi2tc to produce a de-nulled file + timecodes, from which you can convert the de-nulled file to Ut. Nulls barely take up any space, but if you wanted, you could mux the timecodes back in so that it at least plays back at the right speeds. I use the above method for when I want to produce ffvhuff-encoded files with nulls.

[zerowalker]

It didn´t work, it time went with no progression by using that method on UT.

[qyot27]

Since I actually had need to use the null frame tactic again, I actually tested it like I said above.

Source video consists of essentially a slideshow with some fade-in/out transition effects interspersed throughout.
In other words, lots of static frames that could be dropped as nulls to show the difference it can make.

11095 total frames, 29.97fps, 06:10.203 in length

Script:
FFmpegSource2("video.flv",fpsnum=30000,fpsden=1001)
deen("w3d",3,6,8,8,8)
Dupped()

Save as Lagarith with nulls:
VirtualDub 1.9.11, Fast Recompress, Lagarith 1.3.27, Enable Null Frames, single-threaded*, RGB (Default) preserving YV12 colorspace

Convert Lagarith .avi with nulls to Ut Video with nulls by telling VDub to preserve empty frames
VirtualDub 1.9.11, Fast Recompress, Preserve empty frames, Ut Video 11.1.1 YV12, 1 thread*, Predict median

Compare by converting the same Lagarith file to Ut Video, without preserving empty frames
VirtualDub 1.9.11, Fast Recompress, Ut Video 11.1.1 YV12, 1 thread*, Predict median

*only one thread because I was testing this on an old Celeron Coppermine

I didn't test Predict left since it would only inflate the filesizes and if the goal here is to optimize space savings then it doesn't make much sense.

The result:
Lagarith 1.3.27 (with nulls) 251 MB avi2tc detected 3490 nulls out of 11095 total frames
Ut Video 11.1.1 (with nulls) 345 MB avi2tc detected 3490 nulls out of 11095 total frames,
proving that VirtualDub correctly preserved them
Ut Video 11.1.1 (without nulls) 499 MB avi2tc detected 0 nulls, proving that VirtualDub did
not preserve them when 'Preserve empty frames' was unchecked

[Gser]

YCbCr (BT.601) 4:2:0 8bit limitedSo basically it doesn't support BT.709?

[Keiyakusha]

So basically it doesn't support BT.709?
Its not something that have to be "supported"

[kolak]

So basically it doesn't support BT.709?

It won't when you feed RGB and request to encode as YUV- it will be always 601 (as far as I understand it).
When you encode YUV signal than it works as there is no conversion needed. I stay all the time with YUV and had no issues at all for HD sources.

Overall it could/should be improved- anything above PAL resolution should be rather converted with 709 matrix.

[Zarxrax]

I also found lagarith's null frames very useful for a few use cases that I had several years back. Basically creating graphic overlays for tv shows. The null frames were a huge help in compressing something like that, both in regards to filesize and in encoding/decoding speed.
However, utvideo's design philosophy seems very focused. Every frame is a keyframe. Adding new frametypes into the mix might be something that the developer doesn't want to do, and I certainly wouldn't fault him for that.

[qyot27]

I'd still consider it somewhat external, since it can be done without any input from the codec at all. The program (VirtualDub, ffmpeg, etc.) could be the one doing the comparison and determining whether a frame should be stored as a null or with the designated codec. Lagarith's approach internalizes the check, but there's nothing saying it has to.

[JEEB]

There is now (http://git.libav.org/?p=libav.git;a=commit;h=1ab5a780424ae8755858e153def1173a50a44e4c) Ut Video encoding support in libav, and ffmpeg has merged (http://git.videolan.org/?p=ffmpeg.git;a=commit;h=1ab5a780424ae8755858e153def1173a50a44e4c) it as well.

Multithreading isn't implemented yet, but all available prediction modes (none, left, median) are available, and all supported colorspaces are implemented. The interlaced encoding mode isn't supported at the moment.

[kolak]

Is it possible to add 10bit+ colour spaces?

[easy2Bcheesy]

Love UT Codec but am perplexed by encoding at BT.601 - why can't BT.709 be supported for acknowledged HD resolutions?

[keepert]

Preview thumbnail does not work in win8 x64. Looking in Logs is show that error was happened in dllhost.exe in module utv_core.dll. Looks like bug appeared only in x64 version. In 32 version is okay.

[Jynks]

can not read in adobe premier.. any way to get that to work.. earlier in the thread people said to install the 64 and 32 bit version.. but now there is only 1 installer for wndows.

Any ideas? On howto get prem to read the files?

[kypec]

earlier in the thread people said to install the 64 and 32 bit version.. but now there is only 1 installer for wndows.

Yes, there's only one Windows installer but it should install both versions (x86 & x64) on 64-bit OS. ;)

[foxyshadis]

can not read in adobe premier.. any way to get that to work.. earlier in the thread people said to install the 64 and 32 bit version.. but now there is only 1 installer for wndows.

Any ideas? On howto get prem to read the files?

Which version of Premiere? I might be able to help, I have CS6 at work and an old one at home, don't remember exactly which offhand.

[yup]

New version available!
For windows http://umezawa.dyndns.info/archive/utvideo/utvideo-12.1.0-win.exe
Some speed up using AVX instruction.
yup.

[kolak]

On Mac decoder always outputs (A)RGB, but UTvideo always assumes 601 color spaces, so does it mean all HD (except originally RGB) files will always look wrong?

[RTW47]

Current version 13.0.1 readme (http://umezawa.dyndns.info/archive/utvideo/utvideo-13.0.1-readme.en.html) (English) / installer for Windows (http://umezawa.dyndns.info/archive/utvideo/utvideo-13.0.1-win.exe) / zip for MacOSX (http://umezawa.dyndns.info/archive/utvideo/utvideo-13.0.1-macosx.zip) / source (http://umezawa.dyndns.info/archive/utvideo/utvideo-13.0.1-src.zip)

Others:

Fix packaging


Version 13.0.0

New features: Add codecs that use BT.709-based colorspace conversion coefficient. (FourCC: ULH2, ULH0)Others: Changed codec names.
Changed recommended CPU: Nehalem or later.

[kolak]

Great- just 10 or 16 bit mode and we are done :)

[SubOne]

Great- just 10 or 16 bit mode and we are done :)

Exactly! 10-bit would be a good start.

[kolak]

Or forget 10bit and do 16bit (probably even easier). Anything which is not 16bit fill to 16bits with 0- this will compress nicely, so won't add much of bitrate (at least in my theory:) )

[filler56789]

Version 13.1.0

Performance Improvements

ULRG,ULRA,ULY2,ULH2: Speed up encoding from native colorspace if SSSE3 instructions are available.

ULRG,ULRA,ULY2,ULH2: Speed up decoding video that were encoded with "Optimize for compression ratio" option to native colorspace if SSSE3 instructions are available.

ULY2,ULH2: Speed up encoding video from RGB formats if SSSE3 instructions are available.

ULY2,ULH2: Speed up decoding video to RGB formats if SSE4.1 instructions are available.

http://www.videohelp.com/tools/Ut-Video-Codec-Suite

[cjplay]

This codec is great, but the HDYC support is what's getting me to use it. However, I noticed that the FFMPEG library still only shows the one 422 codec. Is there any chance of updating FFMPEG with the 709 version? Maybe "utvideo422709" for the codec or something like that?

Is there a library I can compile to use the 709 codec with FFMPEG?

Thanks for all your efforts. This codec is very welcome here.

Chris.