Ut Video Codec Suite - a new lossless video codec for Windows! - Page 14

_gl · 2nd May 2012, 09:45

Quote:

Originally Posted by SubOne

Gamma shift is a "feature" of Quicktime/MOV. As with everything Apple, there are no bugs, only features...

Well, useless then. But the extra colour shift seems to be Premiere using the wrong matrix - or is that also a known Quicktime issue?

Quote:

Plus DNx is lossy.

Sure, but studio 4:4:4 is sometimes more important to me.

Quote:

We really need a 10-bit lossless codec though, yes.

Ideally yeah. In Premiere you at least have the option of v210 (uncompressed 10bit 4:2:2), but no 4:4:4 YUV options at all.

qyot27 · 2nd May 2012, 20:36

*cough* FFV1 already supports it *cough* (it's had YUV420P10 and YUV422P10 support since May 2011)

If the underlying video is 4:2:0, then 4:4:4 rendering isn't lossless to start with. And unless Premiere can handle 4:4:4 exporting internally (and especially at higher bit depths), it'll always be doing some sort of lossy conversion before the final output render. There's a lot more involved in the situation than just making sure the export or import codecs support 10-bit 4:4:4.

_gl · 2nd May 2012, 20:58

Quote:

Originally Posted by qyot27

If the underlying video is 4:2:0, then 4:4:4 rendering isn't lossless to start with.

Combinations of processing (grading, sharpening etc) and/or overlaying graphics (logos, gradients etc) can create 4:4:4 detail from 4:2:0 etc. source footage.

Quote:

And unless Premiere can handle 4:4:4 exporting internally

I'm pretty sure it can, it didn't loose chroma when I tested rendering to DNxHD 4:4:4, and Premiere's internal YUV pipe is actually 4:4:4. Of course it has to be tested for AVI specifically, but there's a good chance it's supported.

SubOne · 6th May 2012, 09:51

The underlying acquisition footage is RGB 4:4:4, hence the requirement for a 4:4:4 lossless codec. Premiere can definitely export 4:4:4, as evidenced from testing each channel from likes of Uncompressed, DPX or DNx 444.

Mr_Khyron · 13th May 2012, 23:33

Version 11.1.0

read me / installer for Windows / zip for MacOSX / source

Performance Improvements

(Mac) Add assembly language version of colorspace conversion routine from RGB to ULY2.
Common: Speed up decoding a little.
Common: Speed up encoding on x64 about 10%

Others

(Mac) encoded size becomes a little smaller by omitting unnecessary media sample flags.

johnsonlam · 28th May 2012, 03:01

Quote:

Originally Posted by Mr_Khyron

Version 11.1.0
(Mac) Add assembly language version of colorspace conversion routine from RGB to ULY2.
Common: Speed up decoding a little.
Common: Speed up encoding on x64 about 10%
Others
(Mac) encoded size becomes a little smaller by omitting unnecessary media sample flags.

I fail to find the Win 64bit binary, can somebody kindly tell me where?

the_weirdo · 28th May 2012, 05:12

Quote:

Originally Posted by johnsonlam

I fail to find the Win 64bit binary, can somebody kindly tell me where?

The installer will now install both 32-bit and 64-bit version of UtVideo on Windows 64-bit.

infinityloop · 29th May 2012, 06:28

Quote:

Originally Posted by the_weirdo

The installer will now install both 32-bit and 64-bit version of UtVideo on Windows 64-bit.

I'd like to add that 32-bit versions are also required under W7-64 bit, because a 32bit editing or player application can not use the 64-bit codec.

johnsonlam · 30th May 2012, 13:03

Quote:

Originally Posted by infinityloop

I'd like to add that 32-bit versions are also required under W7-64 bit, because a 32bit editing or player application can not use the 64-bit codec.

Thanks.
If Umezawa can write down (32 and 64bit) should avoid confusion.

kypec · 1st June 2012, 13:35

Quote:

Originally Posted by johnsonlam

Thanks.
If Umezawa can write down (32 and 64bit) should avoid confusion.

He did write about it and you can always look it up in the changelog.

zerowalker · 24th June 2012, 08:12

Can Null Frames be added?
As that is one of the Main reasons i love Lagarith, as it´s just a waste to recompress identical frames.

Or maybe it´s a reason it can´t be used, something in the encoder or how it works?

Keiyakusha · 24th June 2012, 16:27

Can you show any real example of where can you find identical frames in video? Most of the time even if they look the same, they are different mathematically and cant be replaced by selecting one of them or their average since this is lossless format.
edit: something like 2-3 100% black frames after my just created CG clip ends and before credits start can be it, but i doubt you'll actually save something with this...

zerowalker · 25th June 2012, 06:17

There is many times where this can be found.
One is Games, let´s say.

You play 60 fps, it records 30 fps, all i fine.
Then you enter some are, and it goes down to 20 fps, then it will have to repeat some frames to make it 30 fps.

Or let´s say you go to a desk and pick up some paper with text, then it will probably be a still frame with the text on the paper.

Get what i mean

?

Not really a big realistic use, but it´s very useful for not wasting space on digital recordings(mostly games).

For archival stuff, like analogue, it won´t matter at all, as no screen is identical, as long as it doesn´t lag or something.

Keiyakusha · 25th June 2012, 06:27

Quote:

Originally Posted by zerowalker

You play 60 fps, it records 30 fps, all i fine.
Then you enter some are, and it goes down to 20 fps, then it will have to repeat some frames to make it 30 fps.

Or let´s say you go to a desk and pick up some paper with text, then it will probably be a still frame with the text on the paper.

I'm not convinced. This is may be true with games but then this indicates some problem. In this case you usually want to go for lower graphic settings or upgrading hardware, not compressing badly recorded footage... As for the desk its not true. You probably will be shooting this with camera, means frames wont be the same. They will have different noise pattern and if at least 1 pixel between 2 frames is different, one of them can't be replaced by "duplicate" mark.
Also not sure how UTvideo works, but if not every frame is keyframe I wonder how it will handle absence of the actual frame in the middle of the group of picture. If group will be split, probably it will only hurt compression.
And if compression algorithm is advanced enough, it almost doesn't uses extra bits for compressing almost the same frames. This is what should be improved as much as possible.

zerowalker · 25th June 2012, 08:09

Don´t get what you mean with lower settings and compressing badly recorded footage.
With recording, i am talking about Dxtory for example.
And yes it´s true that there often is movement, or if it´s some grain effect.

But in some games, it will be a completely still frame, for example a Menu or when you press start etc.
It´s now very common maybe, but it can be very useful, think about 2D games, there it´s very useful, as there isn´t really stuff moving all the time (especially old games).

Now i understand that this isn´t something to strive for as it´s not a miracle to save some frames here and there.
But i was thinking, if the UT system allows it, i can´t see that it´s hard to add (May be wrong).

And if it can be added with ease, then i don´t see any problems. It will be the best of 2 worlds if you get what i mean.

And of course if it is as you say with the keyframes, and there is really no use, then it´s not worth anything.

But i have a Digital movie i made from a gameplay which has some identical frames (it´s from N64, so it´s 20 fps at 30 fps pretty much all the time in reality, if i didn´t speed upp or add stuff).

And there it´s quite a difference between Lagarith with Null Frames and UT, i think there is about 30% size difference.
And in the end of the video there is like 3 frames which are still for 1-2 sec or more, and there the null feature is really showing it´s usefulness.

Okay it was much more:

Lagarith: 961 MB
UT Video Codec RGB: 2.01GB

I compared them in Avisynth and they are identical, so no color conversion was made.

Keiyakusha · 25th June 2012, 08:35

Quote:

Originally Posted by zerowalker

Don´t get what you mean with lower settings and compressing badly recorded footage.

I probably got wrong the reason why fps drops. I just remembered something like skyrim where you have 60 fps in one location then you go into some house or out of it and woohoo, its 24 fps now!

Quote:

Originally Posted by zerowalker

Lagarith: 961 MB
UT Video Codec RGB: 2.01GB

I'm sure most of that difference is because difference in algorythm, disregarding duplicates. UTvideo doesnt aims to be best compression-wise, decoding speed is important too. That's why lagarith is not so popular now. Even h264 lossless looks more attractive. I havent tried lagarith for a long time but it was really slow compared to haffman's implementations, they was even saying this on front page. I dont doubth that artificially generated footage indeed may have identical frames. I don't mind anything that may improve compression without affecting speed. I just think this particular feature not worth even slightest effort.

Edit: lets better make Umezawa-san add 10bit support. There is no good solutions right now. x264 vfw can be potential solution, but developers (or should i say maintainers?) are not motivated enough. Yes I know that x264 vfw is bad and everyone hate it but for example... i really want to see some working alternative to all those commercial plugins for 1K$ each.

zerowalker · 25th June 2012, 10:07

Oh, well then you did get a bit wrong impression there. But that is also a case.

Hmm maybe, but when i use it for normal stuff, they are pretty much the same, let´s say Lagarith is 100gb, then UT may be like 110gb or something like that.
And yeah i know many say it´s slow, i myself hasn´t really run into any problems, even on 1920x1200, even less if i use LAV Filter for decoding.
But i guess the normal cpu doesn´t handle the stress to well, as i got 4ghz i5 and it uses about 100% of 2 cores when it´s on high resolutions, and i think it´s only optimized for 2 cores.

qyot27 · 25th June 2012, 19:27

The most recent versions of Lagarith are a lot more speed optimized, although I don't have any definitive numbers. It's better than it was years ago, though. It was enough for me to actually notice it on a computer that's 11 years old, so if it had to do with assembly it was in the SSE or below realm. Most (all?) of Ut's optimizations are in SSE2, as I understand it, and that's why on newer computers it can run circles around HuffYUV.

Maximizing the impact of the null frame filters for normal footage usually means pulling some AviSynth trickery beforehand, as it can level the sources enough to make it worthwhile (for instance, using Dupped on it, especially if a smoother's been run on it first). It also probably has more of an effect on animation.

Feasibly, one could test the impact of null frames on a source by encoding it in Lagarith+nulls first, and then transcoding to Ut whilst preserving the nulls (so that only the real frames are being compared). And then compare the null-containing Ut file to a regular Ut encode of the same source. VirtualDub has a 'Preserve empty frames' option that might let you do a transcode from Lagarith to Ut while keeping nulls intact. If not, it can be forced to work by using avi2tc to produce a de-nulled file + timecodes, from which you can convert the de-nulled file to Ut. Nulls barely take up any space, but if you wanted, you could mux the timecodes back in so that it at least plays back at the right speeds. I use the above method for when I want to produce ffvhuff-encoded files with nulls.

zerowalker · 26th June 2012, 06:24

It didn´t work, it time went with no progression by using that method on UT.

qyot27 · 13th August 2012, 08:08

Since I actually had need to use the null frame tactic again, I actually tested it like I said above.

Source video consists of essentially a slideshow with some fade-in/out transition effects interspersed throughout.
In other words, lots of static frames that could be dropped as nulls to show the difference it can make.

11095 total frames, 29.97fps, 06:10.203 in length

Script:

Code:

FFmpegSource2("video.flv",fpsnum=30000,fpsden=1001)
deen("w3d",3,6,8,8,8)
Dupped()

Save as Lagarith with nulls:
VirtualDub 1.9.11, Fast Recompress, Lagarith 1.3.27, Enable Null Frames, single-threaded*, RGB (Default) preserving YV12 colorspace

Convert Lagarith .avi with nulls to Ut Video with nulls by telling VDub to preserve empty frames
VirtualDub 1.9.11, Fast Recompress, Preserve empty frames, Ut Video 11.1.1 YV12, 1 thread*, Predict median

Compare by converting the same Lagarith file to Ut Video, without preserving empty frames
VirtualDub 1.9.11, Fast Recompress, Ut Video 11.1.1 YV12, 1 thread*, Predict median

*only one thread because I was testing this on an old Celeron Coppermine

I didn't test Predict left since it would only inflate the filesizes and if the goal here is to optimize space savings then it doesn't make much sense.

The result:

Code:

Lagarith 1.3.27 (with nulls)	251 MB	avi2tc detected 3490 nulls out of 11095 total frames
Ut Video 11.1.1 (with nulls)	345 MB	avi2tc detected 3490 nulls out of 11095 total frames, 
                                          proving that VirtualDub correctly preserved them
Ut Video 11.1.1 (without nulls)	499 MB	avi2tc detected 0 nulls, proving that VirtualDub did 
                                          not preserve them when 'Preserve empty frames' was unchecked

2nd May 2012, 20:36	#262 \| Link
qyot27 ...? Join Date: Nov 2005 Location: Florida Posts: 1,420	cough FFV1 already supports it cough (it's had YUV420P10 and YUV422P10 support since May 2011) If the underlying video is 4:2:0, then 4:4:4 rendering isn't lossless to start with. And unless Premiere can handle 4:4:4 exporting internally (and especially at higher bit depths), it'll always be doing some sort of lossy conversion before the final output render. There's a lot more involved in the situation than just making sure the export or import codecs support 10-bit 4:4:4. Last edited by qyot27; 2nd May 2012 at 20:45.

6th May 2012, 09:51	#264 \| Link
SubOne Registered User Join Date: May 2009 Posts: 32	The underlying acquisition footage is RGB 4:4:4, hence the requirement for a 4:4:4 lossless codec. Premiere can definitely export 4:4:4, as evidenced from testing each channel from likes of Uncompressed, DPX or DNx 444. Last edited by SubOne; 6th May 2012 at 09:53.

24th June 2012, 16:27	#272 \| Link
Keiyakusha 契約者 Join Date: Jun 2008 Posts: 1,576	Can you show any real example of where can you find identical frames in video? Most of the time even if they look the same, they are different mathematically and cant be replaced by selecting one of them or their average since this is lossless format. edit: something like 2-3 100% black frames after my just created CG clip ends and before credits start can be it, but i doubt you'll actually save something with this... Last edited by Keiyakusha; 24th June 2012 at 16:30.

25th June 2012, 06:17	#273 \| Link
zerowalker Registered User Join Date: Jul 2011 Posts: 1,121	There is many times where this can be found. One is Games, let´s say. You play 60 fps, it records 30 fps, all i fine. Then you enter some are, and it goes down to 20 fps, then it will have to repeat some frames to make it 30 fps. Or let´s say you go to a desk and pick up some paper with text, then it will probably be a still frame with the text on the paper. Get what i mean? Not really a big realistic use, but it´s very useful for not wasting space on digital recordings(mostly games). For archival stuff, like analogue, it won´t matter at all, as no screen is identical, as long as it doesn´t lag or something.

25th June 2012, 08:09	#275 \| Link
zerowalker Registered User Join Date: Jul 2011 Posts: 1,121	Don´t get what you mean with lower settings and compressing badly recorded footage. With recording, i am talking about Dxtory for example. And yes it´s true that there often is movement, or if it´s some grain effect. But in some games, it will be a completely still frame, for example a Menu or when you press start etc. It´s now very common maybe, but it can be very useful, think about 2D games, there it´s very useful, as there isn´t really stuff moving all the time (especially old games). Now i understand that this isn´t something to strive for as it´s not a miracle to save some frames here and there. But i was thinking, if the UT system allows it, i can´t see that it´s hard to add (May be wrong). And if it can be added with ease, then i don´t see any problems. It will be the best of 2 worlds if you get what i mean. And of course if it is as you say with the keyframes, and there is really no use, then it´s not worth anything. But i have a Digital movie i made from a gameplay which has some identical frames (it´s from N64, so it´s 20 fps at 30 fps pretty much all the time in reality, if i didn´t speed upp or add stuff). And there it´s quite a difference between Lagarith with Null Frames and UT, i think there is about 30% size difference. And in the end of the video there is like 3 frames which are still for 1-2 sec or more, and there the null feature is really showing it´s usefulness. Okay it was much more: Lagarith: 961 MB UT Video Codec RGB: 2.01GB I compared them in Avisynth and they are identical, so no color conversion was made. Last edited by zerowalker; 25th June 2012 at 08:45.

13th May 2012, 23:33	#265 \| Link
Mr_Khyron Member Join Date: Nov 2002 Posts: 203	Version 11.1.0 read me / installer for Windows / zip for MacOSX / source Performance Improvements (Mac) Add assembly language version of colorspace conversion routine from RGB to ULY2. Common: Speed up decoding a little. Common: Speed up encoding on x64 about 10% Others (Mac) encoded size becomes a little smaller by omitting unnecessary media sample flags.

24th June 2012, 08:12	#271 \| Link
zerowalker Registered User Join Date: Jul 2011 Posts: 1,121	Can Null Frames be added? As that is one of the Main reasons i love Lagarith, as it´s just a waste to recompress identical frames. Or maybe it´s a reason it can´t be used, something in the encoder or how it works?

25th June 2012, 10:07	#277 \| Link
zerowalker Registered User Join Date: Jul 2011 Posts: 1,121	Oh, well then you did get a bit wrong impression there. But that is also a case. Hmm maybe, but when i use it for normal stuff, they are pretty much the same, let´s say Lagarith is 100gb, then UT may be like 110gb or something like that. And yeah i know many say it´s slow, i myself hasn´t really run into any problems, even on 1920x1200, even less if i use LAV Filter for decoding. But i guess the normal cpu doesn´t handle the stress to well, as i got 4ghz i5 and it uses about 100% of 2 cores when it´s on high resolutions, and i think it´s only optimized for 2 cores.

25th June 2012, 19:27	#278 \| Link
qyot27 ...? Join Date: Nov 2005 Location: Florida Posts: 1,420	The most recent versions of Lagarith are a lot more speed optimized, although I don't have any definitive numbers. It's better than it was years ago, though. It was enough for me to actually notice it on a computer that's 11 years old, so if it had to do with assembly it was in the SSE or below realm. Most (all?) of Ut's optimizations are in SSE2, as I understand it, and that's why on newer computers it can run circles around HuffYUV. Maximizing the impact of the null frame filters for normal footage usually means pulling some AviSynth trickery beforehand, as it can level the sources enough to make it worthwhile (for instance, using Dupped on it, especially if a smoother's been run on it first). It also probably has more of an effect on animation. Feasibly, one could test the impact of null frames on a source by encoding it in Lagarith+nulls first, and then transcoding to Ut whilst preserving the nulls (so that only the real frames are being compared). And then compare the null-containing Ut file to a regular Ut encode of the same source. VirtualDub has a 'Preserve empty frames' option that might let you do a transcode from Lagarith to Ut while keeping nulls intact. If not, it can be forced to work by using avi2tc to produce a de-nulled file + timecodes, from which you can convert the de-nulled file to Ut. Nulls barely take up any space, but if you wanted, you could mux the timecodes back in so that it at least plays back at the right speeds. I use the above method for when I want to produce ffvhuff-encoded files with nulls. Last edited by qyot27; 25th June 2012 at 19:32.

26th June 2012, 06:24	#279 \| Link
zerowalker Registered User Join Date: Jul 2011 Posts: 1,121	It didn´t work, it time went with no progression by using that method on UT.

13th August 2012, 08:08	#280 \| Link
qyot27 ...? Join Date: Nov 2005 Location: Florida Posts: 1,420	Since I actually had need to use the null frame tactic again, I actually tested it like I said above. Source video consists of essentially a slideshow with some fade-in/out transition effects interspersed throughout. In other words, lots of static frames that could be dropped as nulls to show the difference it can make. 11095 total frames, 29.97fps, 06:10.203 in length Script: Code: FFmpegSource2("video.flv",fpsnum=30000,fpsden=1001) deen("w3d",3,6,8,8,8) Dupped() Save as Lagarith with nulls: VirtualDub 1.9.11, Fast Recompress, Lagarith 1.3.27, Enable Null Frames, single-threaded, RGB (Default) preserving YV12 colorspace Convert Lagarith .avi with nulls to Ut Video with nulls by telling VDub to preserve empty frames VirtualDub 1.9.11, Fast Recompress, Preserve empty frames, Ut Video 11.1.1 YV12, 1 thread, Predict median Compare by converting the same Lagarith file to Ut Video, without preserving empty frames VirtualDub 1.9.11, Fast Recompress, Ut Video 11.1.1 YV12, 1 thread, Predict median only one thread because I was testing this on an old Celeron Coppermine I didn't test Predict left since it would only inflate the filesizes and if the goal here is to optimize space savings then it doesn't make much sense. The result: Code: Lagarith 1.3.27 (with nulls) 251 MB avi2tc detected 3490 nulls out of 11095 total frames Ut Video 11.1.1 (with nulls) 345 MB avi2tc detected 3490 nulls out of 11095 total frames, proving that VirtualDub correctly preserved them Ut Video 11.1.1 (without nulls) 499 MB avi2tc detected 0 nulls, proving that VirtualDub did not preserve them when 'Preserve empty frames' was unchecked