Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > New and alternative video codecs

Reply
 
Thread Tools Search this Thread Display Modes
Old 9th November 2011, 04:25   #121  |  Link
Pat357
Registered User
 
Join Date: Jun 2006
Posts: 452
Quote:
Originally Posted by Chikuzen View Post
ffdshow and LAV Video Decoder
LAV Video works indeed, but very very slooow on playback, even on my i7 990X@4Ghz. (it looks like the decoding is only single treaded)

I can't make it work in FFSshow ... how is the codec called in FFDshow ? I see nothing that looks like Lagarith decoder in the decoder list.

If I write a singe line AVS script like :
AVISource("filename") and open this, the playback is smooth and much faster (the original vfw is than used to decode, multithreaded)

Opening the AVI in FFmpeg (FFPlay) gives me green colored decoded frames (!!) ... both with the AVI saved as YV12 and RGB32.
Is the FFmpeg LAC decoder faulty are am I doing something wrong ?
Pat357 is offline   Reply With Quote
Old 9th November 2011, 10:08   #122  |  Link
Chikuzen
typo lover
 
Chikuzen's Avatar
 
Join Date: May 2009
Posts: 595
Quote:
Originally Posted by Pat357 View Post
I can't make it work in FFSshow ... how is the codec called in FFDshow ? I see nothing that looks like Lagarith decoder in the decoder list.
oops, sorry. I mistook.
ffdshow doesn't support lagarith yet.
Quote:
Opening the AVI in FFmpeg (FFPlay) gives me green colored decoded frames (!!) ... both with the AVI saved as YV12 and RGB32.
Is the FFmpeg LAC decoder faulty are am I doing something wrong ?
If it becomes so, ffmpeg may have some bugs.
I found two bugs of libav's UtVideo decoder about ten days ago.
Since I sent small samples to the developer at #libav-devel(IRC channel) , they were fixed without passing half a day.
http://lists.libav.org/pipermail/lib...er/013542.html
http://lists.libav.org/pipermail/lib...er/013607.html
If libavcodec's Lagarith decoder is important for you, you should do a bug report.
__________________
my repositories

Last edited by Chikuzen; 9th November 2011 at 14:17.
Chikuzen is offline   Reply With Quote
Old 9th November 2011, 11:24   #123  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,419
The Lagarith decoder in libavcodec supports YV12 only. The green frame issue depends on what the underlying stream is like - if these were solid-colored frames, then it would seem that you just happen to be unlucky enough to be using an FFmpeg build from the gap time between when the decoder was committed and the solids-rendered-as-green issue was fixed, which was a week later. If it's not a problem with solids, then my only other guess is, maybe null frames? I really don't know how the decoder treats those. And if it's not solids or nulls, are you sure you're using a build new enough to support Lagarith? It was only committed to SVN this past January (although the branch had been around for a couple years before that).

The slowness of the fflagarith decoder could also possibly be due to not being assembly-optimized, not just multithreading stuff.
qyot27 is offline   Reply With Quote
Old 9th November 2011, 16:13   #124  |  Link
Pat357
Registered User
 
Join Date: Jun 2006
Posts: 452
Quote:
Originally Posted by qyot27 View Post
The Lagarith decoder in libavcodec supports YV12 only. The green frame issue depends on what the underlying stream is like - if these were solid-colored frames, then it would seem that you just happen to be unlucky enough to be using an FFmpeg build from the gap time between when the decoder was committed and the solids-rendered-as-green issue was fixed, which was a week later.
I always update to the latest FFmpeg from Zeranoe : http://ffmpeg.zeranoe.com/builds/
I disabled null frames when encoding in Lagarith, so that could also not be the reason for the green decoded movie.
The movie was a about 2m 1080p HD track to test lagarith.
Quote:
If it's not a problem with solids, then my only other guess is, maybe null frames? I really don't know how the decoder treats those. And if it's not solids or nulls, are you sure you're using a build new enough to support Lagarith? It was only committed to SVN this past January (although the branch had been around for a couple years before that).
My current build is only a couple of days old !!
I see Zeranoe has updated his builds : latest is dd. 2011-11-06
I will download and try this build.
Quote:
The slowness of the fflagarith decoder could also possibly be due to not being assembly-optimized, not just multi-threading stuff.
When I use the original VfW lagarith to decode (open video by opening AviSynth 2.6 MT), I get smooth playback and see that several cores are used to decode.
Opening the AVI directly with FFplay gives me very choppy playback <5 fps with only 1 of my 6 available cores (i7 990x) used and at 100%. The other 5 cores are at < 1 %.
My command line is: "FFplay -threads 8 -i %1"

I found another build that doesn't show green video :
The green color could be a problem with the Zeranoe builds, I want to try other very recent builds.
The oldest build I have is this one :
Code:
ffmpeg\bin>ffplay.exe -threads 8 -loop 0 "K:\film\vdubtest\lagarith_UV12.avi"
FFplay version SVN-r26397, Copyright (c) 2003-2011 the FFmpeg developers
  built on Jan 17 2011 04:07:25 with gcc 4.4.2
  configuration: --enable-gpl --enable-version3 --enable-libgsm --enable-libvorbis --enable-libtheora --enable-libspeex --enable
-libmp3lame --enable-libopenjpeg --enable-libschroedinger --enable-libopencore_amrwb --enable-libopencore_amrnb --enable-libvpx
--disable-decoder=libvpx --arch=x86 --enable-runtime-cpudetect --enable-libxvid --enable-libx264 --enable-librtmp --extra-libs='
-lrtmp -lpolarssl -lws2_32 -lwinmm' --target-os=mingw32 --enable-avisynth --enable-w32threads --cross-prefix=i686-mingw32- --cc=
'ccache i686-mingw32-gcc' --enable-memalign-hack
  libavutil     50.36. 0 / 50.36. 0
  libavcore      0.16. 1 /  0.16. 1
  libavcodec    52.108. 0 / 52.108. 0
  libavformat   52.93. 0 / 52.93. 0
  libavdevice   52. 2. 3 / 52. 2. 3
  libavfilter    1.74. 0 /  1.74. 0
  libswscale     0.12. 0 /  0.12. 0
Input #0, avi, from 'K:\film\vdubtest\lagarith_UV12.avi':
  Duration: 00:00:30.03, start: 0.000000, bitrate: 169237 kb/s
    Stream #0.0: Video: lagarith, yuv420p, 1920x1080, 29.97 tbr, 29.97 tbn, 29.97 tbc
  16.54 A-V:  0.000 s:22.8 aq=    0KB vq= 3529KB sq=    0B f=0/0   /0
The quality is OK : no green decoded frames anymore!! The problem with this one is that it either hangs on playback or is *extremely* slow : I only see the first couple of frames (maybe 50 or so) and then playback stops.

The Zeranoe build on the same file :
Code:
K:\programs\ffmpeg-git-8475ec1-win32-static\bin>ffplay -threads 8 -i "K:\film\vdubtest\lagarith_UV12.avi"
ffplay version N-34318-g8475ec1, Copyright (c) 2003-2011 the FFmpeg developers
  built on Oct 31 2011 17:50:05 with gcc 4.6.1
  configuration: --enable-gpl --enable-version3 --enable-runtime-cpudetect --enable-avisynth --enable-bzlib --enable-frei0r --en
able-libopencore-amrnb --enable-libopencore-amrwb --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-libopenjpeg
--enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvo-aacenc --enable-libvo-amrwbenc --e
nable-libvorbis --enable-libvpx --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib
  libavutil    51. 22. 0 / 51. 22. 0
  libavcodec   53. 26. 0 / 53. 26. 0
  libavformat  53. 18. 0 / 53. 18. 0
  libavdevice  53.  4. 0 / 53.  4. 0
  libavfilter   2. 45. 3 /  2. 45. 3
  libswscale    2.  1. 0 /  2.  1. 0
  libpostproc  51.  2. 0 / 51.  2. 0
[avi @ 04422FC0] parser not found for codec lagarith, packets or times may be invalid.
Input #0, avi, from 'K:\film\vdubtest\\lagarith_UV12.avi':
  Duration: 00:00:30.03, start: 0.000000, bitrate: 169546 kb/s
    Stream #0:0: Video: lagarith (LAGS / 0x5347414C), yuv420p, 1920x1080, 29.97 tbr, 29.97 tbn, 29.97 tbc
   6.33 A-V:  0.000 fd=   0 aq=    0KB vq= 4608KB sq=    0B f=0/0   0/0
This build decodes much faster, but the green color is there.
It looks like the Y channel is OK but the chroma is completely messed up.
Also notice the "parser not found for codec lagarith, packets or times may be invalid" message".
I have to try newer build, but not from Zeranoe.
Pat357 is offline   Reply With Quote
Old 10th November 2011, 03:01   #125  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,419
Quote:
Originally Posted by Pat357 View Post
When I use the original VfW lagarith to decode (open video by opening AviSynth 2.6 MT), I get smooth playback and see that several cores are used to decode.
Opening the AVI directly with FFplay gives me very choppy playback <5 fps with only 1 of my 6 available cores (i7 990x) used and at 100%. The other 5 cores are at < 1 %.
My command line is: "FFplay -threads 8 -i %1"
The reference Lagarith codec has both multithreading and assembly optimizations. Lacking multithreading capability will lead to a drop in speed, but if it's an extremely large drop in speed, it's more than likely due to not having the relevant assembly code to speed things up.

1) Assembly+Multithreading=Fast
2) Assembly+No Multithreading=Fast, but not as fast as 1) if the code in 1) is optimized for multiple threads
3) No Assembly+Multithreading=Slow, even if it can use threads
4) No Assembly+No Multithreading=Slowest option, but likely only by a small degree


This is a comparison using avsmeter to compare the performance of the VfW Lagarith decoder (1.3.26) to the one in libavcodec (via FFMS2):
Code:
$ avsmeter lagtest-avisource.avs

AVSMeter v1.12 (Aug  2 2011) by Groucho2004
AviSynth 2.60, build:May 25 2011 [19:58:41]

Number of frames:            2013
Length (h:m:s.ms):    0:02:14.200
Frame width:                  512
Frame height:                 288
Framerate:                 15.000 (1000000000 / 66666501)
Progressive:                  Yes
Colorspace:                  YV12

Hit ESC to exit...
Frame 2013/2013, fps (min/max/avg): 52.44 | 178.80 | 71.07

$ avsmeter lagtest-ffms2.avs

AVSMeter v1.12 (Aug  2 2011) by Groucho2004
AviSynth 2.60, build:May 25 2011 [19:58:41]

Number of frames:            2013
Length (h:m:s.ms):    0:02:14.200
Frame width:                  512
Frame height:                 288
Framerate:                 15.000 (15015 / 1001)
Progressive:                  Yes
Colorspace:                  YV12

Hit ESC to exit...
Frame 2013/2013, fps (min/max/avg): 29.98 | 126.88 | 44.40
As you can see, the VfW decoder is, on average, a little less than twice as fast as the libavcodec decoder is. My computer does not have multithreading ability at all, and because it's using a Celeron processor based on the Pentium III Coppermine, it's not got the luxury of newer instruction sets like SSE2 or higher. That speed difference in my example is therefore probably entirely due to the libavcodec decoder simply not being as optimized, not from the lack of multithreading.

Note, of course, that the sample I was using was only 512x288. I don't expect the speed to scale linearly up to 1080p, nor to scale linearly across processors, but it still illustrates the performance gap.



The other possibility, is that if you're using 64-bit builds, they're getting miscompiled, which would explain why 32-bit builds don't show the problem. I just downloaded the latest win32-static build from Zeranoe and the sample from the above test played back fine (albeit with the same quirks I also noticed in FFMS2, which coincided with some warnings I got back about length and output bytes or something).

Last edited by qyot27; 10th November 2011 at 03:17.
qyot27 is offline   Reply With Quote
Old 22nd March 2013, 23:27   #126  |  Link
zambelli
Doom9ing since 2001
 
zambelli's Avatar
 
Join Date: Oct 2001
Location: Seattle, WA, USA
Posts: 2,002
Why no Lagarith encoder in Ffmpeg? Lagarith is released under GPL, so from a licensing perspective it should be OK, right?
zambelli is offline   Reply With Quote
Old 22nd March 2013, 23:37   #127  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
Quote:
Originally Posted by zambelli View Post
Why no Lagarith encoder in Ffmpeg? Lagarith is released under GPL, so from a licensing perspective it should be OK, right?
Might be related to the fact that the original Lagarith code is Windows-only and implemented as a VfW Codec.

Also, if you need a lossless encoder, there already are some alternatives available in FFmpeg, such as HuffYUV, FFV1, x264...
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊
LoRd_MuldeR is offline   Reply With Quote
Old 23rd March 2013, 17:22   #128  |  Link
Leeloo Minaï
Registered User
 
Join Date: Nov 2007
Posts: 50
@LoRd_MuldeR : where is the problem with the fact Lagarith is in VfW format ?
HuffYUV codec was originally released a VfW codec, so "windows only", but also as open source, all like Lagarith, and now HuffYUV is part of FFmpeg...
Leeloo Minaï is offline   Reply With Quote
Old 24th March 2013, 22:23   #129  |  Link
LoRd_MuldeR
Software Developer
 
LoRd_MuldeR's Avatar
 
Join Date: Jun 2005
Location: Last House on Slunk Street
Posts: 13,248
Quote:
Originally Posted by Leeloo Minaï View Post
@LoRd_MuldeR : where is the problem with the fact Lagarith is in VfW format ?
HuffYUV codec was originally released a VfW codec, so "windows only", but also as open source, all like Lagarith, and now HuffYUV is part of FFmpeg...
The "problem" is that you don't have a general purpose cross-platfrom encoder library that you can just take "as is" and plug it into FFmpeg

Instead you have to "extract" (decouple) the actual encoder code from the VfW Codec wrapper. Depending on how "clean" the code is written/designed (haven't looked at in detail) this can be straight-forward or really annoying!

Also, code that compiles in Visual Studio and that never has been tested with other compilers is likely to not compile with GCC, especially if a lot of Win32 API functions/types or MSVC-specifics intrinsic are used. It's not fun to fix that

After all, this all should be very doable, but somebody has to do it! Feel free to port the Lagarith encoder and submit a patch to the FFmpeg and/or Lavc guys...
__________________
Go to https://standforukraine.com/ to find legitimate Ukrainian Charities 🇺🇦✊

Last edited by LoRd_MuldeR; 25th March 2013 at 02:03.
LoRd_MuldeR is offline   Reply With Quote
Old 26th March 2013, 01:59   #130  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,558
Porting might not be too difficult, but it isn't in there because no one ever did it, the author doesn't want to, and it must not be a high priority for volunteers.

Lagarith itself is written with intrinsics, which is always nice and makes porting easy, but also includes a bunch of intel-format custom asm code to do colorspace conversion. It also sets and reads options in a very windows-specific way, via the registry. Since ffdshow is primarily made for MSVC it could be included nearly as-is, the way libdts and faad are, but it wouldn't make it into ffmpeg. I don't think it'd be very hard, colorspace conversion is something better handled in libswscale than lavc anyway. Another serious problem is that some features are based on x86 quirks, like the way it performs floating point operations, that aren't be 100% lossless on non-x86 platforms without reimplementing floating-point logic in software. That's a huge part of the slowdown, and one reason some of the intrinsics can't be used.

(The FFDShow includion would be time-consuming more due to creating the GUI than anything.)

Last edited by foxyshadis; 26th March 2013 at 02:10.
foxyshadis is offline   Reply With Quote
Old 27th March 2013, 00:04   #131  |  Link
tuqueque
Registered User
 
Join Date: Jun 2009
Posts: 26
I'm no coder... But as a user, I LOVE Lagarith to record video tutorials, intermediate lossless content and stuff like that. I'd love to have Lagarith on FFmpeg. It compresses better than almost any other (lossless) codec and I'll use anything but x264/H.264 in my work. I think the best way to support real open source projects, is to avoid stuff based on proprietary/patented stuff (not trying to initiate a debate, I'm just exposing the searons why I use Lagarith).

In any case, we may wait for VP9's lossless mode.
tuqueque is offline   Reply With Quote
Old 27th March 2013, 00:16   #132  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,666
FFV1 and Dirac are both (probably) royalty-free and should compress much better than Lagarith.
Dark Shikari is offline   Reply With Quote
Old 27th March 2013, 22:40   #133  |  Link
zambelli
Doom9ing since 2001
 
zambelli's Avatar
 
Join Date: Oct 2001
Location: Seattle, WA, USA
Posts: 2,002
Lagarith is royalty-free too (at least there's no mention of any licensing on its homepage). "This codec was build using the Huffyuv source as a template, and uses some Huffyuv code, most notably the routine to upsample YUY2 video to RGB and to perform pixel prediction on YUY2 video. Other colorspace conversion routines were taken from AviSynth. Lagarith is released under the GPL."

You're correct regarding compression efficiency, but performance wise Lagarith kicks FFV1's butt on a modern Intel i7 quadcore system. Here's a quick test:

Lagarith v1.3.27 x86
FFV1 (Ffdshow) Rev4499 20130104 x86
Source: 1920x1080 29.97 fps YV12

==========================
Lagarith = 0.601x realtime, 33.11% file size
FFV1 AC Large = 0.306x realtime, 27.95% file size
FFV1 VLC Small = 0.408x realtime, 30.04% file size


As you can see, FFV1 is 15% more efficient than Lagarith in its most complex mode (AC Large), but half the speed. IMHO, that's not worth it.

Another advantage of Lagarith over FFV1 is that it can be installed as a standalone codec in Windows, making it very easy to distribute Lagarith-compressed videos to people who aren't well acquainted with codecs, VfW, DirectShow, etc. Telling somebody to "download and install this one executable" is way easier than teaching somebody how to find the latest stable build of FFdshow, install it, and then configure to decode the right codecs.

Last edited by zambelli; 27th March 2013 at 22:44.
zambelli is offline   Reply With Quote
Old 28th March 2013, 00:39   #134  |  Link
Asmodian
Registered User
 
Join Date: Feb 2002
Location: San Jose, California
Posts: 4,406
Doesn't this speed/size ratio look even better for huffyuv? I remember it is much faster than Lagarith and only somewhat larger? I think it is all about picking the option that is "fast enough" while also being "small enough". Of course "enough" is project dependent.
Asmodian is offline   Reply With Quote
Old 29th March 2013, 20:23   #135  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by zambelli View Post
Another advantage of Lagarith over FFV1 is that it can be installed as a standalone codec in Windows, making it very easy to distribute Lagarith-compressed videos to people who aren't well acquainted with codecs, VfW, DirectShow, etc. Telling somebody to "download and install this one executable" is way easier than teaching somebody how to find the latest stable build of FFdshow, install it, and then configure to decode the right codecs.
And that is exactly why I use Lagarith instead of FFV1. I need to be able to send someone an .avi file and have them easily be able to do whatever with it in their existing Windows tools.

If there was a stand-alone installer for FFV1 that provided 32-bit and 64-bit VfW/DirectShow/MediaFoundation encoders and decoders, I'd be over the moon. If it had QuickTime components for Mac and Windows, I'd never use anything else.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 2nd April 2013, 20:57   #136  |  Link
zerowalker
Registered User
 
Join Date: Jul 2011
Posts: 1,121
Is it possible to optimize the Lagarith code to use AVX instructions and stuff?
The encoder is fast as it is, but it would be awesome if itīs possible to improve it.

But i donīt know much of programming, so canīt say if itīs possible or not.
zerowalker is offline   Reply With Quote
Old 2nd April 2013, 21:02   #137  |  Link
Dark Shikari
x264 developer
 
Dark Shikari's Avatar
 
Join Date: Sep 2005
Posts: 8,666
Almost all the time in something like Lagarith is going to be spend in the arithmetic coder, which is almost necessarily not-SIMDable.
Dark Shikari is offline   Reply With Quote
Old 2nd April 2013, 21:04   #138  |  Link
zerowalker
Registered User
 
Join Date: Jul 2011
Posts: 1,121
So, in other words, instructions doesnīt help Lagarith?

Dark Shikari, you wouldnīt know how to implement YCoCg support in Lagarith?

Last edited by zerowalker; 2nd April 2013 at 21:13.
zerowalker is offline   Reply With Quote
Old 3rd April 2013, 02:36   #139  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,558
Quote:
Originally Posted by Dark Shikari View Post
Almost all the time in something like Lagarith is going to be spend in the arithmetic coder, which is almost necessarily not-SIMDable.
It's actually a reasonably fast range coder, and keeps separate state in both threads; most of the time seems to be spent in prediction, which is already SIMD'd to hell and back. (There is no straight-C fallback code, just SIMD for most operations.)

Quote:
Originally Posted by zerowalker View Post
Dark Shikari, you wouldnīt know how to implement YCoCg support in Lagarith?
It's not knowing how, it's someone willing to do it. Although it might not be too hard to rip the guts out of ConvertToYCgCo and insert it into another conversion routine, it would be incompatible with every other Lagarith install out there right now.

I did do some testing with ConvertToYCgCo, though, and it does seem to help a little bit if you don't mind subsampling, but if color fidelity is important to you, YCgCo24 (or YV24) gains you almost nothing size-wise over RGB. Maybe 3-5%.
foxyshadis is offline   Reply With Quote
Old 3rd April 2013, 03:27   #140  |  Link
zerowalker
Registered User
 
Join Date: Jul 2011
Posts: 1,121
Oh well if itīs just 3-5% less size, then itīs pretty much useless, as itīs not perfect, which RGB is.

Is there a way to make Lagarith convert to YV12 correctly?

ConvertToYV12(matrix="Rec709")

For example, if the original need to use this matrix to be correct.
Lagarith will use Rec601 always it seems.

And i donīt think you can solve it afterwards, though if i am wrong, it would be nice.

EDIT:

Okay it seems to be fixable with ColorMatrix afterwards.
But would be nice for Lagarith to do it for many reasons.

Last edited by zerowalker; 3rd April 2013 at 03:44.
zerowalker is offline   Reply With Quote
Reply

Tags
lagarith

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 18:25.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.