Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 24th September 2023, 16:05   #1  |  Link
N'Cha
Registered User
 
Join Date: Sep 2023
Posts: 28
FFVideoSource vs FFM2 vs LWLibAvVideoSource vs L-Smash?

What would be the best "source" choice between FFVideoSource, FFM2, LWLibAvVideoSource and L-Smash? I don't understand the pros and cons in term of quality, speed, features, compression etc. Thanks for the help
N'Cha is offline   Reply With Quote
Old 24th September 2023, 17:19   #2  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,768
FFVideoSource = FFM2
LWLibAvVideoSource = L-Smash

One is the plugin name the other the function name of the plugin (historical reasons I guess)

Both are more or less the same since they both use ffmpeg.

The main difference is how some container formats are handled within each plugin. Just look the table here: https://forum.doom9.org/showthread.php?t=176231

There is no best
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database || https://github.com/avisynth-repository
ChaosKing is offline   Reply With Quote
Old 24th September 2023, 20:14   #3  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,669
For AVC or HEVC (and MPEG2) sources, there's also DGDecodeNV if you have a suitable NVidia GPU available. It's definitely the best option there is for those, no issues with frame accuracy whatsoever.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 24th September 2023, 20:20   #4  |  Link
N'Cha
Registered User
 
Join Date: Sep 2023
Posts: 28
Thanks for the link i'll check it out
http://avisynth.nl/index.php/LSMASHSource o nthis link the wiki says that LWLibAvVideoSource supports video codecs that LSmash, why is that if they are the same jsut with different names? Also Staxrip you can select the 4 of them independetly, why is that if they are the same?
N'Cha is offline   Reply With Quote
Old 24th September 2023, 20:21   #5  |  Link
N'Cha
Registered User
 
Join Date: Sep 2023
Posts: 28
Quote:
Originally Posted by Boulder View Post
For AVC or HEVC (and MPEG2) sources, there's also DGDecodeNV if you have a suitable NVidia GPU available. It's definitely the best option there is for those, no issues with frame accuracy whatsoever.
I have a RTX3080
N'Cha is offline   Reply With Quote
Old 24th September 2023, 22:02   #6  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,768
Quote:
Originally Posted by Boulder View Post
For AVC or HEVC (and MPEG2) sources, there's also DGDecodeNV if you have a suitable NVidia GPU available. It's definitely the best option there is for those, no issues with frame accuracy whatsoever.
Absolutely, the main reason for me is not the higher decoding speed but the frame accuracy.

I whish it would auto generate the index like ffms2/lsmash does...
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database || https://github.com/avisynth-repository
ChaosKing is offline   Reply With Quote
Old 29th September 2023, 23:27   #7  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,719
Quote:
Originally Posted by N'Cha View Post
the wiki says that LWLibAvVideoSource supports video codecs that LSmash, why is that if they are the same jsut with different names?
They're the same plugin, but different functions.

The difference between LSMASHVideoSource() and LWLibavVideoSource() is that the first will use the index inside the container to create an uncompressed video stream living in RAM inside Avisynth/VapourSynth as a frameserver. The second one, instead, will index the file, regardless of whether the container of the source already has one or not.

Pros and cons?

Well:

1) Not all containers have an index inside, which is why you can only use LSMASHVideoSource() for ISO containers like MP4, MOV etc.

2) If, for whatever reason, the index inside the container has some kind of issue, there will be issues in the stream.


Using LWLibavVideoSource() will require more time than LSMASHVideoSource() 'cause it will have to go through the whole file and create an .lwi index, however this indexer will be able to handle more sources as it will work with any container even those that don't have an index like .ts and, even for those who do have an index like mp4, but that perhaps have issues in it for whatever reason, it will be able to index them correctly anyway as it will literally recreate it.


Should you use one or the other? It depends on you and your use cases.

Why? Well, the choice might seem obvious at first as one might think "I'm always gonna go with LWLibavVideoSource() as it handles pretty much everything and the chances of getting the correct results are higher". This is true, but think about the following scenario:

you're in a company, your server has a 1 Gbit/s ethernet connection and a reputable production house just sent you an Apple ProRes masterfile of a movie worth 1.2 TB in a .mov container. Do you really wanna spend 7h indexing it (i.e 7h to create the .lwi file) before you can start encoding it OR you take your chances with LSMASHVideoSource() and start encoding it straight away?

This is one of the dilemmas plenty of people go through every day and it's just an example.

So, to recap, the choice of which indexer to use is yours, there's no "best".

Last edited by FranceBB; 30th September 2023 at 23:37.
FranceBB is offline   Reply With Quote
Old 30th September 2023, 20:17   #8  |  Link
N'Cha
Registered User
 
Join Date: Sep 2023
Posts: 28
Thanks for the detailed answer It's more clear now
N'Cha is offline   Reply With Quote
Old 1st October 2023, 06:28   #9  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,980
Yeah, indexing sucks. Ain't nobody got time for that

But FranceBB - which production servers are 1 Gbps in 2023?! I've not seen anything slower than 10 Gbps in a long time, and many servers are 25 or more.

Do you really have modern hardware that still only has a 1 Gbps connection to storage?
__________________
These are all my personal statements, not those of my employer :)
Blue_MiSfit is offline   Reply With Quote
Old 1st October 2023, 16:11   #10  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,719
Quote:
Originally Posted by Blue_MiSfit View Post
which production servers are 1 Gbps in 2023?! I've not seen anything slower than 10 Gbps in a long time, and many servers are 25 or more.

Do you really have modern hardware that still only has a 1 Gbps connection to storage?
They do have 10 Gbit/s ports, the problem is that we still don't have many 10 Gbit/s capable switches, so we have nothing to connect them to.
As an example:



This is one of the Avisynth (actually FFAStrans) server which does indeed have a 10 Gbit/s port, but I would need two SFP+ transceivers (to connect both the switch and the port) and 20 meters worth of LC cable. The problem is that we have just a few 10 Gbit/s capable switches with very limited ports as the overwhelming majority is still 1 Gbit/s so... on we go with 1 Gbit/s (which is the only one connected as you can see from the picture)

Currently, on prem in Italy we have 6 servers in a farm:

Avisynth Server 1 - Intel Xeon 20c/40th 64GB RAM 1 Gbit/s
Avisynth Server 2 - Intel Xeon 20c/40th 64GB RAM 1 Gbit/s
Avisynth Server 3 - Intel Xeon 56c/112th 128GB RAM 10 Gbit/s
Avisynth Server 4 - Intel Xeon 56c/112th 128GB RAM 10 Gbit/s
Avisynth Server 5 - Intel Xeon 56c/112th 128GB RAM 10 Gbit/s
Avisynth Server 6 - Intel Xeon 10c/20th 32GB RAM 1 Gbit/s


Unfortunately, though, even those connected to 10 Gbit/s don't really use the 10 Gbit/s connection 'cause they're connected to old storages like Harmonic MediaGRID (which was brought up in 2007 and is currently 16 years old) and never exceeds 2 Gbit/s max, while our Isilon is 1.2 PB with I think 12 connections but there are so many systems reading and writing on it that it's always pegged at 90-99% speed so we basically never exceed 1 Gbit/s anyway unless it's one of the rare occasions in which other systems are not reading/writing on it and I get a sweet sweet 4 Gbit/s etc.

The only storage that currently holds its promises is the new AVID Nexis storage for Sport, TG24 (i.e News) and PCE (Promo Cinema and Entertainment) on which I can actually basically saturate 8 Gbit/s per machine easily and it's working remarkably well.


This is in Italy, of course, in the UK they have their own VPC with a 55 servers farm all connected at 10 Gbit/s on each server. As my former boss (now retired) would say "in Italy we're the poorest out of the whole group".
Luckily there's been a shift over the last few years and I now also own 105 8c/8th 16GB of RAM VMs in the cloud (AWS EC2), however processing stuff in the cloud is crazily expensive, so...
FranceBB is offline   Reply With Quote
Old 6th October 2023, 22:43   #11  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,643
Quote:
Originally Posted by Blue_MiSfit View Post
Yeah, indexing sucks. Ain't nobody got time for that

But FranceBB - which production servers are 1 Gbps in 2023?! I've not seen anything slower than 10 Gbps in a long time, and many servers are 25 or more.

Do you really have modern hardware that still only has a 1 Gbps connection to storage?
Could an index be generated on the server where the data lives, and then be passed on to be used by the client trying to access that data. Indexing isn't compute intensive, but bandwidth intensive, so doing it where the data is should be much faster.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 7th October 2023, 23:14   #12  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,719
Quote:
Originally Posted by benwaggoner View Post
Could an index be generated on the server where the data lives, and then be passed on to be used by the client trying to access that data. Indexing isn't compute intensive, but bandwidth intensive, so doing it where the data is should be much faster.
The problem is that generally the data resides on storage solutions, not full blown servers you have control on.
I'm talking about things like Dell Isilon, Harmonic MediaGRID, AVID Nexis etc.
In other words, those storage solutions are generally linux-based firmware locked high availability RAID6 whose only purpose is to keep your data up all the time and you couldn't just randomically run let's say, ffindex on them.
FranceBB is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 14:32.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2023, vBulletin Solutions Inc.