Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > New and alternative video codecs

Reply
 
Thread Tools Search this Thread Display Modes
Old 29th October 2018, 19:43   #61  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,786
How did you do that? Did you apply a custom patch? Would that work with GCC 7.3.0 too, or only with GCC 8.2.0?
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 30th October 2018, 13:11   #62  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 95
First thing I did was modify line 99 in CMakeList.txt from
if( UNIX )
to
if( NOT MSVC )
which is the same thing aom does to set the compiler flags for GCC/ICC/Clang.

Then I had to add
add_compile_options( "-mavx2" )
right below
add_compile_options( "-msse4.1" )

All the above BEFORE the "cmake -G "MSYS Makefiles" -DBUILD_STATIC=ON -DCMAKE_BUILD_TYPE=Release .." command of course

However! This triggers something in GCC and the compilation process bugs out on IntraPrediction.cpp because of some -Wmaybe-uninitialized which appears ONLY when compiling with -mavx2
So what I do is, make -j6 until it bugs out, then make VERBOSE=1 to intercept the "bad" compilation command, remove -mavx2 from it, run that command, reposition myself at the top of the build directory, then resume building normally.
Gotta do the same thing twice because IntraPrediction.cpp is used in a couple of libraries, so the object file ends up in different directories each time.

Not the most straightforward way, but everything seems to work fine afterwards

Last edited by SmilingWolf; 30th October 2018 at 13:15.
SmilingWolf is offline   Reply With Quote
Old 30th October 2018, 13:37   #63  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,786
Too much manual meddling for me as autobuild suite user...

I made it an official issue and hope it will be taken seriously.

At least it has a future. Unlike mencoder which may already be dead.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 30th October 2018, 14:02   #64  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 95
Well I did all that to stay on the "safe" side of things.

However one could modify line 75 in CMakeLists.txt from
bb_enable_warnings( gcc warnings-as-errors -Wno-sign-compare -Wno-class-memaccess )
to
bb_enable_warnings( gcc warnings-as-errors -Wno-sign-compare -Wno-class-memaccess -Wno-maybe-uninitialized )
along with the other CMakeLists.txt described above to enable SSE4.1 and AVX2. It compiles fine without manual intervention, altough I haven't tested yet the encoder built this way (just finished compiling).
There shouldn't be any problem though, -Wmaybe-uninitialized is known to be overzealous at times

Also, it's always strange to see my nick in quotes like 'SmilingWolf' or 'lupo...'. Makes me feel kinda like a shady character lol
Note that I don't mean anything by this. I'm merely too used to considering it my first name when I'm online

EDIT:
summing up, a git diff ready to apply to the repo looks like this:
Code:
diff --git a/CMakeLists.txt b/CMakeLists.txt
index 2d59e74..3c3430c 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -72,7 +72,7 @@ endif()
 # bb_enable_warnings( gcc -Wno-unused-variable )
 # bb_enable_warnings( gcc-4.8 warnings-as-errors -Wno-unused-variable )
 # for gcc 8.2:
-bb_enable_warnings( gcc warnings-as-errors -Wno-sign-compare -Wno-class-memaccess)
+bb_enable_warnings( gcc warnings-as-errors -Wno-sign-compare -Wno-class-memaccess -Wno-maybe-uninitialized )
 
 if( XCODE )
   bb_enable_warnings( clang warnings-as-errors
@@ -96,8 +96,9 @@ endif()
 bb_enable_warnings( msvc warnings-as-errors "/wd4996" )
 
 # enable sse4.1 build for all source files for gcc and clang
-if( UNIX )
+if( NOT MSVC )
   add_compile_options( "-msse4.1" )
+  add_compile_options( "-mavx2" )
 endif()
 
 # enable parallel build for Visual Studio
Attached the diff, too
Attached Files
File Type: diff doom9.diff (840 Bytes, 124 views)

Last edited by SmilingWolf; 30th October 2018 at 14:10.
SmilingWolf is offline   Reply With Quote
Old 30th October 2018, 14:08   #65  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,786
Well, I looked from the opposite side, pointing out it's just a nick, not a real name ... tastes are different.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 3rd November 2018, 11:04   #66  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 95
Quote:
Originally Posted by LigH View Post
Well, I looked from the opposite side, pointing out it's just a nick, not a real name ... tastes are different.
Sure, it's really not a big deal

64bits single-threaded binaries only (MSYS2/GCC 8.2):
VVCSoftware_VTM-2.2-156-g0326358: https://mega.nz/#!k4w3iQJI!0zN4oMEP2...rhoPcxcwzJ4RP8
SmilingWolf is offline   Reply With Quote
Old 10th November 2018, 15:08   #67  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 95
64bits single-threaded binaries only (MSYS2/GCC 8.2):
VVCSoftware_VTM-3.0rc1: https://mega.nz/#!klR2xAqI!-6CgrU7UI...WRHNtmK38xAO6o
SmilingWolf is offline   Reply With Quote
Old 12th November 2018, 22:16   #68  |  Link
Jamaika
Registered User
 
Join Date: Jul 2015
Posts: 709
I will add commands for the 8bit codec. They still need to work on decoding.

EncoderApp.exe -i image_21447.yuv -b video.vvc -wdt 1920 -hgt 1080 --InputBitDepth=8 --InputChromaFormat=420 --QP=0
--SEIDecodedPictureHash=0 --Verbosity --FramesToBeEncoded=1 --ConformanceWindowMode=0 --FrameRate=25
--QuadtreeTULog2MaxSize=5 --QuadtreeTUMaxDepthIntra=4 --QuadtreeTUMaxDepthInter=4 --Profile=next
--IntraPeriod=1 --GOPSize=1 --TransformSkip=1 --TransformSkipFast=1 --SEIDecodedPictureHash=3 --QTBT=1 --CTUSize=32

DecoderApp.exe -b video.vvc -o output.yuv --SEIDecodedPictureHash=0 --OutputBitDepth=8

https://www.sendspace.com/file/7ho3k5
Jamaika is offline   Reply With Quote
Old 12th November 2018, 22:31   #69  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 95
The official testing guidelines actually suggest using the config files already included in the repo, so the cmdline should look more like:

bin/EncoderApp.exe -c cfg/encoder_randomaccess_vtm.cfg --InputFile=orig.i420.yuv --ReconFile=vvc.qp31.3.0rc1.yuv --BitstreamFile=vvc.qp31.3.0rc1.bin --FrameRate=24000/1001 --FramesToBeEncoded=2159 --SourceWidth=320 --SourceHeight=240 --InputBitDepth=8 --OutputBitDepth=8 --InternalBitDepth=8 --QP=31

Last edited by SmilingWolf; 12th November 2018 at 22:35.
SmilingWolf is offline   Reply With Quote
Old 19th November 2018, 07:25   #70  |  Link
Jamaika
Registered User
 
Join Date: Jul 2015
Posts: 709
Quote:
Originally Posted by SmilingWolf View Post
The official testing guidelines actually suggest using the config files already included in the repo, so the cmdline should look more like:

bin/EncoderApp.exe -c cfg/encoder_randomaccess_vtm.cfg --InputFile=orig.i420.yuv --ReconFile=vvc.qp31.3.0rc1.yuv --BitstreamFile=vvc.qp31.3.0rc1.bin --FrameRate=24000/1001 --FramesToBeEncoded=2159 --SourceWidth=320 --SourceHeight=240 --InputBitDepth=8 --OutputBitDepth=8 --InternalBitDepth=8 --QP=31
Thanks for info
Quote:
Originally Posted by benwaggoner View Post
These theoretical encoding times are really JM versus HM; encoders which are far too slow for any practical use, and really not very good psychovisually. And for decoder complexity, that's what Profiles and Levels are for. If there is a 16x worst case, probably 85% of the practical efficiency gains can be found in the first 2x of decoder complexity. Hitting the right balance between efficiency gains and decoder complexity is a huge part of designing a major bitstream format, and one of the reasons that the MPEG/ITU codecs are so good. There are just so many eyeballs from so many different industry sectors pounding on it to squeeze every last bit and MIPS out.
A little advertising. What is the difference between HEVC and VVC?

VVC Test Model 1.0 New block partitioning
• 128x128 Coding Tree Units (CTU)
• Recursive quadtree partitioning (QT)
• Nested recursive multi-type tree partitioning (MTT) with binary or ternary split

Oh, but it means more precision as well as a computer adapted to 8K movies.
Otherwise, the film is processed four times longer than JCTVC using the parameters from the official test.

Last edited by Jamaika; 19th November 2018 at 07:29.
Jamaika is offline   Reply With Quote
Old 23rd November 2018, 22:58   #71  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,771
Quote:
Originally Posted by Wolfberry View Post
VVCSoftware_VTM-3.0-ae9412fc (x64 SSE2/AVX/AVX2 MSVC 19.16.27024)
Are there release notes for these new builds somewhere?

I might take a whack at some test VVC encodes for my codec challenge over the long weekend.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 24th November 2018, 00:47   #72  |  Link
Wolfberry
Helenium(Easter)
 
Wolfberry's Avatar
 
Join Date: Aug 2017
Location: Hsinchu, Taiwan
Posts: 99
https://vcgit.hhi.fraunhofer.de/jvet...tware_VTM/tags

More like a change log, but this is the only one I can find.
__________________
Monochrome Anomaly
Wolfberry is offline   Reply With Quote
Old 29th November 2018, 00:00   #73  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,786
There is a new cmake-update branch to fix the lack of CMake support in MSYS2/MinGW. The project compiles now with both MinGW32 (GCC 7.3.0) and MinGW64 (GCC 8.2.0), despite some persisting BBuildEnv warnings.

Unfortunately, the created apps crash with an external exception c000001d (illegal instruction) on an AMD Phenom-II; I assume they require more than just SSE2 to run at all. Will test tomorrow on an FX CPU.
__

P.S.:

Code:
        --SIMD                   SIMD extension to use (SCALAR, SSE41, SSE42, AVX, AVX2, AVX512), default: the highest supported extension
That explains. Enjoy parallelism.

I noticed that the MinGW32 build (GCC 7.3.0) reports [SIMD=SSE42], but the MinGW64 build (GCC 8.2.0) reports [SIMD=AVX] on an AMD FX-5600.

VVCSoftware_VTM-2.2 cmake-update 67208da2

BTW, is it correct that the current VTM encoder does not even support Y4M, only raw YUV?
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid

Last edited by LigH; 29th November 2018 at 10:09.
LigH is offline   Reply With Quote
Old 29th November 2018, 21:25   #74  |  Link
Jamaika
Registered User
 
Join Date: Jul 2015
Posts: 709
Quote:
Originally Posted by LigH View Post
BTW, is it correct that the current VTM encoder does not even support Y4M, only raw YUV?
Encoder supports raw yuv and y4m, but decoder has only raw yuv.
If true then output 10-bit and 12-bit YUV data as 5-byte and 3-byte (respectively) packed YUV data. Ignored for interlaced output.
Jamaika is offline   Reply With Quote
Old 29th November 2018, 21:55   #75  |  Link
Tommy Carrot
Registered User
 
Tommy Carrot's Avatar
 
Join Date: Mar 2002
Posts: 863
Quote:
Originally Posted by LigH View Post
BTW, is it correct that the current VTM encoder does not even support Y4M, only raw YUV?
Haven't tried VTM encoder in the last few weeks, so maybe it has been added since, but indeed, when i tested it, it only supported raw yuv. Every input parameter had to be specified manually.

Btw, thanks everybody who share builds here. I really appreciate it.
Tommy Carrot is offline   Reply With Quote
Old 29th November 2018, 23:47   #76  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,786
Input parameters should preferably be collected in configuration files. The repo offers a wide variety of examples.

I'll try a little Y4M tomorrow, thanks.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 4th December 2018, 19:51   #77  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,786
The cmake-update patch has been merged. Compilation with MinGW+GCC now works from the master branch.

VVCSoftware_VTM-3.0 14a70bbb
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid

Last edited by LigH; 4th December 2018 at 21:46.
LigH is offline   Reply With Quote
Old 5th December 2018, 08:55   #78  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,786
The media-autobuild_suite supports building VVC binaries now.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 9th December 2018, 04:13   #79  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,912
Quote:
Originally Posted by LigH View Post
The media-autobuild_suite supports building VVC binaries now.
Thanks for the binary.
I wanted to play a bit with it and I know that it can open .yuv files, so I tried to open Avisynth via ffmpeg and pipe it to vvc.exe via yuv4mpegpipe, but it seems that it can't open it, 'cause it doesn't like yuv4mpeg:

Code:
ffmpeg.exe -i "\\VBOXSVR\Share_Windows_Linux\Production\AVS\test.avs" -strict -1 -an -f yuv4mpegpipe - | vvc.exe -i - --Profile=next --FrameRate=24000/1001 --InputBitDepth=8 --OutputBitDepth=8 --InternalBitDepth=8 --QP=31 --FramesToBeEncoded=21 --QuadtreeTULog2MaxSize=5 --QuadtreeTUMaxDepthIntra=4 --QuadtreeTUMaxDepthInter=4 --GOPSize=1 --IntraPeriod=1 --CTUSize=32 --BitstreamFile=vvc.bitsteam -o "\\VBOXSVR\Share_Windows_Linux\raw_video.vvc"

pause


If I try to use "--y4m -" like in x265, instead, it gives "command not found".

I then tried to use "rawvideo" as pipe, but it still says "impossible to open -".

Code:
ffmpeg.exe -i "\\VBOXSVR\Share_Windows_Linux\Production\AVS\test.avs" -strict -1 -an -f rawvideo - | vvc.exe -i - --Profile=next --FrameRate=24000/1001 --InputBitDepth=8 --OutputBitDepth=8 --InternalBitDepth=8 --QP=31 --FramesToBeEncoded=21 --QuadtreeTULog2MaxSize=5 --QuadtreeTUMaxDepthIntra=4 --QuadtreeTUMaxDepthInter=4 --GOPSize=1 --IntraPeriod=1 --CTUSize=32 --BitstreamFile=vvc.bitsteam -o "\\VBOXSVR\Share_Windows_Linux\raw_video.vvc"

pause
Am I doing something wrong with the pipe or does vvc open only uncompressed .yuv file (and not even pipe)?
FranceBB is offline   Reply With Quote
Old 9th December 2018, 07:16   #80  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,786
The command line syntax of VVC is definitely different and a lot more complex than the one of x265 or even x264. And it is not compatible (and there is no reason why it should). Due to the complexity, it is recommended to use configuration files. A bunch of them are available from their source repository.

Only support for raw YUV is documented. But they may support Y4M too, I heard from other testers. Yet, I don't know whether it supports pipes or not. All configuration examples I saw referred to files. The error message "impossible to open -" suggests that it assumes "-" to be a file name, not understanding it means "read from a pipe".
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 03:52.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.