Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > New and alternative video codecs

Reply
 
Thread Tools Search this Thread Display Modes
Old 28th October 2018, 14:52   #1181  |  Link
Selur
Registered User
 
Selur's Avatar
 
Join Date: Oct 2001
Location: Germany
Posts: 5,885
Quote:
Is that happening to everyone else or did i find a rare bug in my build proccess or encoder settings?
Encoded a bunch of files last week with with up-to-date builds of aomenc and rav1e and had no problem.
__________________
Hybrid here in the forum, homepage
Selur is offline   Reply With Quote
Old 29th October 2018, 08:41   #1182  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 92
Quote:
Originally Posted by utack View Post
the past ~2 weeks libaom has been producing files for me it can't decode any more, a few frames into the stream it dies and never recovers

dav1d is not dying, but shows a lot of blocky colorful artifarcts in many GOPs
Is that happening to everyone else or did i find a rare bug in my build proccess or encoder settings?
If you're using more than one slice and more than one thread, you've just met my old friend BUG 2054
So far I have reached the conclusion it happens (but not always!) when forcing the encoder to use columns with the width of 2 superblocks (128 pixels). Doesn't happen when the columns have a width of 256 pixels or more on the same file with all the other settings being the same.
I have been doing single threaded encodes for weeks because of this.

Last edited by SmilingWolf; 29th October 2018 at 09:08.
SmilingWolf is offline   Reply With Quote
Old 29th October 2018, 10:59   #1183  |  Link
hajj_3
Registered User
 
Join Date: Mar 2004
Posts: 901
MPC-BE beta 1.5.3 v4106 has been released which supports AV1: https://sourceforge.net/projects/mpc...r.zip/download
hajj_3 is offline   Reply With Quote
Old 29th October 2018, 11:50   #1184  |  Link
utack
Registered User
 
Join Date: Apr 2018
Posts: 40
Quote:
Originally Posted by SmilingWolf View Post
If you're using more than one slice and more than one thread, you've just met my old friend BUG 2054
That might be it. Or the 10bit pipeline.
I switched to 8bit and no "threads" option and that works for now.
utack is offline   Reply With Quote
Old 29th October 2018, 18:48   #1185  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 2,972
Quote:
Originally Posted by PatchWorKs View Post
Yes, but - as you probably know - more bits results in better compression...

Anyway source is AVC 264 FHD 8bits 4:2:0


Yes, I made some tests @444 and encoded videos seems - to me- less contrasted/colorful (more natural ?)...
There REALLY shouldnít be any value in using more chroma sub samples than the source! Unless you are downscaling in the same operation. A 720p 4:2:0 sourceís chroma samples would be 4:4:4 at 360p.

You might want to try some kind of blind test, though. You shouldnít see more color or more contrast with more samples. You might see sharper edges between areas of strong color. For example, with ClearType text, which plays all sorts of RGB 444 subpixel tricks that donít translate well to even other LCD panels, let along different color spaces.

(Pro tip for doing screen recording; disable chroma subpixel rendering for text. Otherwise it can look unpredictably funky on different displays).
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 30th October 2018, 21:30   #1186  |  Link
Revan654
Registered User
 
Revan654's Avatar
 
Join Date: May 2004
Posts: 324
1. With Rav1e anyone know what type -r Reconstruction takes (Boolean, String, Int, etc...) and what it actually does. I can not find any info on it and the help menu is blank for that option.

2. This has been likely asked a dozen times over, Whats the main reason why aomenc is slower compared to other encoders out there?

I'm just starting to research AV1.
Revan654 is offline   Reply With Quote
Old 30th October 2018, 22:56   #1187  |  Link
marcomsousa
Registered User
 
Join Date: Jul 2018
Posts: 60
Quote:
Originally Posted by Revan654 View Post
2. This has been likely asked a dozen times over, Whats the main reason why aomenc is slower compared to other encoders out there?
  1. libaom it was developed for research purposes during AV1 design.
  2. Isn't optimized for speed, only to be a good reference encoder/decoder
  3. Almost all other encoder, the reference encoder isn't fast enough.
  4. That's why there are 3ļ parties implementation of the same codecs dav1d and rav1e
  5. Now that the AV1 design is finished, the reference encoder is now optimizing there code for speed.
  6. We need to wait until there are hardware acceleration (for decoding 4k/8k UHD) (2, 3 years to be mainstream)
  7. New codecs are (always) more complex that the codecs before.
  8. New codecs that are releasing now are too much complex for todays CPU, but they are designed to be using mainstream in future CPU 3 years from now.
  9. Before AV1 became mainstream, AOM was to begin working in AV2, that will me more complex that AV1, and will be design to CPU released 7 or 8 years from now.
  10. Increase the complex is the only way to produce better quality with less space. And that is ok if the hardware continues to improve over time..
__________________
AV1 win64 VS2017 builds
Last build here | History
I also open source the build scripts at Github: here

Last edited by marcomsousa; 31st October 2018 at 10:10.
marcomsousa is offline   Reply With Quote
Old 2nd November 2018, 14:19   #1188  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,943
New uploads: (MSYS2; MinGW32: GCC 7.3.0 / MinGW64: GCC 8.2.0)

AOM v1.0.0-864-g351711076
now with TPL model (RDO modulation based on frame temporal dependency) and block based denoiser

rav1e 0.1.0 (7492fc5 / 2018-11-01)

dav1d 0.0.1 (287ba91 / 2018-11-02)
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 2nd November 2018, 18:10   #1189  |  Link
Mjpeg
Registered User
 
Join Date: Jun 2018
Posts: 7
16x FTW

Report from a 1 day meeting on future codecs, h264 through VVC.

http://www.streamingmedia.com/Articl...P9-128213.aspx

The most interesting quote is from a Youtube encoding engineer that AV1 encoding time is down to 16x slower vs. VP9, so that's a nice performance trend (no doubt giving up some % of quality). What's important (as benwaggoner always says) is what quality@perf tradeoffs are available.
Mjpeg is offline   Reply With Quote
Old 2nd November 2018, 20:39   #1190  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 2,972
Quote:
Originally Posted by Mjpeg View Post
Report from a 1 day meeting on future codecs, h264 through VVC.

http://www.streamingmedia.com/Articl...P9-128213.aspx

The most interesting quote is from a Youtube encoding engineer that AV1 encoding time is down to 16x slower vs. VP9, so that's a nice performance trend (no doubt giving up some % of quality). What's important (as benwaggoner always says) is what quality@perf tradeoffs are available.
Yeah, YouTube is sort of a special case for encoding. Given how much content they get and the average views/upload, economically theyíre going to spend fewer MIPS/pixel than for premium content. And they do their stuff (last I heard) single-threaded on unused-at-that-moment Google servers, ala a spot instance. Which is why libvpx never got a lot of multithreading, nor were the VPx series of bitstream vetted for parallelizability.

Flip side is no one expects spectacular quality from YouTube. Itís free. So even though video game captures always look terrible (lots of high frequency sharp edges...), thatís what people are used to and so they donít really think about it any more. So YouTube can experiment a lot at how to make good-enough quality fast, which is a different direction from a lot of other folks, and a very useful one in encoder development.

It can be a good starting point for live encoders, although a YouTube can handle some content taking 3x longer to encode than other content due to complexity.

HEVC and AV1 have great tools for making text and video game footage a LOT better. But they are also pretty expensive to add to normal mode detection, so lot of heuristics to figure out when to use them are important.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 2nd November 2018, 22:26   #1191  |  Link
Mjpeg
Registered User
 
Join Date: Jun 2018
Posts: 7
Live encoding

I'm all for AV1 encoders getting faster, but of course on an absolute scale it's pretty slow still, spending a lot of CPU for all that efficiency.

I always figured live-encoding would be quite a wait, although the official framing of AV! always mentions that live-stream/chat was an important case they had thought about.

So anyway I find a talk by Nathan Egge of Mozilla

Most of the slides look familiar but there's a claim about live encoding I had not seen before.

https://people.xiph.org/~negge/AVIF2018.pdf

page 55:

rav1e Live Encoding
Shown at IBC in Sept 2018
● 640x480 @ 30 fps
● Single tile / thread
● Simplified feature set

I guess one sign of AV1 progress will be when someone posts a link here to an AV1 webcam.
Mjpeg is offline   Reply With Quote
Old 3rd November 2018, 11:01   #1192  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 92
32/64bits binaries (GCC 8.2):
av1-1.0.0-877-ge5761e020: https://mega.nz/#!1sgl3QrB!x6F6SfLzz...vWZ-IMSzZukIWs

A long standing multithreading bug has been fixed tonight, so here's a new build
Cc @utack
SmilingWolf is offline   Reply With Quote
Old 3rd November 2018, 16:11   #1193  |  Link
v0lt
Registered User
 
Join Date: Dec 2008
Posts: 1,193
Quote:
Originally Posted by SmilingWolf View Post
A long standing multithreading bug has been fixed tonight, so here's a new build
I still donít see aomenc.exe using more than one thread.
v0lt is offline   Reply With Quote
Old 3rd November 2018, 17:44   #1194  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 92
It won't by default. Tile columns, rows and threads count are all set to 0.
You have to give at least --tile-columns=1 (and/or --tile-rows=1) --threads=2 to have it use multithreading.
SmilingWolf is offline   Reply With Quote
Old 3rd November 2018, 18:12   #1195  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,943
32 bit GCC 8.2? ... Does it exist for Windows? MSYS2 did not yet solve internal compiling errors, I believe.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 3rd November 2018, 18:24   #1196  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 92
I'm cross compiling from a linux VM. Made it easier to switch between compiler versions back when I was investigating the optimization related bug, then the bug was worked around and the environment stuck.
Silver lining, I'm not stuck with an outdated compiler and I don't have to manage my own MSYS2 package.
SmilingWolf is offline   Reply With Quote
Old 3rd November 2018, 18:35   #1197  |  Link
Clare
Registered User
 
Join Date: Apr 2016
Posts: 60
Quote:
Originally Posted by SmilingWolf View Post
It won't by default. Tile columns, rows and threads count are all set to 0.
You have to give at least --tile-columns=1 (and/or --tile-rows=1) --threads=2 to have it use multithreading.
Just use --row-mt=1 instead of tiles, it maxes out all my threads.
Clare is offline   Reply With Quote
Old 3rd November 2018, 18:57   #1198  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 92
What does your command line look like? So far I've been unable to make row-mt work myself

My tries so far:
this one generates invalid bitstream:
../../bin8/aomenc --frame-parallel=0 --tile-columns=2 --tile-rows=2 --row-mt=1 --threads=4 --auto-alt-ref=1 --cpu-used=4 --tune=psnr --passes=2 --end-usage=q --cq-level=40 --test-decode=fatal -o test.av1.cq40.webm orig.i420.y4m
Pass 1/2 frame 480/481 92352B 1539b/f 36899b/s 17849 ms (26.89 fps)
Pass 2/2 frame 19/0 0B 17871 ms 1.06 fps [ETA unknown] 2423FFailed to decode frame 2 in stream 0: Corrupt frame detected
Failed to decode tile data

This one works but uses only 12-13% (one core) of my 4c/8t CPU:
../../bin8/aomenc --frame-parallel=0 --tile-columns=2 --tile-rows=2 --row-mt=1 --auto-alt-ref=1 --cpu-used=4 --tune=psnr --passes=2 --end-usage=q --cq-level=40 --test-decode=fatal -o test.av1.cq40.webm orig.i420.y4m
Pass 1/2 frame 480/481 92352B 1539b/f 36899b/s 18130 ms (26.47 fps)
Pass 2/2 frame 16/0 0B 18152 ms 52.88 fpm [ETA unknown]

Last edited by SmilingWolf; 3rd November 2018 at 19:06.
SmilingWolf is offline   Reply With Quote
Old 3rd November 2018, 19:30   #1199  |  Link
v0lt
Registered User
 
Join Date: Dec 2008
Posts: 1,193
Quote:
Originally Posted by SmilingWolf View Post
It won't by default. Tile columns, rows and threads count are all set to 0.
You have to give at least --tile-columns=1 (and/or --tile-rows=1) --threads=2 to have it use multithreading.
I always ask 4 threads, but it never worked.
aomenc --codec=av1 --cq-level=20 --threads=4

Added:
I do not understand what are the columns and rows in this context. If a codec divides a frame into identical independent cells, it is unclear how it can effectively compress in a multi-thread mode.

Last edited by v0lt; 3rd November 2018 at 19:38.
v0lt is offline   Reply With Quote
Old 3rd November 2018, 22:20   #1200  |  Link
SmilingWolf
I am maddo saientisto!
 
SmilingWolf's Avatar
 
Join Date: Aug 2018
Posts: 92
Tile columns (click to enlarge):


The frame is divided in N (10 in my case) columns of equal width.
Each column in indipendent, so every thread can indipendently work on a tile.
Using tile columns (and/or rows) makes decoding faster too, because each tile can be decoded by a separate thread, making playback way smoother

You command line doesn't show any multithreading because you only have a single big tile, which is being worked on by thread 0, leaving threads 1,2,3 with nothing to do.

Last edited by SmilingWolf; 3rd November 2018 at 22:28.
SmilingWolf is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 11:52.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.