Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 3rd July 2019, 08:51   #1  |  Link
tarun93
Registered User
 
Join Date: Jun 2019
Posts: 10
XYZ to RGB with ffmpeg

Hi Guys,

I was trying to transcode a DCP to a lossless x265 10 bit for archival purposes.
The main issue I'm facing, is the colors are washed up. Tried playing the DCP with DCP-o-matic 2 main app and the colors are right but with vlc, ffplay, mplayer there is a color shift as explained before, even DCP-o-Matic 2 Player there is the same issue.
I have also tried to re encode the DCP mxf with ffmpeg to x265 yuv444p10le / yuv444p12le lossless but the output has similar color shift.


DCP-o-Matic export of ProRes and H264 both have its a few issues.
H264 downmixes the audio channels and any and all frequencies over 15kHz are gone.
with ProRes the audio layout is messed up and I either ways have to fix it with ffmpeg.

ffmpeg xyz2rgb filter is too outdated and could not get it compiled successfully.

Lastly, I am sorry if this is the wrong sub section of the forum to post this question.
Attached Images
    
tarun93 is offline   Reply With Quote
Old 3rd July 2019, 18:56   #2  |  Link
Asmodian
Registered User
 
Join Date: Feb 2002
Location: San Jose, California
Posts: 4,406
You do need to do the XYZ to RGB conversion somehow...

How about exporting the video by itself and dealing with the audio later?
__________________
madVR options explained
Asmodian is offline   Reply With Quote
Old 4th July 2019, 00:55   #3  |  Link
Sparktank
47.952fps@71.928Hz
 
Sparktank's Avatar
 
Join Date: Mar 2011
Posts: 940
Try this, but change for x265 use (and not c:v ffvhuff).
Quote:
ffmpeg -i "video.mxf" -y -c:v ffvhuff -pix_fmt yuv420p10le -an -map_metadata -1 "output-yuv420p10le.avi"
I don't think the xyz2rgb code has changed much since it's been put in (that is, outside of FFMPEG development).

If you added a bunch of x265 commands to enable HDR, you're going to get those bottom screens.
DCP's/MXF/XYZ sources don't normally come with HDR. They're SDR, so you don't need x265 HDR commands.

Sounds like copy/paste from somewhere else.
__________________
Win10 (x64) build 19041
NVIDIA GeForce GTX 1060 3GB (GP106) 3071MB/GDDR5 | (r435_95-4)
NTSC | DVD: R1 | BD: A
AMD Ryzen 5 2600 @3.4GHz (6c/12th, I'm on AVX2 now!)
Sparktank is offline   Reply With Quote
Old 4th July 2019, 06:29   #4  |  Link
tarun93
Registered User
 
Join Date: Jun 2019
Posts: 10
Quote:
Originally Posted by Asmodian View Post
How about exporting the video by itself and dealing with the audio later?
what do you have on your mind ? I am open to suggestions.

Quote:
Originally Posted by Sparktank View Post
Try this, but change for x265 use (and not c:v ffvhuff).


I don't think the xyz2rgb code has changed much since it's been put in (that is, outside of FFMPEG development).

If you added a bunch of x265 commands to enable HDR, you're going to get those bottom screens.
DCP's/MXF/XYZ sources don't normally come with HDR. They're SDR, so you don't need x265 HDR commands.

Sounds like copy/paste from somewhere else.
agreed xyz2rgb havent changed alot over time.

I'm aware the content in the DCP is not HDR and I have not enabled any HDR prams in my ffmpeg command.

ffvhuff outputs similar result with faded colours.


lastly i dont under what do you mean by
Quote:
Originally Posted by Sparktank View Post
Sounds like copy/paste from somewhere else.
here is the full ffmpeg cmd i use

Code:
ffmpeg -y -r 24 -i video.mxf -i audio.mxf -c:v libx265 -profile:v main10-444 -level:v 5.1 -c:a copy -crf 17 -vf scale=2048:858 -preset slower -pix_fmt yuv444p10le -x265-params scenecut=0:open_gop=0:min-keyint=72:keyint=72 -movflags faststart -color_range pc -color_primaries smpte431 -color_trc bt709 -colorspace bt709 -r 24 output.mkv

Last edited by tarun93; 4th July 2019 at 06:33.
tarun93 is offline   Reply With Quote
Old 4th July 2019, 07:21   #5  |  Link
Asmodian
Registered User
 
Join Date: Feb 2002
Location: San Jose, California
Posts: 4,406
Quote:
Originally Posted by tarun93 View Post
what do you have on your mind ? I am open to suggestions.
I was only noticing that your issues with 'DCP-o-Matic export of ProRes and H264' were audio related so I figure you could simply remux that video with good audio using mkvtoolnix or similar.
__________________
madVR options explained
Asmodian is offline   Reply With Quote
Old 4th July 2019, 07:24   #6  |  Link
tarun93
Registered User
 
Join Date: Jun 2019
Posts: 10
Quote:
Originally Posted by Asmodian View Post
I was only noticing that your issues with 'DCP-o-Matic export of ProRes and H264' were audio related so I figure you could simply remux that video with good audio using mkvtoolnix or similar.

Currently i export it as prores 422 HQ via DCP-O-Matic and fix the audio layout of the mov via ffmpeg, but as we all know ProRes ain't a lossless format.
tarun93 is offline   Reply With Quote
Old 4th July 2019, 07:37   #7  |  Link
Sparktank
47.952fps@71.928Hz
 
Sparktank's Avatar
 
Join Date: Mar 2011
Posts: 940
What is your source?
There's still a wide range of info you are not giving.
Mediainfo isn't that useful.
I'd rather trust FFprobe, since that's how FFmpeg will read it.
They both rely on different processes, so go with the one that will be processing it.

Why -color_primaries smpte431 ?
I largely suspect this is why you're getting that HDR look if you were to work with a UHD-BD that has HDR10.

Make the primaries 709 like the rest.
Whatever the source is, if you convert to 709, you'll get the proper pictures. Unless the player you use can do 431 properly.
If FFprobe shows everything is 709, don't force smpte431.

If it truly is an XYZ colorspace source, then converting it to any other format should turn out alright when you watch in something like MPC-HC with MadVR as the renderer. I don't know about any other player.


Lastly, I know a lot of people come in here from torrent sites that copy settings from some random encoder, expecting those to be the one-for-all encoding settings.
X265 is not really needed unless you have HDR to work with.
OR want hardware playback compatibility.
I just suspected you copied some random setting that's useful for HDR encodes and applied to an SDR encode (since no info was really given).

Otherwise, you can achieve better transparency with x264 (that's been around for centuries).
__________________
Win10 (x64) build 19041
NVIDIA GeForce GTX 1060 3GB (GP106) 3071MB/GDDR5 | (r435_95-4)
NTSC | DVD: R1 | BD: A
AMD Ryzen 5 2600 @3.4GHz (6c/12th, I'm on AVX2 now!)
Sparktank is offline   Reply With Quote
Old 4th July 2019, 09:03   #8  |  Link
tarun93
Registered User
 
Join Date: Jun 2019
Posts: 10
Quote:
Originally Posted by Sparktank View Post
What is your source?
There's still a wide range of info you are not giving.
Mediainfo isn't that useful.
I'd rather trust FFprobe, since that's how FFmpeg will read it.
They both rely on different processes, so go with the one that will be processing it.
the got the movie from cinema movie distributor legally. Here is a ffmpeg probe.
Code:
  libavutil      56. 30.100 / 56. 30.100
  libavcodec     58. 53.101 / 58. 53.101
  libavformat    58. 28.101 / 58. 28.101
  libavdevice    58.  7.100 / 58.  7.100
  libavfilter     7. 55.100 /  7. 55.100
  libswscale      5.  4.101 /  5.  4.101
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
[mxf @ 0x5578d51420c0] "OPAtom" with 2 ECs - assuming OP1a
Input #0, mxf, from 'j2c_5577dce3-54d0-4838-bb17-31ee7b6f2f56.mxf':
  Metadata:
    operational_pattern_ul: 060e2b34.04010101.0d010201.10000000
    uid             : 28380a8d-7195-491d-890c-efea1530c91f
    generation_uid  : 2d26452b-4b28-4275-b90e-99a750a8e974
    company_name    : libdcp
    product_name    : libdcp
    product_version : 1.5.5
    product_uid     : 43059a1d-0432-4101-b83f-736815acf31d
    modification_date: 2019-02-18T09:35:36.000000Z
    application_platform: linux
    material_package_umid: 0x060A2B340101010501010F2013000000CABB969FD28849F58D8BF92078241A21
    material_package_name: AS-DCP Material Package
    timecode        : 00:00:00:00
  Duration: 01:19:57.63, start: 0.000000, bitrate: 144602 kb/s
    Stream #0:0: Video: jpeg2000 (JPEG 2000 digital cinema 2K), xyz12le(progressive), 2048x858, SAR 1:1 DAR 1024:429, 24 tbr, 24 tbn, 24 tbc
    Metadata:
      file_package_umid: 0x060A2B340101010501010F20130000005577DCE354D04838BB1731EE7B6F2F56
      file_package_name: File Package: SMPTE 429-4 frame wrapping of JPEG 2000 codestreams
      track_name      : Picture Track
Quote:
Originally Posted by Sparktank View Post
Why -color_primaries smpte431 ?
I largely suspect this is why you're getting that HDR look if you were to work with a UHD-BD that has HDR10.

Make the primaries 709 like the rest.
Whatever the source is, if you convert to 709, you'll get the proper pictures. Unless the player you use can do 431 properly.
If FFprobe shows everything is 709, don't force smpte431.
smpte431 bt709 both give a similar output with colors faded away.

Quote:
Originally Posted by Sparktank View Post
If it truly is an XYZ colorspace source, then converting it to any other format should turn out alright when you watch in something like MPC-HC with MadVR as the renderer. I don't know about any other player.
One of the reason I am trying to transcode the whole thing off XYZ to RGB is because it is a client requirement.

Quote:
Originally Posted by Sparktank View Post
Lastly, I know a lot of people come in here from torrent sites that copy settings from some random encoder, expecting those to be the one-for-all encoding settings.
X265 is not really needed unless you have HDR to work with.
OR want hardware playback compatibility.
I just suspected you copied some random setting that's useful for HDR encodes and applied to an SDR encode (since no info was really given).

Otherwise, you can achieve better transparency with x264 (that's been around for centuries).
As i mentioned the DCP is legally purchased and I ain't here to get settings or make/release any pirated copies of the content. I would be more then happy to just export the DCP as ProRes/Hx264 from dcp-o-matic. I am here because my knowledge regarding transcoding in general is limited.
Lastly, I thought x265 lossless will have better compression ratios for 2K or 4K content compared to x264.

Last edited by tarun93; 4th July 2019 at 09:07.
tarun93 is offline   Reply With Quote
Old 4th July 2019, 09:17   #9  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,565
-crf 17 isn't lossless.
sneaker_ger is offline   Reply With Quote
Old 4th July 2019, 09:41   #10  |  Link
tarun93
Registered User
 
Join Date: Jun 2019
Posts: 10
Quote:
Originally Posted by sneaker_ger View Post
-crf 17 isn't lossless.
agreed . I was first trying x264 lossless. i read on ffmpeg site
Quote:
The range of the CRF scale is 0–51, where 0 is lossless, 23 is the default, and 51 is worst quality possible. A lower value generally leads to higher quality, and a subjectively sane range is 17–28. Consider 17 or 18 to be visually lossless or nearly so; it should look the same or nearly the same as the input but it isn't technically lossless."
so was trying it out to see if i could make out any difference.
once i have the colors right there are 2 changes that i can think of
1. color_trc color_primaries colorspace will be set to bt709
2. crf value will be zero and/or if we settle with HEVC then x265-params lossless
tarun93 is offline   Reply With Quote
Old 4th July 2019, 09:45   #11  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,565
x264 10 bit -crf 0 is not lossless. -x264-params qp=0 is.
sneaker_ger is offline   Reply With Quote
Old 4th July 2019, 09:51   #12  |  Link
tarun93
Registered User
 
Join Date: Jun 2019
Posts: 10
Quote:
Originally Posted by sneaker_ger View Post
x264 10 bit -crf 0 is not lossless. -x264-params qp=0 is.
thank you mate. will keep in mind.


are there anyway to fix the colors guys ????
tarun93 is offline   Reply With Quote
Old 4th July 2019, 19:13   #13  |  Link
Asmodian
Registered User
 
Join Date: Feb 2002
Location: San Jose, California
Posts: 4,406
Quote:
Originally Posted by tarun93 View Post
are there anyway to fix the colors guys ????
Have you tried MPC-HC with madVR? Many players do not respect the tags. If BT.709 and smpte431 look the same the player is not using the tags.

Otherwise you will need to convert to BT.709.
__________________
madVR options explained
Asmodian is offline   Reply With Quote
Old 4th July 2019, 22:54   #14  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,988
Tried Resolve for this kind of thing? You'd probably need the paid version, but it's worth trying I think.
Blue_MiSfit is offline   Reply With Quote
Old 8th July 2019, 06:56   #15  |  Link
tarun93
Registered User
 
Join Date: Jun 2019
Posts: 10
Quote:
Originally Posted by Blue_MiSfit View Post
Tried Resolve for this kind of thing? You'd probably need the paid version, but it's worth trying I think.
what do you have in mind ????
tarun93 is offline   Reply With Quote
Old 9th July 2019, 08:24   #16  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,988
Import the J2k into a resolve timeline, export as something like ProRes 4444 or DNxHR?

I know Resolve tears through MXF wrapped J2K thanks to Kakadu, but not sure if / how it handles XYZ Just a thought!
Blue_MiSfit is offline   Reply With Quote
Old 10th July 2019, 10:53   #17  |  Link
tarun93
Registered User
 
Join Date: Jun 2019
Posts: 10
ping !!!

Last edited by tarun93; 10th July 2019 at 13:35.
tarun93 is offline   Reply With Quote
Old 10th July 2019, 18:07   #18  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
And broadly speaking, does anyone know of any free/open tools that can do proper linear light conversions/resampling? Some of us have access to higher end tools, but sometimes it's nice to be able to do a one-off on a laptop while on the road or whatever. Just something that can do a decent bicubic 4K->1080p conversion in 2100 would be handy.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 10th July 2019, 21:27   #19  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
Out of curiosity, in case you wanna do it with ffmpeg and/or Avisynth in the future, can you upload a very short sample of the original JPEG 2000 XYZ file or just a few screenshot that are in XYZ?
I was thinking about making a matrix of linear transformation (LUT) from your XYZ, output a regular BT709 Linear and add it to my free public LUT Collection.
FranceBB is offline   Reply With Quote
Old 11th July 2019, 13:00   #20  |  Link
tarun93
Registered User
 
Join Date: Jun 2019
Posts: 10
Thanks for the pointer FranceBB

Quote:
Originally Posted by FranceBB View Post
Out of curiosity, in case you wanna do it with ffmpeg and/or Avisynth in the future, can you upload a very short sample of the original JPEG 2000 XYZ file or just a few screenshot that are in XYZ?
I was thinking about making a matrix of linear transformation (LUT) from your XYZ, output a regular BT709 Linear and add it to my free public LUT Collection.
The original copy is encrypted so i cant get original XYZ J2K/TIFF shots.
if i decrypt it and make a dcp copy i get the color issue.

I got a free 3D LUT which fixed the colors.
here is the link.

Is there any "DCI P3 XYZ to DCI P3 YUV/RGB" 3D lut ???

another question what do you guy suggest for the colors should i keep it limited or full range ??? i have mostly noticed movies and TV shows have Limited range

Last edited by tarun93; 11th July 2019 at 14:13.
tarun93 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 21:39.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.