Cineform Decoder now in ffmpeg

[poisondeathray]

nice job kieranrk!

https://medium.com/@kierank_/reverse-engineering-the-gopro-cineform-codec-7411312bfe1c#.tcm0sjtmi

ffmpeg -codecs


.
.
D.V.L. cfhd Cineform HD
.
.



I know it's early but some early observations/test data

UHD 10bit 4:2:2 cineform stream

VFW decode through official cineform decoder (via avisynth, avsmeter) ~85FPS
ffmpeg decode ~15FPS

[smok3]

Interesting, so for now I can conclude that ffmpeg decoder is substantially slower than ffmpeg prores one?

[kolak]

Even if it's than give it some time and it will be way faster. Cineform can be faster than ProRes and because it's wavelet based than decoding to 1/4, 1/8 etc resolution can be blazing fast. It's quite useful feature to be able to watch 4K file at half res on very low CPU. You can't do it with any DCT based codec.
You should be able to decode 4K 60p Cineform on dual core i3 without problems. Original decoder is very fast, so lets hope ffmpeg will be the same soon.

[nevcairiel]

The FFmpeg decoder is not optimized yet, nor is it fully complete yet when it comes to all variations of CineForm, so yes, give it time.
From what I've been told by kierank, he thinks it can be made really fast.

[kolak]

Exactly- if original decoder can be fast than I'm sure community can make ffmpeg one very fast also.
Just typical optimisation gives 3-4x speed boost compared to some basic/first code no?

[Jamaika]

That's great! I was convinced that patents do not allow. Codec are not responsive to the color matrix. I think it is a bad writing yuv444p12le as gbrp12le (10 bpc).
Of course, I lack BT2020.

[kolak]

CF has 12bit RGB mode. You can have whatever video data you want as CF- rest is just flagging.

[Jamaika]

I'm surprised. AVI codec description is misleading.

[kolak]

AVI container is crap- no standardised matadata, timecode, field order etc.
You better to use MOV or MXF.

[kieranrk]

If you do have files that don't play - please let me know.

Upload them here:
https://trac.ffmpeg.org/ticket/1087

[smok3]

It does work for files that premiere calls "10 bit yuv" and does not work with files that premiere calls "12 bit RGB with alpha".

[Jamaika]

Test decoder Cineform yuv422p10le:
Three films and three different screenshots output.
http://i63.tinypic.com/2eggx7t.png
for %%f in ("%~dp1*.avi") do ffmpeg.exe -loglevel info -y -i %%f -ss 00:00:00.500 -f image2 -c:v png -frames:v 1 -pix_fmt rgb48be %%~nf.png
yuv422p10 noRGB, noBT709 | size file: 77.127.688
yuy422p10 RGB | size file: 83.961.528 --> color change
yuv422p10 BT709 | size file: 77.174.536 --> color change

[shekh]

AFAIK vfw encoder uses these options to convert RGB to YUV when you feed it RGB, but it does not store these options nowhere in video.
This is easy to confirm when you read from vfw back as RGB, the colors will be different depending on what options are set (while typing I got feeling that I need to doublecheck this again).
Btw from my research "use video systems RGB" has same meaning as "full range", can someone please confirm it.

edit:
I repeated my tests and now it looks more complicated.
"enc:709" option is always preserved within video. When sending RGB it is used for conversion. When sending yuv (v210) it is just saved without affecting data.
"dec:709" option is never used at all, instead it behaves like it is retrieved from the video.
"enc:video rgb" option affects conversion when sending RGB but never saved in video (or never retrieved by decoder, I have no way to know).
"dec:video rgb" used when decoding as RGB.

So what it means for ffmpeg decoder, at least it has chance to setup 601/709 choice.

[WorBry]

Re: the encoder.

Great stuff ! I can now convert some archived Cineform.avi edits encoded with the Cineform vfw codec (via AVISynth/VDub) to mov files for loading in DaVinci Resolve 12 (the Windows standalone version, now 12.5 beta). Resolve wouldn't even acknowledge the Cineform.avi files, but accepts the "re-wrapped" mov files just fine.

[shekh]

I can now convert some archived Cineform.avi edits...

Are you sure anything has changed? Re-wrapping does not care about cineform codec (avi->mov conversion worked fine for me 2 years ago).

ps. Glad you found the solution that suits you

[WorBry]

Are you sure anything has changed? Re-wrapping does not care about cineform codec (avi->mov conversion worked fine for me 2 years ago).

Oh you're right of course - it's just a direct stream copy.:o

I only started using Resolve a few days back, and every NLE I used or tried-out before (that supports third-party vfw codecs) accepted these Cineform.avi files without any problem. So I hadn't had need to convert to mov files until now.

Glad you found the solution that suits you

Yes, I was delighted to find that merely "re-wrapping" to mov is the solution. At first, I figured the reason why these Cineform.avi files were not accepted by Resolve was because they lacked some "legitimacy" metadata - given that the Cineform.avi render files created by Resolve are (of course) accepted as are those produced from GoPro Studio and Vegas.

[kolak]

You could make just QT ref files with QT 7 PRO and this should also work.

[WorBry]

I don't have QT 7 Pro, but thanks, that's useful to know.

[ianken]

Oh god, where was this last year when I had to build a solution to migrate 100k+ Cineform mezzanine files to H264? Oh, huge manatee!

[zcream]

Is the Cineform option also available in the OSX build ? Not sure if there are any extra options to include when I build with homebrew.

[kolak]

It should work with standard build.

[Malow]

Cineform is now open-source:

https://gopro.com/news/gopro-open-sources-the-cineform-codec

https://gopro.github.io/cineform-sdk/

https://github.com/gopro/cineform-sdk/releases

[kolak]

Great news!

[WorBry]

Thought that would never happen ! What do you think the prospects are then of seeing a Cineform encoder in ffmpeg any time soon?

[kolak]

I hope it will happen. ffmepg decoder is not very well optimised. "Original" code is few times faster and has all features, so I hope someone will port it.
Clineform due to its wavelet nature is quite unique and fact that you can now edit 4K streams at 1/2 resolution on i5 laptop in Premiere is amazing. No need for any proxy or optimised media.

[Midzuki]

What do you think the prospects are then of seeing a Cineform encoder in ffmpeg any time soon?

Assuming the answer from Paul B Mahol is to be taken seriously...

https://forum.videohelp.com/attachments/43740-1510717165/cineform+ffmpeg.png

[WorBry]

What an oddly worded reply from Paul B Mahol. I'm not a programmer, but if the quality of the coding is of "astonishing quality" (implying high level), surely that's a good thing - or is it just that he personally finds it too complex to tackle ? So is his the final word on the matter or do we hope for some other FFMPEG developer able and willing to step up to the plate ?

[kolak]

Yes, I was thinking about the same.

I'm not a programmer either, so don't understand what it's involved, but surely this is SDK, so you just plug into it. Don't have to re-write core.

[Rumbah]

I think it's irony (bad code quality and not well documented).

[kolak]

I couldn't decide if it's irony or not.
So it means ffmpeg code is total rubbish as it's few times slower for decoding :)

[nevcairiel]

Speed doesn't relate to code quality. But instead of wise-cracking here, maybe contribute improvements to ffmpeg? :)

[kolak]

Not directly, I understand this.
ffmepg code is written by many people and overlooked to have "fixed structure".
Cineform SDK was written by 1 (or 2) guys. You do write it very different in such a case. I know this from writing Python scripts.

This still doesn't change fact that Cineform SDK is few times faster than ffmpeg and I assume it can be speed up even further.
This just shows that ffmpeg code may have nice structure but it's far from being optimal. This applies to many parts of ffmpeg code.

I would help, but I'm not far good enough.

[Rumbah]

I couldn't decide if it's irony or not.
So it means ffmpeg code is total rubbish as it's few times slower for decoding :)

Actually it was meant the other way round. The code for integrating the SDK has to be maintained in ffmpeg. And if the SDK's code quality and documentation is not very good (and that is very likely in a 1-2 man project) then maintaining is nearly impossible.

[shekh]

To anyone. What do you think about the Active Metadata? Good/bad? Do you imagine it in FFMpeg? It is very weird part of the sdk...

[kolak]

It's quite cool. Industry is so legacy and lazy. Those sort of things should be already "present" years ago.
There was FirstLight (I think) programme which made a use of it. You could open video with it, do adjustment and if you had this file loaded in some player or NLE change were visible straight away.
Matada or rather active metadata is something very underestimated in my opinion. We still can't even pass properly simple things like color space etc properly, so maybe active meted is just to much :)
David's (guy who wrote Cineform) ideas were just to much ahead of the time and lacked support+money from bigger company. Even so he did manage to make it SMPTE standard.

[WorBry]

The code for integrating the SDK has to be maintained in ffmpeg. And if the SDK's code quality and documentation is not very good (and that is very likely in a 1-2 man project) then maintaining is nearly impossible.

So if Paul B Mahol was being ironic/sarcastic/snobbish and that actually is the case then, are all hopes of integrating the SDK in ffmpeg dashed ?

DaVinci Resolve 14.1 has now incorporated the Cineform SDK and I believe Lightworks Pro 14.1 beta and Hitfilm Express 2017 have also. There is this expectation that it is just a matter of time before FFMPEG incorporates full Cineform support.

https://hitfilm.com/forum/discussion/45239/cineform-now-fully-open-source

If that is not to be, it would be helpful to get an unambiguous statement from FFMPEG on that.

[shekh]

VDFM is also in the list. I think the FFMpeg statement was right there: "patch welcome" :)

[nevcairiel]

If that is not to be, it would be helpful to get an unambiguous statement from FFMPEG on that.

It all depends on the contributions of volunteers. Like already said above, contribute! Thats all the statement you'll ever get. FFmpeg already has a cineform decoder, so maybe someone will improve that based on the knowledge gained from the SDK, and maybe someone will clean up the encoder for integration, but until someone does, we won't know.

On that note, the code quality of the Cineform SDK and the fact that it doesn't present a clear and usable interface for clean integration (basically, its not meant to be a re-usable library) mean that any such attempts would require a *significant* effort to meet the FFmpeg guidelines.

[WorBry]

It all depends on the contributions of volunteers. Like already said above, contribute! Thats all the statement you'll ever get.

I would, but I'm not a programmer. Thanks for the explanation anyway.

[WorBry]

A comment from David Newman, who wrote the Cineform code:

http://www.dvinfo.net/forum/cineform-software-showcase/534863-cineform-sdk-now-open-source.html#post1939158

[kolak]

And there are quite a few software which did use of it and have direct integration.
Hopefully someone will take this task :)

[WorBry]

VDFM is also in the list. I think the FFMpeg statement was right there: "patch welcome" :)

Hey shekh, great to see that VDFM does now have native Cineform support also. Ran a quick test comparing Cineform (YUV 422 10-bit) avi encode bitrates at the different quality levels and they tally with those obtained with the vfw Cineform codec (v 9.2.1) from GoPro Studio (Quik).

But where did "Filmscan 3" come from ? Not that I'm complaining, but I was under the impression that "Filmscan 2" was the max quality level. I haven't done any quality metric tests (yet), but with the same source clip the bitrate of the 'Filmscan 3" encode was about 5% higher than that of the 'Filmscan 2" encode.

Edit:
I haven't done any quality metric tests (yet)....

With this one test clip (HD-AVC 1080/30p mp4) as source:

Bitrate (Mb/s) SSIM score (%)

Filmscan 3 346 98.28
Filmscan 2 327 98.09
Filmscan 1 239 96.43

And that's after converting the Cineform encodes back to YV12 for the AVISynth SSIM test.

[shekh]

From the sdk:


CFHD_ENCODING_QUALITY_FILMSCAN3, // overkill but useful for get higher data-rates from animation or extremely clean sources

[WorBry]

Thanks shekh.

[kolak]

FilmsScan3 is a "special" mode made for Disney for very clean sources. For typical or noisy source it may rise bitrate a lot for not much quality gain.

Cineform was made with idea of high-end RAW (or film scan) sources, not already pre-compressed ones. When you test it with h264 etc (or even already compressed ProRes) source it will tend to use relatively low bitrates even for FS2 mode. If you take some RAW source (RED, Arri) etc then it's a different story. It will nicely preserve quality over whole source.
Here is an example:

http://s32.postimg.org/u54dzr079/test.png

Great to see native Cinefrom in Vdub!

[WorBry]

Interesting. I knew 'Film Scan' per se was developed to accurately reproduce the grain characteristics in high grade film scans but had seen no mention of 'Film Scan 3' before:

https://cineform.zendesk.com/hc/en-us/articles/206528816-Understanding-CineForm-Quality-Settings

[kolak]

FilmScan3 or mode 6 in old cmd tools was rather a "secret" mode for typical users. You could turn it on with registry "hack" on PC.

[poisondeathray]

Ran a quick test comparing Cineform (YUV 422 10-bit) avi encode bitrates at the different quality levels and they tally with those obtained with the vfw Cineform codec (v 9.2.1) from GoPro Studio (Quik).


I'm not getting bit for bit identical the same result.... Very slightly different bitrate and differences on a 10bit422 source . Anyone else ?

[shekh]

Very slightly different bitrate and differences on a 10bit422 source . Anyone else ?

I am not sure it is possible to compare results bit-to-bit from current sdk vs vfw 9.2.1 implementation. Probably something was changed.

[poisondeathray]

There seem to be decoder differences as well , between ffmpeg and VFW (regardless if file was encoded with 9.2.1 or native vdfm sdk implementation)

Or how do you test "proper" 10-bit VFW decoding ? Maybe some testing methodology errors ? I'm testing in vapoursynth, with avisource and pixel_type="v210" . The differences don't look like a simple 10bit vs 8bit issue . And the difference pattern is quite different when testing v9.2.1 vs SDK


import vapoursynth as vs
core = vs.get_core()

c0 = core.ffms2.Source(source=r'F:\_Video Tests\QT Prores\cineform_filmscan2_native.avi')
a0 = core.avisource.AVISource(r'F:\_Video Tests\QT Prores\cineform_filmscan2_native.avi', pixel_type="v210")
#c0 = core.ffms2.Source(source=r'F:\_Video Tests\QT Prores\cineform_filmscan2_v9.2.1.avi')
#a0 = core.avisource.AVISource(r'F:\_Video Tests\QT Prores\cineform_filmscan2_v9.2.1.avi', pixel_type="v210")

d = core.std.MakeDiff(c0,a0)
da = core.std.Levels(d, min_in=511, max_in=513, gamma=1, min_out=0, max_out=1023, planes=[0,1,2])

da.set_output()