Lossless workflow for CrowdRun sequence?

[qyot27]

I'm currently in the process of fetching the frames for CrowdRun from here, and in the PDF in the parent directory it specifies that the sequences are stored in 16bits per plane RGB.

I want to preserve the full information during the conversion process, with the only loss incurred being during the switch from RGB to YUV after it's been assembled into a proper video file.

What lossless video options are there to store this? 16bits per plane would result in 'RGB48', if that's the right term for it. I would guess the Q16 release of ImageMagick could be used to compress the frames themselves more easily so I can move it around (with PNG), but are there any readily-available video codecs that can handle that colorspace? If I do indeed have to sacrifice that during the conversion to video, I might as well also switch to YUV at that time.

Which brings me to another question: what would be the optimal YUV-based colorspace to preserve the most of the original information? A cursory look at FFV1 has 444P listed, for instance. I understand that there'd be an additional conversion to YV12 when I use the clip for fprofiling x264, but I'd prefer the archival copy to be as least touched if possible.

My confusion, therefore, is finding the 'right' (subjective as that might be) workflow for getting the images from the SGI files that are downloaded to the archival video copy with the least amount of error introduced while making the filesize more manageable. Unfortunately storing the frames as they are isn't an option.

[Manao]

First, you have to choose the colorspace you want and stand by it. That choice depends solely on what you plan to do with the archive. If you only plan to encode it afterward, then do not choose 4:4:4, it has barely any real life uses. I'd go for 4:2:2 8 or 10 bits. There are some consumer-level codecs that handle 4:2:2 10 bits (prores, avc intra, red ray...) while almost none support 4:4:4. However, 10 bits support is still hard to come by (the only lossless solution I know of which is uncompressed and more or less supported is V210, which means 1920x1080 would take 5.5 MB / frames), so in the end I guess I'd go for 4:2:2 8 bits.

Then I'd use a lossless codec that can handle that. I'd go for huffyuv, because it's fairly fast and common enough, but ffv1 would also do.

[qyot27]

My primary intention here is to store a copy that's as close to the original colorspace (or in the original colorspace, as the case may be) as possible, while getting at least reasonable size savings from an appropriate lossless format. That way, when 4:4:4 and 10 bit (or higher, including RGB colorspaces) do become viable, I can still work from the same 'master' copy. If I need to work in 4:2:2 or 4:2:0, in 8 or 10 bit, etc. I can just create a new downsampled file and use that when I need to.

As it's standing, though, using the Q16 version of ImageMagick to process the frames into appropriate 16-bit PNG files will only shave about 3GBs off the total size - I'll end up with around 21GBs instead of 24.

[akupenguin]

16-bit PNG is sad. You get better compression from 8-bit PNG + uncompressed lsbs (1.21x in this case).
I doubt you'll do better than uncompressed lsbs; paq8 only compresses them by 0.2%. Which is also evidence for them being pure noise, so 10bit is probably enough.

btw, FFV1 supports up to 16bit depth, but that part of the libavcodec API is pretty new so it may not have wide application support.

[qyot27]

Thanks all.

Originally Posted by akupenguin

btw, FFV1 supports up to 16bit depth, but that part of the libavcodec API is pretty new so it may not have wide application support.

That'd be this commit from April 24th, right? I remember doing some compiles of ffmpeg around or after that date when I was looking at the MLP encoding patch, but I can always grab it from SVN or git again (actually, I probably should do that anyway, as I'm not altogether sure how recent the builds I'm using are, especially the one in Ubuntu).

[Dark Shikari]

Originally Posted by akupenguin

16-bit PNG is sad. You get better compression from 8-bit PNG + uncompressed lsbs (1.21x in this case).
I doubt you'll do better than uncompressed lsbs; paq8 only compresses them by 0.2%. Which is also evidence for them being pure noise, so 10bit is probably enough.

btw, FFV1 supports up to 16bit depth, but that part of the libavcodec API is pretty new so it may not have wide application support.

Does the context model in FFV1 really work for that high a bit depth?

[akupenguin]

No, it looks like it was only tuned up to 10bit. But that far it should use just about the same contexts as 8bit would, which lets the entropy coder not-compress lsbs if needed.

[Biggiesized]

Does anyone have frames 07602 through 07610? Every mirror which I've searched doesn't seem to have them. You can tell something's wrong when those particular frames were modified at a later date according to the FTP. Thus I'm 9 frames short of all 500.

EDIT: Wow, nevermind. I never knew Xiph mirrored them too. Their copy is intact:

http://media.xiph.org/svt/2160p50_Cg...TER_SVTdec05_/

[qyot27]

As an update, I've reached an impasse. I can't seem to figure out how to properly convert RGB48 to 16-bit 4:4:4, and my tests with ffmpeg writing uncompressed RGB48 can't even properly decode with ffmpeg or ffplay, and results in a massively distorted image (it encodes with rgb48le, but decodes with bgr24le, and this was running with an up-to-date SVN build, which at that time was 19816, I believe).

It seems I may have to hold out until AviSynth 2.6 reaches a state where those higher colorspaces are usable, although the highest one mentioned in the thread for the newest alpha is YV24/4:4:4 8bit, not YV48/4:4:4 16bit. Even then, conversions to those higher colorspaces is currently broken.

But I fully acknowledge that I could be overlooking something.

[Biggiesized]

qyot27, how did you encode the sgi16 sequence into RGB48? What container are you using? I think the best I'll get out of this is 10-bit 4:2:2 with After Effects.

[qyot27]

Originally Posted by Biggiesized

qyot27, how did you encode the sgi16 sequence into RGB48? What container are you using? I think the best I'll get out of this is 10-bit 4:2:2 with After Effects.

As I said, it doesn't decode correctly, but I convert the SGI(s) into TIFF with ImageMagick, and then give the TIFF(s) to ffmpeg. Container is AVI.

convert 07111.sgi -compress none -endian LSB 07111.tiff
ffmpeg -i 07111.tiff -vcodec rawvideo test-rgb48.avi

(for some reason using ffmpeg's sequence import syntax didn't seem to work for me; kept coming up with warnings about possible corruption in the files)

If the Q16 version of ImageMagick is the one being used, the resulting TIFF files will be 16 bits per channel, as will the colorspace ffmpeg automatically chooses - i.e. rgb48le. If you use the Q8 version of ImageMagick, then any of the conversions (to TIFF, PNG, etc.) will be downsampled to 8 bits per channel, which would obviously work correctly but doesn't preserve the rest of the original data.

[Biggiesized]

Where do I get the ffmpeg tools? Perhaps ImageMagick is improperly rendering those SGI files?

I can try rendering to TIFF via After Effects and then try to build a container like you did in ffmpeg once I get it.

[Boolsheet]

Apparently there's still an issue with 16 bit colorspaces and ffv1. I'm trying to build a patched version but if the decoder can't handle it we have to wait until someone implements it correctly.

I'm not sure if FFmpeg converts the colorspaces from RGB to YUV correctly in 16 bit. Oh well, something more to do manually.


Edit: It actually worked. I forced ffv1 to use 16 bits and the output is decodable with ffmpeg.
ffv1 compressed the raw version from 24'883'200'000 to 23'201'531'146 bytes.

Here is the 'special' ffmpeg executable. It's ffmpeg only, no other libraries and includes a really ugly hack, so only use it if everything else fails.
I used something like this to convert it:
ffmpeg.exe -f rawvideo -vcodec rawvideo -s 3840x2160 -pix_fmt rgb48be -r 50 -i c:\Crowdrun_rgb48.raw -vcodec ffv1 -strict -2 -pix_fmt yuv444p16le c:\Crowdrun_yuv444.avi

FFmpeg expects an interleaved upside down rawstream. The channels in the sgi are planar and the first byte of the image data is the upper left pixel.

[qyot27]

Originally Posted by Boolsheet

Apparently there's still an issue with 16 bit colorspaces and ffv1. I'm trying to build a patched version but if the decoder can't handle it we have to wait until someone implements it correctly.

That issue was fixed a week ago (here) - there's no need to apply the yuv16 patch anymore.

I'm not sure if FFmpeg converts the colorspaces from RGB to YUV correctly in 16 bit. Oh well, something more to do manually.

From what I read, ffmpeg doesn't convert colorspaces at all, but maybe there is a trick to it like the command you posted.

Originally Posted by Biggiesized

Where do I get the ffmpeg tools? Perhaps ImageMagick is improperly rendering those SGI files?

Well, ffmpeg is in MeGUI's update pipeline, but for a build new enough for this you'd have to compile it yourself. I wouldn't know how well the SGI->TIFF conversion gets done, I just know that the resulting TIFFs are perfectly viewable in Photoshop and the Windows Fax/Image Viewer. Restricting myself to bgr24le (by using Q8 to render 8-bit files first) is handled fine by all parties involved.

[Boolsheet]

Originally Posted by qyot27

That issue was fixed a week ago (here) - there's no need to apply the yuv16 patch anymore.

Huh, doesn't work for me with a self compiled unpatched build and the auto builds anyway.

From what I read, ffmpeg doesn't convert colorspaces at all, but maybe there is a trick to it like the command you posted.

Yes yes, it has to. PNG only supports rgb, there would be no way to save a YUV video frame to a PNG.
-pix_fmt does the job, tell ffmpeg what colorspace your raw source is by putting it in front of the -i switch and one behind it for the encoder colorspace.

[qyot27]

Edit: It actually worked. I forced ffv1 to use 16 bits and the output is decodable with ffmpeg.
ffv1 compressed the raw version from 24'883'200'000 to 23'201'531'146 bytes.

Here is the 'special' ffmpeg executable. It's ffmpeg only, no other libraries and includes a really ugly hack, so only use it if everything else fails.
I used something like this to convert it:
ffmpeg.exe -f rawvideo -vcodec rawvideo -s 3840x2160 -pix_fmt rgb48be -r 50 -i c:\Crowdrun_rgb48.raw -vcodec ffv1 -strict -2 -pix_fmt yuv444p16le c:\Crowdrun_yuv444.avi

I don't know what's going on. None of the suggested has worked. Not the special patched build posted earlier, with or without the accompanying commandline, nor my own builds (which were built with gcc 4.4.1, not 3.4.5) with that command, or the reverse - taking video encoded in the patched build and attempting to decompress it in the patched build, my own, or even ffplay - all three outright crash. Also, the difference in size between the rgb48 and (raw or ffv1, I can't remember) yuv444 files is miniscule - less than 20KB. 48,606 MB vs. 48,591 MB or something like that.

[Boolsheet]

FFmpeg threw errors and warnings at me while testing but it never crashed. Does it on other systems happen too?

yuv444p16le still has 3 channels and uses 16 bit for each so the size of the raw version shouldn't change at all (except for container overhead if you use one).
By the way I'm an idiot.. forgot the set the arithmetic coder (-coder ac) for ffv1. Now it compresses it to 7'992'751'690 bytes but takes around 4 spf for me. Decoding still works, ffplay doesn't crash.
There's also the -context switch for the context model, but I don't think it changes much if it's not optimized for 16 bit.

[Reimar]

ffplay indeed seems to have some issues with these files, though MPlayer can handle them - it seems related to colourspace conversion, probably libswscale has some issues with such high resolutions (it originally only handled up to 2048x...).
With this patch: http://article.gmane.org/gmane.comp....eg.devel/96725 ffmpeg can read the .sgi files directly.
[EDIT] it should also be possible to use that same patch with latest MPlayer/mencoder to make that support those files directly via the mf:// syntax
[EDIT2] Patch is integrated, latest FFmpeg and MPlayer/Mencoder should now handle those files out-of-the-box.

[aloshious]

I am doing project in video compression... for that i need some red colour dominating video sequence... can any one help me with this