Workflow: XAVC-I -> Avisynth -> Resolve -> Avisynth

[chmars]

Hello,

Codecs, compression, conversion... Jungle.

I have a footage which is XAVC-L from an FS7 camera EDIT: from this project. (https://forum.doom9.org/showthread.php?t=182868)
I need to pass it through:

-one script in Avisynth,
-grade it in Davinci Resolve +17 Free,
-eventually remove dirt and sharpen in Avisynth
-deshake.

Heres are infos about the file:
ID : 2
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High 4:2:2@L4.2
Format settings : CABAC / 2 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 2 frames
Format settings, GOP : M=3, N=12
Format settings, wrapping mode : Frame
Codec ID : 0D01030102106001-0401020201316001
Duration : 22 s 560 ms
Bit rate mode : Variable
Bit rate : 52.5 Mb/s
Maximum bit rate : 50.0 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate : 50.000 FPS
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 10 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.507
Stream size : 141 MiB (92%)
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

Here is a sample (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/R0EN4VYFyWQUFev/download)

I am lost with formats and codecs.
Would you have an advice to keep 10bit and as max quality as possible through this chain?
I read the deshaker from VirtualDub should be preferred. Is that correct?

[Frank62]

First thoughts:

The bottlenecks will be

-Deshaker, because you will need RGB32 (REC 601). So no way to keep 10 Bits. Nevertheless I'd prefer Deshaker compared to all other ways.

-RemoveDirt: even worse, as far as I know, all motion Avisynth compensation techniques are based on YV12. Correct me if I'm wrong. If right you might use a technique to build a RemoveDirt-mask only and pass RGB32 through it.

----------

The order should be:

1.Resolve - Import no problem, grading in 10 Bits no problem
Export to something (quite) lossless, that avisynth can import, f. e. DNxHR

2. Inside Avisynth:
-Import f. e. with LWLibavVideosource
-Take care of interlacing (many possibilites and ways)
-Deshake by using Deshaker, progressive frames are neccessary for Deshaking
-RemoveDirt (better after Deshaking, motion compensation can work more exact on stable picture, can avoid a lot of artefacts)
-Sharpen if at all necessary (?)

[StainlessS]

2. Inside Avisynth:
-Import f. e. with LWLibavVideosource
-Take care of interlacing (many possibilites and ways)
-Deshake by using Deshaker, progressive frames are neccessary for Deshaking
-RemoveDirt (better after Deshaking, motion compensation can work more exact on stable picture, can avoid a lot of artefacts)
-Sharpen if at all necessary (?)
For chmars's project, there should be no interlacing.
Any shaking is only a pixel or two of jiggling, really. DePan might work ok.
If DPan dont work ok, then maybe could deshake on RGB8, convert back to YUV Planar, and do motion vectors on the de-shook'ed,
YUV, and apply the vectors to the original 10 bit clip, I think that might be do-able [although I aint done anything like that myself].
(I know/think that Pinterf made it possible to do anaysis on low bit depth, and apply vectors to higher bit depth).
EDIT: motion vectors, not quite sure how to do that, maybe by interleave of original YUV8 and de-shook YUV8,
analysis on that. Might be tricky, need somebody clever to figure that out.
EDIT: I betcha Zorr could do it, or maybe not :)

[zorr]

as far as I know, all motion Avisynth compensation techniques are based on YV12. Correct me if I'm wrong.

There's no such limitation anymore, MVTools2 accepts YUV, YUY2 and RGB 8-32 bit formats.

maybe could deshake on RGB8, convert back to YUV Planar, and do motion vectors on the de-shook'ed, YUV, and apply the vectors to the original 10 bit clip

Yes it could be
1) Deshake in RGB8, convert it into 10bit and save as a new clip
2) recreate Deshaked frames using MCompensate on the original 10bit frames

Or perhaps you could do the first pass of DeShake in VDub and the second pass in AviSynth. But I'm only aware of one plugin, wonkey_monkey's Deshaker3D (http://avisynth.nl/index.php/Deshaker3D) and it only supports 8bit RGB.

[kedautinh12]

There's no such limitation anymore, MVTools2 accepts YUV, YUY2 and RGB 8-32 bit formats.



Yes it could be
1) Deshake in RGB8, convert it into 10bit and save as a new clip
2) recreate Deshaked frames using MCompensate on the original 10bit frames

Or perhaps you could do the first pass of DeShake in VDub and the second pass in AviSynth. But I'm only aware of one plugin, wonkey_monkey's Deshaker3D (http://avisynth.nl/index.php/Deshaker3D) and it only supports 8bit RGB.

Do you have Deshaker3D x64??

[chmars]

Hi guys, thanks for interresting!

First thoughts:
...
1.Resolve - Import no problem, grading in 10 Bits no problem
...

Remember, I am using the free version. (Not (yet?) making money with that project).
ATM, the only 10 bit I can import in Resolve is prores.
I might have missed something. Have to try more.


Any shaking is only a pixel or two of jiggling, really
)
Would be proud but no.
Some pixel yes, for the gatewave, on good days.
But 8/s8mm footage is shaking a lot. Example below.
There's no such limitation anymore, MVTools2 accepts YUV, YUY2 and RGB 8-32 bit formats.



Yes it could be
1) Deshake in RGB8, convert it into 10bit and save as a new clip
2) recreate Deshaked frames using MCompensate on the original 10bit frames

Or perhaps you could do the first pass of DeShake in VDub and the second pass in AviSynth. But I'm only aware of one plugin, wonkey_monkey's Deshaker3D (http://avisynth.nl/index.php/Deshaker3D) and it only supports 8bit RGB.
Do you have Deshaker3D x64??
Haem, yes on my drive...like a car I wouldn't know how to drive correctly.

This is all above the sky over my head.
I am studying John's version of Fred's script. Trying to understand steps involved in deshake ans removedirt.

This prores video file for example (very soft because of Slog3):
Camera file -> Avisynth to remove useless BLUE_LED frames -> outputed to avi -> converted to prores 422 HQ for Resolve input:

https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/YaSBta5uGAnuwuP/download (https://nmldqjct.preview.infomaniak.website/files/index.php/s/uxERSPAyg2MeU3U/download)


How do I deshake and dedirt it?

Can we do better in Avisynth than in Resolve, stab wise? :D (Or should I stay with the easy way?)

.

[Arx1meD]

In my opinion, the best
Deshaker: Stab3 (https://github.com/realfinder/AVS-Stuff/raw/master/avs%202.5%20and%20up/StabilizationToolsPack.avsi), Stab_Light (https://forum.doom9.org/showthread.php?t=182532).
Dedirt: SpotLess (https://forum.doom9.org/showthread.php?t=181777), SpotRemover (https://forum.doom9.org/showthread.php?t=182407).

[StainlessS]

Would be proud but no.
Some pixel yes, for the gatewave, on good days.
But 8/s8mm footage is shaking a lot. Example below.

Oops, yes. I was only thinking about the contraption jiggle, forgot about the motion due to original camera motion.

[johnmeyer]

The movie you linked to, Blue_LED_Detect_3_0-64-trimed_prores422.mov, will not play on any software I have, and Mediainfo reports no audio and no video.

[chmars]

Hi John,
Strange, when I click on the pic, it downloads and opens directly in VLC, in example.
Maybe download problem?

File:Blue_LED_Detect_3_0-64-trimed_prores422.mov (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/gx5POZeXBxYBoF6/download)
md5:17ce4e179753404f437a4a998d4c1e03

[zorr]

Do you have Deshaker3D x64??

No, sorry. And I haven't even tried Deshaker3D. Perhaps wonkey_monkey is willing to develop / compile a 64 bit version and then add support for > 8bit color spaces... :)

[Stereodude]

Has Resolve fixed up their screwy chroma handling yet? (up to 16.2 it doesn't convert from 4:2:0 correctly)

[chmars]

Really I try hard, read, READFAGLOLZ, threads and pages.
But still, I can't really figure out how to handle my pixels.
Stuck.

To summ:

-I have this MXF file out of the camera, AVC High 422 Intra@L5.2.
-First step: script to remove the duplicate frames with Vdub2.

Then, save as?

When I simply try uncompressed avi, I can't then reopen it with my trim+crop script because P210 seems not to be handeled.
If I compress and re-encode, what should I choose not to lose information?
The original file is AVC, so I tried h264/mp4 10bit lossless but it is very slow (2.5 fps).

As I only have to remove frames at this point, I thought ffmpeg could "just" remove frames from the mxf file.
I found the trim parameter for it, but only in a "trim=n:n" way.
I didn't find how to remove frames from a text file list.

And I'll anyway have to find some solution for the file get in Resolve...

Well...tiny hint?

[poisondeathray]

To summ:

-I have this MXF file out of the camera, AVC High 422 Intra@L5.2.
-First step: script to remove the duplicate files with Vdub2.

Then, save as?

When I simply try uncompressed avi, I can't then reopen it with my trim+crop script because P210 seems not to be handeled.
If I compress and re-encode, what should I choose not to lose information?
The original file is AVC, so I tried h264/mp4 10bit lossless but it is very slow (2.5 fps).

As I only have to remove frames at this point, I thought ffmpeg could "just" remove frames from the mxf file.
I found the trim parameter for it, but only in a "trim=n:n" way.
I didn't find how to remove frames from a text file list.

And I'll anyway have to find some solution for the file get in Resolve...

Well...tiny hint?

Use v210. It's uncompressed 10bit422 in "packed" not "planar" arrangement . It's compatible in all professional programs - resolve, NLE's etc...

What are you doing "remove frames" ? Why not remove them in the script ?

It says "remove duplicate files with vdub2" - did you mean "frames" ?

[chmars]

Use v210. It's uncompressed 10bit422 in "packed" not "planar" arrangement . It's compatible in all professional programs - resolve, NLE's etc...
Thanks, lots!
Such a pain at my state of knolage.

What are you doing "remove frames" ? Why not remove them in the script ?
It says "remove duplicate files with vdub2" - did you mean "frames" ?
S..t! Yes, brain washed. absolutely, I meant "frames", post corrected.

[chmars]

Well, back here, after been helped a lot to excellently solve the workflow steps, now this is all well working:

-Shoot(8mm/S8 capture): 4k 10bit 422 Xavc-I
-Script 1: Remove unneeded frames to become frame accurate -> uncompressed AVI V210 (https://forum.doom9.org/showthread.php?t=182809)
-Script 2: Trim and crop -> uncompressed AVI V210
-Script 3: Remove Dirt, restore grain -> uncompressed AVI V210 (https://forum.doom9.org/showthread.php?t=182831)
-Grade, sometimes stabilize in Resolve -> final mp4

Steps 2 and 3 are separated as I may sometimes use the "dirty" version too, in Resolve as under-layer with transparency.

Realized cannot shoot with XAVC-L (much lighter) because in 10bit, only 2k is available.
Enough for S8, you say. Right in theory but as I have to crop a lot (ATM, but will be solved) when shooting 8mm, 4k ends in 2k.


Format : MXF
Format version : 1.3
Format profile : OP-1a
Format settings : Closed / Complete
File size : 12.0 GiB
Duration : 6 min 55 s
Overall bit rate : 248 Mb/s
Encoded date : 2021-05-29 17:03:47.000
Writing application : Sony Mem 2.00

Video
ID : 2
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High 4:2:2 Intra@L5.1
Format settings, CABAC : No
Format settings, GOP : N=1
Format settings, wrapping mode : Frame
Codec ID : 0D01030102106001-0401020201323001
Duration : 6 min 55 s
Bit rate : 239 Mb/s
Maximum bit rate : 250 Mb/s
Width : 3 840 pixels
Height : 2 160 pixels
Display aspect ratio : 16:9
Frame rate : 25.000 FPS
Color space : YUV
Chroma subsampling : 4:2:2
Bit depth : 10 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 1.151
Stream size : 11.5 GiB (96%)
Color range : Limited
Transfer characteristics : BT.709

This is all excellent but I realize files are huge. AVIs are +/- 3x bigger than source.

Question:

Knowing that sometimes, footages are very washed out and differences between colors/brightness very reduced, there is a risk of banding if I correctly understand, because differences are very thin between tones.

What would you convert These XAVC-I files into?
I mean, in your opinion, what would be a good compromise and still, readable with Resolve Free version?

Sample here. (https://nmldqjct.preview.infomaniak.website/shared_files/index.php/s/zWQ9PJjMsywYn0e/download)

[poisondeathray]

This is all excellent but I realize files are huge. AVIs are +/- 3x bigger than source.

Question:

Knowing that sometimes, footages are very washed out and differences between colors/brightness very reduced, there is a risk of banding if I correctly understand, because differences are very thin between tones.

What would you convert These XAVC-I files into?
I mean, in your opinion, what would be a good compromise and still, readable with Resolve Free version?


Are you asking about a compromised ( in terms of filesize and quality) intermediate , since v210 files are rather large for UHD ?

The commonly used ones are Prores, DNxHR

[chmars]

Thank you.
Yes, a compromise, trade-off not to loose these sometimes poor colors and levels in a very tight histogram.
Btw for others like me, if interested, I found back your in depth explanation:
Avisynth+ <--> Davinci Resolve 16. (https://forum.doom9.org/showthread.php?t=176877)

[chmars]

Thank you poisondeathray. Yes, a trade-off, compromise between size vs enough informations to hold sometimes shrinked histogram and poor colors.
I found back the thread where you deeply discussed this.
For others like me : Avisynth+ <--> Davinci Resolve 16. (https://forum.doom9.org/showthread.php?t=176877)
I don't know yet what's all about but with Prores max quality, the file came back from 3 x size to approx. the original file size.
I approximate it must not be too bad :--)