Problem processing large video

[Cary Knoop]

I am trying to process a 4K video encoded ProRes 4:2:2 HQ.

Even when I try to simply open (commenting out the decimate) nothing seems to happen (no significant disk, CPU or memory activity):


from vapoursynth import core
import vapoursynth as vs

c = core.ffms2.Source('a:\video.mov')
#c = core.std.SelectEvery(c, cycle=3, offsets=[0,1])

c.set_output()


Running: Vapoursynth R45. Windows 10 on a Threadripper 1950X with 64GB of memory, source disk A:\ is an NVME drive.

[Cary Knoop]

Well, I just was proven to be impatient.
It's super slow though.

[Myrsloik]

Well, I just was proven to be impatient.
It's super slow though.

Your drive is super slow

[Cary Knoop]

Your drive is super slow
My A:\ is mapped to an NVME PNY CS2030 480GB drive.

Not really my idea of super slow:

https://cdn2.techadvisor.co.uk/cmsdata/reviews/3655600/pny-cs2030-review-results.jpg

Source: https://www.techadvisor.co.uk/review/solid-state-drives/pny-cs2030-review-3655600/

[VS_Fan]

You could try:

accessing your disk directly without mapping it;
download (https://crystalmark.info/en/download/)and run CrystalDiskMark on your disk

[asarian]

Well, I just was proven to be impatient.
It's super slow though.

You realize FFMS2 needs to create an index of the file first, right?! On a large 4k film. that could takes minutes even before actual processing starts.

[poisondeathray]

Another option is LSmash , which does not need to index ISO base media formats , such as MOV, MP4

[Cary Knoop]

Another option is LSmash , which does not need to index ISO base media formats , such as MOV, MP4
I tried that as well, but it was very slow as well.

Never had problems opening files but in this case, for a very large and low compressed video, it is definitely slow. And this was only a 12 minutes master, what if it is a 2-hour master.

The suggestion that it must be my hard drive I frankly find not very helpful.

[Cary Knoop]

You realize FFMS2 needs to create an index of the file first, right?! On a large 4k film. that could takes minutes even before actual processing starts.
I do, but it's very slow.

[poisondeathray]

I tried that as well, but it was very slow as well.


Slow before opening ? As if it were indexing ?

or slow seeking performance "after" you were "patient ?"

Did you use core.lsmas.LibavSMASHSource , which is for ISO base media formats ? - This doesn't index and should open immediately

(core.lsmas.LWLibavSource is for other formats, and indexes)

[Cary Knoop]

Slow before opening ? As if it were indexing ?

or slow seeking performance "after" you were "patient ?"

Did you use core.lsmas.LibavSMASHSource , which is for ISO base media formats ? - This doesn't index and should open immediately

(core.lsmas.LWLibavSource is for other formats, and indexes)
Ah, I tried core.lsmas.LWLibavSource() instead of core.lsmas.LibavSMASHSource()!

And indeed, once it is working, consecutive access is almost instantly.

Thanks!

[Selur]

When using the ffms2 I usually create the index file with ffmsindex (add the created index to the call) and then open the Vapoursynth script.
This doesn't really speed things up, but it allows to know what time is spend on creating the index. :)

[asarian]

Ah, I tried core.lsmas.LWLibavSource() instead of core.lsmas.LibavSMASHSource()!

And indeed, once it is working, consecutive access is almost instantly.

Thanks!

If you have a nVidia card, you could also give DGDecodeNV a try. Creating the .dgi file may take a while, but its frame-serving processing power is phenomenal.

[poisondeathray]

If you have a nVidia card, you could also give DGDecodeNV a try. Creating the .dgi file may take a while, but its frame-serving processing power is phenomenal.

Won't help for prores

[asarian]

Won't help for prores

My bad. Fully missed he was talking about ProRes material. :o

[ChaosKing]

Is mov in ffms2 frame accurate? You should test it.

[poisondeathray]

I mov in ffms2 frame accurate? You should test it.

I-frame formats, like prores, are usually frame accurate

long gop in mov, such as some xdcam variants, h264 are usually frame accurate too with both lsmash/ffms2 in my experience