continue after broken encoding

[Terka]

using command line to encode to x264.
but sometimes happens that i need to break it or the computer gets off. is it possible continue on place where the break was done?

[[P]ako]

As far as I know, you will have to start again. If you are doing a two-pass encoding, then you can stop the process right after the 1st pass is done, then continue with the 2nd pass later.

[Terka]

its a pity. maybee explore the encoded part, count the nr of frames (N) and trim in avisynth-encode only from the frame N+1. then join.

[LoRd_MuldeR]

its a pity. maybee explore the encoded part, count the nr of frames (N) and trim in avisynth-encode only from the frame N+1. then join.

This works okay for CRF mode, but it won't work with 2-Pass mode. Even if you manage to trim the stats file accordingly, the results will still be suboptimal.

That's because both segments would be encoded independently and thus rate-control restricted to the current segment...

[nm]

This works okay for CRF mode, but it won't work with 2-Pass mode. Even if you manage to trim the stats file accordingly, the results will still be suboptimal.

That's because both segments would be encoded independently and thus rate-control restricted to the current segment...
But x264 had already determined good sizes for both segments and that information is available.

[stax76]

but sometimes happens that i need to break it or the computer gets off

Why would this happen? It's possible to pause x264 or to hibernate the PC. With 4 cores a HD encode should be done in 4 hours I think so starting over should not be dramatic.

[LoRd_MuldeR]

But x264 had already determined good sizes for both segments and that information is available.

Well, even in the second pass of a 2-Pass encode x264 constantly (i.e. after each encoded frame) updates the QP's of future frames to compensate for mispredictions:

Now start encoding. After each frame, update future QPs to compensate for mispredictions in size. If the 2nd pass is consistently off from the predicted size (usually because we use slower compression options than the 1st pass), then we multiply all future frames' qscales by the reciprocal of the error. Additionally, there is a short-term compensation to prevent us from deviating too far from the desired size near the beginning (when we don't have much data for the global compensation) and near the end (when global doesn't have time to react).

This obviously won't work if you start the second pass in the middle of the file, because all information about the segment that was already encodes is lost.

[aegisofrime]

This works okay for CRF mode, but it won't work with 2-Pass mode. Even if you manage to trim the stats file accordingly, the results will still be suboptimal.

That's because both segments would be encoded independently and thus rate-control restricted to the current segment...

Actually I have done that before and the results look perfectly fine to me...

Suboptimal to a computer maybe, but to my human eye it's ok.

Sometimes you just can't afford to do the encode all over again especially when the video in question is a TGMC'ed 1080p video that takes 7 days to encode :p

[LoRd_MuldeR]

Actually I have done that before and the results look perfectly fine to me...

Suboptimal to a computer maybe, but to my human eye it's ok.

Sometimes you just can't afford to do the encode all over again especially when the video in question is a TGMC'ed 1080p video that takes 7 days to encode :p

Sure, after a certain number of frames, it should recover. So the longer the video you are encoding, the smaller the problem should be, I guess. But it still may be a more serious problem for shorter clips. Also if you didn't see the difference, that's probably because your bitrate simply was high enough to hide the quality loss...

[aegisofrime]

Sure, after a certain number of frames, it should recover. So the longer the video you are encoding, the smaller the problem should be, I guess. But it still may be a more serious problem for shorter clips. Also if you didn't see the difference, that's probably because your bitrate simply was high enough to hide the quality loss...

That sounds like a reasonable explanation. Still, between a slight (and to me, imperceptible) quality loss and starting a seven day encode I will rather take the quality hit :)

[Terka]

ive done it 1pass quality based:
--preset slow --level 4.1 --crf 22.5 --tune film
it will be great if a such new feature could be implemented, - encodings resume.
similar like resuming a download of a file.
you can then start the encoding when you work with the computer, when you shut it down, encoding stops. next time you start
the pc, you resume encodings...

[nurbs]

You can already resume an encode if you hibernate/sleep the computer instead of shutting it down.

[bnshrdr]

Is the only thing holding x264 back from being capable of 2pass recovery the fact that all memory used by x264 is lost?

If so, how difficult would it be for someone to write a patch to dump the memory to a file and throw an extra switch onto the x264 command line like "--resume memory.bin"?
By no means am I volunteering but this is just hypothetical thinking.

[LoRd_MuldeR]

Is the only thing holding x264 back from being capable of 2pass recovery the fact that all memory used by x264 is lost?

Yes, the problem is that the current state of the encoder is lost when you abort the encode prematurely.

And the developers said that adding functions to dump the current state to a file plus adding functions to restore the encoder's state from that file would be a lot of work!

Most important: If the encoder crashes, you have no chance to dump the state anyway. While for a planned break you can use hibernation. So...

[Maccara]

Most important: If the encoder crashes, you have no chance to dump the state anyway. While for a planned break you can use hibernation. So...

True. It would be nice to have something like "checkpoint every 1000 encoded frames" or similar, in case of crashes/power outage/etc.

But this is moot anyway as you also reminded us of:

And the developers said that adding functions to dump the current state to a file plus adding functions to restore the encoder's state from that file would be a lot of work!

(I don't remember the details of that discussion anymore, but I do remember there were some significant barriers to why it wouldn't be simple to implement)

[J_Darnley]

By no means am I volunteering
If you aren't then who's going to write it?