change frame rates between passes?

[Blue_MiSfit]

Hey Folks,

I haven't done this in a long time, but I'm trying to dust off the cobwebs.

I'm encoding some stuff for an adaptive streaming demo. The highest quality layers will be 60p and the lowest quality layers will be 30p. This decimation is quite common, but i'm not sure how to do it with x264.

--vf select_every:2,0 works, but since I'm trying to encode everything with GOP alignment using a single first pass and multiple output passes this breaks the process:


C:\Users\dprestegard\Desktop>x264 bbb_sunflower_1080p_60fps_normal.mp4 --index b
bb.idx --force-cfr --fps 30 --preset slower --tune film --bitrate 3000 --vbv-max
rate 3000 --vbv-bufsize 6000 --keyint 60 --stats ".stats" --profile main --level
31 --vf resize:1280,720 select_every:2,0 --pass 3 --output BBB_3000.h264
ffms [info]: 1920x1080p 1:1 @ 60062/1001 fps (cfr)
resize [info]: resizing to 1280x720
x264 [info]: using SAR=1/1
x264 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x264 [error]: timebase mismatch with 1st pass (1/30 vs 1/60)
x264 [error]: x264_encoder_open failed


What am I missing?

[kypec]

What am I missing?
I guess you're missing half of your original (1st pass) frames to be encoded since you've instructed x264 pre-processor to skip every other frame.

[kabelbrand]

The highest quality layers will be 60p and the lowest quality layers will be 30p.

I guess you would have to use fixed GOP durations and separate encoding passes for this.

[kabelbrand]

I guess you would have to use fixed GOP durations and separate encoding passes for this.

Just for kicks: You could use ffmpeg to do each pass in a single run. For 2-pass you still need separate logs for each frame rate.
ffmpeg -r 60 -vsync drop -i bbb_sunflower_1080p_60fps_normal.mp4 -codec:v libx264 -b:v 5000k -g 120 -sc_threshold 0 -an -y BBB_5000.h264 -vf select=not(mod(n\,2)) -codec:v libx264 -b:v 3000k -g 60 -sc_threshold 0 -x264opts fps=30 -an -y BBB_3000.h264

[benwaggoner]

The best solution for this would probably be to:

Encode your reference 30p stream
Parse the .stats output and generate a new .qpfile that where you set an IDR frame for every IDR frame from the source
Make a 60p qpfile that doubles all the frame numbers
Use that qpfile in your 60p encodes


I don't know that there would be any way to reuse the .stats and .mbtree data when inserting or removing frames like this. But the above should result in all your IDR frames having the same timestamps.

Share a link if you get it working! I'm not sure how well real-world players handle changing frame rate mid playback. In theory it should work.

But as we all know: "in theory there's no difference between theory and practice, but in practice there is."

[sneaker_ger]

Wouldn't it be better to use the 60p stream as reference since scene changes are not limited to even frames? Then half the frame numbers for 30p (round up).

/edit: or could this produce problems when switching streams? Don't really know how it works...

[benwaggoner]

Wouldn't it be better to use the 60p stream as reference since scene changes are not limited to even frames? Then half the frame numbers for 30p (round up).

/edit: or could this produce problems when switching streams? Don't really know how it works...
Half the frames in the 60p file wouldn't exist in the 30p version, so if those got picked as IDRs, you wouldn't have GOP alignment. A clever client might be able to work around that, but never bet on a clever client unless you control its source code.

If you pick IDR frames based on 30p, you now that all those frames will have frames at the same exact timestamps in 60p.

[foxyshadis]

If you're going to do that, you might also want to pre-process your input to just remove scene changes from the odd frames, so you don't get kicked by two I-frames in a row. Drop & dup a neighbor frame if one occurs.

Technically, you can turn them into B-frames whose blocks all refer forward (so it becomes open GOP, although it's only "open" in the forward direction), but I don't know how you'd explain that to x264. You'd probably have to enable regular open GOP, add them to the qpfile and hope for the best.

[benwaggoner]

If you're going to do that, you might also want to pre-process your input to just remove scene changes from the odd frames, so you don't get kicked by two I-frames in a row. Drop & dup a neighbor frame if one occurs.
Rate control on the second pass would know which are forced IDR frames, and then not do a duplicate unless there's an edit aligned on a different frame than in the 30p or something. And even in those cases the next frame 30p frame would have the correct image.

Technically, you can turn them into B-frames whose blocks all refer forward (so it becomes open GOP, although it's only "open" in the forward direction), but I don't know how you'd explain that to x264. You'd probably have to enable regular open GOP, add them to the qpfile and hope for the best.
I suppose the 60p pass could be set to Open GOP because the forced IDR frames for the first pass would force Closed GOPs at the proper interval anyway. That would allow a forward-predicted b-frame in the 60p stream as you describe above

I don't know that any adaptive streaming packagers would fail on just seeing an Open GOP stream as long as there were periodic aligned IDR frames, but that is a possible (but easy to test) risk.

Blue_MiSfit, keep us posted on the results of your experiment, please!

[Blue_MiSfit]

I was on a tight timeline so I just used our Elemental boxes to do this. They generated GOP aligned MP4s at 60p and 30p that dynamically packetize into HLS nicely using Wowza. Playback is absolutely flawless on iOS.

It _works_ on Android (tablet, phone, and FireTV STB) but with some issues (~1 second pause when shifting from 30p to 60p, and seeking issues).