CRF is no rate distortion algorithm but a bitrate control mode.
CRF lets you specify a constant picture QP for the whole sequence like constant QP, but in float.
This picture QP is then scaled by the 'complexity' of the local pictures if I remember correctly, someone correct me here please.
the mbtree algorithm and aq mode, which work on a macroblock level, then add to this picture QP to derive a QP for a macroblock.
I think when mbtree is enabled the complexity of a picture is always 1.0, ie no scale, but again I am not so sure about this one.
So I suggest to look at the mbtree, aq mode and bitrate control code.
There is a paper on mbtree here:
https://huyunf.github.io/blogs/2017/...tree paper.pdf