[nm]

Originally Posted by skyjuice

I would like to compare the effectiveness of the in-built deblocking filter of x264.

Are you going to replace H.264 in-loop deblocking with your own (in which case you need to tweak a decoder too)?

Originally Posted by skyjuice

I have done a few readings where there are these alpha (strength) and beta (threshold) parameters.

Yep, they can be set on the command line with --deblock alpha:beta

Originally Posted by skyjuice

I tried encoding a set of frames with the filter turned-on (default) and turned-off (--no-deblock) and hardly notice any difference.

Encode at low bitrates for stronger effect. Try with --crf 30

Note that in-loop deblocking is mainly supposed to reduce blocking that is produced during encoding, not blocking that is in the source video! x264 automatically lowers deblocking strength when low quantizers (high bitrate) are used for encoding.

Originally Posted by skyjuice

If I want to disable the deblocking filter (other than using the command line), can I erase them in the source code (e.g x264.c and encoder.c)?

You can remove most of the deblocking code, which is also spread around a few other files. Or you could just leave it there and add your own code paths where needed.