It's a problem on certain blu-ray discs. tsMuxer isn't reading correctly the playlist and the chapters get screwed up, as well as the end time of the movie. The fix for this is to open the original BD in BDEdit, go to Playlist TAB, choose the movie playlist and then click save chapters. This will save a list of chapters, the correct list and you paste it in tsmuxer, in Custom Chapters List.
As far as your second problem goes, that's because you didn't replace the original clpi and mpls with the edited ones.