I really like HCEnc and it works great on Linux via Wine with an avs script. :-)
For creating mkv's mkvmergeGUI is available for Linux, I use it on occasions and it's available via bunkus's PPA (if on Ubuntu or download otherwise) which is at:
http://www.bunkus.org/videotools/mkv...downloads.html