Its the "quite some playing around" I want to avoid
But I think you're right, feeding the audio to a separate tool then remuxing it seems like the way to properly encode audio for webm until someone produces an opensource or freeware tool that works properly.