I did a lot of reading but it was a few months ago, there was lots of talk, but the majority basically says as long as you use HDMI/DP then just by design it will always send a video signal to support the audio.
There were threads talking about splitters, but in the end I think it is something that has to be endured just by design I am afraid.
"There is no way to transmit only audio data via HDMI.
The HDMI white paper (page 10) notes that the signal is sent in HDMI packets consisting of both audio and video. Despite all the pins, there are no separate audio and video signals, so there are also no HDMI A/V splitter cables.
https://www.hdmi.org/pdf/whitepaper/...aperv73(2).pdf