It is easy, even if interoperability is enabled, do not send messages out of Signal. It would be your option. But other people with non military-grade privacy requirements could benefit of improved privacy when it sends messages to whatsapp users from signal app because signal app is foss and signal would enforce better security and privacy than whatsapp app. Signal would gain traction and it could reach more people willing to abandon Meta and corps.
So, Matrix (federated) and XMPP (federated) would also have “metadata leaks”. I imagine there would be metadata exchanged between federated servers and in addition the E2EE of XMPP and Matrix is not so good/modern as Signal’s. When Signal-Whatsapp interoperability is mentioned, all people is worried about metadata leaks but it seems that concern dissapears when federation of Matrix or XMPP is mentioned.
Apart from that and one very personal opinion, I always connected Matrix to IRC, I mean, it is used more for the groups functionality than for the person-to-person functionality. And IRC was never considered an Instant Messaging alternative. But this is a very personal feel.