I along with a lot of other people have tried to test OMEMO and we’ve all spent some time trying to figure out what is going on with OMEMO on multiple clients and compatibility. I think I know at least one reason why clients on different platforms can’t talk to each other. If you look at my earlier blog post I mentioned fixing an issue with the wrong size initialization vector in my AES-GCM code. I was using 16 instead of the standard 12 bytes. I used conversations to test compatibility and this is an issue/bug in conversations. Conversations can read 16 or 12 but sends the incorrect 16. This is apparently because Chatsecure had/has a bug where it is using 16. This results in the following issues:
- If someone is implementing OMEMO and doesn’t know to replicate this bug, it will not work. It is not documented or written down anywhere.
- If you are using a library that provides GCM and does not use the 16 byte iv. For example, Apple’s CryptoKit in ios13. OMEMO will not work with conversations or chat secure.
At the moment Monal for iOS and Mac will be updated to make 12 byte ivs. It will still be able to receive 16 byte ivs. This means monal will be able to send/receive to Conversations. It may not be able to communicate with chat secure or other clients that have replicated the bug depending on how it was implemented. Monal for catalyst uses apple’s Cryptokit and will be able to send to conversations but not receive. I have also observed that Beagle sends 16 byte but will accept 12 bytes like Conversations.
In a nutshell, if you are testing OMEMO compatibility across the ecosystem with different clients and platforms, don’t do it. It’s a waste of time until this is resolved by every client.