The Flaw in ISIS's Favorite Messaging App

In some corners of Washington, D.C., cryptography is becoming a dirty word. Since the rise of the Islamic State, hardly a day goes by that politicians don’t raise the specter of a terrorist attack planned on encrypted messaging platforms. In December’s Democratic debate, viewers heard Hillary Clinton call for a “Manhattan-like project” to ensure that law enforcement would always be able to implement a wiretap. For more and more lawmakers, encryption is that perfect, pitch-black night in which radicalized things go bump.

But according to security experts, that attitude attributes much too much power to computer scientists. “I appreciate their faith in my field,” the researcher Matt Blaze recently told Politico, “but I don’t share it.” In fact, as a new paper on the secure-messaging application Telegram reminds readers, it’s easier to market impenetrable encryption than it is to implement it.

Backed by Pavel Durov, the Russian founder of the social network VK, Telegram allows its users to exchange messages one-on-one, in groups, or on large public forums called channels. The platform is best known in policy circles for its popularity with the Islamic State, though it moved recently to disband a number of channels hosting jihadist material.

Telegram is also known for its security. The app’s encrypted Secret Chat function—“meant for people who want more secrecy than the average fella,” according to the organization’s website—earned a seven out of seven on the Electronic Frontier Foundation’s Secure Messaging Scorecard. And in February, Telegram announced that a $300,000 competition to decipher the Secret Chats would end without a winner.

For years now, though, cryptographers have been warning that the application has strange guts. Telegram uses a custom protocol, MTProto, to secure its messages, a decision that breaks a cardinal rule of cryptography: Don’t try to design your own, not if you can use an established approach instead. In 2014, for example, when Telegram’s more popular competitor WhatsApp decided to upgrade its security, it did so by working with Open Whisper Systems to implement that organization’s well-regarded TextSecure protocol.

In that light, Telegram’s choice to pursue an original design stands out. “They came up with something totally new, and a little weird, and mysterious,” the Johns Hopkins University professor Matthew Green said. “It’s like coming up and finding a submarine where the doors are made out of Saran Wrap. I guess if you use enough Saran Wrap you could build a pretty secure submarine; it doesn’t mean that it’s going to sink. But it does mean it’s not something I would want to trust with my life.”

The more novel an approach to encryption is, the less time researchers have had to spot vulnerabilities; older algorithms wear the scars of past decryption attempts like armor. What’s more, the more unusual or non-standard an encryption scheme is, the more difficult it can be to identify points of failure that are hidden in a tangle of unconventional choices. As a result, Telegram’s decision to “roll its own crypto” had long drawn scrutiny. Now, findings published in December give critics fresh reason for suspicion.

Claudio Orlandi is an associate professor at Denmark’s Aarhus University, where, along with the graduate student Jakob Jakobsen, he recently audited Telegram’s source code. While the application claims on its website that MTProto helps “achieve reliability on weak mobile connections as well as speed when dealing with large files,” Jakobsen was skeptical that the security tradeoff made any sense. “There should be lots of provably secure approaches that work just as nicely on a smartphone,” he said in a phone interview. And when the pair went looking for a flaw in the protocol, they found one.

In particular, Orlandi and Jakobsen found that MTProto lacks a property called “indistinguishability under chosen ciphertext attack,” or IND-CCA. That standard is meant to imply that an attacker trying to decipher a message can’t wring information out of the encrypted version. Formally, it can be tested against a sort of game. If an adversary asks Telegram to encrypt one of two messages, and receives the encrypted version in return, it shouldn’t be possible to guess which message was enciphered, at least not with better odds than chance—even if you give the adversary access to a “decryption oracle” that can crack any message secured by the same algorithm.

By design it’s a heavy burden of proof, and on a narrow, technical level, Telegram fails the test. In MTProto, it turns out that an attacker could fiddle with the encrypted message to create a new version, one that would look different on its face but that would still decrypt to the same underlying text. Under the somewhat literal-minded rules of the IND-CCA game, the adversary could then feed this doctored message to its handy oracle, since it isn’t technically the encrypted text that it was originally challenged to identify. That probably sounds like a groan-worthy loophole, and it doesn’t map onto a real-life vulnerability. At least, not yet. “Most attacks start as theoretical,” Orlandi noted, “and are dismissed by developers as irrelevant.”

The Telegram team told me that they were aware of the IND-CCA issue before this paper, but that the question was academic. “A postal worker can write ‘Haha’ (using invisible ink!) on the outside of a sealed package that he delivers to you,” the app’s FAQs currently read. “It doesn‘t stop the package from being delivered, it doesn’t allow them to change the contents of the package, and it doesn't allow them to see what was inside.” Jakobsen acknowledged that this was a fair analogy for the flaw he and Orlandi found. This October, a little over a month after the two first alerted Telegram to their finding, the organization answered that a future patch would address the concern they had raised.

The dispute has less to do with this specific attack than what cryptographers say it signals, a deeper shakiness in Telegram’s home-brewed approach to encryption. “It’s a theoretical flaw,” Nicholas Weaver, a senior researcher at the University of California, Berkeley’s International Computer Science Institute, said in an email, “but it’s a huge red flag.” In other words, because custom crypto is presumptively insecure, the odds of MTProto having exactly one flaw are slim. What’s more, this insecurity is as much a signal for attackers as it is for users, an invitation to try widening a known theoretical hole into a practical vulnerability.

“Everyone agrees on a simple principle,” Orlandi said. “Attacks only get better.”

That debate over Telegram highlights just how differently security researchers and law-enforcement officials view the cryptographic balance of power. As the rules of the IND-CCA game highlight, cryptographers often test their protocols against what they assume will be a nearly-omnipotent adversary, one with what amounts to a Magic 8-Ball. They do so because they know who they’re up against. A finite number of men and women are trying to guard sprawling mathematical terrain against the predation of nation-states. Their opponents have immense financial and intellectual capital; one high-profile paper this fall alleged that the NSA may well have spent hundreds of millions of dollars—and an entire year—cracking a single prime number.

The impenetrable encryption that lawmakers fear? Privacy advocates should be so lucky.

Setting aside Telegram’s own speed and reliability explanation, then, none of the experts I spoke to could guess why the organization chose MTProto instead of an established protocol. Green wondered, “Why would somebody spend a lot of effort building something that’s mediocre?” Effort is a scarce resource, and “don’t roll your own crypto” is a heuristic that conserves it. By the same token, exercises like Orlandi and Jakobsen’s might be theoretical, but they help speed the discovery of the next fatal flaw. “Even the smartest cryptographers are not good at finding these attacks just by poking at something. Proofs are a rigorous structure that helps us to avoid outsmarting ourselves,” Green said.

Because what looks like steel could be, from another angle, Saran Wrap.

Meanwhile, Telegram’s Markus Ra said in an email, “This matter will be addressed, among other things, in a future revision of the protocol—for aesthetic reasons.” With an appropriate overhaul, it’s imaginable that Telegram could become the digital fortress that many policymakers, many in the media, and many of its users seem to think it already is.

The Islamic State, for its part, is moving away from the application. Their preference? In some cases, custom crypto. Green laughed when I questioned the wisdom of the move. “I have to say that that doesn’t fill me with confidence,” he said. “But they seem to like it.”