Current Issue
This Month's Print Issue

Follow Fast Company

We’ll come to you.

7 minute read

Inside The Making Of Allo, Google’s AI-Powered Messaging App

Google's plan for beating Apple and Facebook in the messaging wars.

  • <p>Google launched its AI-assisted messaging app, Allo, today.</p>
  • <p>Allo creates smart replies that are carefully calibrated to both the content and context of your conversation.</p>
  • <p>If someone sends you a picture of them skydiving, you can send a pre-selected response, like “So brave,” “How fun,” and “So exciting!”</p>
  • <p>The app is designed to compete with messaging apps from Facebook and Apple.</p>
  • <p>It's an example of how Google is implementing the machine learning technology in which it has invested heavily.</p>
  • 01 /05

    Google launched its AI-assisted messaging app, Allo, today.

  • 02 /05

    Allo creates smart replies that are carefully calibrated to both the content and context of your conversation.

  • 03 /05

    If someone sends you a picture of them skydiving, you can send a pre-selected response, like “So brave,” “How fun,” and “So exciting!”

  • 04 /05

    The app is designed to compete with messaging apps from Facebook and Apple.

  • 05 /05

    It's an example of how Google is implementing the machine learning technology in which it has invested heavily.

Let the Great Messaging War of 2017 begin. Today, Google is launching Allo, a revamped messaging app meant to compete with the likes of Apple Messages and Facebook Messenger—two products that in recent months have steadily unveiled a slew of new features. Google thinks it could leapfrog them both, thanks to its most valuable assets: a wealth of data about what we do and search for online, and the billions of dollars it has invested in machine learning.

In Google Allo, those advantages manifest themselves in a couple simple but tantalizing ways. When you message people, Allo creates smart replies akin to those found in Inbox—but those smart replies are carefully calibrated to both the content and context of your conversation. So, for example, if someone sends you a picture of them skydiving, you can immediately tap on a series of well-tuned responses: "So brave," "How fun," and "So exciting!"

But perhaps even more interesting is that with Allo, you can now chat with Google directly, via a feature called Google Assistant. "We’ve been having conversations with Google ever since olden days, when you typed something in a box and then maybe typed more into the box to get better answers," explains Scott Huffman, Google’s VP of engineering for Allo. As Huffman points out, Google has invested heavily in making that conversation more natural over time—so that now instead of typing "weather NYC," you can do complex, locally tuned queries such as "What’s the weather going to be like next Tuesday?" "The search engine and the messaging thread are the two most understood interfaces ever created," he says. "If you’re having a conversation, why wouldn’t you have a UI that’s really a conversation?"

The intent, according to Huffman, is to make Google Assistant into a true conversational partner—and not just one that you talk with one-on-one. In Google Allo, if you’re chatting with friends about dinner, you can simply type "@google" and the assistant chimes in with restaurant recommendations nearby. It’s not magic yet—there are too many instances where it doesn't simply spit out exactly what you wanted—but the path toward magic isn’t hard to imagine.

Can Messaging Fix All Our App Woes?

Google doesn’t talk about it much, but almost all of its products begin with intensive ethnographic research. The research for Allo began in Indonesia and India. While visiting the homes of dozens of people, Google's staffers observed messaging as it's used in the wild, by those who have never had desktop computers. They saw, more than anything else, how much messaging had come to dominate every facet of mobile computing.

There was, for example, a middle-aged family man in Indonesia who ran a second business making potato chips and conducted all his transactions via messaging; an Indonesian fashion designer who procured fabrics, inspected samples, and sent designs all via SMS; and an Indian woman who got all her health news via group chat. To the Googlers, all those uses seemed both far removed from the way we use messaging in the West—but also futuristic, in the sense that those Indonesians and Indians gave a hint of all the services that you might one day build atop a messaging app.

The fact that Google, Facebook, and Apple are all focusing so heavily on messaging isn’t a fad. Rather, it’s a logical evolution, in a world that doesn't assume that apps are king: Messaging consumes the greater share of the time we spend on our devices, so why not make messaging into the portal for all the things we do on our phone? "Messaging is so woven into our lives that you don’t even think about how many things you do with it," says Jason Cornwell, who lead the UX for Allo. "As an industry, we’ve barely scratched the surface of what messaging can do."

The Chinese messaging app WeChat is a taste of that potential. You can chat with millions of businesses and pay them without ever leaving the app; you can make restaurant reservations and hail a cab. It is less of a messaging app than a portal for your entire digital life. Chat hasn't evolved that way in the West, partly because millions of us have come of age with apps and desktop computers. But the mobile generation isn't so biased.

The features announced for Messages, Messenger, and Allo are all inching toward becoming heirs to the home screen. Messages now allows third-party apps to graft themselves onto the service, so that you can, for example, find Open Table reservations when you’re chatting with a friend. Messenger is creating both chatbots and payments, so that you could, one day soon, chat with Everlane’s bot and buy a new shirt without ever having to launch an app or a web page.

For now, Allo is both more modest and more advanced. Take that skydiving example, which really does work. To come up with the suggested smart replies—"So brave," How fun," and "So exciting"—Google’s servers have to analyze your picture, figure out what’s happening, and spit out a list of socially acceptable responses. They might need a little work on the punctuation of course: "How fun" is the dismissive, bitchy cousin to "How fun." Both are a far cry from the chipper and polite, "How fun!"

Right now, Allo's responses are partly based on donated chat data from Google employees; as the real world sample gets larger, the results should improve. But there are interesting signs that it's already adapting to context. If you say "Yo" to someone, for example, the auto reply is "What up." According to Nick Fox, Google’s VP of communications products, those smart replies are meant to solve a fundamental problem with messaging today. We know what we want to say, but it still takes too long to actually say it, whether we’re trying to type with only one hand, or we’re dashing from the car to the grocery store. "It makes chat more efficient," says Fox. "But sometimes, the smart replies are slightly different than what you would have said, and they push the conversation ahead in a new way."

But the most interesting thing about Allo is Google Assistant. "One of the biggest opportunities we saw was when someone in a group chat would have to leave and bring some piece of information back to the group," says Cornwell. Pulling the assistant into your group chats hints at a future where the home screen or the app store cease to be bottlenecks in how you use your phone. The assistant could replace an immense amount of hassle that we take for granted on our phones. Instead of chatting with a friend about dinner, bouncing out of your chat app to find a restaurant, and then coming back to your chat armed with a link that you’ve copied, it’s far more elegant and intuitive to bring the assistant in, so that you and the person you’re chatting with can both look at the results.

The Discoverability Problem

Of course, the hard part is knowing what an assistant like that can do—what’s known, in UX circles, as the discoverability problem. Amazon Echo is a good example of discoverability gone wrong. It can now do hundreds of things thanks to third-party integrations it calls "skills," but you still find out about those new skills with an email that Amazon sends every week. When you’re chatting with the assistant, it solves that problem by pushing messages about its range of capabilities, from translation to games to sports to travel.

But we are clearly in early days. For example, when I ask Google Assistant, "Find me flights to San Francisco on Delta, this October 30," it still has to push me to Google’s web results. But currently, I can ask the assistant to "Show me emails I’ve send to Mark." The point is, Google has a great deal of scaffolding to build before Allo and the assistant do all the things we can imagine. But Google says that the scaffolding is coming. Third-party integrations with other apps, such as Uber or Open Table, should be arriving this winter, and the AI itself will get better at responding to users—while also giving the developers more and more data about the things people want Allo to do.

For us, the users, the key detail might be the one that’s also easiest to take for granted: the new paradigm of bringing a virtual assistant into chats you’re already having, as you can now do simply by typing "@google" when you're chatting with a friend. "That’s transformative," says Cornwell, who has been working with bot UIs ever since he graduated from college, more than a decade ago. That would solve the two biggest problems about apps—getting people to spread them, and getting them to actually use them. Cornwell believes that we’ve reached "peak app" and that we’re due for a paradigm shift in computing—one with that makes services easier to build and spread.

"The job of designers like me is to make UI go away," he adds. "We’re just getting to the point where natural language processing and AI is good enough that you don’t have to create UI artifice anymore."

[All Images: courtesy Google]

The Fast Company Innovation Festival