Co.Design

The Future Of Technology Isn't Mobile, It's Contextual

Next up: Machines that understand you and everything you care about, anticipate your behavior and emotions, absorb your social graph, interpret your intentions, and make life, um, "easier."

You’re walking home alone on a quiet street. You hear footsteps approaching quickly from behind. It’s nighttime. Your senses scramble to help your brain figure out what to do. You listen for signs of threat or glance backward. What you learn may prompt you to turn down another street, confront the person, or relax. Whether he or she turns out to be a mugger or a jogger, your brain rapidly cycled through many scenarios seeking an answer.

It’s called situational awareness. The way we respond to the world around us is so seamless that it’s almost unconscious. Our senses pull in a multitude of information, contrast it to past experience and personality traits, and present us with a set of options for how to act or react. Then, it selects and acts upon the preferred path. This process--our fundamental ability to interpret and act on the situations in which we find ourselves--has barely evolved since we were sublingual primates living on the Veldt.

Here’s the rub: Our senses aren’t attuned to modern life. A lot of the data needed to make good decisions are unreliable or nonexistent. And that’s a problem.

Contextual Computing: Our Sixth, Seventh, and Eighth senses

In the coming years, there will be a shift toward what is now known as contextual computing, defined in large part by Georgia Tech researchers Anind Dey and Gregory Abowd about a decade ago. Always-present computers, able to sense the objective and subjective aspects of a given situation, will augment our ability to perceive and act in the moment based on where we are, who we’re with, and our past experiences. These are our sixth, seventh, and eighth senses.

Hints of this shift are already arriving. Mobile devices with GPS deliver location-based services, which sets a baseline for the many ways your phone can gather information it will use to make your life easier down the line. Amazon’s and Netflix’s recommendation engines, while not magnificently intuitive, feed you book and video recommendations based on your behavior and ratings. Facebook’s and Twitter’s valuations are premised on the notion that they can leverage knowledge of your acquaintances and interests to push out relevant content and market to you in more effective ways.

These merely scratch the surface. The adoption of contextual computing--combinations of hardware, software, networks, and services that use deep understanding of the user to create tailored, relevant actions that the user can take--is contingent on the spread of new platforms. Frankly, it depends on the smartphone. Mobile technology isn’t interesting because it’s a new form factor. It’s interesting because it’s always with the user and because it’s equipped with sensors. Future platforms designed from the ground up for contextual computing will make such devices seem closer to toys than to a phone with cool tools.

For that to happen, computer scientists, technology companies, and users all need to understand and buy into the requirements and possibilities of contextual computing. It’s a cultural moment that’s not dissimilar to the way in which graphical, and then networked computing, were introduced in conceptual and technical forms 10 years before reaching commercial success.

You Need Four Data Graphs to Make It Work

At Jump, we’ve identified four data graphs essential to the rise of contextual computing: social, interest, behavior, and personal. Some are well-established and others have emerged seemingly out of thin air in the last few years. By mastering all four of these graphs, players seeking to dominate the next era of the web will be wildly successful.

There are legitimate ethical concerns about each of these graphs. They throw into relief the larger questions of privacy policy we’re currently wrestling with as a culture: Too much disclosure of the social graph can lead to friends feeling that you’re tattling on them to a corporation. The interest graph can turn your passions into a marketing campaign. The behavior graph can allow people who wish you harm to know where you are and what you’re doing. And revealing the personal graph can make it feel like an outside entity is quite literally reading your mind. We’re all trying to understand what to do about this from an individual standpoint, let alone a legal one.

Despite the ethical ambiguity around contextual computing, what matters is that companies are actively constructing these graphs already. These products and services are in the market today, but most in existence target only one or two of these graphs. Few are pursuing all four, both given the immaturity of the space and a lack of clear targets to shoot for. This has the unintentional effect of highlighting the risks of using such services, without demonstrating their benefits. For the potential of contextual computing to be realized, these data sets must be integrated.

The Social Graph Is About Connections

This data set shows how you connect to other people and how they are connected to one another. It also reveals the nature and emotional relevance of those connections. Most people associate this with Facebook, but it’s actually an idea and data set that spread far beyond its walls. In an ideal contextual computing state, this graph would be complete--so gentle nudges by software and services can bring together two people who are strangers but who could get along brilliantly and are in the same place at the same time. It could be two people who share a friend and who simultaneously move to Omaha, where neither person knows a soul.

Only when this graph is open to a wide variety of services will it reach its potential. And all the social data in the world won’t be helpful in the slightest if you know little about a specific person’s beliefs, activities, and interests.

Your Personal Graph Contains (Gulp) All Your Beliefs

This is the set of data relating to a person’s deepest held beliefs, core values, and personality. It’s what makes a person unique in the world, just as the social graph helps to show what makes her similar to others. The data set is under-developed at the moment, and it’s quite difficult to design for, even conceptually.

Given that psychology still struggles to explain exactly how our personal identities function, it’s not surprising that documenting such information in a computable form is slow to emerge. There are early indicators that this will change, however. For example, Proust.com, a relatively new (and struggling) social-networking service, asks users to document intimate details of their lives and their beliefs based on the idea of the famed Proust Questionnaire. People have, quite reasonably so, been reluctant to share such information in a publicly viewable social network.

A more successful example is Evernote, which has built a large business based on making it incredibly easy and secure to document both recently consumed information and your innermost thoughts. Scraping such intimate files for data is currently the questionable realm of the NSA, however. Entirely new solutions will need to be created if the potential of the personal graph is to be reached.

The Interest Graph--What You Like--Is About Curiosity

Your tastes and preferences are largely organized around the subjects that tend to correlate with one another. It’s also about the overlaps in taste between the individuals whose lives closely resemble your own. Many companies have made early bets in this arena; Twitter is a fan and believes it’s well on its way to fully charting how all subjects connect to all others.

For now, such applications are notoriously narrow. For example, a book site like Goodreads.com is capable of predicting what other books you might read based on your expressed interests. What’s problematic is that the interest graph falls far short of depicting your real interests and tastes. It cannot yet tackle the way your curiosity might lead you to new directions. And it could never effectively recommend a restaurant or a vacation spot based on what it knows you read.

Your Behavior Can Be Easily Graphed

It’s easy for data to depict what you actually do instead of what you claim to do. Sensors do the job. So do, if less elegantly, self-reporting mechanisms. This data can sit in pivotal contrast to the interest graph, allowing computers to know, perhaps better than you, how likely you are to go for a jog. It would be useful, too, for a travel site that notes how you tell friends you’d like to visit China but records that you only vacation in Europe. Rather than uselessly recommending vacation deals to Beijing, a smart travel app would instead feed you deals to Paris or Berlin. The behavior graph provides the foundation, to some extent, of Google Search, Netflix recommendations, Amazon recommendations, iTunes Genius, Nike+ run tracking, FourSquare, FitBit, and the entire "quantified self" movement. When mashed against the other three graphs, there’s a potential for real insight.

In the Best Possible Light, Contextual Computing Helps You Out

The real potential of contextual computing isn’t about just one of these graphs. It’s about connections that resonate between them and which get tailored to different kinds of experiences. Early entrants like Google’s Now and Glass projects, Highlig.ht, and Siri are just beginning to experiment with these technologies. Just as the visionaries at Xerox PARC (who developed the foundational technologies of every desktop PC) could not have fully grasped the long-term impact of the mouse and graphical computing when they began working on them in 1973, we cannot say now which contextual applications will emerge as most vital. The way to the future will be paved on many thousands of interesting failures.

Granted, true contextual computing is a little further around the corner than the most optimistic pundits would have you believe. That should not be mistaken as a caveat that it’s unlikely to fully arrive. As Bill Gates astutely pointed out, “There’s a tendency to overestimate how much things will change in two years and underestimate how much change will occur over 10 years.” (Notably, the tablet computers he introduced in 2001 didn’t achieve commercial success until the launch of the iPad in 2010.)

Within a decade, contextual computing will be the dominant paradigm in technology. Even office productivity will move to such a model. By combining a task with broad and relevant sets of data about us and the context in which we live, contextual computing will generate relevant options for us, just as our brains do when we hear footsteps on a lonely street today. Then and only then will we have something more intriguing than the narrow visions of wearable computing that continually surface: We’ll have wearable intelligence.

[IMAGE: Cursor via Shutterstock

Add New Comment

26 Comments

  • We often pick bad movies, bad books, knowing ourselves we often make bad choices and/or mistakes. So the fact that some database has our data does not guarantee we would love the next movie our favorite actor is in.

    We love chinese food yet there are plenty of chinese restaurants we hate. We love Detective Shows yet there are plenty of Detective Shows we won't watch. Perhaps context computing can get closer to correct recommendations even at some point recommending new items we've never considered. It will be nice to have such enhancements.

    Even bad recommendations can help. We have a friend who recommends chinese restaurants to use and we to him. Yet we don't like his chinese food recommendations and he doesn't like ours. So when he recommends a chinese restaurant to us now we avoid that restaurant and he ignores the restaurants we recommend based on the experience of each of us hating the others choices.

    I guess context computing can help with experiences...

  • Collin Davis

    What happens when one person can do the work of three because of contextual computing? The efficiency is great but that ultimately benefits a corporation more. What if this increase in efficiency is noted and the management decides "Why have three when we can have one do the same amount?" Then what, 2/3 layoffs? Even less people with jobs? How is that in anyway sustainable? Tell me its unlikely please.

  • John Akwei, ECMp ERMp

    My theory is that AI is flawed because Human Intelligence is based on wish fulfillment leading to the continuation of life, and that isn't relevant for inorganic constructs. Machine contextual analysis would encounter the same problem.

  • Mark Bernstein

    The Mark Weiser vision for Ubiquitous Computing at PARC 20 years ago had two key aspects - 1) that computation would permeate our environment in the form of many devices, displays, sensors and actuators; and 2) that the emerging computational environment would help us in our work and play. Clearly, we're in the middle of "1)" and spending way too much time attending to our computational platforms. And, I hope, we are finally about to approach "2)", where a multitude of contextual computing technologies enable surprisingly productive consequences in many aspects of our daily lives. NSA willing, of course...

  • quesadam

    remember when computers were going to deliver artificial intelligence? AI, Contextual-Computing, both buzzwords of the same family of unrealistic human thinking 

  • Naomi Colb

    I believe that the world is a giant holodeck that already quickly responds to the thoughts, emotions, intentions expectations we cultivate within. 

    I believe that rather than focusing on ever more intuitive and humanoid micro-robots, we would be much better off focusing on developing our own capabilities....calming our inner voices, becoming ever more present, aware of, receptive to and appreciative of our 5 senses.
    I believe that when we become conscious of our implicit interconnectedness with everyone and everything in this universe, we have the power to attract exactly the experiences, people and things that we desire.
    I believe that we are over-stimulated, technology - dependant and addicted.We are chronically distracted from optimal living in a human body.  Desensitized, we miss the fulfillment of sensual living and the joy of generating our experience, rather than living in continual reaction to the illusion of imagined threat.  

  • Andy Galaxy

    If you give computers (or a brain for that matter) control over context, well, ultimately you're going to fail. Intuition wins every time, and that's what computers will never have. This 'privacy' issue is a scam by information mongers. If you don't want anyone to know - don't write it down.

  • Peter

    The issue is not so much the graphs and information itself, but who controls it. If the individual has ownership and is able to reveal or conceal at will then this sort of contexual computing can provide benefit. When corporate giants have control over the data, they MAY allow these sort of useful apps to be constructed, but only as a side effect of their own goals.
    The missing link then is the personal data governance. Until this is manifests, the 'privacy' concerns are going to override the percieved benefits and the full power of contextual computing will not be realised.

  • Brad Arnold

    There is a growing divide in the human population between haves and have nots, literate and not functionally literate, technologically savvy and technological avoidance, but now a real big one is opening up between those who don't have as big a privacy concerns and can embrace technology like social networking or location detectors or even the sharing of personal information to gain access, and those who are privacy hounds who jealously guard their privacy.

    Seriously, I have a friend that refused to register a free software product because it required an email address.  Another won't allow any digital photos to be taken of his family. A third friend won't join any social networks, and is afraid of having a wireless network in his home.  Obviously, allowing a computer to obtain your  social, interest, behavior, and personal data profiles is very intimate.  Furthermore, I believe soon there will be computer sensors you implant in your body that broadcast your physical state to the net for monitoring and analysis.

    Those that refuse such very very handy technologies will be left far behind those that utilize them - in the case of a heart attack for instance, those that are monitored will get immediate intervention, while those who are don't utilize that service might very well die or be permanently disabled.

  • Greg Lloyd

    I like your take on contextual computing, and agree that capturing context in a form that agents like Siri or Google Now can reason about is the key to radically improving their performance. Both systems do a pretty good job using just contacts, calendar and perhaps a bit of trace and navigation data.

    Somewhat ironically, it seems that recording important parts of person's social, interest and behavior graph may be simpler in the context of work, where shared tasks, discussions, work product and the related activity streams can already directly recorded rather than inferred.

    People on the Enterprise 2.0 / Social Business side talk and argue a lot about "What's do people mean by the context of a business activity?" and how to leverage context to "socialize" business objects.

    I've done my share - focused on making tasks, discussions, work product first class objects that can be connected, tagged, discussed, shared - and used to augment human powered content navigation as well as faceted search.

    My two cents on how this relates to contextual computing here: http://traction.tractionsoftwa...
    Summary: A representation of business objects in the context of work can make some useful parts of social, interest and activity graphs straightforward to represent and reason about.

  • Pete Mortensen

    Hi everyone,

    Pete here. Thanks for the incredible response and good discussion that's starting to happen! The piece has struck a nerve, and I can only assume it's because I have done nothing more than synthesize the meta-intent behind a few thousand startups and fledgling offerings in the market. 

    There are, however, a few questions I wanted to address that don't currently warrant their own follow-up piece.

    1. These are the graphs of data about an individual necessary to do good contextual computing, not the graphs of all possible data required.
    -- As has been raised by more than one commenter, yes, things like Google's Knowledge Graph are incredibly important, too, but it was, in the past, a far more objective thing. Page Rank was once absolute but has become, well, contextual, largely through the addition of data from the Behavior Graph (location, search history, browser history from Chrome users, the contents of email, though we don't like to think about it) and the Social Graph (Google+ mostly, though they have leveraged both Facebook and Twitter in the past). And it's gotten better with mobile, because LBS are a key component of Behavior Graph data. There are almost certainly other data graphs that are really useful to include, like the current weather, but that is only relevant based on the interests and behavior of the user in question, if that makes sense.
    2. Yes, other people have talked about context for a long time. And no, this article is not a comprehensive literature survey.
    -- I will freely acknowledge that most of the thoughts in this piece come from my own heads and those of some of my colleagues at Jump Associates. No intentional snubs here, though omissions also should not be taken as endorsements or non-endorsements. Scoble, in particular, based on the one piece he's written about context so far, has a very different take on the space at the moment, which is why I didn't write about him here (for one thing, I don't think there is close to a general-use case scenario for Google Glass -- though I think the potential in healthcare is intriguing). I'm actually curious to see what my thinking looks like through his lens and vice versa -- and it's certainly to early to argue for one definitive take.

    3. Yes, it might be creepy. But so is most of what we do on an everyday basis, in the eyes of 10 years ago.
    -- There are a lot of people who swore they would never use cell phones or smartphones a few years back. Their numbers have dwindled. This doesn't mean that things won't go differently than I assume they will from the current vantage point, but something that is extremely contextual is what everyone is driving toward today. We need to have a serious conversation today so we can anticipate and avoid some of the pitfalls that make people feel weird. Not talking in a serious way about privacy in service of context is the one way to be sure we don't like what emerges.

    Thanks, everyone. I'll be periodically checking back in, so feel free to ask me anything.

    Pete

  • Giulio Coraggio

    Don't you think there will be massive privacy issues connected to such practices? The problem has been recently raised with reference to mobile apps (http://www.gamingtechlaw.com/2... and data collected through them which are becoming more and more invasive in our life.