Co.Design

Will Apple's Patent Victory Create A Usability Hell?

We talk to an IP lawyer about whether Apple really can own a gesture that now seems foundational.

Pinching to zoom on touch-screen devices is such a common gesture today that it’s hard to believe Steve Jobs wowed audience members (who actually cheered and applauded for close to 20 seconds) when he first stretched his fingers against the iPhone’s glass face. Now, the interaction is used by nearly all device makers in electronics ranging from smartphones to tablets to laptop trackpads, a universal gesture for magnifying photos, maps, and webpages.

But the ubiquity of that hand gesture could soon go the way of the Macarena. In Apple’s $1 billion patent lawsuit against Samsung, which the company won last week, a jury decided that a slew of Samsung’s mobile devices had violated a number of Apple’s utility patents for interaction designs, including some that relate to distinctions between one-finger scrolling and two finger gestures that could include pinch-to-zoom. In an industry already wary of patent litigation, the verdict could lead to significant fragmentation in the user experiences created by tech companies eager to avoid the wrath of Apple’s rabid legal squad.

"With regard to gestures, I think it will be hard to change the status quo because they’ve already gained such widespread acceptance," says one top interaction designer at HP. "Ultimately, I’m worried it would hurt the end user--it concerns me that you could have these inconsistencies in terms of user experience as you go from different product to product."

As my colleague Kyle Vanhemert outlined Monday, without usability standards, the consumer experience suffers. The pinch-to-zoom gesture, for example, is part of an intuitive, cross-platform interaction language, but companies might look at the Samsung verdict and hesitate to mimic the experience, even though the Apple v. Samsung case didn’t hinge on pinch-to-zoom specifically. The jury also found Samsung had violated other Apple utility patents, for example, including patents for its tap-to-zoom gesture and rubberbanding technology. "It’s very much possible that pinch-to-zoom is the best way of zooming in on a map," Vanhemert wrote. "Forcing smartphone developers to come up with a different way of doing things for the sake of coming up with a different way of doing things is outrageously stupid."

But Christopher Carani, an IP lawyer with McAndrews, Held & Malloy, Ltd. and the former chair of the Industrial Designs Committee at the American Intellectual Property Law Association, questions whether it will ever get to that point. Carani explains that we shouldn’t think of the jury’s verdict like a binding Supreme Court decision. "This is a district court decision--it doesn’t have some sort of precedential effect from one court to the next, or one country to the next," he says. "So before it all gets gloom and doom, which is basically what I’ve read so far, with respect to these types of functional utility patents, it’s critical not to paint with a broad brush and say this is just any type of pinch-to-zoom or tap-to-zoom function."

Rather, he continues, "In these types of situations, you’re not actually patenting the exact concept, but with something like pinch-to-zoom, you have to drill down the specific language of the patent claim itself, so there might be some actual command that has to be used or some underlying instructions that are covered. By now Samsung’s attorneys have studied this thing inside and out; they know the prior art; they know the scope of the claim; and they’re going to be counseling their clients to try and find a way to design a workaround." Canari cites his own BlackBerry phone, for example, which has pinch-to-zoom functionality. "But I don’t know," he adds with a laugh, "Perhaps Apple will go after [RIM] next."

Arguably, Apple doesn’t even have to. James Schauer, who spent several years at RIM before becoming CEO of Harmonic Interactive Research and Design, which develops gesture-recognition technology, saw firsthand the impact potential litigation can have on product experience. "I can assure you there was a chilling effect on innovation and design at RIM. One of the reasons RIM didn’t do any [rubberbanding-like] bounce back was because of Apple’s patents," Schauer says. "They took a very conservative approach--anywhere we could possibly face litigation was looked at with extraordinary scrutiny. Arguably that became the straw that broke the camel’s back at RIM. When the lawyers are in charge, that’s game over."

Still, Schauer believes the jury made the right decision in the Samsung-Apple patent lawsuit. "We were pretty upset when we saw Android came out--we knew they hacked all over Apple’s patents, and we were like, 'This is not fair,'" he says. "I think the jury made the right ruling, but I think the real question is: Should pinch-to-zoom be patentable?"

Many share his concerns. Slate said Apple’s new "pinch-to-zoom monopoly is bad news." The New York Times reported that some designers are concerned the patent case’s outcome could lead to unnecessarily differentiated user experiences, akin to being forced to create triangular steering wheels for cars because of patent concerns, despite rounded wheels being the universal standard.

Triangular steering wheels? "I think that’s a healthy dose of hyperbole," Canari says.

[Image: ivelly/Shutterstock]

Add New Comment

37 Comments

  • Mark

    It's only a problem because apple has patented ridiculous things (like the rectangle with rounded corners) as well as things that they are yet to actually use in their designs.

  • Matt

    I'm not familiar with patent law, but is there not a "fair use" caveat, as there is in copyright? Essentially, if you copyright something but dont defend it, and it become common practice, you essentially lose the copyright. IE if I had copyrighted the design of a Stop sign, but hadn't filed a complaint when i discovered cities were using it without permisssion, once "stop sign" became part of the common lexicon my copyright becomes worthless.
    Again, I'm not familiar with the specifics of the laws in place, but i'm surprised that something similar does not apply here. Why is it that several years and hundreds of Android devices from multiple companies later, Apple can now suddenly single out Samsung for a small handful of infringements?
    Anyone want to argue, please do. I'm honestly just cuious..

  • Stuart

    (if you apply you reasoning to pinch to zoom, you have to apply it too all UI behavior, otherwise it's an empty argument)

  • Stuart

    Standardization would mean reduced vendor differentiation... that's the same as removing choice... the way it works right now is exactly right.  

  • Eric Tan

    If Samsung (or Android) or other companies infringing on the pinch to zoom patent want to continue using it, wouldn't it be easier just to pay Apple a licensing fee.  I don't know what additional demands Apple might have of Samsung, but financially it would make sense.  I find it rather doubtful that a better way to zoom will ever be invented and patented for all to use for free...

  • pmstest

    I'll gladly give up pinch to zoom and use anything else so that we can get rid of the Apple dominance outside the lines of their ridiculous patents. Android-based phones are taking over whether people like it or not.

  • Tyler Fastcompany Gray

    Read that story from Verge again. Apple does have a ltd Pinch to Zoom patent. It just didn't come up in this case. We've clarified language (as did the NYT after this initial story came out) to make it clear that whether or not this case was specifically about pinch-to-zoom, it was close enough, and it involved enough significant other related utility patents to potentially scare designers away from anything resembling those behaviors. 

  • Vance McAlister

    I really am trying to get a grip on this issue, and would love to hear back from Austin on this issue.  How does this article fit together with the Verge analysis that Apple doesn't even HAVE a patent on "pinch to zoom".  Are they wrong? Is this article wrong?  Are they somehow both right?

  • Murray Sanders

    The notion is ridiculous. So, any improvements to the design or any modifications of the gesture all belong to Apple? I would bet lawyers did not put up the right fight. I don't see how this judgement can stand for long. Appeal now, Samsung.

  • jimmoffet

    I'm incredibly suspicious that patents encourage innovation. If you want to be rewarded for inventing something, bring it to market first. You'll have the market all to yourself. 

    If someone is able to bring the same product to market at the same time, I'm sorry but you need to compete against it. 

    I think that the chilling effect on innovation of imitations is probably grossly overstated. I'd like to see some research on what the effects might actually be of severely limiting what is patentable.

  • Priyank Ahuja

    I really don't understand why such patents are approved or why do people want to limit the technology. At one time you oppose SOPA and then you destroy the openness of software like andoid. If you have invented something then you could let people enjoy it and use it on a large scale to get appriciation and not bind it.

  • dabossguy

    Samsung, I've got a solution for you as an alternate to pinch to zoom. Wait for it, wait, wait - pinch a loaf!

  • kavok

    People are forgetting that these companies can be licensed to use the features that were in the patents. It's certainly not the end of the world. It's just that Samsung didn't want to have to pay to use them, and now they are being forced to.

  • Dave Haynie

    Apple does not deserve to "own" pinch-to-zoom.. there's just way too much prior art. Multi-touch goes back to 1982, at least; capacitive multitouch to 1984, at least. Apple and Microsoft engineers both got to see it, via John Underkoffler's brilliant view of the future in 2002, in the film "Minority Report". Of course, Apple also saw it in an in-house demo given by Adam Bogue in 2003. 

    Microsoft actually had a guy, Bill Buxton, who worked on such things, elsewhere, back in the early 80s, and has a great page about them here: http://www.billbuxton.com/mult...

    Apple does have one additional pinch and zoom patent I didn't see called up, which is basically a double pinch-zoom. That may well be unique. 

    There's supposedly a good bit of prior art on the "bounce-back" patent as well, but that's one Samsung should have known better... that was a very well know Apple software patent, and Android doesn't do that. Which is a good thing... Apple's all about the skeuomorphisms, and for that, it makes sense that your scrolling book might hit a rubber bumper when it gets to the end. For an actual computer screen, no need. It was stupid for them to add that in. 

    Were I on the jury, I'd certainly have given the prior art more attention than these guys did (I did dozens of similar patent reviews at Commodore, usually when being sued by someone like IBM, so I was far better versed in examining prior art than the guy with the really shaky patent who was the foreman), but I would also have taken that as a sign that Samsung pretty blatantly disregarded Apple's patents. As many have written, this shouldn't have been such an easy decision. Except maybe in Apple's neighborhood. 

  • loopdiloop

    If companies weren't so quick to copy the ideas and inventions of others this wouldn't be a problem. Accept risk and invent your own ways of interacting with mobile devices.

    Furthermore, this isn't a problem propagated by design teams but of businesses cutting corners in order to ride on the coat tails of another companies breakthrough. 

    Create a better way. There is plenty of talent out there (including teams at Samsung) that have dreamt up a myriad of different ways of dealing with the problem 'pinch to zoom' solves. Let those ideas appear in the next great product.