Privacy and the Market for Lemons, or How Websites Are Like Used Cars

March 18, 2011 at 4:37 pm 6 comments

I had a fun and engaging discussion on the “Paying With Data” panel at the South by Southwest conference; many thanks to my co-panelists Sara Marie Watson, Julia Angwin and Sam Yagan. I’d like to elaborate here on a concept that I briefly touched upon during the panel.

The market for lemons

In a groundbreaking paper 40 years ago, economist George Akerlof explained why so many used cars are lemons. The key is “asymmetric information:” the seller of a car knows more about its condition than the buyer does. This leads to “adverse selection” and a negative feedback spiral, with buyers tending to assume that there are hidden problems with cars on the market, which brings down prices and disincentivizes owners of good cars from trying to sell, further reinforcing the perception of bad quality.

In general, a market with asymmetric information is in danger of developing these characteristics: 1. buyers/consumers lack the ability to distinguish between high and low quality products 2. sellers/service providers lose the incentive to focus on quality and 3. the bad gradually crowds out the good since poor-quality products are cheaper to produce.

Information security and privacy suffer from this problem at least as much as used cars do.

The market for security products and certification

Bruce Schneier describes how various security products, such as USB drives, have turned into a lemon market. And in a fascinating paper, Ben Edelman analyzes data from TRUSTe certifications and comes to some startling conclusions [emphasis mine]:

Widely-used online “trust” authorities issue certifications without substantial verification of recipients’ actual trustworthiness. This lax approach gives rise to adverse selection: The sites that seek and obtain trust certifications are  actually less trustworthy than others. Using a new dataset on web site safety, I demonstrate that sites certified by the best-known authority, TRUSTe, are more than twice as likely to be untrustworthy as uncertified sites. This difference remains statistically and economically significant when restricted to “complex” commercial sites.

[...]
In a 2004 investigation after user complaints, TRUSTe gave Gratis Internet a clean bill of health. Yet subsequent New York Attorney General litigation uncovered Gratis’ exceptionally far-reaching privacy policy violations — selling 7.2 million users’ names, email addresses, street addresses, and phone numbers, despite a privacy policy exactly to the contrary.

[...]
TRUSTe’s “Watchdog Reports” also indicate a lack of focus on enforcement. TRUSTe’s postings reveal that users continue to submit hundreds of complaints each month. But of the 3,416 complaints received since January 2003, TRUSTe concluded that not a single one required any change to any member’s operations, privacy statement, or privacy practices, nor did any complaint require any revocation or on-site audit. Other aspects of TRUSTe’s watchdog system also indicate a lack of diligence.

The market for personal data

In the realm of online privacy and data collection, the information asymmetry results from a serious lack of transparency around privacy policies. The website or service provider knows what happens to data that’s collected, but the user generally doesn’t. This arises due to several economic, architectural, cognitive and regulatory limitations/flaws:

  • Each click is a transaction. As a user browses around the web, she interacts with dozens of websites and performs hundreds of actions per day. It is impossible to make privacy decisions with every click, or have a meaningful business relationship with each website, and hold them accountable for their data collection practices.
  • Technology is hard to understand. Companies can often get away with meaningless privacy guarantees such as “anonymization” as a magic bullet, or “military-grade security,” a nonsensical term. The complexity of private browsing mode has led to user confusion and a false sense of safety.
  • Privacy policies are filled with legalese and no one reads them, which means that disclosures made therein count for nothing. Yet, courts have upheld them as enforceable, disincentivizing websites from finding ways to communicate more clearly.

Collectively, these flaws have led to a well-documented market failure—there’s an arms race to use all means possible to entice users to give up more information, as well as to collect it passively through ever-more intrusive means. Self-regulatory organizations become captured by those they are supposed to regulate, and therefore their effectiveness quickly evaporates.

TRUSTe seems to be up to some shenanigans the online tracking space as well. As many have pointed out, the TRUSTe “Tracking Protection List” for Internet Explorer is in fact a whitelist, allowing about 4,000 domains—almost certainly from companies that have paid TRUSTe—to track the user. Worse, installing the TRUSTe list seems to override the blocking of a domain via another list!

Possible solutions

The obvious response to a market with asymmetric information is to correct the information asymmetry—for used cars, it involves taking it to a mechanic, and for online privacy, it is consumer education. Indeed, the What They Know series has done just that, and has been a big reason why we’re having this conversation today.

However, I am skeptical that the market can be fixed though consumer awareness alone. Many of the factors I’ve laid out above involve fundamental cognitive limitations, and while consumers may be well-educated about the general dangers prevalent online, it does not necessarily help them make fine-grained decisions.

It is for these reasons that some sort of Government regulation of the online data-gathering ecosystem seems necessary. Regulatory capture is of course still a threat, but less so than with self-regulation. Jonathan Mayer and I point out in our FTC Comment that ad industry self-regulation of online tracking has been a failure, and argue that the FTC must step in and enforce Do Not Track.

In summary, information asymmetry occurs in many markets related to security and privacy, leading in most cases to a spiraling decline in quality of products and services from a consumer perspective. Before we can talk about solutions, we must clearly understand why the market won’t fix itself, and in this post I have shown why that’s the case.

Update. TRUSTe president Fran Maier responds in the comments.

Update 2. Chris Soghoian points me to this paper analyzing privacy economics as a lemon market, which seems highly relevant.

Thanks to Jonathan Mayer for helpful feedback.

To stay on top of future posts, subscribe to the RSS feed or follow me on Twitter.

Entry filed under: Uncategorized. Tags: , , , , , .

Link Prediction by De-anonymization: How We Won the Kaggle Social Network Challenge The Master Switch and the Centralization of the Internet

6 Comments Add your own

  • 1. Fran Maier  |  March 18, 2011 at 6:13 pm

    I’d like to address several aspects of this blog

    First regarding Edelmen’s research from 2006 (although it appears that he’s updated some references) I’d like to point out the following:

    – Ben’s research uses Site Advisor (now part of McAfee) as the authority without providing details of the methodology, why it is better, and the definition of trust that he uses (and how that compares to TRUSTe’s). You should note that Ben served as an advisor to Site Advisor and was also an “Expert Reviewer” at the time.

    – TRUSTe frequently takes compliance enforcement action. In fact, we terminated Gratis Internet (Free iPods) http://www.engadget.com/2005/02/11/truste-says-freeipods-com-not-so-trusty/ and cooperated with the NY State AG office and Classic Closeouts http://www.truste.com/about_TRUSTe/press-room/news_truste_helps_consumers_recover_funds_from_retailer.html

    On a daily basis companies that apply to TRUSTe to not earn the seal because they are unwilling to make changes to their privacy policy or practices. At least 12% fail the certification process. EVERY client makes some kind of change to get TRUSTe certified.

    As to our requirements, we just issued an update to our Certification Criteria
    http://www.truste.com/privacy-program-requirements/index.html. Since 2006 we’ve also applied additional web site scanning and other tools to strengthen the monitoring.

    In regard to TPL, TRUSTe has published criteria for the Allow on the IE9 TPL List and at the beginning of April we will be using this criteria which as you can see require rigor and review:

    http://www.truste.com/privacy-program-requirements/3rd-party-data-collection/index.html

    By the way, we’re up the road in San Francisco and our blog is always available, we’d be happy to demonstrate more about what we do and how we do it.

    Fran Maier
    President, TRUSTe

    Reply
    • 2. Arvind  |  March 18, 2011 at 6:19 pm

      Thanks for the response. I’ve added a note to the post.

      Edit. Having looked at your TPL Allow criteria, I have to say that it does little to assuage the concerns that have been raised.

      The whole reason that there is a privacy outcry and that the TPL feature exists is that the existing system is broken, including “non-PII,” notice-and-choice, etc, which are some of the factors I pointed out in my post. Yet these are precisely the measures that your critiera emphasize, and thus it seems to be aimed at preserving the status quo rather than changing it.

      The only admirable part of the criteria is the one about DNT header, but since you only require “accepting” them and not honoring them, it’s not clear if it even means anything.

      And finally, from an empirical standpoint, you have several thousand domains on the allow list and a couple dozen on your block list. This is starkly at odds with the reality of user expectations; your criteria are a secondary matter.

      Reply
  • 3. Adam Thierer  |  March 18, 2011 at 6:38 pm

    Shorter version of your argument:

    (1) Technology is complicated.
    (2) People are stupid / personal responsibility can’t work.
    (3) We must regulate to save the poor saps.

    Same logic has driven the online child safety debates for years, except it’s parents instead of average consumers accused of being too ignorant for their own good.

    Regardless, if (1) and (2) are true, your Do Not Track silver-bullet solution ain’t gunna save the day. The only solution would be an outright ban on all online data collection.

    You should have a little more faith in your fellow humans as well as the evolution of code markets to come up with creative solutions for those who need them. Remember, your values are not necessarily representative of everyone else’s out there. Just as in the child safety context, there ARE solutions out there for those who are really sensitive about these things.

    — Adam Thierer

    Reply
    • 4. Arvind  |  March 18, 2011 at 6:47 pm

      That’s not a shorter version of my argument, it’s a twisted and incorrect version.

      Reply
    • 5. cw337  |  July 31, 2011 at 4:05 am

      I agree with Arvind. 2) Isn’t true at all: Even for technical experts, it is very difficult to browse the web without being tracked. I use 12 browser add-ons to try to keep some privacy https://addons.mozilla.org/en-US/firefox/collections/gracefool/privacy/. But it’s time-consuming and error-prone, and I still often fail because so many websites require cross-site requests to popular providers in order to work.

      Reply
  • 6. The Quest to Find Privacy Market Failure  |  March 19, 2011 at 4:31 pm

    [...] recent market failure blog post called “Privacy and the Market for Lemons, or How Websites Are Like Used Cars,” seems to have piqued Adam’s interest. (See the comments.) In it, privacy and [...]

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Trackback this post  |  Subscribe to the comments via RSS Feed


About 33bits.org

I'm an assistant professor of computer science at Princeton. I research (and teach) information privacy and security, and moonlight in technology policy.

This is a blog about my research on breaking data anonymization, and more broadly about information privacy, law and policy.

For an explanation of the blog title and more info, see the About page.

Subscribe

Be notified when there's a new post — subscribe to the feed, follow me on Google+ or twitter or use the email subscription box below.

Enter your email address to subscribe to this blog and receive notifications of new posts by email.

Join 213 other followers