Towards a Science of Security and Human Behaviour

Towards a Science of Security and Human Behaviour

Towards a Science of Security and Human Behaviour Ross Anderson Cambridge University Traditional View of Infosec People used to think that the Internet was insecure because of lack of features crypto, authentication, filtering So we all worked on providing better, cheaper security features AES, PKI, firewalls About 1999, some of us started to realize that this is not enough

SOUPS 2008 July 24th 2008 Economics and Security Since 2000, we have started to apply economic analysis to IT security and dependability It often explains failure better! Electronic banking: UK banks were less liable for fraud, so ended up suffering more internal fraud and more errors Distributed denial of service: viruses now dont

attack the infected machine so much as using it to attack others Why is Microsoft software so insecure, despite market dominance? SOUPS 2008 July 24th 2008 New View of Infosec Systems are often insecure because the people who guard them, or who could fix them, have insufficient incentives Bank customers suffer when poorly-designed

bank systems make fraud and phishing easier Casino websites suffer when infected PCs run DDoS attacks on them Insecurity is often what economists call an externality a side-effect, like environmental pollution SOUPS 2008 July 24th 2008 New Uses of Infosec Xerox started using authentication in ink cartridges to tie them to the printer and its competitors soon followed

Carmakers make chipping harder, and plan to authenticate major components DRM: Apple grabs control of music download, MS accused of making a play to control distribution of HD video content SOUPS 2008 July 24th 2008 IT Economics (1) The first distinguishing characteristic of many IT product and service markets is network effects

Metcalfes law the value of a network is the square of the number of users Real networks phones, fax, email Virtual networks PC architecture versus MAC, or Symbian versus WinCE Network effects tend to lead to dominant firm markets where the winner takes all SOUPS 2008 July 24th 2008 IT Economics (2) Second common feature of IT product and

service markets is high fixed costs and low marginal costs Competition can drive down prices to marginal cost of production This can make it hard to recover capital investment, unless stopped by patent, brand, compatibility These effects can also lead to dominant-firm market structures SOUPS 2008 July 24th 2008 IT Economics (3)

Third common feature of IT markets is that switching from one product or service to another is expensive E.g. switching from Windows to Linux means retraining staff, rewriting apps Shapiro-Varian theorem: the net present value of a software company is the total switching costs So major effort goes into managing switching costs once you have $3000 worth of songs on a $300 iPod, youre locked into iPods SOUPS 2008 July 24th 2008 IT Economics and Security

High fixed/low marginal costs, network effects and switching costs all tend to lead to dominantfirm markets with big first-mover advantage So time-to-market is critical Microsoft philosophy of well ship it Tuesday and get it right by version 3 is not perverse behaviour by Bill Gates but quite rational Whichever company had won in the PC OS business would have done the same SOUPS 2008 July 24th 2008 IT Economics and Security (2)

When building a network monopoly, you must appeal to vendors of complementary products Thats application software developers in the case of PC versus Apple, or now of Symbian versus Linux/Windows/J2EE/Palm Lack of security in earlier versions of Windows made it easier to develop applications So did the choice of security technologies that dump usability costs on the user (SSL, not SET) Once youve a monopoly, lock it all down! SOUPS 2008 July 24th 2008 Economics and Usability

Make your products usable by newbies but much more usable with practice! To what extent can you make skill a source of asymmetric lockin? Hypothesis: this underlies the failure of user programmability to get traction! We have nothing now as good as BASIC was in the 1980s SOUPS 2008 July 24th 2008 Economics and Usability (2)

How many features should my product have? Marginal benefit of new feature concentrated in some target market Marginal cost spread over all users So we get chronic featuritis! At equilibrium, a computer / phone / anything programmable will be just on the edge of unacceptability to a significant number of users The same happens with laws, services, SOUPS 2008 July 24th 2008 Why are so many security

products ineffective? Akerlofs Nobel-prizewinning paper, The Market for Lemons introduced asymmetric information Suppose a town has 100 used cars for sale: 50 good ones worth $2000 and 50 lemons worth $1000 What is the equilibrium price of used cars? If $1500, no good cars will be offered for sale Started the study of asymmetric information Security products are often a lemons market SOUPS 2008 July 24th

2008 Products worse then useless Adverse selection and moral hazard matter (why do Volvo drivers have more accidents?) Application to trust: Ben Edelman, Adverse selection on online trust certifications (WEIS 06) Websites with a TRUSTe certification are more than twice as likely to be malicious The top Google ad is about twice as likely as the top free search result to be malicious (other

search engines worse ) Conclusion: Dont click on ads SOUPS 2008 July 24th 2008 Privacy Most people say they value privacy, but act otherwise. Most privacy ventures failed Why is there this privacy gap? Odlyzko technology makes price discrimination both easier and more attractive Acquisti et al people care about privacy when buying clothes, but not cameras (phone viruses

worse for vendor than PC viruses?) Loewenstein et al its not clear that there are stable and coherent privacy preferences! Student disclosure more for How bad RU and less with detailed privacy notice SOUPS 2008 July 24th 2008 Conflict theory Does the defence of a country or a system depend on the least effort, on the best effort, or on the sum of efforts? The last is optimal; the first is really awful

Software is a mix: it depends on the worst effort of the least careful programmer, the best effort of the security architect, and the sum of efforts of the testers Moral: hire fewer better programmers, more testers, top architects SOUPS 2008 July 24th 2008 How Much to Spend? How much should the average company

spend on information security? Governments, vendors say: much much more than at present But theyve been saying this for 20 years! Measurements of security return-oninvestment suggest about 20% p.a. overall So the total expenditure may be about right. Are there any better metrics? SOUPS 2008 July 24th 2008 Skewed Incentives

Why do large companies spend too much on security and small companies too little? Research shows an adverse selection effect Corporate security managers tend to be riskaverse people, often from accounting / finance More risk-loving people may become sales or engineering staff, or small-firm entrepreneurs Theres also due-diligence, government regulation, insurance and agency to think of SOUPS 2008 July 24th 2008 Skewed Incentives (2)

If you are DirNSA and have a nice new hack on XP and Vista, do you tell Bill? Tell protect 300m Americans Dont tell be able to hack 400m Europeans, 1000m Chinese, If the Chinese hack US systems, they keep quiet. If you hack their systems, you can brag about it to the President So offence can be favoured over defence SOUPS 2008 July 24th 2008 Security and Policy Our ENISA report, published in March, has 15 recommendations:

Security breach disclosure law EU-wide data on financial fraud Data on which ISPs host malware Slow-takedown penalties and putback rights Networked devices to be secure by default See links from my web page SOUPS 2008 July 24th 2008 Security and Sociology

Theres a lot of interest in using social network models to analyse systems Barabsi and Albert showed that a scale-free network could be attacked efficiently by targeting its high-order nodes Think: rulers target Saxon landlords / Ukrainian kulaks / Tutsi schoolteachers / Can we use evolutionary game theory ideas to figure out how networks evolve? Idea: run many simulations between different attack / defence strategies

SOUPS 2008 July 24th 2008 Security and Sociology (2) Vertex-order attacks with: Black normal (scalefree) replenishment Green defenders replace high-order nodes with rings Cyan they use cliques (c.f. system biology ) Application: traffic analysis (see my Google tech talk) SOUPS 2008 July 24th 2008 Psychology and Security

Phishing only started in 2004, but in 2006 it cost the UK 35m and the USA perhaps $200m Banks react to phishing by blame and train efforts towards customers But we know from the safety-critical world that this doesnt work! We train people to keep on clicking OK until they can get their work done and learned helplessness goes much wider People dont notice missing padlock the dog that didnt bark. Is there anything we can do? SOUPS 2008 July 24th 2008

Psychology and Security (2) Folklore: systems designed by geeks for geeks also discriminate against women, the elderly and the less educated We set out to check whether people with higher systemizing than empathizing ability would detect phishing more easily Methodology: tested students for phishing detection, and also on Baron-Cohen test Presented at SHB07: re-examined by sex SOUPS 2008 July 24th

2008 SOUPS 2008 July 24th 2008 Results SOUPS 2008 July 24th 2008 Ability to detect phishing is correlated with SQ-EQ It is (independently) correlated with gender

Folklore is right the gender HCI issue applies to security too Psychology and Security (3) Social psychology has long been relevant to us! Solomon Asch showed most people would deny the evidence of their eyes to conform to a group Stanley Milgram showed that 60% of people will do downright immoral things if ordered to

Philip Zimbardos Stanford Prisoner Experiment showed roles and group dynamics were enough The disturbing case of Officer Scott How can systems resist abuse of authority? SOUPS 2008 July 24th 2008 Psychology and Security (4) Why does terrorism work? The bad news: its evolved to exploit a large number of our heuristics and biases! Availability heuristic; mortality salience;

anchoring; loss aversion in uncertainty; wariness of hostile intent; violation of moral sentiments; credence given to images; reaction against outgroup; sensitivity to change; The good news: biases affect novel events more, and so can be largely overcome by experience SOUPS 2008 July 24th 2008 Psychology and Security (5) Deception from its role in evolution, to everyday social poker; self-deception; how deception is different online, and policy Would you really vote for a president you didnt think could lie to you?

Many inappropriate psychological interfaces are sustained by money or power compare why we fear computer crime too little, and terrorism too much SOUPS 2008 July 24th 2008 The Research Agenda The online world and the physical world are merging, and this will cause major dislocation for many years Security economics gives us some of the tools we need to understand whats going on

Sociology gives some cool and useful stuff too And security psychology is not just usability and phishing it might bring us fundamental insights, just as security economics has SOUPS 2008 July 24th 2008 More See www.ross-anderson.com for a survey article, our ENISA report, my security economics resource page, and links to: WEIS Annual Workshop on Economics and

Information Security SHB Workshop on Security and Human Behaviour (www.lightbluetouchpaper.org) Security Engineering A Guide to Building Dependable Distributed Systems 2e just out! SOUPS 2008 July 24th 2008 SOUPS 2008 July 24th 2008

Recently Viewed Presentations

  • Day 1: Course Overview and Derivatives Basics

    Day 1: Course Overview and Derivatives Basics

    A standard tool for analyzing an option position is the payoff diagram (often called a "hockey stick" diagram because of its common shape). Plot the option value as a function of the underlying asset price. Typically, what is plotted is...
  • Chapter 11

    Chapter 11

    Wealth used to support deceased in afterlife. New ruler compelled to acquire new wealth through conquest to ensure his one care in afterlife. Gender-specific roles in the home and in society. Emphasis on cooperation. ... Aztec trade and markets more...
  • joinup.ec.europa.eu

    joinup.ec.europa.eu

    Training Module 2.1Le cycle de vie des Données et MétadonnéesPubliquesLiéesOuvertes. PwC firms help organisations and individuals create the value they're ...
  • Clayfighter Pitch - Callum Rourke

    Clayfighter Pitch - Callum Rourke

    Bernie's Low attack will involve Bernie throwing a Liquorice projectile underarm like a bowling ball and it will slide along the floor and send the opponent on the floor. His High attack involves Bernie Bassett taking off his top hat...
  • What was a Roman Banquet like? Erebinthoi syn

    What was a Roman Banquet like? Erebinthoi syn

    The Titans vs The Olympians. The Titans ruled before the Olympians. Cronos was the leader of the Titans. Cronos ate his children out of fear that they would take over. One of his sons, Zeus, was hidden from Cronus and...
  • NCA Regional Debate: Peterborough Wednesday 28th September 2016

    NCA Regional Debate: Peterborough Wednesday 28th September 2016

    So on the face of it, the chances of being rated outstanding are minimal but there are things you can do….. www.qcs.co.uk. Betting person - wouldn't bet on something that had a 0.6% chance of return
  • USC Clinical Trials Office (CTO) Answers to CTOs

    USC Clinical Trials Office (CTO) Answers to CTOs

    Clinical Trial Initiation In True 2.0. PI/Research Coordinator responds to all questions and provides mandatory documents (i.e. study protocol, proposed Clinical Trial Agreement if provided, and sponsors first proposed budget) for CTO's review (please refer to True 2.0 training slides).
  • 700 BC - Discovery in Lydia (Turkey) of Schist

    700 BC - Discovery in Lydia (Turkey) of Schist

    700 BC - "Touchstone" A schist mineral in Lydia (Turkey) Standardized precious metals and alloys. Gave credibility to the king's mint. Provided a unifying effect on the state and permitted selective buying. Increased trade. * Lydian Coinage 500 BC *...