DIY: Usability Testing for Business Analysts Pittsburgh IIBA,

DIY: Usability Testing for Business Analysts Pittsburgh IIBA,

DIY: Usability Testing for Business Analysts Pittsburgh IIBA, January 9, 2017, Carol Smith, @carologic Portions of this presentation were created with Thyra Rauch DIY Usability Testing for BA's - @carologic Usability Testing http://creativecommons.org/licenses/by-sa/2.0/ http://www.flickr.com/photos/raphaelquinet/513351385/sizes/l/in/photostream/ http://www.flickr.com/photos/raphaelquinet/ DIY Usability Testing for BA's - @carologic Usability Tests Measure the UIs ability to support users in achieving specific goals

Real users, doing real tasks Prototypes or live products Observed, not guided Image from Steve Krugs Rocket Surgery Made Easy DIY Usability Testing for BA's - @carologic It is not Testing the participants abilities Quality testing

Full accessibility testing System testing Acceptance testing DIY Usability Testing for BA's - @carologic Why Test? You are not your user May miss details; too close to design

Unforeseen requirements Validate understanding of tasks and context Improved UX = more satisfied users Less training needed Fewer services calls DIY Usability Testing for BA's - @carologic Test early to avoid mistakes http://www.flickr.com/photos/dwulff/12256075/sizes/m/in/photostream/ http://www.flickr.com/photos/dwulff/

DIY Usability Testing for BA's - @carologic Getting Started DIY Usability Testing for BA's - @carologic Design the Test Set a goal Type of participants Tasks for participants to attempt Share with team (if applicable)

Keep concise DIY Usability Testing for BA's - @carologic Create Tasks Representative of typical tasks Describe what, not how You want to buy a shirt for your father. Find a mans blue dress shirt for under $40. As a systems administrator for your company,

you want to install, configure, and test the xyz application. Avoid leading or matching UI wording DIY Usability Testing for BA's - @carologic Roles: Facilitator Welcomes participant Ensures consent is in place (NDAs, recording permissions) Moderates study Prompts for more information and manages frustration

DIY Usability Testing for BA's - @carologic Observers* Careful, meaningful notes Participants movements Interactions with interface Success/failures of UI and task completion Non-verbal communication (body language)

Participants comments Observers own inferences (noted as interpretations) Context, additional notes *See guidance for Observers in Resources section DIY Usability Testing for BA's - @carologic Think Aloud Share what they are thinking DIY Usability Testing for BA's - @carologic Pilot Study: Test the test

Critical Step Discover Problems with study Problems with concept/software being tested If tasks are typical (is this something users actually do?) Confirm estimated time for test

Refine script and tasks for test Get new ideas for follow-on questions or things to observe DIY Usability Testing for BA's - @carologic Planning Test Logistics DIY Usability Testing for BA's - @carologic Recruiting Participants Methods 2 weeks to recruit/schedule Social media

Paper postings at coffee places or in offices Email invitations and surveys Live recruiting DIY Usability Testing for BA's - @carologic Whats a lab? Do I need one? Comput er / Concept Participant Facilitator Observer Timer

Rubin, Jeffrey. Handbook of Usability Testing. John Wiley & Sons, Inc.; Logger DIY Usability Testing for BA's - @carologic No, but it is nice Anywhere Any Phase Anytime Participant observed through 2 way mirror and on screens Photo by Roebot at http://www.flickr.com/photos/roebot/2964156413/

http://www.flickr.com/photos/nzdave/491411546/sizes/o/in/photostream/ http://www.flickr.com/photos/nzdave/ DIY Usability Testing for BA's - @carologic Portable Labs Temporary setup Laptop, external mouse Capture screen, video and audio Anywhere

Conference room Waiting room Share screen for observation Photo by Roebot at http://www.flickr.com/photos/roebot/2964156413/ DIY Usability Testing for BA's - @carologic Remote Testing Online screen sharing software Capture screen, video and audio

Use separate phone line Test setup and connections ahead of time! Bandwidth, operating system, browser DIY Usability Testing for BA's - @carologic Interacting with Participants DIY Usability Testing for BA's - @carologic Welcome Make sure the participant feels welcomed and appreciated Emphasize you are not testing them, rather you are testing

Encourage feedback, positive or negative I was not involved in the design of this so you cant hurt my feelings DIY Usability Testing for BA's - @carologic Gain informed consent Your responsibility to inform participant Every time Ethical and legal responsibility Do no harm (physical, emotional)

Explain consent form Rarely lose participants over form DIY Usability Testing for BA's - @carologic Why Use a Script? Promote consistency in conducting the test Help remind you to welcome and thank people Help remember to reset/configure between participants

Ensures NDAs and other legal requirements are not missed DIY Usability Testing for BA's - @carologic Parts of a Script Welcome to participants Steps in test (forms, tutorials, tasks, questions) Thank you to participants (and incentives if any) Notes to yourself

Reset/configuration prompts DIY Usability Testing for BA's - @carologic Facilitation Remain passive (body, face) Ask open-ended questions and let them talk Avoid leading or giving clues Encourage to elaborate Probe dont demand

Gently prompt to think-aloud if necessary Let them tell you when they are done DIY Usability Testing for BA's - @carologic Example Phrases How does this compare with your expectations? What specifically are you referring to? What are you looking for?

DIY Usability Testing for BA's - @carologic Getting Participants Unstuck Its not a question of if but when Embarrassed at what they perceive to be their fault Not be able to articulate what they dont understand Dont directly answer initial questions Move on before frustrated, or take a break DIY Usability Testing for BA's - @carologic

Progressive Assistance What are you trying to do right now? What do you think the next step is? What would you do in this situation if you were at work? DIY Usability Testing for BA's - @carologic Hints Do you see anything that might help you? (small hint) Have you checked the Help? (medium hint)

What do you think the xxx button does? (large hint) DIY Usability Testing for BA's - @carologic Common Issues Says its great, but cannot complete task Watch what they DO, not what they SAY Takes control - diverts to their issue Show you are listening - writing it down

Refocus on study Sometimes you need to end the session Thank and give incentive DIY Usability Testing for BA's - @carologic Activity! DIY Usability Testing for BA's - @carologic Test an item in the room Break into teams of ~3 Select item to test

Create a task E.g. Using this phone set an alert for 45 minutes DIY Usability Testing for BA's - @carologic Light Script Five minutes in total Introductions Explain what youll be doing Gain consent from participant to participate in your study

Task Thank you for participating 33 DIY Usability Testing for BA's - @carologic Run the study Identify Roles Facilitator Observer

One team member rotates Conduct study with new team member as participant (5 minutes) Participant moves to next study Run again 34 DIY Usability Testing for BA's - @carologic Discussion How did it feel?

What surprised you? 35 DIY Usability Testing for BA's - @carologic Improving Products with Usability Testing DIY Usability Testing for BA's - @carologic Can Test Websites, mobile apps, blenders, airport service Clickability is recommended Simulations or mockups

Prototypes Help documentation Processes (receipt of materials, purchase) DIY Usability Testing for BA's - @carologic Tweak, Dont Redesign Small iterative changes Make it better now

Dont break something else Take something away Reduce distractions Dont add question it Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems. By Steve Krug DIY Usability Testing for BA's - @carologic Focus on Serious Problems All interfaces have usability problems

limited resources to fix them more problems than resources Less serious problems distract Put intense focus on fixing most serious problems first Adapted from: Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems. By Steve Krug DIY Usability Testing for BA's - @carologic How to Focus Identify top 5 or 10 most serious issues

Top 3 from each observers list Prioritize combined list Commit resources for next effort Stop Adapted from: Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems. By Steve Krug DIY Usability Testing for BA's - @carologic Make Useful & Usable Recommendations Quickly DIY Usability Testing for BA's - @carologic Just Do It!

Anywhere (conference room, remotely) Any Stage (earlier in process the better) Anytime (remote, un-moderated) Start small Test an idea (not a feature) A few participants Photo by Roebot at http://www.flickr.com/photos/roebot/2964156413/

DIY Usability Testing for BA's - @carologic Recommended Readings DIY Usability Testing for BA's - @carologic Contact Carol in/CarolJSmith @Carologic slideshare.net/carologic DIY Usability Testing for BA's - @carologic Resources DIY Usability Testing for BA's - @carologic Myths of Usability Testing Need a lab

Testing occurs at the end of the dev cycle Must test everything Need to test 100 people A good test means no changes Usability is system testing, user-acceptance testing, etc. Need a large test team, each playing

a specific role/part DIY Usability Testing for BA's - @carologic Usability by the numbers 80% of SW life cycle costs occur during maintenance 80% of maintenance costs are due to unmet or unforeseen user requirements Only 20% are bugs or reliability problems Typical software program released with 40 usability design problems

Over 70% of CIO's state that one of their biggest problems is communicating with users to understand their needs DIY Usability Testing for BA's - @carologic Number of Participants Contentious issue Nielsen - 5-7 participants uncover ~80% of major usability problems IF representative IF doing representative tasks.

Testing more = significant diminishing return. Virzi and Carol Barnum Number of Test Participants Small number of users is sufficient. http://www.useit.com/alertbox/20000319.html DIY Usability Testing Jakob Nielsens Alertbox. Why You Only Need to Test with 5 Users. March 19, for BA's - @carologic Recruitees Could be

Customers General consumers Members of organization or company Difficulty of recruiting Specificity of screener Availability/interest in study (doctors vs. consumers) Experience (highly experienced vs. novice)

DIY Usability Testing for BA's - @carologic Recruiting Considerations Accessibility of test location Include people with disabilities We are all only temporarily able-bodied. Accessibility is good for us all. Language interpreters Remote and on-site studies Technology availability

Administrative rights Internet connectivity Quote by @mollydotcom at #stirtrek 2011 via @carologic DIY Usability Testing for BA's - @carologic Data Collection Forms Customize for each study Space for writing notes each scenario spelled out.

Include space for tickmarks regarding easily measurable activities. E.g. Participant accesses video clip E.g. Participant selects advertisement DIY Usability Testing for BA's - @carologic Tool Considerations In person or remote? Lab or on-site? Prototype limitations (can it be online?, is it a document or a clickable site?)

Number of observers, number of participants? Number of facilitators? Logging and video editing needs (time on task, highlight video creation)? Surveys before or after? Eye tracking? DIY Usability Testing for BA's - @carologic Study Measurements

Success % of tasks completed/not completed Number of steps to accomplish task Learning time Number of errors Number of times needed help

Satisfaction Time on task Photo By Kenyaboy7Andrew Chipley http://www.flickr.com/photos/kenyaboy7/3749535540/ DIY Usability Testing for BA's - @carologic Interview & Observation Notetaking Form Date: __________________ Time: __________________ Location: ________________ Note-Taker: ______________ Activity ______________________________________ Who are you observing? (e.g. SME in soft-goods manufacturing, application admin) What is the situation? ___________________________ (e.g. phone interview, their work at their desk in person, remote usability study, etc.)

People (P) Objects (O) Environment (E) Messages (M) Services (S) Who are you observing? Who else is involved in the experience? What objects do they interact with? What apps? Phone, etc.? Characteristics of the setting, number/types of interruptions What is communicated?

How is it transmitted? What services are available to them? Physical What are your biggest takeaways? Cognitive What did you learn? What surprised you? What is the participants need? What do they interact with? How associate meanings? How learn? Social Interactions, decision making, scheduling, work? Cultural

What are shared norms, habits, values? Emotional What emotions are expressed and how? Watch for these types of observations and put additional Questions/Ideas on the back. DIYinUsability Testing for BA's Content in this document described by Vijay Kumar his book: 101 Design Methods @carologic Observer Rules Everyone who observes a design session is asked to abide by a set of rules. The purpose of these rules is to minimize stress for the participants and to maximize the amount of information we get from the study. Stay for the entire study Distractions are unhelpful and participants may get the impression that youre leaving because theyve

done something wrong (e.g. walking out in middle of a movie). If you can attend only part of a study, discuss with the facilitator beforehand to determine whether there is a way to accommodate this. Dont reveal information about the study to the participant It is often more useful to explore an area of difficulty in detail rather than try to get through all the topics. The facilitator will track time so that we can cover as many of the important areas as possible. Respect participants and the confidentiality of their data We have promised the participants that their participation is confidential. This means that we should not include their names in any reports or other communication such as email, and we should refrain from discussing them by name outside the test setting. Do not make negative comments about peoplethere is always a risk that a derogatory comment could

be overheard or otherwise make its way back to the user. Adapted from the book Paper Prototyping by Carolyn Snyder, published by Morgan Kaufmann Publishers. Copyright (c) 2003 Elsevier. All rights reserved. DIY Usability Testing for BA's - @carologic Observer Rules Remain silent and silence phones (continued) You may notice something so surprising that you are tempted to laugh or exclaim out loud. This is not unusual. Unfortunately, participants might think you are laughing at them. Keep as quiet as possible. You will have opportunities to ask questions at the end. If necessary, pass a note to the facilitator. Observe only - no questions or support

If reviewing an interface, its likely participants will have problems, and it is normal to feel a temptation to help. Please dont. Instead, try to understand why it was that the user got stuck or went down the wrong path. Its the facilitators role to get users back on track if they get really stuck. And if the facilitator poses a question during the test, he or she is asking the users, not youplease dont answer unless the facilitator specifically directs a question to you. Avoid Design Questions - Questions that ask the user their opinions about how to design aspects of the application (such as, Where would you like to see these navigation buttons?) can take a lot of time to answer and produce only limited results. Instead, focus on trying to understand the problemwell come up with solutions later, outside the test. In person, be conscious of your body language Although most studies are interesting, not every moment will be fascinating. If something is happening that isnt of interest to you but may be to others, sit quietly without fidgeting. Take notes to stay alert. Adapted from the book Paper Prototyping by Carolyn Snyder, published by Morgan Kaufmann Publishers. Copyright (c) 2003 Elsevier. All rights reserved. DIY Usability Testing for BA's - @carologic

Ethics Resources Treatment of human subjects Web-based course Gives certificate of completion http://cme.cancer.gov/clinicaltrials/learning/humanparticipant-protections. asp Guidelines for writing informed consent http://ohsr.od.nih.gov/info/sheet6.html

UXPA Code of Conduct DIY Usability Testing for BA's - @carologic NDAs Do I need a non-disclosure agreement (NDA)? Maybe Legal department (be friendly) One-way, two-way* AKA

Confidentiality agreement (CA) Confidential disclosure agreement (CDA) Proprietary information agreement (PIA) Secrecy agreement (SA) DIY Usability Testing for BA's - @carologic References & Further Reading Cato, John. User-Centered Web Design. Addison Wesley Longman; 2001.

Gaffney, Gerry. (2000) What is Card Sorting? Usability Techniques Series, Information & Design. http://www.infodesign.com.au/usabilityresources/design/cardsorting.asp Hackos, JoAnn T., PhD and Redish, Janice C. User and Task Analysis for Interface Design. Wiley; 1998. Henry, S.L. and Martinson, M. Evaluating for Accessibility, Usability Testing in Diverse Situations. Tutorial, 2003 UPA Conference. (Activity) Krug, Steve. Don't Make Me Think: A Common Sense Approach to Web Usability. Krug, Steve. Rocket Surgery Made Easy: The Do-It-Yourself Guide to Finding and Fixing Usability Problems Kuniavsky, Mike. Observing the User Experience: a Practitioner's Guide to User Research. Morgan Kaufmann, 2003.

Mandel, Theo. The Elements of User Interface Design. Wiley; 1997. Nielsen, Jakob and Robert L. Mack. Usability Inspection Methods. John Wiley & Sons, Inc. 1994. Powell, Thomas A. The Complete Reference: Web Design. Osborne/McGraw-Hill; 2000. Redish, Janice (Ginny). Letting Go of the Words: Writing Web Content that Works. Rubin, Jeffrey and Dana Chisnell. Handbook of Usability Testing: How to Plan, Design, and Conduct Effective Tests. John Wiley & Sons, Inc. Snyder, Carolyn. Paper Prototyping. Morgan Kaufmann Publishers. 2003 Elsevier. DIY Usability Testing for BA's - @carologic

Recently Viewed Presentations

  • La sant des enfants du Canada : Un

    La sant des enfants du Canada : Un

    Parmi toutes les provinces, c'est au Québec que les familles comptant sur un seul revenu avaient le revenu après impôt le plus faible (46 500 $) et c'est en Alberta que les familles comptant sur un seul revenu ont le...
  • Habit 1 - Community Unit School District 308

    Habit 1 - Community Unit School District 308

    Habit 1, Being Proactive, is the key to unlocking all the other habits, that's why it comes first. Habit 1 says, "I am the force. I am the captain of my life. I can choose my attitude. I'm responsible for...
  • 3.4 Rates of change - Oxford School District

    3.4 Rates of change - Oxford School District

    Rates of change. Example 1: Find the rate of change of the Area of a circle with respect to its radius. Evaluate the rate of change of A at . r = 5. and . r = 10. If ....
  • Tool 13 Short Form "Statistical Foundations for Data Based ...

    Tool 13 Short Form "Statistical Foundations for Data Based ...

    Lean, Six Sigma, Quality Transformation Toolkit Phase II Final Report Lean Six Sigma Improvement Project: Conducted for Chilco Presented to: Dr. John Sinn Prepared by: Team 682 Summer 2007 Project Team The project team is comprised of Ph.D. students and...
  • El Fútbol project - ZIMFLEX

    El Fútbol project - ZIMFLEX

    To be an independent enquirer. To be creative and innovative in my work. To manage my time and resources. Your task: You are going to complete your own French Football Album. Just follow the step by step process. You will...
  • Avaya Servers Upgrades & Migrations

    Avaya Servers Upgrades & Migrations

    Endpoints: Avaya scored a perfect 20 out of 20 points getting praise for its impressive range of endpoints including IP hard phones,multiple softphone options and "Splashy and unique" features including Extension to Cellular, Seamless Communications and point to point video...
  • Sheet Music - Murdoch University

    Sheet Music - Murdoch University

    Dimensional Salience. r = .87. Prince, Thompson, & Schmuckler (2009) Existing work proposesdimensional salience is important factor-when equated for discriminability (difficulty of perceiving), one dimension dominates the other
  • Impact Investing and Catholic Social Teaching

    Impact Investing and Catholic Social Teaching

    IMPACT INVESTING AND CATHOLIC SOCIAL TEACHING REV SÉAMUS P FINN OMI "Investing for the Poor: ... and redistribution of resources and assets that are essential for life has long been considered a hallmark of the Christian/Catholic tradition. Beginning with the...