Designing and Implementing Usability Testing For E-textbook ...

Designing and Implementing Usability Testing For E-textbook ...

Designing and Implementing Usability Testing For E-textbook Webpage Enhancements DAVID COMEAUX AND EMILY FRANK LOUISIANA STATE UNIVERSITY #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana Agenda User experience and why it matters Our e-textbook initiative

Usability testing Tests Findings Lessons Learned #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana User Experience How we interact with things Designing products and services that are: Useful Usable Desirable

Keep in mind that: We are not the users Good user experience requires research and intentionality #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana User Experience Lots of techniques: Attitudinal: Surveys Focus Groups Interviews

Behavioral Usability testing Behavioral mapping / journey mapping Other Personas Card sorting #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana E-textbook Initiative Began buying books with licensing terms: DRM Free Unlimited user

Perpetual access Aligned them with courses Matching required textbooks Buying titles Promoted via webpage (the student page) Enable faculty discovery Collection of e-books that meet criteria Promoted via webpage (the faculty page) #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana Usability Testing Objectives:

Measuring Quesenberys 5 Es Process: Observe users as they use a product Apply lessons learned to improve user experience Iterative Testing: Test to learn from users Make changes Test again to confirm changes, identify additional issues #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana

Usability Testing Planning sessions Selecting tasks Preparing scripts IRB #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana Usability Testing Selecting location and recruiting participants Process varied by test Establish incentive (if any)

#LUCetexts October 5-6, 2016 Baton Rouge, Louisiana Usability Testing Performing test Greeting Consent form

Brief: We are testing the product, not you Pre-test Using script, ask user to carry out task Paper with scenario in case user needs reminder Encourage thinking out loud Take notes Where user got tripped up, quotes, subjective thoughts Benefit of 2 observers Questionnaire for quantitative feedback Debrief Provide incentive #LUCetexts October 5-6, 2016 Baton Rouge,

Louisiana Usability Testing Analyzing and Reporting Results Confirm understanding between observers What do we see? What does it mean? What should we do? Write report Apply fixes #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana Usability Testing

Guerrilla Usability Testing Approach people Public spaces Foot traffic Mix of people Similar model Introduce, tasks, observe, debrief, repeat #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana Our Testing Usability Testing is most effective as part of an iterative design process:

tests are done and potential improvements are identified improvements are implemented new tests are done, followed by implementing improvements #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana Our Testing Students Page Our first round of usability testing was done in the early spring of 2016. 5 students were tested - all were student workers We observed the students as they worked at the Circulation Desk computers Students were asked some pre-test questions (major and year) and post-test questions (impressions and suggestions)

Observer Worksheet Ebooks page in Fall 2015 #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana Our Testing Students Page - Round 1 Results Findings Students had some trouble finding the page Actions decided to add a spotlight image to draw attention at the beginning of the semester partnered with Moodle development team to include e-textbooks on course page as an alternate entry point

continued to promote program to raise awareness #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana Our Testing Students Page Findings Students wanted to search for all courses by department in addition to specific courses Actions we added a field labelled Dept Prefix, and included an example, to enable students to do this #LUCetexts

October 5-6, 2016 Baton Rouge, Louisiana Our Testing Our second round of usability testing was done in the early fall of 2016. The second round of testing focused more on students ability to use the books themselves to access content. LSUs test is part of a cross-university effort including several peer libraries also testing e-book platforms We tested e-books from two popular platforms; Springer and Project Muse. Other institutions will use similar methods and test these and other platforms #LUCetexts October 5-6, 2016 Baton Rouge,

Louisiana Our Testing Goals included: explore student interactions with e-books, specifically: Searching for information in an e-book Downloading a chapter Printing or copying identify obstacles to learning Sharing what we learned and possible solutions with publishers and vendors #LUCetexts October 5-6, 2016

Baton Rouge, Louisiana Our Testing Students Page A trial or beta test was performed; to test the test 4 students participated in the pre-test No major problems identified with test scenarios made some tweaks to post-test questionnaire #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana Our Testing

Students Page, Round 2 Methodology Target audience included all students, undergraduate or graduate Testing was done in the main lobby of the library Students were awarded a $5 gift card, and lured by candy 6 students participated Simple pre-test questions Several post-test questions Observer Worksheet #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana Our Testing Students Page, Round 2 Results

Prompt: In this book, find the chapter or chapters that discuss The Heaviside Class. Students used three paths to access the prompt information. #LUCetexts Two students used Search within this book to search for the prompt term. This action lead the students to a search results page, which listed basic metadata. Two students visually scanned through the Table of Contents listing on the main page, in search of an Index. Both noticed the heading labeled Back Matter and, with some hesitancy, clicked on it.

Two students used the Download Book option. This downloaded a pdf of the entire book. Both students then used the ctrl + f shortcut to search for the prompt term. October 5-6, 2016 Baton Rouge, Louisiana Our Testing Students Page, Round 2 Results Prompt: Find a map of West Tennessee in 1833. Students explored two paths to access the prompt information, only 1 was successful.. #LUCetexts Two students scanned the Contents list and identified the correct document with ease.

Two students scanned the list, but did not see the Maps header at first. They instead clicked on the Search Inside this Book tab, then entered search terms. Two students did not scan the Contents list, but instead chose the Search Inside this Book tab immediately. October 5-6, 2016 Baton Rouge, Louisiana Our Testing Students Page, Round 2 Results

Prompt: In this book, find information about the Fenian invasions. Students explored two paths to access the prompt information. #LUCetexts Three students visually scanned the Contents list, downloaded the index file, then searched the index for the prompt term. The other three students used the Search Inside this Book tab to search for the prompt term. October 5-6, 2016 Baton Rouge, Louisiana

Our Testing Students Page, Round 2 Results Behavioral Observations #LUCetexts Regardless of platform, when seeking information, patrons would often visually browse the books Table of Contents as a first strategy. Some commented on the lack of information on this page, and suggested that it would be helpful to include more information, such as a chapter summary, on the Table of Contents pages. Patrons had no difficulty with printing, cutting and pasting, or downloading chapters.

October 5-6, 2016 Baton Rouge, Louisiana Our Testing Students Page, Round 2 Results Post-test Questions #LUCetexts 5 of the 6 students reported using an e-textbook for a class. Only one was clearly found through our etextbook page. Others were from textbook vendors or obtained from friends.

Two mentioned having a preference for print. Two mentioned specific apps (Bluefire and Notability) for reading and note-taking tasks. October 5-6, 2016 Baton Rouge, Louisiana Our Testing Instructor Page, Round 1 Goals Goal was to assess the findability, ease of use, and satisfaction with the E-Books for Instructors page. #LUCetexts October 5-6, 2016 Baton Rouge,

Louisiana Our Testing Instructor Page, Round 1 Methodology 5 participants (2 Librarians, 3 Teaching Faculty) Tasks included: o finding the E-Textbooks for Faculty page o find a specific book (known title) o find a title of your choosing #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana Our Testing

Instructor Page, Round 1 Results Prompt: You heard through the grapevine that LSU Libraries has a page that allows you to search for books that can be assigned in your classes. Try to find the page. Results: 3 patrons found the page with little difficulty 2 got lost due to the E-books limiter in our search box #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana Our Testing Instructor Page, Round 1 Results Improvements:

Add slide to home page mid-semester Send communications directly to faculty Mass-mail postcards to teaching faculty Encourage liaisons to notify faculty #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana Our Testing Instructor Page, Round 1 Results Prompt: 1. Find the title Nutrition and Type 2 Diabetes: Etiology and Prevention 2. Request that this title be made available for your course Results:

All participants found the book with no difficulty Most had no trouble requesting the title #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana Our Testing Instructor Page, Round 1 Results Prompt: 1. Find a title within your subject area. 2. Request that this title be made available for your course Results: Some participants had difficulty finding a title Most had no trouble requesting the title

#LUCetexts October 5-6, 2016 Baton Rouge, Louisiana Our Testing Instructor Page, Round 1 Results Improvements: Changed search from Exact phrase to Contains all words Distinguished items we already owned from those that could be purchased Added Search Tips to no results screen Added explanatory text to emails #LUCetexts

October 5-6, 2016 Baton Rouge, Louisiana Our Testing Instructor Page, Round 2 Methodology 5 participants (4 Teaching Faculty, 1 Librarian) Rapid Iterative Testing (RIT) Tasks included: o finding the E-Textbooks for Faculty page o find a specific book (known title) o find a title of your choosing o assign the title to your class #LUCetexts October 5-6, 2016

Baton Rouge, Louisiana Our Testing Instructor Page, Round 2 Results Searching was successful; no changes made Labels were improved o Title or Keyword o Purchase this Book - Prompt LSU tp Purchase o Available Now - Assign o Links to assign or purchase made more prominent #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana

Best Practices and Lessons Learned Test frequently and regularly Just because a site or service is running doesn't mean testing should stop Incorporate feedback into the test: dont assume youve fixed the problems. Once youve incorporated changes, perform another round of testing. Test with a variety of types of users Represent the profile of your users Consider the role of distractions Distractions as authentic experience #LUCetexts October 5-6, 2016

Baton Rouge, Louisiana Best Practices and Lessons Learned Dont worry about recruiting a huge number of people Small samples good for revealing big usability issues As few as 5 participants is sufficient for a standard usability test Combine a variety of data collection techniques Our changes were informed by Google Analytics and usability testing #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana

Questions What role does user experience research play in your organization? Have you performed usability testing? Where could you apply usability testing? #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana References Barnum, C. M. (2010). Usability testing essentials: Ready, set-- test. Burlington, MA : Morgan Kaufmann Publishers. Rios, Jess. (2016, August 26). User experience resources. Retrieved from http://guides.library.harvard.edu/ux.

Schmidt, A., & Etches, A. (2014). Useful, usable, desirable: Applying user experience design to your library. Chicago: ALA Editions. #LUCetexts October 5-6, 2016 Baton Rouge, Louisiana

Recently Viewed Presentations

  • Infusion Therapy - Metropolitan Community College

    Infusion Therapy - Metropolitan Community College

    Extravasation - Inadvertent administration of vesicant solution or medication into surrounding tissues. Phlebitis - Inflammation of the vein's tunica intima. Thrombosis. Formation of a blood clot within the vein. Caused by injury to the cells lining the vein. Infection ....
  • Science Nights in Wake County

    Science Nights in Wake County

    Wiley International Magnet School: Science Night. Coordinated by PTA parents and teachers to volunteer. Includes hands-on experiences in all of the classrooms, student presenters, Broughton high school students, (assisting presenters), presenters (teachers, parent scientists, college faculty, community scientists, local organizations)
  • EDFGC 5809 Teacher as a Professional - WordPress.com

    EDFGC 5809 Teacher as a Professional - WordPress.com

    If you look closely at this drawing, Helen has drawn each person happy and smiling. She said, They are playing a ball game. They are all smiling because they are working together. Whatever is said in the circle stays in...
  • AUSOUG e-Training Webcast Managing Workflows Managing your Oracle

    AUSOUG e-Training Webcast Managing Workflows Managing your Oracle

    AUSOUG e-Training Webcast Managing Workflows Managing your Oracle E-Business Suite workflows as you would Aged Accounts Receivable By Gary Piper 13 November 2008
  • PowerPoint-Präsentation - inIPF

    PowerPoint-Präsentation - inIPF

    Recent clinical trials that served as a basis for the approval of two new medications for IPF differed regarding their inclusion criteria. The ASCEND study enrolled patients without a surgical biopsy only if they showed a definite UIP pattern on...
  • CSE 455 - homes.cs.washington.edu

    CSE 455 - homes.cs.washington.edu

    Assignment 1 is a big set of exercises to code functions that are basic and many of which are needed for future assignments. Sample functions are provided at the beginning of the code, so you get an idea how to...
  • Evropa - hradninadvori.cz

    Evropa - hradninadvori.cz

    Podle něj výhradní práva k dílu náleží autorům, nikoli tiskařům, jak tomu bylo doposud. Král Slunce - Ludvík XIV. Bourbon 1643 - 1715 = 72 let absolutismus kardinál Mazarin Colbert a merkantilismus 1653 zrušil Edikt nantský 1701-14 válka o dědictví...
  • The Effects of Gatorade and Salinity on Biofilms

    The Effects of Gatorade and Salinity on Biofilms

    Microtiter plate absorbance reader. Survivorship Procedure. 1. Staph. E was grown overnight in sterile LB Media. 2. The culture was added to fresh media in a sterile sidearm flask. 3. The cultures were placed in an incubator (37°C) until a...