#UTwebinar
UserTesting’s on demand customer insights platform enables companies to create powerful products and exceptional customer experiences.
Across All Platforms, At All Stages Of Development
Watchcustomersinteractwith:
DESKTOP SMARTPHONE TABLET
WEBSITES APPS&PROGRAMS
PROTOTYPES&WIREFRAMES
SURVEYS PHYSICALPRODUCTS
Anywhere people interact with your brand:
ATHOME ATASTORE ONTHEGO
On any pla6orm or device:
#UTwebinar
Traditional Testing
2 to 3 Weeks 2 to 3 Weeks 1 Week
Results In Hours, Not Weeks. Save Time. Save Money.
#UTwebinar
Plan & Recruit Run Study Analyze Data & Write Report
1 Day 1 Day
Analyze Data & Iterate
Plan, Recruit & Run Study
Within Sprint 2
Days
4-7 Weeks
Agenda
• Welcome
• IBM Design Thinking
• Watson Design Team
• Agile
• Usability Testing
• Q&A
Adam Rector Webinar Manager, UserTesting @iamadamrector
Carol Smith Sr Design Manager, IBM Watson
#UTwebinar
@carologic
8
Carol Smith Sr Design Manager, IBM Watson
#UTwebinar
Carol Smith conducted her first usability study in 2000 and has been researching to understand user needs and then advocating for improved designs on their behalf since then. She has led projects for the US government, non-profits, and corporations in a wide variety of industries. She currently manages a team of designers at IBM, creating and improving Watson software products. She has presented over 70 talks and workshops around the world and has collaborated with luminaries in the field on a variety of topics.
@carologic
Carol Smith • User Experience
• Mix of consulting and in-house • Government, non-profits
and corps in many industries • Usability testing, research, interaction design • Web, software, mobile
• MS, Human-Computer Interaction, DePaul University, 2002
• Sr. Design Manager at IBM Watson, Pittsburgh, PA
• My thoughts – not those of my employer • Twitter: @carologic
IBM
• 105 year old company “Big Blue” • Continuous reinvention • Patents: ATM, floppy disk, relational
database, SQL, UPC barcode • Over 170 countries • Nearly 380,000 employees • 100’s of Designers
https://en.wikipedia.org/wiki/IBM and https://en.wikipedia.org/wiki/IBM_Selectric_typewriter
A focus on useroutcomes
Multidisciplinaryteams
Restlessreinvention
IBM Design Thinking © 2016 IBM Corporation | IBM Confidential
Principles
Hills Playbacks Sponsor users
IBM Design Thinking © 2016 IBM Corporation | IBM Confidential
The Keys
Watson Design Team (~60 Designers)
Explore & Discover (8)
Management & Guild Leads
Director Watson Design Director
Carol
Researchers Ix Designers Viz Designers
Staffing • Balance of skills and experience • Provide mentoring and guidance to less experienced folks • Best designers working on toughest problems • Focus on skill set
• Enable team to do what they enjoy and practice skill • Encourage shadowing of others to learn new skills • Encourage depth and T shape
The Work • Individuals focus on one primary product/problem at a time
• Secondary short projects • 10% time is available • Tuesdays are ‘No Meeting’ day
• Management facing constant change • Multiple projects • Customers around the world • Big challenges
Remote Research • Interviews • Observations
• Limited, but helpful • Usability testing
• Prototypes and working software • Moderated through screen shares • Unmoderated with usertesting.com
• Card Sorting and/or co-creation activities (limited)
Ask for artifacts! hHp://www.flickr.com/photos/camknows/viahHp://creaOvecommons.org/licenses/by-nc-sa/2.0/
In Person Research • At customer’s location
• With sales or product owners (Offering Managers) • Initial meetings are typically to gain trust
• Return to do additional work • Interviews • Observations • Usability Testing • Co-Creation activities
• Limitations • Recording can be very difficult • Access only to whom they approve
Photo: http://koin.com/2016/07/12/how-to-protect-yourself-from-computer-camera-hackers/
Security
Conferences and Events • World of Watson • Design Studio Sessions
• Listed as Feedback Sessions • Use social media and signage
to attract attendees
• Primary Goal: Meet many customers and build relationships for future work.
• Secondary Goal: Learn as much as possible through research activities.
Workshops • Start with Design and product owners (Offering Management)
• Incubator phase • Create Concept Cars and determine value • Review with Development teams
• Collaboration sessions • Design, Development and Offering Management (QA and Content as well)
Remote Work and Collaboration • Screen sharing
• SmartCloud, WebEx, GoToMeeting, etc. • Sharing ideas
• Virtual post-its - Mural.ly • Real-time co-editing (Google Docs, Box) • Document cameras (higher quality)
This is only a small sample of available tools. Endorsement is not implied by inclusion here.
Co-located and international teams • Communication
• Skype, Google Hangout, Appear.in • Chatting
• Slack • WhatsApp desktop version
This is only a small sample of available tools. Endorsement is not implied by inclusion here.
IBM Design Thinking and Agile • Emphasize focus on:
• Team collaboration • Continuous learning • Outcomes
• Sprints or Kanban • 3 week cadence • Demos near end • Daily squad standups • Weekly cross-team reviews
• Usability test should take no more than 2 sprints to complete
Agile Manifesto
Full version: http://agilemanifesto.org/
Individuals and Interactions Customer Collaboration
over processes and tools over contract negotiation
Working Software Responding to Change
over comprehensive documentation over following a plan
We have come to value the items in the dark boxes more.
Agile and Prioritizing Work • Things move very quickly • Constant re-prioritization of work
• Offering Management sets overall priority (fairly stable) • Teams pick what to work on
• Dependencies • What we learn through research • What we learn through dev process • Time available (if no time to implement, don’t do the research) • Staff availability (other projects, personal leave, etc.) • Customer availability
Agile Frustrations via Spotify
Scaling Agile at Spotify via Slideshare of Vlad Mysla http://www.slideshare.net/vmysla/scrum-at-spotify?qid=2345c3ad-7e68-4383-9673-9e715ff47a75&v=default&b=&from_search=14
squad member
Agile UX: The Good • Most important functionality is done first • Working together – not “over the wall” • Keep up with technology and environmental changes • Enables iteration of requirements • Less “design drift” and less wasted design
Agile UX: The Good (continued) • Issues get fixed • “Done” includes design • Satisfying to see designs in real use • Learn from actual product use • User data has effect on current release
Be Part of the Team • They’re not going to “stop the train” for you • Make UX processes Agile • Build trust by providing value of work • Attend daily standups/scrums
Design Adapts to Agile • Falling short of end goals is a constant
• Focus on meeting user’s primary needs • Constant Improvement is key
• Widen the Design family • Distribute the work • Empower entire squad to inform design
• Agile is reactive – no time for predictive work • Prepare to react
Adapted from Jim Laing’s presentation at UX Pittsburgh, May 2014
Moderated Usability Testing • Everyone is invited to observe
• Separate viewing room and project to a screen • Development, offering management, QA, content, etc. • Box note to track progress within Design team • Slack channel for larger group discussion and to ask questions
• Discussions start during the study • Team gets excited – especially those that are new to usability testing • I have mixed emotions – sometimes wrong emphasis - but at least they
are communicating!
Moderated Usability Testing • Raise and/or discover significant issues through usability testing
• Hard to argue with what you see • Have to more carefully vet our participants when testing high visibility
functionality • Have had situations with intense scrutiny – which we welcome!
• Create highlight reels of most important topics • Optional editing – frequently we tell them at what time marker to watch
Unmoderated Usability testing • UserTesting.com
• Clickable prototypes and working software • IBM Bluemix for hosting
• One participant completed and watched before inviting more • Find issues with study design • Determine if getting desired results • If not, edit and re-start
• Small studies • Up to 5 participants • Review and repeat as needed
Remote Tool Considerations • Prototype
• Must be online (impossible to test if not online) • Document or clickable site/prototype
• How is recruiting handled? • How selective can I be? • Can I provide my own list?
• Limited number of observers? • Limited number of participants?
44
Remote Tool Considerations (cont) • Typically only one facilitator • Logging and video editing needs met?
• Time on task, highlight video creation • Ease of use
• Able to provide surveys before or after?
45
Towards Building Teams • Hire great people • Balance skills and experience • Enable to work to their strengths • Encourage to grow, attend webinars, conferences, etc. • Make time for 1:1’s
Shared Documents Aren’t Shared Understanding
CartoonbyLukeBarreH©JeffPaHon,allrightsreserved,www.AgileProductDesign.com
IBM Health Corps • Three weeks, completed entire software project • Partnered with American Cancer Society • Built a tool to help Ministries of Health in low-income countries get
access to more complete and cheaper therapies for cancer.
Blog: https://ibmhealthcorpsblog.wordpress.com/2016/09/29/delivering-high-value-in-a-short-time-colocation-agile-and-design-thinking/ Video: https://www.instagram.com/p/BK9N5K_jEN_/?taken-by=ibmhealthcorps
World Community Grid – unused computing
Web Site: https://www.worldcommunitygrid.org/discover.action
Top Related