Matthew Hale, PhD

Jeff Sauro and James R. Lewis, could you help clarify why it’s recommended to report the frequency of users who encounter specific issues (e.g., problem_a, problem_b, etc.) in iterative formative usability tests, particularly with small sample sizes (e.g., 4 out of 6 participants experienced problem_a; 2 out of 6 participants experienced problem_b)? From what I understand, using the frequency of issues from a small sample in a single formative usability test isn’t reliable for (1) generalizing the likelihood of occurrence across the broader population, or (2) prioritizing which problems are more “important” based on their frequency in the test. This is especially true since other factors, like severity, can be equally or more important, and as you’ve outlined in your work, severity is independent of how often a problem is encountered. Is this a correct assumption? I’d appreciate any insights on what I might be misunderstanding I understand the binomial probability formula, as explained in Quantifying the User Experience, is helpful for estimating the likelihood of problem detection in terms of minimum probability, likelihood of detection, and the cumulative likelihood of meeting discovery goals. However, I’m stuck on this point: If we set the minimum number of detections to 1, doesn’t the actual probability of occurrence in the broader population for a small sample usability test remain unknown, even if 4 out of 6 participants experienced problem_a? This doesn’t necessarily imply that problem_a occurs more frequently than problem_b, which was experienced by only 2 out of 6 participants, right? I realize the binomial probability model isn’t meant to be used deterministically, but I’m struggling to understand why we report issue frequencies within small formative usability samples when operating iteratively. From my experience, doing so can lead stakeholders to prioritize issues based purely on their frequency in a given round, even when other criteria should be considered Wouldn’t it be more useful to frame it differently, such as saying, “All of these issues are likely to affect 31% or more of users,” assuming we use a probability of .31, aim for an 85% chance of detecting the issue, and set the minimum number of detections to 1? Then we could apply additional evaluation criteria, such as David Travis’ problem severity framework, for prioritization, among other framework Can you help me understand what I’m missing or misunderstanding?

9

6 Comments

On the buildo blog, Sara Bianchini explains how you can ‘Discover how usability tests with users who have #Disabilities can enhance your product’s #Accessibility …’ https://lnkd.in/eDGfYS45

2

If your customers are complaining, are in pain, or are angry, your best action is to listen to what they have to say with empathy and patience. Then use what you learn to make your product or service better. This is standard research practice. This is what we counsel the companies we work with as user experience professionals. That's not the feeling I get from this post—especially the part that can't help but be taken to suggest some light suing.

10

2 Comments

Effective icons depend on recognizability and interpretation. Evaluate them with methods appropriate for your specific research questions. #userresearch #icons #iconography

6

Earlier this year, I had the privelege of moderating a panel of industry execs from ServiceNow, Google, Netflix, Nvidia along with the faculty at Savannah College of Art and Design to understand the implications of AI and its impact on the design industry. The outcome of the panel has just been published by SCAD in the AI insights book which I’d highly recommend giving a read. SCAD has always been at the forefront of disruption and bridging the gap between academia and industy. This was one of a kind summit to bring industry and academia together to think about AI, it’s state in the industry and how we can prepare students better in this changing landscape. https://lnkd.in/gzM9Xf8h

61

#BadassBookAlert Jeremy Miller’s Beyond UX is a compelling guide that transcends the surface-level practices of user experience (UX) design and dives into the deeper strategies that shape impactful, human-centered solutions. Miller positions UX as more than a set of tools or methods—he frames it as a strategic discipline that can drive organizational change and innovation when implemented effectively. The book is rich with actionable insights, blending theory with real-world examples that resonate across industries. Miller emphasizes the importance of aligning design practices with business goals, fostering cross-functional collaboration, and embedding UX thinking into organizational culture. His writing strikes a balance between accessible advice for beginners and nuanced strategies for seasoned professionals. One of the standout aspects of Beyond UX is its practical approach. The frameworks and methodologies outlined are not only clear but adaptable to varying levels of organizational maturity. Miller’s focus on metrics and outcomes ensures that readers understand how to measure the impact of their UX efforts—a critical skill for gaining executive buy-in. Jeremy also delves into the often-overlooked dynamics of team collaboration, highlighting how empathy and communication can break down silos and enhance the effectiveness of design processes. His emphasis on storytelling and advocacy equips UX practitioners to champion their work within the complexities of corporate environments. Overall, Beyond UX is an essential read for UX professionals looking to level up their strategic impact. By addressing the "why" and "how" behind effective UX, Miller provides a roadmap for moving beyond the tactical execution of design work into the realm of business and organizational transformation. "Great UX isn't just about pleasing interfaces—it's about aligning design with purpose, bridging silos, and proving that empathy is a business strategy, not just a buzzword." —Jeremy. Ah, you're speaking my love language sir. Don't just take my word for it Tom Greever did the forward and says nice things too. And Jeremy didn't even pay either of us. But he did let me write a little bit about Power Mapping and Stakeholder management in it. Buy it now and Krampus won't come get you. It's the perfect stocking stuffer for anyone who cares about experiences. In case you're too cheap to buy it, you can sample my bit in the comments. #UX #Innovation #Design #Craft

98

41 Comments

More UX and Product leaders need to hear Diana DeMarco Brown (she/her)'s wisdom. Diana is a pioneer in the UX field who takes a thoughtful and nuanced approach to assessing the current state of the tech industry. In our recent conversation, Diana shared many insights, including: - How to evaluate a company's culture during the hiring process - Why private equity ownership can result in design divestment - How Product and UX leaders can effectively influence business decisions - Why we don't need to panic with the rise of AI, but we need to think critically and prioritize regulation during this early stage Link to our full conversation: https://lnkd.in/eemDimVb

42

6 Comments

Episode 237 of The World of UX is now available! In this potpourri segment, Darren addresses four topics: 1) Clarifying skill needs for those interested in UX freelancing or consulting; 2) an overlooked critical need for those building an in-house UX team; 3) identification and management of strawman discussions and 4) the need to view UX job postings with a critical eye. Enjoy! https://lnkd.in/eQRBjRDV

6

1 Comment

✨ I'm so excited to see my blog post on Neurodiversity and UX shared by UXPA International! 🙌 Writing this was such a personal and eye-opening journey. It got me thinking about how we can design experiences that truly work for everyone—especially for neurodivergent individuals. This is just the start of a much-needed conversation about accessibility, empathy, and breaking down barriers in design. A big thank you to UXPA International for sharing my work and giving me the chance to contribute to such an important topic. 🌟 👉 Here’s the post: https://lnkd.in/gf6Ti9Z8 I’d love to hear your thoughts—have you ever designed with neurodiversity in mind? Let’s chat and learn from each other! 🤝 #UXDesign #Neurodiversity #Accessibility #InclusiveDesign

29

9 Comments

Join me at CreativePro Design + Accessibility Summit! https://lnkd.in/gDjet68S Explore the foundational principles and guidelines of accessible web design. Learn how to create web experiences that are inclusive, user-friendly, and compliant with accessibility standards, ensuring equal access for all users. Key Takeaways: - Understanding of key principles and guidelines for accessible web design - Techniques for optimizing web content for accessibility and usability - Best practices for creating accessible websites and digital experiences

40

Looking for inspiration for your next user research report? Check out these strategies (and template!) that I put together. 😊 📖

4

Among the things I appreciate in this piece on accessibility (#a11y) testing and research… * De-risking projects by frontloading research instead of pushing for testing at the end. * Addressing limitations of Figma-based approaches. * Advocating for co-design and other research methods which go beyond mere testing (or survey-based analysis). * Acknowledging issues with some prototyping methods. * Embedding accessibility in a broader frame for inclusion. The first and last points have been on my mind for years (and I was mentioning them in a public conversation on aging at Concordia University’s University of the Streets Café). At some point, we need to find ways to “frontload all the things”. From Gender-Based Analysis Plus (GBA+, which includes aging and indigenous status) to usability, from accessibility to cybersecurity, from copyright to feasibility, from Environmental Impact Assessment to language support and localization (#l10n). Sounds like a tall order… so many of us propose approaches which take all sorts of things into account through #SystemsThinking and, in the case of #UXresearch … co-creation and #ethnography. What this requires most is to avoid “converging too quickly”. It’s still important not to “diverge too long”. The tipping point is easier to find when diverse stakeholders are involved at key steps, early in the process. Not just users, specialists, customers, and “decision-makers”. People who have a stake in the project, even if their influence is low. In the context of many #LivingLabs, such approaches are standard procedure. And it helps push for a solid embedding in community-driven innovation. So, yes, let’s do actual research on accessibility through every step of the process behind any project where humans will access the things we create. We can also establish research (protocols and all) as a core function which integrates other aspects of the service or product we create together. And while scale does make a huge difference on the way this is done, there are ways to undertake very small projects which hit all those notes, with commensurate effort. https://lnkd.in/eT56yN5D

1

User research no longer requires paperwork, reduction act clearance! If you’re not in the federal government, you maybe don’t understand the significance of this decision, but it’s the reason why so many government services seem to pop up without any acknowledgment of how to use them or how people would like to use them. It’s because we simply haven’t been able to ask before!! Now, that all changes. Finally, finally, finally government designers and dove teams can talk to users about products and services before they ship and while they function. This is huge, huge news, and I’m so grateful to everyone who worked on this!

27

Methodology is a key pillar of any research project. Back in October 2020, we had Meghan Rennie join UXRS to talk about how you learn is key to what you learn. Bringing over 30 years experience in design, she talked about the importance of identifying what you know, what you don’t know, and selecting the appropriate research method to answer your question. Check out a replay of this talk on our YouTube channel: https://lnkd.in/g7EG2_Sq

3

How Feminist Theory Informs My Approach to UX Research ✨ As I dove deeper into UX research, one idea from feminist theory that deeply resonated with me was the resistance to essentialism—the belief that there’s no single, universal experience for any group of people. This concept has transformed how I approach user personas, user testing, and the design process. Feminist theory challenges us to reject oversimplified categories, just as UX asks us to rethink the notion of a "typical" user. By applying an intersectional lens to UX research, we can better understand how overlapping identities—such as race, gender, class, and ability—influence user interactions and experiences. This leads to more inclusive, accessible solutions that reflect the full diversity of human experiences. Bridging feminist theory and UX research has enriched my approach to design, ensuring we move beyond the "average" user and create for a broader spectrum of people. 💡 Curious to learn more? Check out my full article on Medium, where I dive deeper into how these principles guide my work in UX research.🔍 Have you found insights from other fields that have shaped your thinking about UX? I’d love to hear your thoughts. #UXResearch #FeministTheory #Intersectionality #InclusiveDesign #UserExperience #Essentialism

7

Resharing my appearance on the UX Minds Podcast. I talk about how I got into UX, my general thoughts on the field, and share practical advice for anyone new to UX. Check it out! https://lnkd.in/erpT4kgQ

8

Why AI in User Research? Incorporating AI in UX research can potentially drive research productivity. Experimenting with new tools and methodologies keeps UX researchers (and those wanting to or doing user research) at the forefront of the field, enabling you to explore new possibilities and stay competitive. Embracing AI-assisted tools is crucial for staying relevant. Figure this out, or somebody else will for you (like your Product team). This workshop aims to provide practical skills and an understanding of using AI tools to gather and interpret user data. This includes understanding user needs, preferences, and pain points with greater precision. These insights can inform design decisions, leading to more efficient data analysis and stakeholder decision support. https://lnkd.in/eBC6xSUH

9