Insights Unlocked

Episode web page: https://bit.ly/3V1gzkD

Episode summary:

In this episode of Insights Unlocked, Lija Hogan, principal experience research consultant at Auros, chats with Meg Roebling, director at Citi, where she leads user experience for consumer credit cards across the bank’s retail partners, about AI in UX research and why human judgment still matters as research gets faster. Drawing on more than 20 years in digital design, much of it in financial services, Meg shares what has changed about research since the dot-com boom and which battles researchers are still fighting.

Meg explains that the case for research has been won in principle but still loses to competing priorities, and why tying it to the funder's own goal, such as risk mitigation at a bank, is "usually a winning argument." She walks through her start-small approach: answer a simple question quickly, show the win, become the go-to person, then tell the story with a verbatim clip that makes a user's struggle real for stakeholders. She also lays out where AI synthesis falls short. It surfaces themes but misses nuance, contradictions, body language and important outliers, flattens results and still hallucinates.

The conversation also touches on designing trustworthy AI experiences in financial services, from chatbots that paper over poor information architecture to why screens still matter for complex, multi-step flows. Meg makes the case for transparency and enhanced security as people hand purchases to AI agents, explains the personalization bargain most customers accept, and argues that teams need to start designing for agents now. Lija and Meg also discuss keeping humans in the loop at every gate of an AI-driven product development lifecycle, the central agent registry at Citi, and Meg's updated advice for leaders: match your approach to your organization's research maturity and invest in measurement and rapid research.

You'll learn:

Why research still loses to competing priorities, and how to tie it to the funder's goal

How small, fast wins and verbatim clips build buy-in for research

The gaps in AI research synthesis that humans still need to catch

Why chatbots shouldn't cover up poor information architecture

Principles for designing trustworthy AI experiences in financial services

How to keep humans in the loop in an AI-driven product development lifecycle

How to match your research investment to your organization's maturity

Resources & links

Meg Roebling on LinkedIn (https://www.linkedin.com/in/megpullisroebling)

Citi (https://www.citigroup.com/global)

Lija Hogan on LinkedIn (https://www.linkedin.com/in/lija-hogan-894769/)

Learn more about Insights Unlocked: https://www.usertesting.com/podcast

Episode web page: https://bit.ly/4yCmSZK

Episode summary: What makes a great AI experience? It's about more than getting the right answer.

In this episode of Insights Unlocked, guest host Mike Mace talks with Sara Logel and Kira Orlova from Auros’ Professional Services about why traditional measures of software quality aren't enough to evaluate AI experiences. They introduce ARQ (AI Relationship Quality), a new research framework designed to help companies understand how people experience AI and build products that earn trust, drive adoption, and keep customers coming back.

Sara and Kira explore five essential elements of a successful AI experience: trust, understanding, control, outcome, and affinity. They explain how these factors work together, why trust in AI needs to be calibrated rather than maximized, and what happens when AI delivers impressive results but leaves users feeling confused, powerless, or disconnected.

The conversation also explores how ARQ combines quantitative benchmarking with qualitative human insights to help product and UX teams identify opportunities for improvement, compare experiences against competitors, and demonstrate the business value of better AI experiences.

You’ll learn:

Why evaluating AI experiences requires more than accuracy, usability, and task completion

The five elements of AI Relationship Quality: trust, understanding, control, outcome, and affinity

Why building appropriate trust in AI matters more than simply maximizing trust

How AI can deliver successful outcomes while still leaving users confused or feeling out of control

How quantitative scores and qualitative research work together to uncover what's driving AI experiences

How product and UX teams can benchmark AI experiences, prioritize improvements, and measure progress over time

Resources & links

Sara Logel on LinkedIn (https://www.linkedin.com/in/saralogel/)

Kira Orlova on LinkedIn (https://www.linkedin.com/in/kiraorlova/)

Mike Mace on LinkedIn (https://www.linkedin.com/in/mikemace/)

Learn more about ARQ (AI Relationship Quality): (https://bit.ly/4zHvPlw)

Learn more about Insights Unlocked (https://www.usertesting.com/podcast)

Episode web page: https://bit.ly/4zzktjl

Episode summary:

In this episode of Insights Unlocked, Nathan Isaacs chats with Caitlin Sullivan, founder of AI Customer Research, about how product, UX, and research teams can move beyond one-off AI prompts to build reliable, repeatable workflows that deliver meaningful customer insights. Drawing on more than 15 years of experience in customer research, Caitlin shares practical advice for identifying where AI can add value, measuring its impact, and building workflows teams can actually trust.

Caitlin explains why the rush to automate everything can undermine productivity and how starting small, documenting processes, and testing AI outputs against established research findings can help teams achieve better results. She also explores how to experiment responsibly with sensitive customer data, collaborate on AI workflows using tools like GitHub, and demonstrate the value of AI investments.

The conversation also tackles a bigger question: What shouldn't we automate? Caitlin makes the case for preserving critical thinking, human empathy, and the joy of discovery, reminding researchers and product teams that efficiency shouldn't come at the expense of the human skills that make customer insights valuable.

You’ll learn:

How to move beyond one-off AI prompts and build repeatable, reliable research workflows

Why starting with small, frustrating tasks is more effective than automating everything

How to measure AI's impact and validate outputs against trusted research findings

Ways to experiment safely with AI while protecting sensitive customer data

How to document, share, and continuously improve AI workflows across teams

Why critical thinking, empathy, and human judgment remain essential in AI-powered research

Resources & links

Caitlin Sullivan on LinkedIn (https://www.linkedin.com/in/caitlindsullivan/)

AI Customer Research (https://aicustomerresearch.com/)

Caitlin's AI Customer Research newsletter (https://aicustomerresearch.substack.com/)

Nathan Isaacs on LinkedIn (https://www.linkedin.com/in/nathanisaacs/)

Learn more about Insights Unlocked (https://www.usertesting.com/podcast)

Episode web page: https://bit.ly/4hyuZBr

Episode summary:

In this episode of Insights Unlocked, UserTesting’s Mike Mace talks with Julia Barham, product executive, longtime product leader and author of The Product Management Playbook, about what it takes to build products customers actually value—and how the role of product management is changing as AI accelerates the product development lifecycle.

Julia argues that great product managers are ultimately managers of value: connecting customer needs and business goals while developing enough fluency across design, data and technology to make better decisions. She shares why strong PMs don’t need to be experts in everything, but do need curiosity, judgment and the ability to synthesize insights and work effectively across functions.

The conversation explores why continuous discovery and experimentation are becoming even more important as AI makes it easier to build and ship quickly. Julia explains why teams should embrace “strong opinions, loosely held,” increase their learning velocity and resist the temptation to turn faster development into an excuse to ship everything in the backlog. Instead, teams need to validate ideas, learn quickly and stay focused on the customer experience.

Julia and Mike also discuss how AI could reshape the product manager’s role. As AI takes on more delivery and coordination work, Julia sees greater value shifting toward strategic thinking, discovery and human judgment. She also shares lessons from the cross-functional leaders she interviewed for her book—including why partners want PMs to bring them into the process early, provide business context and start with the problem rather than arriving with a predetermined solution.

You’ll learn

Why product management is increasingly about managing value, not simply managing features or backlogs

How PMs can build product sense across business, customers, design, data and technology without becoming experts in every discipline

Why continuous discovery and synthesis should happen throughout the product lifecycle

How “strong opinions, loosely held” can help teams experiment, learn and adapt faster

Why faster AI-powered development makes customer validation and product judgment more important, not less

Where product managers can focus their time as AI automates more delivery and coordination work

What design, data and engineering partners want from product managers

Why curiosity, influence and cross-functional collaboration remain essential product management skills

How product leaders can identify skill gaps and coach their teams to become stronger product thinkers

Resources and links

Listener offer: Use code UT20 when purchasing The Product Management Playbook directly from Rosenfeld Media for 20% off.

Julia Barham on LinkedIn (https://www.linkedin.com/in/julia-barham/)

The Product Management Playbook from Rosenfeld Media (https://rosenfeldmedia.com/books/product-management-playbook/)

Author interview with Rosenfeld Media (https://rosenfeldmedia.com/master-every-dimension-of-product-management-with-julia-barhams-new-book-the-product-management-playbook-create-ship-and-optimize-winning-products/ )

Progressive Insurance (https://www.progressive.com/)

Mike Mace on LinkedIn (https://www.linkedin.com/in/mikemace/)

Learn more about Insights Unlocked: usertesting.com/podcast

Episode web page: https://bit.ly/4ddeZlN

Episode summary

In this episode of Insights Unlocked, Nathan Isaacs talks with media futurist and author Jack Myers about how leaders can embrace AI without outsourcing the distinctly human qualities that drive better decisions, stronger customer relationships, and meaningful experiences. Drawing from his book Your Third Brain, Jack explores how human intelligence and machine intelligence can work together to expand what people are capable of—not replace human thinking.

Jack argues that one of the biggest risks of AI isn’t simply that machines are getting smarter, but that people may begin outsourcing judgment, curiosity, creativity, memory, and discernment to them. He explains the concept of the “third brain”: an intelligence that emerges when human experience, intuition, values, empathy, and imagination combine with AI’s speed, scale, knowledge, and ability to recognize patterns.

The conversation also explores what the media industry’s decades of technological disruption can teach today’s product, marketing, UX, and CX leaders. Jack warns against gaining efficiency at the expense of the customer relationship, particularly as organizations increasingly turn to AI, synthetic research, and automated customer interactions.

Nathan and Jack discuss why real human feedback remains essential even as AI becomes better at predicting behavior. Synthetic customers can help explore possibilities and prepare better questions, Jack says, but real people provide the context, emotion, contradictions, and unexpected insights that can reveal whether an organization is solving the right problem in the first place.

Finally, they explore empathy as a practical leadership skill and how organizations can reinvest the time AI saves into listening, curiosity, creativity, and deeper human understanding. Jack encourages leaders to define the human outcome before choosing the technology—and to measure not only productivity, but also trust, well-being, and whether technology is strengthening or diminishing human capabilities.

You’ll learn

Why Jack believes AI should expand human intelligence rather than replace human thinking

What the “third brain” is and how human and machine intelligence can complement each other

Why outsourcing judgment, curiosity, creativity, and discernment to AI creates risks for organizations

What the media industry can teach leaders about protecting direct customer relationships

Why synthetic research can support—but shouldn’t replace—feedback from real people

How empathy and human context can lead to better decisions and customer experiences

How leaders can balance AI-enabled speed and efficiency with deeper human understanding

Why organizations should measure trust and human capabilities alongside productivity

Resources and links

Jack Myers’ website: http://www.jackmyers.com/

Jack Myers’ books, including Your Third Brain: https://jackmyersbooks.com/

Nathan Isaacs on LinkedIn: https://www.linkedin.com/in/nathanisaacs/

Learn more about Insights Unlocked: https://www.usertesting.com/podcast