Pilot Study 3

The research team used human-centered design (HCD) methods and the Multiphase Optimization Strategy framework (Preparation, Optimization, Evaluation) in an iterative design approach. In the Preparation phase, they conducted 21 interviews and 3 focus groups to identify facilitators and barriers to breast cancer screening and obtain feedback on chatbot design mock-ups. They then developed a causal pathway diagram (CPD) informed by qualitative analysis and based in conceptual frameworks. In the Optimization phase, the research team tested CPD-identified moderators of chatbot effectiveness –chatbot communication style, familiarity, and expertise – using a 2×2 factorial experiment design with a baseline condition (see Figure 1). They recruited Black participants via online platforms to complete surveys with different versions of the chatbot design, testing different levels of each moderator. The primary outcomes assessed in the factorial experiment were Intention to Use and a composite Trust score. They incorporated the results of the experiment into the chatbot design and conducted co-design sessions with community members to craft chatbot messages. In the Evaluation phase, they plan to pilot the chatbot and subsequently evaluate it in a randomized controlled trial.

Through qualitative analysis of the interview and focus group data, the research team identified mistrust as a key barrier to both breast cancer screening and chatbot use. Based on this finding, they developed a CPD for the initial chatbot interaction to engender trust (see Figure 2). In the factorial experiment, the research team analyzed data from 496 completed surveys. Briefly, they found the Expert (Physician)-Polite condition was associated with higher intention to use (0.34 points on 5-point scale, p< 0.05) and the Familiar (Breast Cancer Survivor)-Polite condition was associated with significantly higher trust (0.12 points on 5-point scale, p< 0.05). Co-design sessions iterated on messaging and framing of chatbot communication.