Researchers at the University of California, San Francisco wanted to use smartphone questionnaires to get quick samples of behavior.
I worked closely with each team to transform their research program into one based on fast, user-friendly mobile questions.
I developed PingQuest, a platform for automated bidirectional text messages and links to mobile web questionnaires. I consulted with each research user to create the right questions, scheduling, and pinging methods for their specific research goals.
The available survey tools were poorly optimized for mobile
PingQuest uses an extremely simple, mobile-first template that can be customized for each study
Survey tools at the time either crowded lots of questions onto a single mobile page or require repeated page loads
PingQuest pages preload all questions and display one at a time, minimizing respondents’ visual and cognitive burden without added load time.
Smartphone experience sampling tools require users to install apps, and can only run on certain devices
PingQuest uses text messages and standard mobile web components for easy setup and cross-platform consistency
Researchers are used to infrequent surveys that ask large numbers of complex questions
I consult closely with every new PingQuest user to create simple, rapid mobile-format questions that efficiently address their big questions.
So far PingQuest has been used by 12 research labs and collected over 6,000 participant reports.
One of the great things about this project was getting to collaborate with so many different teams, with different topics and different approaches to science.
This researcher was measuring moods and medication use in HIV+ people with substance abuse problems. The system responded to their reports with an individually personalized, mood dependent affirmation.
- Adapted questions and responses to SMS format for low-income or homeless participants with government-issued feature phones.
- Detailed parsing rules for participant SMS reports to allow automated responses.
- Iteratively improved question text and scheduling by observing participants' response patterns, misspellings, malformattings, and side comments.
Result: Participants enjoyed the two-way texting. Results are being submitted for publication.
A nutrition researcher needed to know whether people's food cravings changed after a change in diet (such as cutting carbs, or drinking a liter of soda every day).
- Helped clarify research questions -- e.g., recognizing the distinction between having fewer cravings vs becoming better at resisting cravings.
- Rapid, targeted pilot testing to cut questions and adjust for unexpected response patterns.
- Collected pilot data on frequency of food cravings among users, allowing us to ask temporally fine-grained questions while still catching relevant events.
Result: Response rates are consistently >90%. The first study is being submitted for publication. These questions are being used in an evaluation study of a commercial mindful eating app.
This researcher studied stress, exercise, and relationship quality among people providing care to elderly spouses. Because of the demands of caregiving, their responses were often delayed, cut short, or out of order.
- Created specifications for adapting questionsbased on time of day and user's past responses
- Determined how to salvage partial or interrupted responses, increasing usable data
- Created logic for handling missed, delayed, or duplicate responses
- Taught staff to recognize anomalous data and promptly contact participants for clarification
Result: The system has functioned reliably for three years, and data collection is nearly complete.
A MySQL DB stores relationships among participants, pings, schedules, questions, and responses. A variety of metadata fields and flags can be used for study-specific information.
The backend uses PHP and talks to the Twilio API. Mobile web questionnaire pages are coded in HTML and JQuery Mobile, and emphasize a clean and accessible interface.
PingQuest is a simple, robust, automated system that current projects can continue using indefinitely at very low cost (~$40 per year for hosting and $.0075 per ping).
Developing PingQuest was an amazing exercise in information management and design under constraints. Every client had a unique approach to research, unique needs, and unique participants. It let me collaborate on so many topics I never thought I'd work on, from drug use to diet adherence to reproductive health!
Are you a researcher who's interested in using PingQuest? It's not a main focus right now, but some collaborations are just irresistible. I'd love to hear what you're up to.
Special thanks to my mentor Dr. Judith Moskowitz for supporting the first PingQuest study, and to my friend Dr. Ashley Mason for encouragement, tons of evangelism, and long talks about research design.