<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom"><channel><title>Product Work | Allison Londerée</title><link>/tags/product-work/</link><atom:link href="/tags/product-work/index.xml" rel="self" type="application/rss+xml"/><description>Product Work</description><generator>HugoBlox Kit (https://hugoblox.com)</generator><language>en-us</language><lastBuildDate>Wed, 15 Apr 2026 12:00:00 -0400</lastBuildDate><image><url>/media/icon_hu_da05098ef60dc2e7.png</url><title>Product Work</title><link>/tags/product-work/</link></image><item><title>Early Indicators of Student Success</title><link>/work/education-intervention-analytics/</link><pubDate>Wed, 15 Apr 2026 12:00:00 -0400</pubDate><guid>/work/education-intervention-analytics/</guid><description>&lt;p&gt;At PERTS, I contributed to research on whether classroom learning conditions could function as early, actionable indicators of later academic outcomes. The question was simple but important: if we can measure whether students feel supported, challenged, and meaningfully engaged in class, can that help educators respond before end-of-term outcomes make problems obvious?&lt;/p&gt;
&lt;p&gt;&lt;em&gt;Figure note: Chance of earning a B or better by student group under negative (&amp;lt;5) versus positive (&amp;gt;=5) learning conditions on a seven-point composite scale, including race and reduced-price lunch status.&lt;/em&gt;&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Highlights&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Learning conditions are early indicators, not just &amp;ldquo;soft&amp;rdquo; context.&lt;/li&gt;
&lt;li&gt;Better classroom conditions are strongly associated with better math outcomes.&lt;/li&gt;
&lt;li&gt;This supports practical, repeated measurement so educators can intervene sooner.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id="problem"&gt;Problem&lt;/h2&gt;
&lt;p&gt;Schools often rely on lagging indicators such as final grades or end-of-year test scores. Those measures matter, but they arrive late. Educators need earlier signals that point to whether students are experiencing the kinds of classroom conditions that support learning.&lt;/p&gt;
&lt;p&gt;This project focused on three conditions in math classrooms:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Teacher Caring&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Meaningful Work&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Feedback for Growth&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id="approach"&gt;Approach&lt;/h2&gt;
&lt;p&gt;We analyzed whether these learning conditions predicted later math performance and whether changes in those conditions over time were associated with changes in student outcomes. The broader goal was to support a continuous-improvement approach in which educators could measure conditions, respond, and reassess rather than waiting for outcomes after the fact.&lt;/p&gt;
&lt;h2 id="methods"&gt;Methods&lt;/h2&gt;
&lt;p&gt;The report draws on data collected with the &lt;strong&gt;Character Lab Research Network&lt;/strong&gt; from &lt;strong&gt;more than 4,000 U.S. students in grades 8 through 12&lt;/strong&gt; during the &lt;strong&gt;2019-20&lt;/strong&gt; school year. Students rated learning conditions on a seven-point Likert scale, and those measures were linked to math grades over time.&lt;/p&gt;
&lt;p&gt;Analyses examined:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;the relationship between learning-condition ratings and the likelihood of earning a &lt;strong&gt;B or better&lt;/strong&gt; in math&lt;/li&gt;
&lt;li&gt;whether the same relationships held when controlling for demographics and prior grades&lt;/li&gt;
&lt;li&gt;whether changes in learning conditions between October and February predicted later changes in achievement&lt;/li&gt;
&lt;li&gt;whether results differed across student groups, including students eligible for &lt;strong&gt;Free and Reduced Price Lunch&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id="outcome--impact"&gt;Outcome / Impact&lt;/h2&gt;
&lt;p&gt;The findings make a strong case for treating learning conditions as decision-useful signals rather than soft background context.&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Students who rated learning conditions most positively were &lt;strong&gt;more than twice as likely&lt;/strong&gt; to earn a &lt;strong&gt;B or better&lt;/strong&gt; in math.&lt;/li&gt;
&lt;li&gt;Each step up in the composite learning-conditions score was associated with about &lt;strong&gt;6% more students&lt;/strong&gt; earning A or B grades.&lt;/li&gt;
&lt;li&gt;A positive two-point shift in learning conditions was associated with roughly a &lt;strong&gt;17% higher&lt;/strong&gt; likelihood of earning a &lt;strong&gt;B or better&lt;/strong&gt; in the following term.&lt;/li&gt;
&lt;li&gt;Positive learning conditions were especially meaningful for students who had been less well served, including students eligible for &lt;strong&gt;FRPL&lt;/strong&gt; and Black students.&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;What I like most about this work is that it connects careful behavioral measurement to a concrete intervention model. It is not just an explanatory report; it points toward a system educators can actually use to monitor conditions and improve them over time.&lt;/p&gt;
&lt;h2 id="tools"&gt;Tools&lt;/h2&gt;
&lt;p&gt;Education research, survey measurement, longitudinal analysis, regression-style modeling, applied behavioral science, research communication.&lt;/p&gt;</description></item><item><title>Next Mission Group Chrome Extension</title><link>/work/next-mission-group-extension/</link><pubDate>Wed, 15 Apr 2026 12:00:00 -0400</pubDate><guid>/work/next-mission-group-extension/</guid><description>&lt;p&gt;I built an MVP &lt;strong&gt;Chrome extension&lt;/strong&gt; for &lt;strong&gt;Next Mission Group&lt;/strong&gt; around a concrete product problem: making military experience legible to civilian hiring workflows in real time—not as a one-off résumé rewrite, but &lt;strong&gt;in the browser&lt;/strong&gt;, on the same pages where veterans already search and apply.&lt;/p&gt;
&lt;p&gt;The work combined &lt;strong&gt;product flow&lt;/strong&gt;, &lt;strong&gt;ontology design&lt;/strong&gt;, &lt;strong&gt;heuristic scoring&lt;/strong&gt;, &lt;strong&gt;LLM-assisted narrative&lt;/strong&gt;, and &lt;strong&gt;production auth/API&lt;/strong&gt; choices (Manifest V3, JWT connect flow, secrets kept off the client).&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Highlights&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Manifest V3&lt;/strong&gt; extension with a &lt;strong&gt;side panel&lt;/strong&gt; UX: sign in, maintain a profile, open a supported job posting, run a fit analysis, save results, and leave feedback.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Job-board coverage&lt;/strong&gt; for the MVP path: detection and parsing on &lt;strong&gt;LinkedIn&lt;/strong&gt;, &lt;strong&gt;USAJOBS&lt;/strong&gt;, &lt;strong&gt;Workday&lt;/strong&gt;, &lt;strong&gt;Greenhouse&lt;/strong&gt;, and &lt;strong&gt;Lever&lt;/strong&gt; (with a pattern for adding more hosts).&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Ontology-backed mapping&lt;/strong&gt; between military roles, civilian occupations, and transferable skills—separating a &lt;strong&gt;canonical backbone&lt;/strong&gt; from an &lt;strong&gt;inferred&lt;/strong&gt; layer so LLM suggestions can be &lt;strong&gt;reviewed and traced&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Backend on Wix Velo&lt;/strong&gt; for the MVP operational path: member auth integration, server-side API routes, and &lt;strong&gt;Wix Secrets&lt;/strong&gt; (or env vars locally) so provider keys never ship in the extension bundle.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id="problem"&gt;Problem&lt;/h2&gt;
&lt;p&gt;Military experience often &lt;strong&gt;does not align&lt;/strong&gt; with how civilian roles are written: different vocabulary, credential expectations, and occupational categories. Veterans may be strong fits for a posting while still looking “off-profile” to both &lt;strong&gt;applicants&lt;/strong&gt; and &lt;strong&gt;automated systems&lt;/strong&gt;. The goal was to shrink that gap &lt;strong&gt;at the moment of job evaluation&lt;/strong&gt;—not only after the fact on a separate site.&lt;/p&gt;
&lt;h2 id="user-flow"&gt;User flow&lt;/h2&gt;
&lt;ol&gt;
&lt;li&gt;Install the extension and open the &lt;strong&gt;side panel&lt;/strong&gt;.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Connect&lt;/strong&gt; via the short-lived &lt;strong&gt;JWT&lt;/strong&gt; flow tied to &lt;strong&gt;Wix Members&lt;/strong&gt; (extension behaves like a first-party client without embedding API secrets).&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Sync or edit&lt;/strong&gt; profile data used for matching (roles, skills, history—details governed by product spec).&lt;/li&gt;
&lt;li&gt;Navigate to a &lt;strong&gt;supported job posting&lt;/strong&gt;; the extension detects the page type and extracts structured signals where possible.&lt;/li&gt;
&lt;li&gt;Request an &lt;strong&gt;analysis&lt;/strong&gt;: ontology + scoring produce an interpretable match; &lt;strong&gt;LLM copy&lt;/strong&gt; is used where it adds explanation, with guardrails and provenance in mind.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Save&lt;/strong&gt; outputs and optionally submit &lt;strong&gt;feedback&lt;/strong&gt; to improve mappings and messaging over time.&lt;/li&gt;
&lt;/ol&gt;
&lt;h2 id="architecture-high-level"&gt;Architecture (high level)&lt;/h2&gt;
&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Layer&lt;/th&gt;
&lt;th&gt;Role&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Extension (MV3)&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Side panel UI, content scripts / service worker per MV3 constraints, calls to Velo HTTP functions&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Wix Velo&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Authenticated APIs, persistence, orchestration; &lt;strong&gt;secrets&lt;/strong&gt; for third-party / LLM providers&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Ontology&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Canonical identifiers + reviewable inferred links; scoring and narrative sit on top&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;LLM&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Explanations and assistive text where helpful—not a black-box replacement for structured mapping&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;
&lt;h2 id="security-and-operations"&gt;Security and operations&lt;/h2&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;No long-lived API keys in the browser&lt;/strong&gt;; privileged calls go through &lt;strong&gt;your&lt;/strong&gt; backend.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;JWT-based connect&lt;/strong&gt; aligns the extension with the same membership model as the main site.&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Provider keys&lt;/strong&gt; live in &lt;strong&gt;Wix Secrets&lt;/strong&gt; (or local env for development), consistent with a path toward harderening for review / pilot.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id="outcomes"&gt;Outcomes&lt;/h2&gt;
&lt;p&gt;The MVP establishes an end-to-end path: &lt;strong&gt;install → connect → profile → analyze → save → feedback&lt;/strong&gt;, with documentation for &lt;strong&gt;ontology growth&lt;/strong&gt;, &lt;strong&gt;human review of inferred edges&lt;/strong&gt;, and &lt;strong&gt;broader military–civilian coverage&lt;/strong&gt; over time. Quantitative pilot metrics (activation, analyses per user, qualitative hiring outcomes) can be layered in as releases move beyond internal and friendly-user testing.&lt;/p&gt;
&lt;h2 id="demo"&gt;Demo&lt;/h2&gt;
&lt;p&gt;&lt;strong&gt;MP4 (recommended)&lt;/strong&gt; — Put your file in this same folder as &lt;code&gt;index.md&lt;/code&gt;, e.g. &lt;code&gt;content/work/next-mission-group-extension/demo.mp4&lt;/code&gt;. Optional poster image: &lt;code&gt;demo.jpg&lt;/code&gt; (same base name; used as the thumbnail before play).&lt;/p&gt;
&lt;video controls &gt;
&lt;source src="/work/next-mission-group-extension/demo.mp4" type="video/mp4"&gt;
&lt;/video&gt;
&lt;p&gt;You can use any filename: change &lt;code&gt;src&lt;/code&gt; above to match. Files in &lt;strong&gt;&lt;code&gt;assets/media/&lt;/code&gt;&lt;/strong&gt; also work (use a path like &lt;code&gt;my-folder/recording.mp4&lt;/code&gt;). A direct &lt;strong&gt;HTTPS link&lt;/strong&gt; to an &lt;code&gt;.mp4&lt;/code&gt; works as &lt;code&gt;src&lt;/code&gt; too.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;YouTube&lt;/strong&gt; — If you prefer, set &lt;strong&gt;&lt;code&gt;demo_youtube_id&lt;/code&gt;&lt;/strong&gt; in the front matter (the &lt;code&gt;v=&lt;/code&gt; ID only) and uncomment the line below; remove or comment out the &lt;code&gt;video&lt;/code&gt; shortcode above to avoid two players.&lt;/p&gt;
&lt;h2 id="github"&gt;GitHub&lt;/h2&gt;
&lt;p&gt;Your public &lt;strong&gt;&lt;code&gt;links&lt;/code&gt;&lt;/strong&gt; entry points to &lt;strong&gt;
&lt;/strong&gt;. When the extension (or a mono-repo) is published under your account or an org, add a second &lt;strong&gt;&lt;code&gt;code&lt;/code&gt;&lt;/strong&gt; link in front matter with that repository URL so the header shows &lt;strong&gt;Extension source&lt;/strong&gt; alongside &lt;strong&gt;GitHub profile&lt;/strong&gt;.&lt;/p&gt;
&lt;h2 id="tools"&gt;Tools&lt;/h2&gt;
&lt;p&gt;JavaScript, Chrome Extension APIs (Manifest V3), Wix Velo, JWT auth, ontology modeling, heuristic scoring, LLM integration, product and architecture documentation.&lt;/p&gt;</description></item><item><title>Pesticide Risk &amp; Respiratory Health Modeling</title><link>/work/pesticide-risk-health-burden/</link><pubDate>Wed, 15 Apr 2026 12:00:00 -0400</pubDate><guid>/work/pesticide-risk-health-burden/</guid><description>&lt;p&gt;As part of a Spring 2026 team project, I worked on a county-level modeling pipeline that linked pesticide exposure estimates to respiratory health burden in the United States. The goal was to build a planning-oriented model that could help public health stakeholders identify where asthma and COPD burden may warrant closer attention—while explicitly accounting for the ethical and equity implications of doing so.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Highlights&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Built county-level XGBoost models to estimate relative asthma/COPD burden tied to pesticide-related context.&lt;/li&gt;
&lt;li&gt;Published an interactive map for planning and resource prioritization, not diagnosis.&lt;/li&gt;
&lt;li&gt;Paired performance metrics with equity/error analysis to make limits and risks explicit.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id="problem"&gt;Problem&lt;/h2&gt;
&lt;p&gt;Agricultural pesticide use varies substantially by crop and region, and exposure risk is not evenly distributed. These differences often intersect with existing structural inequities in environmental exposure and healthcare access.&lt;/p&gt;
&lt;p&gt;Public health agencies, health systems, and insurers need tools to prioritize prevention and outreach, especially when resources are limited. However, tools that surface “high-risk” areas can unintentionally reinforce stigma or obscure underlying structural drivers if not carefully designed and interpreted.&lt;/p&gt;
&lt;p&gt;This project asked a practical question: can we combine public exposure, land-use, and health data to flag counties where pesticide-related respiratory burden may be higher—while maintaining transparency about uncertainty and equity impacts?&lt;/p&gt;
&lt;h2 id="approach"&gt;Approach&lt;/h2&gt;
&lt;p&gt;We built a county-year modeling pipeline for &lt;strong&gt;2018 and 2019&lt;/strong&gt; that joined pesticide use estimates with respiratory-health outcomes and county context. The final product included both a documented modeling workflow and a public-facing GitHub Pages site with an interactive county risk map.&lt;/p&gt;
&lt;p&gt;From the outset, the work was framed as a &lt;strong&gt;population-level planning tool&lt;/strong&gt;, not a diagnostic or causal system. We paired model development with a “justification-for-proceeding” framework to surface assumptions, risks, and potential downstream harms before they became embedded in the system.&lt;/p&gt;
&lt;p&gt;This included:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;defining appropriate use (resource prioritization, not individual prediction)&lt;/li&gt;
&lt;li&gt;identifying potential misuse (e.g., stigmatizing communities or over-attributing causality)&lt;/li&gt;
&lt;li&gt;evaluating how model errors might differentially impact populations&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id="methods"&gt;Methods&lt;/h2&gt;
&lt;p&gt;The modeling dataset combined:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;CDC PLACES&lt;/strong&gt; county-level indicators, including asthma, COPD, smoking, obesity, and diabetes&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;USGS / EPA pesticide use estimates&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;USDA Cropland Data Layer&lt;/strong&gt; features&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;ACS demographic covariates&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The final selected model was an &lt;strong&gt;XGBoost regressor&lt;/strong&gt; using a feature set that included &lt;strong&gt;445 pesticide mass features&lt;/strong&gt; plus baseline demographic, health, cropland, and time covariates.&lt;/p&gt;
&lt;p&gt;In addition to standard modeling steps (exploratory analysis, spatial train/test splitting, hypothesis testing, and holdout validation), we conducted &lt;strong&gt;equity-focused evaluation&lt;/strong&gt;:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;compared model error across demographic and geographic subgroups&lt;/li&gt;
&lt;li&gt;examined whether prediction accuracy varied systematically in counties with different socioeconomic or population characteristics&lt;/li&gt;
&lt;li&gt;assessed where data sparsity or measurement limitations could bias results&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The project also included model-card style documentation to make assumptions, limitations, and intended use explicit.&lt;/p&gt;
&lt;h2 id="outcome--impact"&gt;Outcome / Impact&lt;/h2&gt;
&lt;p&gt;The final XGBoost models performed strongly on held-out county-level validation data:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Asthma (CASTHMA):&lt;/strong&gt; R² = &lt;strong&gt;0.835&lt;/strong&gt;, RMSE = &lt;strong&gt;0.389&lt;/strong&gt;, n = &lt;strong&gt;1219&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;COPD:&lt;/strong&gt; R² = &lt;strong&gt;0.885&lt;/strong&gt;, RMSE = &lt;strong&gt;0.765&lt;/strong&gt;, n = &lt;strong&gt;1219&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;The project also produced a live interactive map that translates model output into a more usable planning interface for non-technical stakeholders.&lt;/p&gt;
&lt;p&gt;Importantly, the equity analysis surfaced a meaningful gap:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;model performance was &lt;strong&gt;not uniform across counties&lt;/strong&gt;, with higher error observed in some regions with different demographic compositions and data coverage&lt;/li&gt;
&lt;li&gt;this suggests that the areas most affected by environmental and health inequities may also be those where predictions are &lt;strong&gt;less reliable&lt;/strong&gt;&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This finding shaped how we positioned the tool:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;as a &lt;strong&gt;starting point for investigation&lt;/strong&gt;, not a definitive ranking&lt;/li&gt;
&lt;li&gt;as a way to guide &lt;strong&gt;additional data collection and local validation&lt;/strong&gt;, especially in underrepresented areas&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;More broadly, the project reinforced that building useful models in public health requires more than predictive performance. It requires clear communication about:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;what the model can and cannot say&lt;/li&gt;
&lt;li&gt;where it may fail&lt;/li&gt;
&lt;li&gt;and how its outputs could impact real communities if used uncritically&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id="tools"&gt;Tools&lt;/h2&gt;
&lt;p&gt;Python, Jupyter, XGBoost, county-level public health data, geospatial joins, GitHub Pages, model-card documentation, responsible AI frameworks.&lt;/p&gt;</description></item><item><title>RSV Vaccine Attitudes in a Digital Health Intervention</title><link>/work/digital-health-experimentation/</link><pubDate>Wed, 15 Apr 2026 12:00:00 -0400</pubDate><guid>/work/digital-health-experimentation/</guid><description>&lt;p&gt;This case study is based on published work from my time at Lirio on a digital health intervention designed to increase RSV vaccination among adults over 60. Rather than treating outreach as a one-way campaign, the project looked closely at what people actually texted back and used those replies to better understand attitudes, friction points, and behavior-change opportunities.&lt;/p&gt;
&lt;p&gt;&lt;strong&gt;Highlights&lt;/strong&gt;&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;Analyzed unsolicited SMS replies to understand RSV vaccine attitudes at scale.&lt;/li&gt;
&lt;li&gt;Used structural topic modeling plus expert thematic coding for interpretable insights.&lt;/li&gt;
&lt;li&gt;Findings informed how live intervention messaging could be improved over time.&lt;/li&gt;
&lt;/ul&gt;
&lt;h2 id="problem"&gt;Problem&lt;/h2&gt;
&lt;p&gt;When the RSV vaccine was introduced for older adults in the United States in 2023, many of the drivers of uptake were still unknown.&lt;/p&gt;
&lt;p&gt;For teams designing outreach interventions, this creates a familiar challenge: how do you improve messaging when beliefs, concerns, and barriers are still emerging in real time?&lt;/p&gt;
&lt;p&gt;Rather than relying solely on structured surveys or downstream outcomes, we turned to a more immediate signal: what people actually texted back.&lt;/p&gt;
&lt;h2 id="approach"&gt;Approach&lt;/h2&gt;
&lt;p&gt;The intervention was launched with a large community pharmacy chain using SMS precision nudging to encourage vaccination. As patients replied, we accumulated a large, messy, and highly valuable dataset of unsolicited responses.&lt;/p&gt;
&lt;p&gt;However, the scale of the data introduced a new constraint: there were far too many replies for behavioral experts to review manually, and insights were needed quickly to inform an active intervention.&lt;/p&gt;
&lt;p&gt;My role as the data scientist was to bridge that gap—developing a scalable way to extract structure and meaning from free-text responses, while preserving the nuance required for behavioral interpretation.&lt;/p&gt;
&lt;h2 id="methods"&gt;Methods&lt;/h2&gt;
&lt;p&gt;The study population included &lt;strong&gt;2,481,987&lt;/strong&gt; eligible adults aged 60 and older, of whom &lt;strong&gt;35,716&lt;/strong&gt; people replied, generating &lt;strong&gt;105,848&lt;/strong&gt; text responses. After removing operational texts such as &lt;code&gt;STOP&lt;/code&gt; and &lt;code&gt;HELP&lt;/code&gt;, the analytic dataset included &lt;strong&gt;46,964&lt;/strong&gt; unsolicited replies.&lt;/p&gt;
&lt;p&gt;We used a mixed-methods workflow that combined computational modeling with expert interpretation:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;&lt;strong&gt;Structural topic modeling (STM)&lt;/strong&gt; to identify latent themes across the full dataset&lt;/li&gt;
&lt;li&gt;&lt;strong&gt;Thematic analysis&lt;/strong&gt; on a hand-coded subset, conducted by behavioral health experts&lt;/li&gt;
&lt;li&gt;Alignment of model-derived topics with expert-reviewed themes to ensure interpretability&lt;/li&gt;
&lt;li&gt;Characterization of topics along dimensions such as &lt;strong&gt;sentiment&lt;/strong&gt; and &lt;strong&gt;practical vs. emotional function&lt;/strong&gt;&lt;/li&gt;
&lt;li&gt;Comparisons across time, message condition, and available covariates (e.g., flu vaccination history, insurance type)&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;This approach allowed us to scale insight generation without losing the behavioral context needed for intervention design.&lt;/p&gt;
&lt;figure&gt;&lt;img src="/work/digital-health-experimentation/topic-map-rsv.png"
alt="Topic Map of 30 topics in text message replies to a precision nudging digital health intervention for RSV vaccination. Shape and colour indicate the mapping of structural topic model topics onto the 10 thematic analysis topics. Black star = Wrong recipient, teal circle = Help, red cross = Stop, purple plus = Already vaccinated, brown square = Will not get vaccinated, yellow triangle = Benign, and grey diamond = Nonsense. Size indicates topic proportions. Topics 1-10 are largest, topics 11-20 are of middle size, and topics 21-30 are smallest."&gt;&lt;figcaption&gt;
&lt;p&gt;Topic Map of 30 topics in text message replies to a precision nudging digital health intervention for RSV vaccination. Shape and colour indicate the mapping of structural topic model topics onto the 10 thematic analysis topics. Black star = Wrong recipient, teal circle = Help, red cross = Stop, purple plus = Already vaccinated, brown square = Will not get vaccinated, yellow triangle = Benign, and grey diamond = Nonsense. Size indicates topic proportions. Topics 1-10 are largest, topics 11-20 are of middle size, and topics 21-30 are smallest.&lt;/p&gt;
&lt;/figcaption&gt;
&lt;/figure&gt;
&lt;h2 id="outcome--impact"&gt;Outcome / Impact&lt;/h2&gt;
&lt;p&gt;The mixed-method approach of modeling &lt;em&gt;and&lt;/em&gt; expert review converged to produce a grounded, interpretable map of how adults responded to a newly introduced vaccine in a real-world setting.&lt;/p&gt;
&lt;p&gt;Key findings included:&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;expressed attitudes became less negative later in the intervention&lt;/li&gt;
&lt;li&gt;individuals without prior flu vaccination and those with commercial insurance were more likely to express refusal&lt;/li&gt;
&lt;li&gt;messages framed around &lt;strong&gt;emotional consequences&lt;/strong&gt; generated the highest engagement&lt;/li&gt;
&lt;li&gt;messages framed around &lt;strong&gt;anticipated regret&lt;/strong&gt; generated the lowest engagement&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;Beyond the publication, this project demonstrated a practical pattern: behavioral data generated during live interventions can be rapidly structured and fed back into design decisions.&lt;/p&gt;
&lt;p&gt;Instead of treating outreach as static, this approach enables a tighter loop between &lt;strong&gt;what people say&lt;/strong&gt;, &lt;strong&gt;how we interpret it&lt;/strong&gt;, and &lt;strong&gt;how interventions evolve&lt;/strong&gt;.&lt;/p&gt;
&lt;h2 id="tools"&gt;Tools&lt;/h2&gt;
&lt;p&gt;Behavioral science, thematic analysis, structural topic modelling, text analysis, digital health experimentation, intervention evaluation.&lt;/p&gt;</description></item></channel></rss>