A plateau hidden by account migration
A UK B2B app believed its new onboarding had reduced week-eight retention. The comparison crossed an identity migration that split some returning users into fresh profiles. We rebuilt the denominator, isolated post-migration accounts and compared meaningful workspace activity rather than logins.
The corrected view did not prove onboarding was successful; it showed that the reported decline was overstated and that teams inviting a second colleague in week one had a more durable return pattern. Product chose to instrument invitation completion properly before running the next test.
“Mara kept the conclusions narrower than some of us wanted, which was frustrating for about ten minutes and useful for the next three months. We left with a reliable cohort definition and one onboarding behaviour worth testing.” — Product Director
When paid acquisition looked unusually loyal
A consumer app saw striking retention among one paid channel. Source properties were incomplete for users who reinstalled, moving a portion of returners into the paid cohort. The audit repaired channel rules and documented where attribution remained uncertain.
“The first result was uncomfortable: our best-looking cohort was partly a classification artefact. The revised analysis gave finance and growth the same numbers and stopped an increase in spend based on a false comparison.” — Growth Lead
A release read beyond conversion
For a membership app, a simplified registration flow increased completion but had no clear effect on four-week value activity. The readout separated faster account creation from continued use and identified a sharp difference between invited and self-started members.
“The memo was brief enough to use in the release meeting, but every claim had a definition and caveat behind it. We retained the simpler registration and moved the next experiment toward first-session value.” — Senior Product Manager