I've been on a lot of data team calls over the years — at Fivetran, at Atlan, and with the customers I've worked with since. A pattern I keep seeing: bad analytics data doesn't announce itself. It quietly corrupts dashboards, nudges product decisions in the wrong direction, and drains the data team's credibility one awkward meeting at a time.
By the time anyone officially notices, the damage is measured in months of wrong decisions and a lot of rebuilt trust.
Here are five symptoms I keep running into, and what I'd do about each one.
1. Your funnel numbers don't add up
You're looking at your conversion funnel. The numbers don't make sense. More users completed checkout than started it. Your signup-to-activation rate is 120%. A step in onboarding shows zero users for a week, then jumps back to normal.
What's usually happening: events are being dropped, duplicated, or misfired.
Common causes I've seen:
- An event fires on the wrong trigger (
checkout_startedon page load instead of on button click). - A deploy broke an event and nobody noticed for days.
- Web and mobile fire the same event with different naming (
checkout_startedvscheckoutStarted). - An A/B test variant is missing tracking code entirely.
How I'd fix it: define event contracts with explicit trigger conditions and validate them at the CDP layer. Monitor event volumes per-platform — a 50% drop on iOS while web stays flat is an instant signal that something broke.
2. Three teams, three numbers for the same metric
Product says monthly active users is 45,000. Marketing says 52,000. Finance says 38,000. All three pulled from the same warehouse. None of them are wrong — they're using different event definitions.
What's usually happening:
- Product counts
session_startedevents. - Marketing counts users who triggered any event.
- Finance counts users with
subscription_active = true.
They're all measuring "active users" using different underlying events and filters, and nobody has written down which one is canonical.
How I'd fix it: establish canonical event definitions with explicit business meaning. A contract for session_started should include a description like "Fires once per user session when the application loads. A session expires after 30 minutes of inactivity." When every team reads the same contract, they build metrics from the same foundation.
3. You can't explain a metric change
Your activation rate dropped 15% last week. The head of product asks why. You spend three days investigating. Was it a tracking change? A product change? A pipeline issue? A seasonal pattern? You can't tell, because you have no visibility into what changed in the event data.
This one is personal for me. I've sat in more than one of these war rooms, and the worst part is always the same: you can't prove whether it's a real product signal or a tracking regression. A property rename, a new SDK version changing event timing, a feature flag accidentally suppressing tracking for a cohort — any of these looks identical to a product metric change in your dashboard.
How I'd fix it: monitor the events themselves, not just the aggregate metrics. When feature_activated volume drops, you want to know within hours. Correlate event changes with git deploys to narrow it down: "this event dropped 40% two hours after commit abc123 shipped." That single line of context turns a three-day investigation into a three-minute one.
4. Your warehouse has events nobody recognises
Query your event table for distinct event names. Almost every team I've worked with finds events nobody can explain. test_click_v2. old_checkout_flow. [object Object]. Phantom events polluting the table, eating warehouse storage, and confusing anyone who queries it directly.
What's usually happening: without governance, anyone can ship any event. Test events leak to production. Deprecated events keep firing because nobody removed the tracking code. Bugs produce malformed event names. Over time, the signal-to-noise ratio degrades.
How I'd fix it: enforce a tracking plan at your CDP. Events not in the plan get flagged or blocked. Auto-discover events from your warehouse and diff them against your contracts — anything in the warehouse but not in a contract is an unplanned event that needs a decision.
5. Nobody trusts the data
This is the final stage, and it's the one that worries me most.
Stakeholders stop using dashboards and ask analysts to pull custom reports. Analysts spend more time validating data quality than doing analysis. Product managers make decisions on intuition instead of metrics. The data team becomes a bottleneck instead of an accelerator.
Trust erosion is cumulative. Each wrong number, each metric that can't be explained, each report that takes a week because the analyst has to validate the underlying data first — trust drops. Once lost, it takes months to rebuild.
How I'd fix it:
- Publish your event contracts so everyone can see what events exist, what they mean, and who owns them.
- Surface real-time health scores per event — volume, violation rate, SLA compliance.
- Alert proactively, so the data team finds problems before stakeholders do.
- Automate the fixes where you can — auto-generated PRs, AI-assisted root cause, so the loop closes in hours, not weeks.
The common thread
All five of these trace back to the same root cause: there's no enforceable governance over the event data. Spreadsheets, tribal knowledge and manual QA don't scale. They break when teams grow, when products evolve, and when the pace of shipping picks up.
The fix isn't more documentation. It's infrastructure:
- Contracts that define what events should look like.
- Enforcement at the CDP layer that blocks bad data at ingestion.
- Monitoring that detects anomalies before they reach dashboards.
- Automation that fixes problems without manual investigation.
Broken analytics data isn't inevitable. It's a governance problem with a governance solution, and it's the one I'm building suky to solve.
If any of the five above felt uncomfortably familiar, I'd love to take a look at your data. Come have a look and we can start there.
— Kevin