Mosaic Data Partners — Insights

Why AI Pilots Stall Before Production

2026-07-22T00:00:00Z

The demo went well. Leadership was impressed. Someone said "we should roll this out." That was five months ago, and the pilot is still a pilot (or quietly dead in a Slack channel nobody opens).

We hear a version of this story in most first conversations about AI work. The details vary, but the failure modes repeat. Here are the six we see, in rough order of frequency.

1. The demo was the deliverable

A pilot built to impress in a 30-minute meeting is optimized for the happy path: clean inputs, cherry-picked examples, a presenter steering around the rough edges. Production is the opposite: adversarial inputs, edge cases, users who type things you didn't anticipate. Teams that treat the demo as 80% done discover it was closer to 20%, get discouraged by the remaining grind, and stall.

The fix is scoping the pilot as a thin slice of the production system (real data access, real auth, one real workflow end to end) rather than a facsimile of it. That version takes longer to demo and months less to ship.

2. No evals, so nobody can say whether it works

Ask "is the AI good enough to roll out?" and the room goes quiet, because "good enough" was never defined. Without an eval set (a few hundred representative inputs with known-good outputs, scored automatically), every quality conversation runs on anecdotes. One bad example from an exec kills the project, one good one over-inflates it, and neither is evidence.

Teams skip eval sets because building them is tedious, and skipping them is the most common technical reason pilots die. Build the eval set before scaling the pilot. If you can't define what a good output looks like, that's not an eval problem; the use case isn't ready.

3. It can't reach the data that would make it useful

The pilot ran on ten uploaded PDFs. The production version needs the warehouse, the docs wiki, the ticketing system & the CRM, with the user's actual permissions enforced. That's an integration and access-control project, and it's routinely discovered late because the pilot sidestepped it.

This is also where AI projects turn out to be data projects in disguise: if the warehouse is untrusted or undocumented, the AI inherits every one of those problems and adds hallucination risk on top. We've redirected more than one AI engagement into a transformation engagement first, because a system that confidently answers from bad data is worse than no system.

4. Nobody owns it after the demo

Pilots get built by an enthusiast (an engineer with spare-time energy, an innovation team, an intern). Production systems need an owner with a budget line: someone accountable for accuracy drift, prompt updates when the model version changes, cost monitoring & the support queue. When the enthusiast moves on and no owner was named, the pilot doesn't fail so much as stop.

Name the owner before the build starts. If no team wants to own it, that tells you the organization doesn't actually value the use case, which is cheaper to learn before the build.

5. Cost & latency were never budgeted

At demo scale, inference cost rounds to zero and nobody times the responses. At production scale, both matter: a use case that costs $0.40 a query is viable for contract review and absurd for autocomplete, and a 20-second response that felt fine in a demo is unusable inside a support workflow. Set the per-query cost and latency targets in week one and engineer to them; model choice, caching strategy & how much context goes into each call all follow from that budget.

6. Security review arrived at the end instead of the beginning

The pilot ran on someone's laptop with an API key in a notebook. Then, at rollout, security asks the reasonable questions: where does the data go, what's the retention policy, who can query what, what happens when the model is prompted to leak? Answering these late can force a re-architecture; answering them early is mostly a set of known patterns (zero-retention API terms, scoped access, injection defenses, audit logging). Invite security to the kickoff and your longest-lead blocker becomes a checklist.

What the successful ones do differently

The through-line isn't talent or budget. Teams that ship treat the pilot as the first increment of a production system: real data, an eval set, a named owner, cost and latency budgets, security in the room, all from week one. That front-loads the effort, and it's why the project is still alive in month four.

That's how we run AI engagements: an audit-plus-prototype phase (typically 2–4 weeks) that produces the eval set and a real go/no-go, before anyone commits to a full build. Sometimes the answer coming out of it is "not this use case, but here's a better one" (cheaper to hear in week three than month six).

If you've got a stalled pilot (or a list of them), send us two or three sentences about it. We'll tell you within a business day whether it looks recoverable and what we'd check first.

How Much Does a Data Warehouse Migration Actually Cost?

2026-07-08T00:00:00Z

Every migration conversation starts with the same question and gets the same useless answer: "it depends." It does depend, but on knowable things. If you can count your source systems, estimate how much undocumented logic lives in your current pipelines, and list the reports that absolutely cannot break, you can get to a defensible budget number before a single vendor call.

Here's how we think about it.

The five cost drivers

1. Source system count. Each source (Salesforce, your production database, Stripe, the ERP) needs a connector, historical backfill, and someone who understands its quirks. A company with 4 sources and a company with 20 are running different projects. Managed connectors (Fivetran, Airbyte) keep this cheap for standard sources; the expensive ones are the internal systems and the vendors with bad or rate-limited APIs.

2. Undocumented transformation logic. This is the driver most teams underestimate. Fifteen years of stored procedures, scheduled SQL jobs & Excel macros encode business logic nobody wrote down. Migrating the data is easy. Reverse-engineering what the old pipeline meant to do, deciding what to keep, and re-implementing it in tested models is where the hours go. A rough tell: if nobody at your company can explain how revenue is calculated end to end, budget more.

3. The report surface you have to protect. The migration isn't done when the data lands; it's done when the reports leadership already relies on produce the same or better answers from the new platform. More reports, more validation. We handle this with a parallel run (old & new systems side by side, reconciled weekly). It adds cost up front and removes the much larger cost of a broken report surface at cutover.

4. Team availability on your side. Someone who knows the business logic has to answer questions. If your analysts can give a few hours a week, the project moves. If every question waits five days, the timeline (and the bill) stretches. This is free for you to fix and worth real money.

5. Compliance & access requirements. SOC 2, HIPAA, PII handling, row-level security. None of these are exotic, but each adds design and review time. Flag them in the first conversation, not week six.

What the numbers look like

With the caveat that these are services costs (platform costs below), based on the engagements we scope:

Small (~4–6 sources, modest logic, a handful of critical reports): roughly $25K–$60K, 8–10 weeks.
Mid (~8–15 sources, real legacy logic, a report surface people fear): roughly $60K–$150K, 10–14 weeks.
Complex (15+ sources, compliance requirements, embedded/customer-facing reporting): $150K–$250K+, and worth splitting into phases rather than one heroic cutover.

The spread inside each band is mostly driver #2. Two companies with identical source counts can differ 2x on cost because one has documented, comprehensible logic and one has a decade of archaeology.

Platform costs are separate and recurring: for mid-market workloads, expect roughly $2K–$10K/mo across warehouse compute (Snowflake, BigQuery, Databricks) and ingestion tooling once you're in steady state. Anyone quoting platform costs without seeing your workload is guessing; a two-week audit produces a real estimate.

The hidden costs nobody quotes

The parallel run. You pay for two systems for 4–8 weeks. That overlap is why the cutover itself is uneventful.
Report rebuilds. Dashboards pointed at the old warehouse need re-pointing, and a chunk of them were broken already. Decide deliberately which ones deserve to survive (usually fewer than you think, which pulls the budget back down).
Your team's hours. Interviews, validation reviews, access provisioning. Budget a few hours a week from the people who know the data.
Deciding what not to migrate. Every warehouse accumulates dead tables, and migrating everything "to be safe" means paying to move tables nobody queries. An audit that deletes before it moves pays for itself.

When you shouldn't migrate

We sell migrations, so read this list as against our own interest:

Your current setup is slow but correct, and the real complaint is dashboard usability. That's an analytics problem, not a platform problem, and it's cheaper to fix.
Nobody trusts the numbers. A new platform won't change that on its own (the doubt migrates along with the tables). Fix the modeling and testing layer first or alongside.
You're pre-product-market-fit with three sources and one analyst. Postgres and a BI tool are fine. Come back in a year.

How to get a real number

Our modernization engagements start with a one-week discovery: source inventory, logic audit, report tracing, and a fixed-price SOW with the phasing laid out. You can run the same exercise internally: count the sources, list the reports that can't break, and try to get one revenue number explained end to end. However long that explanation takes is your best single predictor of migration cost.

If you'd rather we did it, tell us what you're running today and we'll tell you which band you're in, within one business day.

Dashboards Nobody Opens: Diagnosing a Failed BI Rollout

2026-06-24T00:00:00Z

The pattern repeats across companies: buy Tableau, Looker or Power BI, spend a quarter building dashboards, hold a launch meeting. Six months later the CFO is still asking an analyst for one-off SQL pulls, and the license renewal is a line item nobody can defend.

The instinctive diagnosis is "wrong tool." In our experience running analytics engagements, it's almost never the tool. Unused dashboards have three root causes, and each has a different fix, so it's worth diagnosing before rebuilding anything.

Cause 1: The metrics were never defined

Symptom: two dashboards show different revenue numbers, and both are "right" by their own logic. One counts bookings, one counts invoiced revenue; one includes refunds, one doesn't. Nobody wrote down which definition is canonical, so every dashboard embeds its author's assumption.

Once a leader sees two conflicting numbers, they stop trusting both (rationally). From then on they ask an analyst, because the analyst can explain the number and a dashboard can't.

The fix isn't visual. It's a metrics layer: each metric defined once (i.e., in dbt or your semantic layer), version-controlled, tested & documented, with every dashboard reading from the same definition. This is transformation work, and skipping it is why the second BI rollout fails the same way the first one did.

Cause 2: The data underneath isn't trusted

Symptom: the dashboard was correct at launch, then a pipeline silently broke, and for two weeks it showed stale numbers before anyone noticed. One or two incidents like that and the team permanently downgrades the dashboard to "directionally interesting" and goes back to asking for pulls.

Trust in a report takes months to build and one incident to lose, and teams rarely re-adopt a dashboard they've caught being wrong. The fix is plumbing: freshness monitoring, data tests that fail loudly, an alert channel someone owns, and a visible "data as of" stamp so staleness is at least visible. If the underlying platform is the real constraint, that's a modernization conversation, but monitoring alone fixes more trust problems than replatforming does.

Cause 3: The dashboards answer questions nobody asks

Symptom: the dashboard has 40 charts, was built from "what's in the data," and answers everything except the question the VP has on Tuesday morning.

Dashboards get built forward from the schema (here are the columns, here's a chart for each) when they should be built backward from decisions. The method we use:

List the decisions. For each intended audience: what do they decide weekly or monthly? (e.g., pricing, hiring pace, campaign budget, inventory buys).
Map each decision to the 2–3 metrics that inform it. Not every metric that's related; the ones that would change the decision.
Write the definition of each metric before charting it (see Cause 1).
Build the smallest surface that covers the list. Usually this is 3–5 focused dashboards rather than 30, and every chart has to name the decision it serves.

A useful audit question for any existing chart: "what would we do differently if this number moved 20%?" If nobody can answer, cut the chart.

Which one do you have?

A quick self-diagnosis:

Conflicting numbers across reports, arguments about whose figure is right → Cause 1, fix the metric definitions first.
"The dashboard is wrong/stale" said from memory of a past incident → Cause 2, fix monitoring and rebuild trust deliberately.
Dashboards are accurate and fresh but meetings still run on exported screenshots and ad-hoc pulls → Cause 3, rebuild backward from decisions.

Most real situations are a blend, but one cause usually dominates, and fixing the wrong one first is how companies end up on their third BI tool with the same complaints (a tool switch changes the look, not the causes).

If you want a second opinion before committing budget, describe your setup in a few sentences: which tool, how many dashboards, and what leaders complain about. We'll tell you which cause it sounds like and whether it's worth an engagement or just a pointed internal fix.

The Modern Data Stack for Mid-Market Companies: What to Buy, What to Skip

2026-06-10T00:00:00Z

The "modern data stack" diagrams floating around have a dozen logos on them, and vendors are happy to sell you every layer. For a mid-market company (~50–1,000 employees, data team of zero to five), most of those layers are premature. The stack that works is four pieces, and the discipline is in what you don't buy yet.

This is the opinionated version we give clients in strategy engagements. We don't resell any of these tools or take referral fees, so the recommendations are just recommendations.

The four pieces you actually need

1. A cloud warehouse. Snowflake, BigQuery, or Databricks. Rough rule: BigQuery if you're already deep in GCP (the integration and pricing gravity is real), Databricks if ML workloads dominate your roadmap, Snowflake for most everything else. At mid-market scale the capability differences matter less than vendors suggest; pick for your team & cloud rather than the benchmarks. Postgres is a legitimate answer below ~50M rows and a handful of sources, and don't let a vendor talk you off it early.

2. Managed ingestion. Fivetran or Airbyte for standard sources (Salesforce, Stripe, HubSpot, ad platforms). Writing your own connector for a SaaS API that Fivetran already supports is a poor use of engineering time; save custom pipelines for the internal systems nobody sells a connector for. Watch consumption pricing as row volumes grow (this is the line item that surprises people at renewal).

3. dbt for transformation. This is the layer that turns raw tables into defined, tested, documented models, and the layer companies skip (which is why their numbers disagree across reports). dbt Core is free; the cost is the modeling discipline. If you adopt one practice from this post, adopt this one. (It's also most of what our transformation practice builds.)

4. One BI tool. Looker, Tableau, Power BI, or Metabase; the ranking matters far less than having exactly one, pointed at dbt-modeled tables rather than raw data. Metabase is underrated for teams without a BI specialist; Power BI is the path of least resistance in Microsoft shops. Two BI tools means two versions of every number.

Orchestration (Airflow, Dagster) earns its place once you have real dependencies between jobs; dbt Cloud or even scheduled queries are fine before that.

What it costs

Rough steady-state monthly numbers for a typical mid-market setup:

Warehouse compute: $500–$5K/mo (varies widely with workload; watch for dashboards set to refresh hourly)
Ingestion: $300–$3K/mo depending on connector count and volume
dbt: $0 (Core) to a few hundred/mo (Cloud)
BI: $0 (self-hosted Metabase) to $3K+/mo (Looker et al., per-seat)

Call it $1K–$10K/mo all-in for the platform. The larger cost is the people and the modeling work, which is where the build-vs-hire-vs-contract math comes in (getting that math right for your situation is most of what a strategy diagnostic is for).

What to skip (for now)

Each of these is a real category with real value at a scale most mid-market companies haven't reached. Buying them early adds integration surface and spend without moving a metric.

A data catalog. Under ~10 data people, a maintained dbt docs site and a naming convention do the job. Catalogs earn their keep when cross-team discovery is a daily problem.
Reverse ETL. Skip until there's a specific, named workflow ("scores back into Salesforce for the SDR team") with an owner. Bought speculatively, it becomes an unused pipe with a subscription.
MDM (master data management). A real discipline, priced and staffed for the enterprise. Mid-market entity-resolution problems are usually solvable in the modeling layer.
Streaming infrastructure. Kafka because "we might need real-time" is the classic over-buy. Most "real-time" requirements are actually "hourly is fine." Build streaming when a named use case (fraud, ops alerting, in-product features) needs sub-minute data.
A second BI tool. Someone will ask for one within the year. Say no and consolidate definitions instead.
An ML platform. Skip feature stores & model registries until at least one model is in production. Ship the first one with plain tooling.

The pattern behind the list: buy the tool when the problem has a name, an owner & a frequency, not because the reference diagram has a gap.

Sequencing, if you're starting from spreadsheets

Warehouse and ingestion in the first couple of weeks (a week or two of real work now), dbt models for your 10–20 core entities and metrics next (the actual work), BI on top of those models last. Resist building dashboards on raw data "temporarily": those temporary dashboards are the single most common mess we're hired to unwind in modernization work.

If you're weighing this build for your own company (what to buy, what your current setup is worth keeping, whether to hire or contract the work), that's the shape of our 2–6 week strategy diagnostic. Or just tell us where you are and we'll give you the two-paragraph version of what we'd do, free, within a business day.