What happens when a platform changes its API.

Platforms change what they return. They always have, and the changes are not equally survivable, which is worth knowing before you decide what to defend against.

• A version is retired

  Announced in advance, with a date. Calls fail, somebody is blocked, and it is fixed that day. This is the best case and it is the one everybody plans for, which is part of why the others go unhandled.

• A field is renamed

  The old name simply stops arriving. A pipeline that writes what it receives carries on, the column fills with nothing, and the table loads successfully. Every report reading that column now shows zero, which looks like performance rather than a fault.

• A field changes type or unit

  An amount reported in millionths rather than whole units, or a rate given as a fraction where it used to be a percentage. The rows arrive, the types are valid, and every figure is out by a constant factor that nobody spots because the shape of the chart does not change.

• A metric is redefined

  The name is the same, the type is the same, and what it counts has moved. There is nothing in the response to detect. This one is only findable by comparing your figure against what the platform's own interface now shows.

A pipeline that stops gets attention immediately, because somebody cannot do their job. The cost is a day and some irritation.

A pipeline that keeps running after a rename produces confident reports, for every client on that platform, for as long as it takes somebody to notice, and when it is noticed, every report issued in between is either re-sent or quietly left wrong. Both are expensive, and the second is expensive in the way that is hard to recover from.

So the instinct to invest in keeping the pipeline running is aimed at the cheaper failure. The investment that pays is detection, because the failures that matter here are the ones that do not announce themselves.

Each catches a class the others miss, which is why it is three rather than one.

• Schema, on every run

  Compare the columns you received against the columns you expected. Raise something on anything missing, anything new, or anything whose type moved. New columns are usually harmless and you still want to be told, because a new column is often how a rename announces itself.

• Volume, against the same weekday

  Row count for a source against the same day last week rather than against yesterday. This catches the partial response, where the call succeeded and returned a fraction of the data, which a schema check cannot see because the shape was perfect.

• A spot reconciliation, on a schedule

  One client account, one metric, one period, checked against the platform's own interface by a person. It is the only check that catches a redefinition, and it is the only one on this list that cannot be automated, which is exactly why it gets dropped.

Store the response before you transform it. It costs storage, which is the cheaper of the two bills a warehouse produces, and it pays for itself the first time a definition changes underneath you.

With the raw response kept, a changed definition is a re-derivation from data you already hold. Without it, the same fix means asking the platform for history it may no longer serve. It also turns the worst question in this job, whether a figure was always wrong or started being wrong, from an argument into a query.

When a tool is priced on connectors, or a data engineer is hired to own ingestion, this is most of what is being bought. Not the initial build, which is a known quantity and usually the part that gets estimated.

What is being bought is that platforms change and somebody notices. Costing a pipeline as a build rather than as a standing obligation is the single most common reason a home-built version is quoted at a fraction of what it turns out to cost, and the gap does not appear until the first silent change.

Connectors are maintained centrally, so a platform changing what it returns is handled once rather than separately for every client on that source. Missing connectors are built on request, triaged and tracked, which is the same work seen from the other end.

Syncs add to the tables rather than replacing them, so when a platform restates a day that had already closed, both the original figure and the correction stay on record. Freshness and threshold checks run against the modelled tables, which covers the case where a source stops arriving and the case where it arrives looking wrong.