Back to resources

Resources

How to write metric definitions that hold across clients.

Most arguments about a number are arguments about what the number means. Write it down once.

  • 6 minute read
  • 5 sections

Part 1

The argument you keep having

Two people bring two figures for the same metric to the same client meeting. The next hour goes on working out which is right, and it usually ends with both being right, because they were measuring different things.

That meeting is not a data quality problem. It is a definition problem, and it repeats because nothing was written down after the last one. A metric definition is the artefact that ends it.

Part 2

What a definition has to contain

Six parts. A definition missing any of them will produce the same argument again.

  • The name, exactly as it appears in the data

    Not a friendly label. The column name, so anyone reading a query and anyone reading the definition are looking at the same thing.

  • What it counts, in one sentence

    Plain words, no formula yet. If you cannot say it in a sentence, the metric is doing two jobs and should be two metrics.

  • The grain

    What one row is. Per day, per campaign per day, per customer. This is the part most often left out and the part that causes the largest errors when a table is joined.

  • What is excluded

    Test transactions, internal traffic, refunds, a particular account. Exclusions are where two honest implementations diverge, so they belong in the definition rather than in the query.

  • The currency and the timezone

    Both, always, even when there is only one of each today. The day you add a second market, every definition without them becomes ambiguous at once.

  • Who decides

    A name. Definitions need an owner, because the question is not usually what the metric is but whether it should change, and that is a decision rather than a lookup. Where the client defines the thing being counted, such as a qualified lead, that owner sits on the client's side.

Part 3

Where to keep it

Next to the data, not in a document nobody opens. A definition that lives in a shared drive is a definition that will be out of date within two quarters and will not be consulted before somebody writes a query.

The version that works is the one attached to the thing it describes: a comment on the column, a description in the modelling layer, a line in the same repository as the transform. Then the person most likely to get it wrong is the person most likely to see it.

Part 4

Start with the five you argue about

Do not try to define every metric. That project is always abandoned, usually around the fortieth definition, and the forty produced are already stale.

Write down the ones that have caused an argument. For most teams that is spend, conversions, revenue, an acquisition cost and one channel definition. Five definitions that end five recurring arguments are worth more than a complete dictionary nobody reads.

Across a roster, write them once for every client and record exceptions by client name. A client who counts a booked call as a conversion, where everybody else counts a form, is an exception worth a line in the definition rather than a quiet change to one report.

In practice

Where SightShark fits

The modelling layer is where these definitions become enforced rather than agreed. One name per metric, one type, one grain and one currency, across every source and every client, applied in SQL inside a warehouse you own.

Because the transform is readable in your own account, the definition and the implementation are in the same place. Anybody who wants to know how a figure was built can read how it was built, which is the only version of a metric dictionary that cannot drift from the data.

See how the platform works

Questions

Questions people ask

What should a metric definition include?

Six things: the column name as it appears in the data, one plain sentence on what it counts, the grain or what one row represents, what is excluded such as test orders or internal traffic, the currency and timezone, and the name of the person who decides when it changes. Missing any of these reopens the argument.

Where should metric definitions be stored?

Next to the data rather than in a separate document. A definition in a shared drive goes stale within a couple of quarters and is not consulted before somebody writes a query. Attached to the column or held in the same place as the transform logic, it is seen by the person most likely to get it wrong.

How many metrics should we define first?

Only the ones that have already caused an argument, which for most teams is about five: spend, conversions, revenue, an acquisition cost and one channel definition. Attempts at a complete dictionary get abandoned partway through, and the definitions written by then are already out of date.

Pricing

Priced by the clients you report on.

Compare plans
  • Starter

    $499 /month

    Up to 20 clients

  • Growth

    $999 /month

    Up to 50 clients

  • Enterprise

    Custom

    No cap on clients

Get started

Want this running on your own data?

Book a call with someone who can answer the technical questions.