Back to resources

Resources

What a marketing data warehouse costs to run.

Two bills, and only one of them is decided by how your queries are written.

  • 7 minute read
  • 6 sections

Part 1

The two bills

Storage is the first: what you keep, charged by how much of it there is and how long you keep it. It is the predictable one. It grows slowly, in a straight line, and you can forecast it from how many clients and sources you have connected.

Compute is the second: what you ask. It is charged either by how much data each query reads or by a block of capacity you reserve in advance. This is the one that surprises people, because a table costs almost nothing while it sits there and costs something every single time anybody looks at it.

Part 2

What actually drives the bill

On the usual pricing model you pay for the data a query reads, not the answer it returns. A query that produces one number can read a year of rows to produce it, and it is billed for the year.

That single fact explains most of the surprises. Cost tracks what was scanned rather than what was shown, so the cheapest looking thing on a dashboard, a headline figure in a large font, is often the most expensive query running anywhere in the agency.

Part 3

The levers, in the order worth pulling

Roughly in order of what they return for the effort. The first two are most of the saving.

  • Ask for the columns you need

    Selecting everything is the most expensive habit on a columnar store, because you are billed for the columns a query touched rather than for the table it named. Naming six columns instead of sixty is a proportional saving and it takes a minute.

  • Partition by date

    A table split by day lets a query about last month read last month. Without it every query reads all of history, which means the cost of running an unchanged dashboard rises every month purely because you have been collecting for longer.

  • Cluster by what you filter on

    Usually the client, the account or the campaign. It is the column that appears in the WHERE clause of nearly every query you run, and ordering the data by it lets a query skip the rows it was never going to use.

  • Pre-compute anything read repeatedly

    A summary built once a day is read many times. The alternative is every viewer re-deriving the same figure from raw rows, paying for it separately, and getting the same answer.

  • Point dashboards at modelled tables, never raw

    Raw exports are wide and deep because they carry everything the platform sends. A modelled table carries what the question needs, which makes it both cheaper to read and the same shape as every other source.

Part 4

The pattern that quietly costs the most

A dashboard pointed at a raw table, refreshing every time somebody opens it, copied for every client and opened by a lot of people. The cost is the size of the query multiplied by the number of viewers multiplied by how often each one looks, once for every client.

Nobody involved is doing anything wrong, which is why it survives. It grows with headcount and with the roster, it arrives as a slowly rising total rather than an incident, and it does not appear as a line item that anybody owns. If you look at one thing after reading this, look at which dashboards read raw tables and how many people open them.

Part 5

Deciding what to keep

Deleting history is rarely the right saving. Storage is the smaller and steadier of the two bills, and history is the one thing you cannot buy back later once a platform's window has closed on it.

The better version is to change how old data is held rather than whether it is held. Keep recent periods at full grain where they are queried often, summarise older periods to the grain anybody actually asks them at, and move what is left to colder storage. That reduces what gets scanned, which is the bill that was large, rather than the one that was not.

In practice

Where SightShark fits

The warehouse is your own, in your own cloud project, so the cloud bill arrives to you directly and itemised rather than bundled into somebody else's pricing. That is worth stating plainly because it cuts both ways: you see exactly what the roster's data costs, and it is yours to control.

The modelling step is what makes the levers above the default rather than a project. Dashboards read tables shaped for the question rather than raw exports, staging and production stay separate so experiments do not run against the tables reports depend on, and daily syncs add to the tables rather than rewriting them.

See how the platform works

Questions

Questions people ask

What drives the cost of a data warehouse?

Two things. Storage, charged by how much data you keep and for how long, which is predictable and grows slowly. Compute, charged either by how much data each query reads or by capacity reserved in advance, which is the variable one. On the usual model you pay for the data a query scanned rather than the answer it returned, so a single headline figure can be an expensive query.

How do you reduce data warehouse query costs?

Name the columns you need instead of selecting everything, partition tables by date so a query about last month reads only last month, cluster by whatever you filter on most often such as the client or account, pre-compute anything that is read repeatedly, and point dashboards at modelled tables rather than raw exports. The first two usually account for most of the saving.

Should I delete old marketing data to reduce cost?

Usually not. Storage is the smaller and steadier of the two bills, and history cannot be recovered once a platform's retention window has passed. Changing how old data is held works better than deleting it: keep recent periods at full grain, summarise older periods to the grain they are actually asked at, and move the remainder to colder storage.

Pricing

Priced by the clients you report on.

Compare plans
  • Starter

    $499 /month

    Up to 20 clients

  • Growth

    $999 /month

    Up to 50 clients

  • Enterprise

    Custom

    No cap on clients

Get started

Want this running on your own data?

Book a call with someone who can answer the technical questions.