Inmon and the Corporate Information Factory: one integrated core, many marts

Also called: CIF, the enterprise data warehouse, the 3NF core, top-down data warehousing

The approach that treats integration as the hard problem and reporting as the easy one — and is right about that whenever more than one system describes the same customer.

What it is

Bill Inmon’s warehouse has two layers with two different jobs. The core is normalised, integrated, and modelled on the business rather than on any one system: one customer, one definition, whatever each source calls them. The marts are dimensional, built on top of the core, and shaped for whoever is asking.

Nobody queries the core directly. It is the single version of the truth and it is not built to be read — it is built to be right, and to stay right while systems come and go. The marts are the readable surface, and they are deliberately disposable: rebuild one whenever the questions change.

This is the “top-down” half of the old Inmon-versus-Kimball argument. Model the enterprise first, then serve the departments. Kimball’s bottom-up answer — serve one process well, then conform the dimensions — is faster to first value, and the two camps have long since stopped pretending either is universal.

How the tables are actually shaped

The core: third normal form, and time

The core looks like a well-designed operational schema with one difference: it keeps history. Entities are normalised, relationships are real foreign keys, and each entity version carries the period it was true for.

Ent_Customer
  CustomerKey     INT       -- surrogate, STABLE across every version
  EffectiveFrom   DATETIME2 -- key is (CustomerKey, EffectiveFrom)
  EffectiveTo     DATETIME2
  IsCurrent       BIT
  CustomerNumber  NVARCHAR  -- business key
  SourceSystem    NVARCHAR  -- business key: the same customer, twice over
  Name, Address, ...

• The surrogate key identifies the customer; EffectiveFrom identifies which version of them. A changed customer reuses its key and gains a row. Only a genuinely new one takes a new key.
• A version is matched on the business key and the source system, because the entire point of the core is that the same customer arrives from two places and has to be reconciled rather than duplicated.
• A key map records which source key became which core key — derived from the core after it loads, since nothing in the source could say that two IDs are the same person.

The marts: current-only, and rebuilt

Mart dimensions read the core with IsCurrent = 1 and are replaced whole on each run. That sounds wasteful and is the correct trade: the history lives in the core, so a mart has nothing to preserve, and a mart that can be rebuilt from scratch is a mart you can change your mind about.

It does mean mart surrogate keys move between runs. Acceptable precisely because the facts are rebuilt in the same run — but it is the reason nothing outside the mart may store one of its keys.

Where staging fits

Staging lands the source data, the core integrates it, the marts present it. Three layers, and the middle one is the only one that is really Inmon — the outer two are what every methodology does under different names.

When it fits, and when it does not

The question that decides this one is not technical. It is whether your organisation is willing to argue about what a customer is, and then write the answer down.

Those two lists are Netune’s own judgement of this methodology, copied from the rule file it scores a real database with.

What it costs

The mistakes that are expensive

Treating the marts as a later phase

The core alone reports on nothing. A project that spends its budget perfecting the integration layer and leaves the marts for next year delivers a warehouse that is finished and unusable at the same time — which is how this methodology earned its reputation for slowness.

Budget for both, and get one thin mart working end to end before the core is anywhere near complete.

Building a core when there is nothing to integrate

With a single source system the integration layer is doing very little, and a star schema reaches the same reports with half the work. The core earns its cost when two systems disagree about the same entity — and not before.

Letting the core surrogate key move

The one invariant here is that a customer’s key is the same in every version of them. A load that renumbers from scratch turns the history into several unrelated customers, silently, and every fact pointed at the old numbers is now pointed at somebody else. Assign from the current maximum and reuse the existing key wherever the business key matches.

How it compares

Inmon or Kimball?

The honest answer is: how many source systems, and how much time. Kimball gives reports sooner and handles one or two well-behaved sources beautifully. Inmon handles the case where the sources contradict each other and somebody has to decide who is right — and that decision has to live somewhere other than in each report.

In practice most large warehouses end up with both: an integrated core and dimensional marts. That is Inmon’s design, and it is also what a Kimball shop builds after its fifth star.

Inmon or Data Vault?

Both put an integration layer between the sources and the reports, and both keep history there. The difference is what happens when a source changes shape: Data Vault absorbs it by adding tables, while an Inmon core absorbs it by somebody remodelling the entity. Vault is more mechanical and less opinionated; the core is more readable and asks more of its modellers.

Questions people ask