How we cover every critical data element (CDE) without writing a rule for each one.
Where we started
Policy asked for one rule per critical data element
30,000rules needed, each one built and maintained with a steward
325 rules in place todayRules the policy would still need
It gets worse
The CDE tag follows the data into every copy
CDECRM applicationCustomer enters their emailOwn rules, own steward
CDELake raw zoneDaily load, untouchedOwn rules, own steward
CDELake curated zoneCleaned and joinedOwn rules, own steward
CDEPower BI dashboardWhere people read itOwn rules, own steward
If every tagged copy needs its own checks, one customer email becomes four sets of rules. The workload grows every time data moves.
The insight
Not all data carries equal risk
Wrong value has consequencesCustomer name in a regulatory submission
Regulatory, financial or customer-facing impact when it fails.
Wrong value is an inconvenienceInternal product code in one team's report
Worth watching, not worth a steward's weekly attention.
One rule per CDE spends the same effort on both. So we stopped counting rules and matched the depth of control to the risk.
The model
Automation covers everything. Judgment goes where the risk is.
Tier 1 · Curated rulesHighest-risk CDEs only
Steward-set thresholds, a named owner and a remediation path.
Tier 2 · Automated completenessEvery CDE
One line of configuration per CDE. No rule-building, and the policy is met.
Tier 3 · ObservabilityEvery CDE
Learns what normal looks like for each field and flags unusual change.
Tiers 2 and 3 apply to every CDE automatically. The only placement decision is which CDEs earn Tier 1, typically 10 to 20 percent.
Why Tier 1 stays small
A curated rule costs more than it looks
Steward timeA strong rule takes expert hours
Thresholds, valid values and a remediation path all come from a steward. Their time is the bottleneck.
Reporting noiseMore results, less attention
Every Tier 1 failure goes to a named steward. With too many, the ones that matter get lost.
Processing costHeavier checks cost real money
A curated rule tests several dimensions per CDE. That compute lands on the bottom line.
Finding the balance is the job. Too few CDEs in Tier 1 leaves high-risk failures without an owner. Too many overwhelms the people who fix them.
Why all three
Each tier catches what the others miss
Tier 1 · CuratedUNKNOWN typed into a regulatory field. The field is filled, so only a validity rule with an owner catches it.
Tier 2 · CompletenessA field that has gone blank, on any CDE, including the ones nobody had time to write a rule for.
Tier 3 · ObservabilityA null rate drifting from 1.1% to 4.2%. No threshold was crossed, but something changed.
The authoritative data source
Check once at the source. Every copy inherits the result.
CDECRM applicationAuthoritative data sourceAll three tiers run here
→
CDELake raw zoneConsumption pointTransfer check only
→
CDEMarketing curated zoneInherits the result
→
CDECampaign dashboardsDisplay the result
CDEFinance curated zoneInherits the result
→
CDEBilling dashboardsDisplay the result
CDEService curated zoneInherits the result
→
CDEService dashboardsDisplay the result
and every curated zone added later
The tag travels. The obligation does not. One raw zone element is copied into many curated zones and many dashboards, but the rules run once. For a vendor feed we cannot reach, the source is the raw landing zone.
Placement is not permanent
CDEs move up and down as risk changes
Tier 1 · curated rules
Tiers 2 and 3 only
Customer phone ↑Internal code ↓
PromoteRepeated completeness failures or recurring anomalies show the risk is higher than first judged.
DemoteNo regulatory, financial or customer-facing risk. Stepping down frees steward time for CDEs that need it.
Tiers 2 and 3 keep running either way. Stewards and data owners make the call, because they know the business consequence.
The point
Rule count is not a measure of maturity
Three tiers give every CDE baseline coverage and give the riskiest ones real accountability.
The authoritative data source means each CDE is checked once, not once per copy.
Promotion and demotion keep steward effort pointed at current risk.
Policy met
Every CDE has documented coverage that fits its risk.
Stewards in control
Their time goes to the CDEs that matter, with alerts they can act on.
What we measure instead: coverage quality, actionable alerts and the capacity to fix what they find.