Nice Lakehouse: Do You Own or Rent?

Dave: We're building a Data Lake.

Jen: Cool. What problem does it solve?

Dave: All of them.

Jen: Which ones specifically?

Dave: The data ones.

Jen: Do we have a data strategy?

Dave: The Data Lake IS the strategy.

Jen: That's like saying the swimming pool is the water treatment plan.

Dave: We're also hiring a Data Lake architect.

Jen: Do we have a data governance lead?

Dave: That's a different budget line.

Jen: It's the SAME problem, Dave.

Dave: Look, leadership wants a Data Lake by Q3.

Jen: Leadership also wanted a Data Warehouse by Q3 in 2018.

Dave: That was different.

Jen: Was it though?

Dave: ...

Jen: Where is that Data Warehouse now?

Dave: We don't talk about that.

Jen: You are building a swamp with a dashboard.

Welcome back. If you read Part 1, you already know the four words and the order they go in. If you didn't, go read it. We'll wait.

This one builds on it. In this series, Dave and Jen will have some interesting conversations which (could be) based on actual conversations in your Federal agency. Are you Dave or Jen in this story? More characters coming soon.

Part 1 was about what happens when agencies buy tools before building governance. Part 2 is about something more uncomfortable: why everyone involved (vendors, IT managers, contractors) has a vested interest in making sure the buying never stops. Regardless of whether the tools work.

Let's start with a question nobody wants to answer honestly.

Why is "build a Data Lake" always urgent?

Not important. Urgent. There's a difference. Important is "we need to understand our data before we can modernize." Urgent is "we need a Data Lake by Q3." Important means asking hard questions. Urgent means skipping them. And urgency, real or manufactured, is the engine that keeps this cycle running. "We are in a crisis, there's no time for this stuff!"

Here's what I've observed over decades in federal IT: the urgent need for vendors to sell "Data" solutions and the urgent need for agencies to implement them. Those are not two separate problems. They are the same problem. They are both symptoms of a confirmed, enterprise-wide absence of data governance. And here's the part that makes people uncomfortable: in the current model, everybody wins. The vendor wins a deal. The IT manager wins a bigger budget line. The contractor wins a support contract. The only thing that loses? The mission. And it loses quietly, over years, one seven-figure project overrun at a time.

(What's a Data Lake without governance? A very expensive swamp? A Data Puddle for $40M? You decide.)

WARNING: What follows is going to sound harsh. It's meant to.

The Tools Promotion Office

We love them, right? The people I mean.

In most large federal agencies, the IT department functions less as an objective technical authority and more as a tools promotion office. There are individual exceptions everywhere. Talented, mission-focused IT professionals who fight this pattern every single day. But the institutional pattern is clear: the loudest voice in the room isn't asking "what does the mission need?" It's asking "which platform should we buy?" And that person already has an answer. They've been to the demos. They've been to the conferences. They've got vendor swag on their desk and a relationship with the Federal Sales Engineer that pre-dates the RFI.

The evaluation was over before it started. Facts.

[TRUE STORY] Decades ago, one evening after 8pm working as a young network admin, I was waiting for all the staff to go home then update the LAN (uh... with 50 floppy disks). A senior federal IT colleague appeared on my floor. He was unloading boxes, lots of them, to every desk. "Don't mind me Greg! Everyone is getting a brand new color printer on this floor!" Me: "For what?" Him: "We have to spend IT budget before end of September." I was maybe 21 yrs old in a server closet (also my office) and had already asked my one allowed question. I knew nobody on my floor "needed" a color printer but someone else "needed" to buy 20 of them to maintain the annual budget request to OMB. I find the phrase "hey, it was the 80's" works often, but as each decade since has come and gone, some might say not much has changed, unless you work for GAO.

Is this procurement gone wild or culture? You decide, but in most agencies it's a system that rewards procurement over outcomes, platform adoption over mission impact, and budget growth over operational maturity. The IT manager who procures a seven-figure platform gets promoted. The IT manager who spends two years building a data governance framework that prevents the need for that platform? Nobody writes that success story. There's no ribbon-cutting for "we didn't buy something."

And the vendors? They're doing what vendors do. They sell. They're good at it. The demo is always incredible. The deck always aligns with your strategic plan. The references are always glowing. I am not anti-vendor. I am pro-honest-outcomes. Although not a catchy buzzword, let's face it, the vendor's job ends at the purchase order. Your job doesn't. And the gap between "purchased" and "operational"? That's where seven figures go to die.

"We need a Data Lake" = Federal Enterprise Surrender

So when someone in your agency says "Our problem is that we need a Data Lake!" (eyes wide, voice rising, hands gesturing at an imaginary architecture diagram like they just discovered religion)...

...ask yourself: is this a strategy or is this a purchase order dressed up as one?

A familiar pattern. Do you recognize it?

The cries for a Data Lake almost always indicate what I call enterprise surrender. The moment an agency admits it cannot make sense of its own data. The frustration is legitimate. The data is scattered across 40 systems. Nobody trusts it. Getting a straight answer to a simple question requires calling three people and opening four spreadsheets. The Human Middleware from Part 1? Still running. Still duct-taping answers together every morning before leadership walks in. "Happy Monday. I can't wait to see our Dashboard."

But the instinct to reach for technology (something big, something shiny, something with a logo and a sales engineer) is the instinct that's been costing us millions for decades.

Buying a Data Lake to solve a data governance problem is like buying a gym membership to fix a broken leg. You're not wrong for wanting to be healthy. You're just starting in the wrong place. And the gym doesn't care. They got your money.

The lifecycle of a Data Lake (without governance)

Here's where the pattern gets expensive. I've seen this play out enough times to set my Apple iOS watch by it.

Are AI platforms the Data Warehouses of 2016? Let's hope not.

[side note: I've seen this movie before. In the late 2000s it was Business Intelligence platforms. In the mid-2010s it was Data Warehouses. By 2016 it was Data Lakes and now it's AI. The vendors change. The acronyms change. The PowerPoint decks get prettier. The outcome doesn't change.]

I am witnessing a project now which has stuffed 200+ documents (TWO-HUNDRED) into an agency AI chatbot which will be flung at light speed into the workforce. Context? None. Structure? Meh. Use cases? Zero. Next train stop: Hallucination Island. Success? Not a flippin chance unless you define the purchase itself as success. What could possibly go wrong using AI? Can't wait to read the "Lessons Learned" report in 2027.

You cannot "Data Lake" what you cannot govern. Same as you cannot "AI" what you cannot govern. Same answer. Same four words.

The Data Lake IS legitimate (when it's earned)

Now, to be fair. A Data Lake absolutely has a place. This is not an anti-Data Lake article. For silo-based analytics use cases, a Data Lake can and should exist as part of a permanent, live, operational Data Mesh. One does not replace the other, despite what many are led to believe by vendor whitepapers that conveniently blur the line between the two.

A Data Lake and a Data Strategy are two very different topics. One is a thing you build. The other is how you make sure the thing doesn't become the most expensive puddle in the federal government.

Your systems and databases aren't waiting to be centralized. They're a decentralized Data Mesh. They always have been. They always will be. Every large agency is a collection of Lines of Business, each with their own systems, their own data, their own use cases, and their own operational reality. Pretending you can pour all of that into one lake and have it magically make sense is like expecting a library to organize itself just because you put all the books in the same building.

(More on Data Mesh in Part 3. It deserves its own deep dive.)

The question that exposes everything

Here's the question that exposes the gap every single time: who owns the data going into your Data Lake?

Not who maintains the system. Not who manages the servers. Not who has admin credentials. Who owns the data: the business rules, the authoritative version, the lineage from source to destination? If nobody can answer that question? You don't have a Data Lake project. You have a very expensive ingestion pipeline with no quality controls. Congratulations. You just automated the chaos. Enter the spreadsheet jockey, right on cue.

Ask four questions before you write a single purchase order:

1. Does your agency have a documented, enterprise-wide data governance framework? Not policy sitting in a binder collecting dust, but an active, living framework with named data owners, defined authoritative sources, and verification and validation processes?
2. Are your data standards defined at the enterprise level? Not just inside each silo?
3. Do you know which systems hold the authoritative version of your most critical data? And does everyone agree? [side note: they don't]
4. Can your enterprise describe itself in three seconds during a crisis, or does someone need to open a spreadsheet?

If you can answer yes to all four, a Data Lake might be exactly the right next step. If you can't? The Data Lake isn't your problem. It's your escape from the problem. And it's going to cost seven figures to learn that lesson again.

The real conversation: buying over governing

Here's what nobody wants to say out loud. The problem isn't that agencies buy bad tools. The tools are fine. The problem is that the entire incentive structure (from vendor compensation to IT budget cycles to how federal managers get promoted) rewards buying over governing. Until data governance becomes a 1st Class discipline with the same organizational weight as program management, the pattern will repeat. Different tool name. Same outcome. Same landfill.

The good news? This is fixable. And it doesn't require a seven-figure purchase order.

Document your authoritative sources. Define your enterprise data standards. Assign data ownership that lives with the mission, not with IT. Build the foundation. Then, and only then, let the tools do what they were designed to do.

When you do this right, something remarkable happens. The tools you already own start working better. The APIs start delivering trustworthy data. The dashboards stop lying. And the next tool you buy? It walks into an environment where it can actually succeed. That's not a vendor pitch. That's math, not magic.

The vendor who understands this and helps the agency mature its data environment before pushing product? That vendor earns a partnership that lasts. The IT manager who champions governance over procurement? That person changes the trajectory of the entire enterprise. And the agency that gets the four words right? They stop feeding the landfill and start building something that actually works.

Four words. Same order. Same answer.

Data Governance. Data Standards. Data Management. Data Exchange.

The lake can wait. The governance can't.

Part 2 of The Governance Series by daitagov. First published on LinkedIn.