The Black Hole

Your cloud database bill just arrived, and it’s a real doozey. Worst yet is the fact that it is a black hole. In other words when you ask for the spare change in the companies couch cushions to pay for it, you have no idea where the money is going when asked. You expect clarity on your bank/credit card statements. In this post I’m going to explain the hidden lens that can be used to gain it with your cloud database bill as well. And help you bring what’s really driving your cloud database spend into clear focus.

Clear Accountability

The first step would be ensuring clear accountability. That means you should be able to track:

• Which business unit spent which part of that $85,200? Perhaps you went to a cloud database environment to support sales initiative. Is that what it’s being used for?
• Which projects are consuming what dollar amounts of spend? Every project in the organization sounded good at some point to the person who started it. But do the projects have a sound return on investment or are we throwing money down the drain.
• Which environments are the most costly? Is our development environment used way more than our production environment?
• Whether the spend  is going towards populating your cloud database, or transforming values, or reading the data or AI against the data?

Your black hole, should begin moving forward on the path of insight.

Actionable Intelligence

In my eyes those items for clear accountability are just the minimum you need to ensure your billing data is brought into focus. However, for actionable intelligence we need our lens to zoom a little more so that we can track:

• What priority are each of the processes that are running? Are we spending more on what the company deemed as high priority, or is more being spent on nice to have tasks?
• What applications are driving the spend? Is your IT spend focused on the cost of the tools, or can you optimize your spend with a different tool?
• Which deployments are you running and what is each costing? Your company set out to modernize it’s data strategy so does your cloud database spend reflect that?
• Who are the owners of the processes? The process owners are typically different than the people running the queries against your cloud database.

The Hidden Lens

If you are anything like me you might have a lifetime of pictures at your fingertips. But who in the world is that adorable baby covered in cake icing? Is that your own daughter, or your Uncle Bob’s, neighbors, best friends daughter that you met one time on vacation? I guess we should have written something on the back of the photos we developed 42 years again. We had access to that important information, but at the time like me I’m sure you just assumed you would never forget that moment. Without recording it, that knowledge is now hidden from us and the picture is pretty much a throw away, instead of a legacy picture.

Just like an unlabeled photo eventually loses its meaning, an unlabeled project in your cloud database quickly loses its context — and when the context is gone, so is your ability to know whether the ongoing cost is justified.

Imagine Carol’s manager frantically tells her that the CEO must have access to some information from some system(s) for the Board and they need it YESTERDAY. So, Carol rushes off and builds what is needed and delivers what we will call Project Big Cheese. Imagine that Project Big Cheese is responsible for $3,371. It’s the price of doing business. Right?

Certainly, but it also means that every single day a $3,371 is being added to your cloud database bill. What if Carol had labeled the picture, I mean her work, with information about it being for Project Big Cheese?

Well you were in the meeting as well, and you know, that Project Big Cheese was a 1 time thing. The company needed that information for a bond renewal that you got. But Carol’s manager seems to have forgotten to give her that key piece of information. If you had the clarity into the charges on your bill, you could stop the project. But you don’t. So, after a year you have accumulated $1,230,415 in completely unnecessary charges. So sorry! I probably should have lead with that, instead of burying it down here on page 12.

In data terms that information you would write on the back of your photos, if you choose, is called “metadata.” Your data and analytics teams have access to that traceable knowledge when they they run their queries to insert data to, transform data in, or consume data from your cloud database. All the major cloud database providers (Snowflake, Databricks, Amazon, Microsoft and Google) provide the ability for your data teams to “tag” the data flowing in or out with that “metadata.” Thus they literally provide a lens, but it’s hidden from view unless your organization wants to use it.

Put another way, think of that of you and your college mates going over Niagara Falls in a barrel. Which would have been a classic, had your goofy friend not left the lens cover on. Your organization needs to take action to remove the cover, by adding tags, in order for the lens to be of any use.

Query Tagging

Without getting into all of the technical weeds, these “query tags” are the hidden lens that provides your organization with clarity into it’s spend. That’s what financial operations is all about for leadership.

For those reading this who deal with the data, understand they are more than just a #finops checkbox. It’s what makes your work visible, accountable and visible to leadership. It’s the checks/balances that are completely missed on all of the frantic calls. They are what provide you insight in reviews of your spend. Obviously each organization will differ in size and intent and the way costs are broken down. As a starting point I recommend this “minimum” list of values to ensure accountability:

• • Business Unit
  • Project
  • Environment
  • Process

If your organization wants full traceability then I recommend this extended “tag” list:

• • Business Unit
  • Project
  • Environment
  • Process
  • Priority
  • Application
  • Deployment
  • Process Owner

🧠 If you are the type who lives in the technical weeds be sure to read my follow-up post “Query Tags: Turning Precision into Visibility.” 🧠