Nobody Knows What Your Company Spends on AI

It is six in the evening and the working day is over. Somewhere in the company, a support agent pasted three customer emails into a chat window to get help drafting a reply. A lawyer uploaded a draft contract to summarize its risk clauses. A developer ran an agent across a private repository for four hours. A marketing manager generated forty product descriptions. An intern, told to be resourceful, put an export of the CRM into a prompt to find patterns in it. Some of those people used a company account. Most used a personal one, because it was faster and nobody said not to.

Now ask the questions any board would eventually ask. How much did the company spend on artificial intelligence today? Which departments spent it? Did any regulated data leave the building, and under which provider's terms? Did any of it produce something worth the money? In the overwhelming majority of organizations, the honest answer to all four is the same: we do not know, and we have no mechanism that could tell us.

This is not a failure of discipline, and the people in the story are not doing anything unreasonable. It is a structural gap. The tools arrived faster than the controls, they are consumed through a channel nobody instrumented, and they are billed in a unit that no finance process was designed to handle. The instinct in most companies is to close the gap with a policy: publish an AI charter, list the approved tools, forbid the rest. That instinct is wrong, and this article is about why, and about what actually works instead.

Shadow IT at least left a receipt

We have seen this shape before. In the SaaS era, teams bought tools their IT department had never heard of, and the industry called it shadow IT. The comparison is useful but it flatters the current situation, because shadow AI is worse in three specific ways.

Shadow IT left artifacts. Someone had a subscription, on a corporate card, renewing monthly, with an invoice arriving in an inbox. Finance could find it. There was an account with an admin, a domain, a login page, a contract with terms someone could eventually read. The trail was thin but it existed, and enough companies eventually built SaaS discovery tooling on exactly that trail. A prompt leaves nothing. It is a request, a response, and a token count on a bill that belongs to the employee's personal credit card. When the employee leaves, the trail leaves with them.

Shadow IT moved data on purpose. Uploading a customer list to an unapproved CRM was an act with intent behind it, and it was rare enough to be an incident. In the AI case, moving data is the interaction. There is no way to ask a model for help with a contract without sending it the contract. Exfiltration is not the abuse of the tool, it is the normal, correct, everyday use of the tool. Any control model built on the assumption that data movement is exceptional will be wrong on the first day.

Shadow IT had a fixed price. That is the difference that breaks the finance side. A SaaS seat is a known number per user per month, forecastable a year ahead, and the worst case equals the best case. Inference is metered by the token, which means consumption is unbounded by construction and the same headcount can produce a bill that varies by an order of magnitude between two consecutive months. There is no seat count to multiply. This is the observation that leads most executives to the question that opens the next section, and it is worth taking seriously rather than dismissing.

The three unknowns

Strip the problem down and there are exactly three things a company cannot currently answer about its own AI usage.

The first is attribution: who spent what, on which model, for which team, on which project. This is an accounting question. It has a well understood shape, it maps onto structures the company already has, and it is entirely tractable with the right plumbing.

The second is exposure: what data left the organization, to which processor, under which retention and training terms, and whether any of it falls under a regime that requires a declaration. This is a legal and security question. It is harder, because answering it fully requires inspecting content, and inspecting content creates its own problems.

The third is value: what came back. Did the four hours of agent time save a day of work or produce a branch nobody merged. Did the forty product descriptions convert. This is the question everyone actually cares about and the one nobody in the industry can currently answer with rigor, including the vendors selling the answer.

These three are not independent, and their relationship is the argument of this article. Attribution is the only one that is cheap to measure. It is also the one that, once you have it, materially advances the other two. A system that knows which key made which call, to which model, with which volume, is one configuration change away from also knowing which calls carried a redacted payload and which team is producing them. You do not get exposure control and value measurement by demanding them directly. You get them as a byproduct of building the accounting layer, because the accounting layer is the point of passage that all three questions need.

That is the whole strategy in a sentence. Chase the money, because the money is the only thread you can pull that drags the rest along with it.

Why the AI charter does not work

The standard first response to shadow AI is a document. It names the approved tools, forbids the others, tells employees not to paste confidential information into public models, and asks them to acknowledge that they have read it. Some version of this document now exists in most large organizations. Almost none of them are enforced, because none of them can be.

A rule is only as real as the mechanism that observes it. A policy that says "do not send customer data to unapproved models" has no observer. Nothing in the network stack, the endpoint, or the identity provider is watching for it by default. There is no gate the employee passes through where the rule could be applied. The rule is therefore not a control, it is a statement of preference with a signature attached, and its main function is to move liability from the company to the employee. That may be worth something to a legal team. It is worth nothing to the person who has to answer the board's four questions.

Security learned this lesson twenty years ago with web filtering, and again with data loss prevention. Nobody writes a policy saying "do not visit malicious sites" and considers the problem solved. They put a proxy in the path, because the proxy is what turns the sentence into an outcome. The same logic applies here and it is not a new insight, but it has a consequence for AI that is easy to miss, and the rest of this article turns on it.

There is a second, sharper reason the charter fails. The prohibited behavior is also the productive behavior. Web filtering works partly because the employee gains nothing from the blocked site. Here, the employee who ignores the policy does better work, faster, than the one who follows it. A control that makes compliant employees less effective than non compliant ones will lose, every time, regardless of how senior the person who signed it. Any workable model has to be one where using the sanctioned path is the easier path, not the more virtuous one.

The fixed budget is the wrong question

When the charter fails, the next move is usually financial: give the AI spend a line in the budget. Allocate a fixed amount per employee per year, approve it once, and treat it like any other cost center. And then the objection arrives immediately, and it is a real one: these services are billed per token, so the amount consumed cannot be known in advance, so a fixed budget cannot be committed in advance. The finance director is right about the mechanics and stops there.

But look at what the objection actually says. It says: this cost is variable, consumption driven, unbounded in principle, attributable only after the fact, and generated by decentralized decisions made by people who do not see the price of what they are doing. That is not a new category of problem. That is a description of public cloud, circa 2010, almost word for word.

No company has a fixed AWS bill either. What they have, if they are mature about it, is something better than a fixed bill: they have per team attribution, a unit cost they track, a forecast with a known error range, budgets with alerts, and a set of guardrails that make the pathological cases impossible rather than merely discouraged. Nobody asks the cloud provider for a flat rate. They build the practice that makes a variable cost governable. That practice has a name, a body of work behind it, and a foundation that maintains its framework. Turning it on AI workloads is not an analogy stretched for the sake of an article. It is the same problem with a different meter.

So the question to bring to the finance director is not "how much will we spend on AI next year". It is "what is our unit of consumption, who owns it, and what is the maximum we can lose in a day". Those three have answers.

What FinOps actually offers here

The FinOps lifecycle is usually described in three phases, and each one maps onto the AI problem cleanly enough to be worth walking through. The Foundation has since expanded the framework into domains and extended its scope beyond public cloud to cover SaaS and AI workloads, but the original three phase framing remains the clearest way to sequence the work.

Inform comes first and it is the phase most companies skip. Before any budget, any limit, any policy, you need visibility: a showback that tells each team what it consumed last month, broken down by model and by use case. Showback is not chargeback. Nothing is billed internally yet, nothing is blocked, the only output is a number sent to a team lead who has never seen one before. The effect of that number is consistently underestimated. A large share of wasteful AI spend is not malicious or even deliberate, it is a default that nobody revisited: a frontier model used for a classification task a small one would do, a system prompt carrying twelve thousand tokens of context on every call, a nightly job that reprocesses the same documents. None of that survives contact with a team lead who can see it.

Optimize comes second, and it has more levers than the discourse suggests. Model selection is the obvious one, and the largest: the price spread between the cheapest usable model and the most capable one is not a few percent, it is a multiple, and a great deal of production traffic is running on a model chosen once during a prototype and never reconsidered. Context size is the second, and it is where the real waste hides, because prompt tokens are billed on every single call and a bloated system prompt multiplies its own cost by the request count. Caching is the third, and it is close to free money for repeated prefixes. Batching is the fourth where latency permits. None of these are visible without the first phase, which is why the order matters.

Operate comes last: budgets that reset on a schedule, alerts that fire before the limit rather than after it, rate limits that contain a runaway process, and an allowlist of models per population. This is the phase people want to start with, because it feels like control. Starting here produces a system that blocks work nobody understood the shape of, generates escalations, and gets switched off within a quarter.

The token is not a business unit

The most important thing FinOps brings to this problem is not the tooling, it is the insistence on a unit. Cloud FinOps matured the day teams stopped reporting a monthly total and started reporting cost per transaction, per customer, per gigabyte served. The total tells you nothing: it goes up when the business grows, which is good, and it goes up when the system degrades, which is bad, and the number looks identical in both cases. Only a unit cost separates the two.

AI spend needs the same treatment, and the token is emphatically not that unit. The token is the vendor's meter, not the company's measure of anything. It has no relationship to any outcome the business recognizes. A million tokens spent resolving support tickets and a million tokens spent generating unread drafts appear identically on the invoice.

Useful units are the ones the business already counts. Cost per employee per month is the crudest one and still worth having, because it makes the population visible. Cost per resolved ticket is better, and it is available to any support organization that already tracks resolutions. Cost per merged pull request is the developer equivalent and it is more honest than counting agent hours. Cost per generated document, per processed invoice, per onboarded customer: the pattern is to divide the spend by whatever the department already puts in its own reporting. The moment a team can say "this workflow costs eleven cents per resolved ticket", every subsequent conversation becomes an ordinary business conversation. Is that too much, compared to what. Would a cheaper model raise it or lower it, once you count the reopened tickets. That is a discussion an operations manager can have. "We spent four hundred thousand tokens" is not.

A note on that currency, because it is a deliberate choice rather than an oversight. The model vendors are American, their price tables are denominated in dollars, and the invoice arrives in dollars regardless of where the company is incorporated. The dollar is therefore the native unit of this cost, and every unit figure in this series stays in it. Converting to a local currency for internal reporting looks like a courtesy to the finance team and is in fact a measurement error, because it folds a second variable into a number whose entire purpose is to isolate the first. A cost per resolved ticket expressed locally moves when consumption moves, which is the signal, and it also moves when the exchange rate moves, which is noise from the perspective of the team being asked to explain it. Two months with identical usage produce two different numbers, and nobody in the review meeting can tell which part of the variation they are supposed to act on.

This does not make the exchange rate irrelevant, it puts it in a different line of the analysis. For a company in the CFA zone the exposure is real and slightly counterintuitive: the franc is pegged to the euro at a fixed parity, so the local cost of a dollar denominated AI bill does not track any West African variable at all, it tracks the euro against the dollar. Consumption is governed by the gateway, currency exposure is governed by treasury, and collapsing the two into a single reported figure means neither one gets managed. Keep the unit cost in dollars where the meter is, convert once at the top for the consolidated accounts, and let the treasury conversation happen on its own terms. The third article in this series returns to the payment side of this, which in the UEMOA context is not only a question of rates but of card ceilings and settlement friction.

This is also where the third unknown, value, stops being unanswerable. Nobody is going to produce a defensible company wide ROI figure for artificial intelligence, and the attempts are mostly marketing. But a single workflow with a known unit cost and a known outcome count is measurable, and enough of those add up to something real.

Agents break the per seat budget

There is one more reason the fixed per employee allocation fails, and it is recent enough that many finance teams have not encountered it yet.

Conversational use is roughly predictable. A person reads, thinks, types, waits for an answer, reads it. Human reading speed puts a natural ceiling on consumption, and across a population it averages out into something you can plan around. The per seat model works acceptably for this population, which is why the vendors' seat based enterprise plans exist and are reasonable value.

Agentic use has no such ceiling. When a developer runs a coding agent over a repository, a single instruction expands into a long chain of calls: the agent reads files, reasons, calls a tool, receives output, reasons again, and each step resends an accumulating context. One human sentence can produce hundreds of model calls and replay the same context dozens of times. The consumption is bounded by the task and the loop, not by human attention. The same is true of any autonomous pipeline: document processing chains, evaluation harnesses, anything with retries.

The practical consequence is that a uniform allocation is not conservative, it is broken in both directions at once. Set it at a level that suits the sales team and the engineering team hits it on day three, so they route around it and you have lost the visibility you built the system for. Set it at a level that suits engineering and you have handed the entire company a budget it will never use, which is not a control at all. Budgets have to be per role, sized from observed consumption, which is another reason the Inform phase is not optional. You cannot size a limit for a population you have never measured.

Governing by giving

Here is the turn. Every control discussed so far fails for the same reason: there is no point of passage. The charter has no observer, the budget has no meter, the security team has no gate. Everything becomes possible the moment all AI traffic goes through one place the company operates, and nothing is possible until then.

That place is a gateway: a service that speaks the same API the providers speak, sits between the company's people and tools and the model vendors, holds the real provider credentials, and issues its own keys to internal consumers. Every call passes through it, which means every call can be attributed, limited, logged, filtered and priced. It is the proxy of the web filtering era, applied to inference.

The strategic point is not the architecture, it is the deal it lets you offer. A gateway can be deployed as an instrument of restriction, and if it is, it will be treated as one and worked around. It should be deployed as an instrument of access. The employee currently paying for a personal subscription out of their own pocket, or worse, using a free tier whose terms permit training on their input, gets something concrete in exchange for coming inside: a single key that reaches every model from every vendor, no personal card, no expense report, access to the expensive models the company has decided to pay for, and no ambiguity about whether they are allowed to use this for work. That is a better deal than the one they have. They take it, and they take it voluntarily.

The company gets, as a byproduct of that adoption, the attribution layer it could not obtain by writing a document. This is what "govern by giving" means concretely: the control is not paid for by the employee in friction, it is paid for by the company in provisioning, and the compliance falls out of the incentive rather than out of the rule. The second article in this series builds this gateway with LiteLLM, and shows what the budget, tagging and limit primitives actually look like in configuration.

What a gateway will never see

An honest account has to state the boundary, because a gateway solves less of this problem than its proponents imply.

A gateway sees API traffic. It does not see an employee opening a vendor's consumer web application in a browser tab and pasting a contract into it on a personal account. That is a large fraction of real world usage, quite possibly the majority in a non technical department, and no amount of gateway configuration reaches it. Closing that path is a different project: enterprise agreements with the vendors so there is a sanctioned web interface tied to corporate identity, single sign on so accounts are company accounts, network policy or endpoint controls for the consumer endpoints, and an offer good enough that the sanctioned path is the convenient one. The gateway is necessary and it is not sufficient.

It also does not see AI embedded inside other software. The assistant inside the document suite, the summarizer in the ticketing tool, the copilot in the CRM, the meeting notetaker: these send company data to models through their vendor's own infrastructure, on terms set in a SaaS contract nobody read with this in mind. That spend is invisible to the gateway by construction, and it is governed, if at all, through procurement and vendor review rather than through infrastructure. Browser extensions are the same category with worse terms.

None of this argues against the gateway. It argues against declaring victory when it is deployed. The realistic claim is that a gateway gives you full control of the traffic your engineering and your automated systems generate, a strong sanctioned path for everyone else, and an inventory that is far better than nothing. The claim it cannot support is total coverage.

An operating model that survives contact

Assume the gateway exists. The design decisions that determine whether the system is still running in six months are organizational, not technical, and they are worth stating in advance.

Make the team the accounting unit and the person the identity unit. Budgets belong to teams, because teams have a manager who owns a cost center and can arbitrate. Keys belong to individuals, because attribution to a shared key is not attribution. This mirrors how cloud accounts and IAM identities already work, and it means the escalation path for "we need more" is the one the company already uses for every other resource.

Separate the soft limit from the hard limit, and put real distance between them. The soft limit exists to start a conversation before anything breaks, so it fires to the team lead, not to the user. The hard limit exists to make the catastrophic case impossible: the misconfigured loop, the agent that retries forever, the credential that leaked. Do not set the hard limit at a level where ordinary good work reaches it, because a limit that regularly blocks legitimate work teaches people to route around the system, and once they route around it you have lost the data, which was the point.

Bound the day as well as the month. A monthly ceiling alone permits the entire month to be consumed on a Tuesday morning by one runaway process, which is the exact failure the ceiling was supposed to prevent. A daily bound stacked underneath the monthly one contains the blast radius without constraining normal usage, and this is a primitive the gateway can provide directly.

Tier the models. Route the default traffic to a cheap, fast model that handles the large majority of requests acceptably, and make the expensive frontier models available to the roles and workflows that demonstrably need them. Do not make this a request form. Make it a group membership that a manager can grant, otherwise the friction reappears exactly where you removed it.

Publish an exception path and honor it quickly. Every control system in a company is judged by what happens when someone legitimately needs to exceed it. If the answer is a ticket that takes a week, the control has taught the organization to bypass it. If the answer is a manager approval that takes an hour, the control survives.

Where to start

The sequencing matters more than any individual decision, and it is the inverse of the intuition.

For the first month, measure and block nothing. Stand up the gateway, provision keys generously, set limits high enough that nobody encounters them, and let the traffic arrive. Resist every request to enforce something during this period. The output of the month is a picture of actual consumption by team, by model and by use case, which is the thing the company has never had, and which every subsequent decision depends on.

For the second month, publish showback and do not attach consequences to it. Each team lead receives their number and their breakdown. Expect the first round of optimization to happen without any instruction, because a visible number changes behavior on its own, and expect to find at least one workflow whose cost is absurd relative to what it produces.

For the third month, set the budgets, and set them from what you measured rather than from what you guessed. Now the per role sizing has evidence behind it, the soft limits sit above real usage, and the hard limits are set where a genuine anomaly lives. This is also when chargeback becomes possible, if the company wants it, and when the first unit costs can be published for the workflows that have a countable outcome.

The whole sequence is inform, then optimize, then operate, and the reason to insist on it is that the failure mode of starting at the end is not merely inefficiency. It is that the system gets bypassed in week two and never recovers the trust it needs to be useful.

Limits and open questions

Everything above rests on one assumption worth making explicit: that measured cost is a reasonable proxy for the thing the company actually cares about. It is a proxy, not the thing. A gateway's cost figures are computed from a price table maintained in software, and they will drift from the provider's invoice for ordinary reasons: cached prefixes billed at a discount, batch tiers, negotiated enterprise rates, changes the vendor makes on a Tuesday. Reconciliation against the real invoice is an operational task that has to be owned by someone, and any organization treating gateway numbers as accounting truth without that reconciliation is going to be surprised.

The value question remains genuinely open. Unit costs per workflow are a real advance over token totals, but a unit cost is only half of a return, and the other half requires measuring what the output was worth, which for most knowledge work is not measurable at the resolution anyone would like. Claims of company wide AI ROI should be read with that in mind, including the ones that are flattering.

And there is a question this article has deliberately not resolved: whether routing every prompt through a single company operated chokepoint, which by construction sees every question every employee asks, is a governance improvement or a new concentration of risk. It is both. The gateway becomes an extremely attractive target, its logs are a surveillance capability whether or not anyone intends to use them that way, and the decision about whether to log prompt content or only metadata is a genuine one with no default safe answer. The third article in this series takes that question up under a specific legal regime, where the answer stops being a matter of preference.

Sources

• FinOps Foundation, FinOps Framework: domains, capabilities and the inform, optimize and operate lifecycle. finops.org/framework
• FinOps Foundation, FinOps for AI and the extension of framework scopes beyond public cloud to SaaS and AI workloads. finops.org/introduction/what-is-finops
• LiteLLM documentation, Proxy: cost tracking, budgets and rate limits, for the gateway primitives referenced here and detailed in the next article. docs.litellm.ai
• Provider pricing documentation from OpenAI, Anthropic and Google for the per token billing model and the price spread between model tiers.