Picture three companies.

1. A manufacturer uses computer vision to catch defects near the beginning of a production run rather than at the end. The company sells the same physical object, but consumes less material and machine time for every acceptable unit.

2. A hospital uses AI to draft clinical notes. The care has not become an “AI product.” But clinicians spend less time documenting it, which changes the cost and capacity of delivering care.

3. A software company sells an agent that researches a question, calls external tools, checks its work, and produces an answer. The product can do something conventional software could not. But every successful answer now carries a variable cost.

These look like three different AI stories. Economically, they begin with the same two questions:

What got cheaper, and who keeps the difference?

In the previous essay, we separated two ideas commonly collapsed into the label “AI company”: AI in the product and AI in operations. That distinction tells us where AI enters a business. It does not yet tell us what happens to the economics once it gets there.

For that, the practical lens is cost structure.

Here, “product” means whatever outcome the customer pays for: software, a physical good, professional work, healthcare, transportation, or another service. AI does not need to be visible in that outcome to change its economics. The model may be the thing being sold, or it may quietly alter how much labor, material, time, computation, and capital the company requires.

This gives us three parts of the argument:

1. AI changes the cost of producing an outcome.

2. That changes what customers build, buy, and pay for.

3. Someone captures the difference.

Subscribe now

1. AI changes the cost of producing an outcome

Every business turns inputs into outcomes. The inputs differ by industry.

A software company relies largely on talent, computing infrastructure, data, and third-party services. A manufacturer adds raw materials, components, energy, equipment, and inventory. A hospital coordinates clinicians, beds, supplies, diagnostic capacity, and information. A logistics company combines labor, vehicles, fuel, warehouse capacity, and time.

AI can change how much of those inputs is required for each successful result.

Less waste before more output

The simplest gains come from avoiding waste.

A retailer that forecasts demand more accurately can carry less excess inventory. A manufacturer that detects a defect earlier can avoid spending additional material and machine time on a unit that will eventually be discarded. A logistics company that reduces empty miles gets more deliveries from the same fleet. A hospital that anticipates patient volume can align staffing before an expensive mismatch appears.

In none of these cases is the model the customer-facing product. It sits underneath the product and changes the amount of economic input consumed.

This is an important corrective to the idea that an “AI company” must sell an AI feature. A business can expose no model to its customers and still be transformed by one. Conversely, a company can add a conspicuous AI interface without materially changing its underlying economics.

The feature is visible. The cost curve is consequential.

The work changes shape

Software development is the most visible example. Code-generation tools reduce the time required to draft routine code, interpret unfamiliar systems, write tests, produce documentation, and explore implementation options.

That does not make software engineering free. It shifts the scarce input. Typing code becomes less important relative to defining the system, understanding the user, choosing an architecture, and evaluating whether the output is correct.

The same pattern appears in services.

In a law firm, AI can produce a first-pass document or search a body of case law. Attorney time moves toward judgment, negotiation, and accountability. In healthcare, AI can draft documentation while the clinician remains responsible for its accuracy and for the care itself. In insurance, a system can assemble the evidence for a routine claim while an experienced adjuster handles ambiguous cases.

In each example, AI makes an intermediate artifact abundant: code, text, analysis, or classification. The scarce resource moves elsewhere.

That distinction matters because abundant output can create its own costs. Faster code generation can produce more software to maintain. Faster document generation can create more material to review. A factory system optimized for throughput can push defects downstream if quality is poorly measured.

When generation becomes cheaper faster than evaluation improves, the result is not leverage. It is clutter or risk.

Measure the successful outcome, not the activity

This is why “we reduced headcount” is an incomplete account of AI’s value. So are prompt counts, generated lines of code, and numbers of AI licenses.

The useful denominator depends on the business:

• Cost per acceptable manufactured unit.

• Clinician time per patient treated.

• Cost per correctly resolved claim.

• Cost per successful delivery.

• Cost per validated product improvement.

• Cost per trustworthy answer.

The distinction can reverse an apparent conclusion. Suppose an AI legal product doubles its model expense but reduces professional review time by 80 percent. Its inference bill has worsened, but the total cost of producing a trustworthy work product has improved.

Or consider an agent that completes a task only after twenty attempts. The final result may look impressive. The economics may be terrible once we include model calls, retrieval, tool use, verification, and human intervention.

The relevant unit is not the token, prompt, document, or feature. It is the successful outcome.

The operating question remains: What got cheaper?

2. The customer’s alternative moves too

A lower internal cost is only half the story. AI also changes what the customer can do without the company.

Every customer has alternatives. They can buy a product, build an internal version, hire people to perform the work, outsource it, tolerate the problem, or stop doing the work entirely. When AI changes the cost of one option, it changes the value of the others.

Build versus buy gets recalculated

Software provides the clearest case. For years, vendors benefited from a straightforward calculation: building internally was slow, expensive, and risky, so buying a specialized product was usually rational.

AI lowers the cost of creating the first version. A customer can prototype a workflow, build a lightweight internal tool, or automate a narrow process faster than before. That puts pressure on products whose primary value was packaging a simple workflow behind an interface.

But “easier to build” is not the same as “easy to own.” Internal software still requires maintenance, security, integration, evaluation, support, and adaptation. AI may make the prototype cheap while leaving the long-term obligation expensive.

The same recalculation extends beyond software. A professional-services firm may automate work it previously outsourced. A manufacturer may bring some design or quality analysis in-house because a smaller team can now perform it. A health system may buy an AI-enabled workflow rather than add administrative staff, while continuing to send difficult exceptions to specialists.

Companies will continue to buy software and services. The threshold will simply move.

That creates an uncomfortable question for every vendor:

If the customer can reproduce the visible part, what remains uniquely ours?

The durable answer is unlikely to be “our AI.” Models will improve and diffuse. Stronger answers include trusted data, deep integration, reliability, security, regulatory capability, distribution, maintained domain knowledge, and accountability for the final outcome.

The interface is increasingly easy to imitate. A maintained system that works under real-world constraints is not.

Customer-facing AI keeps the meter running

AI products introduce a special cost problem.

Traditional software taught companies and investors to expect very low marginal costs. Once the code was written and the infrastructure was running, serving another customer was often inexpensive. The meter largely switched off after the product was built.

With customer-facing AI, the meter keeps running.

Every generated answer, analyzed document, autonomous action, or multimodal interaction consumes computation. A difficult task may require repeated reasoning, retrieval, tool calls, verification, and review. The product performs work each time the customer uses it, and that work has a cost.

This does not make the business unattractive. Runtime inference enables products that could not previously exist. It does mean the company must understand the total cost of making the product work reliably.

A compelling demonstration can conceal weak economics. An agent may appear magical while consuming more resources than the outcome is worth. On the other hand, an expensive model call can be entirely rational if it eliminates a much larger amount of labor or prevents a costly error.

Again, the question is not “How many tokens did we use?” It is “What did a successful result cost?”

Price must reflect the changed alternative

Traditional SaaS pricing is organized around seats. That works when software assists a person whose job remains the basic unit of production.

If the software begins doing the work, seat count becomes a poor proxy for value. A customer may need fewer users precisely because the product is effective. At the same time, greater use can raise the vendor’s inference cost.

Token pricing solves only part of the problem. Tokens represent a vendor’s input, not a customer’s outcome. A customer no more wants to buy tokens than a restaurant customer wants to pay according to the kitchen’s electricity consumption.

Different products will therefore require different models:

• Seat pricing when AI primarily assists existing workers.

• Usage pricing when each interaction has clear incremental value.

• Workflow pricing when the product completes a recognizable unit of work.

• Outcome pricing when the result can be measured and attributed.

• Hybrid pricing when a stable platform and variable consumption both matter.

Whatever the model, the price must be compared with the customer’s new alternative. A vendor cannot price forever against the labor cost of an old process if AI has made that process much cheaper.

The external question is: What can the customer now obtain more cheaply elsewhere?

3. Someone captures the difference

Suppose AI reduces the cost of producing a successful outcome from ten dollars to six. Four dollars of economic value has appeared. But that does not tell us who receives it.

The company might keep the difference as margin. It might lower its price to win market share. Employees might capture some of it through higher compensation or reduced workload. A model provider might absorb it through inference fees. Customers might demand it during procurement. A new competitor might use it to enter the market with a different business model.

Technology creates the possibility of surplus. Competition, pricing, and organizational design determine where the surplus goes.

Lower cost is not automatically higher profit

A company can make a process cheaper and still fail to benefit.

If every competitor has access to the same model and the customer can switch easily, much of the saving may flow into lower prices. If the company adds AI subscriptions without removing old systems or workflows, costs may rise. If faster production creates more review, compliance, or support work, the apparent gain may migrate to another line of the budget.

This is why local productivity can coexist with flat company performance. One employee completes a task faster, but the surrounding process remains unchanged. The saved time is absorbed by another queue, another meeting, or simply more low-value output.

Real transformation should appear in the operating model: shorter cycle times, lower support expense, less waste, faster onboarding, improved quality, greater capacity, or more revenue from the same resources.

If none of those move, AI adoption may be visible while its economic value remains missing.

New bottlenecks determine the advantage

When one input becomes cheaper, another often becomes more important.

When code becomes abundant, product judgment and system design become scarcer. When analysis becomes abundant, trusted data and evaluation matter more. When prototypes become easy, integration and distribution become harder differentiators. When autonomous systems act faster, governance and accountability become more valuable.

The winning company is not necessarily the one that reduces the most obvious cost. It may be the one that identifies the next bottleneck first.

Return to the manufacturer. Early defect detection saves material, but only if the system is trusted enough to influence production. Better scheduling raises throughput, but only if downstream logistics can absorb it. More output has little value if demand is unchanged.

Or return to the hospital. Drafting notes faster can return time to clinicians. It can also produce a larger volume of text that must be checked, stored, interpreted, and defended. The benefit depends on whether the full workflow improves, not whether one step accelerates.

A cost curve never moves in isolation.

The strategic question is: Who keeps the difference after the entire system adjusts?

A preview: green trees, dead trees, and blue sky

Once a cost curve moves, two further questions follow.

Which existing companies can reorganize around the new economics? And where do those economics create opportunities that did not previously exist?

We can think of incumbents as green trees or dead trees. Green trees can redirect resources toward new growth while preserving what customers value. Dead trees may still have revenue, brand, and scale, but their structures cannot adapt. They add AI to the surface while the economics underneath remain unchanged.

Then there is blue sky: opportunities that become viable when AI makes a previously impractical product, service, customer segment, or business model affordable. Blue sky is not simply “the AI market.” It appears wherever the boundary of what can be profitably produced or delivered has moved.

Cost structure tells us where to begin looking. It does not tell us which incumbents can move or which apparent openings are real. That is the next question.

Follow the cost curve

AI will continue to attract arguments about intelligence, labor, consciousness, and power. Those arguments matter. Business strategy requires a more disciplined frame.

Start with three questions:

1. What got cheaper?

2. How did the customer’s alternatives change?

3. Who keeps the difference?

These questions apply whether a company sells software, physical goods, professional services, healthcare, or transportation. They apply whether AI is visible in the product or buried inside operations.

AI does not create business value merely by making something technically possible. It creates business value when it changes the cost of producing a successful outcome and someone finds a way to capture the difference.

To understand what AI will do to a business, follow the cost curve.

On a pitch deck they look identical. On a P&L they behave very differently. Confuse them and you misread the business, valuing a category-creating product as if it were ordinary software, or overlooking a quiet operator that has rebuilt its cost base from the inside.

So here is the whole argument in one line: AI in the product reshapes what you sell and how you price it; AI in operations multiplies what your people can do. Those are two different axes, not two points on one line, and the most useful thing you can do with any company wearing the AI label is figure out where it lands on each.

Subscribe now

Two bets hiding under one name

When someone calls a business an AI company, they almost always mean one of two things, and they rarely say which.

The first is AI in the product. The customer is paying for model inference, and in return gets something software could never deliver before: a draft written, a problem reasoned through, an action taken. Every query runs a meter. Picture it that way. In classic software the meter switched off the day the code shipped, and the next customer was free. In an AI product the meter keeps running, because the product keeps doing real work for every user, every time. That one fact, that the product performs work on each use and that work has a cost, is what rewrites the economics, and it is also what makes these products worth paying for.

The second is AI in operations. The company points AI at its own work to get more done with fewer people. This never touches the price the customer sees. It lands in operating expense and headcount, which means it widens the gap between revenue and cost from the inside.

The two move in different directions on the P&L. AI in the product trades some gross margin for capability and growth, because serving each customer now carries a real cost. AI in operations lifts operating leverage, because the same revenue rides on a smaller payroll. A company can be strong on one axis and untouched by the other, which is exactly why a single “how much AI” score tells you nothing. You need two.

Axis 1: AI in the product

The SaaS baseline

For twenty years, software taught investors to expect gross margins of 75 to 85 percent (ICONIQ, State of Software 2025). The reason was simple. Once the code was written and the servers were running, the next customer cost almost nothing. Cost was fixed, and revenue piled on top.

The meter never switches off

The meter changes that, and it is worth being clear about what you get in return. Inference is what lets these products do things no fixed-cost software ever could, which is a large part of why they grow as fast as they do. The trade is a different margin shape, not a broken one. When inference is part of the product, cost rises with use, so the gross margin simply looks different from classic software.

The numbers are already in. ICONIQ’s State of AI: Bi-Annual Snapshot puts the average AI-product gross margin near 52 percent, up from 41 percent two years earlier and still climbing. Put plainly: an AI product keeps about half of every dollar it bills today, where classic software kept four-fifths, and that half is funding growth a SaaS company could only envy. Inference runs about 23 percent of revenue at scaling-stage AI companies. Bessemer’s State of AI 2025 shows the range: its fastest-growing “Supernovas” run near 25 percent margin on roughly $1.13M of revenue per employee, trading early margin for a land grab, while the steadier “Shooting Stars” hold near 60 percent on about $164K per employee.

When pricing lags the cost

This does ask for a new discipline, and Replit is the cleanest public lesson. After it shipped a popular new agent, its gross margin reportedly swung from 36 percent to negative 14 percent in just two months in 2025, according to The Information. The feature was a hit; the pricing simply had not caught up with the cost of serving it. That is the lesson, not a verdict on the model. It is cost of goods sold, the direct cost of delivering what you sell, showing up where SaaS never had any, and it rewards the companies that price for it from day one.

The cost curve is falling

There is good news in the trend. Bain Capital Ventures argues in “Gross Margin is a BS Metric” that today’s thinner margins are a passing artifact of expensive model APIs, and the data backs it. When Mixtral 8x7B arrived, the going rate for that class of model fell from about $2 to $0.24 per million tokens in a matter of days. Input costs are falling fast, even as usage climbs. The fair conclusion is that today’s margins are a snapshot, not a destiny. Which means, for a buyer, the question is not “what is the margin today,” it is “which way is it heading, and has the company priced for the answer.”

What is genuinely new

Not every AI feature earns the label. There is a real line between products that bolt a model onto a classic application, where margin drifts down toward 60 to 70 percent, and products that could not exist at all without runtime inference: open-ended generation, agents that take multi-step actions, plain-language interfaces over messy data. The second group is what people mean by “AI-native” (HBS, IBM), and it is where both the magic and the margin pressure run hottest.

Pricing has to follow the meter

Here is the trap that catches the most companies: if usage drives your cost but seats drive your price, you lose money precisely when customers love you most. Aaron Levie of Box put the scale of it bluntly at TechCrunch Disrupt 2025: there will soon be “100x, maybe 1,000x more agents than people,” and no per-seat model survives that. Sarah Tavel of Benchmark gives the fix a name in “Sell work, not software”: price against the labor you replace, not the seats you fill.

The market is already moving. ICONIQ reports outcome-based pricing leaping from 2 percent of companies in mid-2025 to 18 percent by January 2026, the fastest pricing shift in modern software, and Salesforce at feature launch charged about $2 every time an Agentforce agent did something, a business it says has reached $800M in revenue (Q4 FY26 release). Which means a high-inference product still selling by the seat is not leaving money on the table. It is funding its customers’ usage out of its own margin.

Axis 2: AI in operations

The second axis asks something different. Not “is AI in what you sell,” but “has AI changed how you run.” Two questions hide inside that one. Is AI making your existing people faster, or has it changed the shape of the company itself?

The fast version: people get more done

The cleanest evidence comes from Brynjolfsson, Li and Raymond, who studied real customer-support agents and found a 14 percent jump in output on average, and 34 percent for the newest workers (Generative AI at Work). Operators feel the same pull. Shopify’s Farhan Thawar ordered 1,500 Cursor licenses, then immediately needed 1,500 more, and the fastest-growing users were not engineers but the support and revenue teams (First Round). Shopify’s CEO turned it into law: in his April 2025 memo, Tobi Lütke told the company that reflexive AI use is now the baseline, and that no team gets new headcount until it can prove AI cannot do the job. Which means, on this rung, the win shows up as the same work done by fewer hands.

The deep version: the company changes shape

The profound version is not faster typing. It is a different org chart, with fewer managers, wider spans of control, and far more revenue riding on each person.

Watch Klarna do it in public. Its F-1 filing shows headcount falling from 5,527 at the end of 2022 to 3,422 at the end of 2024, roughly two thousand jobs, credited directly to AI, with management saying the number will keep dropping. Over the same stretch, CEO Sebastian Siemiatkowski says revenue per employee climbed from about $300K to $1.3M (Charter, February 2026). Each remaining person now carries more than four times the revenue they did three years earlier. That is not a productivity tweak. That is a new operating model.

The maturity frameworks grade this axis

This is where the maturity frameworks earn their keep, because they grade exactly this axis. MIT CISR sorts companies into four stages and finds the bottom two lag their industry financially while the top two beat it, so depth here is not vanity, it shows up in the numbers. Gartner offers a five-level version of the same climb. And the macro picture is humbling: McKinsey finds roughly 88 percent of companies have adopted AI but only about 38 percent have scaled it past pilots, while Deloitte finds just 34 percent are genuinely rebuilding the business rather than chasing a few productivity wins. Which means most companies that call themselves AI-driven are sitting near the bottom of the ladder.

The skeptic deserves a chair

We are not here to sell a fairy tale to a room full of PE partners. Daron Acemoglu’s “The Simple Macroeconomics of AI” projects total productivity gains of at most 0.66 percent over a decade. A 2026 NBER working paper from Yotzov, Barrero, Bloom and colleagues found that about nine in ten firms report no measurable change in employment or productivity at all, which fits perfectly with most companies still stuck on the bottom rungs. And Klarna itself walked part of it back in 2025, bringing humans back for its highest-value customers when pure automation went too far. Cutting too deep carries its own cost.

Quadrant breakdown

Why it all comes down to cost structure

Strip away the noise and every one of these stories is the same question wearing different clothes: what cost structure do you actually have, and did you price for it? A high-product company still selling by the seat is carrying inference as cost of goods sold and charging as if it had none. A lean operator with an ordinary product can defend rich margins for years. The phrase “AI company” tells you almost nothing about which of those you are looking at. Its place on these two axes tells you almost everything.

So before you ask whether a company uses AI, ask the two questions that actually move a valuation. Does AI sit in the product, creating value and reshaping how it is priced? Does AI sit in operations, multiplying what each person can do? The label is marketing. The position is the business.

We’ll cover that impact in the next post.

Key references

Industry reports & benchmarks

• ICONIQ Growth, State of AI: Bi-Annual Snapshot (Jan 2026) — https://www.iconiq.com/growth/reports/2026-state-of-ai-bi-annual-snapshot

• ICONIQ Growth, State of Software 2025 — https://www.iconiq.com/growth/reports/2025-state-of-software

• Bessemer Venture Partners, The State of AI 2025 — https://www.bvp.com/atlas/the-state-of-ai-2025

• McKinsey, The State of AI 2025 — https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai

• Deloitte, State of AI in the Enterprise 2026 — https://www.deloitte.com/us/en/what-we-do/capabilities/applied-artificial-intelligence/content/state-of-ai-in-the-enterprise.html

• MIT CISR, Enterprise AI Maturity Model (Weill, Woerner, Sebastian) — https://cisr.mit.edu/publication/2025_0801_EnterpriseAIMaturityUpdate_WoernerSebastianWeillKaganer

• Gartner, AI Maturity Model & Roadmap Toolkit — https://www.gartner.com/en/chief-information-officer/research/ai-maturity-model-toolkit

Operator & investor commentary

• Sarah Tavel (Benchmark), “AI startups: Sell work, not software” —

Sarah Tavel's Newsletter

AI startups: Sell work, not software

For the past 25 years, application software startups have had a singular focus: increasing company and employee (including developer) productivity. This looked like building software that increased productivity at the employee level, increased collaboration across employees and teams, and/or enabled better oversight and management at the leadership leve…

Read more

3 years ago · 272 likes · 27 comments · Sarah Tavel

• Bain Capital Ventures, “Gross Margin is a BS Metric” — https://baincapitalventures.com/insight/gross-margin-is-a-bs-metric/

• Aaron Levie (Box), TechCrunch Disrupt 2025 — https://techcrunch.com/2025/10/29/box-ceo-aaron-levie-on-how-ai-is-changing-the-enterprise-saas-landscape/

• Tobi Lütke (Shopify), “Reflexive AI usage” memo (Apr 7, 2025) —

• Farhan Thawar (Shopify) via First Round, “From Memo to Movement” (Jul 2025) — https://www.firstround.com/ai/shopify

• Sebastian Siemiatkowski (Klarna), Charter Leading with AI Summit (Feb 10, 2026) — https://www.charterworks.com/what-klarna-learned-from-its-ambitious-ai-rollout/

• Harvard Business School Online, “How to Architect an AI-Native Business” — https://online.hbs.edu/blog/post/ai-native

• IBM, “What is AI Native?” — https://www.ibm.com/think/topics/ai-native

Public filings & disclosures

• Klarna Group plc, Form F-1 (SEC, Mar 2025) — https://www.sec.gov/Archives/edgar/data/2003292/000162828025012824/klarnagroupplcf-1.htm

• Salesforce, Q4 FY26 earnings release (Feb 25, 2026) — https://www.salesforce.com/news/press-releases/2026/02/25/fy26-q4-earnings/

• The Information, “Replit’s Margins Illustrate the High Costs of Coding Agents” (2025) — https://www.theinformation.com/articles/replits-margins-illustrate-high-costs-coding-agents

Academic papers

• Brynjolfsson, Li & Raymond, “Generative AI at Work” (NBER 31161; QJE 2025) — https://www.nber.org/papers/w31161

• Acemoglu, “The Simple Macroeconomics of AI” (NBER 32487, May 2024) — https://www.nber.org/papers/w32487

• Yotzov, Barrero, Bloom et al., “Firm Data on AI” (NBER WP 34836, Feb 2026) — https://www.nber.org/papers/w34836

What we set out to do

We started MTN with a thesis any clinician would recognize. The bedside is full of decisions that depend on patterns no single person can hold in working memory. Vitals, waveforms, labs in motion, and health records encode much of that pattern. Modern models can learn from them when the data is treated with care.

So we built clinical AI on top of that data. Real models, trained on real cohorts. We cleared the institutional review boards at two major health systems. Cleared their IT and security reviews after that. Each step took longer than the science had, and each step taught us more about the system. We were standing at the door, approvals in hand, ready to run.

We expected the constraint to be model quality. It was not.

What the work taught us

The lesson came from the part of the job that did not look like medicine.

Most of the engineering effort behind any clinical AI deployment is not the model. It is the data underneath. Every site looks different. Every integration breaks again when an upstream system changes a field, a code set, or a vendor. EHR upgrades shift what a clinical observation looks like at the storage layer. New billing rules redefine how the same encounter is coded. Long before a model could run live, the integration questions had already decided whether it ever would. The schema work, the billing-code reconciliation, the reimbursement pathway, the data-access layer, all of it sat upstream of the deployment we never had to actually run to understand.

By the end of a typical month, we found ourselves spending far less time designing models than navigating billing codes and the surrounding apparatus of reimbursement and operations. The work that determined whether a deployment lived or died was rarely the model itself. It was the reconciliation of the data deciding whether the model’s recommendations could ever be paid for. The science was not the bottleneck. The data layer was, and the data layer decided what could be built and what could be paid for.

That experience repeated often enough to stop being an anecdote. It was the structural shape of the work.

The deeper realization

If the data layer decides what can be deployed, then the question is what is wrong with the data layer. The answer is older than any of our models, and it is hiding in plain sight on every health system’s income statement.

Roughly a third of every healthcare dollar in the United States is spent on administration rather than care.[1] That is the binding constraint on American medicine. Even the best clinical AI lives downstream of a financial system that cannot afford to absorb new costs without offsetting them elsewhere. A model whose validation looks strong on retrospective data but adds friction to billing, prior authorization, or reconciliation will not survive the budget conversation that follows the approvals. It will be politely retired, framed as a learning, and quietly replaced by whatever protects the operating margin.

The problem is not primarily clinical. It is financial. And the financial problem is not primarily about insurance design or pricing or any of the other places the policy conversation usually settles. It is about the cost of running the system itself: the cost of making the data agree with itself across the dozens of platforms a single patient interaction touches.

Why the two problems are the same problem

The administrative tax is not a separate disease from the deployment problem. It is the same disease in a different costume.

Look at where the money actually goes. Denial rework. Reconciliation across payers and providers. Parallel platforms that grew through acquisition, and the perpetual schema mapping between them. Back-office teams whose job is to translate one system’s representation of an event into another system’s representation of the same event.

These are the same underlying condition: data that does not agree with itself, in volumes large enough that humans become the integration layer. Fragmented data is what makes administration expensive. Fragmented data is also what makes clinical AI hard to deploy. One disease, two symptoms.

Once we saw it that way, the next move was structural rather than strategic. The technology we had built to make clinical data usable was already pointed at the harder and larger version of the same problem. Per-source semantic annotation. Canonical concepts emerging across many annotated schemas. Integration or migration without requiring raw-data movement. We had been solving the financial problem all along, in the narrow shape of a clinical deployment.

The shift

So we took the engine and pointed it at the financial layer. We call it MTN Data Foundry.

• It makes fragmented data operable by adding a semantic description to each source.

• Each source is annotated at the schema level on its own terms, using evidence from the schema itself rather than a canonical mapping imposed from above.

• Shared semantic concepts emerge across many annotated schemas, either through cross-source alignment or by mapping to a target schema when one is specified.

• Integration and migration follow from those mappings without requiring a raw-data central store.

The clinical work taught us the shape of the problem. Data Foundry is the shape of the answer.

Adjacent verticals as proving grounds

The same fragmented-data pattern shows up wherever companies grow by acquiring others, regardless of industry. The seams left behind by an acquisition rarely disappear; they get papered over by integration projects that ship just enough harmonization to close the books and then never quite finish. Years later, an operator inherits an environment where the same customer or the same transaction or the same asset is represented in three subtly incompatible ways, and the cost of disagreement compounds quietly until a migration, an audit, or a new system of record forces it into the open.

Working with adjacent verticals to healthcare is the fastest way to harden the product. Cycles are shorter outside healthcare. Migrations close in months instead of years. Procurement is faster. Regulatory exposure is narrower. Each engagement sharpens the semantic-annotation primitives and the canonical-concept extraction that the longer healthcare engagements will eventually depend on. The matured version comes back into the healthcare deployments with more reps behind it and fewer assumptions baked into it.

What this opens up

When administrative drag eases:

• The marginal dollar moves toward care.

• Clinicians spend their day with patients rather than with documentation.

• Plans approve in minutes rather than weeks. Audits become a reading exercise rather than a forensic one.

• New therapies reach the people who need them because the system around them can afford to deploy them.

• The deployments themselves stop being heroic projects that require a special team and start being routine ones that any operator can run.

None of that requires a new clinical breakthrough. It requires the system underneath the breakthroughs to compute. The clinical AI we set out to build still gets shipped. It gets shipped on top of a system that can pay for it, in a country whose health spending finally tracks its health outcomes.

An invitation

This is bigger than one company. The operators, consultants, plans, platforms, and investors working on any part of this fabric are the ones who decide whether the next decade looks like the last. We followed our own work upstream until we found the prerequisite, and the prerequisite turned out to be more interesting than what we set out to build. If you are working on any part of this problem, in healthcare or in an industry that looks like it, we would like to be in the conversation with you.

It promises to predict readmissions, flag silent hypoxia, summarize patient notes, or help clinicians triage faster. But how do you know whether the tool was trained fairly? That it performs equitably across racial groups or age brackets? That it won’t quietly drift into irrelevance after a few model updates?

The truth is: most users don’t get a clear answer. Even the vendors building these models often don’t have the internal capacity to apply full ethics frameworks. Worse, the few that do exist often require specialized staff, dedicated committees, or regulatory teams most startups simply don’t have.

That’s the gap the SAFE-AI Framework was built to close. We at MTN are very excited to have played a part.

TL;DR: What Hospitals Should Know About SAFE-AI

SAFE-AI stands for Scalable Agile Framework for Execution in AI. It was designed through a collaborative effort between MTN, the Data Science Alliance, the University of Utah, and Nemsee LLC. The goal is highly practical: embed lightweight but testable ethical oversight into the way small vendors and product teams already work.

How this helps

• Built to fit real Agile/Scrum workflows (no ethics bureaucracy required)

• Designed for resource-limited teams who require hospital-grade in transparency and fairness

• Aligns with emerging federal rules (HHS §1557, FDA SaMD)

• Makes it easy for hospitals to ask vendors the right questions—and get clear, auditable answers

A preprint of the paper is available through this link: http://arxiv.org/abs/2507.01304

The Problem: AI Governance Can’t Be an Afterthought

Hospitals are already integrating AI-powered tools into core operations. But most of these tools are:

• Developed by small-to-mid-sized teams

• Trained on siloed or non-representative datasets

• Tuned and re-tuned rapidly in live environments

These facts create risk. Not just regulatory or reputational risk, but real clinical and operational risks: alarm fatigue, performance drift, and unequal outcomes for vulnerable patients.

SAFE-AI offers a way to make sure every model change, deployment, and update includes an ethics checkpoint—without slowing product delivery.

How SAFE-AI Works: Phases That Repeat

SAFE-AI is not a checklist, it’s a repeating lifecycle that maps directly to Agile build cycles.

Figure 1. SAFE-AI Summary Workflow - highlights each core phase and feedback loop, from prioritization to deployment and monitoring.

Discovery

Select and prioritize among potential projects and their degree of alignment with organizational interests. Identify compliance and regulatory issues, affected stakeholders, and match the level of ethical scrutiny to the real-world impact.

Assessment

Define Acceptance Criteria (e.g. sensitivity, latency), Fairness Metrics (subgroup performance), and Transparency Metrics using the SPAMM approach (more on that below).

Development & Integration

Build, tune, test tolerance metrics. Most importantly, embed those metrics directly into product backlogs. Everyone from data scientists to QA owns ethics.

Monitoring & Evaluation

Set re-entry rules: every model retrain, data update, or environment change triggers another lightweight ethics cycle. Think of it as “post-market surveillance” for your AI signals.

→ For tech teams: Implementation is modular and audit-friendly.

Figure 2. Detailed Ethical Evaluation Process SAFE-AI - conducted continuously throughout product development. The process begins with a discussion of priorities and alignment and proceeds through model building and inference implementation, emphasizing appropriate tolerance levels at each iteration.

Subscribe now

Spotlight: The Metrics That Matter

SAFE-AI helps vendors (and hospitals) move beyond vague principles by using three metric categories:

🟢 Acceptance Criteria: Stakeholder-defined performance thresholds—like “AUC ≥ 0.85 on under-40 patients.”

🟡 Fairness Metrics: Gap analyses across subgroups, like “False-negative rate for Black patients should not exceed overall FNR by >3%.”

🔵 Transparency Metrics: Narrative scenario testing using SPAMM (Scenario-Based Probability Analogy Mapping). Instead of abstract confidence scores, you get human-readable summaries such as:

“In 70% of patients with these vital trends, respiratory failure occurred within 6 hours. In the other 30%, the model missed early signs, mostly in those with comorbidities.”

Why SPAMM Helps Build Trust

One of the paper’s key innovations is the SPAMM technique, which makes it easier for clinicians, executives, and even patients to understand what an AI model is doing, and where it might fail.

This matters because tools like SHAP or LIME (popular explainability techniques) don’t always translate meaningfully to frontline decision-makers. SAFE-AI emphasizes narrative transparency, telling the story of how the model behaves across contexts, including error modes.

Hospitals can demand this kind of scenario framing as part of every AI procurement or evaluation process.

Embedding Ethics into Agile, Not Around It

Many ethics approaches assume a waterfall-style dev cycle and dedicated review boards. That’s fine for pharma, but it doesn’t match the pace or constraints of AI product teams.

SAFE-AI maps ethics tasks onto existing tools like Jira, GitHub Projects, and sprint planning boards. That means:

• No separate documentation silos

• No ethics teams working in isolation

• No extra quarterly committee meetings

Instead, you get “responsibility metrics” that your vendors can track just like latency or uptime—so you can track them too.

Continuous Oversight = Safer AI Over Time

Every time a model is retrained—even if the inputs/outputs don’t change—the SAFE-AI framework requires a re-check of fairness and transparency metrics. That ensures drift doesn’t creep in silently.

This also future-proofs your health system. As AI regulations evolve (including under HHS, ONC, and the AI Bill of Rights), you’ll already have an audit trail showing due diligence and governance maturity.

What This Means for You

If you operate in a regulated environment and are considering AI-powered tools or already deploying them at the edge, in the EHR, or in predictive analytics stacks, SAFE-AI provides a simple question:

“Do you have an Agile-compatible ethics process, with defined fairness and transparency metrics, and do you revisit them after every model update?”

If the answer is “no,” SAFE-AI can make it “yes.”

How Does It Work In Action?

We’re currently piloting SAFE-AI with several systems and partners. More on that to come!

Where do I find the paper?

Preprint is available at: http://arxiv.org/abs/2507.01304

Subscribe now

I am Pranav Koirala, an emergency medicine doctor based in Maryland and the co-founder of MTN. My journey in medicine began in Nepal, where I completed medical school and worked as an expedition physician in remote, high-altitude environments. Those experiences not only solidified my passion for emergency medicine but also exposed me to the challenges of providing care in austere settings with limited resources, where access to timely interventions can mean the difference between life and death. Motivated by this, I came to the U.S. to pursue advanced training in emergency medicine, seeking to expand my knowledge and skills in managing critically ill patients.

Now, working in a busy emergency department, I see firsthand the gaps in the healthcare system—gaps that often lead to preventable adverse outcomes. Patients like Ms. Davis fill my workdays, and frontline healthcare workers everywhere encounter similar situations daily. Hours later, her workup results and medical history finally trickled in. She had diabetes and because of her mild dementia, she frequently forgot to take her medications, likely precipitating a life threatening condition called diabetic ketoacidosis, which almost took her life. An intelligent monitoring system with real time alert would have cautioned not only her rehab team but also her ED providers early enough as she started deteriorating leading to better prioritization and prevention of her near death experience.

I co-founded MTN to help find solutions for high-altitude illness, but as our research evolved, we recognized its potential for a much broader range of applications. The ability to continuously monitor vital signs—now feasible with modern smart wearables— and combine that data with health records can transform patient care by enabling early detection of deterioration and preventing avoidable emergencies.

What excites me even more is the potential of this technology not only in advanced healthcare systems but also in resource-poor settings, where I have worked extensively in the past and where even basic healthcare remains a luxury. Millions of people in remote villages, underserved communities, and low-resource hospitals struggle even in modern times with limited access to doctors, diagnostic tools, and real-time monitoring.

The possibility of bringing cutting-edge solutions to both high-tech hospitals and under-resourced clinics drives my passion for advancing healthcare innovation. At MTN, we are working to bridge these gaps, using machine intelligence to empower healthcare workers and provide patients with the care they deserve—whether in the crowded halls of an urban emergency department or in the most remote corners of the world.

The Problem

But the problem isn’t us. It likely isn’t them either. The problem is that the colloquial definition of “AI” really means two things:

1. The larger set of diverse methods used in the field over the last seventy years.

2. The latest and greatest paradigm – currently large language models.

Given the rising talk of AI’s impact on mental health care, it is important to know what people mean when they use that term. To appreciate the definition, we need to dive into a brief, gap-riddled and over-simplified history of what “AI” has meant over the last 70 years.

The Prevailing Definition of “AI” Has Changed Over Time

The Dartmouth Conference (1956)

Though one theoretically could start with Victorian era fiction (1870s), and it is most appropriate to start with Alan Turing’s Turing machine (circa 1936) or famous Turing test (circa 1950), it is most fun to start with the Dartmouth AI conference in the summer of 1956. The Dartmouth Conference is pointed to as the first official academic gathering of mathematicians and engineers centered on creating artificial intelligence.

Of course, my scientific forbearers didn’t anticipate that it would take nearly seventy years to advance on their goals. In their planning materials for the conference, the organizers famously wrote:

An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves. We think that a significant advance can be made in one or more of these problems if a carefully selected group of scientists work on it together for a summer.

If only.

Symbolic AI (1950s-1960s)

The earliest researchers aimed to create AI using chains of symbolic relationships. In essence their “AI” was built through increasingly detailed dictionaries of logical chains. For example, if a thing is sixteen inches long (give or take), has four legs, lots of fur, makes some people sneeze and displays liquid-like qualities, it is a cat. Alas, if a ferret enters the room, this simple set of rules will classify it as a cat and certain readers will be deeply upset.

Researchers realized this limitation, stopped calling it AI and looked for a new approach.

Expert Systems (1970s-1980s)

A clear problem is that chains of “if-then” statements are too simple. Experts reason through multiple facts, rules and assign probabilities to various possibilities. Thus, in the 1970s the field shifted its definition of “AI” to expert systems.

Here’s how an expert system would approach the ever-difficult cat problem:

Facts:

• Cats typically have four legs, a long tail, and fur.

• Cats can display fluid-like flexibility in their movements.

• Cats are known to cause allergic reactions in some people.

Rules:

• Rule 1: IF an animal has four legs AND a long tail, THEN it might be a cat.

• Rule 2: IF an animal moves with liquid-like flexibility, THEN it might be a cat.

• Rule 3: IF being near the animal causes sneezing or allergic symptoms, THEN it might be a cat.

• Rule 4: IF multiple features (from Rules 1-3) are satisfied, THEN classify the animal as a cat.

The AI Winter (1980s)

But if a Sphynx, Peterbald, or Donskoy shows up at the litter box, the system breaks. One can never add enough facts to cover every single possible type of cat.

And so, in the early 80s, the first AI winter set in. People stopped using the term “AI.” They grew skeptical that intelligent machines were even possible.

However, it is important to note that symbolic or rule-based methods did not go away or fail in their utility. In fact, one could consider diagnostic manuals such as psychiatry’s DSM or most clinical decision support systems as highly useful types of these approaches. People just started seeing them as tools rather than machine brains.

Early Neural Networks (1980s)

Green buds poked through the snow during the 1980s when computer science started emulating what was known about the human brain. By then, it became common knowledge that neurons accept multiple inputs, weigh them, combine them and then produce an output. One neuron’s output becomes another neuron’s input, and the “neural network” grows.

To define a cat, one would start with defined input features, such as body length, leg quantity, fur length, likelihood of eliciting a sneeze, and maximum contortion angle. Each would be assigned a numeric value. Those numbers would feed into artificial neurons, that eventually led to a neuron declaring “cat” or “not cat.”

Return of the AI Winter (1980s-1990s)

But the springtime proved fleeting. Researchers in the field over-promised and under-delivered. Teaching the neural networks was an impossible task and so a deep freeze set in again. Talk of “AI” brought laughter.

Probabilistic Models and Decision Trees (late 1990s-2000s)

However, a cautious groundhog left its den in the era of “big data.” The label of “AI” was still largely taboo, but through training statistical models or simple decision rules on large amounts of data (including 1960’s style symbolic representations), massive progress was made on the usability of systems. When you typed “cat” into Google circa 2003, it didn’t do such a bad job creating a list of web pages full of cats.

Deep Learning (2010s)

Then, in 2012, neural networks triumphantly returned. The breakthrough was not a new theory (methods largely were figured out in the 1980s), but new hardware. Computers were finally fast enough to train the networks and allow them to become “deep” with many layers of artificial neurons. Thus, one could show a deep neural network every cat picture on the internet. The network could learn from raw pixels, not even needing to be told features like body length or sneeze danger. Mind blowing really.

Yet, even as these networks beat world champion chess and Go players, people were noticing they struggled on tasks outside the data they were trained on. Just because a computer could win at StarCraft, doesn’t mean it could divvy up your monthly budget. The term “AI” fell by the wayside and people instead talked about “machine learning.”

Foundation Models (2020s)

However, in 2022 chatGPT was released. These “large language models” (LLM) not only generalize outside of their specific training data, but can also generate human-like responses. The generalizability allows them to form the “foundation” for a wide-variety of tasks. Breakthroughs were largely the result of algorithms that use hardware (i.e., GPUs) more efficiently, and thus can train on the entire Internet. Talk of AI returned unabashed and (in my opinion) largely justified.

It is important to note, these generative AI models are still considered a form of machine intelligence (i.e., built with machine learning), but their capabilities are far beyond earlier models. Not only can these models identify a cat, but they can write a poem about one tap dancing, then create a picture of the feline jig.

ChatGPT’s feline jig: poem and image

On tiny paws with taps so bright,
The cat danced boldly through the night.
Click-clack steps on the wooden floor,
Each move a rhythm, a tale, and more.
A feline star, we all adore.

Final Thoughts

So now the next time someone says “AI,” follow up by asking them what type they mean. Are they referring to an LLM talking someone out of self-harm, or DSM-IV style expert system guiding structured diagnoses?

Then again, will we still be talking seriously about “artificial intelligence” in three years? If performance peaks, probably not. People who recognize the limitations of LLMs have already shifted the discussion to 'artificial general intelligence,' or 'AGI. The goal of AGI systems is to not only tell you about cats, but figure out when to order cat food, or arrange for a house cleaning when your allergy-prone relative is scheduled to visit.

Of course, that’s also what those researchers at Dartmouth thought they could make significant progress on during that 1956 summer. What happens next even ChatGPT can’t guess.