Guide · Operations After Go-Live

Managed AI Operations: What Really Happens After Go-Live

Go-live is the beginning, not the outcome. The first twelve months of operation determine whether an AI solution becomes part of daily business — or a quiet drag on it. What happens during this period is concrete, predictable, and not particularly dramatic — if it's planned for deliberately.

This guide describes what actually happens in ongoing AI operations, what companies rarely assess realistically beforehand, and where the most common pitfalls lie in the first twelve months. Drawn from real DACH business practice, not from a whitepaper.

1. What actually happens in the first twelve months

A realistic picture of how operations unfold, broken into three phases. The figures are ranges drawn from DACH SME projects, not guarantees — they vary with use case, volume, and the maturity of the surrounding environment.

Phase 1 (months 1–3): calibration phase

Highest upkeep effort. Escalation thresholds get recalibrated in practice. Edge cases show up that weren't in the pilot sample. Users learn how to work with the system and find ways around it — which delivers important clues about gaps. Typical: 0.3–0.5 person-days per week.

Phase 2 (months 4–9): stabilization phase

Lowest upkeep effort. The system runs within its defined range, KPIs have settled in. What happens here is mostly quiet observation plus occasional configuration tweaks. Typical: 0.1–0.3 person-days per week. Caution: this is the phase where ownership erodes most often — the sponsor steps back because "it's running fine."

Phase 3 (months 10–12): first major adjustments

Drift becomes measurable. Business processes have shifted (new product lines, new language variants, new suppliers). Model providers have rolled out at least one major version. A planned optimization iteration comes due. Typical: 0.2–0.4 person-days per week plus one iteration package.

2. What companies most often underestimate

Four recurring patterns from business practice. None of them is exotic — all four show up reliably if they haven't been thought through in advance.

Speed of drift

"Drift practically doesn't happen in the first few weeks" — wrong. Drift starts on day one, but stays below the perception threshold for a long time. The problem only becomes visible once the escalation rate rises over several weeks — by which point it has already taken hold. Early measurement beats late reaction.

Dependence on individuals

After go-live, the implementation team disappears and the knowledge stays informal. If the person who built the pilot goes on vacation or leaves the company, a gap opens up. Documenting the configuration, the eval sets, and the escalation rules isn't optional — it's the insurance policy.

Interface maintenance as a hidden cost

The ERP gets a quarterly upgrade, the CRM API changes a version, the DMS gets a new field. If nobody tracks these updates and tests them in advance, you find out the hard way — through an outage. Interface maintenance isn't AI maintenance — but in an AI context it often costs more effort than the AI itself.

Escalation share as an emotional factor

Even at a 95 percent automation rate: the five percent that escalate are the cases everyone sees and everyone talks about. If the operations team doesn't know this is normal, and doesn't know the range it should stay within, the feeling sets in that the system "doesn't work." Expectation management is part of running operations.

3. Where systems actually tend to break

Four concrete failure patterns that show up reliably in production DACH business projects. Expect them and you can manage them. Don't expect them and you'll experience them as escalations.

Data drift

The input mix shifts: new language variants, new suppliers, new product lines, new request types. Model quality declines gradually. Countermeasure: ongoing monitoring of the input data distribution plus sample evaluation of outputs.

A model update breaks something

The provider rolls out a new version, an existing prompt loses precision, a classification shifts. Countermeasure: an eval set built from real examples, run before every update, with a documented acceptance threshold. A rollback path to the previous version defined in advance.

Business process drift

The business unit changes an approval rule, a new regulation comes into force, a new case type needs to be covered — and the workflow no longer fits. Countermeasure: a quarterly alignment round with the business unit, documented change requests, controlled rollout.

Interface breakage

ERP, CRM, or DMS gets a major update, an API version is deprecated. If nobody tracks this, the integration fails. Countermeasure: a documented version status, a roadmap link to the vendor's update plan, staging tests before every production update. More in the ERP integration guide.

4. What an honest operations report must include

The monthly operations report is the tool that lets management, IT, and the business unit see the same set of facts. It can't be replaced by a dashboard — dashboards show state, the report shows development and decisions. Minimum contents:

KPI status with comparison: turnaround time, escalation rate, error rate, volume, and where relevant token costs — all measured against the prior month and against the status at go-live.

Incidents: every anomaly with cause, resolution, and date. Minor ones too — completeness beats selectivity.

Changes: every model or configuration change made, with date, rationale, and effect.

Vendor status: which model versions, which API versions, and which hosting region are actively in production use.

Optimization backlog: prioritized proposals with effort and benefit estimates. Not a wish list — a basis for decisions.

Recommendation: a concrete statement for the coming month — observe, adjust, or iterate. No marketing language, no buzzwords.

A good operations report runs three to five pages. If it regularly exceeds fifteen pages, nobody reads it anymore — which makes it useless.

When managed AI operations makes sense — and when it doesn't

Makes sense when

• The AI solution is embedded in day-to-day operations
• There's no dedicated AI operations role internally
• Processed cases affect customers or cash flow directly
• Volume is above roughly 100 cases per day
• Multiple AI applications run in production in parallel

Doesn't make sense when

• It's a pure experiment with no production business load
• There's an established internal MLOps team with capacity
• Volume is low enough that a quarterly spot-check is sufficient
• The use case is dead — escalating to "discontinue" makes more sense
• The pilot is still in pilot status, not in regular operation

Frequently asked questions about ongoing AI operations

How much effort does running an AI solution actually take?

Noticeably more in the first quarter after go-live than later — typically between 0.3 and 0.5 person-days per week per production AI application. After that, ongoing effort for stable use cases settles at 0.1 to 0.3 person-days per week, plus roughly half a day per month for the operations report and optimization triage. The spread comes from two factors: the complexity of the input data, and how often the surrounding business processes change.

When does Managed AI Operations pay off, and when doesn't it?

It pays off when the AI solution is embedded in day-to-day operations, there's no dedicated AI operations role internally, and outages or creeping quality loss cost real money. It doesn't pay off for pure experiments with no business load, for highly regulated in-house operations with an established MLOps team, or when volume is low enough that a quarterly spot-check is sufficient. Rule of thumb: once you're processing 100+ cases a day, or once there's direct customer impact, the question is no longer "whether" but "run by whom."

What gets underestimated most often in the first quarter after go-live?

Three things: (1) How often the input data shifts — new suppliers, new product lines, new language variants arrive faster than expected. (2) How strongly the escalation rate affects operational morale — the five percent of cases that are hard to escalate feel like they absorb all the attention. (3) How much upkeep the connecting interfaces to the ERP or CRM actually need. The AI itself is usually robust; the environment around it isn't.

What must an honest operations report include?

At minimum: current KPIs against the prior month (turnaround time, escalation rate, error rate, volume), incidents that occurred with cause and resolution, every model or configuration change made with date and rationale, current model and vendor status, an open optimization backlog with effort and benefit estimates, and a concrete recommendation for the coming month. What doesn't belong: marketing language, buzzwords, and vague "it's running great" statements.

What typical mistakes happen in the first twelve months?

(1) Nobody assigned ongoing responsibility — the pilot sponsor steps back, and an operations owner was never defined. (2) Model updates get adopted untested because "the vendor says it's better." (3) Escalation rules don't get adjusted even though input volume or mix has changed. (4) No rollback path documented for when something does break. (5) The pilot's success tempts the team to expand to the next use case before the first one is cleanly in regular operation.

AI solution live — is the upkeep handled?

In a free initial call, we assess the operations gaps in your production AI systems — and what upkeep effort is realistic, whether handled internally or via managed services.

Request a free initial call