Senthil Info

automating business processes, honestly

What to Measure After Go-Live

Most automation projects are measured once, before approval, using estimates. After go-live the measurement stops, because the project has closed and the sponsor has moved on.

Five numbers, monthly, take about twenty minutes. They tell you whether the thing works, and they are the only evidence your organisation will ever accumulate about how automation performs in your operation. A product-side example of how workforce software approaches this topic is available in this article.

The five

One. Straight-through rate. Instances completed with no human touch, as a proportion. The number the whole business case rested on, and now measurable rather than assumed. If it comes in twenty points below the estimate, the project delivered something different from what was approved, and that is worth knowing in month two rather than year two. For broader background and an independent point of comparison, see Stack Overflow.

Two. Hours spent on exceptions. Logged, not estimated. Exceptions take longer once isolated, so this is usually above the pre-automation per-item rate.

Three. Hours spent on the automation itself. Monitoring, restarts, fixes, rule changes. This is the maintenance line that first-year cases omit, and after six months you have a real figure to use in the next case.

Four. Error and rework rate. Frequently the actual benefit and rarely the claimed one. Also the early warning for silent failure after an upstream change.

Five. Volume. Whether the process is growing, and whether volume is routing around the automation. A quiet drop in instances processed usually means someone found a way past it.

The comparison that matters

Against your own pre-automation baseline, not against the vendor's model and not against an industry figure.

The tally sheet you ran before the project is what makes this possible, which is the third distinct reason to run it. Without a baseline, post-go-live numbers are just numbers.

The month-nine review

Put the date in the business case and keep it.

Nine months is long enough for the ramp to finish, for at least one upstream change, and for maintenance to appear. Earlier is too kind and later is too late to act.

Three questions:

Did the straight-through rate hold?

What did maintenance actually cost?

And did the benefit convert the way we said — cash or capacity, and did the cash appear in a budget line?

Write the answers down whatever they are. A project that underdelivered and was measured honestly is worth more to the organisation than one that overdelivered and was never checked, because the first improves the next business case and the second does not.

What most organisations do instead

Run several automation projects and never compare one against its own case.

The reason is structural rather than lazy: the project closes, the budget code shuts, the sponsor is measured on delivery rather than on outcome, and nobody's job includes looking back. The same absence that means nobody publishes small-implementation outcomes operates inside individual companies.

Which means a company that does this consistently has something almost nobody has: actual local data about what automation delivers in its own operation. After three projects that is worth more than any industry figure.

Keeping it cheap

Five numbers in a spreadsheet, monthly, by one named person. Twenty minutes.

Not a dashboard. A monthly figure read by someone with context beats a live screen glanced at by several people without it.

And review it quarterly with the process owner, which is fifteen minutes and is where the numbers turn into decisions — a rule to add, an upstream input to fix, or an automation to turn off.

The short version