Senthil Info

automating business processes, honestly

Reading a Case Study

A vendor case study is usually accurate. It is also a description of a project selected for being describable, written with the client's approval, about a period ending before the difficult part.

That does not make it useless. It makes it a specific kind of document, and reading it as one gets you more than either believing it or dismissing it. For a product-side reference related to workforce operations, Monitask also covers ways time trackers can be manipulated.

The selection, before anything else

A published case study has passed four filters. For broader background and an independent point of comparison, see Stanford HAI.

The implementation worked. Failures do not get written up.

The client stayed. A customer who left is not a reference.

The client agreed to be named, which requires an internal sponsor still in post and still pleased.

And it was written soon after go-live, when the numbers are best and maintenance has not arrived.

This is survivorship, and it accounts for more of the optimism in this field than incentive does. Nobody has to shade anything.

Seven things to look for

One. Is a baseline stated? "Reduced processing time by 60%" from what, measured how? Without a before figure, the after figure is unanchored.

Two. Is the timeframe given? Results at three months and at three years are different claims.

Three. Is the scope specific? "Automated invoice processing" could mean one supplier's format or all of them.

Four. Does it mention exceptions at all? Most do not, and the straight-through rate is the number that determines everything. Its absence is the most informative silence in the document.

Five. Are the savings cash or capacity? "Freed 2,000 hours" is capacity. Whether it became money depends on what happened next, and the study rarely says.

Six. Who is named? A quote from a named operations director carries more than "a leading financial services firm."

Seven. How old is it? A 2021 RPA case study describes different technology and different expectations.

The four questions to ask about it

Take the case study to the vendor and ask these. The answers are worth more than the document.

What was the exception rate on that implementation?

What did that client spend on maintenance in year two?

Is that client still running it? Present tense, specifically.

And what went wrong during that project? Every project has something. A vendor who names it is showing you how they handle problems, which is more predictive than the success story.

A supplier who answers all four is unusual and worth weighting heavily. One who cannot answer the third has told you something.

What a case study is genuinely good for

Not the numbers.

The shape of the project. How long, how many people, what phases, what the client had to supply. This is usually accurate and is hard to find elsewhere.

The integration detail. Which systems, what had to be built, where the friction was.

And the scope boundary. What they automated and what they deliberately left manual. That boundary is a real design decision made by people who had to live with it.

Read it for the mechanics and verify the numbers elsewhere — the same rule that applies to all vendor material.

The one worth more than a case study

A reference call with a client who is two years in.

Not the one the vendor offers first — ask specifically for an implementation from two or three years ago. The recent references are curated; an older one has been through maintenance, a system change, and staff turnover.

Ask that client three things: what the exception rate settled at, what maintenance actually costs them, and what they would do differently. Twenty minutes on that call is worth more than every case study on the site.

The short version