Two Kinds of Visibility

One morning in 2013, a senior manager stopped at our whiteboard at Inland Revenue while my team was running through the day’s incidents, looked at the columns of cards for a while, and said, “Wayne, we don’t do agile here.”

I said, “No, we’re doing TPS. Toyota Production System.”

She wasn’t satisfied with that. “Just remember, Inland Revenue doesn’t do agile.” Then she walked off.

She ran ICT Solutions. She had sat on the panel that hired me. And she had carried Mary and Tom Poppendieck’s Lean Software Development around during her MBA sometime later, which made the line even better.

That was the entire exchange. Nothing came after it, no memo, no meeting invite to discuss my team’s unauthorised working methods. And looking back, the reason nothing came after it is the whole point of this piece. We never asked permission for what we were doing, and we never argued for it in a meeting room either, because we would have lost that argument. We did the work where people could see it, and let the results carry the case.

Some context first. I ran Business Platform Services Channels at Inland Revenue from January 2013 to October 2015, a team of 11 or 12 looking after the systems sitting between New Zealand taxpayers and everything behind the portal. Channel owned about 40 percent of Inland Revenue’s ICT applications in its own right, but as the front end of the service chain, up to 70 percent of all application interfaces flowed through us on their way to the middleware, the databases, and the mainframe at the back. So any incident touching the portal got assigned to us to triage first, whether or not the fault actually lived behind us. We wore every complaint, and that’s how busy we were: sifting through the shit to work out whose it was. When the portal goes down, nobody rings the database team.

The technology was waiting for business transformation to arrive. WebSphere servers out of support from IBM, infrastructure being sweated to survive until the new world got funded and built. Least resources, oldest kit, first in line for blame. And in my first year, 2013, we logged 33 priority one incidents. Thirty-three times in a single year, a system serving every taxpayer in the country fell over badly enough to be declared a P1.

What We Actually Did

Every morning we stood at a whiteboard and went through the work. To do, in progress, done. Whatever needed attention across the entire service line got a card. This wasn’t proper agile; there was no Scrum Master and no sprint cadence, because production support doesn’t run in sprints. Incidents don’t wait for planning day. We were level 2 and level 3 in the support chain, taking what the Service Desk at level 1 passed up, triaging it the way an emergency department triages patients, firming up the diagnosis, fixing what we could ourselves, and passing the rest to the development teams at level 4. Inland Revenue had run Scrum on some projects before, and several of my people knew the ceremonies well, so the muscle memory was there. We simply pointed it at operations instead.

I called it Toyota Production System because that’s what it was: make the work visible, limit what’s in progress, let the team pull the next priority, and treat every incident as a signal about the system rather than a fire to forget once it’s out.

Teru Yanagihasi, my Senior IT Consultant, took the visibility further than I would have on my own. He put a metrics dashboard straight onto the whiteboard, openly displaying where the issues actually lived across the service line, which trends were building, what needed watching next. He also introduced a Niko Niko board, a Japanese practice where each person marks their mood for the day, which sounds soft until you realise it tells you about the condition of the team before the incident queue does. A tired team misses thresholds. Many of the ideas that worked in that team started as Teru’s; his dedication and commitment were second to none.

None of this was electronic. There was no tooling budget. Whiteboard, markers, and a team that showed up every morning.

My peers and my leadership tolerated all of it, I guess because the numbers kept moving in the right direction and it cost them nothing to look the other way. The P1 count kept falling, and they left us alone. Results bought us an autonomy that no argument would have.

The Viral Part

Then something happened that I didn’t plan. Colleagues from the other Business Platform Services teams walked past our corner, saw the board, watched the morning ritual for a while, and whiteboards started appearing in their spaces too. Nobody mandated visual management. There was no workshop, no change programme, no lunchtime session on lean methods. The practice spread because it sat in the line of sight of anyone walking the floor, and because the thing it was attached to, our incident numbers, was visibly improving month after month.

Eventually my Director for ICT Operations bought us a 45 inch TV to display our monitoring dashboards, an envy to the rest of BPS. I suspect that TV did more for adoption across the floor than any presentation I could have given.

The numbers are the part I can still recite. 33 priority one incidents in 2013. Seven in 2014. Three in 2015, my final year.

Behind those numbers was a team watching thresholds against upper control limits, CPU, cache, database log files, queue depths, and taking preventive action in quiet hours before any limit was breached. The whiteboard made the priorities visible. The dashboards made the thresholds visible. The discipline was simply that somebody was always watching and always acting early, and that discipline held availability at 99.9 percent through tax peak seasons, on assets that were officially past their support life. How we sweated those assets through peak season deserves its own piece, so I’ll leave that story for another day.

The Newsletter

Three P1s in a year got attention at the top. The CIO came down and asked what we had done to make that possible, and then he wanted a piece in the internal newsletter about the improvement in operations.

Here’s where I learned something about visibility that the whiteboard hadn’t taught me. I told him the truth: we relied on more eyeballs watching the thresholds of the upper control limits for failure. He looked puzzled. I explained the eyeballs again, and he looked more puzzled. You can’t write “we watched harder and acted earlier” in a newsletter. There’s no chart for it, no milestone, no budget line, nothing that reads like an achievement.

In the end, I suggested he attribute the improvement to the mainframe, since we had a mainframe project running at the time. A project is quantifiable. It has a name, a start date, an end date, a cost. It fits the shape a newsletter expects. So the mainframe got the credit, and I’m the one who offered it, because I understood the constraint he was working under. The improvement he actually came down to ask about was the ingenuity of an entire team monitoring thresholds and sweating aging assets, and that never gets published, because it sounds simple and boring.

Two Kinds of Visibility

I’ve thought about this a lot since. There were two kinds of visibility in that story, and they behave completely differently.

Sideways visibility spreads practice. Peers copied the whiteboard because they could see it working with their own eyes, on their own floor, attached to numbers that kept falling; no persuasion required, no business case, no mandate. When results are physically visible, adoption becomes voluntary, and voluntary adoption sticks in a way that mandated change rarely does. That’s the persistence half of the story. We kept doing the same simple things, quietly, for three years, and the copying followed the results.

Upward visibility grants credit, and it only registers what is countable and project-shaped. The institution could see the mainframe project because it had the right form. It couldn’t see the eyeballs, even when the person responsible stood in front of the CIO and explained them twice. So the official record of that improvement, if anyone ever digs out that newsletter, says the mainframe did it.

Both kinds are real, and both matter. But if you run operations anywhere, it’s worth knowing that the work that spreads among practitioners and the work that gets credited by the institution can be entirely different work, and the gap between them is where a lot of quiet operational excellence lives and dies unrecorded.

Teru returned to Japan after 2021. The senior manager was right in the end; Inland Revenue never did do agile, at least while I was there. It did TPS for three years in one corner of the floor, and the whiteboards told everyone who cared to look. Even she was converted before I left.

I still think the eyeballs deserved the newsletter.