Developing in Production (Yes, Really)

There’s a version of data work that exists in documentation, architecture diagrams, and conference talks. It has clean environments, clear requirements, and a sensible deployment process. And then there’s the version I keep encountering, the one where production is already live, the data refresh is scheduled, and the question is no longer “should we test this?” but “how noticeable will this be if it breaks?”

This blog exists because most of what I’ve learned about data, BI, cloud platforms, modelling, and analytics didn’t come from carefully staged environments. It came from doing real work on real systems, often under time pressure, sometimes with incomplete information, and occasionally while hoping no one was actively looking at the dashboard I just changed.

Developing in production isn’t something I aim for. It’s something that happens. Deadlines arrive. Data changes shape. Requirements evolve mid-refresh. The “proper” way gives way to the necessary way, and suddenly production becomes the place where the lesson actually sticks. Not because it’s ideal but because the consequences are real.

I’m starting this blog to document those lessons as they happen. Not the polished, best-practice version you’d present in hindsight, but the version you learn while fixing something that worked yesterday and inexplicably doesn’t today. The moments where a small change reveals a much bigger misunderstanding. The quiet wins. The loud mistakes. The “oh, that’s how that actually works” realisations.

This isn’t a guide to doing things perfectly. It’s a record of learning by building, breaking, and fixing. Sometimes carefully. Sometimes directly on production. Always with the goal of understanding things a little better next time.

If you’ve ever:

• trusted a dashboard and immediately regretted it
• learned more from a production issue than a week of documentation
• made a “small change” that was, in fact, not small

then you already understand the premise.

Future posts will be practical, specific, and occasionally confessional, covering data modelling, BI tools, cloud platforms, and whatever else I happen to be learning the hard way. If nothing else, this will serve as a reminder to my future self of what not to do again.