Remember that time when you accidentally truncated a table in production? Or when you forgot the WHERE clause in your UPDATE statement? You’re not really a seasoned professional if you haven’t. There’s even a very apt name for that moment in time when the realization hits you: The oh-no second.
But what if there was some type of control to prevent this from happening? Like more restrictive controls, perhaps some type of peer-review process before you clicked “go”? Or even…
How we got here
In the last few decades, with the introduction of cheaper hardware, high-speed Internet connections and a number of other important paradigm shifts, IT-driven processes have seen an unparalleled adoption rate, driving a previously unimaginable evolution in all aspects and walks of society. Practically nothing runs without a server these days. IT has made a dramatic ascent from a comically marginal existence – bearded scientists with glasses, swapping tape rolls and crunching Fortran algorithms for the business analysts in suits and ties – to the point where computers are the glue that every organization and individual depends on.
The Fortran guys could easily take a full day or two of downtime if something crashed the server, and the accounting guys and the executives wouldn’t even know about it. But if your e-commerce site goes down for two hours in December, you’re going to hear about it.
And when stuff becomes business-critical, there is considerable incentive to make sure that it isn’t exposed to the wrong people and that it doesn’t crash. Enter restraints and controls. Some controls are mandated by law, others by management, nervously wanting to understand and control what’s going on at the IT department. And of course there are legions of accountants, auditors and lawyers, eagerly waiting to help in return for a nice, fat check.
Risk works. Risk is good.
There’s more than one clash of interests here. The perhaps most apparent one is that between control and risk. Lawyers and auditors, by profession, identify and mitigate risks. Their job is to keep you out of courts and out of prison if anything goes south. By profession, these are seriously risk-avert people.
Businesses, on the other hand, must voluntarily take on risks, in the form of projects or investments. If you’ve ever bought insurance, you know that you could insure literally anything, but the accumulated price would become so steep that it would guarantee you a net loss. By definition, in order to make money, a business has to be a calculated risks.
The four-hour workweek, re-imagined.
Which brings us to the matter of getting stuff done. Imagine if everything you do has to be approved by a stakeholder and a manager, every line of code you write is peer-reviewed, then tested in a dev test environment, then in an acceptance test environment (which should both contain reasonably fresh, yet scrambled copies of the production data), then approved for deployment by the stakeholder (who ideally should also take time to verify the results), and finally deployed to production by two other people, under a four-eyes principle where no single person can perform any change in production alone. Sprinkle this with a bunch of project meetings, all while leaving a long and winding trail of tickets and documentation.
This is how most development cycles look. Except, you know, the test environments are rarely fresh, the tests aren’t really that thorough, and the peer-review could probably be called a peer-glance at best.
And here’s the real problem with controls. Anyone who wants to get stuff done will try to follow the controls just enough to not get fired (or enough to get someone else fired if the shit hits the proverbial fan). And anyone who wants to do bad things (like run away with your money or your client accounts) will probably be smart enough to circumvent those controls.
Excessive controls build a culture that hampers the productive and doesn’t stop the malevolent.
Trust and accountability
I would argue that true IT control boils down to accepting calculated risks and making smart choices with regards to trusting the people you recruit, train and pay. If you don’t really trust them, you probably shouldn’t have taken them on board to begin with.
You could call this “agile control” if you like – if mistakes are allowed to happen, those mistakes should be an acceptable risk, provided that the fix also is really quick to implement. In a bigger perspective, fewer formal controls and processes will allow your business to develop faster, but it places a clear burden of responsibility on the practitioners as well as management.
Don’t get me wrong. You need to have good security in place. By all means, store the administrator password in a safe under your bed, keep your audit logs running, test your backups regularly, and foster a culture where people are not afraid to ask for advice or a peer review.
But the only ones who benefit from cumbersome controls and processes are the lawyers and auditors who bill by the hour to tell you horror stories of fraud, theft and failed deployments.