Self-service ops
Self-service ops is the practice of letting the developer who owns a service perform the operational actions it needs (restart it, roll it back, read its logs) without filing a ticket and without being handed production access. It exists because both of the obvious alternatives are bad. A ticket queue turns a thirty-second restart into a day of waiting and costs two people their concentration; a broad grant of cluster access turns every developer into a permanent risk with a cluster-sized blast radius. Self-service ops is the attempt to remove the queue without granting the access, and an implementation that only manages one of those two has not done it.
The ticket is more expensive than it looks
The obvious cost of a ticket is the waiting. The real cost is the interruption, and it lands twice: once on the developer who has to stop and hand their context off, and once on whoever picks the ticket up and has to load it. Interruptions during engineering work are well studied. Breaking the Flow (Ma, Huang and Leach, ICSE 2024) measured what they do to time-on-task and to stress across code writing, comprehension, and review. The finding that matters here is the unremarkable one: interruptions are not free, and what they cost is out of all proportion to how long they last.
A restart that takes thirty seconds to perform and half a day to get performed has not cost you half a day. It has cost you two people's afternoons.
The broad grant is more expensive than it looks
The other way to kill the queue is to give everybody kubectl. This is fast, and it is why so many teams end up there. It also means every developer permanently holds enough access to take the cluster down, which is a blast radius problem, and holds it as a credential on a laptop between the two times a year they use it, which is a standing access problem. Nobody chose that risk; they chose to stop filing tickets, and this came attached.
The two ways implementations fail
- Fast but unbounded. The team removes the queue by handing out access. Restarts happen in seconds, and so does everything else.
- Bounded but slow. The team keeps the boundary and rebuilds the queue inside it. The internal portal has a beautiful button, and the button opens an approval that an SRE has to click. The developer is still waiting on a human; the ticket has simply been given a nicer interface.
What separates real self-service from a repainted ticket queue is when the decision gets made. In a real self-service action, the question of who may do what was answered once, in advance, in configuration. At the moment of use there is nothing to approve, because it was already approved by a rule rather than by an exception. The approval moves from runtime to configuration.
What a self-service action has to be
- Narrow. One operation with a defined effect. A shell is not a self-service action; it is every action.
- Correct by construction. It does the operation the right way, so the person casting it does not need to know the right way. A rolling restart, not a deleted pod.
- Scoped to ownership. You can act on what you own, and the system knows what you own.
- Auditable. Who, what, to what, and when, written down without anyone having to remember to write it down.
- Safe when it fails. It refuses clearly, says why, and says who to ask. A refusal that explains itself is worth more than a success, because it is the thing that lets the boundary hold without anyone routing around it.