How Revivify works

Setting it up

Setup is done once, by whoever already administers the cluster, and it is the only moment cluster access appears anywhere in this story. Everybody who casts a spell afterwards holds none. The steps below are how the design works; nothing here installs today.

1. 1.

   Install the operator, once per cluster

   The same operator serves every spell: you install it once, not once per spell. It runs in its own namespace and exposes nothing: no Service, no Ingress, no public endpoint, because it dials out rather than being called in.

   shell

   helm repo add spellbook https://charts.spellbook.dev
   helm install spellbook spellbook/operator \
     --namespace spellbook --create-namespace

2. 2.

   Say what your team owns

   This is the file that decides what "checkout" means. Restarting the wrong service is a more interesting failure than being denied permission, and the ownership map is what makes it impossible: a name that matches two workloads is refused, not guessed.

   spellbook.yaml

   teams:
     - name: storefront
       members:
         - ana@northwind.example
         - sam@northwind.example
       owns:
         - deployments.apps/checkout-api  in storefront
         - deployments.apps/cart-api      in storefront

3. 3.

   Grant the spell

   The grant reconciles into an ordinary Kubernetes Role holding one verb. Read the rule it produces: there is no delete in it, so a restart cannot become a deletion even if our code tries. The API server would reject it.

   spellbook.yaml

   grants:
     - team: storefront
       spells: [revivify]
   
   # becomes, in namespace storefront:
   #   rules:
   #     - apiGroups: ["apps"]
   #       resources: [deployments]
   #       verbs:     [patch]
   #
   # One verb. No delete, no create, no exec, no scale.

4. 4.

   Connect Slack, outbound

   Socket Mode holds a connection from your cluster out to Slack, so Slack never calls you. There is no callback URL to expose and no inbound firewall rule to open. The thing that can restart your production service is not reachable from the internet at all.

   shell

   kubectl create secret generic spellbook-slack \
     --namespace spellbook \
     --from-literal=app-token=xapp-… \
     --from-literal=bot-token=xoxb-…

5. 5.

   Cast it

   One PATCH, and Kubernetes performs the rolling restart it already knows how to perform. Ana holds no kubeconfig, and the audit record naming her is written before the call is made.

   slack

   /revivify checkout-api

This is the design, not an install guide: the chart, the file format and the flags do not exist yet. It is written out this far because a mechanism you can argue with is worth more than a promise you cannot.

What it refuses to do

A permission that was never issued cannot be misused, by a person, by a compromised account, or by an agent.

/revivify payments-db

You do not own it. The reply names the team that does and how to ask them. There is no override flag and no confirmation dialog, because a confirmation dialog is a permission you are holding.

kubectl rollout restart deploy/checkout-api

You do not have kubectl. Not revoked: never issued. The spell is the only path, so the version of this action that can be aimed at the wrong namespace does not exist for you.

Restart a StatefulSet or a DaemonSet

Not in the grant. Restarting a database replica set is a different blast radius from restarting a stateless API, and it will be a different spell with its own grant before it is a flag on this one.

Restart while a rollout is in progress

Refused. It names who started the rollout and when. Stacking restarts is how a slow deploy becomes an outage.

Scale it, change its image, edit its env

Not this spell, and not this grant. One spell is one verb on one kind of resource. There is no generic escape hatch: no apply-this-manifest, no run-this-command.

Explain what broke

Revivify restores; it does not diagnose. If you restart before you know the cause, it says so in the output, and the audit record shows that no diagnosis preceded the restart.

Failure modes

What happens when it cannot do the thing.

The new pods never become Ready

The rollout stalls, progressDeadlineSeconds expires, and Revivify reports that it stalled and at what replica count. It does not roll back on your behalf. Under a default rolling strategy the workload sits at reduced capacity until you act, and the report says so plainly rather than reporting success.

The service does not exist, or two things match the name

It refuses and lists the candidates. It never picks one for you, because picking the wrong one is the exact failure this spell exists to make impossible.

The grant is missing or misconfigured

The cast fails before the API call and names the exact verb and resource that is missing, so whoever administers the cluster can fix it in one line instead of bisecting an RBAC error.

Slack drops mid-cast

The PATCH either landed or it did not; there is no partial state to reconcile, because the restart is one call. The operator reports into the thread when it reconnects, and the audit record is written in-cluster whether or not Slack is reachable.

The restart does not fix it

It says the pods came back and the symptom did not clear, and it points you at /scry. A restart that appears to work and quietly leaves the cause in place is the worst outcome available, so the spell is loud about it.