Anthropic enterprise models are very strong at following runbooks and managing infrastructure. To frame this properly, I spent about a month documenting the environment and creating runbooks. I can currently do things like the following:
Follow the steps below to roll over this cluster. {
Choose a server in this cluster by ID in numeric order (this server)
Use Ansible+vault to create a backup AMI of (this server)
Use Ansible+vault to collect health data on (this server) according to the server health runbook
Use Ansible+vault to disable notification for this server
Use Ansible+vault to preform a full kernel update and reboot the vm according to the update runbook.
Use Ansible+vault to collect health data on this server according to the server health runbook post update
Compare the pre and post health and server alarms for (this server), if there is an issue, pause, SOS notify oncall
Use Ansible+vault to enable notification for this server
}
Each runbook contains exact steps required, how to verify success, each one of them is able to pull the E-stop, freeze on that step and SOS notify.