One of my Ceph OSDs was down and would not come back up. The underlying problem turned out to be the USB network adaptor had been knocked and wasn’t connected to the host. There is a magic incantation to get it back up and running (at least in a Proxmox cluster).

The magic incantation to get the OSD up again:

ceph-volume lvm activate --all

(From the Proxmox documentation on troubleshooting Ceph):

A faulty OSD will be reported as down and mostly (auto) out 10 minutes later. Depending on the cause, it can also automatically become up and in again. To try a manual activation via web interface, go to Any node → Ceph → OSD, select the OSD and click on Start, In and Reload. When using the shell, run following command on the affected node:

ceph-volume lvm activate --all