The End of Static Keys: SSH Certificate Authorities Explained

This is Part 6 of the Machine Identity and Workload Auth series.

For platform engineers and SREs, SSH has long been the default tunnel into infrastructure. However, the traditional model of managing authorized_keys files across hundreds of servers is failing. In the static model, a developer generates a key pair, distributes the public key to every server they need access to, and that key remains valid indefinitely. If that developer leaves or their laptop is stolen, the key must be manually removed from every host, creating a significant security liability.

The mechanism to fix this is the SSH Certificate Authority (CA). By introducing a central signing service, you shift from distributing public keys to distributing short-lived certificates. This changes the trust model: servers no longer check a local list of approved keys. Instead, they check a single local CA public key and verify that the presented certificate is signed by that CA and has not expired.

How SSH Certificates Work

An SSH certificate is not a different type of key. It is a wrapper around an existing SSH public key. When you use ssh-keygen -t ed25519, you generate a key pair. When you sign that public key with an SSH certificate authority, you create a certificate file (e.g., user_key-cert.pub).

The critical mechanism here is the principals field within the certificate. Unlike a standard public key, which is just a string of bytes, a certificate binds the public key to specific identities and permissions. As documented in the OpenSSH protocol specifications, the server validates these principals against the connecting user.

1. Principals: These are the usernames or hostnames allowed to use the key. If a certificate lists principals ["alice", "dev-team"], the SSH daemon on a server will only grant access if the connecting user is alice or dev-team. If bob tries to use Alice’s key, the connection is rejected, even if the signature is valid.
2. Validity Period: Certificates have valid-after and valid-before timestamps. This is where short-lived SSH certificates come into play. A certificate might be valid for only 8 hours. Once that window closes, the key is useless, regardless of whether it is still in the authorized_keys file.
3. Extensions: You can restrict what the key can do, such as disabling port forwarding or limiting command execution.

This mechanism ensures that compromise is contained. If a short-lived certificate is stolen, it expires quickly, and it cannot be used by unauthorized principals.

Centralized Issuance Workflow

Managing CA keys manually is error-prone. The robust solution is to use a centralized SSH Certificate Authority (CA) hosted within a PKI system like HashiCorp Vault or smallstep step-ca. The workflow shifts from "copy-paste keys" to "request and receive."

Consider a developer needing access to a production database server via a CI/CD pipeline.

1. Developer Action: The developer authenticates to the internal PKI portal (or uses an API token) and requests a user certificate. They provide their public key (id_ed25519.pub) and specify the required principals (e.g., db-admin). In a CI/CD context, this request is automated using tools like vault ssh or custom scripts that fetch the certificate upon job start.
2. PKI Verification: The PKI system verifies the developer’s identity against the directory service (e.g., LDAP or Okta). It checks policy: "Does this user have permission to request access to db-admin?"
3. Signing: The PKI uses the CA private key to sign the developer’s public key. It embeds the principals, validity period (e.g., 4 hours), and extension restrictions into the certificate.
4. Distribution: The PKI returns the signed certificate (user_key-cert.pub). The developer places this in their ~/.ssh/ directory.

Crucially, the server does not need to be updated. The server already trusts the CA’s public key. As long as the certificate is valid and the principals match, access is granted. This eliminates the need to push configuration changes to servers when employees join or leave.

# Example: How the server verifies the certificate locally
# The server's sshd_config must point to the CA public key
TrustedUserCAKeys /etc/ssh/ca_user_key.pub
 
# When the client connects:
ssh alice@db-prod-01
# The client sends:
# 1. Public key: id_ed25519.pub
# 2. Certificate: id_ed25519-cert.pub
# The server checks:
# 1. Is the certificate signed by ca_user_key.pub?
# 2. Is 'alice' in the principals list?
# 3. Is the current time within valid-after/valid-before?

Host Certificates: Scaling Server Trust

User certificates are only half the battle. In large environments, verifying host keys is equally painful. Developers often bypass warnings when host keys change (due to re-imaging or IP reuse).

SSH Host Certificates solve this. The CA signs the server’s host key. The client machine stores the CA’s public key in ~/.ssh/known_hosts or /etc/ssh/ssh_known_hosts. When connecting, the server presents its host key and the host certificate. The client verifies the signature against the CA.

This allows you to rotate host keys without updating every developer’s known_hosts file. As long as the new host key is signed by the same CA, the client trusts it. This is essential for immutable infrastructure patterns where servers are frequently replaced.

Revocation and Audit

A common misconception is that SSH certificates require a Certificate Revocation List (CRL) like TLS/SSL. While OpenSSH 6.1+ supports Key Revocation Lists (KRLs) [OpenSSH Release Notes], they are rarely used in practice for SSH. Why? Because the primary defense is short TTLs.

If a key is compromised, it expires in hours. The real power of centralized management is audit. Since all certificates are issued through a central SSH Certificate Authority (CA) within the PKI, you have a complete log of:

1. Who requested the certificate.
2. When it was issued.
3. What principals were granted.
4. When it expired.

Additionally, OpenSSH logs authentication attempts. By integrating these logs with your SIEM (Security Information and Event Management) system, you can correlate certificate usage with user activity. If a certificate is used from an unusual IP address, you can alert immediately.

Operational Considerations

The CA private key is the root of trust. It must be stored in a hardware security module (HSM) or a highly secured vault. It should never touch a standard server disk. Since certificates rely on timestamps, all servers and clients must have accurate time (NTP); a skew of more than a few minutes can cause valid certificates to be rejected. Always maintain an emergency access path, often done with a "break-glass" key stored offline. These practices align with core platform engineering principles, ensuring that infrastructure operations are automated, reliable, and secure.

Conclusion

SSH certificates transform key management from a manual, distributed chore into an automated, centralized policy enforcement mechanism. By leveraging the principals field and short validity periods, you reduce the blast radius of key compromise and eliminate the need to manage authorized_keys files across your fleet. For platform engineers, this is a foundational requirement for scalable, secure infrastructure operations.

The shift is straightforward: generate a CA, configure servers to trust it, and automate certificate issuance through your PKI. The result is an infrastructure where access is ephemeral, auditable, and centrally controlled.