CIS Ceph Benchmark

This recommendation verifies that Ceph is running the latest stable version on the Ceph distributed storage system. Implementing this control strengthens the overall security configuration and reduces exposure to potential threats.

Failure to implement this control may leave the Ceph distributed storage system vulnerable to attack or non-compliant with organizational security policies. This control helps establish a consistent, hardened configuration baseline.

# Verify Ceph version
ceph --version
ceph health detail

# Update Ceph to latest stable
apt-get update && apt-get install --only-upgrade ceph ceph-common
# Or use cephadm:
ceph orch upgrade start --ceph-version 18.2.x
ceph orch upgrade status

This recommendation verifies that Ceph daemons run as a dedicated non-root user on the Ceph distributed storage system. Implementing this control strengthens the overall security configuration and reduces exposure to potential threats.

Failure to implement this control may leave the Ceph distributed storage system vulnerable to attack or non-compliant with organizational security policies. This control helps establish a consistent, hardened configuration baseline.

# Check Ceph daemon users
ps -eo user,pid,comm | grep ceph
grep 'setuser' /etc/ceph/ceph.conf

# Ensure daemons run as ceph user
# In ceph.conf [global]:
# setuser match path = /var/lib/ceph/$type/$cluster-$id
useradd -r -s /usr/sbin/nologin ceph 2>/dev/null
chown -R ceph:ceph /var/lib/ceph /var/log/ceph /etc/ceph

This recommendation ensures that CephX authentication is required for all connections on the Ceph distributed storage system. Enforcing this requirement establishes a minimum security standard and prevents insecure configurations.

Without this enforcement, the Ceph distributed storage system may accept insecure configurations that weaken overall security posture. Mandating this control ensures consistent protection against common attack vectors.

# Check CephX authentication
ceph config get mon auth_cluster_required
ceph config get mon auth_service_required
ceph config get mon auth_client_required

# Enable CephX authentication
# In ceph.conf [global]:
# auth_cluster_required = cephx
# auth_service_required = cephx
# auth_client_required = cephx
ceph config set global auth_cluster_required cephx
ceph config set global auth_service_required cephx
ceph config set global auth_client_required cephx

This setting ensures that keyring files have restricted permissions on the Ceph distributed storage system. Restricting this capability limits potential abuse and enforces the principle of least privilege across the environment.

Unrestricted access to this capability could allow unauthorized users or processes to perform actions beyond their intended scope. Applying least-privilege principles to the Ceph distributed storage system is essential for defense in depth.

# Check keyring permissions
find /etc/ceph -name '*.keyring' -exec stat -c '%a %U %G %n' {} \;
ceph auth ls | head -30

# Restrict keyring permissions
chmod 600 /etc/ceph/*.keyring
chown ceph:ceph /etc/ceph/*.keyring
# Remove unnecessary auth keys:
ceph auth rm client.unused_key
# Rotate admin key:
ceph auth get-or-create client.admin mon 'allow *' osd 'allow *' mgr 'allow *' mds 'allow *'

This recommendation verifies that messenger v2 encryption is enabled for all traffic on the Ceph distributed storage system. Implementing this control strengthens the overall security configuration and reduces exposure to potential threats.

Failure to implement this control may leave the Ceph distributed storage system vulnerable to attack or non-compliant with organizational security policies. This control helps establish a consistent, hardened configuration baseline.

# Check messenger v2 encryption
ceph config get global ms_cluster_mode
ceph config get global ms_service_mode
ceph config get global ms_client_mode

# Enable messenger v2 encryption
ceph config set global ms_cluster_mode 'secure crc'
ceph config set global ms_service_mode 'secure crc'
ceph config set global ms_client_mode 'secure crc'
# For strict encryption:
ceph config set global ms_cluster_mode secure
ceph config set global ms_service_mode secure
ceph config set global ms_client_mode secure

This recommendation verifies that RADOS Gateway uses TLS with valid certificates on the Ceph distributed storage system. Implementing this control strengthens the overall security configuration and reduces exposure to potential threats.

Failure to implement this control may leave the Ceph distributed storage system vulnerable to attack or non-compliant with organizational security policies. This control helps establish a consistent, hardened configuration baseline.

# Check RGW TLS configuration
ceph config get client.rgw rgw_frontends
grep 'rgw_frontends\|ssl' /etc/ceph/ceph.conf

# Configure RGW with TLS
# In ceph.conf [client.rgw.<name>]:
# rgw_frontends = beast ssl_port=443 ssl_certificate=/etc/ceph/rgw-cert.pem ssl_private_key=/etc/ceph/rgw-key.pem
ceph config set client.rgw rgw_frontends 'beast ssl_port=443 ssl_certificate=/etc/ceph/rgw-cert.pem ssl_private_key=/etc/ceph/rgw-key.pem'
chmod 600 /etc/ceph/rgw-*.pem

This recommendation verifies that centralized logging with syslog forwarding is configured on the Ceph distributed storage system. Implementing this control strengthens the overall security configuration and reduces exposure to potential threats.

Failure to implement this control may leave the Ceph distributed storage system vulnerable to attack or non-compliant with organizational security policies. This control helps establish a consistent, hardened configuration baseline.

# Check Ceph logging configuration
ceph config get global log_to_file
ceph config get global log_file
ceph config get global mon_cluster_log_file

# Configure centralized logging
ceph config set global log_to_file true
ceph config set global log_file /var/log/ceph/$cluster-$name.log
ceph config set global mon_cluster_log_file /var/log/ceph/$cluster.log
ceph config set global log_to_syslog true
ceph config set global err_to_syslog true

This recommendation verifies that audit logging with log rotation is enabled on the Ceph distributed storage system. Implementing this control strengthens the overall security configuration and reduces exposure to potential threats.

Failure to implement this control may leave the Ceph distributed storage system vulnerable to attack or non-compliant with organizational security policies. This control helps establish a consistent, hardened configuration baseline.

# Check debug and audit logging
ceph config get global debug_mon
ceph config get global debug_osd
ceph config get mgr mgr/cephadm/log_to_cluster

# Configure audit logging (low debug for production)
ceph config set global debug_mon 1/5
ceph config set global debug_osd 1/5
ceph config set global debug_auth 2/5
# Enable manager audit logging:
ceph config set mgr mgr/cephadm/log_to_cluster true
# Log rotation:
cat > /etc/logrotate.d/ceph << 'EOF'
/var/log/ceph/*.log {
    daily
    rotate 14
    compress
    missingok
    notifempty
    sharedscripts
    postrotate
        killall -q -1 ceph-mon ceph-osd ceph-mds ceph-mgr
    endscript
}
EOF

This recommendation verifies that public and cluster networks are separated on the Ceph distributed storage system. Implementing this control strengthens the overall security configuration and reduces exposure to potential threats.

Failure to implement this control may leave the Ceph distributed storage system vulnerable to attack or non-compliant with organizational security policies. This control helps establish a consistent, hardened configuration baseline.

# Check network separation
grep -E 'public_network|cluster_network' /etc/ceph/ceph.conf
ceph config get global public_network
ceph config get global cluster_network

# Separate public and cluster networks
# In ceph.conf [global]:
# public_network = 10.0.1.0/24
# cluster_network = 10.0.2.0/24
ceph config set global public_network 10.0.1.0/24
ceph config set global cluster_network 10.0.2.0/24
# Firewall rules:
ufw allow from 10.0.1.0/24 to any port 6789,3300
ufw allow from 10.0.2.0/24 to any port 6800:7300

This recommendation verifies that management dashboard uses TLS with authentication on the Ceph distributed storage system. Implementing this control strengthens the overall security configuration and reduces exposure to potential threats.

Failure to implement this control may leave the Ceph distributed storage system vulnerable to attack or non-compliant with organizational security policies. This control helps establish a consistent, hardened configuration baseline.

# Check mgr dashboard TLS
ceph mgr module ls | grep -A3 dashboard
ceph config get mgr mgr/dashboard/ssl

# Enable dashboard with TLS
ceph mgr module enable dashboard
ceph dashboard create-self-signed-cert
# Or use proper certs:
ceph dashboard set-ssl-certificate -i /etc/ceph/dashboard-cert.pem
ceph dashboard set-ssl-certificate-key -i /etc/ceph/dashboard-key.pem
ceph config set mgr mgr/dashboard/ssl true
ceph dashboard ac-user-create admin -i /tmp/pw.txt administrator

This recommendation verifies that pools have appropriate replication and application tags on the Ceph distributed storage system. Implementing this control strengthens the overall security configuration and reduces exposure to potential threats.

Failure to implement this control may leave the Ceph distributed storage system vulnerable to attack or non-compliant with organizational security policies. This control helps establish a consistent, hardened configuration baseline.

# Check pool configuration
ceph osd pool ls detail | head -30
ceph osd pool get <pool_name> size
ceph osd pool get <pool_name> min_size

# Harden pool settings
# Set replication factor:
ceph osd pool set <pool_name> size 3
ceph osd pool set <pool_name> min_size 2
# Enable application tags:
ceph osd pool application enable <pool_name> rbd
# Set PG autoscale:
ceph osd pool set <pool_name> pg_autoscale_mode on

This recommendation verifies that OSD encryption at rest is enabled on the Ceph distributed storage system. Implementing this control strengthens the overall security configuration and reduces exposure to potential threats.

Failure to implement this control may leave the Ceph distributed storage system vulnerable to attack or non-compliant with organizational security policies. This control helps establish a consistent, hardened configuration baseline.

# Check OSD encryption
ceph osd tree
lsblk -o NAME,FSTYPE,MOUNTPOINT | grep ceph
dmsetup ls --target crypt 2>/dev/null

# Enable OSD encryption at rest
# For new OSDs with cephadm:
ceph config set global osd_dmcrypt_type luks2
# Deploy encrypted OSD:
ceph orch daemon add osd <host>:<device> --encrypted=true
# Verify:
ceph osd metadata <osd_id> | grep dmcrypt

This recommendation verifies that CRUSH rules use appropriate failure domains on the Ceph distributed storage system. Implementing this control strengthens the overall security configuration and reduces exposure to potential threats.

Failure to implement this control may leave the Ceph distributed storage system vulnerable to attack or non-compliant with organizational security policies. This control helps establish a consistent, hardened configuration baseline.

# Check CRUSH map and erasure coding
ceph osd crush rule ls
ceph osd erasure-code-profile ls
ceph osd crush dump | head -20

# Configure CRUSH rules for failure domains
ceph osd crush rule create-replicated replicated_rack default rack
ceph osd pool set <pool_name> crush_rule replicated_rack
# Create erasure coding profile:
ceph osd erasure-code-profile set ec-4-2 k=4 m=2 crush-failure-domain=rack
ceph osd pool create ec-pool 32 erasure ec-4-2

This recommendation verifies that balancer and scrub maintenance windows are configured on the Ceph distributed storage system. Implementing this control strengthens the overall security configuration and reduces exposure to potential threats.

Failure to implement this control may leave the Ceph distributed storage system vulnerable to attack or non-compliant with organizational security policies. This control helps establish a consistent, hardened configuration baseline.

# Check balancer and scrub settings
ceph balancer status
ceph config get osd osd_scrub_auto_repair
ceph config get global osd_max_scrubs

# Configure balancer and maintenance
ceph balancer mode upmap
ceph balancer on
# Configure scrub settings:
ceph config set global osd_max_scrubs 1
ceph config set osd osd_scrub_auto_repair true
ceph config set osd osd_deep_scrub_interval 604800
# Set maintenance window:
ceph config set osd osd_scrub_begin_hour 2
ceph config set osd osd_scrub_end_hour 6