Graduating security with CNCF projects

Utilising Rook, Ceph and Vitess to achieve security goals for production workloads.

Yes I know I might be a little abrupt with better late than never in understanding the amazing world of graduated CNCF projects. Taking a look at the plethora of options at the table we start off with the basics and build out way up with Rook, Ceph and Vitess.

What is Rook and Ceph?

Consider Rook conducting an orchestra. Within your Kubernetes cluster, Rook oversees the deployment and lifecycle of Ceph storage in its capacity as a Kubernetes operator — a unique kind of controller.

What Rook provides is:

• Simplified Deployment: Ceph on Kubernetes is deployed and configured automatically by Rook. This simplifies the process by doing away with the need for manual Ceph installation and configuration on separate nodes.
• Self-Management: Rook handles scaling, healing, and upgrades throughout Ceph’s lifecycle. This relieves administrators of the burden of overseeing Ceph’s intricate workings.
• Integration with Kubernetes: Rook uses resources and Ceph daemons (background processes) by utilizing Kubernetes primitives like pods, deployments, and services. Your Kubernetes applications’ storage provisioning is made easier by this smooth integration.

Rook orchestrates Ceph, the underlying storage engine. This robust distributed storage system is available as open-source software and offers:

Scalability: Ceph’s horizontal scaling, achieved by adding additional nodes to the cluster, makes it simple to boost performance and storage capacity.
High Availability: Ceph ensures data redundancy and continuous operation even in the event of a node failure by replicating data across multiple nodes.
Flexibility: Ceph can be used for a variety of storage requirements in your Kubernetes environment because it can store files, objects, and blocks.
For your Kubernetes apps, Rook and Ceph work together to offer a reliable and safe persistent storage solution.

Here’s a comparison: Consider a web application that uses a database to store user data. This database used to likely be located on a different server. The database can utilize Ceph’s persistent storage, which is provided by Ceph and managed by Rook inside the Kubernetes cluster. This guarantees your application’s scalability and streamlines management.

Now why consider securing k8s with Rook?

• Expanded Attack Surface: Because of its distributed architecture and containerized design, Kubernetes offers a larger attack surface. It’s critical to secure underlying storage, the Kubernetes API server, and container images.
• Data Persistence with Rook: Rook makes it possible for stateful Kubernetes applications to have persistent storage. Data confidentiality and integrity are guaranteed by protecting this storage layer.
• GCP Integration: By combining Rook’s capabilities with GCP’s security features, the overall security posture is strengthened.

To give Kubernetes persistent storage, Rook integrates with Ceph, protecting Rook entails protecting Ceph as well:

• Rook Operator Security: Use Kubernetes’ Role-Based Access Control (RBAC) to manage access to the Rook operator. Limit who has access to launch and control Rook instances.
• Ceph Authentication: To manage access to the Ceph cluster, make use of the built-in authentication mechanisms provided by Ceph, such as Cephx or Kerberos.
• Ceph Authorization: Manage user or service account permissions for particular Ceph resources, such as pools or monitors, by implementing authorization policies within Ceph.
• Ceph Encryption: Use programs like dmcrypt or the encryption features built into Ceph to enable encryption of data both in-transit and at rest. This keeps sensitive data safe even in the event that the storage layer is breached.

apiVersion: rook.io/v1
kind: CephCluster
metadata:
  name: my-ceph-cluster
spec:
  resources:
    requests:
      mon:
        cpu: 100m
        memory: "1Gi"
  # ... other configurations
  securityContext:
    runAsUser: 1000
    fsGroup: 1000

This sample configures a Rook cluster with a securityContext specifying a dedicated user and group for the Rook operator pods, enhancing isolation.

Similarly Ceph RBD encryption can help out in utilizing an added layer of security to kubernetes clusters. Imagine a situation where a Ceph block device is used by a Kubernetes application to store sensitive data. The data will stay unreadable even if unauthorized access is gained to the underlying storage if RBD (RBD — Block Device) encryption is enabled.

Ceph RBD encryption encrypts data before sending it to the Ceph storage cluster using client-side tools like dmcrypt. Below is a summary of the entire procedure along with some pertinent code samples:

# Create a physical volume for the RBD device
pvcreate /dev/sdX

# Create a logical volume group named "ceph-rbd"
vgcreate ceph-rbd /dev/sdX

# Create a logical volume named "rbd-data" of desired size within the volume group
lvcreate -L 10G -n rbd-data ceph-rbd

# Prepare the logical volume for LUKS encryption
cryptsetup --verbose --type luks1 --cipher aes-256-cbc --key-size 256 --hash sha256 -h /dev/mapper/ceph-rbd-rbd-data luks-rbd

# Open the encrypted logical volume
cryptsetup luksOpen luks-rbd rbd-data

We further configure Rook to utilize this encrypted block device

apiVersion: rook.io/v1
kind: CephBlockPool
metadata:
  name: encrypted-rbd
spec:
  # ... other pool configurations
  replicated:
    # Adjust these values based on your desired replication settings
    replicas: 3
  # Key piece: specify the encrypted device mapper path
  storageClassDeviceSets:
  - name: encrypted-deviceset
    # Replace with the actual path from step 1 (e.g., /dev/mapper/rbd-data)
    dataDevices: ["/dev/mapper/rbd-data"]
  # Optional: Enable encryption at rest with a KMS provider (replace with your provider)
  encryption:
    provider: kubernetes
    # ... KMS provider specific configurations

Now we simply utilize this RBD pool for our Kubernetes application

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: encrypted-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: encrypted-rbd

This code defines a PersistentVolumeClaim that will utilize the encrypted RBD pool for persistent storage.

Integrations with GCP features

You can use the strong security features that GCP provides to further fortify your Kubernetes deployment with Rook:

• Cloud Key Management Service (KMS): Rook and Ceph encryption keys can be managed with KMS. This improves the overall security posture and centralizes key management.
• Cloud Identity and Access Management (IAM): To manage access to GCP resources that Rook uses, like Cloud Storage buckets for Ceph OSDs (Object Storage Daemons), extend IAM policies.

Rook and Ceph Resources

Rook

https://rook.io/docs/rook/latest-release/Getting-Started/intro/

Securing MySQL shard with Vitess

Vitess was created to enable MySQL to scale horizontally. Some of the main features of this versatile tool include:

• Vitess shards your MySQL database horizontally, distributing data among several nodes according to a predetermined sharding key. This makes it possible to scale writes and reads separately for increased performance and high availability.
• Scalability: Vitess enables you to add more MySQL nodes (shards) as your data volume increases by sharding your database. This offers a means of managing growing database workloads without compromising performance.
• Kubernetes Integration: You can deploy and manage Vitess instances as pods within your Kubernetes cluster thanks to Vitess’s good integration with Kubernetes. For Vitess deployments, Kubernetes handles scaling, self-healing, and rolling updates.
• Vitess for Scalability: Vitess distributes data over several nodes by horizontally sharding your MySQL database. For high availability and performance, this makes it possible to scale reads and writes independently.
• Kubernetes orchestration: Vitess pods are managed by Kubernetes throughout their lifecycle, which guarantees rolling updates, self-healing, and automatic scaling.
• GCP Security Integration: By combining Vitess and Kubernetes with GCP’s security features, the overall security posture is strengthened.

When Vitess is implemented on Kubernetes, additional security considerations are introduced. Here’s how to strengthen your setup for Vitess:

• Vitess Authentication: To manage access to the Vitess administration panel and database instances, use Vitess authentication mechanisms like username/password or third-party authentication services like Google Cloud IAM.
• Vitess Authorization: Use Vitess Access Control Lists (ACLs) to provide users and apps with specific permissions. These ACLs regulate the level of access (read, write, etc.) and which users have access to particular keyspaces (Vitess terminology for database schemas).
• Enable TLS encryption so that clients and Vitess instances, as well as between Vitess components (vtctld, vttablet), can communicate with one another. This prevents eavesdropping on data while it is in transit.

vtctld:
  # ... other configurations
  authentication:
    # Replace with your chosen authentication method configuration (e.g., username/password)
    # Here's an example using username/password
    static:
      users:
        - name: myuser
          password: "myStrongPassword"

Similarly, Vitess can configure tailor made ACLs, for example to configure read-only access on a Kubernetes environment

We can define the ACL policy as

name: readonly_acl

# Specify the keyspace (schema) this ACL applies to
keyspace: my_keyspace

# Define user or group (replace with your actual user/group)
user: readonly_user

# Grant read-only access (no write or admin privileges)
rules:
  - path: /my_keyspace/ - rw:r

Now on the alongside Vitess instance run the applyACL command to finalize changes

vtctl ApplyACL -server <vtctld_connection_string> <path_to_acl_file.yaml>

Now put the ACL policy YAML in a file (such as ConfigMap or Secret) that the vttablet pods can access. In order to load the ACL policy file during pod initialization, modify the vttablet startup script.

The entire keyspace indicated in the path is accessible to read-only users with this ACL. By identifying particular tables or even particular columns within tables, you can further fine-tune the access.

Resources

I know I might have been really fast with the explanations here but these are some of the resources you can access to know more:

Vitess

Vitess | Documentation

Connect with me??

Imran Roshan