Your Kubernetes Secrets Are Basically Public Without a KMS

Key Management Service (KMS) is a way to manage your secrets in a more secure manner. But before diving into KMS, let’s do a quick primer on Kubernetes Secrets.

What Are Kubernetes Secrets?

A Secret is any sensitive information—such as a database password, an API token, or cloud credentials. In most applications, you separate such configuration data from the actual application logic.

That’s where Kubernetes Secrets come in. You can store confidential configuration as a separate Kubernetes resource called a Secret.

Where Are Secrets Stored?

Kubernetes stores secrets in etcd, which is the key-value store used by the Kubernetes control plane. Unless encryption at rest is enabled, the secrets are stored in plaintext in etcd—more on this later.

How to Create Kubernetes Secrets?

Let’s walk through three ways to create secrets in Kubernetes. For reference, I’m using an OpenShift cluster with the oc CLI, but everything here applies to kubectl as well.

Create a Namespace

% oc new-project kms

a. Creating a Secret from a File

% echo -n 'somepassword' > password.txt
% oc create secret generic kms-file-secret \
  --from-file=password=password.txt

In the first line, we create a file password.txt with the contents somepassword. The second command creates a secret named kms-file-secret by reading the contents of that file.

To inspect the created secret:

% oc get secret kms-file-secret -o yaml

You’ll see:

% oc get secret kms-file-secret  -o yaml
apiVersion: v1
data:
  password: c29tZXBhc3N3b3Jk
kind: Secret
metadata:
  creationTimestamp: "2025-05-16T13:35:30Z"
  name: kms-file-secret
  namespace: kms
  resourceVersion: "807159"
  uid: 66691c51-4a6c-4a15-a6b1-1d2de6d8fff7
type: Opaque

That string c29tZXBhc3N3b3Jk is just a base64 encoded version of somepassword.

b. Creating a Secret from a Literal Value

% oc create secret generic kms-literal-secret \
  --from-literal=password=somepassword

This creates a secret by passing the value directly as a literal string. The result is the same base64 encoded password in the data field:

% oc get secret kms-literal-secret -o yaml 
apiVersion: v1
data:
  password: c29tZXBhc3N3b3Jk
kind: Secret
metadata:
  creationTimestamp: "2025-05-16T14:05:34Z"
  name: kms-literal-secret
  namespace: kms
  resourceVersion: "819400"
  uid: d4ecd109-cd95-4afe-b2e8-77b2da3bcfca
type: Opaque

c. Creating a Secret Using a Manifest File

You can also define secrets in YAML files using either the data or stringData fields.

stringData: human-readable strings (Kubernetes will encode them to base64) data: requires values to already be base64 encoded

Let’s go with the data field here.

% echo -n 'somepassword' | base64
c29tZXBhc3N3b3Jk

Now use that in a manifest:

% oc apply -f - <

Verifying:

% oc get secret kms-manifest-secret -o yaml    
apiVersion: v1
data:
  password: c29tZXBhc3N3b3Jk
kind: Secret
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"v1","data":{"password":"c29tZXBhc3N3b3Jk"},"kind":"Secret","metadata":{"annotations":{},"name":"kms-manifest-secret","namespace":"kms"},"type":"Opaque"}
  creationTimestamp: "2025-05-16T14:24:58Z"
  name: kms-manifest-secret
  namespace: kms
  resourceVersion: "827156"
  uid: ebb157dd-c493-47af-94c4-ba5d96ff9a29
type: Opaque

Why Use Kubernetes Secrets?

• Security: Better than storing sensitive info in ConfigMaps (which are plaintext).
• Isolation: Keeps secrets separate from your application code.
• Portability: Easily consumed by multiple Pods using environment variables or mounted volumes.

But… What’s the Problem?

At first glance, the secret looks like it’s encrypted due to its unreadable format:

data:
  password: c29tZXBhc3N3b3Jk

But that’s just base64 encoding—it’s easily reversible:

% oc get secret kms-file-secret -o jsonpath='{.data.password}' | base64 --decode
somepassword

So, if RBAC policies are misconfigured or someone has unauthorized read access to secrets, they can easily extract sensitive data. Not good.

So, What’s the Solution?

To make secrets more secure, Kubernetes offers multiple enhancements:

• Encryption at rest: Secrets are encrypted before being stored in etcd.
• KMS providers: Use cloud-based or external Key Management Service to encrypt secrets (e.g., AWS KMS, Azure Key Vault, HashiCorp Vault).
• Secrets Store CSI Driver: Mount secrets from external providers directly into Pods.

We’ll dive into these solutions in Part 2 of the series.