Kubernetes Configuration Management: Mastering ConfigMaps and Secrets

Introduction

Hardcoding configuration values and sensitive data directly into your application code is a recipe for disaster. It makes your applications inflexible, insecure, and difficult to manage across different environments. Kubernetes provides two powerful objects to solve this problem: ConfigMaps for configuration data and Secrets for sensitive information.

In this comprehensive guide, I’ll show you how to effectively manage application configuration and secrets in Kubernetes, following security best practices and production patterns.

Why Separate Configuration from Code?

The Problem:

# Bad Practice - Hardcoded values
containers:
- name: app
  image: myapp:1.0
  env:
  - name: DATABASE_URL
    value: "postgres://admin:password123@db.prod.com:5432/mydb"  # Exposed!
  - name: API_KEY
    value: "sk-1234567890abcdef"  # Security risk!

The Solution:

• ConfigMaps: Non-sensitive configuration (URLs, feature flags, settings)
• Secrets: Sensitive data (passwords, API keys, certificates)

Benefits:

• Security: Secrets are base64 encoded and can be encrypted at rest
• Flexibility: Change configuration without rebuilding images
• Environment separation: Different configs for dev/staging/prod
• Reusability: Share configuration across multiple Pods
• Version control: Track configuration changes

Prerequisites

• kubectl installed and configured
• Access to a Kubernetes cluster
• Understanding of Pods and Deployments
• Basic knowledge of environment variables

Part 1: ConfigMaps

ConfigMaps store non-confidential configuration data as key-value pairs.

Creating ConfigMaps

Method 1: Declarative (YAML)

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
  namespace: default
data:
  # Simple key-value pairs
  database_url: "db.example.com"
  database_name: "myapp"
  log_level: "info"
  
  # Multi-line configuration
  app.properties: |
    color=blue
    padding=25px
    theme=dark
    
  # JSON configuration
  config.json: |
    {
      "api": {
        "timeout": 30,
        "retries": 3
      }
    }

Create:

kubectl apply -f configmap.yaml

Method 2: Imperative (Command Line)

From literal values:

kubectl create configmap app-config \
  --from-literal=database_url=db.example.com \
  --from-literal=database_name=myapp \
  --from-literal=log_level=info

From files:

# Create config file
echo "color=blue" > app.properties
echo "theme=dark" >> app.properties

kubectl create configmap app-config --from-file=app.properties

From directory:

kubectl create configmap app-config --from-file=./config-dir/

From environment file:

# Create .env file
cat <<EOF > app.env
DATABASE_URL=db.example.com
LOG_LEVEL=info
CACHE_ENABLED=true
EOF

kubectl create configmap app-config --from-env-file=app.env

Using ConfigMaps in Pods

Method 1: Environment Variables (Individual Keys)

apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
  - name: app
    image: nginx:alpine
    env:
    - name: DATABASE_URL
      valueFrom:
        configMapKeyRef:
          name: app-config
          key: database_url
    - name: LOG_LEVEL
      valueFrom:
        configMapKeyRef:
          name: app-config
          key: log_level

Method 2: Environment Variables (All Keys)

apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
  - name: app
    image: nginx:alpine
    envFrom:
    - configMapRef:
        name: app-config

Method 3: Volume Mounts

apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
  - name: app
    image: nginx:alpine
    volumeMounts:
    - name: config-volume
      mountPath: /etc/config
      readOnly: true
  volumes:
  - name: config-volume
    configMap:
      name: app-config

Access files in the Pod:

kubectl exec -it app-pod -- sh
ls /etc/config/
cat /etc/config/database_url
cat /etc/config/app.properties

Method 4: Specific Keys as Files

apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
  - name: app
    image: nginx:alpine
    volumeMounts:
    - name: config-volume
      mountPath: /etc/config
  volumes:
  - name: config-volume
    configMap:
      name: app-config
      items:
      - key: app.properties
        path: application.properties
      - key: config.json
        path: config.json

ConfigMap Management

View ConfigMaps:

kubectl get configmaps
kubectl get cm  # Short form
kubectl describe configmap app-config
kubectl get configmap app-config -o yaml

Edit ConfigMap:

kubectl edit configmap app-config

Delete ConfigMap:

kubectl delete configmap app-config

Important: Pods don’t automatically reload when ConfigMaps change. You need to:

• Restart Pods manually
• Use a sidecar container to watch for changes
• Implement application-level config reloading

Part 2: Secrets

Secrets store sensitive information like passwords, tokens, and keys.

Secret Types

Kubernetes supports several Secret types:

1. Opaque (default): Arbitrary user-defined data
2. kubernetes.io/service-account-token: Service account token
3. kubernetes.io/dockercfg: Docker registry credentials
4. kubernetes.io/tls: TLS certificate and key
5. kubernetes.io/ssh-auth: SSH authentication
6. kubernetes.io/basic-auth: Basic authentication

Creating Secrets

Method 1: Declarative (YAML)

apiVersion: v1
kind: Secret
metadata:
  name: app-secrets
type: Opaque
stringData:  # Use stringData for plain text (auto base64 encoded)
  database_password: "MyS3cr3tP@ssw0rd"
  api_key: "sk-1234567890abcdef"
  
# Or use data with base64 encoded values
data:
  username: YWRtaW4=  # base64 encoded "admin"

Create:

kubectl apply -f secret.yaml

Method 2: Imperative (Command Line)

From literal values:

kubectl create secret generic app-secrets \
  --from-literal=database_password='MyS3cr3tP@ssw0rd' \
  --from-literal=api_key='sk-1234567890abcdef'

From files (recommended for security):

# Create secret files
echo -n 'MyS3cr3tP@ssw0rd' > password.txt
echo -n 'sk-1234567890abcdef' > api-key.txt

kubectl create secret generic app-secrets \
  --from-file=database_password=password.txt \
  --from-file=api_key=api-key.txt

# Clean up files
rm password.txt api-key.txt

From JSON file:

cat <<EOF > credentials.json
{
  "apiKey": "sk-1234567890abcdef",
  "clientSecret": "secret123"
}
EOF

kubectl create secret generic app-secrets --from-file=credentials.json
rm credentials.json

TLS Secret:

kubectl create secret tls tls-secret \
  --cert=path/to/cert.crt \
  --key=path/to/key.key

Docker Registry Secret:

kubectl create secret docker-registry regcred \
  --docker-server=docker.io \
  --docker-username=myuser \
  --docker-password=mypassword \
  --docker-email=myemail@example.com

Using Secrets in Pods

Method 1: Environment Variables (Individual Keys)

apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
  - name: app
    image: nginx:alpine
    env:
    - name: DB_PASSWORD
      valueFrom:
        secretKeyRef:
          name: app-secrets
          key: database_password
    - name: API_KEY
      valueFrom:
        secretKeyRef:
          name: app-secrets
          key: api_key

Method 2: Environment Variables (All Keys)

apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
  - name: app
    image: nginx:alpine
    envFrom:
    - secretRef:
        name: app-secrets

Method 3: Volume Mounts (Recommended)

apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
  - name: app
    image: nginx:alpine
    volumeMounts:
    - name: secret-volume
      mountPath: /etc/secrets
      readOnly: true
  volumes:
  - name: secret-volume
    secret:
      secretName: app-secrets

Access secrets in the Pod:

kubectl exec -it app-pod -- sh
ls /etc/secrets/
cat /etc/secrets/database_password
cat /etc/secrets/api_key

Method 4: Specific Keys with Custom Paths

apiVersion: v1
kind: Pod
metadata:
  name: app-pod
spec:
  containers:
  - name: app
    image: nginx:alpine
    volumeMounts:
    - name: secret-volume
      mountPath: /etc/secrets
  volumes:
  - name: secret-volume
    secret:
      secretName: app-secrets
      items:
      - key: database_password
        path: db-password
        mode: 0400  # Read-only for owner

Using Docker Registry Secrets

apiVersion: v1
kind: Pod
metadata:
  name: private-app
spec:
  containers:
  - name: app
    image: myregistry.com/myapp:1.0
  imagePullSecrets:
  - name: regcred

Secret Management

View Secrets:

kubectl get secrets
kubectl describe secret app-secrets
kubectl get secret app-secrets -o yaml

Decode Secret values:

kubectl get secret app-secrets -o jsonpath='{.data.database_password}' | base64 --decode

Edit Secret:

kubectl edit secret app-secrets

Delete Secret:

kubectl delete secret app-secrets

Complete Example: Multi-Tier Application

Let’s create a complete application with both ConfigMaps and Secrets:

configmap.yaml:

apiVersion: v1
kind: ConfigMap
metadata:
  name: web-app-config
  namespace: production
data:
  # Application settings
  app_name: "MyWebApp"
  environment: "production"
  log_level: "info"
  cache_ttl: "3600"
  
  # Feature flags
  feature_flags: |
    new_ui=true
    beta_features=false
    analytics=true
  
  # Nginx configuration
  nginx.conf: |
    server {
      listen 80;
      server_name example.com;
      location / {
        proxy_pass http://backend:8080;
      }
    }

secret.yaml:

apiVersion: v1
kind: Secret
metadata:
  name: web-app-secrets
  namespace: production
type: Opaque
stringData:
  database_url: "postgres://db.prod.example.com:5432/myapp"
  database_user: "app_user"
  database_password: "Str0ngP@ssw0rd!"
  jwt_secret: "your-256-bit-secret"
  api_key: "sk-prod-1234567890abcdef"

deployment.yaml:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web-app
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web-app
  template:
    metadata:
      labels:
        app: web-app
    spec:
      containers:
      - name: app
        image: myapp:1.0
        ports:
        - containerPort: 8080
        
        # Environment variables from ConfigMap
        env:
        - name: APP_NAME
          valueFrom:
            configMapKeyRef:
              name: web-app-config
              key: app_name
        - name: ENVIRONMENT
          valueFrom:
            configMapKeyRef:
              name: web-app-config
              key: environment
        - name: LOG_LEVEL
          valueFrom:
            configMapKeyRef:
              name: web-app-config
              key: log_level
        
        # Environment variables from Secret
        - name: DATABASE_URL
          valueFrom:
            secretKeyRef:
              name: web-app-secrets
              key: database_url
        - name: DATABASE_USER
          valueFrom:
            secretKeyRef:
              name: web-app-secrets
              key: database_user
        - name: DATABASE_PASSWORD
          valueFrom:
            secretKeyRef:
              name: web-app-secrets
              key: database_password
        - name: JWT_SECRET
          valueFrom:
            secretKeyRef:
              name: web-app-secrets
              key: jwt_secret
        
        # Mount ConfigMap as volume
        volumeMounts:
        - name: config-volume
          mountPath: /etc/config
          readOnly: true
        - name: secret-volume
          mountPath: /etc/secrets
          readOnly: true
        
        resources:
          requests:
            memory: "128Mi"
            cpu: "100m"
          limits:
            memory: "256Mi"
            cpu: "200m"
      
      volumes:
      - name: config-volume
        configMap:
          name: web-app-config
      - name: secret-volume
        secret:
          secretName: web-app-secrets
          defaultMode: 0400

Deploy:

kubectl create namespace production
kubectl apply -f configmap.yaml
kubectl apply -f secret.yaml
kubectl apply -f deployment.yaml

Security Best Practices

1. Never Commit Secrets to Git

# Add to .gitignore
echo "*.secret.yaml" >> .gitignore
echo "secrets/" >> .gitignore

2. Use RBAC to Restrict Access

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: secret-reader
rules:
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get", "list"]
  resourceNames: ["app-secrets"]  # Specific secret only

3. Enable Encryption at Rest

# EncryptionConfiguration
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources:
  - secrets
  providers:
  - aescbc:
      keys:
      - name: key1
        secret: <base64-encoded-secret>
  - identity: {}

4. Use External Secret Management

• AWS Secrets Manager
• Azure Key Vault
• HashiCorp Vault
• Google Secret Manager

Example with External Secrets Operator:

apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: app-secrets
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secrets-manager
    kind: SecretStore
  target:
    name: app-secrets
  data:
  - secretKey: database_password
    remoteRef:
      key: prod/database/password

5. Rotate Secrets Regularly

# Update secret
kubectl create secret generic app-secrets \
  --from-literal=api_key='new-key' \
  --dry-run=client -o yaml | kubectl apply -f -

# Restart pods to pick up new secret
kubectl rollout restart deployment web-app

6. Use Immutable ConfigMaps/Secrets

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
immutable: true  # Cannot be updated
data:
  key: value

7. Limit Secret Scope

# Use namespaces to isolate secrets
kubectl create namespace dev
kubectl create namespace prod

# Secrets in dev namespace can't be accessed from prod

Troubleshooting

Issue 1: Pod Can’t Access ConfigMap/Secret

Check if exists:

kubectl get configmap app-config
kubectl get secret app-secrets

Check namespace:

kubectl get configmap app-config -n production

Verify Pod reference:

kubectl describe pod app-pod
# Look for events related to ConfigMap/Secret mounting

Issue 2: Secret Values Not Decoded

Decode manually:

kubectl get secret app-secrets -o jsonpath='{.data.password}' | base64 --decode

Issue 3: ConfigMap Changes Not Reflected

Restart Pods:

kubectl rollout restart deployment web-app

Or use annotation to force update:

kubectl patch deployment web-app \
  -p '{"spec":{"template":{"metadata":{"annotations":{"configmap-version":"'$(date +%s)'"}}}}}'

ConfigMap vs Secret Comparison

Feature	ConfigMap	Secret
Purpose	Non-sensitive config	Sensitive data
Encoding	Plain text	Base64 encoded
Encryption	No	Optional (at rest)
Size Limit	1MB	1MB
Use Cases	URLs, settings, flags	Passwords, keys, certs
Visibility	Visible in YAML	Hidden in kubectl describe

Advanced Patterns

1. ConfigMap/Secret Versioning

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config-v2  # Version in name
data:
  version: "2.0"
  setting: "value"

2. Multiple Environments

# Different configs per environment
kubectl create configmap app-config --from-file=config.dev.yaml -n dev
kubectl create configmap app-config --from-file=config.prod.yaml -n prod

3. Shared Secrets Across Namespaces

# Copy secret to another namespace
kubectl get secret app-secrets -n source -o yaml | \
  sed 's/namespace: source/namespace: target/' | \
  kubectl apply -f -

Conclusion

Proper configuration management is crucial for secure, maintainable Kubernetes applications. Key takeaways:

• Use ConfigMaps for non-sensitive configuration
• Use Secrets for sensitive data
• Never hardcode credentials in images
• Implement RBAC to restrict access
• Enable encryption at rest for Secrets
• Rotate secrets regularly
• Consider external secret management solutions
• Use volumes over environment variables for sensitive data
• Implement proper namespace isolation

Mastering ConfigMaps and Secrets enables you to build secure, flexible applications that can easily adapt to different environments without code changes.

In the next post, we’ll explore Kubernetes Storage with Persistent Volumes and Persistent Volume Claims.

Happy configuring! 🔐

Resources

• ConfigMaps Documentation
• Secrets Documentation
• Encrypting Secret Data at Rest
• External Secrets Operator