1. Introduction:
Kubernetes has become the de facto platform for container orchestration in modern cloud-native environments. However, managing and securing sensitive information like passwords, API keys, and tokens within Kubernetes can be challenging. HashiCorp Vault is a powerful tool designed to manage secrets and protect sensitive data across distributed systems. In this tutorial, we’ll explore different ways to securely manage and access spring vault kubernetes secrets in Spring Boot applications using HashiCorp Vault integration.
2. Quick Recap: Kubernetes Secrets with Vault
HashiCorp Vault is a secrets management solution that provides a secure way to store and access secrets. It offers various authentication methods and access control mechanisms to ensure secrets are accessed only by authorized entities.
3. Providing Secrets to Spring Applications
Integrating HashiCorp Vault with a Spring Boot application running in Kubernetes allows for secure retrieval and management of sensitive configuration data such as database credentials, API keys, and other secrets. In this example, we’ll demonstrate how to use Spring Boot to fetch secrets from Vault using the Kubernetes authentication method.
Prerequisites
Before proceeding, ensure you have the following:
- HashiCorp Vault deployed and accessible from your Kubernetes cluster.
- A Kubernetes Service Account configured with appropriate permissions to authenticate with Vault.
- A Spring Boot application with dependencies for Vault integration.
4. Vault Spring Boot Authentication Kubernetes Setup
Vault can authenticate Kubernetes pods using Service Accounts, allowing pods to request secrets from Vault based on their identity and policies.
4.1. Example: Setting up Kubernetes Authentication in Vault
4.1.1. Enable the Kubernetes authentication method in Vault:
vault auth enable kubernetes
4.1.2. Configure Vault to trust the Kubernetes service account:
vault write auth/kubernetes/config \
token_reviewer_jwt="$(cat /var/run/secrets/kubernetes.io/serviceaccount/token)" \
kubernetes_host="https://kubernetes.default.svc.cluster.local" \
kubernetes_ca_cert=@/var/run/secrets/kubernetes.io/serviceaccount/ca.crt
4.1.3. Create a Vault policy to grant access to the secrets:
vault policy write myapp-policy - <<EOF
path "secret/data/myapp/*" {
capabilities = ["read"]
}
EOF
4.1.4. Map the Kubernetes role to the Vault policy:
vault write auth/kubernetes/role/myapp-role \
bound_service_account_names=myapp-sa \
bound_service_account_namespaces=default \
policies=myapp-policy \
ttl=24h
5. Vault Kubernetes Secrets Explicit Retrieval
With explicit retrieval, applications directly interact with Vault to retrieve secrets by making API calls. This method gives applications fine-grained control over secret access.
5.1. Example: Spring Boot Integration with Vault
Ensure you have the following dependencies in your Spring Boot pom.xml for integrating with Vault:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-vault-config</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
5.1.1. Configure Spring Boot to Retrieve Secrets from Vault
Add configuration in bootstrap.properties or bootstrap.yml to connect Spring Boot to Vault:
spring.cloud.vault:
host: <vault-host>
port: <vault-port>
scheme: https
authentication: KUBERNETES
kubernetes:
role: myapp-role
5.1.2. Access Secrets from Spring Boot Application
Now, you can access the secrets (myapp.username and myapp.password) in your Spring Boot application using the @Value annotation:
@RestController
public class SecretController {
@Value("${myapp.username}")
private String username;
@Value("${myapp.password}")
private String password;
@GetMapping("/secret")
public String getSecret() {
return "Username: " + username + ", Password: " + password;
}
}
When the Spring Boot application starts in Kubernetes, it will authenticate with Vault using the Kubernetes authentication method and retrieve the secrets (username and password) defined in Vault’s secret/data/myapp path.
Integrating Spring Boot with HashiCorp Vault in Kubernetes allows for secure management and retrieval of secrets. By leveraging Vault’s Kubernetes authentication method and Spring Boot’s configuration capabilities, developers can easily access sensitive data without hardcoding them into application properties. This approach enhances security and facilitates the management of secrets in cloud-native environments.
6. Vault Kubernetes Secrets Semi-Explicit Retrieval
Semi-explicit retrieval involves applications making requests to a sidecar container that manages interaction with Vault. The sidecar retrieves secrets from Vault and injects them into the application’s environment.
Example (Kubernetes YAML with Vault Agent sidecar):
apiVersion: v1
kind: Pod
metadata:
name: myapp
spec:
containers:
- name: myapp
image: myapp:latest
# Application container
- name: vault-agent
image: vault:latest
# Vault Agent sidecar container
env:
- name: VAULT_ADDR
value: "https://vault.example.com"
- name: VAULT_TOKEN
value: "my_vault_token"
volumeMounts:
- name: vault-secret
mountPath: "/vault/secrets"
volumes:
- name: vault-secret
emptyDir: {}
7. Transparent Support Using Vault Sidecar
Using a Vault sidecar container provides a seamless integration where applications don’t directly interact with Vault. The sidecar handles secret retrieval and injection into the application’s runtime environment.
To demonstrate “Transparent Support Using Vault Sidecar” in Kubernetes, we’ll set up a simple example where a Vault sidecar container retrieves secrets from Vault and injects them into the main application’s environment variables.
7.1. Example: Kubernetes Deployment with Vault Sidecar
Let’s create a Kubernetes Deployment where our main application container and a Vault sidecar container work together to securely fetch and inject secrets.
7.1.1. Create a Vault policy and role
First, create a policy in Vault that grants access to the secrets needed by your application. For example, let’s assume we have a policy named myapp-policy:
# myapp-policy.hcl
path "secret/data/myapp/*" {
capabilities = ["read"]
}
Next, create a role (myapp-role) and map it to this policy:
vault write auth/kubernetes/role/myapp-role \
bound_service_account_names=myapp-sa \
bound_service_account_namespaces=default \
policies=myapp-policy \
ttl=24h
7.1.2. Configure Kubernetes Deployment
Create a Kubernetes Deployment YAML file (myapp-deployment.yaml) that includes both your main application container and the Vault sidecar container.
apiVersion: apps/v1
kind: Deployment
metadata:
name: myapp
spec:
replicas: 1
selector:
matchLabels:
app: myapp
template:
metadata:
labels:
app: myapp
spec:
serviceAccountName: myapp-sa
containers:
- name: myapp
image: myapp:latest
env:
# Application-specific environment variables
- name: DB_USERNAME
valueFrom:
secretKeyRef:
name: myapp-secrets
key: DB_USERNAME
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: myapp-secrets
key: DB_PASSWORD
- name: vault-agent
image: vault:latest
env:
- name: VAULT_ADDR
value: "https://vault.example.com"
- name: VAULT_ROLE
value: "myapp-role"
# Additional configuration for Vault sidecar
In this YAML:
myappcontainer is our main application.vault-agentcontainer is the Vault sidecar.myapp-sais the Kubernetes service account associated with the Vault role (myapp-role).
7.1.3. Configure Vault Sidecar Container
The Vault sidecar container (vault-agent) will use the myapp-sa service account’s token to authenticate with Vault and retrieve secrets.
You can configure the Vault agent container to perform the following actions:
- Authenticate with Vault using the Kubernetes authentication method.
- Retrieve secrets from Vault and inject them into the environment of the
myappcontainer.
Here’s an example of how the Vault sidecar container configuration might look:
- name: vault-agent
image: vault:latest
env:
- name: VAULT_ADDR
value: "https://vault.example.com"
- name: VAULT_ROLE
value: "myapp-role"
- name: VAULT_SA_TOKEN
value: "/var/run/secrets/kubernetes.io/serviceaccount/token"
volumeMounts:
- name: vault-token
mountPath: "/var/run/secrets/kubernetes.io/serviceaccount"
readOnly: true
volumes:
- name: vault-token
projected:
sources:
- serviceAccountToken:
path: "vault-token"
expirationSeconds: 7200
7.1.4. Apply the Deployment
Apply the Kubernetes Deployment YAML to create the deployment:
kubectl apply -f myapp-deployment.yaml
This deployment will start a pod running your myapp container alongside the Vault sidecar. The Vault sidecar will handle secret retrieval from Vault and inject them into the environment variables of your myapp container securely and transparently.
Using a Vault sidecar container in Kubernetes provides a seamless way to integrate secrets management with your applications. The sidecar handles the complexities of secret retrieval and injection, allowing your main application container to focus solely on its core functionality. This approach enhances security by ensuring that secrets are never exposed directly to the application and are managed centrally by Vault.
8. Transparent Support Using Vault Secret CSI Provider
In Kubernetes, the Container Storage Interface (CSI) enables the integration of external volume plugins with the kubelet without having to modify the Kubernetes codebase. The Vault Secret CSI Provider extends this concept to securely mount Vault secrets directly into pods as volumes, providing a seamless and transparent way to inject secrets into applications running in Kubernetes.
8.1. Example (Kubernetes CSI driver for Vault):
To use the Vault Secret CSI Provider, you’ll need to perform the following steps:
8.1.1. Configure Vault Secrets Engine
First, configure Vault with a secrets engine (e.g., Key-Value Version 2) that stores the secrets you want to access.
vault secrets enable -path=secret kv-v2
vault kv put secret/myapp username=admin password=secretpassword
8.1.2. Deploy Vault CSI Provider
Deploy the Vault CSI Provider in your Kubernetes cluster. This component acts as an intermediary between Kubernetes and Vault, facilitating the retrieval and injection of secrets into pods.
# vault-csi-provider.yaml
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: vault-csi
provisioner: vault.csi.k8s.io
parameters:
vaultAddress: "https://vault.example.com"
vaultToken: "my_vault_token"
Apply this configuration using:
kubectl apply -f vault-csi-provider.yaml
8.1.3. Mount Vault Secrets in Pod
Now, you can define a Pod specification that mounts Vault secrets as volumes using the CSI driver.
apiVersion: v1
kind: Pod
metadata:
name: myapp
spec:
containers:
- name: myapp
image: myapp:latest
env:
- name: DB_USERNAME
valueFrom:
secretKeyRef:
name: myapp-secrets
key: username
- name: DB_PASSWORD
valueFrom:
secretKeyRef:
name: myapp-secrets
key: password
volumes:
- name: myapp-secrets
csi:
driver: vault.csi.k8s.io
readOnly: true
volumeAttributes:
path: "secret/data/myapp"
In this example:
- We define a Pod named
myappwith a container runningmyapp:latest. - We specify environment variables
DB_USERNAMEandDB_PASSWORDsourced from the Vault secrets mounted as a volume (myapp-secrets). - The
vault.csi.k8s.ioCSI driver is used to mount the Vault secrets at the specified path (secret/data/myapp).
Benefits of Vault Secret CSI Provider
- Centralized Secrets Management: Vault remains the central authority for secret management, ensuring security and policy enforcement.
- Dynamic Secrets: The CSI Provider can dynamically fetch and rotate secrets from Vault, providing fresh credentials to applications without manual intervention.
- Transparent Integration: Applications consume secrets as if they were mounted volumes, abstracting away the complexities of secret retrieval and injection.
- Access Control: Vault policies govern which pods and containers can access specific secrets, enhancing security and compliance.
The Vault Secret CSI Provider enables Kubernetes workloads to securely access Vault secrets directly as volumes, simplifying secrets management and enhancing application security. By leveraging the Kubernetes CSI framework, Vault seamlessly integrates with Kubernetes environments, providing a robust solution for managing and injecting secrets into containerized applications. This approach aligns with best practices for securing sensitive data within cloud-native architectures.
9. Transparent Support Using Vault Secrets Operator
The Vault Secrets Operator is a Kubernetes operator designed to automate the management of Vault secrets within Kubernetes environments. It simplifies the process of provisioning, updating, and deleting secrets in Vault, ensuring that secrets are securely synchronized and made available to applications running in Kubernetes.
Features of Vault Secrets Operator
- Custom Resource Definitions (CRDs): The Vault Secrets Operator introduces custom resource definitions (CRDs) that represent Vault resources within Kubernetes. These CRDs allow you to declare and manage secrets using Kubernetes-native YAML manifests.
- Automated Secret Lifecycle Management: The operator automates the lifecycle management of secrets in Vault based on the defined CRDs. It handles the creation, renewal, and deletion of secrets according to specified policies and configurations.
- Synchronization with Vault: The operator continuously synchronizes the state of secrets defined in Kubernetes CRDs with the corresponding secrets stored in Vault. This ensures that secrets are always up-to-date and consistent across the Kubernetes cluster.
9.1. Example: Setting up Vault Kubernetes Secrets Operator
To use the Vault Secrets Operator, follow these steps:
9.1.1. Deploy Vault Kubernetes Secrets Operator
Deploy the Vault Secrets Operator in your Kubernetes cluster. This operator watches for custom resources related to secrets and interacts with Vault to manage them accordingly.
# vault-secrets-operator.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: vault-secrets-operator
spec:
replicas: 1
selector:
matchLabels:
app: vault-secrets-operator
template:
metadata:
labels:
app: vault-secrets-operator
spec:
containers:
- name: vault-secrets-operator
image: vault-secrets-operator:latest
Apply this deployment using:
kubectl apply -f vault-secrets-operator.yaml
9.1.2. Define VaultSecret CRDs
Create custom resource definitions (CRDs) that represent the secrets you want to manage in Vault.
# myapp-secret.yaml
apiVersion: vault.banzaicloud.com/v1alpha1
kind: VaultSecret
metadata:
name: myapp-secrets
spec:
path: secret/data/myapp
type: kv-v2
data:
username: admin
password: secretpassword
Apply this CRD using:
kubectl apply -f myapp-secret.yaml
9.1.3. Watch the Operator in Action
The Vault Secrets Operator will reconcile the VaultSecret CRD and automatically create or update the specified secrets in Vault (secret/data/myapp).
Benefits of Vault Secrets Operator
- Declarative Secrets Management: Define secrets as Kubernetes resources using YAML manifests, making it easy to version control and manage through GitOps workflows.
- Automated Lifecycle: The operator automates the lifecycle of secrets, reducing the need for manual intervention and ensuring secrets are consistently managed across environments.
- Integration with RBAC: Leverages Kubernetes RBAC (Role-Based Access Control) for fine-grained control over who can manage and access secrets within the cluster.
- Auditability and Compliance: Changes to secrets are logged and auditable through Kubernetes events, providing visibility into secret management activities.
The Vault Secrets Operator simplifies the integration of Vault with Kubernetes by automating the management of secrets using Kubernetes-native resources. By leveraging custom resource definitions and automated reconciliation, organizations can streamline the provisioning and lifecycle management of secrets in a secure and auditable manner. This approach aligns with best practices for managing secrets within cloud-native environments and enhances the overall security posture of Kubernetes workloads.
10. Vault Kubernetes Secrets Method Comparison
Each approach has its trade-offs. Explicit retrieval offers direct control but requires more development effort. Semi-explicit methods like sidecars and CSI providers simplify integration but introduce additional complexity into the deployment.
11. Conclusion
Integrating HashiCorp Vault with Kubernetes provides robust security for managing secrets within containerized applications. By leveraging Vault’s capabilities, organizations can ensure secrets are stored securely, accessed only by authorized applications, and automatically rotated as per policy. Choosing the right integration method depends on the specific security and operational requirements of the Kubernetes environment. Code examples illustrate how different methods can be implemented to securely retrieve and manage secrets from Vault within Kubernetes deployments.
