Sensitive data & secrets

Should you store secrets in environment variables?

Short answer: avoid env vars for production secrets. They enable lateral movement after RCE, leak through logs and process dumps, and cannot be audited or rotated cleanly. Put a secret ID in the environment and resolve it at startup with koanf (correct spelling) plus AWS Secrets Manager, the cloud SDK, or a vault product.

6 min read
In short: Short answer: avoid env vars for production secrets. They enable lateral movement after RCE, leak through logs and process dumps, and cannot be audited or rotated cleanly. Put a secret ID in the environment and resolve it at startup with koanf (correct spelling) plus AWS Secrets Manager, the cloud SDK, or a vault product.

Should you store secrets in environment variables?

Short answer: avoid it for production secrets. Environment variables are a fine place for non-secret config (feature flags, public URLs, log level). They are a poor vault. Once a process can run code on the box, or once a log line dumps process.env, every value in that environment is available in plaintext. That is CWE-526: Exposure of Sensitive Information Through Environmental Variables, and groups such as TeamTNT have automated it against cloud workloads.

The twelve-factor app told you to put config in the environment so the codebase could be opened without leaking credentials. That part is still right: do not commit the database URL. The part that aged badly is treating the environment as the store. The store should be a secrets manager. The environment, if you use it at all, should hold a reference.

If a secret does leak, the incident looks like secrets exfiltration. If it leaks through a log line, the fix is redacting sensitive data from logs.

Why do env vars help an attacker move laterally?

Think of the environment as a label on every process: DATABASE_URL, STRIPE_SECRET_KEY, AWS_SECRET_ACCESS_KEY. Your application might be locked down. Those strings open the next hop.

You can see the shape locally:

Terminal window
env
import os
for key, value in os.environ.items():
print(f"{key}: {value}")

Every language has the same call. Remote code execution, a debug endpoint, a template that prints config, or a malicious dependency that reads os.environ is enough. The attacker stops attacking your app and starts using your database, your object store, and your third-party APIs. The env var was the pivot, not the prize.

How do environment variables leak by accident?

console.log(process.env) during a local debug session is the usual story. The line ships. A request in production hits it. The logging vendor now has every secret that process was started with. Crash dumps, Kubernetes env: in a support bundle, and docker inspect are the same class of mistake.

Redact before the record leaves the process. That means structured logging with an allow-list of fields, not a denylist of key names you remember to hide. See the slog LogValuer pattern in redacting sensitive data from logs.

Why are env vars a bad way to manage secrets?

There is no audit trail for who read DATABASE_URL last Tuesday. There is no version history when someone overwrites it. Rotation means restarting every process that cached the old value, in every environment, without mixing staging and production. Drift is the steady state: your laptop, the preview deploy, and production disagree, and you find out during an incident.

A secrets manager exists to give you those operations: who accessed what, which version is live, and a rotation that does not require a human to edit three dashboards.

How does runtime secrets injection work?

Fetch the value when the process needs it, from a store that authenticates the workload. Three patterns show up in production.

Set the environment variable to an ID, not the secret

This is the portable option when you span more than one cloud. DATABASE_PASSWORD holds arn:aws:secretsmanager:us-east-1:123456789012:secret:payments/db-9xYz (or a GCP Secret Manager resource name). At startup you pattern-match the ID, call the provider, and cache the result in memory for the process lifetime (or until a refresh). If the value does not look like an ID, fall back to the raw string so local development and CI can still use a .env that is never deployed.

Libraries that already abstract this include koanf and Konf in Go, and similar config loaders in other languages. The tool name is koanf (not "konaf"). A worked Go example using koanf plus AWS Secrets Manager:

package main
import (
"context"
"fmt"
"log"
"os"
"strings"
"github.com/aws/aws-sdk-go-v2/aws"
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/secretsmanager"
"github.com/knadh/koanf/providers/env"
"github.com/knadh/koanf/v2"
)
func resolveSecret(ctx context.Context, raw string) (string, error) {
if !strings.HasPrefix(raw, "arn:aws:secretsmanager:") {
return raw, nil // local or CI plaintext fallback
}
awsCfg, err := config.LoadDefaultConfig(ctx)
if err != nil {
return "", fmt.Errorf("load aws config: %w", err)
}
client := secretsmanager.NewFromConfig(awsCfg)
out, err := client.GetSecretValue(ctx, &secretsmanager.GetSecretValueInput{
SecretId: aws.String(raw),
})
if err != nil {
return "", fmt.Errorf("get secret: %w", err)
}
if out.SecretString == nil {
return "", fmt.Errorf("secret %s has no string value", raw)
}
return *out.SecretString, nil
}
func main() {
ctx := context.Background()
k := koanf.New(".")
err := k.Load(env.Provider("APP_", ".", func(s string) string {
return strings.ReplaceAll(strings.ToLower(strings.TrimPrefix(s, "APP_")), "_", ".")
}), nil)
if err != nil {
log.Fatal(err)
}
password, err := resolveSecret(ctx, k.String("database.password"))
if err != nil {
log.Fatal(err)
}
// Use password to open the database. Do not log it.
_ = password
fmt.Println("resolved database password from", os.Getenv("APP_DATABASE_PASSWORD"))
}

A developer laptop sets APP_DATABASE_PASSWORD=local-dev-only. Production sets APP_DATABASE_PASSWORD to the ARN. The same binary runs in both places. IAM on the task role, not a copied string, is what authorizes the fetch. The log line prints the ARN or the words "local-dev-only", never the live password.

Call the platform secrets manager directly

If you are committed to one cloud, skip the indirection and use the SDK with the secret name. Grant the task or function an IAM role that can read only that secret. You get type-safe APIs, CloudTrail on GetSecretValue, and no plaintext in DescribeTaskDefinition.

func getSecret(ctx context.Context, name string) (string, error) {
awsCfg, err := config.LoadDefaultConfig(ctx, config.WithRegion("us-east-1"))
if err != nil {
return "", err
}
client := secretsmanager.NewFromConfig(awsCfg)
result, err := client.GetSecretValue(ctx, &secretsmanager.GetSecretValueInput{
SecretId: aws.String(name),
})
if err != nil {
return "", fmt.Errorf("error fetching secret: %w", err)
}
return aws.ToString(result.SecretString), nil
}

Call this at startup or on a refresh interval. Do not call it on every request unless the SDK's cache is in front.

Use a dedicated secrets product

Doppler, HashiCorp Vault, 1Password, and Infisical sit above the clouds and sync the same name into AWS, GCP, and Vercel. You define the secret once. They handle rotation policy, audit, and per-environment versions. Use one of these when the alternative is three copies of the same value and a spreadsheet of which dashboard is source of truth.

What should you do instead of env-as-vault?

  1. Fetch production secrets at runtime from a manager, authenticated as the workload.
  2. If you must put something in the environment, put the ID, not the value.
  3. Sanitize logs. An env dump is a breach report.
  4. Keep local plaintext out of git and out of the production image.

Environment variables are convenient. Convenience is how a single RCE becomes a database dump. Put the secret behind an identity that can be revoked without restarting the universe, and keep the plaintext out of env.

Frequently asked questions

Should you store secrets in environment variables?

Avoid it for production secrets. Env vars are fine for non-secret config. For secrets, store a reference and fetch the value at runtime from a manager that authenticates the workload.

What is CWE-526?

Exposure of Sensitive Information Through Environmental Variables. If an attacker can run code or dump the process environment, they get every value in plaintext.

What is koanf?

A Go configuration library (knadh/koanf) that can load env vars and other sources. The correct name is koanf. Use it to read a secret ID, then resolve that ID against AWS Secrets Manager or another vault.

What should local development do?

Keep a plaintext fallback when the value is not a secret ID, and never deploy that plaintext. Production sets the ARN or resource name; IAM on the task role authorizes the fetch.

Application security in your code

Protect your application with Arcjet

Get rate limits, bot detection, and attack blocking in your request handlers.