Securing LLM API Keys with AWS KMS

A prompt-injection researcher points your app’s own debug endpoint at itself, triggers an unhandled exception, and your error handler helpfully dumps the full environment into the log stream. Ten minutes later a Discord bot is running GPT-4 class completions against your OPENAI_API_KEY at full tilt. You find out from a billing alert, not a security alert.

That is the uncomfortable shape of an LLM provider key sitting in process.env. It is not a password that unlocks your data. It is a metered spend faucet that a stranger can open, and the meter runs in dollars per second. Here is the fix: encrypt the key with AWS KMS, keep the plaintext out of the environment entirely, and decrypt it only at the exact moment you call the provider.

Why an LLM key is a special kind of dangerous

A database credential leaks blast radius that maps to your data. An LLM provider key leaks something worse in one respect: uncapped, immediate, attacker-controlled cost.

Three properties compound:

• It is metered spend. Every request costs money. A leaked key is not a door into your system, it is a direct line to your invoice. Attackers resell stolen keys precisely because they are fungible compute.
• It is trivial to exfiltrate and use. A single header, Authorization: Bearer sk-..., works from anywhere on the internet with no VPC, no signing, no client certificate. Paste it into curl and it works.
• The blast radius is your org, not one project. On most providers, a key inherits the billing account behind it. One leaked key from a side service can burn the budget your production traffic depends on.

If you have not modeled this yet, start with the Node.js secrets threat model. The threat here is not “someone reads my secret,” it is “someone spends my money and rate-limits my real users.”

The problem with keys in .env and process.env

process.env is a single flat, global, plaintext bag. Anything running in your process can read all of it, and so can anything that can make your process talk about itself:

• An unhandled exception whose handler logs the environment.
• An SSRF or debug route that reflects config.
• A crash reporter or APM agent that snapshots env by default.
• A dependency in your node_modules that reads process.env on import.

None of these need to target your key specifically. They dump the whole bag, and your key is in the bag. That is the core failure: the secret lives in plaintext, in a shared namespace, for the entire lifetime of the process. Committing a .env file is the crude version of this mistake. Leaving decrypted secrets in process.env at runtime is the subtle one.

Envelope encryption with KMS, decrypted on demand

The durable fix is envelope encryption. You do not ship the LLM key in plaintext anywhere. You ship ciphertext, and the ability to turn it back into plaintext is gated by IAM, not by file permissions.

The AWS KMS flow for a value larger than a bare token, or for a bundle of secrets, is:

1. Call GenerateDataKey with KeySpec: "AES_256". KMS returns a Plaintext data key and a CiphertextBlob (that same key, encrypted under your KMS key).
2. Encrypt your secret locally with the plaintext data key (AES-256-GCM).
3. Erase the plaintext data key from memory. Store only the ciphertext and the CiphertextBlob.
4. At runtime, call Decrypt on the CiphertextBlob to recover the data key, decrypt the secret, use it, and drop it.

KMS never lets the KMS key itself leave its FIPS-validated hardware, and the plaintext data key exists only for a few milliseconds inside your process. If you want the mechanics end to end, what is envelope encryption walks the whole thing.

sequenceDiagram
    participant App as App process
    participant KMS as AWS KMS
    participant Prov as LLM provider
    App->>App: read ENC[...] from .env (ciphertext only)
    App->>KMS: Decrypt(CiphertextBlob, EncryptionContext)
    KMS-->>App: plaintext key (in memory, short-lived)
    App->>Prov: POST /v1/... Authorization: Bearer <key>
    Prov-->>App: completion
    App->>App: drop plaintext, never touch process.env

The decrypted key lands in an in-memory keystore that you read from at the point of use, and it never touches process.env or disk.

A keystore that never populates process.env

This is exactly what @faizahmed/secret-keystore is built for. You keep an ordinary .env, but the sensitive values are stored as ENC[...] ciphertext, encrypted under a KMS key. Developers only ever handle a KMS Key ID, which is not a secret. There is no passphrase and no private key to leak.

Put the raw key in .env once, then encrypt it in place with the CLI:

# .env already contains OPENAI_API_KEY=sk-proj-your-real-key
npx @faizahmed/secret-keystore encrypt \
  --kms-key-id="alias/llm-secrets" \
  --keys="OPENAI_API_KEY"
# rewrites the file in place: OPENAI_API_KEY=ENC[AQICAHh...]

At runtime, config() discovers and cascades your env files (the way dotenv-flow does), decrypts the ENC[...] values via KMS, and loads everything into an in-memory SecretKeyStore. Plaintext values pass through untouched. Nothing is written to disk, and nothing lands in process.env:

import { config } from "@faizahmed/secret-keystore";

const secrets = await config({ kmsKeyId: process.env.KMS_KEY_ID });

async function complete(prompt: string) {
  // key is read at the point of use, straight from memory
  const res = await fetch("https://api.openai.com/v1/responses", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${secrets.get("OPENAI_API_KEY")}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ model: "gpt-4.1-mini", input: prompt }),
  });
  return res.json();
}

Because the secret is never in process.env, the classic leaks stop working: an env dump shows ENC[...], a crash reporter snapshots ENC[...], and a nosy dependency reads ENC[...]. On shutdown, secrets.destroy() clears the map. The full pattern, including the zero-config loader and run command for unmodified apps, is in the encrypted .env complete guide. If you need cryptographic proof that only your attested code can decrypt, the keystore supports Nitro Enclave remote attestation, so a leaked ciphertext plus stolen IAM credentials still cannot decrypt outside the enclave.

A gateway so app code never holds the raw key

The strongest version removes the key from your application process entirely. Stand up a small internal LLM gateway. It is the only workload with kms:Decrypt permission, it holds the provider key in memory, and it exposes an internal-only endpoint. Your app services call the gateway with their own short-lived internal auth, never the provider key.

// gateway service: the ONLY place that decrypts the provider key
app.post("/llm/complete", requireInternalAuth, async (req, res) => {
  const upstream = await fetch("https://api.openai.com/v1/responses", {
    method: "POST",
    headers: {
      Authorization: `Bearer ${secrets.get("OPENAI_API_KEY")}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify(req.body),
  });
  res.status(upstream.status).json(await upstream.json());
});

Now a leak in any product service exposes an internal gateway URL, not a spendable provider key. The gateway is also the natural home for per-tenant quotas, request logging, and a kill switch. If you already centralize model calls, this is a small extension. Pair it with the habits in using AI without leaking secrets.

Least privilege, rotation, and provider-side caps

Encryption is one layer. Wrap it with three cheap controls:

Scope kms:Decrypt tightly. Only the gateway role gets it, only for the one key, and pin the encryption context so a stolen blob cannot be decrypted for a different purpose:

{
  "Effect": "Allow",
  "Action": "kms:Decrypt",
  "Resource": "arn:aws:kms:us-east-1:111122223333:key/abcd-1234",
  "Condition": {
    "StringEquals": { "kms:EncryptionContext:app": "llm-gateway" }
  }
}

Rotate on a schedule and on suspicion. secret-keystore rotate decrypts with the old KMS key and re-encrypts under the new one in a single pass, so rotating the wrapping key is a one-command change. Rotate the provider key itself from the provider dashboard whenever a decrypt path changes hands.

Cap spend at the provider. OpenAI does not offer per-API-key spend limits, but it does offer project-level usage limits (a hard cap plus a soft warning threshold) with email alerts. Give each app its own scoped project key, set a limit per project, and one compromised key cannot drain the whole org. Treat this as the last line, not the first.

Takeaway

An LLM key is a spend faucet, and process.env leaves it wide open to every log dump and SSRF in your stack. Store it as KMS ciphertext, decrypt it on demand into an in-memory keystore that never touches process.env, front it with a gateway so app code never holds the raw key, and cap the blast radius with scoped IAM, rotation, and project-level limits. The key that is never in plaintext, in a place an attacker can read, is the key they cannot spend.

References

• AWS KMS Developer Guide: envelope encryption and key concepts
• AWS KMS API Reference: Decrypt
• AWS KMS API Reference: GenerateDataKey
• @faizahmed/secret-keystore on npm
• OpenAI API: production best practices (API key safety)