Request secrets redacted before LLM calls
A new sanitization layer prevents API keys, tokens, and other credentials from leaking when request data is sent to language models in the eval mock handler. Oversized payloads are also truncated to avoid flooding prompts.
When the eval mock handler sends request data to an external LLM to generate mock responses, sensitive information like API keys, tokens, and authorization headers could end up in the prompt. This creates a data exposure risk if the handler is accidentally used with real user requests instead of synthetic test data.
Request bodies and query strings are now filtered before reaching the LLM. A pattern-matching system identifies secret-looking field names—tokens, passwords, authorization headers, bearer credentials—and replaces their values with a redacted placeholder. Safe fields like primaryKey or keyword are preserved since they contain structural data the LLM needs to generate accurate mocks. Large request bodies over 4KB are truncated with a warning log, since bulk data and file uploads don't improve mock generation.
The changes live in the instance-ai eval module in the CLI package, part of ongoing work to make AI-assisted evaluation features safer for production use.
View Original GitHub Description
Summary
- Adds a request sanitization layer (
eval-request-sanitizer.ts) that redacts secret-looking keys from request bodies and query strings before they are included in the LLM prompt - Redacts values whose keys match credential patterns (apiKey, token, password, Authorization header, etc.) while preserving safe keys that contain secret-ish substrings (keyword, primaryKey, author)
- Truncates oversized request bodies (>4KB) with a warning log — large bodies are usually file uploads or bulk data that don't help the LLM generate a better mock response
Related Linear ticket
https://linear.app/n8n/issue/TRUST-17
Review / Merge checklist
- Tests pass — 75 tests (isSecretKey, redactSecretKeys, truncateForLlm)
- Typecheck clean (pre-existing failures in integration tests only)
- Lint clean
Note: This is a clean reimplementation of #27909, rebased onto current master to avoid the merge conflicts from the
feature/instance-ai-evals-v2base branch.