Credential config values preserved in mock layer
A fix in n8n's execution engine now correctly distinguishes between secret credentials (API keys, tokens) and configuration values (URLs, regions, endpoints), preserving realistic defaults for the LLM mock handler instead of masking everything behind a generic placeholder.
When n8n runs workflows that interact with external APIs, the eval mock layer generates test responses by intercepting HTTP requests. Previously, all credential properties were replaced with a generic eval-mock-value placeholder — including configuration like API endpoints and region settings. This left the LLM mock handler blind to critical routing information, producing inaccurate responses.
The fix introduces a classification system that examines each credential property to determine whether it holds a secret that must be masked or a config value whose default should be preserved. Secrets (identified by typeOptions.password, naming patterns like *key*, *token*, *secret*) receive descriptive placeholders like <api-key> or <secret-access-key>. Config properties (URLs, regions, hosts, endpoints) keep their schema defaults so the mock handler sees realistic request targets.
The classification uses four signals: explicit password markers, secret naming patterns, non-string types (booleans and options are always config), and config naming patterns. Properties that don't match any pattern default to secret — a safe fallback that prevents accidental credential exposure.
This change lives in the @n8n/core execution engine, specifically in the eval mock helpers that power AI-assisted workflow testing.
View Original GitHub Description
Summary
- Credential properties used as config (URL, region, endpoint, host) were blindly replaced with a generic placeholder, misleading the LLM mock handler into generating inaccurate responses
- Now classifies each credential property as secret vs config using schema metadata (
typeOptions.password, property type, name patterns) and preserves schema defaults for config properties - Secret properties get descriptive placeholders like
<api-key>,<secret-access-key>instead of the opaqueeval-mock-value
Related Linear ticket
https://linear.app/n8n/issue/TRUST-18
Review / Merge checklist
- Tests pass — 29 tests (6 new for
isSecretCredentialProperty, 4 updated forbuildEvalMockCredentials) - Typecheck clean
- Lint clean
Note: This is a clean reimplementation of #27908, rebased onto current master to avoid the merge conflicts from the
feature/instance-ai-evals-v2base branch.