surshar / claude-code

Claude Code is an agentic coding tool that lives in your terminal, understands your codebase, and helps you code faster by executing routine tasks, explaining complex code, and handling git workflows - all through natural language commands.

Built by Anthropic · github.com/anthropics

validate-agent.sh: don't abort at the first warning (set -e + ((x++))) and stop false-flagging valid agentsBrowse files

Two defects made the validator fail on plugin-dev's own agent files
(anthropics/claude-code#83803):

1. Under `set -e`, `((warning_count++))` / `((error_count++))` return a
   nonzero status when the counter was 0, so the script died at the first
   warning or error instead of finishing the run. Increments now use
   `count=$((count + 1))`, which always returns 0.

2. Field extractions like `TOOLS=$(... | grep '^tools:' ...)` aborted the
   script under `set -e` when the field was absent (grep exits 1 on no
   match), instead of reporting the missing field. They now end in
   `|| true`.

3. The description check only read the first physical line of the
   `description:` value, so multi-line descriptions with <example> blocks
   (as in plugin-dev's own agents) were false-flagged as missing examples.
   The extraction now captures the full multi-line value.

Adds validate-agent.test.sh: plugin-dev's own agents must exit 0, a
warning-only file must complete with exit 0, and an invalid file must
still exit 1 with all errors reported.

No-Verification-Needed: standalone shell script in the public repo; driven end-to-end directly plus new regression harness

Boris Cherny authored on Aug 25, 2026 parent 8b6ef81 · commit 0989f29b0bca

2 changed files+101 -23compared with 8b6ef81

Mplugins/plugin-dev/skills/agent-development/scripts/validate-agent.sh

Aplugins/plugin-dev/skills/agent-development/scripts/validate-agent.test.sh

Mplugins/plugin-dev/skills/agent-development/scripts/validate-agent.sh+31 -23

		@@ -56,74 +56,82 @@
56	56	warning_count=0
57	57
58	58	# Check name field
59		-NAME=$(echo "$FRONTMATTER" \| grep '^name:' \| sed 's/name: //' \| sed 's/^"$.$"$/\1/')
	59	+# The "\|\| true" on each field extraction keeps grep's no-match exit status from
	60	+# killing the script under set -e; a missing field is reported below instead.
	61	+NAME=$(echo "$FRONTMATTER" \| grep '^name:' \| sed 's/name: //' \| sed 's/^"$.$"$/\1/' \|\| true)
60	62
61	63	if [ -z "$NAME" ]; then
62	64	echo "❌ Missing required field: name"
63		- ((error_count++))
	65	+ error_count=$((error_count + 1))
64	66	else
65	67	echo "✅ name: $NAME"
66	68
67	69	# Validate name format
68	70	if ! [[ "$NAME" =~ ^[a-zA-Z0-9][a-zA-Z0-9-]*[a-zA-Z0-9]$ ]]; then
69	71	echo "❌ name must start/end with alphanumeric and contain only letters, numbers, hyphens"
70		- ((error_count++))
	72	+ error_count=$((error_count + 1))
71	73	fi
72	74
73	75	# Validate name length
74	76	name_length=${#NAME}
75	77	if [ $name_length -lt 3 ]; then
76	78	echo "❌ name too short (minimum 3 characters)"
77		- ((error_count++))
	79	+ error_count=$((error_count + 1))
78	80	elif [ $name_length -gt 50 ]; then
79	81	echo "❌ name too long (maximum 50 characters)"
80		- ((error_count++))
	82	+ error_count=$((error_count + 1))
81	83	fi
82	84
83	85	# Check for generic names
84	86	if [[ "$NAME" =~ ^(helper\|assistant\|agent\|tool)$ ]]; then
85	87	echo "⚠️ name is too generic: $NAME"
86		- ((warning_count++))
	88	+ warning_count=$((warning_count + 1))
87	89	fi
88	90	fi
89	91
90	92	# Check description field
91		-DESCRIPTION=$(echo "$FRONTMATTER" \| grep '^description:' \| sed 's/description: *//')
	93	+# The description may span multiple lines (unquoted prose plus <example> blocks),
	94	+# so capture everything from "description:" until the next top-level agent key.
	95	+DESCRIPTION=$(echo "$FRONTMATTER" \| awk '
	96	+ /^description:/ { in_description=1; sub(/^description:[[:space:]]*/, ""); print; next }
	97	+ /^(name\|model\|color\|tools):/ { in_description=0 }
	98	+ in_description { print }
	99	+')
92	100
93	101	if [ -z "$DESCRIPTION" ]; then
94	102	echo "❌ Missing required field: description"
95		- ((error_count++))
	103	+ error_count=$((error_count + 1))
96	104	else
97	105	desc_length=${#DESCRIPTION}
98	106	echo "✅ description: ${desc_length} characters"
99	107
100	108	if [ $desc_length -lt 10 ]; then
101	109	echo "⚠️ description too short (minimum 10 characters recommended)"
102		- ((warning_count++))
	110	+ warning_count=$((warning_count + 1))
103	111	elif [ $desc_length -gt 5000 ]; then
104	112	echo "⚠️ description very long (over 5000 characters)"
105		- ((warning_count++))
	113	+ warning_count=$((warning_count + 1))
106	114	fi
107	115
108	116	# Check for example blocks
109	117	if ! echo "$DESCRIPTION" \| grep -q '<example>'; then
110	118	echo "⚠️ description should include <example> blocks for triggering"
111		- ((warning_count++))
	119	+ warning_count=$((warning_count + 1))
112	120	fi
113	121
114	122	# Check for "Use this agent when" pattern
115	123	if ! echo "$DESCRIPTION" \| grep -qi 'use this agent when'; then
116	124	echo "⚠️ description should start with 'Use this agent when...'"
117		- ((warning_count++))
	125	+ warning_count=$((warning_count + 1))
118	126	fi
119	127	fi
120	128
121	129	# Check model field
122		-MODEL=$(echo "$FRONTMATTER" \| grep '^model:' \| sed 's/model: *//')
	130	+MODEL=$(echo "$FRONTMATTER" \| grep '^model:' \| sed 's/model: *//' \|\| true)
123	131
124	132	if [ -z "$MODEL" ]; then
125	133	echo "❌ Missing required field: model"
126		- ((error_count++))
	134	+ error_count=$((error_count + 1))
127	135	else
128	136	echo "✅ model: $MODEL"
129	137
		@@ -133,17 +141,17 @@
133	141	;;
134	142	*)
135	143	echo "⚠️ Unknown model: $MODEL (valid: inherit, sonnet, opus, haiku)"
136		- ((warning_count++))
	144	+ warning_count=$((warning_count + 1))
137	145	;;
138	146	esac
139	147	fi
140	148
141	149	# Check color field
142		-COLOR=$(echo "$FRONTMATTER" \| grep '^color:' \| sed 's/color: *//')
	150	+COLOR=$(echo "$FRONTMATTER" \| grep '^color:' \| sed 's/color: *//' \|\| true)
143	151
144	152	if [ -z "$COLOR" ]; then
145	153	echo "❌ Missing required field: color"
146		- ((error_count++))
	154	+ error_count=$((error_count + 1))
147	155	else
148	156	echo "✅ color: $COLOR"
149	157
		@@ -153,13 +161,13 @@
153	161	;;
154	162	*)
155	163	echo "⚠️ Unknown color: $COLOR (valid: blue, cyan, green, yellow, magenta, red)"
156		- ((warning_count++))
	164	+ warning_count=$((warning_count + 1))
157	165	;;
158	166	esac
159	167	fi
160	168
161	169	# Check tools field (optional)
162		-TOOLS=$(echo "$FRONTMATTER" \| grep '^tools:' \| sed 's/tools: *//')
	170	+TOOLS=$(echo "$FRONTMATTER" \| grep '^tools:' \| sed 's/tools: *//' \|\| true)
163	171
164	172	if [ -n "$TOOLS" ]; then
165	173	echo "✅ tools: $TOOLS"
		@@ -173,23 +181,23 @@
173	181
174	182	if [ -z "$SYSTEM_PROMPT" ]; then
175	183	echo "❌ System prompt is empty"
176		- ((error_count++))
	184	+ error_count=$((error_count + 1))
177	185	else
178	186	prompt_length=${#SYSTEM_PROMPT}
179	187	echo "✅ System prompt: $prompt_length characters"
180	188
181	189	if [ $prompt_length -lt 20 ]; then
182	190	echo "❌ System prompt too short (minimum 20 characters)"
183		- ((error_count++))
	191	+ error_count=$((error_count + 1))
184	192	elif [ $prompt_length -gt 10000 ]; then
185	193	echo "⚠️ System prompt very long (over 10,000 characters)"
186		- ((warning_count++))
	194	+ warning_count=$((warning_count + 1))
187	195	fi
188	196
189	197	# Check for second person
190	198	if ! echo "$SYSTEM_PROMPT" \| grep -q "You are\\|You will\\|Your"; then
191	199	echo "⚠️ System prompt should use second person (You are..., You will...)"
192		- ((warning_count++))
	200	+ warning_count=$((warning_count + 1))
193	201	fi
194	202
195	203	# Check for structure

Aplugins/plugin-dev/skills/agent-development/scripts/validate-agent.test.sh+70 -0

		@@ -1,0 +1,70 @@
	1	+#!/bin/bash
	2	+# Regression tests for validate-agent.sh (anthropics/claude-code#83803):
	3	+# - warnings must not abort the run: under `set -e`, `((x++))` returns nonzero
	4	+# when x was 0, so the first warning killed the script with exit 1
	5	+# - multi-line descriptions (prose plus <example> blocks) must not be
	6	+# false-flagged as missing examples
	7	+set -uo pipefail
	8	+
	9	+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
	10	+VALIDATOR="$SCRIPT_DIR/validate-agent.sh"
	11	+PLUGIN_ROOT="$(cd "$SCRIPT_DIR/../../.." && pwd)"
	12	+TMP_DIR="$(mktemp -d)" \|\| exit 1
	13	+trap 'rm -rf "$TMP_DIR"' EXIT
	14	+
	15	+failures=0
	16	+
	17	+check() {
	18	+ local label="$1" expected="$2" file="$3"
	19	+ local actual=0
	20	+ bash "$VALIDATOR" "$file" > "$TMP_DIR/out.txt" 2>&1 \|\| actual=$?
	21	+ if [ "$actual" -eq "$expected" ]; then
	22	+ echo "PASS: $label (exit $actual)"
	23	+ else
	24	+ echo "FAIL: $label (expected exit $expected, got $actual)"
	25	+ cat "$TMP_DIR/out.txt"
	26	+ failures=$((failures + 1))
	27	+ fi
	28	+}
	29	+
	30	+# The plugin's own agents are valid and must pass.
	31	+for agent in "$PLUGIN_ROOT"/agents/*.md; do
	32	+ check "own agent $(basename "$agent")" 0 "$agent"
	33	+done
	34	+
	35	+# A valid agent whose fields only trigger warnings must still exit 0,
	36	+# and must reach the summary line instead of aborting at the first warning.
	37	+cat > "$TMP_DIR/warning-agent.md" <<'EOF'
	38	+---
	39	+name: warning-agent
	40	+description: A valid description that has no example blocks and no trigger phrase
	41	+model: sonnet
	42	+color: blue
	43	+---
	44	+
	45	+You are a test agent. Your job is to exist so the validator has something to warn about.
	46	+EOF
	47	+check "valid agent with warnings" 0 "$TMP_DIR/warning-agent.md"
	48	+if ! grep -q "Validation passed" "$TMP_DIR/out.txt"; then
	49	+ echo "FAIL: summary line missing (script aborted before finishing)"
	50	+ failures=$((failures + 1))
	51	+fi
	52	+
	53	+# An invalid agent (bad name, missing color) must still fail with exit 1.
	54	+cat > "$TMP_DIR/invalid-agent.md" <<'EOF'
	55	+---
	56	+name: x
	57	+description: A valid description for an otherwise invalid agent file
	58	+model: sonnet
	59	+---
	60	+
	61	+You are a test agent with an invalid name and no color field.
	62	+EOF
	63	+check "invalid agent" 1 "$TMP_DIR/invalid-agent.md"
	64	+
	65	+echo ""
	66	+if [ "$failures" -gt 0 ]; then
	67	+ echo "$failures test(s) failed"
	68	+ exit 1
	69	+fi
	70	+echo "All tests passed"