surshar / claude-code

Claude Code is an agentic coding tool that lives in your terminal, understands your codebase, and helps you code faster by executing routine tasks, explaining complex code, and handling git workflows - all through natural language commands.

Built by Anthropic · github.com/anthropics

claude-code / plugins / plugin-dev / skills / command-development / references / testing-strategies.md

702 lines · 14.5 KBPreview CodeRaw

1	# Command Testing Strategies
2
3	Comprehensive strategies for testing slash commands before deployment and distribution.
4
5	## Overview
6
7	Testing commands ensures they work correctly, handle edge cases, and provide good user experience. A systematic testing approach catches issues early and builds confidence in command reliability.
8
9	## Testing Levels
10
11	### Level 1: Syntax and Structure Validation
12
13	What to test:
14	- YAML frontmatter syntax
15	- Markdown format
16	- File location and naming
17
18	How to test:
19
20	```bash
21	# Validate YAML frontmatter
22	head -n 20 .claude/commands/my-command.md \| grep -A 10 "^---"
23
24	# Check for closing frontmatter marker
25	head -n 20 .claude/commands/my-command.md \| grep -c "^---" # Should be 2
26
27	# Verify file has .md extension
28	ls .claude/commands/*.md
29
30	# Check file is in correct location
31	test -f .claude/commands/my-command.md && echo "Found" \|\| echo "Missing"
32	```
33
34	Automated validation script:
35
36	```bash
37	#!/bin/bash
38	# validate-command.sh
39
40	COMMAND_FILE="$1"
41
42	if [ ! -f "$COMMAND_FILE" ]; then
43	echo "ERROR: File not found: $COMMAND_FILE"
44	exit 1
45	fi
46
47	# Check .md extension
48	if [[ ! "$COMMAND_FILE" =~ \.md$ ]]; then
49	echo "ERROR: File must have .md extension"
50	exit 1
51	fi
52
53	# Validate YAML frontmatter if present
54	if head -n 1 "$COMMAND_FILE" \| grep -q "^---"; then
55	# Count frontmatter markers
56	MARKERS=$(head -n 50 "$COMMAND_FILE" \| grep -c "^---")
57	if [ "$MARKERS" -ne 2 ]; then
58	echo "ERROR: Invalid YAML frontmatter (need exactly 2 '---' markers)"
59	exit 1
60	fi
61	echo "✓ YAML frontmatter syntax valid"
62	fi
63
64	# Check for empty file
65	if [ ! -s "$COMMAND_FILE" ]; then
66	echo "ERROR: File is empty"
67	exit 1
68	fi
69
70	echo "✓ Command file structure valid"
71	```
72
73	### Level 2: Frontmatter Field Validation
74
75	What to test:
76	- Field types correct
77	- Values in valid ranges
78	- Required fields present (if any)
79
80	Validation script:
81
82	```bash
83	#!/bin/bash
84	# validate-frontmatter.sh
85
86	COMMAND_FILE="$1"
87
88	# Extract YAML frontmatter
89	FRONTMATTER=$(sed -n '/^---$/,/^---$/p' "$COMMAND_FILE" \| sed '1d;$d')
90
91	if [ -z "$FRONTMATTER" ]; then
92	echo "No frontmatter to validate"
93	exit 0
94	fi
95
96	# Check 'model' field if present
97	if echo "$FRONTMATTER" \| grep -q "^model:"; then
98	MODEL=$(echo "$FRONTMATTER" \| grep "^model:" \| cut -d: -f2 \| tr -d ' ')
99	if ! echo "sonnet opus haiku" \| grep -qw "$MODEL"; then
100	echo "ERROR: Invalid model '$MODEL' (must be sonnet, opus, or haiku)"
101	exit 1
102	fi
103	echo "✓ Model field valid: $MODEL"
104	fi
105
106	# Check 'allowed-tools' field format
107	if echo "$FRONTMATTER" \| grep -q "^allowed-tools:"; then
108	echo "✓ allowed-tools field present"
109	# Could add more sophisticated validation here
110	fi
111
112	# Check 'description' length
113	if echo "$FRONTMATTER" \| grep -q "^description:"; then
114	DESC=$(echo "$FRONTMATTER" \| grep "^description:" \| cut -d: -f2-)
115	LENGTH=${#DESC}
116	if [ "$LENGTH" -gt 80 ]; then
117	echo "WARNING: Description length $LENGTH (recommend < 60 chars)"
118	else
119	echo "✓ Description length acceptable: $LENGTH chars"
120	fi
121	fi
122
123	echo "✓ Frontmatter fields valid"
124	```
125
126	### Level 3: Manual Command Invocation
127
128	What to test:
129	- Command appears in `/help`
130	- Command executes without errors
131	- Output is as expected
132
133	Test procedure:
134
135	```bash
136	# 1. Start Claude Code
137	claude --debug
138
139	# 2. Check command appears in help
140	> /help
141	# Look for your command in the list
142
143	# 3. Invoke command without arguments
144	> /my-command
145	# Check for reasonable error or behavior
146
147	# 4. Invoke with valid arguments
148	> /my-command arg1 arg2
149	# Verify expected behavior
150
151	# 5. Check debug logs
152	tail -f ~/.claude/debug-logs/latest
153	# Look for errors or warnings
154	```
155
156	### Level 4: Argument Testing
157
158	What to test:
159	- Positional arguments work ($1, $2, etc.)
160	- $ARGUMENTS captures all arguments
161	- Missing arguments handled gracefully
162	- Invalid arguments detected
163
164	Test matrix:
165
166	\| Test Case \| Command \| Expected Result \|
167	\|-----------\|---------\|-----------------\|
168	\| No args \| `/cmd` \| Graceful handling or useful message \|
169	\| One arg \| `/cmd arg1` \| $1 substituted correctly \|
170	\| Two args \| `/cmd arg1 arg2` \| $1 and $2 substituted \|
171	\| Extra args \| `/cmd a b c d` \| All captured or extras ignored appropriately \|
172	\| Special chars \| `/cmd "arg with spaces"` \| Quotes handled correctly \|
173	\| Empty arg \| `/cmd ""` \| Empty string handled \|
174
175	Test script:
176
177	```bash
178	#!/bin/bash
179	# test-command-arguments.sh
180
181	COMMAND="$1"
182
183	echo "Testing argument handling for /$COMMAND"
184	echo
185
186	echo "Test 1: No arguments"
187	echo " Command: /$COMMAND"
188	echo " Expected: [describe expected behavior]"
189	echo " Manual test required"
190	echo
191
192	echo "Test 2: Single argument"
193	echo " Command: /$COMMAND test-value"
194	echo " Expected: 'test-value' appears in output"
195	echo " Manual test required"
196	echo
197
198	echo "Test 3: Multiple arguments"
199	echo " Command: /$COMMAND arg1 arg2 arg3"
200	echo " Expected: All arguments used appropriately"
201	echo " Manual test required"
202	echo
203
204	echo "Test 4: Special characters"
205	echo " Command: /$COMMAND \"value with spaces\""
206	echo " Expected: Entire phrase captured"
207	echo " Manual test required"
208	```
209
210	### Level 5: File Reference Testing
211
212	What to test:
213	- @ syntax loads file contents
214	- Non-existent files handled
215	- Large files handled appropriately
216	- Multiple file references work
217
218	Test procedure:
219
220	```bash
221	# Create test files
222	echo "Test content" > /tmp/test-file.txt
223	echo "Second file" > /tmp/test-file-2.txt
224
225	# Test single file reference
226	> /my-command /tmp/test-file.txt
227	# Verify file content is read
228
229	# Test non-existent file
230	> /my-command /tmp/nonexistent.txt
231	# Verify graceful error handling
232
233	# Test multiple files
234	> /my-command /tmp/test-file.txt /tmp/test-file-2.txt
235	# Verify both files processed
236
237	# Test large file
238	dd if=/dev/zero of=/tmp/large-file.bin bs=1M count=100
239	> /my-command /tmp/large-file.bin
240	# Verify reasonable behavior (may truncate or warn)
241
242	# Cleanup
243	rm /tmp/test-file*.txt /tmp/large-file.bin
244	```
245
246	### Level 6: Bash Execution Testing
247
248	What to test:
249	- !` commands execute correctly
250	- Command output included in prompt
251	- Command failures handled
252	- Security: only allowed commands run
253
254	Test procedure:
255
256	```bash
257	# Create test command with bash execution
258	cat > .claude/commands/test-bash.md << 'EOF'
259	---
260	description: Test bash execution
261	allowed-tools: Bash(echo:), Bash(date:)
262	---
263
264	Current date: !`date`
265	Test output: !`echo "Hello from bash"`
266
267	Analysis of output above...
268	EOF
269
270	# Test in Claude Code
271	> /test-bash
272	# Verify:
273	# 1. Date appears correctly
274	# 2. Echo output appears
275	# 3. No errors in debug logs
276
277	# Test with disallowed command (should fail or be blocked)
278	cat > .claude/commands/test-forbidden.md << 'EOF'
279	---
280	description: Test forbidden command
281	allowed-tools: Bash(echo:*)
282	---
283
284	Trying forbidden: !`ls -la /`
285	EOF
286
287	> /test-forbidden
288	# Verify: Permission denied or appropriate error
289	```
290
291	### Level 7: Integration Testing
292
293	What to test:
294	- Commands work with other plugin components
295	- Commands interact correctly with each other
296	- State management works across invocations
297	- Workflow commands execute in sequence
298
299	Test scenarios:
300
301	Scenario 1: Command + Hook Integration
302
303	```bash
304	# Setup: Command that triggers a hook
305	# Test: Invoke command, verify hook executes
306
307	# Command: .claude/commands/risky-operation.md
308	# Hook: PreToolUse that validates the operation
309
310	> /risky-operation
311	# Verify: Hook executes and validates before command completes
312	```
313
314	Scenario 2: Command Sequence
315
316	```bash
317	# Setup: Multi-command workflow
318	> /workflow-init
319	# Verify: State file created
320
321	> /workflow-step2
322	# Verify: State file read, step 2 executes
323
324	> /workflow-complete
325	# Verify: State file cleaned up
326	```
327
328	Scenario 3: Command + MCP Integration
329
330	```bash
331	# Setup: Command uses MCP tools
332	# Test: Verify MCP server accessible
333
334	> /mcp-command
335	# Verify:
336	# 1. MCP server starts (if stdio)
337	# 2. Tool calls succeed
338	# 3. Results included in output
339	```
340
341	## Automated Testing Approaches
342
343	### Command Test Suite
344
345	Create a test suite script:
346
347	```bash
348	#!/bin/bash
349	# test-commands.sh - Command test suite
350
351	TEST_DIR=".claude/commands"
352	FAILED_TESTS=0
353
354	echo "Command Test Suite"
355	echo "=================="
356	echo
357
358	for cmd_file in "$TEST_DIR"/*.md; do
359	cmd_name=$(basename "$cmd_file" .md)
360	echo "Testing: $cmd_name"
361
362	# Validate structure
363	if ./validate-command.sh "$cmd_file"; then
364	echo " ✓ Structure valid"
365	else
366	echo " ✗ Structure invalid"
367	((FAILED_TESTS++))
368	fi
369
370	# Validate frontmatter
371	if ./validate-frontmatter.sh "$cmd_file"; then
372	echo " ✓ Frontmatter valid"
373	else
374	echo " ✗ Frontmatter invalid"
375	((FAILED_TESTS++))
376	fi
377
378	echo
379	done
380
381	echo "=================="
382	echo "Tests complete"
383	echo "Failed: $FAILED_TESTS"
384
385	exit $FAILED_TESTS
386	```
387
388	### Pre-Commit Hook
389
390	Validate commands before committing:
391
392	```bash
393	#!/bin/bash
394	# .git/hooks/pre-commit
395
396	echo "Validating commands..."
397
398	COMMANDS_CHANGED=$(git diff --cached --name-only \| grep "\.claude/commands/.*\.md")
399
400	if [ -z "$COMMANDS_CHANGED" ]; then
401	echo "No commands changed"
402	exit 0
403	fi
404
405	for cmd in $COMMANDS_CHANGED; do
406	echo "Checking: $cmd"
407
408	if ! ./scripts/validate-command.sh "$cmd"; then
409	echo "ERROR: Command validation failed: $cmd"
410	exit 1
411	fi
412	done
413
414	echo "✓ All commands valid"
415	```
416
417	### Continuous Testing
418
419	Test commands in CI/CD:
420
421	```yaml
422	# .github/workflows/test-commands.yml
423	name: Test Commands
424
425	on: [push, pull_request]
426
427	jobs:
428	test:
429	runs-on: ubuntu-latest
430	steps:
431	- uses: actions/checkout@v2
432
433	- name: Validate command structure
434	run: \|
435	for cmd in .claude/commands/*.md; do
436	echo "Testing: $cmd"
437	./scripts/validate-command.sh "$cmd"
438	done
439
440	- name: Validate frontmatter
441	run: \|
442	for cmd in .claude/commands/*.md; do
443	./scripts/validate-frontmatter.sh "$cmd"
444	done
445
446	- name: Check for TODOs
447	run: \|
448	if grep -r "TODO" .claude/commands/; then
449	echo "ERROR: TODOs found in commands"
450	exit 1
451	fi
452	```
453
454	## Edge Case Testing
455
456	### Test Edge Cases
457
458	Empty arguments:
459	```bash
460	> /cmd ""
461	> /cmd '' ''
462	```
463
464	Special characters:
465	```bash
466	> /cmd "arg with spaces"
467	> /cmd arg-with-dashes
468	> /cmd arg_with_underscores
469	> /cmd arg/with/slashes
470	> /cmd 'arg with "quotes"'
471	```
472
473	Long arguments:
474	```bash
475	> /cmd $(python -c "print('a' * 10000)")
476	```
477
478	Unusual file paths:
479	```bash
480	> /cmd ./file
481	> /cmd ../file
482	> /cmd ~/file
483	> /cmd "/path with spaces/file"
484	```
485
486	Bash command edge cases:
487	```markdown
488	# Commands that might fail
489	!`exit 1`
490	!`false`
491	!`command-that-does-not-exist`
492
493	# Commands with special output
494	!`echo ""`
495	!`cat /dev/null`
496	!`yes \| head -n 1000000`
497	```
498
499	## Performance Testing
500
501	### Response Time Testing
502
503	```bash
504	#!/bin/bash
505	# test-command-performance.sh
506
507	COMMAND="$1"
508
509	echo "Testing performance of /$COMMAND"
510	echo
511
512	for i in {1..5}; do
513	echo "Run $i:"
514	START=$(date +%s%N)
515
516	# Invoke command (manual step - record time)
517	echo " Invoke: /$COMMAND"
518	echo " Start time: $START"
519	echo " (Record end time manually)"
520	echo
521	done
522
523	echo "Analyze results:"
524	echo " - Average response time"
525	echo " - Variance"
526	echo " - Acceptable threshold: < 3 seconds for fast commands"
527	```
528
529	### Resource Usage Testing
530
531	```bash
532	# Monitor Claude Code during command execution
533	# In terminal 1:
534	claude --debug
535
536	# In terminal 2:
537	watch -n 1 'ps aux \| grep claude'
538
539	# Execute command and observe:
540	# - Memory usage
541	# - CPU usage
542	# - Process count
543	```
544
545	## User Experience Testing
546
547	### Usability Checklist
548
549	- [ ] Command name is intuitive
550	- [ ] Description is clear in `/help`
551	- [ ] Arguments are well-documented
552	- [ ] Error messages are helpful
553	- [ ] Output is formatted readably
554	- [ ] Long-running commands show progress
555	- [ ] Results are actionable
556	- [ ] Edge cases have good UX
557
558	### User Acceptance Testing
559
560	Recruit testers:
561
562	```markdown
563	# Testing Guide for Beta Testers
564
565	## Command: /my-new-command
566
567	### Test Scenarios
568
569	1. Basic usage:
570	- Run: `/my-new-command`
571	- Expected: [describe]
572	- Rate clarity: 1-5
573
574	2. With arguments:
575	- Run: `/my-new-command arg1 arg2`
576	- Expected: [describe]
577	- Rate usefulness: 1-5
578
579	3. Error case:
580	- Run: `/my-new-command invalid-input`
581	- Expected: Helpful error message
582	- Rate error message: 1-5
583
584	### Feedback Questions
585
586	1. Was the command easy to understand?
587	2. Did the output meet your expectations?
588	3. What would you change?
589	4. Would you use this command regularly?
590	```
591
592	## Testing Checklist
593
594	Before releasing a command:
595
596	### Structure
597	- [ ] File in correct location
598	- [ ] Correct .md extension
599	- [ ] Valid YAML frontmatter (if present)
600	- [ ] Markdown syntax correct
601
602	### Functionality
603	- [ ] Command appears in `/help`
604	- [ ] Description is clear
605	- [ ] Command executes without errors
606	- [ ] Arguments work as expected
607	- [ ] File references work
608	- [ ] Bash execution works (if used)
609
610	### Edge Cases
611	- [ ] Missing arguments handled
612	- [ ] Invalid arguments detected
613	- [ ] Non-existent files handled
614	- [ ] Special characters work
615	- [ ] Long inputs handled
616
617	### Integration
618	- [ ] Works with other commands
619	- [ ] Works with hooks (if applicable)
620	- [ ] Works with MCP (if applicable)
621	- [ ] State management works
622
623	### Quality
624	- [ ] Performance acceptable
625	- [ ] No security issues
626	- [ ] Error messages helpful
627	- [ ] Output formatted well
628	- [ ] Documentation complete
629
630	### Distribution
631	- [ ] Tested by others
632	- [ ] Feedback incorporated
633	- [ ] README updated
634	- [ ] Examples provided
635
636	## Debugging Failed Tests
637
638	### Common Issues and Solutions
639
640	Issue: Command not appearing in /help
641
642	```bash
643	# Check file location
644	ls -la .claude/commands/my-command.md
645
646	# Check permissions
647	chmod 644 .claude/commands/my-command.md
648
649	# Check syntax
650	head -n 20 .claude/commands/my-command.md
651
652	# Restart Claude Code
653	claude --debug
654	```
655
656	Issue: Arguments not substituting
657
658	```bash
659	# Verify syntax
660	grep '\$1' .claude/commands/my-command.md
661	grep '\$ARGUMENTS' .claude/commands/my-command.md
662
663	# Test with simple command first
664	echo "Test: \$1 and \$2" > .claude/commands/test-args.md
665	```
666
667	Issue: Bash commands not executing
668
669	```bash
670	# Check allowed-tools
671	grep "allowed-tools" .claude/commands/my-command.md
672
673	# Verify command syntax
674	grep '!\`' .claude/commands/my-command.md
675
676	# Test command manually
677	date
678	echo "test"
679	```
680
681	Issue: File references not working
682
683	```bash
684	# Check @ syntax
685	grep '@' .claude/commands/my-command.md
686
687	# Verify file exists
688	ls -la /path/to/referenced/file
689
690	# Check permissions
691	chmod 644 /path/to/referenced/file
692	```
693
694	## Best Practices
695
696	1. Test early, test often: Validate as you develop
697	2. Automate validation: Use scripts for repeatable checks
698	3. Test edge cases: Don't just test the happy path
699	4. Get feedback: Have others test before wide release
700	5. Document tests: Keep test scenarios for regression testing
701	6. Monitor in production: Watch for issues after release
702	7. Iterate: Improve based on real usage data