cweill / gotests / 18735290381

94%

master: 94%

Build Type

Pull #194

github

Committed by cweill

Commit Message

fix: improve AI prompt to prevent duplicate test cases

Improved the LLM prompt and validation to ensure test cases have unique,
diverse values instead of duplicates or placeholder names.

**Prompt Improvements:**
- Emphasize "UNIQUE, DIFFERENT input values" for each test case
- Changed example from "descriptive_test_name" to "specific_scenario_name"
  with concrete examples (e.g., "valid_input", "empty_string", "negative_value")
- Simplified instructions for small model compatibility
- Added explicit instruction to show scaffold at end of prompt
- Added "Requirements" section with clear expectations

**Validation Enhancements:**
- Added `hasDuplicates()` function to detect identical test cases
- Validation now rejects test cases where all args+want values are the same
- LLM will retry with error feedback when duplicates are detected

**Golden Files:**
- Regenerated all 11 golden files with improved prompt
- Test case names now follow pattern: "valid_input", "empty_string", "negative_value"
- No more placeholder "descriptive_test_name" in golden files
- All E2E tests pass with strict golden validation

**Tests:**
- Updated unit tests to match new prompt text
- All 9 E2E tests pass with deterministic qwen2.5-coder:0.5b output
- Full test suite passes (24 tests, 95.2% coverage)

Note: Small model limitations still produce some type errors in generated code
(e.g., `price: ""` for float64), but output is deterministic for E2E validation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>

Pull Request Pull Request #194: feat: AI-powered test case generation

Run Details

682 of 743 new or added lines in 10 files covered. (91.79%)

6 existing lines in 2 files now uncovered.

1544 of 1637 relevant lines covered (94.32%)

1328.25 hits per line