cweill / gotests / 18735038392

94%

master: 94%

Build Type

Pull #194

github

Committed by cweill

Commit Message

fix: implement strict golden file validation and regenerate all goldens

## Problem
E2E tests were using weak validation that only checked if *any* test case
name appeared in the golden file. This allowed bad golden files with:
- Placeholder names ("descriptive_test_name")
- Function names as test names ("FilterPositive", "ParseKeyValue")
- Null/empty values instead of realistic test data

## Solution

### 1. Strict Golden File Validation (internal/ai/e2e_test.go)
- Parse both generated and golden files with parseGoTestCases()
- Compare test cases field-by-field (names, args, want values, wantErr)
- Fail immediately on ANY mismatch (not just warnings)
- Ensures temperature=0 determinism is actually validated

### 2. Golden File Regeneration Script (scripts/regenerate-goldens.sh)
- Automates regeneration of all 11 AI golden files
- Strips CLI output ("Generated TestXxx", warnings)
- Uses real Ollama + qwen2.5-coder:0.5b for generation
- Ensures consistency across all golden files

### 3. Fixed/Added Golden Files

**Added (4 new test cases):**
- calculator_multiply_ai.go
- calculator_divide_ai.go
- string_utils_reverse_ai.go
- string_utils_contains_any_ai.go

**Regenerated (7 existing files):**
- business_logic_calculate_discount_ai.go - cleaned format
- business_logic_format_currency_ai.go - fixed placeholder names
- data_processing_filter_positive_ai.go - fixed nil values → real data
- math_ops_clamp_ai.go - cleaned format
- math_ops_factorial_ai.go - fixed placeholder names
- string_utils_parse_key_value_ai.go - fixed nil values → real data
- user_service_hash_password_ai.go - cleaned format

## Results

✅ All 9 E2E tests pass with strict validation:
- business_logic_calculate_discount ✓
- math_ops_clamp ✓
- data_processing_filter_positive ✓ (was failing)
- user_service_hash_password ✓
- calculator_multiply ✓ (new)
- calculator_divide ✓ (new)
- string_utils_reverse ✓ (new)
- string_utils_parse_key_value ✓ (was failing)
- string_utils_contains_any ✓ (... (continued)

Pull Request Pull Request #194: feat: AI-powered test case generation

Run Details

650 of 705 new or added lines in 10 files covered. (92.2%)

6 existing lines in 2 files now uncovered.

1512 of 1599 relevant lines covered (94.56%)

1359.95 hits per line