umputun / newscope

81%

master: 81%

LAST BUILD ON BRANCH feature/ai-classification-improvements

branch: feature/ai-classification-improvements

CHANGE BRANCH

x

Reset

Sync Branches

• feature/ai-classification-improvements
• add-claude-github-actions-1754853952019
• dependabot/github_actions/actions/checkout-5
• dependabot/github_actions/actions/setup-go-6
• dependabot/go_modules/github.com/go-pkgz/rest-1.20.4
• dependabot/go_modules/github.com/sashabaranov/go-openai-1.40.5
• dependabot/go_modules/github.com/sashabaranov/go-openai-1.41.0
• dependabot/go_modules/github.com/sashabaranov/go-openai-1.41.1
• dependabot/go_modules/github.com/sashabaranov/go-openai-1.41.2
• dependabot/go_modules/golang.org/x/net-0.42.0
• dependabot/go_modules/golang.org/x/net-0.43.0
• dependabot/go_modules/golang.org/x/net-0.44.0
• dependabot/go_modules/golang.org/x/sync-0.16.0
• dependabot/go_modules/golang.org/x/sync-0.17.0
• dependabot/go_modules/modernc.org/sqlite-1.38.1
• feat/summary-validation-theme-toggle
• feature/multi-user-auth
• master

Build # 16818656031

Build Type

Pull #14

github

Committed by umputun

Commit Message

feat(llm): implement batch processing for AI classification with 90% cost reduction

- Add configurable batch processing with size (default: 10) and timeout (default: 5s)
- Remove 500-character content truncation for improved classification accuracy
- Update OpenAI library from v1.40.3 to v1.40.5 and upgrade model to gpt-5
- Implement ProcessBatch method in FeedProcessor for efficient batch handling
- Add comprehensive token usage monitoring and logging for cost tracking
- Fix async test patterns using require.Eventually instead of time.Sleep
- Add setupBasicItemManagerMocks helper to reduce test code duplication
- Update integration tests to handle asynchronous batch processing properly

The batch processing reduces API calls from ~50 individual calls to 2-3 batch calls
for the same number of articles, providing significant cost savings while maintaining
classification quality through full article content analysis.

Pull Request Pull Request #14: feat(llm): implement batch processing for AI classification with 90% cost reduction

Run Details

100 of 124 new or added lines in 2 files covered. (80.65%)

3623 of 4466 relevant lines covered (81.12%)

28.17 hits per line