agentjido / req_llm / f39d9fc48b3bd95eb576cce3f440d58fbc9a2e3d-PR-148

53%

main: 49%

Build Type

Pull #148

github

Committed by neilberkman

Commit Message

Refactor Meta/Llama into generic provider for code reuse

Extract Meta's native Llama prompt format into a reusable generic provider
that can be shared across cloud hosts and self-hosted deployments.

Changes:
- Created ReqLLM.Providers.Meta for Meta's native format
  - Handles prompt formatting with special tokens
  - Parses native response format (generation, prompt_token_count, etc.)
  - Extracts usage metadata with all required fields
- Refactored ReqLLM.Providers.AmazonBedrock.Meta to delegate to generic provider
  - Keeps only Bedrock-specific AWS Event Stream handling
  - Streaming usage now includes cached_tokens and reasoning_tokens
- Updated test error message for clarity

Documentation:
- Clarifies that most providers use OpenAI-compatible APIs (Azure, Vertex AI, vLLM, Ollama)
- AWS Bedrock is primary user of Meta's native format
- Generic provider handles native format with prompt/max_gen_len/generation fields
- Provides guidance for future provider implementations

This enables future Azure AI Foundry and Vertex AI support to correctly
delegate to OpenAI provider, while providing a native format option for
providers that need it.

Pull Request Pull Request #148: Refactor Meta/Llama into generic provider for code reuse

Run Details

30 of 35 new or added lines in 2 files covered. (85.71%)

6 existing lines in 4 files now uncovered.

3660 of 6969 relevant lines covered (52.52%)

433.13 hits per line