agentjido / req_llm / 44742923e05a883faa62e59b2a0c06258aaf37d2

49%

push

github

Refactor Meta/Llama into generic provider for code reuse (#148)

Extract Meta's native Llama prompt format into a reusable generic provider
that can be shared across cloud hosts and self-hosted deployments.

Changes:
- Created ReqLLM.Providers.Meta for Meta's native format
  - Handles prompt formatting with special tokens
  - Parses native response format (generation, prompt_token_count, etc.)
  - Extracts usage metadata with all required fields
- Refactored ReqLLM.Providers.AmazonBedrock.Meta to delegate to generic provider
  - Keeps only Bedrock-specific AWS Event Stream handling
  - Streaming usage now includes cached_tokens and reasoning_tokens
- Updated test error message for clarity

Documentation:
- Clarifies that most providers use OpenAI-compatible APIs (Azure, Vertex AI, vLLM, Ollama)
- AWS Bedrock is primary user of Meta's native format
- Generic provider handles native format with prompt/max_gen_len/generation fields
- Provides guidance for future provider implementations

This enables future Azure AI Foundry and Vertex AI support to correctly
delegate to OpenAI provider, while providing a native format option for
providers that need it.

30 of 35 new or added lines in 2 files covered. (85.71%)

3659 of 6969 relevant lines covered (52.5%)

457.17 hits per line