anthonypdawson / vector-inspector / 27910899718

80%

Build Type

push

github

Committed by web-flow

Commit Message

Dynamic content column detection + Ollama embedding support (#39)

* feat: add dynamic content column detection for cross-database compatibility

- Add automatic detection of content/text columns (document, text, content, etc.)
- Implement per-collection caching for performance
- Support manual override via set_content_column()
- Update all connection providers (pgvector, lancedb, chroma, pinecone, qdrant, weaviate)
- Add comprehensive test suite (11 tests)
- Add documentation

Fixes hardcoded 'document' assumption that caused issues with:
- Milvus tables using 'text' column
- Custom schemas with different column names
- Cross-database migration scenarios

All connections now call super().__init__() to initialize cache.

* fix: use pandas dtypes instead of PyArrow schema for LanceDB DataFrames

LanceDB returns pandas DataFrames which don't have .schema attribute.
Changed to use .dtypes.items() for constructing schema dict.

* feat: add Ollama embedding support for collections

- Add 'ollama' as supported model_type alongside sentence-transformer and CLIP
- Update load_embedding_model() to handle Ollama models (returns model name as-is)
- Update encode_text() to use ollama.embed() for Ollama models
- Add Ollama support to batch encoding in base_connection and pgvector
- Add 'Ollama' to embedding config dialog type names
- Graceful import error with installation instructions

Allows using local Ollama models for embeddings when HuggingFace is blocked.
Users can now configure collections with model_type='ollama' and model_name='<ollama-model>'.

* refactor: use HTTP API for Ollama embeddings (no extra dependencies)

Changed from ollama package to direct HTTP API calls, matching the approach
used by OllamaProvider for LLM. No new dependencies required.

Ollama embedding API: POST /api/embed with {model, input}
Same endpoint used by existing Ollama LLM integration.

* docs: add Ollama embedding setup guide

Complete guide for configuring collections to use... (continued)

Coverage Stats

219 of 399 new or added lines in 18 files covered. (54.89%)

6 existing lines in 2 files now uncovered.

15707 of 19553 relevant lines covered (80.33%)

0.8 hits per line