feat: auto-detect embedding dimension during validation

[roomote]

Related GitHub Issue

Closes: #10991

Roo Code Task Context (Optional)

N/A

Description

This PR implements auto-detection of embedding dimensions during embedder validation to address the issue where users can configure incorrect embedding dimensions, causing Qdrant to reject vector upserts with dimension mismatches.

Key Implementation Details:

1. IEmbedder Interface Update: Added optional detectedDimension field to the validation result type
2. Embedder Updates: All 8 embedders (ollama, openai-compatible, openai, gemini, mistral, bedrock, openrouter, vercel-ai-gateway) now parse the test embedding response during validation and return the actual dimension
3. Service Factory Changes:
   • validateEmbedder() now returns the detected dimension
   • createVectorStore() accepts an optional detectedDimension parameter with the following priority order:
     1. Auto-detected from test embedding (most reliable)
     2. Profile-based from getModelDimension()
     3. Manual configuration from modelDimension setting
4. Manager Integration: _recreateServices() validates the embedder first, captures the detected dimension, and passes it to createServices()

Design Choices:

• The auto-detected dimension takes highest priority since it comes directly from the model's actual output
• Existing fallback mechanisms (profile-based and manual) are preserved for compatibility
• Zero or negative detected dimensions are ignored as invalid

Test Procedure

1. Automated Tests: Added new test cases covering:

   • Ollama embedder returning detected dimension (including realistic 4096-dimension embedding)
   • OpenAI-compatible embedder returning dimension from both array and base64-encoded embeddings
   • Service factory prioritizing detected dimension over other sources
   • Service factory falling back correctly when detected dimension is not available

2. Test Commands:

   cd src && npx vitest run services/code-index/__tests__/service-factory.spec.ts services/code-index/embedders/__tests__/ollama.spec.ts services/code-index/embedders/__tests__/openai-compatible.spec.ts

3. Manual Testing Scenario:

   • Configure Ollama with qwen3-embedding model
   • Set an incorrect manual dimension (e.g., 1536)
   • The system should now auto-detect the correct dimension (4096) and use it

Pre-Submission Checklist

• Issue Linked: This PR is linked to an approved GitHub Issue (see "Related GitHub Issue" above).
• Scope: My changes are focused on the linked issue (one major feature/fix per PR).
• Self-Review: I have performed a thorough self-review of my code.
• Testing: New and/or updated tests have been added to cover my changes.
• Documentation Impact: I have considered if my changes require documentation updates (see "Documentation Updates" section below).
• Contribution Guidelines: I have read and agree to the Contributor Guidelines.

Screenshots / Videos

N/A - This is a backend-only change with no UI impact.

Documentation Updates

• No documentation updates are required.
• Yes, documentation updates may be needed. The codebase indexing documentation could note that embedding dimensions are now auto-detected, making manual configuration less critical.

Additional Notes

This addresses the specific scenario described in issue #10991 where a user configured 1536 dimensions but their Ollama qwen3-embedding model actually produces 4096-dimension embeddings, causing Qdrant to reject upserts with the error: "Wrong input: Vector inserting dimension is expected to be 1536."

Get in Touch

N/A - Automated PR

[roomote]

Rooviewer   See task on Roo Cloud

Review complete. No issues found.

• Interface changes: IEmbedder.validateConfiguration() now returns detectedDimension
• All 8 embedders updated to detect dimensions from test embeddings
• Dimension priority correctly implemented (auto-detected > profile > manual)
• Manager integration properly passes detected dimension to vector store creation
• Comprehensive test coverage for new functionality

_{Mention @roomote in a comment to request specific changes to this pull request or fix all unresolved issues.}