Multi-Dataset Queries

​

Querying Multiple Datasets

response = await client.query(
    text="transformer architectures",
    dataset_ids=["papers-ds", "patents-ds", "news-ds"],
    top_k=20,
)

​

Single-Model vs Multi-Model

​

Same embedding model

• Query text is embedded once
• Results are ranked by raw similarity score (0–1, higher = more relevant)
• Scores are directly comparable across datasets

​

Different embedding models

• Query text is embedded once per unique model (in parallel)
• Results are ranked by Reciprocal Rank Fusion (RRF)
• RRF uses 1/(k + rank) where rank is the result’s position within its dataset
• This is fair across models because it only considers rank order, not raw scores

​

Partial Failures

response = await client.query(
    text="search query",
    dataset_ids=["ds-1", "ds-2", "ds-3"],
)

if response.warnings:
    for warning in response.warnings:
        print(f"Warning: {warning}")

# Results from successful datasets are still returned
for result in response.results:
    print(f"[{result.score:.4f}] {result.dataset_id}: {result.metadata}")

​

Pre-Computed Vectors

• All datasets must use the same embedding model — the server can’t split a single vector across different models (returns 400 if they differ)
• Vector dimension must match the model’s expected dimension

# Only works if both datasets use the same model
response = await client.query(
    vector=[0.012, -0.034, ...],
    dataset_ids=["ds-1", "ds-2"],
)