# **Flock** a DBMS extension that deeply integrates LLM and RAG capabilities directly into OLAP systems

> **Flock** is a DBMS extension developed within the Data & AI Systems Laboratory (DAIS Lab) at Polytechnique Montreal. It is designed to bring analytics and semantic analysis capabilities directly into your SQL queries, deeply integrating language models and retrieval-augmented generation through map and reduce functions.

**Flock** allows users to perform text generation, summarization, classification, filtering, fusion, embedding generation, and end-to-end RAG pipelines within DuckDB.

Structured JSON output is provided via `llm_complete` and other LLM functions with `model_parameters` / structured output schemas (replacing the legacy `llm_complete_json` function). See each function page for examples.

## Home

- [What is Flock?](https://dais-polymtl.github.io/flock/): Overview, capabilities, and paper.
- [Installation](https://dais-polymtl.github.io/flock/installation): Install DuckDB and load Flock (native or DuckDB-WASM).

## Getting Started

- [Getting Started Overview](https://dais-polymtl.github.io/flock/getting-started/overview): Configure providers and build from source.
- [OpenAI Provider](https://dais-polymtl.github.io/flock/getting-started/openai): OpenAI and OpenAI-compatible API setup.
- [Azure Provider](https://dais-polymtl.github.io/flock/getting-started/azure): Azure OpenAI setup.
- [Ollama Provider](https://dais-polymtl.github.io/flock/getting-started/ollama): Local inference setup.
- [Anthropic / Claude Provider](https://dais-polymtl.github.io/flock/getting-started/anthropic): Claude model setup.

## Resource Management

- [Resource Management Overview](https://dais-polymtl.github.io/flock/resource-management/overview): Models, prompts, and secrets.
- [Model Management](https://dais-polymtl.github.io/flock/resource-management/models): `max_batch_size`, `is_async`, `rate_limit`, `usage_limit`.
- [Prompt Management](https://dais-polymtl.github.io/flock/resource-management/prompts): Create and manage prompts.
- [API Keys Management](https://dais-polymtl.github.io/flock/resource-management/api-keys): Secure secret storage.
- [Model Parameters](https://dais-polymtl.github.io/flock/model-parameters): Provider-specific generation settings.

## Scalar Functions

- [Scalar Functions Overview](https://dais-polymtl.github.io/flock/scalar-functions/overview): Row-by-row LLM operations.
- [LLM Complete](https://dais-polymtl.github.io/flock/scalar-functions/llm-complete): Completions for text, image, and voice.
- [LLM Filter](https://dais-polymtl.github.io/flock/scalar-functions/llm-filter): Boolean filtering for text, image, and voice.
- [LLM Embedding](https://dais-polymtl.github.io/flock/scalar-functions/llm-embedding): Text embeddings (image/voice via two-step workflows).

## Aggregate Functions

- [Aggregate Functions Overview](https://dais-polymtl.github.io/flock/aggregate-functions/overview): Reduce and rerank operations.
- [LLM Reduce](https://dais-polymtl.github.io/flock/aggregate-functions/llm-reduce): Summarize groups with an LLM.
- [LLM Rerank](https://dais-polymtl.github.io/flock/aggregate-functions/llm-rerank): Relevance-based reranking.
- [LLM First](https://dais-polymtl.github.io/flock/aggregate-functions/llm-first): Top-ranked result.
- [LLM Last](https://dais-polymtl.github.io/flock/aggregate-functions/llm-last): Bottom-ranked result.

## Hybrid Search

- [Hybrid Search](https://dais-polymtl.github.io/flock/hybrid-search): Fusion functions (RRF, CombSum, etc.).

## Operations

- [Performance and Cost](https://dais-polymtl.github.io/flock/performance): Batching, throttling, and tuning.
- [LLM Metrics](https://dais-polymtl.github.io/flock/llm-metrics): `flock_get_metrics()` and observability.

## Contributing

- [Developer Guide](https://dais-polymtl.github.io/flock/developer-guide): Build and contribute to Flock.