FlockMTL Using Ollama

Choosing Ollama is a good option for local model execution. It allows you to run models locally without relying on external APIs. But we should first set up the Ollama API and model. Here's how to do it:

Install Ollama: Ensure Ollama is installed. If not, download it from the Ollama download page.
Download the Model: Confirm that the required model is available on your system.

Ollama Setup

Ollama Setup

After making sure that you have Ollama installed and the model downloaded, you can start using it with FlockMTL. The same as the other providers only two steps are required:

First, create a secret with your Ollama API URL. You can do this by running the following SQL command, where Ollama is running locally, and the default URL is 127.0.0.1:11434 :

CREATE SECRET (
    TYPE OLLAMA,
    API_URL '127.0.0.1:11434'
);

Next, create your Ollama model in the model manager. Make sure that the name of the model is unique, and the model you're using is already downloaded. You can create your model by running the following SQL command:

CREATE MODEL(
   'QuackingModel',
   'llama3.2',
   'ollama',
   {"context_window": 128000, "max_output_tokens": 2048}
);

Now you simply use FlockMTL with Ollama provider. Here's a small query to run to test if everything is working:

SELECT llm_complete(
    {'model_name': 'QuackingModel'},
    {'prompt': 'Talk like a duck 🦆 and write a poem about a database 📚'}
);

Ollama Setup​

Ollama Setup