How to Use Ollama API to Run LLMs and Generate Responses

Ollama is an open-source platform that allows you to run open-source large language models (LLMs) locally on your system.

In this article, I am going to share how you can use the REST API that Ollama provides us to run and generate responses from large language models (LLMs). I will also show how you can use Python to programmatically generate responses from Ollama.

More on AIWhat Are Diffusion Models?

3 Steps to Use Ollama to Run LLMs

1. Ollama API is hosted on localhost at port 11434. You can go to the localhost to check if Ollama is running or not.
2. We will use curl in our terminal to send a request to the API. curl https://localhost:11434/api/generate -d '{ "model": "llama2-uncensored", "prompt": "What is water made of?" }' 

Here, I am using the llama2-uncensored model, but you can use any available models that you downloaded through Ollama. We can also send more parameters such as stream, which when set to false will only return a single JSON object as a response.

3. Now, as we see, the /api/generate endpoint is used to generate a response/completion for a given prompt. There are various endpoints that we can use for different purposes. You can check them out at the API Documentation of Ollama.

More on AIWhat Enterprises Need to Know Before Adopting a LLM

How to Generate Responses Through Ollama API Using Python

Now, we can use Python to generate responses programmatically.

1. Create a Python file. Import requests and json library. import requests import json
2. Create the URL, headers and data variables with values like the image below:

   

3. Now, use the post method of the response library and pass in the URL, headers and data variables that we created above. Store the response in a variable: response = requests.post(url, headers=headers, data=json.dumps(data))
4. Now, check the status code of the response. If it is 200, print the response text, else print the error. We can extract the exact response text from the JSON object like the snapshot below.

   

5. Run the program.

The complete snapshot of the code should look like this:

By following the steps above you will be able to run LLMs and generate responses locally using Ollama via its REST API. You can now use Python to generate responses from LLMs programmatically.

Frequently Asked Questions