LocalAI

LocalAI is the free, Open Source OpenAI alternative. LocalAI act as a drop-in replacement REST API that’s compatible with OpenAI API specifications for local inferencing. It allows you to run LLMs, generate images, audio (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families and architectures.

caution

For proper compatibility, please ensure you are using the openai SDK at version 0.x.

Installation and Setup

We have to install several python packages:

pip install tenacity openai

Embedding models

See a usage example.

from langchain_community.embeddings import LocalAIEmbeddings

API Reference:LocalAIEmbeddings