Local OpenAI LLM _(Custom Integration for Home Assistant)_

Allows use of generic OpenAI-compatible LLM services, such as (but not limited to):

• DeepSeek
• Google (Gemini)
• llama.cpp
• LM Studio
• LocalAI
• Ollama
• OpenRouter
• Requesty
• Scaleway
• vLLM

• Added server URL to the initial server configuration
• Made the API Key optional during initial server configuration: can be left blank if your local server does not require

• Uses streamed LLM responses
• Conversation Agents support TTS streaming
• Automatically strips tags from responses
• Added support for image inputs for AI Task entities
• Added support for reconfiguring Conversation Agents
• Added option to trim conversation history to help stay within your context window
• Added temperature control (conversation agent + ai task)
• Added option to strip emojis from responses
• Added support for parallel tool calling
• Added experimental Retrieval Augmented Generation capability
• Added chat template arguments support
• Added image generation support for AI Task entities
• Added tools support for Generate Data actions for AI Task entities
• Emits a local_openai_llm_generation_complete event for all LLM completion requests, with usage/timings data if available

Installation

Install via HACS (recommended)

Have HACS installed, this will allow you to update easily.

Adding Tools for Assist to HACS can be using this button:

Integration Configuration

After installation, configure the integration through Home Assistant's UI:

1. Go to Settings → Devices & Services.
2. Click Add Integration.
3. Search for Local OpenAI LLM.
4. Follow the setup wizard to configure your desired services.

Configuration Notes

• The Server URL must be a fully qualified URL pointing to an OpenAI-compatible API.

• A Server Type configuration can be set to expose some additional options for different inference servers and

• Assist requires a fairly lengthy context for tooling and entity definitions.

• Tool calling must be enabled in your inference engine, eg:

• Parallel tool calling requires support from both your model and inference server.

• Chat Template Arguments allow you to provide custom arguments to your model

• Request Body Parameters allow you to provide additional top-level parameters to the chat completion request body

• AI Task entities can be configured for Text and/or Image generation capabilities

• Always continue conversation — When a voice agent finishes speaking, passes the active voice turn back to the user in

All Server Types

Custom HTTP Headers

Add custom HTTP headers as key-value pairs to all LLM API requests. Useful for passing custom metadata or provider-specific headers.

DeepSeek Cloud Configuration

Reasoning Effort

When the server type is set to DeepSeek Cloud, both conversation and AI task agents show a new **DeepSeek Configuration section with a Reasoning Effort** option. This controls the depth of reasoning when thinking is enabled.

• High (default) — standard reasoning effort.
• Max — maximum reasoning effort.

Thinking Mode

Controls when reasoning is enabled. Leave the option unset to disable thinking.

• Always on — thinking remains enabled for every iteration.
• First loop iteration only — thinking is disabled on iterations after the first (e.g., during tool-call rounds).

llama.cpp Configuration

When the server type is set to llama.cpp, both conversation and AI task agents show a llama.cpp Configuration section with the following options.

Thinking Mode

Controls when reasoning is enabled. Leave the option unset to disable thinking.

• Always on — reasoning is enabled for every iteration of the agent loop.
• First loop iteration only — thinking is disabled on iterations after the first (e.g., during tool-call rounds).

Include prior thinking

Controls whether thinking/reasoning content from prior conversation turns is sent back in new completion requests.

Some reasoning models require this enabled, and others require it disabled. Check the documentation for your model if unsure.

• Enabled (default) — prior-turn thinking_content is passed as reasoning_content in the next request, allowing

• Disabled — prior thinking context is stripped before sending. Use for models that reject prior reasoning context

Slot ID

Pins requests to a specific llama.cpp server slot for prompt-cache reuse. Leave empty to allow any slot to be used.

Use loaded model

When enabled, the integration prioritizes using models already resident in memory to minimize latency.

It follows these rules:

• Primary: Uses the configured model if it is currently loaded.
• Fallback: If the configured model is not loaded, it selects the first loaded model that supports the required modality (e.g., image inputs). If no compatible loaded models exist, it defaults back to the configured model.

Sampling Parameters

These options control how llama.cpp selects tokens during text generation.
Please refer to the llama.cpp documentation for further information and usage.

| Parameter | Description | Range | |----------------------|------------------------------------------------------------------------------------------------------------|--------| | Top-P | Restricts sampling to the top-p probability mass of tokens. | 0–1 | | Min-P | Minimum probability threshold for nucleus sampling, providing additional control when combined with top-p. | 0–1 | | Top-K | Limits sampling to the k highest-probability tokens. | 1–1000 | | Repeat Penalty | Penalizes repeat sequences of tokens. | -2–2 | | Presence Penalty | Penalizes tokens already present in the context. | -2–2 |

Model naming

llama.cpp exposes the value supplied via its --alias flag on the model object. When an alias is set it is used as the model's display name; otherwise the raw model id (typically the full model file path) is used, with the path and .gguf extension stripped for a cleaner name.

LocalAI Configuration

When the server type is set to LocalAI, Chat Template Arguments are sent via the OpenAI metadata request field rather than a top-level chat_template_kwargs field, as this is where LocalAI reads chat template variables from. Values are coerced to strings, per LocalAI's metadata convention.

See LocalAI model configuration.

vLLM Configuration

When the server type is set to vLLM, both conversation and AI task agents show a vLLM Configuration section with the following options.

Thinking token budget

Caps how many tokens the model may spend on reasoning, via the thinking_token_budget request parameter. Once the budget is reached, vLLM forces the reasoning block to be closed and the model proceeds to its answer.

• Empty (default) — no budget is sent, and the server default applies.
• 0 — the reasoning block is closed immediately, so the model answers without thinking.
• Any higher value — limits reasoning to that number of tokens.

Thinking Mode

Controls when reasoning is enabled. Leave the option unset to disable thinking.

• Always on — reasoning is enabled for every iteration and the token budget (if set) applies to all

• First loop iteration only — thinking is disabled on iterations after the first (e.g., during tool-call rounds).

When the budget is reached the reasoning block is closed with the parser's end string, which can leave the model cut off mid-thought. A wrap-up message can be added there to transition into the answer more gracefully, though this is configured on the vLLM server rather than per request:

--reasoning-parser qwen3
--reasoning-config '{"reasoning_start_str": "<think>", "reasoning_end_str": "\n\nTime to wrap up my reasoning and answer.</think>"}'

Experimental: Date/Time Context Injection Role

This integration supports injecting some dynamic content, presently the date and time, into the active Conversation Agent prompt when making a request. This was added as it is beneficial for the model to be grounded with this context in its role as an assistant, and was previously added to the system prompt by Home Assistant itself before later being removed due to negative effects on prompt caching and performance.

This was previously always-on but has been extracted as an experimental configuration option as this is not a once-size-fits-all for all models. To this end I have provided a number of options so that users can try them out and select the one that works best, or disable entirely if none work well, for their chosen model.

The available options are:

Tool Result:

The date and time are inserted as a Tool Call Result message to the model, before the current user message.

As long as the model does not reject it, this is the recommended method to use and produces the most reliable results during testing.

Assistant:

The date and time are inserted as an additional Assistant message to the model, before the current user message.

In cases where the Tool Call Result role method does not work for a model, this is the next recommended to test with.

User:

The date and time are inserted as an additional User message to the model, before the current user message.

Recommended only where neither the System nor Assistant injection methods work for the model, but may not produce desirable results. Some models have been known to repeat the date/time back to the user without request.

Disabled (no selection):

If your model simply refuses to work well with any method, simply remove the value from the configuration option to disable this again.

Experimental: Retrieval Augmented Generation (RAG) with Weaviate

Retrieval Augmented Generation is used to pre-feed your LLM messages with related data to provide contextually relevant information to the model based on the user input message.

This integration supports connecting your Agent to a Weaviate vector database server. Once configured, user messages to the Agent will be queried against the Weaviate database first, and the result data pre-emptively injected into the current conversation as contextual data for the Agent to utilise in their response.

This is not a general-purpose "memory" for the Agent: content is only provided to the Agent if it matches on the current user input message to the model.

See the Weaviate documentation for further information on Weaviate.

Weaviate Configuration

1. Install Weaviate locally

1. Reconfigure your LLM Server entity (not the Agent entity) in Home Assistant.

1. Optional: Reconfigure your AI Agent entities in Home Assistant.

Managing Data

Self-hosted Weaviate does not come with a front-end to manage data at all at this current point in time.

I have included a simple NodeJS-based WebApp server within the /weaviate directory of this repository, that can be used to connect to your local Weaviate instance and view, query, and manage the data in your object class. This is also setup into the supplied docker-compose.yml and exposed on port 9090 by default.

This tool supports the following basic functionality:

• Connect to your server and list the available object classes.
• View the available entry data in each class.
• Add new entry data to a class.
• Perform vector and hybrid searches against an object class.

Notes

• Only the current generations user message is queried in the database, no prior user messages are included.
• Search results are used for the current user/assistant turns only (including multiturn tool usages), and do not

• Objects are stored as 2 pieces of data: the query, and the content:

• Useful for providing contextual information to the LLM for different types of requests, without having all of it in

• I have performed basic testing of this with a variety of models across a few inference providers:

• A service action, local_openai.add_to_weaviate, can be used from within Home Assistant to add content to the

Web Search & Additional Tools

Looking to add some more functionality to your Home Assistant conversation agent, such as web and localised business/location search? Check out my Tools for Assist integration here!

These tools exist as a separate integration for compatibility across the wider Home Assistant Conversation ecosystem.

Acknowledgements

This integration was forked from the OpenRouter integration for Home Assistant by @joostlek.

Contributors

I would like to thank the following people for their contributions to this project:

NickM-27	JulienDeveaux	BryanCLieberman	PollyBot13	srwalter	alexjurkiewicz
ordex	gtrev500	whinis	Johnson145	ksmarty	Mugga6315
Thibaultjaigu	Thyraz	jgancedo	geryduyck

Support Development

If you find this integration useful and would like to support development, please consider buying me a coffee.

" style="max-width:100%;">