Documentation · Examples Repo · Discord · Contributing
[!NOTE]
Contributing: pull requests are by invitation, and only for changes motivated
by actual use of Langroid in a real application. To propose a change,
open an issue naming
your organization (or that you are independent) and the application that needs
it. If it fits, we will implement it or invite you to send a PR. For custom work,
contact Prasad Chalasani (pchalasani at gmail dot com) to discuss terms. See
CONTRIBUTING.md for details.
Langroid is an intuitive, lightweight, extensible and principled
Python framework to easily build LLM-powered applications, from CMU and UW-Madison researchers.
You set up Agents, equip them with optional components (LLM,
vector-store and tools/functions), assign them tasks, and have them
collaboratively solve a problem by exchanging messages.
This Multi-Agent paradigm is inspired by the
Actor Framework
(but you do not need to know anything about this!).
Langroid is a fresh take on LLM app-development, where considerable thought has gone
into simplifying the developer experience;
it does not use Langchain, or any other LLM framework,
and works with practically any LLM.
🔥 ✨ A Claude Code plugin is available to accelerate Langroid development with built-in patterns and best practices.
🔥 Read the (WIP) overview of the langroid architecture, and a quick tour of Langroid.
🔥 MCP Support: Allow any LLM-Agent to leverage MCP Servers via Langroid's simple
MCP tool adapter that converts
the server's tools into Langroid's ToolMessage instances.
📢 Companies are using/adapting Langroid in production. Here is a quote:
Nullify uses AI Agents for secure software development.
It finds, prioritizes and fixes vulnerabilities. We have internally adapted Langroid's multi-agent orchestration framework in production, after evaluating CrewAI, Autogen, LangChain, Langflow, etc. We found Langroid to be far superior to those frameworks in terms of ease of setup and flexibility. Langroid's Agent and Task abstractions are intuitive, well thought out, and provide a great developer experience. We wanted the quickest way to get something in production. With other frameworks it would have taken us weeks, but with Langroid we got to good results in minutes. Highly recommended!
-- Jacky Wong, Head of AI at Nullify.
🔥 Just published in ML for Healthcare (2024): a Langroid-based Multi-Agent RAG system for pharmacovigilance, see blog post
Are you building LLM Applications, or want help with Langroid for your company, or want to prioritize Langroid features for your company use-cases? Prasad Chalasani is available for consulting (advisory/development): pchalasani at gmail dot com.
Sponsorship is also accepted via GitHub Sponsors
Questions, Feedback, Ideas? Join us on Discord!
Quick glimpse of coding with Langroid
This is just a teaser; there's much more, like function-calling/tools, Multi-Agent Collaboration, Structured Information Extraction, DocChatAgent (RAG), SQLChatAgent, non-OpenAI local/remote LLMs, etc. Scroll down or see docs for more. See the Langroid Quick-Start Colab that builds up to a 2-agent information-extraction example using the OpenAI ChatCompletion API.🔥 just released! Example script showing how you can use Langroid multi-agents and tools to extract structured information from a document using only a local LLM (Mistral-7b-instruct-v0.2).
import langroid as lr
import langroid.language_models as lm
set up LLM
llm_cfg = lm.OpenAIGPTConfig(
# any model served via an OpenAI-compatible API
chat_model=lm.OpenAIChatModel.GPT4o, # or, e.g., "ollama/mistral"
)
use LLM directly
mdl = lm.OpenAIGPT(llm_cfg)
response = mdl.chat("What is the capital of Ontario?", max_tokens=10)
use LLM in an Agent
agent_cfg = lr.ChatAgentConfig(llm=llm_cfg)
agent = lr.ChatAgent(agent_cfg)
agent.llm_response("What is the capital of China?")
response = agent.llm_response("And India?") # maintains conversation state
wrap Agent in a Task to run interactive loop with user (or other agents)
task = lr.Task(agent, name="Bot", system_message="You are a helpful assistant")
task.run("Hello") # kick off with user saying "Hello"
2-Agent chat loop: Teacher Agent asks questions to Student Agent
teacher_agent = lr.ChatAgent(agent_cfg)
teacher_task = lr.Task(
teacher_agent, name="Teacher",
system_message="""
Ask your student concise numbers questions, and give feedback.
Start with a question.
"""
)
student_agent = lr.ChatAgent(agent_cfg)
student_task = lr.Task(
student_agent, name="Student",
system_message="Concisely answer the teacher's questions.",
single_round=True,
)
teacher_task.add_sub_task(student_task)
teacher_task.run()
🔥 Updates/Releases
Click to expand
- Sep 2026:
OpenAIAssistant.
OpenAI sunset the Assistants API beta on 2026-08-26;
every endpoint now returns HTTP 404, so the class could not function. It has been
removed along with its tests and examples. OpenAIGPT -- which nearly all Langroid
code uses -- is unaffected. See the
migration note
for equivalents to threads, file_search, and code_interpreter.
- Aug 2026:
env_prefix for vector-store configs (env-var naming change -- see
migration notes),
generalized taint propagation across tool re-emission paths, and a one-time warning
when FileAttachment payloads inflate context preflight.
- 0.66.0 Big community batch (14 PRs):
Milvus vector store (thanks @zc277584121);
Markdown/HTML document parsing
(thanks @nuthalapativarun); cooperative max_time
task budgets, MCP tool
namespacing for multi-server agents, and portable JSON chat-history snapshots
(thanks @Whxuan0701); video attachments (thanks @octo-patch); retrieval score
thresholds (thanks @Koushik-Salammagari); even
context-overflow truncation
and several routing/parsing fixes -- full details in the
release notes.
- Aug 2025:
- Jul 2025:
- Jun 2025:
TaskTool for delegating tasks to sub-agents -
enables agents to spawn sub-agents with specific tools and configurations.
- 0.55.0 Event-based task termination with done_sequences -
declarative task completion using event patterns.
- 0.54.0 Portkey AI Gateway support - access 200+ models
across providers through unified API with caching, retries, observability.
- Mar-Apr 2025:
LLMPdfParser, generalizing
GeminiPdfParser to parse documents directly with LLM.
- 0.50.0 Structure-aware Markdown chunking with chunks
enriched by section headers.
- 0.49.0 Enable easy switch to LiteLLM Proxy-server
- 0.48.0 Exa Crawler, Markitdown Parser
- 0.47.0 Support Firecrawl URL scraper/crawler -
thanks @abab-dev
- 0.46.0 Support LangDB LLM Gateway - thanks @MrunmayS.
- 0.45.0 Markdown parsing with Marker - thanks @abab-dev
- 0.44.0 Late imports to reduce startup time. Thanks
@abab-dev
- Feb 2025:
GeminiPdfParser for parsing PDF using
Gemini LLMs - Thanks @abab-dev.
- 0.42.0: markitdown parser for pptx,xlsx,xls files
Thanks @abab-dev.
- 0.41.0: pinecone vector-db (Thanks @coretado),
Tavily web-search (Thanks @Sozhan308), Exa web-search (Thanks @MuddyHope).
- 0.40.0: pgvector vector-db. Thanks @abab-dev.
- 0.39.0: ChatAgentConfig.handle_llm_no_tool for
handling LLM "forgetting" to use a tool.
- 0.38.0: Gemini embeddings - Thanks @abab-dev)
- 0.37.0: New PDF Parsers: docling, pymupdf4llm
- Jan 2025:
- Dec 2024:
ToolMessage: _handler field to override
default handler method name in request field (thanks @alexagr).
- 0.27.0 OpenRouter Support.
- 0.26.0 Update to latest Chainlit.
- 0.25.0 True Async Methods for agent and
user-response (thanks @alexagr).
- Nov 2024:
Agents with strict JSON schema output format on compatible LLMs and strict mode for the OpenAI tools API.
(thanks @nilspalumbo).
- 0.23.0:
support for LLMs (e.g. Qwen2.5-Coder-32b-Instruct) hosted on glhf.chat
- 0.22.0:
Optional parameters to truncate large tool results.
- 0.21.0 Direct support for Gemini models via OpenAI client instead of using LiteLLM.
- 0.20.0 Support for
ArangoDB Knowledge Graphs.
- Oct 2024:
- Sep 2024:
o1-mini and o1-preview models.
- 0.15.0 Cerebras API support -- run llama-3.1 models hosted on Cerebras Cloud (very fast inference).
- 0.14.0 DocChatAgent uses Reciprocal Rank Fusion (RRF) to rank chunks retrieved by different methods.
- 0.12.0 run_batch_task new option -- stop_on_first_result - allows termination of batch as soon as any task returns a result.
- Aug 2024:
Task.run(), Task.run_async.
- 0.10.0 Allow tool handlers to return arbitrary result type, including other tools.
- 0.9.0 Orchestration Tools, to signal various task statuses, and to pass messages between agents.
- 0.7.0 OpenAI tools API support, including multi-tools.
- Jul 2024:
- Jun 2024:
RewindTool,
that lets an agent "rewind and redo" a past message (and all dependent messages are cleared out
thanks to the lineage tracking). Read notes here.
- May 2024:
doc-chat, db (for database-related dependencies). See updated
install instructions below and in the docs.
- Few-shot examples for tools: when defining a ToolMessage, previously you were able to include a classmethod named examples,
and a random example from this list would be used to generate a 1-shot example
for the LLM. This has been improved so you can now supply a list of examples
where each example is either a tool instance, or a tuple of (description,
tool instance), where the description is a "thought" that leads the LLM to use
the tool (see example in the docs). In some scenarios this can improve LLM tool
generation accuracy. Also, now instead of a random example, ALL examples are used to generate few-shot
examples.
- Infinite loop detection for task loops of cycle-length <= 10 (configurable
in TaskConfig. Only detects _exact_ loops, rather than _approximate_ loops where the entities are saying essentially similar (but not exactly the same) things repeatedly.
- "@"-addressing: any entity can address any other by name, which can be the name
of an agent's responder ("llm", "user", "agent") or a sub-task name. This is a
simpler alternative to the RecipientTool mechanism, with the tradeoff that
since it's not a tool, there's no way to enforce/remind the LLM to explicitly
specify an addressee (in scenarios where this is important).
- Much-Improved Citation
generation and display when using DocChatAgent.
- gpt-4o is now the default LLM throughout; Update tests and examples to work
with this LLM; use tokenizer corresponding to the LLM.
- gemini 1.5 pro support via litellm
- QdrantDB: update to support learned sparse embeddings.
- Apr 2024:
chat_model="groq/llama3-8b-8192".
See tutorial.
- 0.1.235: Task.run(), Task.run_async(), run_batch_tasks have max_cost
and max_tokens params to exit when tokens or cost exceed a limit. The result
ChatDocument.metadata now includes a status field which is a code indicating a
task completion reason code. Also task.run() etc can be invoked with an explicit
session_id field which is used as a key to look up various settings in Redis cache.
Currently only used to look up "kill status" - this allows killing a running task, either by task.kill()
or by the classmethod Task.kill_session(session_id).
For example usage, see the test_task_kill in tests/main/test_task.py
- Mar 2024:
DocChatAgent, see the
test_doc_chat_agent.py
in particular the test_doc_chat_batch();
New task run utility: run_batch_task_gen
where a task generator can be specified, to generate one task per input.
- 0.1.212: ImagePdfParser: support for extracting text from image-based PDFs.
(this means DocChatAgent will now work with image-pdfs).
- 0.1.194 - 0.1.211: Misc fixes, improvements, and features:
- Big enhancement in RAG performance (mainly, recall) due to a fix in Relevance
Extractor
- DocChatAgent context-window fixes
- Anthropic/Claude3 support via Litellm
- URLLoader: detect file time from header when URL doesn't end with a
recognizable suffix like .pdf, .docx, etc.
- Misc lancedb integration fixes
- Auto-select embedding config based on whether sentence_transformer module is available.
- Slim down dependencies, make some heavy ones optional, e.g. unstructured,
haystack, chromadb, mkdocs, huggingface-hub, sentence-transformers.
- Easier top-level imports from import langroid as lr
- Improve JSON detection, esp from weak LLMs
- Feb 2024:
chat_model="ollama/mistral". See release notes.
- 0.1.183: Added Chainlit support via callbacks.
See examples.
- Jan 2024:
SQLChatAgent works).
See example script using this Agent to answer questions about Python pkg dependencies.
- Support for .doc file parsing (in addition to .docx)
- Specify optional formatter param
in OpenAIGPTConfig to ensure accurate chat formatting for local LLMs.
- 0.1.157: DocChatAgentConfig
has a new param: add_fields_to_content, to specify additional document fields to insert into
the main content field, to help improve retrieval.
- 0.1.156: New Task control signals
PASS_TO, SEND_TO; VectorStore: Compute Pandas expression on documents; LanceRAGTaskCreator creates 3-agent RAG system with Query Planner, Critic and RAG Agent.
- Dec 2023:
DocChatAgent: Ingest Pandas dataframes and filtering.
- LanceDocChatAgent leverages LanceDB vector-db for efficient vector search
and full-text search and filtering.
- Improved task and multi-agent control mechanisms
- LanceRAGTaskCreator to create a 2-agent system consisting of a LanceFilterAgent that
decides a filter and rephrase query to send to a RAG agent.
- 0.1.141:
API Simplifications to reduce boilerplate:
auto-select an available OpenAI model (preferring gpt-4o), simplifies defaults.
Simpler Task initialization with default ChatAgent.
- Nov 2023:
OpenAIAssistant entries below are historical. The class was
> removed in 0.68.0 after OpenAI sunset the Assistants API; the links point at
> the 0.67.5 tag. See the
> migration note.
- 0.1.126: OpenAIAssistant agent: Caching Support. - 0.1.117: Support for OpenAI Assistant API tools: Function-calling, Code-intepreter, and Retriever (RAG), file uploads. These work seamlessly with Langroid's task-orchestration. Until docs are ready, it's best to see these usage examples: - Tests: - test_openai_assistant.py - test_openai_assistant_async.py
- Example scripts:
- The most basic chat app
- Chat with code interpreter
- Chat with retrieval (RAG)
- 2-agent RAG chat
- 0.1.112: OpenAIAssistant is a subclass of ChatAgent that
leverages the new OpenAI Assistant API. It can be used as a drop-in
replacement for ChatAgent, and relies on the Assistant API to
maintain conversation state, and leverages persistent threads and
assistants to reconnect to them if needed. Examples:
test_openai_assistant.py,
test_openai_assistant_async.py
- 0.1.111: Support latest OpenAI model: GPT4_TURBO
(see test_llm.py for example usage)
- 0.1.110: Upgrade from OpenAI v0.x to v1.1.1 (in preparation for
Assistants API and more); (litellm temporarily disabled due to OpenAI
version conflict).
- Oct 2023:
DocChatAgent re-rankers: rank_with_diversity, rank_to_periphery (lost in middle).
- 0.1.102: DocChatAgentConfig.n_neighbor_chunks > 0 allows returning context chunks around match.
- 0.1.101: DocChatAgent uses RelevanceExtractorAgent to have
the LLM extract relevant portions of a chunk using
sentence-numbering, resulting in huge speed up and cost reduction
compared to the naive "sentence-parroting" approach (writing out full
sentences out relevant whole sentences) which LangChain uses in their
LLMChainExtractor.
- 0.1.100: API update: all of Langroid is accessible with a single import, i.e. import langroid as lr. See the documentation for usage.
- 0.1.99: Convenience batch functions to run tasks, agent methods on a list of inputs concurrently in async mode. See examples in test_batch.py.
- 0.1.95: Added support for Momento Serverless Vector Index
- 0.1.94: Added support for LanceDB vector-store -- allows vector, Full-text, SQL search.
- 0.1.84: Added LiteLLM, so now Langroid can be used with over 100 LLM providers (remote or local)!
See guide here.
- Sep 2023:
docx files (preliminary).
- 0.1.72: Many improvements to DocChatAgent: better embedding model,
hybrid search to improve retrieval, better pdf parsing, re-ranking retrieved results with cross-encoders.
- Use with local LLama Models: see tutorial here
- Langroid Blog/Newsletter Launched!: First post is here -- Please subscribe to stay updated.
- 0.1.56: Support Azure OpenAI.
- 0.1.55: Improved SQLChatAgent that efficiently retrieves relevant schema info when translating natural language to SQL.
- Aug 2023:
GoogleSearchTool to enable Agents (their LLM) to do Google searches via function-calling/tools.
See this chat example for how easy it is to add this tool to an agent.
- Colab notebook to try the quick-start examples:
- 0.1.37: Added SQLChatAgent -- thanks to our latest contributor Rithwik Babu!
- Multi-agent Example: Autocorrect chat
- July 2023:
TableChatAgent to
chat with tabular datasets (dataframes, files, URLs): LLM generates Pandas code,
and code is executed using Langroid's tool/function-call mechanism.
- Demo: 3-agent system for Audience Targeting.
- 0.1.27: Added support
for Momento Serverless Cache as an alternative to Redis.
- 0.1.24: DocChatAgent
now accepts PDF files or URLs.
🚀 Demo
Suppose you want to extract structured information about the key terms of a commercial lease document. You can easily do this with Langroid using a two-agent system, as we show in the langroid-examples repo. (See this script for a version with the same functionality using a local Mistral-7b model.) The demo showcases just a few of the many features of Langroid, such as:- Multi-agent collaboration:
LeaseExtractoris in charge of the task, and its LLM (GPT4) generates questions
DocAgent.
- Retrieval augmented question-answering, with source-citation:
DocAgentLLM (GPT4) uses retrieval from a vector-store to
LeaseExtractor's questions, cites the specific excerpt supporting the answer.
- Function-calling (also known as tool/plugin): When it has all the information it
LeaseExtractor LLM presents the information in a structured
format using a Function-call.
Here is what it looks like in action (a pausable mp4 video is here).
!Demo
⚡ Highlights
(For a more up-to-date list see the Updates/Releases section above)- Agents as first-class citizens: The Agent class encapsulates LLM conversation state,
- Tasks: A Task class wraps an Agent, and gives the agent instructions (or roles, or goals),
Task.run() method has the same
type-signature as an Agent's responder's methods, and this is key to how
a task of an agent can delegate to other sub-tasks: from the point of view of a Task,
sub-tasks are simply additional responders, to be used in a round-robin fashion
after the agent's own responders.
- Modularity, Reusability, Loose coupling: The
AgentandTaskabstractions allow users to design
- LLM Support: Langroid supports OpenAI LLMs as well as LLMs from hundreds of
- Caching of LLM responses: Langroid supports Redis to cache LLM responses.
- Vector-stores: Qdrant, Chroma, LanceDB, Pinecone, PostgresDB (PGVector), Weaviate are currently supported.
- Grounding and source-citation: Access to external documents via vector-stores
- Observability, Logging, Lineage: Langroid generates detailed logs of multi-agent interactions and
ToolMessage mechanism which works with
any LLM, not just OpenAI's.
Function calling and tools have the same developer-facing interface, implemented
using Pydantic,
which makes it very easy to define tools/functions and enable agents
to use them. Benefits of using Pydantic are that you never have to write
complex JSON specs for function calling, and when the LLM
hallucinates malformed JSON, the Pydantic error message is sent back to
the LLM so it can fix it.
---
⚙️ Installation and Setup
Install langroid
Langroid requires Python 3.11+. We recommend using a virtual environment.
Use pip to install a bare-bones slim version of langroid (from PyPi) to your virtual
environment:
pip install langroidsentence-transformers embedding models from HuggingFace,
install Langroid like this:
pip install "langroid[hf-embeddings]"- To use various document-parsers, install langroid with the
doc-chatextra:
pip install "langroid[doc-chat]"- For "chat with databases", use the
dbextra:
pip install "langroid[db]"- You can specify multiple extras by separating them with commas, e.g.:
pip install "langroid[doc-chat,db]"- To simply install _all_ optional dependencies, use the
allextra (but note that this will result in longer load/startup times and a larger install size):
pip install "langroid[all]"Optional Installs for using SQL Chat with a PostgreSQL DB
If you are using SQLChatAgent
(e.g. the script examples/data-qa/sql-chat/sql_chat.py),
with a postgres db, you will need to:
- Install PostgreSQL dev libraries for your platform, e.g.
sudo apt-get install libpq-dev on Ubuntu,
- brew install postgresql on Mac, etc.
- Install langroid with the postgres extra, e.g.
pip install langroid[postgres]
poetry add "langroid[postgres]" or poetry install -E postgres,
(or the corresponding uv versions, e.g. uv add "langroid[postgres]"
or uv pip install langroid[postgres]).
If this gives you an error, try pip install psycopg2-binary in your virtualenv.
📝 If you get strange errors involving mysqlclient, try doing pip uninstall mysqlclient followed by pip install mysqlclient.
Claude Code Plugin (Optional)
This plugin provides two skills:
langroid:patterns- Your Claude Code agent can leverage this skill to produce
langroid:add-pattern- The agent can use this skill to record new patterns it
Step 1: Add the Langroid marketplace
From terminal:
claude plugin marketplace add langroid/langroidOr within Claude Code:
/plugin marketplace add langroid/langroidStep 2: Install the Langroid plugin
From terminal:
claude plugin install langroid@langroidOr within Claude Code:
/plugin install langroid@langroidOnce installed, simply ask your Claude Code agent to implement Langroid patterns in natural language, e.g.,
set up a Langroid agent so it uses the EditTool, and wrap it in a task that ends as soon as the tool is generated
and it will automatically use the langroid:patterns skill to follow the right design pattern.
You can also ask Claude Code to record a new pattern when you discover one, e.g.,
record this as a new Langroid pattern for setting up MCP tools
Set up environment variables (API keys, etc)
To get started, all you need is an OpenAI API Key. If you don't have one, see this OpenAI Page. (Note that while this is the simplest way to get started, Langroid works with practically any LLM, not just those from OpenAI. See the guides to using Open/Local LLMs, and other non-OpenAI proprietary LLMs.)
In the root of the repo, copy the .env-template file to a new file .env:
cp .env-template .env.env file should look like this (the organization is optional
but may be required in some scenarios).
OPENAI_API_KEY=your-key-here-without-quotes
OPENAI_ORGANIZATION=optionally-your-organization-id
Alternatively, you can set this as an environment variable in your shell
(you will need to do this every time you open a new shell):
export OPENAI_API_KEY=your-key-here-without-quotes
Optional Setup Instructions (click to expand)
All of the following environment variable settings are optional, and some are only needed
to use specific features (as noted below).
- Qdrant Vector Store API Key, URL. This is only required if you want to use Qdrant cloud.
Alternatively Chroma,
LanceDB, or Milvus are also
currently supported.
We use the local-storage version of Chroma, so there is no need for an API key.
- Milvus Vector Store URI, token, database name. These are optional.
Without them, Milvus uses local storage at ./milvus.db. Set MILVUS_URI
for Milvus server or Zilliz Cloud, MILVUS_TOKEN for token-based access,
and MILVUS_DB_NAME for a named database. Milvus Lite is unavailable on
Windows, so Windows users must set MILVUS_URI to a Milvus server or Zilliz
Cloud endpoint.
- Redis Password, host, port: This is optional, and only needed to cache LLM API responses
using Redis Cloud. Redis offers a free 30MB Redis account
which is more than sufficient to try out Langroid and even beyond.
If you don't set up these, Langroid will use a pure-python
Redis in-memory cache via the Fakeredis library.
- Momento Serverless Caching of LLM API responses (as an alternative to Redis).
To use Momento instead of Redis:
- enter your Momento Token in the .env file, as the value of MOMENTO_AUTH_TOKEN (see example file below),
- in the .env file set CACHE_TYPE=momento (instead of CACHE_TYPE=redis which is the default).
- GitHub Personal Access Token (required for apps that need to analyze git
repos; token-based API calls are less rate-limited). See this
GitHub page.
- Google Custom Search API Credentials: Only needed to enable an Agent to use the
GoogleSearchTool. To use Google Search as an LLM Tool/Plugin/function-call,
you'll need to set up
a Google API key,
then setup a Google Custom Search Engine (CSE) and get the CSE ID.
(Documentation for these can be challenging, we suggest asking GPT4 for a step-by-step guide.)
After obtaining these credentials, store them as values of
GOOGLE_API_KEY and GOOGLE_CSE_ID in your .env file.
Full documentation on using this (and other such "stateless" tools) is coming soon, but
in the meantime take a peek at this chat example, which
shows how you can easily equip an Agent with a GoogleSearchtool.
An
.env file with commonly used optional variables looks like this:
OPENAI_API_KEY=your-key-here-without-quotes
GITHUB_ACCESS_TOKEN=your-personal-access-token-no-quotes
CACHE_TYPE=redis # or momento
REDIS_PASSWORD=your-redis-password-no-quotes
REDIS_HOST=your-redis-hostname-no-quotes
REDIS_PORT=your-redis-port-no-quotes
MOMENTO_AUTH_TOKEN=your-momento-token-no-quotes # instead of REDIS* variables
QDRANT_API_KEY=your-key
QDRANT_API_URL=https://your.url.here:6333 # note port number must be included
GOOGLE_API_KEY=your-key
GOOGLE_CSE_ID=your-cse-id
Optional setup instructions for Microsoft Azure OpenAI(click to expand)
When using Azure OpenAI, additional environment variables are required in the
.env file.
This page Microsoft Azure OpenAI
provides more information, and you can set each environment variable as follows:
AZURE_OPENAI_API_KEY, from the value of API_KEY
AZURE_OPENAI_API_BASE from the value of ENDPOINT, typically looks like https://your.domain.azure.com.
For AZURE_OPENAI_API_VERSION, you can use the default value in .env-template, and latest version can be found here
AZURE_OPENAI_DEPLOYMENT_NAME is the name of the deployed model, which is defined by the user during the model setup
AZURE_OPENAI_MODEL_NAME Azure OpenAI allows specific model names when you select the model for your deployment. You need to put precisly the exact model name that was selected. For example, GPT-4 (should be gpt-4-32k or gpt-4).
AZURE_OPENAI_MODEL_VERSION is required if AZURE_OPENAI_MODEL_NAME = gpt=4, which will assist Langroid to determine the cost of the model
🐳 Docker Instructions
langroid-examples
repository via this Docker Image.
All you need to do is set up environment variables in the .env file.
Please follow these steps to setup the container:
# get the .env file template from langroid repo
wget -O .env https://raw.githubusercontent.com/langroid/langroid/main/.env-template
Edit the .env file with your favorite editor (here nano), and remove any un-used settings. E.g. there are "dummy" values like "your-redis-port" etc -- if you are not using them, you MUST remove them.
nano .env
launch the container (the appropriate image for your architecture will be pulled automatically)
docker run -it --rm -v ./.env:/langroid/.env langroid/langroid:latest
Use this command to run any of the scripts in the
examples directory
python examples/<Path/To/Example.py>
🎉 Usage Examples
These are quick teasers to give a glimpse of what you can do with Langroid
and how your code would look.
⚠️ The code snippets below are intended to give a flavor of the code
and they are not complete runnable examples! For that we encourage you to
consult the
langroid-examples
repository.
ℹ️
The various LLM prompts and instructions in Langroid
have been tested to work well with GPT-4 (and to some extent GPT-4o).
Switching to other LLMs (local/open and proprietary) is easy (see guides mentioned above),
and may suffice for some applications, but in general you may see inferior results
unless you adjust the prompts and/or the multi-agent setup.
Getting Started Guide
for a detailed tutorial.
Click to expand any of the code examples below.
All of these can be run in a Colab notebook:
Direct interaction with LLM
``python
import langroid.language_models as lm
mdl = lm.OpenAIGPT( lm.OpenAIGPTConfig( chat_model=lm
... (README truncated for length)