02/11/2023, 04:49 Deploy a Langchain agent as an API endpoint in 10 minutes | by Pierre-Louis Biojout | Sep, 2023 | Medium

Get unlimited access to the best of Medium for less than $1/week. Become a member

Deploy a Langchain agent as an API endpoint

in 10 minutes
Pierre-Louis Biojout · Follow
5 min read · Sep 20

Listen Share More

Github repository at the end of the article

So you coded a cool Langchain agent in Python. Congrats! In this tutorial, we will
walk you through the process of making it an OpenAPI endpoint, which can be
deployed and called as an API, allowing you to seamlessly integrate it into your
product or workflows. All that in only 10 minutes. Let’s get started!

https://medium.com/@plbiojout/deploy-a-langchain-agent-as-an-api-endpoint-in-10-minutes-266959613a2c 1/15
02/11/2023, 04:49 Deploy a Langchain agent as an API endpoint in 10 minutes | by Pierre-Louis Biojout | Sep, 2023 | Medium

Photo by Pawel Czerwinski on Unsplash

Installation and setup

You need to have Python 3.8 or higher installed.
(Optional) Create a new virtual environment

python3 -m venv venv

source venv/bin/activate

Install the required packages

langchain for the orchestration of the agent

openai for the OpenAI API

phospho for the deployment of the agent

python-dotenv for the environment variables (like api keys)

https://medium.com/@plbiojout/deploy-a-langchain-agent-as-an-api-endpoint-in-10-minutes-266959613a2c 2/15
02/11/2023, 04:49 Deploy a Langchain agent as an API endpoint in 10 minutes | by Pierre-Louis Biojout | Sep, 2023 | Medium

pip install --upgrade langchain openai phospho python-dotenv

pip freeze > requirements.txt

Store your OpenAI API key

In this example, we are going to use OpenAI as our LLM provider for our Langchain
agent. But you can use any LLM provider you wish.

Create an .env file and store your OpenAI API key in it. For extra security, you can
create a new OpenAI key for this project.

OPENAI_API_KEY=your-api-key

Build the agent logic

Create a new langchain agent
Create a main.py python file at the route of the project. You must name it
main.py since phospho will look for this file to initialize the agent.

Let’s load the environment variables from the .env file :

# main.py

# Load environment variables

from dotenv import load_dotenv

load_dotenv()

Now, let’s create a basic chatbot using the OpenAI API. As you can imagine, the
process would be roughly the same for more complex agents.

# main.py

# Load environment variables

from dotenv import load_dotenv

https://medium.com/@plbiojout/deploy-a-langchain-agent-as-an-api-endpoint-in-10-minutes-266959613a2c 3/15
02/11/2023, 04:49 Deploy a Langchain agent as an API endpoint in 10 minutes | by Pierre-Louis Biojout | Sep, 2023 | Medium

load_dotenv()

# use langchain to easily interact with the OpenAI API

from langchain.schema import (

AIMessage,
HumanMessage,
SystemMessage
)
from langchain.chat_models import ChatOpenAI

chat = ChatOpenAI(model_name="gpt-3.5-turbo",temperature=0.3)
messages = [
SystemMessage(content="You are a funny and helpful assistant."),
HumanMessage(content="Explain to me quantum physics."),
]
response=chat(messages)

print(response.content,end='\n')

Try it locally :

python main.py

Now, let’s package this logic into a function that we can use in our agent :

# main.py

# Load environment variables

from dotenv import load_dotenv

load_dotenv()

# import schema for chat messages and ChatOpenAI in order to query chatmodels G

from langchain.schema import (

AIMessage,
HumanMessage,
SystemMessage
)
from langchain.chat_models import ChatOpenAI

chat = ChatOpenAI(model_name="gpt-3.5-turbo",temperature=0.3)

https://medium.com/@plbiojout/deploy-a-langchain-agent-as-an-api-endpoint-in-10-minutes-266959613a2c 4/15
02/11/2023, 04:49 Deploy a Langchain agent as an API endpoint in 10 minutes | by Pierre-Louis Biojout | Sep, 2023 | Medium

def my_agent(input):
messages = [
SystemMessage(content="You are a funny and helpful assistant."),
HumanMessage(content=input),
]

Open in app
response=chat(messages)

returnSearch
response.content

Deploy the agent

Having the agent run locally is cool. But now, let’s deploy it so everyone can use it.

To do so, we are going to package our langchain agent into a phospho agent.
phospho is an agnostic Plateform as a Service to deploy and manage autonomous
agents. They also provide an SDK to help developers go from prototype to
production with their agents.

Here, we only want to create a basic conversational agent. So we only need to create
a chat route.

# main.py

from langchain.schema import (

AIMessage,
HumanMessage,
SystemMessage
)
from langchain.chat_models import ChatOpenAI
from phospho import Agent, Message

# Load environment variables

from dotenv import load_dotenv

load_dotenv()

# Define our langchain agent

chat = ChatOpenAI(model_name="gpt-3.5-turbo",temperature=0.3)

def my_agent(input):
messages = [
SystemMessage(content="You are a funny and helpful assistant."),
HumanMessage(content=input),
]

https://medium.com/@plbiojout/deploy-a-langchain-agent-as-an-api-endpoint-in-10-minutes-266959613a2c 5/15
02/11/2023, 04:49 Deploy a Langchain agent as an API endpoint in 10 minutes | by Pierre-Louis Biojout | Sep, 2023 | Medium

response=chat(messages)

return response.content

# Package it into a phospho agent

agent = Agent()

@agent.chat()
def my_chat(message):
# Use our agent logic to generate a response
response = my_agent(message.content)
# Return the response in a Message object
return Message(response)

That’s it!

With this code, we have created a new agent with a single route /chat that will call
our my_agent function and send the response back to the user. Note that you need to
send an object of type Message as a response (view the documentation for more
info).

Create a new Github repository​

At the root of your project, create a new Github repository and push your code to the
main branch. Do not forget to add a .gitignore file to ignore the .env file (there is
your API key inside) and the venv folder :

# Ignore virtualenv
venv/

# Ignore .env
.env

# Ignore pycache
__pycache__/

Create a new phospho project

Go to your phospho account dashboard and create a new project.

For the project name :

https://medium.com/@plbiojout/deploy-a-langchain-agent-as-an-api-endpoint-in-10-minutes-266959613a2c 6/15
02/11/2023, 04:49 Deploy a Langchain agent as an API endpoint in 10 minutes | by Pierre-Louis Biojout | Sep, 2023 | Medium

respect naming conventions, use just alphanumeric characters (no spaces, no

special characters, no emojis, etc…)

if the name is already taken, choose a new one. Come on, be creative!

Select the github repository you want to link to your phospho project.

Project creation on phospho.app

Setup the environment variables

Remember, we have stored our OpenAI API key in a .env file. But this file is not
available on the server since we didn't push it to Github. Our code won't work if the
API key is not available as an environment variable to the langchain agent.

On the phospho webapp, go to your project page and click on the Manage Env

Variables tab. Then, click on the Environment variables tab and add a new variable
https://medium.com/@plbiojout/deploy-a-langchain-agent-as-an-api-endpoint-in-10-minutes-266959613a2c 7/15
02/11/2023, 04:49 Deploy a Langchain agent as an API endpoint in 10 minutes | by Pierre-Louis Biojout | Sep, 2023 | Medium

called OPENAI_API_KEY with your OpenAI API key as a value.

At the next restart of your agent, it will be able to use the OpenAI API (to force a
restart, you can trigger a new deployment).

Environment variables management on phospho.app

Deploy the agent

Once project is created, just push some code to the main branch or go to the project
page and trigger the deployment manually using the button.

After a one or two minutes, your agent should be deployed and ready to use. You
have the endpoint URL on your project page.

Test your agent

Your agent is now deployed as an OpenAPI endpoint. You can find the URL of the
API endpoint of your agent on the project page. You can test it directly using the API
or by going in your browser to the URL and adding /docs at the end.

Unless you go viral, you should be able to use the free plan (no credit card required
to start).

https://medium.com/@plbiojout/deploy-a-langchain-agent-as-an-api-endpoint-in-10-minutes-266959613a2c 8/15
02/11/2023, 04:49 Deploy a Langchain agent as an API endpoint in 10 minutes | by Pierre-Louis Biojout | Sep, 2023 | Medium

interactive API documentation (Sagger UI)

Github repo with the source code

You can find all the source code of this tutorial in this github repo :

GitHub - phospho-app/agent-basic-chatgpt: This is an exemple of

a basic agent behaving like…
This is an exemple of a basic agent behaving like chatGPT, that can
be deployed on phospho. - GitHub …
github.com

Congratulations! You have successfully deployed a simple conversational agent

using Langchain, OpenAI and phospho. Feel free to explore and enhance the
capabilities of your agent further.

Want to give anonymous feedback? Fill in the google form here.

Langchain Llm Autonomous Agent Deployment API

Follow

https://medium.com/@plbiojout/deploy-a-langchain-agent-as-an-api-endpoint-in-10-minutes-266959613a2c 9/15
02/11/2023, 04:49 Deploy a Langchain agent as an API endpoint in 10 minutes | by Pierre-Louis Biojout | Sep, 2023 | Medium

Written by Pierre-Louis Biojout

8 Followers

X19

More from Pierre-Louis Biojout

Pierre-Louis Biojout

Knowledge-Based Agent Deployment with Langchain, Pinecone and

Phospho
This tutorial guides you through all the steps, using Python.

5 min read · Jun 18

See all from Pierre-Louis Biojout

https://medium.com/@plbiojout/deploy-a-langchain-agent-as-an-api-endpoint-in-10-minutes-266959613a2c 10/15
02/11/2023, 04:49 Deploy a Langchain agent as an API endpoint in 10 minutes | by Pierre-Louis Biojout | Sep, 2023 | Medium

Recommended from Medium

Vikas Tiwari in GoPenAI

Langchain Agent Tools for Functions and APIs

In the world of software development, we often find ourselves working with multiple functions,
each serving a different purpose or…

5 min read · Jul 9

https://medium.com/@plbiojout/deploy-a-langchain-agent-as-an-api-endpoint-in-10-minutes-266959613a2c 11/15
02/11/2023, 04:49 Deploy a Langchain agent as an API endpoint in 10 minutes | by Pierre-Louis Biojout | Sep, 2023 | Medium

Matt Tengtrakool in Artificial Corner

How to build tool-using agents with LangChain

Learn to use LangChain and Pinecone to build LLM agents that can access the internet and
knowledge bases

6 min read · Jun 4

217 3

Lists

Natural Language Processing

772 stories · 353 saves

Coding & Development

11 stories · 248 saves

Company Offsite Reading List

8 stories · 57 saves

New_Reading_List
174 stories · 170 saves

https://medium.com/@plbiojout/deploy-a-langchain-agent-as-an-api-endpoint-in-10-minutes-266959613a2c 12/15
02/11/2023, 04:49 Deploy a Langchain agent as an API endpoint in 10 minutes | by Pierre-Louis Biojout | Sep, 2023 | Medium

Diptiman Raichaudhuri

Amazon Bedrock Part 1 : RAG with Pinecone Vector Store, Anthropic

Claude LLM and LangChain— Income…
Disclosure: All opinions expressed in this article are my own, and represent no one but myself
and not those of my current or any previous…

8 min read · Oct 2

54

Dion

https://medium.com/@plbiojout/deploy-a-langchain-agent-as-an-api-endpoint-in-10-minutes-266959613a2c 13/15
02/11/2023, 04:49 Deploy a Langchain agent as an API endpoint in 10 minutes | by Pierre-Louis Biojout | Sep, 2023 | Medium

A deep dive into LangChain agents

A technical walkthrough of the mechanics that drive autonomous agents

5 min read · Jun 18

31 2

Guangya Liu

LangFlow Quick Start

Background

· 3 min read · Aug 26

https://medium.com/@plbiojout/deploy-a-langchain-agent-as-an-api-endpoint-in-10-minutes-266959613a2c 14/15
02/11/2023, 04:49 Deploy a Langchain agent as an API endpoint in 10 minutes | by Pierre-Louis Biojout | Sep, 2023 | Medium

Kelvin Lu

Tracing How LangChain Works Behind the Scenes

Introduction of LLM Tracing technique with both LangChain built-in tracing funciton and
LangChain Visualizer.

7 min read · Jun 26

See more recommendations

https://medium.com/@plbiojout/deploy-a-langchain-agent-as-an-api-endpoint-in-10-minutes-266959613a2c 15/15