We have [recently announced](https://blog.langchain.dev/langgraph-platform-announce/) LangGraph Platform, a ***significantly*** enhanced solution for deploying agentic applications at scale.
Sign in to like and favorite skills
We have recently announced LangGraph Platform, a significantly enhanced solution for deploying agentic applications at scale.
LangGraph Platform incorporates key design patterns and capabilities essential for production-level deployment of large language model (LLM) applications.
In contrast to LangServe, LangGraph Platform provides comprehensive, out-of-the-box support for persistence, memory, double-texting handling, human-in-the-loop workflows, cron job scheduling, webhooks, high-load management, advanced streaming, support for long-running tasks, background task processing, and much more.
The LangGraph Platform ecosystem includes the following components:
LangServe was built as a deployment solution for LangChain Runnables created using the LangChain Expression Language (LCEL). In LangServe, the LCEL was the orchestration layer that managed the execution of the Runnable.
LangGraph is an open source library created by the LangChain team that provides a more flexible orchestration layer that's better suited for creating more complex LLM applications. LangGraph Platform is the deployment solution for LangGraph applications.
We recommend using LangGraph Platform rather than LangServe for new projects.
We will continue to accept bug fixes for LangServe from the community; however, we will not be accepting new feature contributions.
If you would like to migrate an existing LangServe application to LangGraph Platform, you have two options:
Runnableadd_routesLangGraphLangGraphThis option is the quickest way to migrate your application to LangGraph Platform. You can wrap the existing
Runnableadd_routesLangGraphOriginal LangServe code:
from langserve import add_routes app = FastAPI() # Some input schema class Input(BaseModel): input: str foo: Optional[str] # Some output schema class Output(BaseModel): output: Any runnable = .... # Your existing Runnable runnable_with_types = runnable.with_types(input_type=Input, output_type=Output) # Adds routes to the app for using the chain under: add_routes( app, runnable_with_types, )
Migrated LangGraph Platform code:
@dataclass class InputState: # Equivalent to Input in the original code """Defines the input state, representing a narrower interface to the outside world. This class is used to define the initial state and structure of incoming data. See: https://langchain-ai.github.io/langgraph/concepts/low_level/#state for more information. """ input: str foo: Optional[str] = None @dataclass class OutputState: # Equivalent to Output in the original code """Defines the output state, representing a narrower interface to the outside world. https://langchain-ai.github.io/langgraph/concepts/low_level/#state """ output: Any @dataclass class SharedState: """The full graph state. https://langchain-ai.github.io/langgraph/concepts/low_level/#state """ input: str foo: Optional[str] = None output: Any runnable = ... # Same code as before async def my_node(state: InputState, config: RunnableConfig) -> OutputState: """Each node does work.""" return await runnable.ainvoke({"input": state.input, "foo": state.foo}) # Define a new graph builder = StateGraph( SharedState, config_schema=Configuration, input=InputState, output=OutputState ) # Add the node to the graph builder.add_node("my_node", my_node) # Set the entrypoint as `call_model` builder.add_edge("__start__", "my_node") # Compile the workflow into an executable graph graph = builder.compile() graph.name = "New Graph" # This defines the custom name in LangSmith
This option is recommended if you want to take advantage of more advanced features in LangGraph Platform.
RunnableWithMessageHistoryFor example, LangGraph comes with built-in persistence that is more general than LangChain's
RunnableWithMessageHistoryPlease refer to the guide on upgrading to LangGraph memory for more details.
If you're relying on legacy LangChain agents, you can migrate them into the pre-built LangGraph agents. Please refer to the guide on migrating agents for more details.
If you created a custom chain and used LCEL to orchestrate it, you will usually be able to refactor it into a LangGraph without too much difficulty.
There isn't a one-size-fits-all guide for this, but generally speaking, consider creating a separate node for any long-running step in your LCEL chain or any step that you would want to be able to monitor or debug separately.
For example, if you have a simple Retrieval Augmented Generation (RAG) pipeline, you might have a node for the retrieval step and a node for the generation step.
Original LCEL code:
... rag_chain = ( {"context": retriever | format_docs, "question": RunnablePassthrough()} | prompt | llm | StrOutputParser() ) rag_chain.with_types(input_type=Input, output_type=Output)
Using LangGraph for the same pipeline:
@dataclass class InputState: # Equivalent to Input in the original code """Input question from the user.""" question: str @dataclass class OutputState: # Equivalent to Output in the original code """The output from the graph.""" answer: str @dataclass class SharedState: question: str docs: List[str] response: str async def retriever_node(state: InputState) -> SharedState: """Rettrieve documents based on the user's question.""" documents = await retriever.ainvoke({"context": state.question}) return { "docs": documents } async def generator_node(state: SharedState) -> OutputState: """Generate an answer using an LLM based on the retrieved documents and question.""" context = " -- DOCUMENT -- ".join(state.docs) prompt = [ SystemMessage( content=( "Answer the user's question based on the list of documents " "that were retrieved. Here are the documents: \n\n" f"{context}" ) ), HumanMessage(content=state.question), ] ai_message = await llm.ainvoke(prompt) return {"answer": ai_message.content} # Define a new graph builder = StateGraph( SharedState, config_schema=Configuration, input=InputState, output=OutputState ) builder.add_node("retriever", retriever_node) builder.add_node("generator", generator_node) builder.add_edge("__start__", "retriever") builder.add_edge("retriever", "generator") graph = builder.compile() graph.name = "RAG Graph"
Please see the LangGraph tutorials for tutorials and examples that will help you get started with LangGraph and LangGraph Platform.