LangChain Chat Models: Overview

🚀   Launch interactive mode!

Ready to play? Fire up the playground and test it live!

👨‍💻 Author: Mohankumar Ramachandran

Master LangChain chat models with this comprehensive guide covering initialization, execution, structured outputs, tool calling, and multimodal inputs with practical examples.langchain, langchain-openai2025-07-12T00:00:00Z

Welcome to your comprehensive guide to LangChain Chat Models! 🚀

This hands-on notebook will help you understand basics of LangChain chat models, including:

• ⚡ Initialize chat models (various methods & providers)
• 🏃 Run & Stream outputs
• 💡 Track tokens for smart cost control
• 📦 Structured outputs with Pydantic, TypedDict, or JSON
• 🛠️ Tool calling (let AI use your functions)
• 🖼️ Multimodal: Mix text & images

Each section includes ready-to-run examples with OpenAI, but the concepts apply to all LangChain supported providers.

Ready to unlock the full power of LangChain chat models? Let's dive in! 🎉

Setup Model API key

Run the next cell to setup a free OpenAI API key using your GitHub account, or use your own OpenAI API key.

Note: May not work for enterprise GitHub accounts due to company-specific policies.

main.py

# NOTE: Set support_with_star to False if you do not want to support our work yet.
from foreverpython import init_github_models

# Run below if you already have your own OpenAI API key
# import os
# os.environ["OPENAI_API_KEY"] = "sk-..."

# else, run below to setup a free GitHub Models API key using your GitHub account.
await init_github_models(use_browser_cache=True, support_with_star=True)

Install packages

First, let's install the official Langchain and Langchain OpenAI Python package so you can access all the latest features and models! 📦✨

main.py

# Install the Langchain and Langchain OpenAI libraries
%pip install langchain==0.3.26 langchain-openai==0.3.27

Initialize Chat Models

Let's start with the basics: initializing chat models. LangChain makes it easy to work with various chat models, whether you're using OpenAI, Hugging Face, or any other provider by providing a standardized interface for initialization and execution.

🥇 Method 1: Universal Setup with init_chat_model()

Best for most cases! Simple and works with any provider.

When to use: If you want a quick, consistent way to start any chat model.

main.py

# Import the necessary modules from LangChain
from langchain.chat_models import init_chat_model

# Initialize the chat model
chat_model = init_chat_model(model="openai/gpt-4.1-mini", model_provider="openai")

# Send a simple message to the model
response = chat_model.invoke("How are you doing?")
print(response.content)

🥈 Method 2: Provider-Specific Classes

Use classes like ChatOpenAI if you want settings unique to that provider.

When to use: If you need a feature only supported by a specific provider (e.g., OpenAI-only settings).

main.py

# Import the ChatOpenAI class from langchain_openai
from langchain_openai import ChatOpenAI

# Initialize Chat OpenAI with specific parameters
chat_model = ChatOpenAI(model="openai/gpt-4.1-mini")

# Send a message using the provider-specific model
response = chat_model.invoke("How are you doing?")
print(response.content)

🥉 Method 3: Configurable Models

Make a model that lets you pick the provider each time you use it! Great for apps that support many model vendors.

When to use: If you want to switch models/providers dynamically in your code.

main.py

# Import the necessary modules from LangChain
from langchain.chat_models import init_chat_model

# Create a configurable model without specifying model/provider upfront
configurable_model = init_chat_model()

# Invoke with runtime configuration
response = configurable_model.invoke(
    "How are you doing?",
    config={
        "configurable": {
            "model_provider": "openai",
            "model": "openai/gpt-4.1-mini",
        }
    },
)

# Print the response content
print(response.content)

Execute Chat Models

Now that you know how to initialize chat models, let's explore the different ways to execute them and get responses. Each method serves different use cases:

1. Invoke Method - Perfect for single requests where you need one response 📝

2. Stream Method - Great for real-time responses and better user experience with long outputs 🌊

💡 Method 1: Using invoke()

Gets the full answer in one go. Super easy!

Use this for: Short questions, APIs, or when you don't care about partial results.

main.py

# Import the necessary modules from LangChain
from langchain.chat_models import init_chat_model

# Initialize the model
chat_model = init_chat_model(model="openai/gpt-4.1-mini", model_provider="openai")

# Send a single message and get a complete response
single_response = chat_model.invoke("How are you doing?")

# Print the content of the single response
print(single_response.content)

🌊 Method 2: Using stream()

Get the answer bit by bit, as it's written. Feels more like chatting!

Use this for: Long responses, user chat UIs, or when you want to display text as it comes in.

main.py

# Import the necessary modules from LangChain
from langchain.chat_models import init_chat_model

# Initialize the model
chat_model = init_chat_model(model="openai/gpt-4.1-mini", model_provider="openai")

# Stream the response chunk by chunk with pipe character as a separator
for chunk in chat_model.stream("How are you doing?"):
    print(chunk.content, end="|", flush=True)

Track Token Usage

When deploying LLM applications to production, monitoring token usage is crucial for managing costs effectively. LangChain makes it easy to track token consumption across different execution methods.

Why Track Tokens?

• Budget control - Know exactly how much each request costs 💸
• Optimization - Identify opportunities to reduce token usage 📊
• Monitoring - Set up alerts for unusual consumption patterns 🚨

🕹️ Scenario 1: Token Usage with invoke()

Easiest way! Just check the usage_metadata after getting your response.

main.py

# Token tracking with standard invoke method
from langchain.chat_models import init_chat_model

# Initialize the model
chat_model = init_chat_model(model="openai/gpt-4.1-mini", model_provider="openai")

# Invoke the model and get response
response_with_usage = chat_model.invoke("How are you doing?")

# Display the total token usage
print(f"Token usage: {response_with_usage.usage_metadata}")

🕹️ Scenario 2: Token Usage with stream()

Token info is in the last chunk. Combine all chunks to get the total usage.

Use this if: You're using streaming and want to know total token usage after.

main.py

# Token tracking with streaming
from langchain.chat_models import init_chat_model

# Initialize the model
chat_model = init_chat_model(model="openai/gpt-4.1-mini", model_provider="openai")

# Initialize aggregate to accumulate chunks
aggregate_response = None

# Stream with usage tracking enabled
for chunk in chat_model.stream("How are you doing?", stream_usage=True):
    # Accumulate chunks to get total usage
    if aggregate_response is None:
        aggregate_response = chunk
    else:
        aggregate_response = aggregate_response + chunk

# Display total token usage after streaming
print(f"Total token usage: {aggregate_response.usage_metadata}")

Structured Output

One of the most powerful features in LangChain is the ability to get structured, validated outputs from language models. This is essential for building reliable applications that need to process model outputs programmatically.

The Power of Structured Output

• Data extraction - Pull specific information from unstructured text 📋
• Database insertion - Create records ready for your database 💾
• Type safety - Ensure outputs match expected formats 🛡️
• API integration - Generate responses that match your API schemas 🔌

🥇 Method 1: Structured Output with Pydantic

Define your expected output with a Pydantic class (validates types, adds descriptions).

When to use: You want strong validation and easy error handling.

main.py

# Structured Output Method 1: Using Pydantic for validation
from typing import Optional
from pydantic import BaseModel, Field
from langchain.chat_models import init_chat_model

# Initialize the model
chat_model = init_chat_model(model="openai/gpt-4.1-mini", model_provider="openai")


# Define a Pydantic schema for jokes
class Joke(BaseModel):
    """Schema for a joke with setup, punchline, and rating."""

    setup: str = Field(description="The setup of the joke")
    punchline: str = Field(description="The punchline to the joke")
    rating: Optional[int] = Field(
        default=None, description="How funny the joke is, from 1 to 10"
    )


# Create a structured model that returns Joke objects
structured_joke_model = chat_model.with_structured_output(Joke)

# Get a joke as a validated Pydantic object
joke_response = structured_joke_model.invoke("Tell me a joke about cats")

# Display the joke in a structured format
joke_response

🥈 Method 2: Structured Output with JSON Schema

Define your output with a standard JSON Schema dict.

When to use: You're working with APIs or want to use standard JSON validation tools.

main.py

# Structured Output Method 2: Using JSON Schema
from langchain.chat_models import init_chat_model

# Initialize the model
chat_model = init_chat_model(model="openai/gpt-4.1-mini", model_provider="openai")

# Define a JSON schema for jokes
joke_json_schema = {
    "title": "Joke",
    "type": "object",
    "properties": {
        "setup": {"type": "string", "description": "The setup of the joke"},
        "punchline": {"type": "string", "description": "The punchline to the joke"},
        "rating": {
            "type": "integer",
            "description": "How funny the joke is, from 1 to 10",
        },
    },
    "required": ["setup", "punchline"],  # Rating is optional
}

# Create a structured model using JSON schema
json_structured_model = chat_model.with_structured_output(joke_json_schema)

# Get a joke as a dictionary matching the schema
joke_dict = json_structured_model.invoke("Tell me a joke about cats")

# Display the joke in a structured format
joke_dict

🥉 Method 3: Structured Output with TypedDict

Use Python's TypedDict for simple structured outputs (like typed dictionaries).

When to use: You want just a typed dictionary—not full validation.

main.py

# Structured Output Method 3: Using TypedDict
from typing_extensions import TypedDict
from langchain.chat_models import init_chat_model

# Initialize the model
chat_model = init_chat_model(model="openai/gpt-4.1-mini", model_provider="openai")


# Define a TypedDict schema for jokes
class JokeDict(TypedDict):
    setup: str  # The joke setup
    punchline: str  # The joke punchline
    rating: int  # Joke rating from 1-10


# Create a structured model using TypedDict
typed_structured_model = chat_model.with_structured_output(JokeDict)

# Get a joke as a typed dictionary
typed_joke = typed_structured_model.invoke("Tell me a joke about cats")

# Display the joke in a structured format
typed_joke

Tools / Function Calling

Tool calling allows language models to interact with external functions and APIs, dramatically expanding their capabilities. This feature is essential for building agents that can perform real-world actions.

Why Use Tool Calling?

• System integration - Connect models to your existing tools and services 🔧
• Mathematical operations - Let models perform accurate calculations 🧮
• Data retrieval - Fetch real-time information from databases or APIs 📡
• Action execution - Enable models to perform tasks beyond text generation 🎯

🥇 Method 1: Python Functions as Tools

Just write Python functions with type hints. Easy and quick!

When to use: Most cases—simple math, string tools, etc.

main.py

from langchain.chat_models import init_chat_model

# Initialize the model
chat_model = init_chat_model(model="openai/gpt-4.1-mini", model_provider="openai")


# Define tools as simple Python functions
def add(a: int, b: int) -> int:
    """Add two integers together.

    Args:
        a: First integer to add
        b: Second integer to add
    """
    return a + b


def multiply(a: int, b: int) -> int:
    """Multiply two integers together.

    Args:
        a: First integer to multiply
        b: Second integer to multiply
    """
    return a * b


# Create a list of tools
math_tools = [add, multiply]

# Bind tools to the model
model_with_math_tools = chat_model.bind_tools(math_tools)

# Ask the model to use the tools
tool_response = model_with_math_tools.invoke("What is 3 * 12? Also, what is 11 + 49?")

# Display the tool calls made by the model
tool_response.tool_calls

🥈 Method 2: Pydantic Models as Tools

Define tools as Pydantic classes for input validation.

When to use: You want strong input validation or need to document tool inputs.

main.py

from langchain.chat_models import init_chat_model
from pydantic import BaseModel, Field

# Initialize the model
chat_model = init_chat_model(model="openai/gpt-4.1-mini", model_provider="openai")


# Define tools as Pydantic models
class AddOperation(BaseModel):
    """Add two integers together."""

    a: int = Field(..., description="First integer to add")
    b: int = Field(..., description="Second integer to add")


class MultiplyOperation(BaseModel):
    """Multiply two integers together."""

    a: int = Field(..., description="First integer to multiply")
    b: int = Field(..., description="Second integer to multiply")


# Create a list of Pydantic tool schemas
pydantic_tools = [AddOperation, MultiplyOperation]

# Bind tools to the model
pydantic_model_with_tools = chat_model.bind_tools(pydantic_tools)

# Use the tools
pydantic_tool_response = pydantic_model_with_tools.invoke(
    "What is 3 * 12? Also, what is 11 + 49?"
)

# Display the tool calls made by the model
pydantic_tool_response.tool_calls

🥉 Method 3: TypedDict Classes as Tools

Use TypedDict for simple tool schemas (dictionary-like inputs).

When to use: Simple cases where you want clear input keys, but don't need full validation.

main.py

from typing_extensions import Annotated, TypedDict
from langchain.chat_models import init_chat_model

# Initialize the model
chat_model = init_chat_model(model="openai/gpt-4.1-mini", model_provider="openai")


# Define tools as TypedDict classes
class AddTypedDict(TypedDict):
    """Add two integers together."""

    # Annotated format: [type, default, description]
    a: Annotated[int, ..., "First integer to add"]
    b: Annotated[int, ..., "Second integer to add"]


class MultiplyTypedDict(TypedDict):
    """Multiply two integers together."""

    a: Annotated[int, ..., "First integer to multiply"]
    b: Annotated[int, ..., "Second integer to multiply"]


# Create tool list
typed_dict_tools = [AddTypedDict, MultiplyTypedDict]

# Bind tools to model
typed_dict_model_with_tools = chat_model.bind_tools(typed_dict_tools)

# Execute with tools
typed_dict_response = typed_dict_model_with_tools.invoke(
    "What is 3 * 12? Also, what is 11 + 49?"
)

# Display the tool calls made by the model
typed_dict_response.tool_calls

🏅 Method 4: LangChain @tool Decorator

Use LangChain's @tool decorator for extra features (like docstrings and metadata).

When to use: When you want more advanced features or documentation for your tools.

main.py

from langchain_core.tools import tool
from langchain.chat_models import init_chat_model

# Initialize the model
chat_model = init_chat_model(model="openai/gpt-4.1-mini", model_provider="openai")


# Define tools using the @tool decorator
@tool
def add_numbers(a: int, b: int) -> int:
    """Add two numbers together."""
    return a + b


@tool
def multiply_numbers(a: int, b: int) -> int:
    """Multiply two numbers together."""
    return a * b


# Create tool list
decorated_tools = [add_numbers, multiply_numbers]

# Bind tools to model
decorated_model_with_tools = chat_model.bind_tools(decorated_tools)

# Use the decorated tools
decorated_response = decorated_model_with_tools.invoke(
    "What is 3 * 12? Also, what is 11 + 49?"
)

# Display the tool calls made by the model
decorated_response.tool_calls

Tool Invocation

Let's put it all together! We'll define a tool 🛠️, let the model decide when to call it, and show how you can execute the tool call and return the result to the model ✨

main.py

# Import necessary libraries
import httpx
from langchain_core.messages import AIMessage, AnyMessage, HumanMessage, ToolMessage
from langchain.chat_models import init_chat_model
from typing import cast, List

# Initialize the model
chat_model = init_chat_model(model="openai/gpt-4.1-mini", model_provider="openai")

# Define the message history
message_history: List[AnyMessage] = [
    HumanMessage("What is the current temperature in New York City?")
]


# Define a tool to call the Open-Meteo weather API
def get_weather(latitude: str, longitude: str):
    """Calls Open-Meteo weather API to fetch the current temperature for given coordinates"""
    with httpx.Client() as client:
        response = client.get(
            "https://api.open-meteo.com/v1/forecast",
            params={
                "longitude": longitude,
                "latitude": latitude,
                "current": "temperature_2m,wind_speed_10m",
                "hourly": "temperature_2m,relative_humidity_2m,wind_speed_10m",
            },
        )
    return str(response.json()["current"]["temperature_2m"])


# Bind tools to model
model_with_tools = chat_model.bind_tools([get_weather])

# Use the decorated tools
response = cast(AIMessage, model_with_tools.invoke(message_history))

# Append the response to the message history
message_history.append(response)

# Execute the tool calls made by the model in loop
for tool_call in response.tool_calls:
    message_history.append(
        ToolMessage(
            content=get_weather(**tool_call["args"]), tool_call_id=tool_call["id"]
        )
    )

# Use the decorated tools
final_response = model_with_tools.invoke(message_history)

# Display the final response content
final_response.content

Multimodal Inputs

Modern language models can process more than just text! Many models now support multimodal inputs, allowing you to send images alongside text for richer interactions.

Multimodal Capabilities

• Image analysis - Describe, analyze, or answer questions about images 📸
• Content moderation - Check images for inappropriate content 🛡️
• Data extraction - Extract text or information from images 📄
• Visual reasoning - Solve problems that require visual understanding 🧩

🥇 Method 1: Base64-Encoded Data

Best for local files or when you want to upload the file itself.

When to use: If the file is not available online.

main.py

import base64
from langchain_core.messages import HumanMessage
from langchain.chat_models import init_chat_model

# Initialize the model
chat_model = init_chat_model(model="openai/gpt-4.1-mini", model_provider="openai")

# Specify your local image file path here
local_image_path = "./sample_data/green_pathway.jpg"

# Read an image file and return its Base64 encoded string
with open(local_image_path, "rb") as image_file:
    encoded_image = base64.b64encode(image_file.read()).decode("utf-8")

# Create a multimodal message with text and image
multimodal_message = HumanMessage(
    content=[
        {
            "type": "text",
            "text": "Describe the weather in this image:",
        },
        {
            "type": "image",
            "source_type": "base64",
            "data": encoded_image,
            "mime_type": "image/jpeg",
        },
    ]
)

# Send the multimodal message
image_analysis = chat_model.invoke([multimodal_message])
print(f"Image analysis: {image_analysis.content}")

🥈 Method 2: External file URLs

Let the model fetch the file from the internet.

When to use: If the file is already hosted online (http/https link).

main.py

from langchain_core.messages import HumanMessage
from langchain.chat_models import init_chat_model

# Initialize the model
chat_model = init_chat_model(model="openai/gpt-4.1-mini", model_provider="openai")

# Same image URL as before
landscape_url = "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg"

# Create a multimodal message with URL reference
url_multimodal_message = HumanMessage(
    content=[
        {
            "type": "text",
            "text": "Describe the weather in this image:",
        },
        {
            "type": "image",
            "source_type": "url",
            "url": landscape_url,
        },
    ]
)

# Send the message with URL-based image
url_image_analysis = chat_model.invoke([url_multimodal_message])
print(f"URL image analysis: {url_image_analysis.content}")

Acknowledgements

💡 If you found this guide helpful, consider exploring the official docs and community resources below for deeper learning and up-to-date best practices!

Content and inspiration for this guide were drawn from the following resources:

• 🔗 LangChain: ChatOpenAI
• 🔗 LangChain: Chat models