ChatGradient

This will help you getting started with DigitalOcean Gradient chat models.

Overview

Integration details

Class	Package	Package downloads	Package latest
ChatGradient	langchain-gradient

Setup

langchain-gradient uses DigitalOcean Gradient Platform.

Create an account on DigitalOcean, acquire a DIGITALOCEAN_INFERENCE_KEY API key from the Gradient Platform, and install the langchain-gradient integration package.

Credentials

Head to DigitalOcean Gradient

1. Sign up/Login to DigitalOcean Cloud Console
2. Go to the Gradient Platform and navigate to Serverless Inference.
3. Click on Create model access key, enter a name, and create the key.

Once you've done this set the DIGITALOCEAN_INFERENCE_KEY environment variable:

import os
os.environ["DIGITALOCEAN_INFERENCE_KEY"] = "your-api-key"

Installation

The LangChain Gradient integration is in the langchain-gradient package:

pip install -qU langchain-gradient

Instantiation

from langchain_gradient import ChatGradient

llm = ChatGradient(
    model="llama3.3-70b-instruct",
    api_key=os.environ.get("DIGITALOCEAN_INFERENCE_KEY")
)

Invocation

messages = [
    (
        "system",
        "You are a creative storyteller. Continue any story prompt you receive in an engaging and imaginative way.",
    ),
    ("human", "Once upon a time, in a village at the edge of a mysterious forest, a young girl named Mira found a glowing stone..."),
]
ai_msg = llm.invoke(messages)
ai_msg
print(ai_msg.content)

Chaining

from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate(
    [
        (
            "system",
            "You are a knowledgeable assistant. Carefully read the provided context and answer the user's question. If the answer is present in the context, cite the relevant sentence. If not, reply with \"Not found in context.\"",
        ),
        ("human", "Context: {context}\nQuestion: {question}"),
    ]
)

chain = prompt | llm
chain.invoke(
    {
        "context": (
            "The Eiffel Tower is located in Paris and was completed in 1889. "
            "It was designed by Gustave Eiffel's engineering company. "
            "The tower is one of the most recognizable structures in the world. "
            "The Statue of Liberty was a gift from France to the United States."
        ),
        "question": "Who designed the Eiffel Tower and when was it completed?"
    }
)

API Reference:ChatPromptTemplate