Setup

Prior to leveraging the Donovan APIs, please ensure you have a Scale API Key to pass in the Authorization headers for all requests. If you don't have a Scale API Key or don't know how to pass it in the headers, read through Connecting to the Donovan API to properly configure your Live API Key and pass it in the Authorization header.

This guide will assume that your Donovan account has been provisioned and you are a member of an organization.

Authentication

Once you have obtained your Live Scale API Key, you can place that into the username field of the Basic Authentication header.

from requests.auth import HTTPBasicAuth
basic = HTTPBasicAuth('live_xxxxxx', '')

View Models

Before you get started with Donovan Chat, you can view the variety of large language models (LLMs) available to your organization. Donovan is a model-agnostic platform where you are able to swap in different base models, as well as your own custom fine-tuned models, to better serve your mission.

To view what models you can use to chat with, you can call our Models endpoint with the following code:

url = "https://api.donovan.scale.com/v1/models"

headers = {
    "accept": "application/json",
    "content-type": "application/json"
}

response = requests.get(url, headers=headers, auth=basic)

Aside: Data Organization

Organizations, which you should be a member of at this point in the guide, are the highest level of hierarchy in Donovan. Below organizations are workspaces, which each contain unique datasets. Users can be members of multiple workspaces, but only one organization. Think of workspaces as a common area for teams or projects to operate in and share data. You must always designate a workspace when you are chatting with Donovan. The next sections will help you identify existing workspaces, or create new workspaces from scratch. (For more information on Organizations and Workspaces, visit Organizations and Workspaces)

Choosing a Workspace

In order to utilize Donovan's retrieval-augmented generation over your data to fuel insights, you must first choose what groups of data you would like to focus on. The first step to this is choosing your workspace, whether it's existing or you would like to create a new workspace. Workspaces typically correspond to a group or a team, analogous Google Shared Drive where you and other team members can contribute to by adding documents and datasets within it.

View Existing Workspaces

Now that we have tested our API Key and checked what models our organization has access to, the next step is to view the current workspaces in our organization. You can use the Get Workspaces endpoint to view the workspaces you have access to in your organization.

url = "https://api.donovan.scale.com/v1/workspaces"

headers = {
    "accept": "application/json"
}

response = requests.get(url, headers=headers, auth=basic)

Creating a New Workspace

In the event where you are standing up a new repository of data that might not be related to any other data that you or your team has uploaded to Donovan, this might be the perfect use-case for creating a new workspace! Below, you can call the Create a Workspace endpoint.

url = "https://api.donovan.scale.com/v1/workspaces"

payload = { "name": "Finance Team" }
headers = {
    "accept": "application/json",
    "content-type": "application/json"
}

response = requests.post(url, json=payload, headers=headers, auth=basic)

Datasets

Now before you upload your documents, it'd be best to identify whether or not you need to create a dataset, which acts as a folder of data pertaining to a subject, theme, or concept. Donovan supports chatting with multiple datasets in a given workspace, so feel free to organize your datasets around time periods, subject matter, or whatever works best for your team.

Viewing a Workspace's Datasets

Datasets are scoped to one workspace, so if your workspace has datasets that you and your team are already contributing to, you can use the Get Workspace Datasets endpoint to retrieve the datasets that are available to the workspace you are looking to contribute in. Note: replace {workspace_id_or_name_here}in the URL with the workspace ID or name.

url = "https://api.donovan.scale.com/v1/workspaces/{workspace_id_or_name_here}/datasets"

headers = {
    "accept": "application/json"
}

response = requests.get(url, headers=headers, auth=basic)

Creating a New Dataset

If you would like to create a new dataset for new data that you plan to search across, you can call the Create Dataset endpoint. Description and sample prompts (or questions th

url = "https://api.donovan.scale.com/v1/datasets"

payload = {
    "name": "Team Meeting Minutes",
    "workspace": "Finance Team",
    "description": "Meeting minutes, agendas, and other documents pertaining to the Finance team.",
    "samplePrompts": ["What are the top three topics discussed across meetings?"]
}
headers = {
    "accept": "application/json",
    "content-type": "application/json"
}

response = requests.post(url, json=payload, headers=headers, auth=basic)

Uploading Data (Almost there!)

Now that you have a workspace with at least one dataset picked out, you can start uploading data to the datasets in the workspaces that you and your team are collaborating in. You can now upload documents that you have locally to these datasets using the Upload File from Local endpoint. Accepted file types can be found in the endpoint page, as well. Please note that you must update {dataset_id_here} with the dataset you would like to upload to, as well as convert your file to a data URL like the example below. If you aren't familiar with data URLs, you can use the commented out example to read a file into the proper format. To find the dataset ID, you can use Get Workspace Datasets for seeing your workspace datasets and grab the dataset ID from the response.

url = "https://api.donovan.scale.com/v1/upload/local"

# If you want to construct the data URL:
# "file": f"data:{filetype};base64," + base64.b64encode(open(local_file_path, 'rb').read()).decode('utf-8')

payload = {
    "mimeType": "text/plain",
    "datasetId": "{dataset_id_here}",
    "fileName": "test.txt",
    "file": "data:text/plain;name=test.txt;base64,aGVsbG8gZnJvbSB0aGUgRG9ub3ZhbiB0ZWFtIQo="
}
headers = {
    "accept": "application/json",
    "content-type": "application/json"
}

response = requests.post(url, json=payload, headers=headers, auth=basic)

Chunk and Embed Uploaded Documents

This step is crucial! Once you've uploaded your documents to a dataset, you can now kick off a batch job to process your assets to ensure they get chunked and embedded so that our retrieval logic can leverage the text chunks. You can utilize the Send Assets to Embedding endpoint. If assets are not chunked and embedded into our system, Donovan will not be able to search over these documents.

url = "https://api.donovan.scale.com/v1/upload/datasets/{dataset-id-goes-here}/staging-assets"

response = requests.post(url, headers=headers, auth=basic)

You've Made It! It's Time to Chat

Congratulations! You have setup at least one workspace, as well as a dataset with locally uploaded files that you can start to query on. Our Create Chat Message Response endpoint can support zero-to-many dataset chat. What does this mean? Let's dig into the different types of chat use-cases we see.

Zero Dataset Chat (i.e. No-RAG Chat)

This means that the chat prompt will not use any retrieval augmented generation on any data that you have uploaded, which can be good for more creative writing, general tasks not specifically tied to your data, or if you want to compare responses from different models available in Donovan.

import requests

url = "https://api.donovan.scale.com/v1/chat"

payload = {
    "model": "SCALE_LLAMA_2_13B_CHAT",
    "text": "Hello",
    "workspace": "<workspace ID here>"
}
headers = {
    "accept": "application/json",
    "content-type": "application/json",
    "authorization": "Basic <encoded key>"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

One-to-Many Dataset Chat (RAG Chat)

When passing in at least one dataset to chat against, our retrieval-backed chat infrastructure will search the documents in these dataset(s) to cite sources that the model uses to craft a fact-driven response.

url = "https://api.donovan.scale.com/v1/chat"

payload = {
    "model": "SCALE_LLAMA_2_13B_CHAT",
    "text": "Tell me about the Finance team's best practices",
    "workspace": "workspace_id_here",
    "datasets": ["dataset_1_id_here", "dataset_2_id_here"]
}
headers = {
    "accept": "application/json",
    "content-type": "application/json"
}

response = requests.post(url, json=payload, headers=headers, auth=basic)

Add Chat History through Threads

If you want to hold chat history to capture multiple messages between you and the model, you can utilize threads mentioned in Threads. In efforts to do this, you can create a thread to pass to the API call above.

Create Thread Create Chat Thread

import requests

url = "https://api.donovan.scale.com/v1/threads"

payload = {
    "workspace": "nf4zjsur1zgqu7u3oyt6cb6r",
    "publicName": "My Public API thread",
    "datasets": ["irgtf3o10x552nmxoyxlqp9c"]
}
headers = {
    "accept": "application/json",
    "content-type": "application/json",
    "authorization": "Basic bGl2ZV83NGJmNDY2NzllNDk0MThiOWVhOTVhOWM0NDhmYWUyZDo="
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

Create a chat message in a thread

url = "https://api.donovan.scale.com/v1/threads"

payload = {
    "model": "SCALE_LLAMA_2_13B_CHAT",
    "datasets": ["{dataset_1_id_here}", "{dataset_2_id_here}"],
    "workspace": "{workspace_id_here}",
    "thread": "{thread_id_here}"
}
headers = {
    "accept": "application/json",
    "content-type": "application/json"
}

response = requests.post(url, json=payload, headers=headers, auth=basic)

Parsing Citations from the Response

Donovan utilizes span citations to denote the start and end of the chunks that were cited from a particular source in your data. The citation ID maps to a fragment in the chat response.