Skip to main content
Version: 0.0.0

Files API

The Files API is available at https://api.endpoints.anyscale.com/v1/files. This API allows you to upload training and validation datasets to use for your fine-tuning jobs.

This API is compatible with the OpenAI SDK.

APIs

Upload file

POST https://api.endpoints.anyscale.com/v1/files

Uploads a file for use in fine-tuning. The file format must be in the JSONL format, which is a newline separated list of JSON objects.

Follow the instructions here for more details on the exact file format schema.

Request body

  • file string - The contents of the file to upload.

Returns

A file object

Examples

Use the OpenAI SDK

import openai

client = openai.OpenAI(
base_url = "https://api.endpoints.anyscale.com/v1",
api_key = "esecret_yourAuthTokenHere"
)

file_name = "train.jsonl"
file = client.files.create(
file=open(file_name, "rb"),
purpose="fine-tune",
user_provided_filename=file_name,
)

Use cURL

curl https://api.endpoints.anyscale.com/v1/files \
-H "Authorization: Bearer esecret_yourAuthTokenHere" \
-F 'file=@train.jsonl'
info

If you are seeing 504 errors with Your request has exceeded the timeout of 3 minutes when uploading, change the base URL for uploading to:

https://app.endpoints.anyscale.com/endpoints-api/v1

List files

GET https://api.endpoints.anyscale.com/v1/files

Gets a list of all the files that you uploaded to the Endpoints service.

Returns

A list of file object in the following format:

{
"data": [File, ...]
}

Examples
Examples

Use the OpenAI SDK

import openai

client = openai.OpenAI(
base_url = "https://api.endpoints.anyscale.com/v1",
api_key = "esecret_yourAuthTokenHere")
files = client.files.list()

Use cURL

curl https://api.endpoints.anyscale.com/v1/files \
-H "Authorization: Bearer esecret_yourAuthTokenHere"

Get file

GET https://api.endpoints.anyscale.com/v1/files/{file_id}

Gets the metadata of a single file that you uploaded to the Endpoints service.

Returns

A file object.

Examples

Use the OpenAI SDK

import openai

client = openai.OpenAI(
base_url = "https://api.endpoints.anyscale.com/v1",
api_key = "esecret_yourAuthTokenHere")
file = client.files.retrieve("file_123")

Use cURL

curl https://api.endpoints.anyscale.com/v1/files/file_123 \
-H "Authorization: Bearer esecret_yourAuthTokenHere"

Get file contents

GET https://api.endpoints.anyscale.com/v1/files/{file_id}/content

Returns the contents of the file that you uploaded to the Endpoints service.

📌 When using cURL, you must pass in a -L parameter to automatically follow redirects and retrieve the file contents.

Returns

The contents of the uploaded file.

Examples

Use the OpenAI SDK

import openai

client = openai.OpenAI(
base_url = "https://api.endpoints.anyscale.com/v1",
api_key = "esecret_yourAuthTokenHere")
content = client.files.retrieve_content("file_123")

Use cURL

curl -L https://api.endpoints.anyscale.com/v1/files/file_123/content \
-H "Authorization: Bearer esecret_yourAuthTokenHere"

Loading JSONL files

Many of the files are in jsonl format. The training data, the validation data, and the fine-tuning results are all in this format. The following code snippet shows how to load a jsonl file into a list of JSON objects.

from json import loads
jsonl_lines = content.decode("utf-8").split("\n")
content = [loads(line) for line in jsonl_lines if line != ""]

Delete file

DELETE https://api.endpoints.anyscale.com/v1/files/{file_id}

Deletes a file that was previously uploaded to the endpoints service.

Returns

The deleted file object.

Examples

Use the OpenAI SDK

import openai

client = openai.OpenAI(
base_url = "https://api.endpoints.anyscale.com/v1",
api_key = "esecret_yourAuthTokenHere")
content = openai.files.delete("file_123")

Use cURL

curl -X DELETE https://api.endpoints.anyscale.com/v1/files/file_123 \
-H "Authorization: Bearer esecret_yourAuthTokenHere"

Models

File

The file object represents a file that you uploaded to the Endpoints service. These files are referenced by ID in the fine-tuning API to use for training and validation.

📌 The following fields that are available in OpenAI's File object aren't supported by Anyscale Endpoints: object, purpose, status, status_details

Attributes

  • id string - The id of the file.
  • bytes int | null - The size of the file in bytes or null, if the size is unknown.
  • created_at string - The date and time the uploaded file in ISO 8601 format.
  • filename string - The original name of the uploaded file that.
Example
{
"id": "file_123",
"bytes": 123,
"created_at": "2023-09-14T18:00:00Z",
"filename": "train.jsonl"
}