Evaluation and optimization API

Use Datasets to collect examples, Evals to score Action versions, Experiments to compare two Actions, Feedback to label individual Data records, and fine-tunes to train an OpenAI model from selected records.

All routes require Authorization: Bearer YOUR_API_KEY and use the https://api.klu.ai/v1 base URL.

Dataset endpoints

curl --request POST 'https://api.klu.ai/v1/datasets' \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{
    "name":"Support golden set",
    "description":"Reviewed support examples",
    "app":"APP_GUID",
    "data":["DATA_GUID_1","DATA_GUID_2"]
  }'

The response contains guid, name, description, app, and created_by_id. On create, the API copies each resolved Data record's input and output into a Dataset Item. On update, datapoints replaces all existing Dataset Items; omitted or unresolvable GUIDs do not create items. name, description, and app are otherwise optional update fields, although the current controller retains the existing App relationship.

Dataset lists, Dataset Items, and the general Dataset list accept skip and limit, defaulting to 0 and 100. General and Item lists return { data, total_count, has_next_page }; the App-filtered route returns a bare array. Dataset Data returns a bare array and includes only Items linked to a Data record.

Eval endpoints

Create an Eval after obtaining Eval type GUIDs from /evals/types.

curl --request POST 'https://api.klu.ai/v1/evals' \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{
    "name":"Support answer quality",
    "dataset":"DATASET_GUID",
    "action":"ACTION_GUID",
    "version":"ACTION_VERSION_GUID",
    "eval_types":[{
      "guid":"EVAL_TYPE_GUID",
      "metadata":{"variables":[{"name":"threshold","value":0.8}]}
    }],
    "sampling_rate":1,
    "alert_on_fail":false
  }'

name, action, and eval_types are required. Dataset and version are optional. Each Eval type requires a guid; its optional metadata supports named string, number, boolean, or object variables. An update requires guid from the URL and an eval_types array; each entry can include new_record.

Eval configuration responses contain the Eval identity, metadata, timestamps, creator, and optional Dataset summary. Eval run responses include Action and version GUIDs, run number, deleted state, score strings, status, cost and latency summaries, and completed/total item counts. Run execution is asynchronous: treat metadata.status and run_items_completed as the progress source of truth.

This public router exposes the run list but does not register routes to start a run or fetch one run directly. The Python SDK defines run and get_eval_run methods for those older paths, but they are not backed by the current public router; use Klu to start a run and then read /evals/{guid}/runs.

Experiment endpoints

{
  "name": "Prompt comparison",
  "app": "APP_GUID",
  "primary_action": "PRIMARY_ACTION_GUID",
  "secondary_action": "SECONDARY_ACTION_GUID"
}

Both Actions must belong to the selected App. Experiment responses include guid, name, both Action GUIDs, status, timestamps, and deleted. The list accepts skip and limit with 0/100 defaults and returns the standard pagination envelope.

POST /experiments/{guid}/prompt runs a generation through the Experiment. It chooses between the primary and secondary Action and records the assignment, unless the caller forces an Action GUID. It supports synchronous, streaming, and asynchronous modes, and accepts the same input shapes as an Action prompt. The Python and TypeScript SDKs expose matching prompt, stream, and async-prompt helpers.

Feedback endpoints

curl --request POST 'https://api.klu.ai/v1/feedback/DATA_GUID' \
  --header 'Authorization: Bearer YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{
    "type":"rating",
    "value":"Positive",
    "source":"product-ui",
    "metadata":{"reason":"resolved the issue"}
  }'

The body-based route requires data, type, value, and source. The URL-based route requires type, value, and source; it accepts either metadata or legacy meta_data. Responses include the Data GUID, type, value, source, creator, metadata, timestamps, and deleted state. Feedback listing uses the standard 0/100 pagination envelope.

Fine-tune endpoints

{
  "name": "Support tone model",
  "app": "APP_GUID",
  "dataset": "DATASET_GUID",
  "data_ids": [101, 102, 103, 104, 105, 106, 107, 108, 109, 110],
  "base_model": "gpt-4.1-mini-2025-04-14",
  "workspaceModelProviderId": 12
}

data_ids are internal numeric Data IDs, not Data GUIDs, and at least 10 matching records are required. The public Data API returns GUIDs and does not expose these numeric IDs, so this create route is usable only when you already obtained the internal IDs through another authorized Klu surface. Creation builds a dedicated Dataset, uploads JSONL, starts an OpenAI fine-tune, and returns after the provider accepts the job with local status processing. It fails if the App is missing, fewer than 10 records resolve, no OpenAI model connection exists, or the provider rejects file/job creation. Poll /finetune/{guid}/status; its response is { "status": "...", "openai_finetune_name": "..." }.

Fine-tune lists use the standard pagination envelope with 0/100 defaults. High-level Python and TypeScript SDK clients intentionally do not support create; use REST for creation. Their get, update, delete, and status methods map to registered routes. Their older process helper targets /finetune/process, which this public router does not register.

Common errors and limits

Invalid bodies or enum values return 400. Missing Apps, Actions, Datasets, Evals, Data, or provider connections return 404. All resources are scoped by the authenticated workspace where the controller enforces that relationship. Delete operations are destructive to the API-visible resource; Dataset deletion also removes its Dataset Items. Preserve source Data separately if you need it later.