Feedback API

The default Ship API reduces cost while keeping outputs equivalent to your chosen model. The Feedback API removes that constraint — and unlocks a fundamentally better experience.

Why Feedback Matters

Without feedback, Ship is constrained to match the behavior of the base model you specify. It can save you money, but it can't do better than the original model — because it doesn't know what "better" means for your application.

Feedback changes this. Once you tell Ship what you care about — accuracy, helpfulness, tone, format, or any other metric — it no longer has to hew to the behavior of the original model. This removes constraints that prevent it from doing better than any individual model, and enables four capabilities:

Higher Quality

Different models get different questions right. With feedback, Ship learns which models excel at which types of requests in your specific workload. Instead of matching one model's quality, it can exceed it by selecting the best model for each request.

The effect is especially pronounced in agentic applications, where errors compound across multi-step chains. Even small per-step improvements translate to dramatically better end-to-end outcomes.

Improved Reliability

Imagine you're grading an exam and you want a reliable answer to a hard question. If you only ask one student, your outcome depends entirely on whether that particular student happens to know the answer. But if you ask several students and pick the best response, you get a much more consistent result — even if each individual student is unpredictable on their own.

LLMs work the same way. A single model might produce a great answer one time and a mediocre one the next. By drawing on multiple optimizations and selecting the best response for each request, Ship smooths out that variance. The result is more predictable, more consistent outputs than any single non-deterministic LLM can provide on its own.

With feedback, Ship learns which outputs to select — turning the inherent randomness of LLMs from a liability into an advantage.

Further Cost Reduction

The default ship-like/ mode saves money through inference-time optimization against a generic set of feedback signals. With feedback, Ship learns that some requests can be handled by more specific optimizations.

Latency Reduction

When an inference-time optimization can reduce latency, Ship uses it. With feedback, Ship learns which requests in your workload can benefit from additional inference-time optimizations without sacrificing quality.

The Feedback API

The /v1/feedback endpoint accepts an array of feedback events. Each event is tied to an inference by its id. Submit one or many events in a single request.

Endpoint

POST https://api.thesean.ai/v1/feedback

Request Body

The request body is a JSON array of feedback event objects. Each object has the following fields:

Required Fields

Optional Fields

Response

Returns a JSON array of result objects, one per input event, in the same order. Each result contains either feedback_ids (on success) or error (on failure). The endpoint always returns HTTP 200; inspect each item to determine success or failure.

Success example

[
  {
    "feedback_ids": {
      "helpful": "019503a1-b2c3-7d4e-8f01-234567890abc",
      "quality.accuracy": "019503a1-b2c3-7d4e-8f01-234567890def"
    },
    "error": null
  }
]

Partial failure example

If one event in the batch fails, it is reported per-item. Successful events are not affected.

[
  {
    "feedback_ids": { "helpful": "019503a1-b2c3-7d4e-8f01-234567890abc" },
    "error": null
  },
  {
    "feedback_ids": null,
    "error": { "status_code": 400, "message": "ID does not exist" }
  }
]

For nested feedback, the response keys use the flattened dot-notation form.

Example Request

curl https://api.thesean.ai/v1/feedback \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $THESEAN_API_KEY" \
  -d '[
    {
      "id": "019503a1-b2c3-7d4e-8f01-234567890abc",
      "feedback": {
        "helpful": true,
        "rating": 4.5
      },
      "optimize": "max"
    }
  ]'

Submitting Multiple Events

You can submit feedback for multiple inferences in a single request. Each event can target a different ID and have its own optimize direction and tags.

curl https://api.thesean.ai/v1/feedback \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $THESEAN_API_KEY" \
  -d '[
    {
      "id": "019503a1-b2c3-7d4e-8f01-234567890abc",
      "feedback": { "helpful": true }
    },
    {
      "id": "019503a2-d4e5-7f60-a1b2-c3d4e5f60718",
      "feedback": { "quality": 4.5 },
      "optimize": "max"
    }
  ]'

Obtaining IDs

The id field is returned as a top-level field in all inference responses. For streaming, the ID is included in the first event.

{
  "id": "019503a1-b2c3-7d4e-8f01-234567890abc",
  ...
}

Feedback Values

Each value in the feedback object can be one of the following types:

Example with Mixed Types

curl https://api.thesean.ai/v1/feedback \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $THESEAN_API_KEY" \
  -d '[
    {
      "id": "019503a2-d4e5-7f60-a1b2-c3d4e5f60718",
      "feedback": {
        "thumbs_up": true,
        "relevance": 0.92,
        "comment": "Answered the question clearly",
        "accuracy": {
          "score": 4,
          "reason": "Correct but could include more detail"
        }
      },
      "tags": {
        "evaluator": "human",
        "task": "question-answering"
      }
    }
  ]'

Nested Feedback

Feedback values can be nested to arbitrary depth. Nested keys are automatically flattened with dot notation before submission.

[
  {
    "id": "019503a1-b2c3-7d4e-8f01-234567890abc",
    "feedback": {
      "evaluation": {
        "quality": {
          "precision": 0.95,
          "recall": 0.87
        }
      },
      "helpful": true
    }
  }
]

This produces feedback entries with keys evaluation.quality.precision, evaluation.quality.recall, and helpful, each with its own feedback ID in the response:

[
  {
    "feedback_ids": {
      "evaluation.quality.precision": "019503a3-1111-7000-8000-000000000001",
      "evaluation.quality.recall": "019503a3-2222-7000-8000-000000000002",
      "helpful": "019503a3-3333-7000-8000-000000000003"
    }
  }
]