Support Classification Quickstart

This walkthrough mirrors the Example: Support Classification sample in the tenant Postman collection.

Scenario

A support team wants to classify incoming tickets by category, urgency, and destination team before routing them to engineering, billing, product, customer success, or developer relations.

What you will build

a support routing task
a labeled support dataset
an evaluation across two models
one active deployment
one live ticket classification

Step 1: Create the task

Create a task that accepts a support ticket and returns structured routing output.

bash

curl https://api.useorlo.com/v1/tasks \
  -H 'Content-Type: application/json' \
  -H 'X-Orlo-Org-Id: YOUR_ORG_ID' \
  -d '{
    "name": "Support Ticket Router",
    "description": "Classify incoming support tickets by category, urgency, and assigned team.",
    "input_schema": {
      "type": "object",
      "properties": {
        "subject": { "type": "string" },
        "body": { "type": "string" },
        "customer_tier": { "type": "string", "enum": ["free", "pro", "enterprise"] }
      },
      "required": ["subject", "body"]
    },
    "output_schema": {
      "type": "object",
      "properties": {
        "category": { "type": "string", "enum": ["billing", "bug_report", "feature_request", "account_access", "integration", "onboarding"] },
        "urgency": { "type": "string", "enum": ["critical", "high", "medium", "low"] },
        "team": { "type": "string", "enum": ["billing_ops", "engineering", "product", "customer_success", "devrel"] },
        "suggested_response": { "type": "string" }
      },
      "required": ["category", "urgency", "team"]
    },
    "prompt_template": "You are an expert support ticket router. Classify the ticket and route it. Respond with JSON only."
  }'

Save:

task_id
task_version_id

Step 2: Upload a labeled dataset

The Postman sample includes 10 labeled tickets. This shortened example shows the structure.

bash

curl https://api.useorlo.com/v1/datasets \
  -H 'Content-Type: application/json' \
  -H 'X-Orlo-Org-Id: YOUR_ORG_ID' \
  -d '{
    "task_id": "TASK_ID",
    "name": "Support Tickets Labeled v1",
    "samples": [
      {
        "input": {
          "subject": "URGENT: Production API returning 500",
          "body": "All API calls to /v1/users are returning 500 and checkout is blocked.",
          "customer_tier": "enterprise"
        },
        "expected_output": {
          "category": "bug_report",
          "urgency": "critical",
          "team": "engineering",
          "suggested_response": "We are investigating the API 500 errors immediately."
        }
      },
      {
        "input": {
          "subject": "Invoice shows wrong amount",
          "body": "My March invoice shows $450 but I am on the $200 plan.",
          "customer_tier": "pro"
        },
        "expected_output": {
          "category": "billing",
          "urgency": "medium",
          "team": "billing_ops",
          "suggested_response": "I will review the invoice and correct any billing discrepancy."
        }
      }
    ]
  }'

Save dataset_id.

Step 3: Choose candidate models

bash

curl https://api.useorlo.com/v1/models \
  -H 'X-Orlo-Org-Id: YOUR_ORG_ID'

Pick two model IDs with has_credentials=true.

Step 4: Run an evaluation

bash

curl https://api.useorlo.com/v1/evaluations/run \
  -H 'Content-Type: application/json' \
  -H 'X-Orlo-Org-Id: YOUR_ORG_ID' \
  -d '{
    "task_id": "TASK_ID",
    "dataset_id": "DATASET_ID",
    "models": ["MODEL_ID_1", "MODEL_ID_2"],
    "budget": {
      "max_tokens": 80000,
      "max_cost_usd": 3.0,
      "max_runtime_minutes": 20
    }
  }'

Save evaluation_id.

Step 5: Poll the result

bash

curl https://api.useorlo.com/v1/evaluations/EVALUATION_ID \
  -H 'X-Orlo-Org-Id: YOUR_ORG_ID'

Wait for status=completed.

Step 6: Create and activate a deployment

bash

curl https://api.useorlo.com/v1/deployments \
  -H 'Content-Type: application/json' \
  -H 'X-Orlo-Org-Id: YOUR_ORG_ID' \
  -d '{
    "task_id": "TASK_ID",
    "task_version_id": "TASK_VERSION_ID",
    "model_id": "WINNING_MODEL_ID",
    "strategy": "prompt"
  }'

Then activate it:

bash

curl -X PUT https://api.useorlo.com/v1/deployments/DEPLOYMENT_ID/activate \
  -H 'X-Orlo-Org-Id: YOUR_ORG_ID'

Step 7: Classify a live ticket

bash

curl https://api.useorlo.com/v1/tasks/TASK_ID/run \
  -H 'Content-Type: application/json' \
  -H 'X-Orlo-Org-Id: YOUR_ORG_ID' \
  -d '{
    "input": {
      "subject": "SSO login broken after your latest update",
      "body": "Since yesterday's update our team of 50 cannot login through Okta.",
      "customer_tier": "enterprise"
    },
    "explain": "debug"
  }'

You should get a routed result with:

category
urgency
team
validation and execution details

What success looks like

By the end of this flow you have a live ticket router that can:

route bugs to engineering
send billing issues to billing ops
handle onboarding and integration questions with the right team

This is a strong first Orlo workflow because the task is easy to reason about and easy to validate.

What Proof You Created

Proof	Where It Appears	Why It Matters
Task definition	The support routing task and task version	Shows the intended workflow, schema, prompt, and output contract
Dataset evidence	The labeled support examples	Shows the model was tested against realistic routing cases
Evaluation result	The completed evaluation	Shows which candidate model performed best for the support task
Runtime validation	The live ticket classification result	Shows the output followed the required category, urgency, and team fields
Operational trace	The inference log in debug mode	Lets support ops and platform teams review what happened later

The support quickstart is intentionally simple: it shows how a domain team can turn a repeated workflow into a measured, deployed, reviewable AI task.