Databricks

Databricks has two OSS surfaces:

• A built-in control-plane source named databricks for Databricks notification destination webhooks.
• An optional deployable Databricks node that exposes lakehouse data and AI capabilities through app.call.

Trigger Source

Databricks workspace notification destinations (Jobs, SQL alerts, Lakehouse Monitoring) POST signed webhooks to AgentField. The control plane verifies the request, deduplicates by event id, and dispatches through the normal execution path.

Create a trigger with:

• source: databricks
• target reasoner: databricks-prod.handle_databricks_event or your own webhook-enabled reasoner
• secret env var: DATABRICKS_TRIGGER_SECRET

{
  "auth_mode": "basic",
  "basic_username": "agentfield",
  "event_type_path": "run_state.life_cycle_state",
  "event_id_path": "run_id",
  "workspace_path": "workspace_id"
}

auth_mode supports basic and bearer. The AgentField trigger secret value becomes the Databricks webhook destination password (basic) or bearer token. Trigger config stores only the env-var name.

The *_path fields are JSON paths into the inbound payload that the source uses to derive a normalized event type, idempotency key, and workspace id. Defaults match the Databricks Jobs run-state payload shape.

Event Payload

Reasoners receive the normalized payload under event and dispatch metadata under _meta:

{
  "event": {
    "event_id": "1024-103045-abcdef",
    "event_type": "databricks.run_state.INTERNAL_ERROR",
    "workspace_id": "1234567890123456",
    "received_at": "2026-06-15T10:30:00Z",
    "payload": {
      "run_id": "1024-103045-abcdef",
      "job_id": 42,
      "run_state": { "life_cycle_state": "INTERNAL_ERROR", "result_state": "FAILED" }
    },
    "databricks": {
      "source": "notification_destination",
      "request_path": "/triggers/databricks-prod-failures"
    }
  },
  "_meta": {
    "source": "databricks",
    "event_type": "databricks.run_state.INTERNAL_ERROR",
    "idempotency_key": "1024-103045-abcdef"
  }
}

Capability Node

Deploy the optional node when agents need to query Databricks SQL, run AI Functions, or invoke a Model Serving endpoint from other AgentField nodes.

The OSS package is in integrations/databricks. The node image is built from integrations/databricks/node/Dockerfile, which expects the repository root as the Docker build context; the package also includes a docker-compose.example.yml.

git clone https://github.com/Agent-Field/agentfield.git
cd agentfield
docker build \
  -f integrations/databricks/node/Dockerfile \
  -t agentfield-databricks-node:latest \
  .

docker run --name agentfield-databricks \
  -p 18112:18112 \
  -e AGENTFIELD_URL=http://host.docker.internal:18080 \
  -e AGENT_NODE_ID=databricks-prod \
  -e AGENT_CALLBACK_URL=http://localhost:18112 \
  -e DATABRICKS_HOST=https://dbc-xxx.cloud.databricks.com \
  -e DATABRICKS_TOKEN=<personal-access-token> \
  -e DATABRICKS_WAREHOUSE_ID=<sql-warehouse-id> \
  -e DATABRICKS_CATALOG=workspace \
  -e DATABRICKS_SCHEMA=agentfield \
  agentfield-databricks-node:latest

Call it from any SDK:

result = await app.call("databricks-prod.query_readonly", input={
    "sql": "SELECT event_type, COUNT(*) FROM main.ops.events GROUP BY event_type",
})

The node registers these callable reasoners. Descriptions come from the OSS package contracts and node registration code.

Typical DX is to let a Databricks trigger wake a domain reasoner, then call the Databricks node only for data and AI work:

@app.reasoner()
async def review_databricks_event(input: dict):
    event = input["event"]
    metrics = await app.call("databricks-prod.query_readonly", input={
        "sql": f"SELECT * FROM ops.events WHERE event_id = '{event['event_id']}'",
    })
    review = await app.call("databricks-prod.ai_query", input={
        "endpoint": "databricks-meta-llama-3-1-8b-instruct",
        "prompt": "Summarize the operational risk in this result set as compact JSON.",
        "fail_on_error": True,
    })
    return {"event_id": event["event_id"], "metrics": metrics, "review": review}

A runnable version of this pattern lives in integrations/databricks/examples/incident-reviewer.

Guardrails

• Default SQL capability is read-only. SELECT, WITH, SHOW, DESCRIBE, and EXPLAIN are allowed; DDL, DML, COPY INTO, CALL, and role switching are denied.
• The Databricks SQL warehouse is the enforcement boundary — even allowed statements that mutate data are refused by Databricks when the warehouse is configured for read-only access.
• ai_query and invoke_serving_endpoint keep inference inside Databricks. They do not wrap AgentField .ai calls.
• Prompt-backed reasoners load prompts from DATABRICKS_PROMPTS_FILE or packaged defaults, not hardcoded customer behavior.