Database-orchestrated AI · .NET 8 · SQL Server

Your database is the AI orchestrator.

Substrata turns SQL Server into the workflow engine for AI. You register a Thread as a row — its Data carries the SPName to run when the answer lands. A fleet of polling workers takes it to the model under a strict JSON schema, and on completion AI.SetRun executes that stored procedure to continue the work — inside the database, where the data already lives. The API only exists so external apps can register the Thread and poll the result.

the hand-off — EXEC @SPName
-- The AI answer just landed as a row. Now, inside the DB:

-- 1 · read the procedure the caller armed in Thread.Data
DECLARE @SPName NVARCHAR(500) =
    (SELECT SPName FROM AI.SPRun
      WHERE ID_Run = @pID_Run AND State = 1);

-- 2 · continue the pipeline where the data already lives
EXEC @SPName @pID_Run;   -- ← your business SP runs on the AI result

The database runs your stored procedure on the AI result — completion is a database event, not an app event.

3
Services (API · Workers · DB)
9
REST endpoints
18 + 20
Tables + stored procedures
4
Polling worker processes
1–5 s
Poll interval
The problem

AI that has to reason over your data usually drags the data to the model.

The business logic that consumes an AI answer lives in SQL Server — campaigns, customer intentions, the rules that fire next. Bolting a separate AI service on top means ETL round-trips, a second state store, and brittle glue: the model answers in one place, the pipeline advances in another, and nothing is transactional or auditable end to end.

Substrata inverts it. The database stays the source of truth and the orchestrator; the model call is the only thing that leaves the data tier — and it leaves asynchronously, so a slow external call never blocks a database connection.

Constraints it had to respect
  • Results must be transactional with the business data.
  • External model calls must never block a SQL connection.
  • Every AI interaction must survive a restart (durable, resumable).
  • Outputs must be structured, not free text.
Architecture

Three tiers, one engine.

The API enqueues, the database orchestrates, the workers execute. Follow a request as it moves through the system.

1 · REST
2 · Thread
3 · poll
4 · infer
5 · write + advance
6 · status
1 · An external app calls the REST API — the only way in.
2 · The API registers a Thread row — its Data column carries the SPName to run once the AI resolves.
3 · The workers poll for pending rows (Complete = 0) — the Thread first, then the Run they create. No message broker.
4 · The worker — the only tier that leaves the data — calls the model with a strict JSON schema.
5 · AI.SetRun writes the result and, from inside the database, EXEC @SPName advances the pipeline.
6 · The external app polls the run status through the API when it needs the answer.
Substrata API

A thin .NET 8 facade. Token auth, Swagger, and Dapper calls to API.* stored procedures — each returns a single JSON string. No AI SDK lives here.

DapperBearer tokenSwaggerFOR JSON
SQL Server engine

The orchestrator. Tables model the AI object graph and a callback registry; AI.Set* procedures write results and dynamically execute the armed business procedure.

state machineSPRun.SPNamelogs.Generalstrict JSON schema
Worker fleet

Four BackgroundService "virtual processes" poll every 1–5 s, call the model with the schema and key handed to them by the poll, and write back. The only tier that touches the model.

BackgroundServicepoll loopOpenAI SDKper-request key
Features

What the engine gives you.

Database as orchestrator

The queue, state machine, and callbacks are SQL rows and procedures — not a separate broker.

SPRun / SPChatCompletions hold the SPName to run on completion.
Run → Stored Procedure hand-off

When the model answers, the write-back procedure executes the next business procedure in the same scope.

EXEC @SPName @pID_Run inside AI.SetRun.
Strict structured output

Every response is contracted with a strict JSON schema, so answers are safe to persist and drive SQL.

ResponseFormat.jsonSchemaIsStrict = 1.
Durable & resumable

Every assistant, thread, and run is a row. Restart a worker and it resumes from Complete = 0.

Poll WHERE Response IS NULL AND Complete = 0.
Per-request provider routing

Model and provider key are resolved per row and handed to the worker — swap providers without redeploying.

Supplier → APIKey joined in the poll payload.
Non-blocking by design

The slow model call runs in the worker, so it never holds a database connection open.

Create-now, poll-later via GET /run/{id_Thread}.
Engineering decisions

Why it's built this way.

01
Make the database the orchestrator, not just storage.
Because the AI reasons over data that already lives here, orchestrating from SQL removes ETL and keeps state transitions atomic and auditable.
02
Delegate inference to a worker fleet, keep the DB thread free.
External model calls are slow and can fail; blocking a SQL connection on them would be a footgun.
03
Completion is a database event.
The logic that consumes an answer belongs next to the data and should run the instant the result lands.
04
Model the AI object graph as tables.
A durable, queryable representation of every interaction survives restarts — no fragile in-memory state.
05
Contract every response with a strict JSON schema.
Structured outputs are safe to persist and drive downstream procedures deterministically, not by parsing free text.
06
Thin API over stored procedures; every SP returns one JSON string.
The database owns the shape via FOR JSON, so the API stays a stateless auth + transport layer.
07
Route providers per request through Supplier / APIKey.
Multiple providers, keys, and models coexist and rotate without a redeploy, and keys stay out of app config.
What I'd harden next
Replace fire-and-forget Parallel.ForEach(async …) fan-out with awaited Task.WhenAll / channels.
Move connection strings and provider keys into a secret store (user-secrets / Key Vault).
Constrain EXEC @SPName beyond the State = 1 allow-list (signed / whitelisted names).
Swap the run-status busy-wait for async polling; consider push over fixed intervals if latency matters.
REST API

The facade, endpoint by endpoint.

Real endpoint shapes; illustrative values. Every call needs Authorization: Bearer <token>.

Operations

What the workers are doing right now.

Pending (Complete = 0)2
ChatCompletions #90344
gpt-4o · intention_v1
queued
Thread #48260 · Data.SPName set
awaiting ThreadWorker
queued
Running1
Run #90311 · Thread 48213
RunWorker · strict schema
model call
Completed → SP fired2
Run #90287 → EXEC biz.OnIntentionClassified
812 ms
SP executed
Run #90280 → EXEC biz.OnOrderParsed
640 ms
SP executed
logs.General
[AI.SetRun] Run 90287 complete → biz.OnIntentionClassified OK (812ms)
The signature flow

Thread → Run → Stored Procedure

Register a thread with an SPName in its Data; when the AI resolves, that stored procedure runs to continue processing — all inside the database. Step through it.

05EXEC @SPName @pID_Run;the pipeline advances inside the database.
The idea

Why let the database drive the AI?

Honest version: the model call runs in a worker, but the database is the engine that orchestrates it.

Why it's powerful
  • Data gravity — the model reasons over rows that already live here; no ETL round-trip.
  • Orchestration by the database — procedures and a callback table are the workflow engine.
  • Transactional & auditable — AI results land with the business data; logs.General records every step.
  • Set-based scale & ops familiarity — batch AI as rows, on intervals the DBA already runs.
Where its limits are
  • Long external calls are pushed to the worker on purpose — never block a DB connection.
  • EXEC @SPName is dynamic procedure execution — allow-listed by State = 1, but a sharp edge.
  • Secrets belong outside the query tier — keys should live in a vault, not a table or config.
  • Poll intervals trade a little latency for a lot of operational simplicity.
AI.SetRun
CREATE PROCEDURE AI.SetRun
    @pID_Run INT, @pResponse NVARCHAR(MAX) = NULL,
    @pMessageResponse NVARCHAR(MAX) = NULL, @pError NVARCHAR(MAX) = NULL
AS
BEGIN
    UPDATE AI.Run SET Complete = 1 WHERE ID_Run = @pID_Run;

    IF @pError IS NULL
    BEGIN
        UPDATE AI.Run
           SET Response = @pResponse, Response_at = GETDATE(),
               MessageResponse = @pMessageResponse
         WHERE ID_Run = @pID_Run;

        BEGIN TRY
            DECLARE @SPName NVARCHAR(500) =
                (SELECT TOP 1 SPName FROM AI.SPRun
                  WHERE State = 1 AND ID_Run = @pID_Run);

            IF @SPName IS NOT NULL
            BEGIN
                DECLARE @StartRun_at DATETIME = GETDATE();
                EXEC @SPName @pID_Run;                      -- ← pipeline advances
                UPDATE AI.SPRun
                   SET State = 0, StartRun_at = @StartRun_at, EndRun_at = GETDATE()
                 WHERE ID_Run = @pID_Run;
            END
        END TRY
        BEGIN CATCH
            INSERT INTO logs.General (Source, LogType, LogMessage)
            VALUES ('AI.SetRun', 'Error', ERROR_MESSAGE());
        END CATCH
    END
END
Stack

Built with

Backend
.NET 8ASP.NET Core Web APIWorker ServiceBackgroundService
Data
SQL ServerDapperADO.NETFOR JSONstored procedures
AI
OpenAI SDKgpt-4ostructured outputsstrict JSON schema
Integrations
Bearer token authSwagger / SwashbuckleMemoryCache
Practices
poll-based asynccallback registrydurable state machineper-request routing
The result

An AI pipeline the database owns end to end.

Every AI interaction is a durable, auditable row; the model call is delegated so nothing blocks; and the business logic runs the instant the answer lands — from inside a stored procedure, transactional with the data it acts on. The API stays thin, the workers stay stateless, and the database stays the single source of truth.

Explore the interactive modules

Everything above is clickable — try the API explorer, step the Run → SP flow, and watch the console.

Back to top