Everything the
library does.

01

Installation

Install core plus the provider you intend to use. Core carries no vendor SDK.

npm install @ts-dspy/core @ts-dspy/anthropic
npm install @ts-dspy/core @ts-dspy/openai
npm install @ts-dspy/core @ts-dspy/gemini

Node.js 22 or newer. Packages ship ESM and CommonJS builds with type declarations for both, so import and require both resolve correctly.

Signatures use decorators, so your tsconfig.json needs experimentalDecorators. Everything else works without it — string signatures are decorator-free.

{
  "compilerOptions": {
    "experimentalDecorators": true,
    "useDefineForClassFields": false,
    "target": "ES2022",
    "module": "ESNext",
    "moduleResolution": "bundler"
  }
}

02

Quick start

Declare what you want back, then ask for it.

import { Signature, InputField, OutputField, Predict, configure } from '@ts-dspy/core'
import { AnthropicLM } from '@ts-dspy/anthropic'

configure({ lm: new AnthropicLM({ apiKey: process.env.ANTHROPIC_API_KEY }) })

class TriageTicket extends Signature {
  static description = 'Classify a support ticket and rate its urgency.'

  @InputField({ description: 'the ticket body' })
  ticket!: string

  @OutputField({ description: 'billing | bug | feature | other' })
  category!: string

  @OutputField({ description: 'urgency from 0 to 1', type: 'number' })
  urgency!: number
}

const triage = new Predict(TriageTicket)
const { category, urgency } = await triage.forward({ ticket })

// urgency is a number here, or forward() threw.
if (urgency > 0.8) escalate(category)

03

Signatures

A signature declares the inputs a task takes and the outputs it must return. There are two forms.

Class form

Decorated fields, with a description per field and an optional type. The description is sent to the model; the type is what the reply is checked against.

class Summarize extends Signature {
  static description = 'Summarize an article for a busy reader.'

  @InputField({ description: 'full article text' })
  article!: string

  @OutputField({ description: 'three sentences at most' })
  summary!: string

  @OutputField({ description: 'key topics', type: 'string[]' })
  topics!: string[]

  @OutputField({ description: 'optional caveat', required: false })
  caveat?: string
}

String form

For short tasks, or when you would rather not use decorators. Types follow a colon.

const rate = new Predict('question -> answer, confidence: number')
const out = await rate.forward({ question: 'When did Apollo 11 land?' })

04

Field types

Every output field is checked against its declared type. Coercion is lenient, because models emit text — what is not lenient is failure.

Commas and a trailing percent sign are stripped before a number is checked, because models write 1,200 and 87% when asked for a figure. That is a deliberate accommodation, not an accident — and it stops well short of guessing at prose.

Enums

A string field asked for a sentiment will accept euphoric as readily as positive. An enum field will not: it names the closed set, validates against it, and — on providers with structured output — puts that set in the schema, so the constraint reaches decoding rather than only the check afterwards. Members are declared on values:

class Classify extends Signature {
  @InputField({ description: 'the review text' })
  review!: string

  @OutputField({
    description: 'overall sentiment',
    type: 'enum',
    values: ['positive', 'negative', 'neutral'],
  })
  sentiment!: string
}

String signatures say the same thing inline. The members are separated by | rather than ,, because commas already separate fields:

const p = new Predict('review -> sentiment: enum(positive|negative|neutral)')

Matching is lenient in the same spirit as the other types —  Positive  is the same answer as positive, and the declared spelling is what you get back. A value outside the set is a ValidationError, not a shrug. An enum declared with no members throws immediately, since it would constrain nothing, and so does a malformed inline declaration such as enum(a|b — quietly falling back to an open string is the one thing this type must never do.

Strict mode and open-ended types

OpenAI’s strict structured output is stricter than JSON Schema itself: every object in the document, nested ones included, must carry additionalProperties: false, and items must be a real schema rather than the empty {} that means “any element”. Both open-ended types are emitted accordingly — an array declares items: { type: 'string' }, and an object declares an empty, closed object.

The array case has a milder version of the same edge. A bare array or list now tells the provider its elements are strings, so a list of figures comes back as ['1', '2'] rather than [1, 2]. Declare number[] or string[] whenever the elements have a type worth naming; validation accepts either way, but only the declared form constrains what the model may emit.

Optional fields are emitted as type: [base, 'null'] rather than being dropped from required. OpenAI documents that form, Anthropic takes plain JSON Schema, and Gemini’s responseJsonSchema documents it too. Gemini’s older responseSchema keyword models nullability with a nullable boolean instead, so a provider pinned to that older shape would have to translate.

05

Modules

A module is the strategy used to fill a signature. All of them accept an optional ILanguageModel as the second constructor argument; without one they use whatever configure() holds.

Predict

One prompt, one completion, one validation. The baseline.

const p = new Predict(Summarize)
const out = await p.forward({ article })

ChainOfThought

Reasons in free text first, then answers with that reasoning in context. Two model calls. The result carries every signature field plus reasoning.

const cot = new ChainOfThought(SolveProblem)
const out = await cot.forward({ problem })
console.log(out.reasoning)  // how it got there
console.log(out.answer)     // validated as declared

RespAct

A reason–act–observe loop over tools you provide. Adds steps to the result. See Tools below.

06

Tools and RespAct

In their simplest form tools are plain functions taking a string and returning a string, or a promise of one. Give each a description — the model picks from those descriptions, so write them as instructions.

import { RespAct } from '@ts-dspy/core'

const agent = new RespAct(
  'question -> answer',
  {
    tools: {
      lookupOrder: {
        description: 'Look up an order by its ID. Input: the order ID.',
        function: async (id) => JSON.stringify(await db.orders.find(id)),
      },
      today: {
        description: 'Return today’s date. Input: ignored.',
        function: () => new Date().toISOString().slice(0, 10),
      },
    },
    maxSteps: 8,
    onEvent: (e) => logger.debug(e),
  }
)

const out = await agent.forward({ question: 'Has order A-4182 shipped?' })
console.log(out.answer, out.steps)

Typed tool arguments

One string is a thin pipe for a tool that really wants three fields. Add a parameters schema — a JSON Schema object, or a Zod schema — and the tool receives named arguments instead. Zod schemas are validated before the tool runs, and a failure comes back to the model as an observation it can correct rather than an exception you have to catch.

import { z } from 'zod'

const agent = new RespAct(
  'question -> answer',
  {
    tools: {
      findFlights: {
        description: 'Find flights between two airports on a date.',
        parameters: z.object({
          from: z.string().describe('departure IATA code'),
          to: z.string().describe('arrival IATA code'),
          date: z.string().describe('ISO date, e.g. 2026-03-14'),
        }),
        function: ({ from, to, date }) => flights.search(from, to, date),
      },
    },
  }
)

Bare functions and { description, function } keep working as they always did; a tool without parameters is still called with a single string.

Native vs. text-mode tool calling

RespAct chooses its path from the model, not from configuration. When the language model reports supportsFunctionCalling: true — every provider in this repo does — the tool schemas travel in the request itself and the model's calls come back as structured data. When it does not, the loop falls back to prompting for Action: / Action Input: lines and parsing them out of the completion, which works on any model that can produce text.

Both paths run the same tools, honour the same repeat guard, and emit the same events, so a program written against one works against the other. That is the point of keeping the text loop: a local model behind an OpenAI-compatible endpoint still runs your agent. Pass forceTextMode: true to pin a tool-capable model to the text loop — useful for comparing the two, or when a particular model's tool mode misbehaves.

onEvent receives each thought, tool call, observation, and validation failure. Route it to your logger for a full trace of a run.

07

Providers

Each vendor provider wraps its vendor's official SDK and reports supportsStructuredOutput: true, so Predict uses the native JSON-schema mode rather than scraping labelled text. OpenAICompatibleLM is the exception — it declares its capabilities from config, because the endpoint behind it might be anything.

Anthropic

import { AnthropicLM } from '@ts-dspy/anthropic'

new AnthropicLM({
  apiKey: process.env.ANTHROPIC_API_KEY,
  model: 'claude-opus-5',   // default
  maxTokens: 16000,
  timeout: 60_000,
  maxRetries: 2,
})

A refusal is surfaced as a typed error rather than empty text: the API returns HTTP 200 with stop_reason: "refusal", which is easy to mistake for a successful empty reply.

OpenAI

import { OpenAILM } from '@ts-dspy/openai'

new OpenAILM({
  apiKey: process.env.OPENAI_API_KEY,
  model: 'gpt-5.2',        // default
  baseURL: 'https://...',  // any OpenAI-compatible endpoint
})

Sampling parameters are sent only when you set them, so reasoning models that reject a non-default temperature work untouched.

OpenAI-compatible endpoints

Ollama, LM Studio, vLLM, Groq, Together, and OpenRouter all speak the OpenAI chat-completions API, so @ts-dspy/openai already talks to them — but the OpenAI defaults are wrong once you leave api.openai.com. OpenAICompatibleLM corrects them: baseURL and model are required, the API key defaults to a placeholder for local servers that want the header but ignore its value, and capabilities are declared rather than assumed.

import { OpenAICompatibleLM } from '@ts-dspy/openai'

new OpenAICompatibleLM({
  baseURL: 'http://localhost:11434/v1',  // required
  model: 'llama3.2',                      // required
  // apiKey defaults to a placeholder Ollama and LM Studio ignore
  supportsStructuredOutput: false,       // default
  maxContextLength: 8192,               // default
})

supportsStructuredOutput is the flag that matters. Predict branches on it: left at OpenAI's true, every call ships a response_format: { type: 'json_schema', strict: true } that most compatible servers reject outright, and the provider looks broken rather than merely unsupported. At false, structured output goes through the prompt-based fallback and the reply is validated against the signature exactly as before.

Those URLs are also exported as OPENAI_COMPATIBLE_BASE_URLS, keyed ollama, lmstudio, vllm, groq, together, and openrouter. The runnable version is examples/ollama-local.ts:

ollama serve      # leave running
ollama pull llama3.2
npm run example:ollama

Note that OPENAI_API_KEY is deliberately not read from the environment here, unlike OpenAILM. baseURL is by definition somewhere other than api.openai.com, so forwarding an OpenAI credential to it would be a leak rather than a convenience — pass the endpoint's own key as apiKey.

Gemini

import { GeminiLM } from '@ts-dspy/gemini'

new GeminiLM({
  apiKey: process.env.GEMINI_API_KEY,
  model: 'gemini-3.5-flash',  // default
  vertexai: false,
})

Writing your own

Extend BaseLM and implement chat() and getCapabilities(). Usage accounting, generate() delegation, and a prompt-based generateStructured() fallback come with the base class.

import { BaseLM, type ChatMessage, type ModelCapabilities } from '@ts-dspy/core'

class LocalLM extends BaseLM {
  async chat(messages: ChatMessage[]): Promise<string> {
    const res = await fetch('http://localhost:11434/api/chat', { /* ... */ })
    return (await res.json()).message.content
  }

  getCapabilities(): ModelCapabilities {
    return { supportsStructuredOutput: false, supportsStreaming: false,
             supportsFunctionCalling: false, supportsVision: false,
             maxContextLength: 8192, supportedFormats: ['text'] }
  }
}

08

Validation and errors

Every reply is checked before it reaches you. A field that cannot satisfy its declared type throws ValidationError naming the field, rather than handing back a string where the type says number.

import { ValidationError, LMError } from '@ts-dspy/core'

try {
  const out = await triage.forward({ ticket })
} catch (err) {
  if (err instanceof ValidationError) {
    err.issues     // [{ field, expected, received, message }]
    err.rawOutput  // exactly what the model said
    metrics.increment('llm.validation_failed', { field: err.issues[0].field })
  } else if (err instanceof LMError) {
    err.cause      // the provider SDK's own error
  }
}

A validation failure is usually a prompt problem, not a bug. The field description is the lever: 'urgency from 0 to 1' gets a number far more reliably than 'urgency'.

The error taxonomy

A failed provider call throws something more specific than LMError whenever the SDK gives us enough to tell — so retry logic is a catch on a class, not a sniff at an HTTP status number. Every one of these still extends LMError, so existing handlers keep working unchanged.

import { RateLimitError, ContextLengthError, LMError } from '@ts-dspy/core'

try {
  return await triage.forward({ ticket })
} catch (err) {
  if (err instanceof RateLimitError) return queue.retryLater(ticket)
  if (err instanceof ContextLengthError) return triage.forward({ ticket: truncate(ticket) })
  if (err instanceof LMError) log.error(err.provider, err.status, err.cause)
  throw err
}

Content filtering is a 200

None of the three providers throws when a reply is filtered — all three answer 200 OK and leave a marker in the body. Anthropic sets stop_reason: 'refusal', Gemini sets promptFeedback.blockReason or finishReason: 'SAFETY', and OpenAI sets finish_reason: 'content_filter'. Each provider inspects the response and raises ContentFilterError, so a declined request never comes back to you as a silently empty string.

AnthropicRefusalError was the older, Anthropic-only name for this. It is now a deprecated alias of ContentFilterError — an alias of that class, not a subclass of it, so an instanceof check under the old name now matches an OpenAI or Gemini filter too. Test err.provider === 'anthropic' if you need to tell them apart, and prefer the new name.

09

Streaming

There are two levels. A module streams fields, so a value fills in as its tokens arrive and is still validated at the end; a provider streams tokens, which is raw text and validates nothing.

Streaming a module

stream() runs the same prediction as forward(). Every yield is a snapshot of the output fields parsed so far, so the object you render grows rather than appearing all at once.

const qa = new Predict(AnswerQuestion)

for await (const partial of qa.stream({ question })) {
  render(partial.answer)  // 'Par' → 'Paris' → 'Paris, France'
}

The last yield is the complete output, validated against the signature exactly as forward() validates it — a stream that ends in something the signature rejects still throws a ValidationError, from the final pull. Streaming is not an opt-out from the runtime checks. The generator's return value is the Prediction wrapper, which for await discards; reach it by driving the iterator yourself.

Only that last snapshot is guaranteed to match the declared types. Coercion belongs to validation, so a field declared number may still be the raw '0.' the model is part-way through writing — which is why a snapshot is typed as PartialOutput<T>, every field optional and possibly still a string.

const stream = qa.stream({ question }, { signal: controller.signal })

let step = await stream.next()
while (!step.done) {
  render(step.value.answer)
  step = await stream.next()
}

step.value.confidence  // the validated Prediction

ChainOfThought streams too. The reasoning step has to finish before the answer step can start, so nothing is yielded until it has; from then on every snapshot carries the finished reasoning alongside the fields filling in.

Reading truncated JSON

The parser behind the structured path is exported, dependency-free and pure. Give it a JSON document cut off anywhere — mid-string, mid-key, after a trailing comma, halfway through an escape — and it returns the fields that are unambiguously there, without throwing. A number or keyword cut off at the end is dropped, because 1 may still become 12.

import { parsePartialJson } from '@ts-dspy/core'

parsePartialJson('{"answer": "Par')              // { answer: 'Par' }
parsePartialJson('{"a": 1, "b": 2,')             // { a: 1, b: 2 }
parsePartialJson('{"a": "he said \\"hi')         // { a: 'he said "hi' }
parsePartialJson('{"score": 0.9')                // {} — 0.9 may become 0.95

Streaming a provider

One level down, all three providers yield raw chunks. This path bypasses signature validation by design — you are consuming tokens before there is a complete reply to check — so use it for display, and use a module when you need the parsed object.

const lm = new AnthropicLM({ apiKey })

for await (const chunk of lm.generateStream('Explain B-trees.')) {
  process.stdout.write(chunk.content)
  if (chunk.done) console.log(chunk.usage)
}

10

Timeouts and retries

LLMCallOptions flows from a module through to the provider SDK, which owns retries and honours retry-after. There is no second retry layer stacked on top, so you never get two loops fighting each other — with one documented exception on Gemini, below.

const out = await triage.forward(
  { ticket },
  { timeout: 30_000, retries: 3, maxTokens: 2000 }
)

Cancellation

A timeout ends a call that is taking too long. signal ends one whose answer nobody is waiting for any more — a React component that unmounted, a server request whose client hung up. Pass any AbortSignal and the call rejects the moment it aborts, instead of running on to a reply that gets thrown away.

const controller = new AbortController()
// React: return () => controller.abort() from the effect.

const out = await triage.forward(
  { ticket },
  { signal: controller.signal, timeout: 30_000 }
)

signal and timeout compose: pass both and whichever fires first ends the call. OpenAI and Anthropic hand each to the SDK separately; Gemini exposes a single abortSignal slot, so ts-dspy folds the two into one with AbortSignal.any(), freshly per request.

Provider defaults

The same two knobs are available at construction on all three providers, and per-call options override them. GeminiLM accepted neither before v0.6, and ignored retries on the call options as well.

new OpenAILM({ apiKey, timeout: 20_000, maxRetries: 2 })
new AnthropicLM({ apiKey, timeout: 20_000, maxRetries: 2 })
new GeminiLM({ apiKey, timeout: 20_000, maxRetries: 2 })

Usage accounting

Every provider tracks tokens across calls.

lm.getUsage()   // { promptTokens, completionTokens, totalTokens, requestCount }
lm.resetUsage()

There is no cost estimate. Prices change faster than a library can track them, and a stale multiplier is worse than no number — earlier versions reported figures roughly twenty times off.

11

Testing

Modules take a language model, so a test needs no network and no API key. The doubles ts-dspy tests itself with ship alongside it, under the @ts-dspy/core/testing subpath — a separate entry point, so nothing in it is pulled into your production bundle.

import { MockLM, CassetteLM } from '@ts-dspy/core/testing'

Scripted replies

MockLM answers with the strings you hand it, in order, and records every call it received — including the prompt the module built, which is usually the part worth asserting on.

import { MockLM } from '@ts-dspy/core/testing'

it('asks about the ticket it was handed', async () => {
  const lm = new MockLM({ responses: ['category: bug\nurgency: 3'] })

  const result = await new Predict(TriageTicket, lm).forward({ ticket: 'login broken' })

  expect(result.urgency).toBe(3)
  expect(lm.lastPrompt()).toContain('login broken')
})

Pass structuredResponses alongside capabilities: { supportsStructuredOutput: true } to exercise the native structured-output path instead, and read lm.calls or lm.structuredCalls for the options each call carried. chatStream and generateStream replay the same scripted text in chunks.

Cassettes

A cassette records real provider replies once and replays them ever after. Capture them with a key on your own machine, commit the file, and CI runs the same test with no key, no network, and no flake.

// Once, against the real provider:
const recorder = CassetteLM.record('cassettes/triage.json', new OpenAILM({ apiKey }))
await new Predict(TriageTicket, recorder).forward({ ticket: 'login broken' })

// Everywhere else, ever after:
const lm = CassetteLM.replay('cassettes/triage.json')

What it writes is a plain array of entries, each keyed by a hash of its request, so a changed prompt reviews as a readable diff rather than an opaque blob.

[
  {
    "key": "6f1c3b0a9e7d5241",
    "request": {
      "kind": "chat",
      "model": "gpt-4o-mini",
      "messages": [{ "role": "user", "content": "Ticket: login broken\n..." }],
      "options": { "temperature": 0 }
    },
    "response": "category: bug\nurgency: 3"
  }
]

The key covers the messages, the model name, and the sampling options that change a reply. timeout and retries are left out, so re-running a test with a longer deadline still hits the recording. Identical requests replay in the order they were recorded, and an unrecorded one fails loudly rather than calling out.

Rolling your own

Anything implementing ILanguageModel will do; extend BaseLM to inherit the usage accounting and the structured-output fallback.

import { BaseLM, Predict } from '@ts-dspy/core'

class StubLM extends BaseLM {
  constructor(private reply: string) { super('stub', 'stub-model') }
  async chat() { return this.reply }
  getCapabilities() {
    return { supportsStructuredOutput: false, supportsStreaming: false,
             supportsFunctionCalling: false, supportsVision: false,
             maxContextLength: 4096, supportedFormats: ['text'] }
  }
}

it('rejects a non-numeric urgency', async () => {
  const lm = new StubLM('category: bug\nurgency: very high')
  await expect(new Predict(TriageTicket, lm).forward({ ticket: 'x' }))
    .rejects.toThrow(ValidationError)
})

Testing the failure path matters more than the happy path. A model that returns the right shape is not the interesting case.

12

API reference

Exports from @ts-dspy/core

configure()

configure({
  lm: new AnthropicLM({ apiKey }),
  cache: false,
  tracing: false,
})

13

Migrating to 0.5

0.5.0 makes the type guarantee real, which means code that relied on the old silent behaviour now throws.

14

Tracing and history

When a signature misbehaves the first question is always the same: what prompt did it actually send? Turn tracing on and every module invocation records one — the prompt, the raw reply, what was parsed out of it, the tokens it cost, and how long it took.

import { configure, inspectHistory, Predict } from '@ts-dspy/core'

configure({ lm, tracing: true })

const out = await new Predict(TriageTicket).forward({ ticket })

out.trace?.rawLMInput    // the prompt as sent
out.trace?.rawLMOutput   // the reply, before parsing
out.trace?.usage         // tokens for this call, not the model's lifetime

The same entries land in a ring buffer, so you can read them after the fact — including from a catch block, where the prediction you would have inspected does not exist.

try {
  await triage.forward({ ticket })
} catch (error) {
  const [failed] = inspectHistory(1)
  console.error(failed.rawLMInput, '\n---\n', failed.rawLMOutput)
}

inspectHistory(n) returns the last n entries, oldest first; omit n for everything still retained. The buffer holds 100 entries by default — raise or lower it with traceHistorySize — and clearHistory() empties it, which is what you want between tests.

What an entry holds

ChainOfThought makes two calls and RespAct makes one per step, so calls is where a multi-step module becomes readable. Failures are recorded too — a ValidationError is precisely when the prompt is worth reading.

Forwarding entries

onTrace receives each entry as it is recorded. Send them to Langfuse, an OpenTelemetry span, or your own logger; the library itself never prints.

configure({
  lm,
  tracing: true,
  onTrace: (entry) => span.addEvent('lm.call', {
    module: entry.moduleId,
    tokens: entry.usage.totalTokens,
    ms: entry.duration,
  }),
})

15

Caching

Repeated prompts are the common case — a test suite re-running the same example, an optimiser sweeping a signature, a retry loop that re-asks a question it already asked. configure({ cache: true }) replays the previous answer instead of paying for it again.

import { configure, MemoryCache } from '@ts-dspy/core'

// Process-wide LRU, 1000 entries.
configure({ cache: true })

// Or size it yourself.
configure({ cache: new MemoryCache({ maxSize: 10_000 }) })

What goes into a key

generate, chat, and generateStructured are all cached, for every provider, because the wrapping lives in BaseLM. The key is a SHA-256 hash of everything that can change the answer.

Prompts and schemas run to tens of kilobytes, so the canonical form is hashed rather than stored. Errors are never cached: a transient 429 must not pin a failure to a prompt for the life of the process.

Provider and model alone do not identify a request. A client built with a default maxTokens, non-default safety settings, or a baseURL pointing at a proxy answers the same prompt differently, so a custom provider extending BaseLM should return those options from cacheScope() and keep its entries apart from a differently configured sibling.

Usage accounting

A cache hit costs nothing, and getUsage() says so. Hits land in a new cacheHits counter and are kept out of requestCount and the token totals, so a figure you multiply by a published price still reflects real provider traffic.

configure({ cache: true })

await lm.generate('Capital of France?')
await lm.generate('Capital of France?')

lm.getUsage()
// { requestCount: 1, cacheHits: 1, promptTokens: 10, completionTokens: 5, ... }

Bring your own store

cache accepts an implementation as well as a boolean. Both methods may be async, so Redis, SQLite, or a directory of files fits without a wrapper — useful when the cache should outlive the process or be shared across workers.

import { configure, type Cache } from '@ts-dspy/core'

const redisCache: Cache = {
  async get(key) {
    const hit = await redis.get(key)
    return hit === null ? undefined : JSON.parse(hit)
  },
  async set(key, value) {
    await redis.set(key, JSON.stringify(value), { EX: 86_400 })
  },
}

configure({ cache: redisCache })

16

Self-repair

A model that fails validation has usually understood the task and merely fumbled the shape — a number written as "very high", a required field left off the end. Set repairAttempts and ts-dspy spends that many extra round-trips telling the model exactly what went wrong before it gives up.

const qa = new Predict(AnswerQuestion, lm)

// One extra round-trip if the first response fails validation.
const result = await qa.forward(
  { question: 'Capital of France?' },
  { repairAttempts: 1 }
)

The default is 0, so behaviour is unchanged unless you ask for repair. Once the attempts are spent the last ValidationError is rethrown, carrying the usual issues and rawOutput.

What the model is told

The follow-up prompt names every failing field with its declared type and the value that actually arrived, so the correction is specific rather than a bare “try again”. For a confidence field declared number that came back as prose, the model is sent:

// Your previous response failed validation for: confidence.
//   - confidence: expected number, received "very high"
//
// Previous response:
// answer: Paris
// confidence: very high
//
// Reply again with every required field on its own "field: value" line,
// using the declared type for each.

Each repair is built from the original prompt rather than the previous repair prompt, so a multi-attempt loop does not accumulate every earlier correction and drift off task.

Where it applies

Both of Predict's paths are covered: the provider's native structured-output mode and the labelled-text fallback. The structured path asks for a corrected object instead of labelled lines, but the field detail is identical.

ChainOfThought inherits it. Repair applies to the answering step only — the reasoning is already settled, so it is reused rather than regenerated, and a repair costs one call rather than two.

const cot = new ChainOfThought(Scored, lm)

// Reason once, then answer - retrying only the answer.
await cot.forward({ question: q }, { repairAttempts: 2 })

RespAct has always repaired a malformed Final Answer inside its own reasoning loop, bounded by maxSteps rather than by repairAttempts. All three modules now build that corrective prompt from the same helper, so the wording never diverges.

17

Batch & concurrency

Every module inherits batch(), so Predict, ChainOfThought and RespAct all take a list of inputs and run them through a bounded worker pool. Eight calls in flight by default.

const rows = tickets.map((ticket) => ({ ticket }))
const results = await triage.batch(rows, { concurrency: 16 })

const done = results.filter((r) => r.status === 'fulfilled')
console.log(`${done.length} of ${results.length} classified`)

Results come back in input order, whatever order the calls actually finished in. This is the detail a hand-written loop gets wrong: Promise.all over fixed-size slices keeps the order but stalls each slice on its slowest call, and a queue that pushes results as they settle quietly loses the correspondence between row and answer.

One bad row does not kill the job

Each input settles independently, in the shape of Promise.allSettled. A model that returns nonsense for row 4 812 of 10 000 costs you row 4 812, not the run.

for (const [i, result] of results.entries()) {
  if (result.status === 'fulfilled') {
    save(rows[i], result.value.category)
  } else {
    // result.reason is the ValidationError or LMError that was thrown
    quarantine(rows[i], result.reason)
  }
}

Pass stopOnError: true when a single failure means the whole run is worthless — the batch then rejects and starts no further inputs. Calls already in flight are still awaited, so nothing is left running behind your back, and the rejection carries the lowest-indexed failure rather than whichever one happened to land first.

Options

Everything else — temperature, timeout, retries, model — is passed through to every underlying call unchanged.

const controller = new AbortController()
setTimeout(() => controller.abort(), 60_000)

await triage.batch(rows, {
  concurrency: 16,
  signal: controller.signal,
  onProgress: (done, total) => bar.update(done / total),
  temperature: 0,
})

The pool on its own

The primitive underneath is exported, so anything else with a rate limit can use it — embedding calls, database writes, an image API.

import { mapWithConcurrency } from '@ts-dspy/core'

const embeddings = await mapWithConcurrency(
  documents,
  (doc) => embed(doc),
  { concurrency: 4, onSettled: (done, total) => log(done, total) }
)

Same guarantees: input order, per-item settlement, a hard ceiling on in-flight work, and no dependency to install — it is roughly eighty lines of core.

18

Zod signatures

A third signature form, built from zod schemas. Decorators record their fields at runtime, so TypeScript never learns them and results come back typed loosely unless you write the output shape out by hand. A zod schema carries that shape in the type system instead — so the result type is inferred exactly, with no type argument and no experimentalDecorators.

import { z } from 'zod'
import { signature, Predict } from '@ts-dspy/core'

const AnalyzeReview = signature({
  description: 'Analyze a product review.',
  input: z.object({ review: z.string() }),
  output: z.object({
    sentiment: z.enum(['positive', 'negative', 'neutral']),
    rating: z.number().int().min(1).max(5),
    themes: z.array(z.string()),
    followUp: z.string().optional(),
  }),
})

const r = await new Predict(AnalyzeReview).forward({ review })

r.sentiment       // 'positive' | 'negative' | 'neutral'
r.themes.join(', ') // string[]
r.followUp?.trim() // string | undefined

Input keys are typed from the input schema too, so a misspelt input is a compile error rather than a field the model never sees. Everything else behaves as it always has: Predict, ChainOfThought and RespAct all accept this form, and decorated classes and string signatures keep working untouched.

What you can express

The zod schema is itself the validator, so every constraint you write is enforced against the model's reply — including the kinds the flat field-type list has no spelling for.

Coercion still applies

Models emit text, so the text path coerces before your schema sees the value — "42" satisfies a number field and "a, b" satisfies a string[], exactly as with the other two forms. Coercion happens field by field on the way in; your schema is used verbatim, so defaults, optionality and refinements all survive.

const out = parseOutput(AnalyzeReview, 'sentiment: positive\nrating: 5\nthemes: price, build')
// { sentiment: 'positive', rating: 5, themes: ['price', 'build'] }

The schema sent to the provider

When the provider supports native structured output, the zod schema is converted with z.toJSONSchema() and then rewritten for OpenAI's strict mode: every property listed in required, additionalProperties: false on every object including nested ones, and optional fields expressed as [base, 'null'] rather than left out of required.

{
  "type": "object",
  "properties": {
    "sentiment": { "type": "string", "enum": ["positive", "negative", "neutral"] },
    "rating": { "type": "integer", "minimum": 1, "maximum": 5 },
    "themes": { "type": "array", "items": { "type": "string" } },
    "followUp": { "type": ["string", "null"] }
  },
  "required": ["sentiment", "rating", "themes", "followUp"],
  "additionalProperties": false
}

A runnable version of everything above lives in examples/zod-signature.ts:

npm run example:zod

19

Images and multimodal

All three providers read images, and until now there was no way to send one. ChatMessage.content was a string and every converter passed it straight through, so supportsVision: true was a claim the library could not honour. Content is now string | ContentPart[] — a plain string still means exactly what it always did, and an array carries text and images in order.

import { imagePart, textPart, type ChatMessage } from '@ts-dspy/core'

const messages: ChatMessage[] = [
  { role: 'user', content: [
    textPart('What does this sign say?'),
    imagePart('data:image/png;base64,iVBORw0KGgo…'),
  ] },
]

await lm.chat(messages)

imagePart() accepts an https:// URL, a data: URI, or an explicit source such as { kind: 'base64', data, mediaType }. A bare base64 blob is refused: nothing in it says whether the bytes are a PNG or a JPEG, and every provider insists on being told.

Image input fields

A signature can declare an input as an image — with @ImageField, or the image type in a string signature. buildPromptContent() then renders the prompt as content parts, with the image sitting where its label falls.

import { Signature, ImageField, OutputField, buildPromptContent } from '@ts-dspy/core'

class ReadSign extends Signature {
  static description = 'Read the sign in the photo.'

  @ImageField({ description: 'photo of the sign' })
  photo!: string

  @OutputField({ description: 'the words on the sign' })
  words!: string
}

const content = buildPromptContent(ReadSign, { photo: dataUri })
await lm.chat([{ role: 'user', content }])

// string signatures take a type too
buildPromptContent('photo: image, question -> answer', { photo, question })

What each provider sends

A data: URI handed to Anthropic or Gemini as a URL is rewritten to inline bytes, because neither can dereference one. Gemini will not fetch an arbitrary web URL at all — fileData resolves only a Files API or Cloud Storage URI — so anything else is refused with an LMError rather than sent on to earn a 400. detail is an OpenAI-only fidelity hint; the others ignore it.

Roles and alternation

Only a user turn may carry an image. OpenAI's message type is a discriminated union in which system and assistant accept text alone, and Anthropic's system and Gemini's systemInstruction are top-level strings — so an image addressed to any of those is flattened to its placeholder rather than dropped without trace.

Anthropic additionally requires strict user/assistant alternation, so consecutive same-role turns are merged. That merge now concatenates block arrays: it used to run only when both turns were strings, which meant a text turn followed by an image turn was sent as two adjacent user messages and rejected outright. Merged content is therefore a block array, with two adjoining text blocks folded into one so the blank line between the turns survives.

20

Evaluation

A prompt change either helped or it did not, and the only way to know is to measure. evaluate runs a program over a dataset, grades every prediction with a metric you choose, and hands back one number plus the working behind it.

import { evaluate, exactMatch, Example, Predict } from '@ts-dspy/core'

const dataset = [
  new Example({ question: 'Capital of France?', answer: 'Paris' }).withInputs('question'),
  new Example({ question: 'Capital of Japan?', answer: 'Tokyo' }).withInputs('question'),
]

const report = await evaluate(new Predict(AnswerQuestion), dataset, exactMatch, {
  concurrency: 8,
})

report.score    // 0.5 — the mean across every example
report.results  // per example: inputs, expected, prediction, score, error?
report.usage    // tokens and latency for this run only

Datasets

A dataset is an array of Example. withInputs() names the input fields; everything else is what the program is expected to produce, which is exactly the split an evaluation needs. Pass inputKeys instead if the whole dataset shares one shape.

const dataset = rows.map((row) => new Example(row))

await evaluate(program, dataset, exactMatch, { inputKeys: ['question'] })

Metrics

A metric is (example, prediction) => number | boolean, awaited if it returns a promise. That is the whole contract, and it is deliberately small: the same function works unchanged as an optimiser's objective.

Anything else is a function you write.

const mentionsSource: Metric = (example, prediction) =>
  String(prediction.get('answer')).includes(example.get('citation'))

const report = await evaluate(program, dataset, mentionsSource)

Read the prediction through get() or toObject(). Prediction defines its fields with Object.defineProperty, so spreading it is not the same thing.

The report

Usage comes from diffing the model's own counters around the run, so it counts this evaluation and nothing that came before it. There is no cost figure, for the reason given under usage accounting.

Failures are results

An example whose program or metric throws is recorded as a zero with the error attached, and the run carries on. An evaluation that dies on row 40 of 500 tells you nothing at all.

const failures = report.results.filter((result) => result.error)

failures[0].error.message
failures[0].inputs       // what was fed in, to reproduce it

Printing

The library never writes to a console. formatReport returns a string, and where it goes — a terminal, a log line, a CI annotation — is your decision.

import { formatReport } from '@ts-dspy/core'

console.log(formatReport(report, { maxRows: 20, includePassing: false }))

// score 0.800  (4.00 / 5)  errors 1
// tokens 3120 (2480 prompt + 640 completion)  requests 5  ...
//
// #  score  detail
// 4  0.000  Error: 429 rate limit exceeded

21

Few-shot and optimizers

Everything above writes prompts by hand. An optimizer writes them from data instead: give it labelled examples and a metric, and it works out which worked examples belong in the prompt. This is the idea TS-DSPy takes from DSPy — a program that improves itself, rather than a prompt you keep editing.

Demos

A demo is an Example rendered into the prompt before the real input, showing the task performed correctly. Demos render in whichever shape the reply is expected to take — labelled field: value for a provider answering in text, JSON for one whose decoding is constrained to a schema — so they teach the output format as well as the task.

import { Example, Predict } from '@ts-dspy/core'

const demos = [
  new Example({ ticket: 'My card was charged twice.', team: 'billing' }).withInputs('ticket'),
  new Example({ ticket: 'Export does nothing on Safari.', team: 'bug' }).withInputs('ticket'),
]

const router = new Predict(RouteTicket, { demos })

withInputs() marks which fields are the question; the rest is the answer. Omit it and the signature decides the split instead. The original two-argument form, new Predict(Sig, lm), still works — pass { lm, demos } when you want both. withDemos() returns a configured copy, leaving the module you called it on alone.

LabeledFewShot

Selects k of your own labelled examples. No model calls, so compiling is instant and free — try this before bootstrapping.

import { LabeledFewShot } from '@ts-dspy/core'

const compiled = new LabeledFewShot({ k: 3, seed: 42 })
  .compile(new Predict(RouteTicket), { trainset })

BootstrapFewShot

Runs the module over the trainset, scores every attempt with your metric, and promotes the runs that passed into demos. The program learns from its own successes.

import { BootstrapFewShot } from '@ts-dspy/core'

const optimizer = new BootstrapFewShot({
  metric: (example, prediction) => example.get('team') === prediction.get('team'),
  maxBootstrappedDemos: 4,
  teacher: strongLM,        // generates the demos, at compile time only
  concurrency: 4,
  seed: 42,
  onProgress: (event) => log(`${event.index + 1}/${event.total} ${event.status}`),
})

const compiled = await optimizer.compile(new Predict(RouteTicket, cheapLM), { trainset })

With a teacher, a stronger model does the trainset runs and the cheaper student ends up imitating its work. You pay for the strong model once, when compiling — never at run time. Without one, the student bootstraps from itself.

The run stops as soon as it has maxBootstrappedDemos, so a trainset of five hundred rows does not cost five hundred calls to keep four demos. Setting maxBootstrappedDemos: 0 alongside maxLabeledDemos compiles from labels alone and makes no model calls whatsoever.

Metrics

A metric is (example, prediction) => number | boolean, and may be async. Booleans are taken at face value; a number counts as a pass when it reaches threshold, which defaults to 0.5. The metric is the entire definition of “good” — everything the optimizer does follows from it.

Options

A full run, including a baseline to compare against, is in examples/optimizer.ts:

export OPENAI_API_KEY="sk-..."
npm run example:optimizer