Prompt Chaining and Multi-Step Workflow Orchestration

"Help me research X, create an outline, write a 3,000-word article, and then polish it." Put a task like this into a single LLM call, and you will probably get something mediocre at everything: shallow research, a disorganized outline, unfocused prose, and polishing that changes almost nothing. Complex tasks cannot be solved merely by writing a longer prompt; they must be split into a multi-step chain so that each step does one thing and wears one hat.

1. Why a Single Call Is Not Enough

A single call has three limitations, none of which can be solved by simply "adding tokens":

1. Task complexity overflow. When one prompt simultaneously asks for retrieval, reasoning, writing, and self-review, every additional objective dilutes the model's attention to each objective. In practice, "doing everything at once" almost always produces work that scores about 70 out of 100 on every dimension.
2. Context contamination. If research material, the outline, the draft, and revision notes are all piled into one context, the model may copy earlier filler and forget later constraints. Long context does not equal long memory; information in the middle gets "submerged."
3. Role conflict. A researcher must diverge, an editor must converge, and a proofreader must find faults. These instructions conflict with one another, so placing them in one prompt forces the model into weak compromises.

The essence of chaining is to split one high-variance task into several low-variance subtasks, each of which can be tested, tuned, and assigned to a different model independently.

2. The Four Basic Chain Patterns

Every prompt chain is a combination of these four patterns.

Sequential       A → B → C → D
Parallel fan-out A → [B,C,D in parallel] → merge
Router           A → (condition) → B | C | D
Loop             A → B → (satisfied?) ─ No ─→ B ...
                         └ Yes ─→ Output

Sequential chain: the most common pattern. The output of one step becomes the input to the next, as in "translate → proofread → format." It suits workflows with fixed steps and clear dependencies.

Parallel chain (fan-out): derives several independent subtasks from one step, runs them in parallel, and then aggregates the results. For example, "generate a summary for each of 10 chapters, then merge them into an overall outline." It offers low latency and high throughput, but aggregation can lose details, so it suits tasks whose results can be combined.

Routing chain: selects a branch based on the input. For example, "classify first: send technical questions to an expert prompt, casual conversation to a lightweight prompt, and policy violations to a refusal." This saves tokens because simple tasks avoid large models and improves quality because each branch uses a specialized prompt.

Loop chain: iterates until an exit condition is met. For example, "generate → self-evaluate → not good enough → regenerate using the critique." It has the highest quality ceiling, but its latency and cost can become unpredictable, so it must have a hard limit of at most N rounds.

A minimal skeleton:

type Step<I, O> = (input: I) => Promise<O>;

// Sequential
const chain = async (topic: string) => {
  const research = await researchStep(topic);
  const outline  = await outlineStep(research);
  const draft    = await draftStep(outline);
  return polishStep(draft);
};

// Parallel fan-out + merge
const summarizeMany = async (chunks: string[]) => {
  const parts = await Promise.all(chunks.map(summarizeStep));
  return mergeStep(parts);
};

// Routing
const route = async (q: string) => {
  const kind = await classifyStep(q);          // 'tech' | 'chat' | 'block'
  return kind === 'tech'  ? expertStep(q)
       : kind === 'chat'  ? lightStep(q)
                          : refuseStep();
};

// Loop (with a hard limit)
const refine = async (topic: string) => {
  let draft = await draftStep(topic);
  for (let i = 0; i < 3; i++) {
    const critique = await critiqueStep(draft);
    if (critique.score >= 8) break;
    draft = await reviseStep(draft, critique);
  }
  return draft;
};

3. Chains vs. Agents: Do Not Repackage a Chain as an Agent

Chains and Agents are often conflated, but their engineering tradeoffs are fundamentally different:

• A chain is a deterministic workflow: you define each step, its order, and the data it receives when writing the code. The model is responsible only for "doing this step well," not for "deciding what to do next." Chains are predictable, replayable, and easy to debug.
• An Agent makes autonomous decisions: the model decides which tool to call, whether to search again, and when to stop based on the situation. It is flexible but unpredictable, difficult to debug, and expensive.

The rule of thumb is: the more deterministic the process, the more it should use a chain; the more open-ended the exploration, the more it should use an Agent. "Research → outline → draft → polish" is deterministic, so use a chain. "Find the root cause of this bug; inspect the code if needed and run tests if needed" is open-ended, so use an Agent.

Many people start with an Agent, only to have the model spend tokens on "deciding what to do next" every round while taking unnecessary detours. First lock down the deterministic workflow with a chain, then introduce an Agent only at branches that genuinely require decisions. This offers the best cost-performance balance. One compromise is to use a chain as the skeleton and place an Agent inside one node, such as allowing the "research" step to retrieve information autonomously, while keeping the other nodes simple and deterministic.

4. Intermediate State: Pass Structured Data, Not Concatenated Natural Language

The easiest place for a chain to fail is how one step passes its output to the next. A beginner's approach is to "concatenate A's natural-language output directly into B's prompt," which introduces a high-noise channel between steps.

// ❌ Bad example: natural-language concatenation is noisy and cannot be parsed reliably
const outline = await llm(`Write an outline based on this research: ${researchText}`);
const draft   = await llm(`Write the article based on this outline: ${outline}`);

// ✅ Good example: pass a structured schema, and let B read only the fields it needs
type Research = { keyFacts: string[]; sources: {title:string;url:string}[]; gaps: string[] };
type Outline  = { sections: { heading: string; points: string[] }[] };

const research: Research = await researchStep(topic);     // Enforce a JSON schema
const outline:  Outline  = await outlineStep(research);   // Read only research.keyFacts
const draft              = await draftStep(outline);

Structured handoffs provide three immediate benefits: step B's prompt can include only the fields it needs instead of receiving everything, intermediate results can be persisted and replayed so one failed step does not require restarting the entire chain, and each step can be evaluated independently.

Context trimming is an underrated optimization. Do not dump the entire 5KB of source material produced during research into the writing step. Writing needs only keyFacts, while proofreading needs only draft. Give each step the minimum context it actually needs. Quality improves because the model is less distracted, and fewer tokens are consumed.

5. Error Recovery: Every Step Can Fail

Once a chain has N steps, its overall failure probability is 1 - (1-p)^N. With a 2% failure rate per step, a ten-step chain has an 18% chance that at least one step will fail. A chain without error recovery is not production-ready.

Every step needs an explicit failure strategy, selected from these four options or a combination of them:

• Retry: retry transient failures such as network issues or JSON parsing errors 2–3 times with exponential backoff. Change the temperature or include feedback explaining the previous error; otherwise, the same input may produce the same failure.
• Degrade gracefully: use a smaller model if the large model is unavailable, use a regex fallback if JSON parsing fails, or use a cached result if the primary path fails. The resulting quality degradation must be acceptable.
• Skip: if a noncritical step such as "image suggestions" fails, skip it and continue the main workflow.
• Human fallback: if a high-value or irreversible step such as final publication fails, send it to a human review queue.

const withRetry = async <T>(fn: () => Promise<T>, times = 3): Promise<T> => {
  let lastErr: unknown;
  for (let i = 0; i < times; i++) {
    try { return await fn(); }
    catch (e) { lastErr = e; await sleep(2 ** i * 500); }
  }
  throw lastErr;
};

// Idempotency: persist intermediate results and deduplicate by (taskId, stepName, inputHash)
const runStep = async (taskId: string, step: string, input: unknown) => {
  const key = `${taskId}:${step}:${hash(input)}`;
  const cached = await db.get(key);
  if (cached) return cached;              // Skip completed steps during replay
  const out = await withRetry(() => steps[step](input));
  await db.set(key, out);
  return out;
};

Idempotency is the lifeline of a chain. Hash each step's input and persist its output. If any step fails, execution can resume from the checkpoint instead of burning tokens by starting over. This is more valuable than any retry logic.

6. Cost and Latency: The Hidden Bill Behind Chains

Chains are not free. The latency of an N-step sequential chain is the sum of its steps, and its token usage is also the sum of all steps. It can easily cost 3–5 times more than a single call. The available optimizations are straightforward:

1. Parallelize whenever possible. Whenever a sequential chain contains mutually independent subtasks, fan them out with Promise.all. Latency changes from the "sum of sequential tasks" to the "duration of the slowest task."
2. Cache intermediate results. Results such as "research" and "classification" can be reused for identical inputs. The classification step in a routing chain usually has an especially high hit rate.
3. Use smaller models for simple steps. Haiku-tier models are sufficient for routing classification, JSON extraction, and format validation. Reserve Opus/Sonnet for demanding reasoning and writing tasks. Mixing models this way can cut costs by more than half.
4. Prompt caching. Each step's system prompt is usually stable. Enable prompt caching to cache these prefixes and substantially reduce the cost of repeated calls.

In short, evaluate a chain by its total economics: can the gains from higher quality, less rework, and less human intervention outweigh the added token usage and latency? If so, use a chain. If not, stick with a single call.

7. In Practice: A "Research → Outline → Draft → Polish" Content Generation Chain

Putting everything together, this is the content generation chain skeleton I use:

type Ctx = { topic: string; research?: Research; outline?: Outline;
             draft?: string; article?: string };

const generateArticle = async (topic: string): Promise<string> => {
  let ctx: Ctx = { topic };

  // 1. Research: Agent substep with autonomous retrieval (open-ended exploration → use an Agent)
  ctx.research = await runStep(topic, 'research', topic);

  // 2. Outline: deterministic step with trimmed context; pass only keyFacts
  ctx.outline  = await runStep(topic, 'outline', ctx.research.keyFacts);

  // 3. Draft: loop chain with self-evaluation and revision, limited to 2 rounds
  ctx.draft    = await runStep(topic, 'draft',
    { outline: ctx.outline, facts: ctx.research.keyFacts, maxRounds: 2 });

  // 4. Polish: a smaller model is sufficient; use a cheaper model + prompt caching
  ctx.article  = await runStep(topic, 'polish', ctx.draft);
  return ctx.article;
};

The design tradeoffs are immediately clear: the open-ended research step uses an Agent, while the deterministic writing steps use a chain. Every step has structured inputs and outputs, and its results are persisted for replay. The loop has a hard limit, and polishing is delegated to a smaller model. Articles produced by this chain consistently outperform versions that "do everything at once" because each step focuses on one clearly defined objective.

Summary

• When to split a chain: consider a chain when a task contains multiple interfering objectives, its context can become contaminated, or it requires different roles.
• Four patterns: sequential, parallel, routing, and loop. These four building blocks can compose 99% of use cases.
• Chains vs. Agents: use chains for deterministic workflows and Agents for open-ended exploration. Lock down the skeleton with a chain first, then use an Agent only at decision points.
• Intermediate state: pass a structured schema, give each step only the minimum context, and persist intermediate results for replay.
• Error recovery depends on idempotency: deduplicate with (taskId, step, inputHash), then combine retries, graceful degradation, skipping, and human intervention. This is more valuable than any retry logic.
• Calculate the total cost: quality gains must outweigh token and latency costs. Parallelism, caching, and mixed model sizes are the essential combination.

The takeaway: A single call is a wish, a chain is division of labor, and an Agent is delegation. Turn a complex task from "getting lucky once" into "controlling every step," and you truly own the process instead of being led by its output.