Benchmarks & Performance

Performance data for Network-AI deployments. Your swarm is only as fast as the backend it calls — this page helps you choose the right setup.

BlackboardValidator Throughput

Layer 1 validation (rule-based, zero LLM calls) measured on Node.js 20, Apple M2, single-thread:

Layer 2 (QualityGateAgent) adds LLM latency and is async — intended for high-value writes, not every write.

Cloud Provider Performance

Not all cloud APIs perform the same. Model size, inference infrastructure, and tier all affect how fast each agent gets a response — and that directly multiplies across every agent in your swarm.

Key insight: A 5-agent swarm using gpt-4o-mini at 500 RPM can fire all 5 agents truly in parallel and finish in ~4s total. The same swarm on gpt-5.2 at 6 RPM must go sequential and takes 60s. The model tier matters more than the orchestration framework.

Choosing a Model for Swarm Agents

• Speed over depth (many agents, real-time) → gpt-4o-mini, claude-3.5-haiku, gemini-2.0-flash, groq/llama-3.3-70b
• Depth over speed (few agents, high-stakes) → gpt-4o, claude-3.7-sonnet
• Free / no-cost testing → Groq free tier, Gemini free tier, or Ollama locally
• Production with budget → multiple keys across providers, route agents to different models

Rate Limit Patterns

When you run a 5-agent swarm sharing one API key and hit the RPM ceiling, the API silently returns empty responses — not a 429 error, just blank content. Network-AI's swarm demos handle this automatically with sequential dispatch and adaptive header-based pacing (reads x-ratelimit-reset-requests to wait exactly as long as needed).

Multiple Keys = True Parallel

import { CustomAdapter, AdapterRegistry } from 'network-ai';

const registry = new AdapterRegistry();

for (const reviewer of REVIEWERS) {
  const adapter = new CustomAdapter();
  const client  = new OpenAI({ apiKey: process.env[`OPENAI_KEY_${reviewer.id.toUpperCase()}`] });

  adapter.registerHandler(reviewer.id, async (payload) => {
    const resp = await client.chat.completions.create({ /* ... */ });
    return { findings: extractContent(resp) };
  });

  registry.register(reviewer.id, adapter);
}

// All 5 dispatch in parallel via Promise.all — ~8–12s instead of ~60s

Local GPU = Zero Rate Limits

const localClient = new OpenAI({
  apiKey : 'not-needed',
  baseURL: 'http://localhost:11434/v1',   // Ollama, vLLM, llama.cpp
});

adapter.registerHandler('reviewer', async (payload) => {
  const resp = await localClient.chat.completions.create({
    model   : 'llama3.2',
    messages: [/* ... */],
  });
  return { findings: extractContent(resp) };
});

Cloud GPU Instances (Self-Hosted)

Running your own model on AWS / GCP / Azure sits between managed APIs and local hardware:

Cost: $1–5/hr for GPU VMs. For high-volume production swarms or teams that want no external API dependency, it is the fastest architecture available. The connection is identical to local Ollama — just point baseURL at your VM's IP.

max_completion_tokens — The Silent Truncation Trap

One of the most common failure modes in agentic output tasks. When a model hits the max_completion_tokens ceiling it stops mid-output and returns whatever it has — no error, no warning. The API call succeeds with finish_reason: "length" instead of "stop".

This is especially dangerous for code-rewrite agents where the output is a full file.

# Real numbers (gpt-5-mini, order-service.ts rewrite):
  Blockers section:  ~120 tokens
  Fixed code:        ~2,800 tokens  (213 lines with // FIX: comments)
  Total needed:      ~3,000 tokens  ← hits the cap exactly → empty output
  Fix: set to 16,000 → full rewrite delivered in one shot

Rule of Thumb by Task

All GPT-5 variants support 128,000 max output tokens — the ceiling is never the model, it is always the cap you set.

Lessons from Building the Code-Review Swarm