User Guide

Delstorm is a multi-agent experimentation platform. You design experiments where AI agents — each with their own persona, backed by an LLM of your choice — engage in structured discourse across configurable rounds.

Each round has a condition prompt that sets the social scenario: a debate topic, a collaborative task, a negotiation, a simulated classroom, or any situation you can describe. Agents respond based on their personas and the conversation history. You control the interaction pattern, number of turns, which agents participate, and how long responses should be.

Use cases include:

• Simulating academic debates grounded in real papers
• Modeling stakeholder negotiations with distinct interests
• Exploring how different ideological perspectives interact on a topic
• Testing how agent personas, temperature, or round structure affect discourse outcomes
• Running batch experiments to observe variance across identical setups
• Generating structured multi-perspective analysis of documents or topics

To run your first experiment, follow these steps:

1. 1.

   Create an account

   Click "Sign In" in the top right. If you don't have an account, click "Create one" to sign up with your email and password. You'll receive a confirmation email — click the link to activate your account, then sign in.

2. 2.

   Check your providers

   Go to Settings. You should see at least one provider already configured (e.g., "claude"). If not, you'll need to add one — see Setting Up Providers below.

3. 3.

   Create an experiment

   Go to New Experiment. Give it a name, select your default provider (e.g., "claude"), enter the default model name (e.g., "claude-sonnet-4-6"), and add at least one agent and one round.

4. 4.

   Launch

   Click "Launch Experiment" and watch the agents respond in real-time. Results appear as they're generated.

Quick test: Try 2 agents, 1 round, "Short" response length. This will finish in under a minute and let you verify everything works before building a larger experiment.

A provider is an LLM backend that powers your agents. Providers are configured in Settings.

Shared vs. Personal Providers

Your administrator may have pre-configured shared providers (like Claude) that are available to all users automatically. These appear in your provider list without any setup. You can also add your own personal providers — these are visible only to you.

Provider Types

Per-Agent Provider Override

By default, all agents in an experiment use the experiment's default provider. However, you can override this on individual agents — for example, to have one agent powered by Claude and another by GPT-4. Set this in the "Provider Override" dropdown on each agent card.

The Experiment Builder is where you design your experiments. It has these sections:

1. 1.

   Load Design (optional)

   If you have saved designs, select one from the dropdown to pre-fill all fields.

2. 2.

   Experiment Info

   Name your experiment, select the default provider (e.g., "claude"), and enter the default model ID (e.g., "claude-sonnet-4-6"). The model ID must match what the provider expects.

3. 3.

   Document Context (optional)

   Upload a document to extract themes and generate grounded agent personas. See Document-Grounded Personas.

4. 4.

   Agents

   Define the participants. Click "Add Agent" to create agent cards. See Agents & Personas.

5. 5.

   Rounds

   Define the sequence of interactions. Click "Add Round" to create round cards. See Rounds, Conditions & Settings.

6. 6.

   Launch / Save / Batch

   "Launch Experiment" runs it once. "Save Design" stores the config for reuse. "Batch Run" runs it multiple times.

Agents are the participants in your experiment. Each agent is defined by these fields:

You can upload a document (PDF, TXT, or MD) to automatically extract themes, positions, and stakeholders, then generate agent personas grounded in the document's actual content.

Step-by-step:

1. 1.

   In the Experiment Builder, go to the "Document Context" section. Upload your file and select which provider to use for analysis (Claude is recommended for best extraction quality).

2. 2.

   Click "Analyze Document". The LLM reads the document and extracts: title, summary, key themes, distinct positions/stances, stakeholders, key arguments, and domain terminology. This takes 10-30 seconds.

3. 3.

   Review the extraction results. The themes, positions, and stakeholders should give you ideas for what agents to create.

4. 4.

   Add agents and give them relevant names and specialties (e.g., if the document is about left-right politics, create "Left_Coalition_Theorist" with specialty "Progressive policy coherence").

5. 5.

   On each agent card, click "Generate Persona from Document". The LLM creates a detailed persona for that specific agent, using the document's themes, arguments, data points, and terminology. The persona is tailored to the agent's name and specialty — not a generic summary.

6. 6.

   Review and edit the generated personas. The generation is a starting point — you can refine it, add specific instructions, or combine it with your own text.

Each round represents a phase of the experiment. Rounds are independent by default — you create continuity through how you write your condition prompts.

The Condition Prompt

This is the most important part of a round. It sets the social scenario that all participating agents operate within. Think of it as setting the stage for a scene.

Round Settings

Each round uses one of five interaction patterns that define how agents communicate:

Tip: You can use different patterns across rounds in the same experiment. Start with Blind Parallel (to capture uninfluenced positions), then Free Discussion (to let agents interact), then Debate (to force a conclusion).

Click "Launch Experiment" to start. You'll be taken to the live view.

The live view shows:

• A progress indicator while the experiment is running
• Live streaming of each agent's response as tokens are generated (you see the text appearing in real-time)
• Color-coded agent names for easy identification
• Round-by-round results with full markdown rendering (headers, bold, lists, etc.)
• An "Export JSON" button for downloading the complete results

Duration estimates:

Batch running lets you execute the same experiment design multiple times to observe how agent behavior varies across identical setups.

How to use:

1. Build your experiment as usual in the experiment builder.
2. Instead of "Launch Experiment", click the purple "Batch Run" button.
3. Enter the number of runs (1-20).
4. A batch progress page shows completion status for each run.
5. Click into individual runs to see their full results.

Each run uses the exact same configuration — same agents, personas, rounds, and conditions — but produces different responses due to LLM sampling variability. Runs execute sequentially (not in parallel) to avoid API rate limits.

Tip: Use "Short" response length for batch runs to save time and API costs. Run 3-5 short batch runs to identify interesting patterns, then do a single thorough run to explore them in depth.

Designs let you save and reuse experiment configurations:

• Save Design: Click the green "Save Design" button in the experiment builder. Your agents (including personas), rounds, and all settings are saved to your account.
• Load Design: At the top of the experiment builder, select a saved design from the dropdown and click "Load". All fields are populated with the saved values.
• Delete Design: Select a design from the dropdown and click "Delete" to remove it.

Click "Export JSON" on any completed experiment to download the full results. The JSON file includes:

• Experiment metadata (name, duration, session ID)
• Agent specifications (names, personas, temperatures, providers)
• All rounds with condition prompts, interaction patterns, and full agent responses
• Turn-by-turn data for multi-turn rounds
• Timing data per round

The exported JSON can be used for further analysis in Python, R, or any data processing tool.

The Evaluate page lets you analyze completed experiments by defining structured evaluation fields and running a hybrid parser + LLM extraction system.

When to Use Evaluation

Evaluation is post-hoc and on-demand — you run it after an experiment completes, not as part of the experiment design. This means you can evaluate the same experiment multiple ways, or decide what to measure after seeing the results.

Access the evaluate page from:

• The "Evaluate" link in the navigation bar
• The "Evaluate" button on any completed experiment's results page
• The "Evaluate All Runs" button on a completed batch page

How It Works

The evaluation system is hybrid — it combines two approaches:

The parser runs first. The LLM runs second, informed by what the parser found. Results are merged with confidence tracking so you know whether each value came from the parser (deterministic), the LLM (inferred), or both (confirmed).

Field Types

Source Filtering

Each field can optionally specify a source round and/or source agent. When set, the parser and LLM only look at that portion of the transcript. This is useful when you have a dedicated evaluation round — you want scores from the evaluator agent's output, not from the full debate.

Evaluation Templates

Save your field definitions as reusable templates. If you always evaluate political debates with the same criteria (Consensus Score, Winner, Key Arguments), save those fields as a template and load it for each new evaluation. Templates are saved to your account.

Batch Comparison

When evaluating multiple experiments (from a batch run), results display as a comparison table — fields as rows, experiments as columns. Numeric fields show an average across runs. This is the core tool for measuring variance: "Did the agents reach consensus more often in runs with lower temperature?"

Example Workflow: Evaluator Agent

A powerful pattern is to include a dedicated evaluator agent in your experiment:

1. Create an agent named "Evaluator" with a persona like "You are an impartial judge who evaluates the quality of discourse."
2. Add a final round with only the Evaluator participating, with a condition like: "Score the preceding discussion on: consensus (1-10), argument quality (1-10), and name the strongest contributor."
3. After the experiment, go to Evaluate and define fields: "Consensus (number, 1-10, source agent: Evaluator)", "Argument Quality (number, 1-10, source agent: Evaluator)", "Strongest Contributor (choice, source agent: Evaluator)".
4. The parser will extract the scores directly from the evaluator's structured output.

Confidence Badges

Delstorm includes a built-in AI assistant that can answer questions about the platform. Look for the blue chat bubble in the bottom-right corner of every page.

Click it to open the chat panel and ask anything about how to use Delstorm — setting up providers, building experiments, configuring agents, using evaluation, or understanding features.

The assistant is powered by Claude and has access to this entire guide as its knowledge base. It provides concise, specific answers with references to relevant features. It will not make up information — if it doesn't know the answer, it will tell you.