Test Your Web App on localhost or a Preview URL

Capability: Strict mode + output_schema (Strict mode is beta — enroll at agent.tinyfish.ai/beta to use it)

Use this to check a user flow — sign-up, checkout, a settings form — on a build that isn’t live yet. Describe the flow in plain English, TinyAgent clicks through it in a real browser, and you get back pass or fail with a reason. Because that browser runs in TinyFish’s cloud, your app needs a URL the internet can reach: a preview deploy or a tunnel to localhost.

TinyAgent logs in to a demo shop, adds a backpack to the cart, checks out, and confirms the order page — returning passed: true.

Steps

Give the agent a URL it can reach. Use your preview deploy’s URL, or open a tunnel to your local server.
Write the test as steps plus an expected result. Number the steps, use exact values, end with “Expected result: …”, and tell the agent to stop and report if any step can’t be done as written.
Run it in Strict mode with a pass/fail schema. Strict mode stops at the first surprise instead of working around it — which is what you want when the surprise is a bug.
Check passed, not the run status. If it failed, open the run in the dashboard to see what happened at each step.

Get a URL the agent can reach

Preview deploy
localhost

Use the preview URL your host gives you (Vercel, Netlify, etc.). If Vercel Deployment Protection is on, add your bypass secret to the URL — it’s under Project Settings → Deployment Protection → Protection Bypass for Automation:

https://my-app-git-feature.vercel.app/?x-vercel-protection-bypass=YOUR_SECRET&x-vercel-set-bypass-cookie=true

Open a tunnel to your local server, then use the https:// URL it prints:

# Cloudflare (no account needed)
cloudflared tunnel --url http://localhost:3000

# or ngrok
ngrok http 3000

Keep both your app and the tunnel running until the test finishes.

Run the test

Beta: agent_config.mode: "strict" and max_steps are both beta — enroll at agent.tinyfish.ai/beta to use either. A non-beta request returns 403 FORBIDDEN for mode: "strict" just as it does for max_steps; max_duration_seconds alone is generally available, no enrollment needed.

Once enrolled in the beta, this example tests the checkout on a public demo shop, so you can run it as-is. Swap in your own URL and steps.

Python
TypeScript

from tinyfish import TinyFish, AgentConfig

client = TinyFish()

run = client.agent.run(
    url="https://www.saucedemo.com",  # your preview or tunnel URL
    goal="""Test the checkout flow:
        1. Log in with username "standard_user" and password "secret_sauce"
        2. Add "Sauce Labs Backpack" to the cart
        3. Open the cart and click "Checkout"
        4. Enter first name "Test", last name "User", postal code "94107", then click "Continue"
        5. Click "Finish"

        Expected result: the page shows "Thank you for your order!"

        If any step can't be completed exactly as written, stop and report which step failed.
        Set failed_step to null if every step passed.
    """,
    agent_config=AgentConfig(mode="strict", max_steps=30),
    output_schema={
        "type": "object",
        "properties": {
            "passed": {"type": "boolean"},
            "failed_step": {"type": "integer", "nullable": True},
            "reason": {"type": "string"},
        },
        "required": ["passed", "failed_step", "reason"],
    },
)

if run.error:
    raise SystemExit(f"Run did not complete: {run.error}")
result = run.result or {}
if not result.get("passed"):
    raise SystemExit(f"FAILED at step {result.get('failed_step')}: {result.get('reason')}")
print("PASSED:", result["reason"])

import { TinyFish } from "@tiny-fish/sdk";

type TestResult = { passed: boolean; failed_step: number | null; reason: string };

const client = new TinyFish();

const run = await client.agent.run({
  url: "https://www.saucedemo.com", // your preview or tunnel URL
  goal: `Test the checkout flow:
    1. Log in with username "standard_user" and password "secret_sauce"
    2. Add "Sauce Labs Backpack" to the cart
    3. Open the cart and click "Checkout"
    4. Enter first name "Test", last name "User", postal code "94107", then click "Continue"
    5. Click "Finish"

    Expected result: the page shows "Thank you for your order!"

    If any step can't be completed exactly as written, stop and report which step failed.
    Set failed_step to null if every step passed.
  `,
  agent_config: { mode: "strict", max_steps: 30 },
  output_schema: {
    type: "object",
    properties: {
      passed: { type: "boolean" },
      failed_step: { type: "integer", nullable: true },
      reason: { type: "string" },
    },
    required: ["passed", "failed_step", "reason"],
  },
});

if (run.error) {
  console.error("Run did not complete:", run.error);
  process.exit(1);
}
const result = run.result as TestResult | null;
if (!result?.passed) {
  console.error(`FAILED at step ${result?.failed_step}: ${result?.reason}`);
  process.exit(1);
}
console.log("PASSED:", result.reason);

Result

{
  "passed": true,
  "failed_step": null,
  "reason": "All steps completed successfully. Logged in, added Sauce Labs Backpack to the cart, checked out as Test User 94107, and clicked Finish. The page now shows 'Thank you for your order!'"
}

The script exits with an error when the test fails, so you can drop it into CI. To run several tests at once, start one run per test — see Run TinyAgent Across Multiple URLs. For tests that log in, use a test account, and keep its password in Vault rather than the goal once you’re past local testing — see Login With Vault.

Tip — from Claude Code, Cursor, ChatGPT or Claude: paste your preview or tunnel URL with the steps, e.g. “Use TinyFish to test sign-up on https://…: create an account with test@example.com and check the dashboard loads.” See Run TinyAgent From ChatGPT or Claude or MCP Integration.

Common Mistakes

Sending http://localhost:3000 as the URL

The agent’s browser runs in the cloud, so localhost points at that cloud machine, not yours — the run reports the site can’t be reached. Open a tunnel and use its https:// URL.

The run is COMPLETED, but the test failed

COMPLETED means the browser ran, not that your app works. Always check result.passed — see Runs.

The agent works around the bug and reports a pass

In default mode the agent tries alternatives when something looks wrong — a missing button, an unexpected popup — which can hide the bug you’re testing for. Use mode: "strict" so it stops instead. See Agent Mode, Max Steps, and Max Duration.

failed_step is filled in even though the test passed

Without an instruction, the agent may fill every field in the schema. Make failed_step nullable and add “Set failed_step to null if every step passed” to the goal, as in the sample.

Yesterday's tunnel URL stopped working

Quick tunnels get a new URL every time they restart. Copy the new one into your test. If your app checks allowed origins or OAuth redirect URLs, add the tunnel URL there too.

Agent Mode, Max Steps, and Max Duration

Why Strict mode suits tests

Runs

Why COMPLETED doesn’t mean your test passed

Get a TinyAgent Run as Structured JSON

Run TinyAgent Across Multiple URLs

Run a whole test suite at once

Start Here

Search API

Monitor

Fetch API

Browser API

Research API

Datasets API

Capabilities

Tools & Integrations

Guides

Use Cases

Reference

Test Your Web App on localhost or a Preview URL

Steps

Get a URL the agent can reach

Run the test

Common Mistakes

Agent Mode, Max Steps, and Max Duration

Runs

Get a TinyAgent Run as Structured JSON

Run TinyAgent Across Multiple URLs

​Steps

​Get a URL the agent can reach

​Run the test

​Common Mistakes

​Related

Agent Mode, Max Steps, and Max Duration

Runs

Get a TinyAgent Run as Structured JSON

Run TinyAgent Across Multiple URLs

Steps

Get a URL the agent can reach

Run the test

Common Mistakes

Related