Run TinyAgent Across Multiple URLs

Capability: concurrent runs

Use this when you need the same extraction on a batch of pages — a product catalog, a list of competitor sites, a set of search results — instead of waiting on one run before starting the next.

Python

import asyncio
from tinyfish import AsyncTinyFish

async def main():
    client = AsyncTinyFish()

    urls = [
        "https://scrapeme.live/shop/",
        "https://books.toscrape.com/",
    ]
    goal = "Extract the first 5 item names and prices. Return as JSON."

    runs = await asyncio.gather(
        *(client.agent.run(url=url, goal=goal) for url in urls)
    )

    for run in runs:
        print(run.result)

asyncio.run(main())

Each run is independent and billed separately — a failure on one URL doesn’t affect the others. For hundreds or thousands of URLs, submit with /run-async and poll instead of holding every request open at once; see Async Bulk Requests.

No code? The Projects area in the dashboard runs the same goal across a list of URLs from the UI. It’s rolling out gradually — if you don’t see it yet, the API pattern above works today.

Common Mistakes

Treating one failed URL as a failed batch

Each run in the batch is independent and billed separately. A failure on one URL doesn’t cancel or affect the others — check each run’s own status and result individually instead of assuming the whole batch failed because one await rejected.

Writing one goal that doesn't fit every URL

The same goal string runs against every URL in the list, so phrasing that only makes sense for one site’s layout will under-perform on the others. Keep the goal generic enough to apply site-agnostically — see the Goal Prompting Guide for batch-specific wording.

Concurrent Requests

Full sync-API batch example with error handling

Async Bulk Requests

Submit thousands of runs and poll for results

Goal Prompting Guide

Write goals that return a consistent shape across every run

Start Here

Search API

Monitor

Fetch API

Browser API

Research API

Datasets API

Capabilities

Tools & Integrations

Guides

Use Cases

Reference

Run TinyAgent Across Multiple URLs

Common Mistakes

Concurrent Requests

Async Bulk Requests

Goal Prompting Guide

​Common Mistakes

​Related

Concurrent Requests

Async Bulk Requests

Goal Prompting Guide

Common Mistakes

Related