Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.tinyfish.ai/llms.txt

Use this file to discover all available pages before exploring further.

​
Overview

The async API pattern is ideal when you want to submit multiple long-running tasks and check their status later. Instead of waiting for each run to complete, you submit all requests and get back run IDs that you can poll for completion.

​
How It Works

  1. Submit requests to /v1/automation/run-async, which returns corresponding run_ids, which you will need if you want to check the status of a particular run.
  2. Check individual runs with GET /v1/runs/:id to check status
  3. Or fetch all runs with GET /v1/runs to monitor batch progress

​
Basic Example

Submit multiple TinyFish Web Agent runs and poll for completion: 
import asyncio
from tinyfish import AsyncTinyFish, RunStatus

async def wait_for_completion(client, run_id, poll_interval=2):
    """Poll a run until it completes"""
    while True:
        run = await client.runs.get(run_id)

        if run.status in (RunStatus.COMPLETED, RunStatus.FAILED, RunStatus.CANCELLED):
            return run

        await asyncio.sleep(poll_interval)

async def main():
    client = AsyncTinyFish()

    # Define your batch of tasks
    tasks_to_run = [
        {
            "url": "https://scrapeme.live/shop/",
            "goal": "Extract all available products on page two with their name, price, and review rating (if available)",
        },
        {
            "url": "https://books.toscrape.com/",
            "goal": "Extract all available books on page two with their title, price, and review rating (if available)",
        },
    ]

    # Step 1: Submit all tinyfish runs and collect run_ids
    print("Submitting tinyfish runs...")
    submit_tasks = [
        client.agent.queue(url=task["url"], goal=task["goal"])
        for task in tasks_to_run
    ]
    responses = await asyncio.gather(*submit_tasks)
    run_ids = [r.run_id for r in responses]
    print(f"Submitted {len(run_ids)} runs: {run_ids}")

    # Step 2: Wait for all runs to complete
    print("Waiting for completion...")
    completion_tasks = [
        wait_for_completion(client, run_id)
        for run_id in run_ids
    ]
    results = await asyncio.gather(*completion_tasks)

    # Step 3: Process results
    for i, run in enumerate(results):
        print(f"Run {i + 1} ({run.run_id}):")
        print(f"  Status: {run.status}")
        if run.status == RunStatus.COMPLETED:
            print(f"  Result: {run.result}")

# Run the async main function
asyncio.run(main())

​
Fire and Forget Pattern

Submit tasks without waiting for completion: 
async def main():
    client = AsyncTinyFish()

    tasks_to_run = [
        {"url": "https://example.com/page1", "goal": "Extract product info"},
        {"url": "https://example.com/page2", "goal": "Extract product info"},
        {"url": "https://example.com/page3", "goal": "Extract product info"},
    ]

    # Submit all tasks
    submit_tasks = [
        client.agent.queue(url=task["url"], goal=task["goal"])
        for task in tasks_to_run
    ]
    responses = await asyncio.gather(*submit_tasks)
    run_ids = [r.run_id for r in responses]

    print(f"Submitted {len(run_ids)} runs")
    print(f"Run IDs: {run_ids}")
    print("Check status later using client.runs.get(run_id)")

asyncio.run(main())

​
When to Use Async vs Sync

Use CaseAPI PatternWhy
Quick tasks (<30s)Sync /runSimpler code, immediate results
Long-running tasksAsync /run-asyncDon’t block, check later
Large batchesAsync /run-asyncSubmit all at once, monitor progress
Fire and forgetAsync /run-asyncNo need to wait
Real-time feedbackSSE /run-sseStream progress events

​
Best Practices

​
Polling Interval

  • Short tasks (under 1 min): Poll every 2-3 seconds
  • Medium tasks (1-5 min): Poll every 5-10 seconds
  • Long tasks (over 5 min): Poll every 30-60 seconds

​
Error Handling

Always check run status and handle failures:
Python
async def process_completed_run(run):
    if run.status == RunStatus.COMPLETED:
        return run.result
    elif run.status == RunStatus.FAILED:
        print(f"Run {run.run_id} failed: {run.error}")
        return None
    elif run.status == RunStatus.CANCELLED:
        print(f"Run {run.run_id} was cancelled")
        return None

​
API Reference

Run Async

Start TinyFish Web Agent run asynchronously

List Runs

Get all runs

Get Run by ID

Check individual run status

Cancel Run

Cancel a running automation

Concurrent Requests

Sync API for immediate results

Web Scraping

Extract data from pages