> ## Documentation Index
> Fetch the complete documentation index at: https://docs.tinyfish.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Async Bulk Requests

> Submit multiple runs and poll for results

## Overview

The async API pattern is ideal when you want to submit multiple long-running tasks and check their status later. Instead
of waiting for each run to complete, you submit all requests and get back run IDs that you can poll for completion.

## How It Works

1. Submit requests to `/v1/automation/run-async`, which returns corresponding `run_id`s, which you will need if you want
   to check the status of a particular run.
2. Check individual runs with `GET /v1/runs/:id` to check status
3. Or fetch all runs with `GET /v1/runs` to monitor batch progress

## Basic Example

Submit multiple TinyFish Web Agent runs and poll for completion:

<CodeGroup>
  ```python Python theme={null}
  import asyncio
  from tinyfish import AsyncTinyFish, RunStatus

  async def wait_for_completion(client, run_id, poll_interval=2):
      """Poll a run until it completes"""
      while True:
          run = await client.runs.get(run_id)

          if run.status in (RunStatus.COMPLETED, RunStatus.FAILED, RunStatus.CANCELLED):
              return run

          await asyncio.sleep(poll_interval)

  async def main():
      client = AsyncTinyFish()

      # Define your batch of tasks
      tasks_to_run = [
          {
              "url": "https://scrapeme.live/shop/",
              "goal": "Extract all available products on page two with their name, price, and review rating (if available)",
          },
          {
              "url": "https://books.toscrape.com/",
              "goal": "Extract all available books on page two with their title, price, and review rating (if available)",
          },
      ]

      # Step 1: Submit all tinyfish runs and collect run_ids
      print("Submitting tinyfish runs...")
      submit_tasks = [
          client.agent.queue(url=task["url"], goal=task["goal"])
          for task in tasks_to_run
      ]
      responses = await asyncio.gather(*submit_tasks)
      run_ids = [r.run_id for r in responses]
      print(f"Submitted {len(run_ids)} runs: {run_ids}")

      # Step 2: Wait for all runs to complete
      print("Waiting for completion...")
      completion_tasks = [
          wait_for_completion(client, run_id)
          for run_id in run_ids
      ]
      results = await asyncio.gather(*completion_tasks)

      # Step 3: Process results
      for i, run in enumerate(results):
          print(f"Run {i + 1} ({run.run_id}):")
          print(f"  Status: {run.status}")
          if run.status == RunStatus.COMPLETED:
              print(f"  Result: {run.result}")

  # Run the async main function
  asyncio.run(main())
  ```

  ```typescript TypeScript theme={null}
  import { TinyFish, RunStatus } from "@tiny-fish/sdk";

  const client = new TinyFish();

  async function waitForCompletion(runId: string, pollInterval = 2000) {
    while (true) {
      const run = await client.runs.get(runId);
      if (
        run.status === RunStatus.COMPLETED ||
        run.status === RunStatus.FAILED ||
        run.status === RunStatus.CANCELLED
      ) {
        return run;
      }
      await new Promise((r) => setTimeout(r, pollInterval));
    }
  }

  async function main() {
    // Define your batch of tasks
    const tasksToRun = [
      {
        url: "https://scrapeme.live/shop/",
        goal: "Extract all available products on page two with their name, price, and review rating (if available)",
      },
      {
        url: "https://books.toscrape.com/",
        goal: "Extract all available books on page two with their title, price, and review rating (if available)",
      },
    ];

    // Step 1: Submit all tinyfish runs and collect run_ids
    console.log("Submitting tinyfish runs...");
    const responses = await Promise.all(
      tasksToRun.map((task) => client.agent.queue(task))
    );
    const runIds = responses.map((response) => {
      if (response.error) {
        throw new Error(`Failed to queue run: ${response.error.message}`);
      }
      return response.run_id;
    });
    console.log(`Submitted ${runIds.length} runs:`, runIds);

    // Step 2: Wait for all runs to complete
    console.log("Waiting for completion...");
    const results = await Promise.all(runIds.map((id) => waitForCompletion(id)));

    // Step 3: Process results
    results.forEach((run, i) => {
      console.log(`Run ${i + 1} (${run.run_id}):`);
      console.log(`  Status: ${run.status}`);
      if (run.status === RunStatus.COMPLETED) {
        console.log(`  Result:`, run.result);
      }
    });
  }

  main();
  ```
</CodeGroup>

## Fire and Forget Pattern

Submit tasks without waiting for completion:

<CodeGroup>
  ```python Python theme={null}
  async def main():
      client = AsyncTinyFish()

      tasks_to_run = [
          {"url": "https://example.com/page1", "goal": "Extract product info"},
          {"url": "https://example.com/page2", "goal": "Extract product info"},
          {"url": "https://example.com/page3", "goal": "Extract product info"},
      ]

      # Submit all tasks
      submit_tasks = [
          client.agent.queue(url=task["url"], goal=task["goal"])
          for task in tasks_to_run
      ]
      responses = await asyncio.gather(*submit_tasks)
      run_ids = [r.run_id for r in responses]

      print(f"Submitted {len(run_ids)} runs")
      print(f"Run IDs: {run_ids}")
      print("Check status later using client.runs.get(run_id)")

  asyncio.run(main())
  ```

  ```typescript TypeScript theme={null}
  import { TinyFish } from "@tiny-fish/sdk";

  const client = new TinyFish();

  async function main() {
    const tasksToRun = [
      { url: "https://example.com/page1", goal: "Extract product info" },
      { url: "https://example.com/page2", goal: "Extract product info" },
      { url: "https://example.com/page3", goal: "Extract product info" },
    ];

    // Submit all tasks
    const responses = await Promise.all(
      tasksToRun.map((task) => client.agent.queue(task))
    );
    const runIds = responses.map((response) => {
      if (response.error) {
        throw new Error(`Failed to queue run: ${response.error.message}`);
      }
      return response.run_id;
    });

    console.log(`Submitted ${runIds.length} runs`);
    console.log("Run IDs:", runIds);
    console.log("Check status later using client.runs.get(runId)");
  }

  main();
  ```
</CodeGroup>

## When to Use Async vs Sync

| Use Case            | API Pattern        | Why                                  |
| ------------------- | ------------------ | ------------------------------------ |
| Quick tasks (\<30s) | Sync `/run`        | Simpler code, immediate results      |
| Long-running tasks  | Async `/run-async` | Don't block, check later             |
| Large batches       | Async `/run-async` | Submit all at once, monitor progress |
| Fire and forget     | Async `/run-async` | No need to wait                      |
| Real-time feedback  | SSE `/run-sse`     | Stream progress events               |

## Best Practices

### Polling Interval

* **Short tasks (under 1 min)**: Poll every 2-3 seconds
* **Medium tasks (1-5 min)**: Poll every 5-10 seconds
* **Long tasks (over 5 min)**: Poll every 30-60 seconds

### Error Handling

Always check run status and handle failures:

```python Python theme={null}
async def process_completed_run(run):
    if run.status == RunStatus.COMPLETED:
        return run.result
    elif run.status == RunStatus.FAILED:
        print(f"Run {run.run_id} failed: {run.error}")
        return None
    elif run.status == RunStatus.CANCELLED:
        print(f"Run {run.run_id} was cancelled")
        return None
```

## API Reference

<CardGroup cols={2}>
  <Card title="Run Async" icon="code" href="https://docs.tinyfish.ai/api-reference/automation/start-automation-asynchronously">
    Start TinyFish Web Agent run asynchronously
  </Card>

  <Card title="List Runs" icon="list" href="https://docs.tinyfish.ai/api-reference/automation/list-runs">
    Get all runs
  </Card>

  <Card title="Get Run by ID" icon="file" href="https://docs.tinyfish.ai/api-reference/automation/get-run-by-id">
    Check individual run status
  </Card>

  <Card title="Cancel Run" icon="xmark" href="https://docs.tinyfish.ai/api-reference/runs/cancel-run-by-id">
    Cancel a running automation
  </Card>
</CardGroup>

## Related

<CardGroup cols={2}>
  <Card title="Concurrent Requests" icon="bolt" href="/examples/bulk-requests-sync">
    Sync API for immediate results
  </Card>

  <Card title="Web Scraping" icon="magnifying-glass" href="/examples/scraping">
    Extract data from pages
  </Card>
</CardGroup>