Botmation Documentation

Concurrency

Concurrency

There is a great open-source npm module for running concurrent Puppeteer Chromium instances called puppeteer-cluster.

Functional bots ran concurrently

Since BotAction's operate solely on Puppeteer pages, you can combine Botmation and Puppeteer-Cluster to run functional web bots concurrently.

Here is a simplified version of the puppeteer-cluster example:

import { Cluster } from 'puppeteer-cluster'


(async () => {
  try {
    const cluster = await Cluster.launch({
        concurrency: Cluster.CONCURRENCY_BROWSER,
        maxConcurrency: 3
    })


    const screenshotBotForClustering = async ({ page, data: url }: {page: Page, data: any}) => 
        await chain(
            goTo(url),
            screenshot(url.replace(/[^a-zA-Z]/g, '_')),
            log('screenshot of ' + url + ' saved')
        )(page)


    cluster.queue('https://nodejs.org/', screenshotBotForClustering)
    cluster.queue('https://github.com/', screenshotBotForClustering)
    cluster.queue('https://www.typescriptlang.org/', screenshotBotForClustering)


    await cluster.idle()
    await cluster.close()


  } catch(error) {
    console.error(error)
  }
})()

Dynamic Clustering Bots

To create a Bot for clustering (bots in a cluster are ran concurrently), you create an async function that takes an object {page: Page, data: any} as a param, for dynamically queueing the Puppeteer code, multiple times with varying options such as starting URL.

Check out puppeteer-cluster for documentation.

Edit this page on GitHub

Prev

Injects

Next

Facebook

On this page

Baby Bot