Overview

Botmation is a simple, declarative framework to build bots with composable functions called BotActions.

“Everything should be made as simple as possible, but no simpler.”
~ Albert Einstein

BotActions are self-reliant, single-purposed, async functions that perform tasks from simple, such as "click this button", to broad complex flows, such as "scrape this feed and like all of my friends' posts". They are ran individually or assembled together in a new BotAction.

BotActions are like bricks. Bricks build walls, walls build buildings, buildings build cities and so forth. Except, at each level of composition (brick, wall, building, etc), they are all the same type of function, a BotAction. Regardless of their compositional complexity, they are all single async functions called BotActions.

Imagine a door 🚪 into a coffee shop ☕ where customers 🧍,🧍,🧍 enter. As they step inside, a customer line forms 🧍🧍🧍. The first customer served, is the first customer in line, then the following, and so forth.

Botmation's bots are assembled in lines of BotActions. Assembled bots run their actions in the order declared, from first to last.

A single BotAction can be an assembled line of other BotActions. Therefore, a single person 🧍 in the coffee shop ☕, can actually be a whole other line of people 🧍🧍🧍. Then any one of those people 🧍 in this "sub" line 🧍🧍🧍, can be a whole other line of people 🧍🧍🧍 infinitely deep ♾️🐇.

The possibilities are endless!

If you're familiar with de-coupling functionality, this compositional pattern creates structure for your code to keep all units of functionality separate, pluggable and 100% reusable.

Running BotActions

BotActions wrap a Puppeteer Page instance as a function param. They are completed once their returned promise resolves.

Some BotActions have higher-order functions that customize their functionality. For example, the BotAction goTo() uses a higher-order function to set the URL for the bot to navigate too, while waitForNavigation does not:

Copy
const page = await browser.newPage() // Puppeteer Browser

await goTo('https://example.com')(page) // call higher-order to get the BotAction
await waitForNavigation(page) // no higher-order, just resolve the BotAction

Higher-order functions are functions that return a function. Botmation uses higher-order functions to customize BotActions during runtime.

So how is this better than using Puppeteer directly? BotActions can be assembled into new reusable functions, of varying compositional complexity. No matter what, regardless of their complexity, they are just single async functions.

Assembling Bots

Bots are built by assembling BotActions, declaratively, in a line. This makes each bot's code highly readable.

Let's edit the code from the example above, by assembling it into a reusable bot via a special kind of BotAction, an Assembly Line. The simplest is called chain()():

Copy
const bot = chain(
  goTo('https://example.com'),
  waitForNavigation
)

This BotAction chains BotActions together in a new reusable BotAction.

This technique is inspired by Currying from Functional Programming.

Now let's run the bot:

Copy
const bot = chain(
  goTo('https://example.com'),
  waitForNavigation
)

const page = await browser.newPage()
await bot(page)

This bot function can be reused on multiple Puppeteer pages. We can even run multiple bots concurrently, using the same code.

An assembled bot is still a BotAction. But, a BotAction is a bot part. Philosophically speaking, parts and bots are differentiated via observation. BotActions are bot parts, until they are ran as a bot, either individually or in an assembled composition.

So how do we create our own BotActions that go beyond the functionality provided by Botmation?

Simple BotActions

There's three main ways to make your own BotActions, from simple to complex.

Every BotAction receives a Puppeteer Page as its first param

The simplest are plain async functions. Here's the code for the waitForNavigation BotAction:

Copy
const waitForNavigation: BotAction = async(page) => {
  await page.waitForNavigation()
}

This BotAction calls the Page's waitForNavigation function. Simplicity is great.

You can call any Puppeteer Page function inside a BotAction. But, you're not limited too just Puppeteer. You can inject other libraries into your bots, which helps with writing tests for your BotActions, as they can easily be mocked. For example, the Scrapers, by default, use the Cheerio library.

Dynamic BotActions

BotActions wrapped in a higher-order functions are dynamic. Let's look at the code for the goTo() BotAction that navigates the page to the URL provided:

Copy
const goTo = (url: string, goToOptions?: Partial<WaitForOptions>): BotAction =>
  async(page) => {
    if (page.url() === url) {
      return
    }

    await page.goto(url, enrichGoToPageOptions(goToOptions))
  }

The higher-order synchronous function returns the customized BotAction function during runtime. Here goTo() uses a higher-order function to dynamically set the url and goToOptions for the BotAction.

Higher order function parameters can be whatever you need them to be. They're typed as a spread array of any, so add more if you need more. Also, you're not limited to one higher-order sync function. Stack them up, as high as need be! The possibilities are endless.

Feeling curious about stacked higher-order functions? Check out the Loops BotActions for practical BotActions that wrap themselves with two higher order functions.

Dynamic BotActions can be reused to create static, more readable BotActions:

Copy
const goToGoogle = goTo('https://google.com')

Compose BotActions

However, the best has been saved for last! The technique for assembling bots is the same technique for assembling complex BotActions. Remember, they are all BotActions, both the parts and the bots.

For example, here is a composed BotAction that logs a bot into a website through a Login form:

Copy
const login = (username: string, password: string): BotAction =>
  chain(
    goTo('https://example.com/login.html'),
    click('form input[name="username"]'),
    type(username),
    click('form input[name="password"]'),
    type(password),
    click('form button[type="submit"]'),
    waitForNavigation,
    log('Login Complete')
  )

login() is a higher-order sync function that returns an assembled line of BotActions by calling the higher-order function of the Chain BotAction.

This BotAction completes a login flow for an example website form. Once started, it runs the assembled BotActions, in the order they are declared. First, it navigates to the login page, then enters the username and password into the login page's form's inputs, submits the form, waits for the Navigation of the page to complete, before finally logging Login Complete in the NodeJS console, but you already know that from reading the code!

Assembly Lines are not the only kind of BotAction that assembles other BotActions. For more complex situations, check out Branching & Loop BotActions for unique ways to assemble BotActions i.e. if statement