#! Dev's Bytes
Random programming stuff
#! Dev's Bytes

Browser automation with Nightmare.js

Share Tweet Share

Nightmare.js allows you to automate browser tasks, as clicking buttons, opening links, navigating pages and much more.

I want to start the first post of this blog writing about something that, for its potential, impressed me when I have seen it for the first time. I'm talking about Nightmare.js, a high level wrapper for already the well-known PhantomJS. Nightmare.js is a rather new module for the Node.js platform, its first stable release (1.0.0), according to its GitHub repository, dates back to May of this year. The simple API of the library allows you to automate most of the actions that a user performs while he browses the web, such as clicking on links, filling out forms, loading new pages, and taking screenshots. Nightmare.js allows you to write, with only a few lines of code, a script to check your incoming emails, read your tweets, or check your online orders status.

Installation

The following instructions are valid for Fedora 19 and Debian 7, if you use a different OS or distro, please refer to the documentation of the respective projects.

Dependencies

Nightmare.js requires Node.js, Python2.x (>=2.5) and PhantomJS.

Node.js

Installation on Fedora

If sudo is enabled for bash and yum, the following command setup the enviroment and installs Node.js and build tools.

curl -sL https://rpm.nodesource.com/setup | sudo bash -
sudo yum install -y nodejs gcc-c++ make

Otherwise, as root

curl -sL https://rpm.nodesource.com/setup | bash -
yum install -y nodejs gcc-c++ make

Installation on Debian

If sudo is enabled for bash and apt-get, the following command setup the enviroment and installs Node.js and build tools.

wget -qO- https://rpm.nodesource.com/setup | sudo bash -
sudo apt-get install -y nodejs build-essential

Otherwise, as root

wget -qO- https://rpm.nodesource.com/setup | bash -
apt-get install -y nodejs build-essential

Python2.x

Python is already installed both on Fedora and on Debian.

PhantomJS

Download PhantomJS from phantomjs.org/download.html. If you use Fedora or another Red Hat based system, follow the instructions for dependecies installation in the download page. After downloading, cd in the directory where the archive is and type

mkdir /tmp/phantomjs
cp phantomjs-* /tmp/phanotmjs
cd /tmp/phantomjs
tar -jxvf phatomjs-*
cd phantomjs-*

Now, if sudo is enabled for mv

sudo mv bin/phantomjs /usr/local/bin/

Otherwise

su
mv bin/phantomjs /usr/local/bin/

Nightmare.js

The installation is as simple as

npm install nightmare

If Python2.x is not the default system version

npm install nightmare --python='/usr/bin/python2'

Search on Wikipedia

Let's write some code to perform a simple search on Wikipedia and give back the corresponding article's URL.

The Wikipedia home page has a search field with a "searchInput" id, while the sending button has a field "name" with a value of "go".

// wiki.js
// Perform a search on Wikipedia and print out the URL of the searched page.
'use strict';

var Nightmare = require('nightmare');

// Check for a search parameter, otherwise, display a help for the user.
if (process.argv.length < 3) {
  console.log('Usage: node wiki.js PARAM');
  return;
}

new Nightmare()
  .goto('https://www.wikipedia.org')
  .type('#searchInput', process.argv[2])
  .click('input[name="go"]')
  .wait()
  .url(function(link) {
    console.log(link);
  })
  .run(function(err, nightmare) {
    if (err) {
      console.log(err);
    }
});

The code is straightforward; wait() is necessary for ensuring the correct page loading before getting the URL.

Running the script with a paramether of "node.js"

node wiki.js 'node.js'
# Display "https://en.wikipedia.org/wiki/Node.js"

Login and screenshot on Twitter

Perform a login and take a screen of your Twitter account home page requires less than twenty lines of code. Looking at the page source, we can find a username field with a "signin-email" id and a password field with a "signin-password" id, while the sending button has the "submit btn primary-btn flex-table-btn js-submit" classes.

// tw.js
// Perfom a login and take a screenshot of the Twitter home of a user
'use strict';

var Nightmare = require('nightmare');
new Nightmare()
  .goto('https://twitter.com')
  .type('input[id="signin-email"]', 'xxxx') // Substitute with your username
  .type('input[id="signin-password"]', 'xxxx') // Substitute with your password
  .click('.submit.btn.primary-btn.flex-table-btn.js-submit')
  .wait()
  .screenshot('screen.png')
  .run(function(err, nightmare) {
    if (err) {
      console.log(err);
    }
    console.log('Done.');
  });

Could you guess what will happen running the script?

node tw.js
# Display "Done.", "screen.png" is your home screenshot

Summary

I have given you a couple of examples only, but the library can do much more than this, among which:

  • Modify the request headers.
  • Upload a file.
  • Check wheter a page element exists and it's visible or not.
  • Handle an event in the DOM with a callback.

As you have seen, the API is stunning and the project, even if new to some degree, is really promising. The complete documentation is really simple and you can find it at the GitHub repository of the library. I recommend you to have a look at the "test" directory for further usage examples.

Do you like this article? Do you believe that Nightmare.js is a efficient project too? Please, leave your comment below and let me know about it.


Receive Updates

ATOM

Contacts