In the final step of the series, we create a web application to access the Elasticsearch managed data and show how to host the web app on IBM's Bluemix.

In the previous article we ran some queries on the nested fields in our petitions data. In this article - the last in the series - we're going to turn our existing code into a fully-fledged app and deploy it using IBM Bluemix. To get an idea of what you'll have by the end of this article, check out our own Petitioneering app.

Before we get to that we'll add in the postcode lookup we promised at the end of the last article. We'll need the get-json library for this so install it before you start.

npm install get-json  

We'll also move the search query into a separate file which we'll call from nestedQuery.js. This will set us up nicely for when we are putting our application together since it will allow us to keep our functions separate from our logic.

Start by creating a new file and call it functions.js. Add the following lines to connect to Elasticsearch and use the get-json library.

var client = require ('./connection.js');  
var getJSON = require('get-json');  

Now move your search function from nestedQuery.js into functions.js. We need to modify it very slightly because of the way we'll be calling it from now on so change

var results = function(constitLookup) {  

to

function results(constitLookup,callback) {  

To allow nestedQuery.js to use this function we need to export it using module.exports. Add this to the end of functions.js:

module.exports = {  
  results: results
};

Back in nestedQuery.js we need to add a require statement to use functions.js and tweak how we call the results function. Replace everything in nestedQuery.js with the following:

var argv = require('yargs').argv;  
var functions = require('./functions.js');

if (argv.search) {  
  functions.results(argv.search, function(results) {
      console.log(results);
  });
}

Run nestedQuery.js to test everything looks ok, and when you're happy we'll move on to adding the postcode lookup.

Adding a postcode lookup

Not everyone knows the exact name of their constituency (easy enough to type in when it's Ipswich, not so much if you live in the constituency of "Inverness, Nairn, Badenoch and Strathspey") so to make it easier for a user to find out what petitions are popular where they live we'll add a lookup that works out a user's constituency from their postcode. For this, we'll use the API from http://postcodes.io/. First we'll use their validate method to check we've got a valid postcode, and then we'll use their postcode method to establish the constituency name. We'll then pass this into our results function.

In functions.js add the following:

function getConstituency(postcode,callback) {  
  getJSON('https://api.postcodes.io/postcodes/'+postcode, function(error, response){
    if(error) {
      console.log(error);
    }
    else {
      results(response.result.parliamentary_constituency,function(response){
        callback(response);
      });
    }
  });
}

function validatePostcode(postcode, callback) {  
  getJSON('https://api.postcodes.io/postcodes/'+postcode+'/validate',function(error,response){
      if(response.result){
        getConstituency(postcode,function(response){
          callback(response);
        });
      }
      else {
        console.log("Please enter a valid postcode");
      }
  });
}

function getResults(userinput, cb) {  
  var results = validatePostcode(userinput,function(response){
    cb(response);
  });
}

We also need to add getResults to our module exports so we can call it from nestedQuery.js:

module.exports = {  
  getResults: getResults,
  results: results
};

Finally, we just need to add a function to nestedQuery.js to invoke the postcode lookup when required. It shouldn't be too much of a surprise that it looks a lot like our existing search function:

if (argv.postcode) {  
  functions.getResults(argv.postcode, function(results) {
      console.log(results);
  });
}

Now you can run nestedQuery.js by supplying it with a postcode, like so:

node nestedQuery --postcode="KA22 8NG"  

Later we'll be able to drop this code right into our application: we'll need to modify how we get the search term to our function and we'll turn the results into some html we can output but the workflow will be essentially the same. Before we get to that stage, though, we need to create a Bluemix account and set up our application.

Creating your app in Bluemix

If you don't already have a Bluemix account, you can quickly get started by signing up for a free 30-day trial. After you've registered and confirmed your email, log in to your account and create your first organization when prompted. We've called ours petitioneering. Create a space and get ready to create your app.

• From your Dashboard click Create App, and choose Cloud Foundry Applications in the Apps section of the menu. We'll be creating a web app using the SDK for Node.js so choose that.
• Give your app an App name and a Host name. The Host name will form part of the url of your app, so choose something so choose a suitable name for the app (we went with petitioneering again).
• Click Create.

At this point Bluemix will start staging your app, and if you return to your Dashboard again you'll see you it listed there. As soon as Bluemix completes the staging you can click on the link under Route to see the default home page for your app and verify that your app is now running.

Downloading your app

Now it's time to download your app and start dropping in some code. Click somewhere in the app's listing that isn't a link to open your app's Overview page, where you can see the app's runtime status, connections and other information. Click Getting Started in the menu, and follow the instructions for downloading your app code, first downloading the CF Command Line Interface and then your starter code. Follow the rest of the instructions on the page to make sure your basic configuration is set up correctly.

Developing your app

Now it's time to download your app and start dropping in some code. Go to your app's Getting Started page, and download the CF Command Line Interface. Download your starter code and follow the rest of the instructions on the page to make sure your basic configuration is set up correctly.

It's time to start adding our code. The first task is just to copy a couple of files that you've already created into your application directory. Copy connection.js and the functions.js file from earlier in this article. Most of the app's functions are already in functions.js - we just need to pass user input into them and format the output for displaying in a web page.

Now, as well as entering their postcode to get information on their own constituency, a user might be interested in the results from any other constituency, so let's add a select box that allows them to do that. To populate the select box we'll use the output from a new Elasticsearch query that returns the constituency names from the documents in our constituencies index.

Add this to functions.js:

function getConstituencies(callback){  
  client.search({
    index: 'gov',
    type: 'constituencies',
    size: 650,
    fields: 'constituencyname',
    body: {
      sort:
        {
          "constituencyname": {
            order: "asc"
          }
        }
    }
  },function (error, response,status) {
      if (error){
        console.log("search error: "+error)
      }
      if (response){
        var constitList = [];
        response.hits.hits.forEach(function(hit){
          constitList.push(hit.fields.constituencyname);
        })
        callback(constitList.sort());
      }
      else {
        console.log("<p>No results</p>");
      }
  });
}

This function will be called when the app's homepage is loaded. That will happen over in app.js, so we'll need to export this function, along with getresults and results. To do that, we need to update module.exports in functions.js again:

module.exports = {  
  getResults: getResults,
  getConstituencies: getConstituencies,
  results: results
};

Back over in app.js, we need to require our new functions.js file so we can pass user input into it as function arguments:

var functions = require('./functions.js');  

When we want to use any of the functions from functions.js we reference them by their name in module.exports:

functions.getResults(argv.postcode, function(results) {  
  console.log("results output...");
    console.log(results);
});

Now it's time to tell our application what to do when a user arrives at the homepage. We're going to use that getConstituencies function to populate a select box so users check the results from any constituency without having to know a valid postcode. Add the following to app.js:

app.get('/', function(request, response) {  
  functions.getConstituencies(function(constituencyList){
    if(constituencyList){
      response.render('index', {
          constituencies: constituencyList
      });
    }
  });
});

This fires when a request is made for the app's index page, and passes the response from getConstituencies (which will be a list of the 650 UK constituencies in our Elasticsearch index) to whatever is going to render our web page.

On the subject of rendering web pages, now might be a good time to introduce Pug, our template engine for this app.

Pug (the template engine formerly known as Jade)

You'll have probably noticed that our basic code from Bluemix uses express.js as its web application framework. To make it even easier (or harder, depending on your viewpoint) to create our html pages we're going to use a template engine as well. We've chosen to use Pug, but express supports many others as well.

Pug allows us to generate static html pages using templates that we define, and variables in those templates that we can pass values into. Starting with a base layout, which we'll use to define a header and a footer for every page, we can then insert different blocks of content depending on which page we want to display to the user. Our app only really has one page, but it should give you an idea of how you could extend it.

Let's add the following to app.js to tell our app to use Pug and specify the directory where it can find the templates (also known as views):

app.set('view engine', 'pug');  
app.set('views', __dirname + '/public/views');  

And let's define our base layout. Save this in your app folder as public/views/layout.pug:

doctype html  
html  
    head
        title='Petitioneering'
        link(rel='stylesheet', href='/stylesheets/style.css')
        link(rel='stylesheet', href='/stylesheets/petitioneering.css')
    body
        block content
        script(src='http://ajax.googleapis.com/ajax/libs/jquery/2.0.3/jquery.min.js')
        script(src='/javascripts/main.js')
    footer
        block footer
          div#footer
            div#footer-content
              p
                | Petitioneering uses open data from
                a(href='https://petition.parliament.uk') UK Government and Parliament
                | , indexed in
                a(href='https://elastic.co') Elasticsearch
                | , hosted by
                a(href='https://compose.com') Compose
              p
                | Application source available from
                a(href='https://github.com/compose-ex/petitioneering') Github

We've defined a fairly straightforward html page, with , and