Use all the Databases - Part 1

Published

Loren Sands-Ramshaw, author of GraphQL: The New REST, shows how to combine data from multiple sources using GraphQL in this Write Stuff two-part series.

Ever wanted to use a few different databases to build your app? Different types of databases are meant for different purposes, so it often makes sense to combine them. You might be hesitant due to the complexity of maintenance and coding, but it can be easy if you combine Compose and GraphQL: instead of writing a number of complex REST endpoints, each querying multiple databases, you set up a single GraphQL endpoint that provides whatever data the client wants using your simple data fetching functions.

This tutorial is meant for anyone who provides or fetches data, whether it’s a backend dev writing an API (in any language) or a frontend web or mobile dev fetching data from the server. We’ll learn about the GraphQL specification, set up a GraphQL server, and fetch data from five different data sources. The code is in Javascript, but you’ll still get a good idea of GraphQL without knowing the language.

In this first part, we'll look at the databases that will be involved. Then we'll introduce GraphQL before moving on to the query we want to make, the schema we need to create and how to setup the server to make that all happen.

In part two, we'll look at resolving queries on SQL, Elasticsearch, MongoDB, Redis and REST data sources and a look at how to get the best performance before calling things done.

Part 1

Part 2

The databases

We at Chirper Fictional, Inc. were building a Twitter clone, and decided to use these databases:

Now we need a way to combine the data from all of these sources together in whichever ways our clients want it, and the best way to do this is with GraphQL.

GraphQL intro

Gotta be honest here... for the first few months of GraphQL's short existence (it launched in July 2015), I thought GraphQL was a query language for accessing your Facebook friend graph 😳. Turns out that’s FQL, and GraphQL is a replacement for REST! And sorry REST, but GraphQL is kinda better than you for most things 😁. Here's why:

Example GraphiQL query

Architecture diagram of REST versus GraphQL

Credit: Jonas Helfer

Fear not—you don't need to rewrite all your REST servers: you can instead add a simple GraphQL server in front of them, as we'll see with the REST data source example below.

Note: you can of course also change data with GraphQL (with functions called mutations), but I won't be covering that in this post.

The query

Let's figure out the query that we'll need for our app's home dashboard. First, here are the things we'd like to display:

For each tweet, we'll want to display the text of the tweet, the author's name and photo, and when it was created. For the mentions and city feeds, we also want the number of times the tweets were viewed and from what city they were made.

Now to make the query, we write out the pieces of data we need in order to display the above list, choosing names for each field and putting it in a JSON-like format! 😄 

const queryString = `  
{
  user(id: 1) {
    firstName
    lastName
    photo
    mentions {
      text
      author {
        firstName
        lastName
        photo
      }        
      city
      views
      created      
    }
  }      
  publicFeed {
    text
    author {
      firstName
      lastName
      photo
    }    
    created
  }
  cityFeed {
    text
    author {
      firstName
      lastName
      photo
    }      
    city
    views
    created
  }
}
`

We'll put mentions as a field of the user query instead of at the top level because we'll need to the user's name in order to query Elasticsearch, and we'll have their name from the first step of the user query (we'll see how this looks when we implement it).

Parentheses are used to pass arguments—for simplicity's sake, we're passing our own user id with (id: 1). Usually when fetching the current user’s data, instead of passing your user id as an argument, you'd put your auth token in the Authorization header, and the server would authenticate you. This is done automatically for you by frameworks like Meteor.

Our query should return the below JSON data. The data mirrors the query format, with values filled in, sometimes with arrays of objects:

{
  "data": {
    "user": {
      "firstName": "Maurine",
      "lastName": "Rau",
      "photo": "http://placekitten.com/200/139",
      "mentions": [
        {
          "text": "Maurine Rau Eligendi in deserunt.",
          "author": {
            "firstName": "Maurine",
            "lastName": "Rau",
            "photo": "http://placekitten.com/200/139"
          },
          "city": "San Francisco",
          "views": 82,
          "created": 1481757217713
        }
      ]
    },
    "publicFeed": [
      {
        "text": "Corporis qui impedit cupiditate rerum magnam nisi velit aliquam.",
        "author": {
          "firstName": "Tia",
          "lastName": "Berge",
          "photo": "http://placekitten.com/200/139"
        },
        "city": "New York",
        "views": 91,
        "created": 1481757215183
      },
      ...
    ],
    "cityFeed": [
      {
        "text": "Edmond Jones Harum ullam pariatur quos est quod.",
        "author": {
          "firstName": "Edmond",
          "lastName": "Jones",
          "photo": "http://placekitten.com/200/139"
        },
        "city": "Mountain View",
        "views": 69,
        "created": 1481757216723
      },
      ...
    ]
  }
}

Now let's write the simple GraphQL server that will return that data!

The schema

The first thing your server needs is a schema. This is what the server will use to provide type safety and power the introspection and improved error messages. Since we've already written out what we'd like our queries to look like, this will be easy - we just need to list out the fields and their types. First, under type Query, we list the possible queries (top-level attributes in our query string):

type Query {  
  user(id: Int!): User

  # A feed of the most recent tweets worldwide 
  publicFeed: [Tweet]

  # A feed of the most recent tweets in your city
  cityFeed: [Tweet]
}

code

Each query is followed by the type that is returned. Besides the basic types (String, Int, Float, Boolean), you can make your own types, which start with a capital letter. So the first line reads, "One possible query is the user query, which takes one required argument (the exclamation point in Int! means required) named id of type Int and which returns something of type User." The last line reads, "One possible query is the cityFeed query, which has no arguments and returns an array of Tweets." The # comments are descriptions, which show up in the GraphiQL IDE described later.

Now to define the User and Tweet types, we'll list the fields we chose in our query string:

type User {  
  firstName: String
  lastName: String
  photo: String
  mentions: [Tweet]
}

type Tweet {  
  text: String
  author: User
  city: String
  views: Int
  created: Float
}

code

That's our schema! The schema goes into a string:

// data/schema.js

const schema = `  
type User { ...  
type Tweet { ...  
type Query { ...

schema {  
  query: Query
}
`;

export default schema;

data/schema.js

Server setup

The reference implementation of the GraphQL specification is GraphQL-JS, and it's used by graphql-server-express, a GraphQL middleware for Express, the most popular Node.js web server. Here's how we set it up:

// server.js:
import express from 'express';  
import { graphqlExpress, graphiqlExpress } from 'graphql-server-express';  
import { makeExecutableSchema } from 'graphql-tools';  
import bodyParser from 'body-parser';

import schema from './data/schema';  
import resolvers from './data/resolvers';

const graphQLServer = express();

const executableSchema = makeExecutableSchema({  
  typeDefs: [schema],
  resolvers,
});

graphQLServer.use('/graphql', bodyParser.json(), graphqlExpress({  
  schema: executableSchema,
}));

graphQLServer.use('/graphiql', graphiqlExpress({  
  endpointURL: '/graphql',
}));

const GRAPHQL_PORT = 8080;

graphQLServer.listen(GRAPHQL_PORT, () => console.log(  
  `GraphQL Server is now running on http://localhost:${GRAPHQL_PORT}`
));

server.js

If you’d like to run this server on your computer, first follow the repo’s setup instructions. Now you can start the server by running server.js:

nodemon ./server.js --exec babel-node

And make queries in GraphiQL:

http://localhost:8080/graphiql

When you edit the code, the server will restart itself, and you can re-run your query in GraphiQL. Reload the page in order to get the docs and autocompletion to update.

You can try out the GraphiQL of the finished Twitter clone server (powered by Compose!) here:

all-the-databases.graphql.guide/graphql

The only differences between the hosted server and the code running on your own computer are environment variables that contain the database connection info that you get when you set up a new Compose database.

We now have a working server and schema. The server setup was short, and specifying types for the schema was intuitive, but we haven’t done anything database-specific yet. In Part 2 we’ll write the server code that fetches the right data from SQL, Elasticsearch, MongoDB, Redis, and a REST API.s. Add Compose Articles to your feed reader to get the next part!

[Loren Sands-Ramshaw](http://lorensr.me/) is an author of the upcoming book [GraphQL: The New REST](https://graphql.guide). He also freelances and helps maintain the [Meteor](https://www.meteor.com/) framework. ☄️🤓

attributionHyberbole and a half

This article is licensed with CC-BY-NC-SA 4.0 by Compose.

Conquer the Data Layer

Spend your time developing apps, not managing databases.