How to bring life to your Node.js microservice

In the first article of this series, I explained how to create a new microservice with Node.js. You might have already figured out, that this service is pretty useless without actual content, so in this article we will add some demo content to our service.

Database connection

If we want to store data persistantly, we need some kind of storage. We will use a MongoDB for this purpose.

npm i mongodb
npm i -D @types/mongodb

We create a file „mongodb-connection.ts“ with this content:

import { Db, MongoClient, MongoClientOptions } from 'mongodb'
import { ServerError } from './errors/server'

export class MongoDbConnector {
  private static Client: Promise
  private static Db: Promise

  public static async Connect (): Promise {
    // do not open new connection if old connection exists
    if (!this.Client) {
      if (!process.env.DB_HOST) throw new ServerError('missing environment setting DB_HOST')
      if (!process.env.DB_PORT) throw new ServerError('missing environment setting DB_PORT')
      if (!process.env.DB_DATABASE) throw new ServerError('missing environment setting DB_DATABASE')
      const host = process.env.DB_HOST
      const port = process.env.DB_PORT
      const db = process.env.DB_DATABASE
      const conString = `mongodb://${host}:${port}/${db}`

      const options: MongoClientOptions = {
        useNewUrlParser: true,
        autoReconnect: true,
        connectTimeoutMS: 2000
      }

      // istanbul ignore if
      if (process.env.DB_USERNAME && process.env.DB_PASSWORD) {
        options.auth = {
          user: process.env.DB_USERNAME,
          password: process.env.DB_PASSWORD
        }
      }

      options.ssl = !!process.env.DB_SSL && process.env.DB_SSL.toLowerCase() === 'true'

      this.Client = MongoClient.connect(conString, options)
      this.Db = this.Client.then(client => client.db())
      await this.Client.then(() => {
        console.log('new mongodb connection established')
      }).catch(err => {
        delete this.Client
        throw new ServerError(`Could not establish database connection (${err})`)
      })
    }
    return this.Client.then(client => client.db())
  }

  static async Disconnect (): Promise {
    // istanbul ignore else
    if (this.Db) {
      delete this.Db
    }
    // istanbul ignore else
    if (this.Client) {
      const client = this.Client
      delete this.Client
      return (await client).close()
    }
  }
}

Keep in mind, that this code is not thread safe and might cause problems when too many users are working parallely or the connection is failing from time to time.

Let’s go to router/health.ts and refactor it to check if the mongodb connection is working before the service respondes to be healthy or ready.

import express from 'express'
import { Router } from 'express-serve-static-core'
import { ServerError } from '../errors/server'
import { MongoDbConnector } from '../mongodb-connector'

export function GetRouter (): Router {
  const router = express.Router()

  router.get('/liveliness', (req: express.Request, res: express.Response, next: express.NextFunction) => {
    MongoDbConnector.Connect().then(() => res.json({ status: 'OK' })).catch(() => {
      next(new ServerError('Could not connect to the db server'))
    })
  })

  router.get('/readiness', (req: express.Request, res: express.Response, next: express.NextFunction) => {
    MongoDbConnector.Connect().then(() => res.json({ status: 'OK' })).catch(() => {
      next(new ServerError('Could not connect to the db server'))
    })
  })

  return router
}

To make sure the mongodb connections are closed correctly when exiting the tests we need to add two more calls to the test/health.ts file.

before(async () => {
  await MongoDbConnector.Connect()
})

after(async () => {
  await MongoDbConnector.Disconnect()
})

The after will close the connection after the tests are done. The before will open the connection. This would happen implicit when using the connection but we don’t want the test to show „test was executed slowly“ just because the db connection wasn’t ready.

Create the swagger definition

For the sake of the length of this post, I need to asume that you already know „how to swagger“. If you don’t, the [online swagger editor](https://editor.swagger.io) might be a good start for your experiments. For this post, I defined this swagger file:

swagger: "2.0"
info:
  description: "This api provides methods to read, write or delete stuff."
  version: "1.0.0"
  title: "StuffService"
  contact:
    name: Johannes Herrmann @ Consort It
    email: "johannes.herrmann@consort-it.de"
  license:
    name: "MIT"
    url: "https://opensource.org/licenses/MIT"
host: "localhost:8080"
basePath: "/api/v1/stuff/"
schemes:
- "http"
paths:
  /:
    get:
      summary: "Get a list of all stuff"
      operationId: "GetStuff"
      produces:
      - "application/json"
      responses:
        200:
          description: "Ok"
          schema:
            type: array
            items:
              $ref: "#/definitions/Stuff"
    post:
      summary: "Insert stuff into the database"
      operationId: "InsertStuff"
      parameters:
      - in: "body"
        name: "createData"
        description: "data for the stuff to be created"
        required: true
        schema:
          $ref: "#/definitions/CreateStuff"
      consumes:
      - "application/json"
      produces:
      - "application/json"
      responses:
        201:
          description: "Created"
          schema:
            $ref: "#/definitions/Stuff"
        400:
          description: "Bad Request"
    put:
      summary: "Update the stuff with the given id in the database"
      operationId: "UpdateStuff"
      parameters:
      - in: "body"
        name: "createData"
        description: "data for the stuff to be updated"
        required: true
        schema:
          $ref: "#/definitions/Stuff"
      consumes:
      - "application/json"
      produces:
      - "application/json"
      responses:
        200:
          description: "Updated"
          schema:
            $ref: "#/definitions/Stuff"
        400:
          description: "Bad Request"
        404:
          description: "Not Found"
  /{stuffId}:
    get:
      summary: "Get a stuff by its id"
      operationId: "GetStuffById"
      parameters:
      - in: "path"
        name: "stuffId"
        description: "id of the stuff to get"
        required: true
        type: string
      produces:
      - "application/json"
      responses:
        200:
          description: "Ok"
          schema:
            $ref: "#/definitions/Stuff"
        404:
          description: "Could not find stuff with the given id"
    delete:
      summary: "Delete a stuff by its id"
      operationId: "DeleteStuffById"
      parameters:
      - in: "path"
        name: "stuffId"
        description: "id of the stuff to be deleted"
        required: true
        type: string
      produces:
      - "application/json"
      responses:
        200:
          description: "Ok"
          schema:
            $ref: "#/definitions/Stuff"
        404:
          description: "Could not find stuff with the given id"

definitions:
  Stuff:
    type: "object"
    required:
      - _id
      - title
      - owner
      - price
    properties:
      _id: 
        type: string
        description: 'the database id of the stuff'
        example: '5beda9e411b64b4fd45109b2'
      title: 
        type: string
        description: 'the title of the stuff'
        example: 'The White House'
      owner: 
        type: string
        description: 'the owner of the stuff'
        example: 'Mr President'
      price: 
        type: number
        description: 'the price of the stuff'
        example: 15.55
  CreateStuff:
    type: "object"
    required:
      - title
      - owner
    properties:
      title: 
        type: string
        description: 'the title of the stuff'
        example: 'The White House'
      owner: 
        type: string
        description: 'the owner of the stuff'
        example: 'Mr President'
      price: 
        type: number
        description: 'the price of the stuff'
        default: 0
        example: 15.55

Create the „Stuff“ model

It looks like the frame for our project is complete and we can finally start the actual work. To define our first data model, we go to the models directory and create a new file called „stuff.ts“. Stuff is a pretty simple object that has only got a couple simple properties.

export class Stuff {
  public _id!: string
  public title!: string
  public owner!: string
  public price?: number
}

Create the „Stuff“ controller

The stuff controller will be used to read/write the stuff from/to the database. We create a file called „stuff.ts“ in the controllers directory.

import { config } from 'dotenv'
import { ObjectID, ObjectId } from 'mongodb'
import { isNumber } from 'util'
import { BadRequestError } from '../errors/bad-request'
import { NotFoundError } from '../errors/not-found'
import { Stuff } from '../models/stuff'
import { MongoDbConnector } from '../mongodb-connector'

config({ path : 'service.env' })

const stuffCollection = 'stuff'

export class StuffController {
  private checkMandatoryProperty (obj: any, propertyName: string) {
    if (!obj[propertyName]) throw new BadRequestError(`missing property: ${propertyName}`)
  }

  private checkStuffProperties (data: any) {
    this.checkMandatoryProperty(data, 'title')
    this.checkMandatoryProperty(data, 'owner')
    if (!isNumber(data.price)) {
      data.price = 0
    }
  }

  public async InsertStuff (data: any): Promise {
    this.checkStuffProperties(data)
    const db = await MongoDbConnector.Connect()
    return db.collection(stuffCollection).insertOne(data).then(r => {
      data._id = r.insertedId
      return data as Stuff
    })
  }

  private getObjectId (stuffId: string): ObjectId {
    try {
      return new ObjectID(stuffId)
    } catch (err) {
      throw new BadRequestError('invalid format for object id')
    }
  }

  public async UpdateStuff (data: any): Promise {
    this.checkMandatoryProperty(data, '_id')
    const stuffId = data._id
    delete data._id
    this.checkStuffProperties(data)
    const db = await MongoDbConnector.Connect()
    return db.collection(stuffCollection).updateOne({ _id: this.getObjectId(stuffId) }, { '$set': data }, { upsert: false }).then((r) => {
      if (r.matchedCount === 0) {
        throw new NotFoundError(`Could not find stuff with id ${stuffId}`)
      } else {
        data._id = stuffId
        return data as Stuff
      }
    })
  }

  public async GetStuffById (stuffId: string): Promise {
    const db = await MongoDbConnector.Connect()
    return db.collection(stuffCollection).findOne({ _id: this.getObjectId(stuffId) }).then(res => {
      if (!res) {
        throw new NotFoundError(`Could not find stuff with id ${stuffId}`)
      } else {
        return res
      }
    })
  }

  public async GetStuff (): Promise<Stuff[]> {
    const db = await MongoDbConnector.Connect()
    return db.collection(stuffCollection).find({}).toArray()
  }

  public async DeleteStuff (stuffId: string): Promise {
    const db = await MongoDbConnector.Connect()
    return db.collection(stuffCollection).deleteOne({ _id: this.getObjectId(stuffId) }).then(r => {
      if (typeof r.deletedCount !== 'number' || r.deletedCount === 0) {
        throw new NotFoundError(`Could not delete the stuff with the id ${stuffId}`)
      }
    })
  }
}

The controllers are responsible for validating their input and executing all kind of actions but they are not responsible for handling the webserver related logic.

Routes

The routes are used to link webserver requests to actions in the controllers. Here we will also map request parameters to the function parameters and handle errors. To create routes for our stuff, we create a file stuff.ts in the routes directory.

import express from 'express'
import { Router } from 'express-serve-static-core'
import HttpStatusCodes from 'http-status-codes'
import { StuffController } from '../controllers/stuff'

export function GetRouter (): Router {
  const router = express.Router()
  const controller = new StuffController()

  router.get('', (req: express.Request, res: express.Response, next: express.NextFunction) => {
    return controller.GetStuff().then(res.json.bind(res)).catch(next)
  })

  router.post('', (req: express.Request, res: express.Response, next: express.NextFunction) => {
    return controller.InsertStuff(req.body).then(created => {
      res.statusCode = HttpStatusCodes.CREATED
      return res.json(created)
    }).catch(next)
  })

  router.put('', (req: express.Request, res: express.Response, next: express.NextFunction) => {
    return controller.UpdateStuff(req.body).then(res.json.bind(res)).catch(next)
  })

  router.get('/:stuffId', (req: express.Request, res: express.Response, next: express.NextFunction) => {
    return controller.GetStuffById(req.params.stuffId).then(res.json.bind(res)).catch(next)
  })

  router.delete('/:stuffId', async (req: express.Request, res: express.Response, next: express.NextFunction) => {
    return controller.DeleteStuff(req.params.stuffId).then(() => res.sendStatus(HttpStatusCodes.NO_CONTENT)).catch(next)
  })

  return router
}

Up to now I didn’t really explain single expressions of my code but in this file, I need to talk about some details as they really had confused me when i started. You can see the snippet .then(res.json.bind(res)) a couple of times. In Typescript passing methods (not functions) as callback parameters is very dangerous as javascripts has some issues with its classes and instances, that I have not seen in any other programming language before. When using typescript we usually avoid those issues by using the arrow-syntax e.g. (result) => { return res.json(result) } or shorter result => res.json(result). When trying to shorten this even further and by using „a pointer“ to the method itself and not an anonymouse function that calls it e.g. .then(res.json) the call will fail because inside the method this will be undefined. To avoid this, we can explicitly bind the method to the object like this .then(res.json.bind(res)).

The other snippet that might not be self explaining is .catch(next). If the controller raises an exception within the promise that executes the action it will return the error as parameter of the catch callback. This snippet will simply pass the error as parameter to the next function. The express server will handle this call and if the parameter is not null, call the next registered error handler. As you might remember, we have provided an error handler some chapters ago and this snippet is making use of it.

Mocking the MongoDB

In the past I’ve spent so much time creating abstractions layers to be able to provide mock data instead of reading the data from a database for testing and to be honest I am tired of it. As our service will be docker based I can’t think of any good reason (except „because one shouldn’t“) why we should not simple start a docker container that runs a temporary database to save us some time.

Testing CRUD stuf

There are so many discussions on what to test in which test stage but here again, we don’t have to make it too complicated.
Create a file called „stuff.ts“ in the test directory with the following content:

import { expect, request, use as chai_use } from 'chai'
import chaiHttp = require('chai-http')
import * as HttpStatusCodes from 'http-status-codes'
import app from '../src/api'
import { MongoDbConnector } from '../src/mongodb-connector'

chai_use(chaiHttp)

const apiPrefix = '/api/v1/stuff'
let stuffId: string

const testObj = {
  title: 'The White House',
  owner: 'Mr President',
  price: 15.55
}

describe('stuff', () => {
  before(async () => {
    await MongoDbConnector.Connect()
  })

  it('can create stuff', async () => {
    const alteredCopy = JSON.parse(JSON.stringify(testObj))
    delete alteredCopy.title
    const res = await request(app).post(apiPrefix).send(alteredCopy)
    expect(res.status).to.equal(HttpStatusCodes.BAD_REQUEST)
    expect(res).to.be.json
    expect(res.body.message).to.equal('missing property: title')
  })

  it('can\'t create stuff with missing property', async () => {
    const res = await request(app).post(apiPrefix).send(testObj)
    expect(res.status).to.equal(HttpStatusCodes.CREATED)
    expect(res).to.be.json
    expect(res.body._id).not.to.undefined
    stuffId = res.body._id
    const expected = JSON.parse(JSON.stringify(testObj))
    expected._id = stuffId
    expect(res.body).to.deep.equal(expected)
  })

  it('can get all stuff', async () => {
    const res = await request(app).get(apiPrefix)
    expect(res.status).to.equal(HttpStatusCodes.OK)
    expect(res).to.be.json
    expect(res.body).to.be.an('array')
    expect(res.body).to.have.length(1)
    const expected = JSON.parse(JSON.stringify(testObj))
    expected._id = stuffId
    expect(res.body[0]).to.deep.equal(expected)
  })

  it('can get stuff by id', async () => {
    const res = await request(app).get(`${apiPrefix}/${stuffId}`)
    expect(res.status).to.equal(HttpStatusCodes.OK)
    expect(res).to.be.json
    expect(res.body).to.be.an('object')
    const expected = JSON.parse(JSON.stringify(testObj))
    expected._id = stuffId
    expect(res.body).to.deep.equal(expected)
  })

  it('can will return BAD_REQUEST for a stuff id in the wrong format', async () => {
    const res = await request(app).get(`${apiPrefix}/123`)
    expect(res.status).to.equal(HttpStatusCodes.BAD_REQUEST)
    expect(res).to.be.json
    expect(res.body.message).to.equal('invalid format for object id')
  })

  it('can will return NOT_FOUND for unkown stuff id', async () => {
    const res = await request(app).get(`${apiPrefix}/1234567890ab`)
    expect(res.status).to.equal(HttpStatusCodes.NOT_FOUND)
    expect(res).to.be.json
    expect(res.body.message).to.equal('Could not find stuff with id 1234567890ab')
  })

  it('can update stuff', async () => {
    const alteredCopy = JSON.parse(JSON.stringify(testObj))
    alteredCopy._id = stuffId
    delete alteredCopy.price
    const res = await request(app).put(apiPrefix).send(alteredCopy)
    expect(res.status).to.equal(HttpStatusCodes.OK)
    expect(res).to.be.json
    alteredCopy.price = 0
    expect(res.body).to.deep.equal(alteredCopy)
  })

  it('returns NOT_FOUND when updating stuff with a unknwon id', async () => {
    const alteredCopy = JSON.parse(JSON.stringify(testObj))
    alteredCopy._id = '1234567890ab'
    alteredCopy.price = 17
    const res = await request(app).put(apiPrefix).send(alteredCopy)
    expect(res.status).to.equal(HttpStatusCodes.NOT_FOUND)
    expect(res).to.be.json
    expect(res.body.message).to.equal('Could not find stuff with id 1234567890ab')
  })

  it('deletes the newly created item', async () => {
    const res = await request(app).delete(`${apiPrefix}/${stuffId}`)
    expect(res.status).to.equal(HttpStatusCodes.NO_CONTENT
      )
  })

  it('deleting an item that does not exist will return NOT_FOUND', async () => {
    const res = await request(app).delete(`${apiPrefix}/${stuffId}`)
    expect(res.status).to.equal(HttpStatusCodes.NOT_FOUND)
  })

  after(async () => {
    await MongoDbConnector.Disconnect()
  })
})

In this file I test all API calls with their „good“ results and I also test some calls for some expected errors. You could write a test for every single mandatory property or every property that has got a default value to reach 100% test coverage but honestly, I don’t see the value.

Kommentar schreiben

Deine E-Mail-Adresse wird nicht veröffentlicht. Erforderliche Felder sind mit * markiert.