Production Architecture

This document gives an overview of the AWS and data architecture of the application.

Frontend

The frontend angular app when deployed exists on S3. It is bundled by content-hash, such that each new revision of the application has a unique filename. The app is then distributed via cloudfront CDN which has direct access to the S3 bucket. A copy of the bundled codebase is cached in each CDN node that downloads it, so that the app is highly available and quick to load for clients globally. Due to the unique hash naming of frontend code assets, whenever we deploy a new revision, we don’t require the user to refresh the cache, and it instead automatically downloads the new revision. This is achieved by not caching the root index.html, which references the latest built revision.

Backend

The backend consists of a Sidekiq background worker and Rails API behind an application load balancer. The API and Background worker both run under Amazon’s Elastic Container Service and automatically scale up and down based on traffic fluctuations. The API uses Redis as a caching layer and job broker, S3 for long term file storage, and Postgres for relational data storage.

Data

Redis

Redis is used as a caching layer for the REST API. It is also used as a broker for our background processing queue.

S3

S3 is used for web hosting for our angular frontend, distributed via cloudfront. S3 is also used for long term application storage and contains every generated PDF resume as well as all candidate uploads, including any original source resume, profile photos etc.

Postgres

Is used to store and access all relational and document based application data. All passwords, and integration credentials, are MD5 hashed and salted for security purposes, but the entire database will contain sensitive candidate information that must be deployed securely in the appropriate regions. There are some stored procedures in the database but it is used very sparingly. Mustard has deployed this postgres instance via RDS for simplicity.

Algolia

Algolia is a powerful external service used to index and retrieve candidate profiles for mustard client databases. Algolia enables us to index candidates by skill, role, company etc, but also allows the application to inform the search engine of synonyms to aid in natural language searches.

Daxtra

Daxtra is the external API the mustard application uses to parse candidate resume files and extract key metadata for post-processing.

Other

AWS Route53

Route53 is used to manage the domain records for the whole stack. It includes email records, configuration for public API endpoints and routes users to our frontend.

AWS CodePipeline

CodePipeline is our deployment pipeline software of choice. The deployment configuration is version controlled with our code, and is triggered by a new release version on GitHub.

Grafana/Graphite

Mustard is built to emit certain metrics from the application and send them to a config specified url. Mustard has a self managed grafana dashboard and graphite cluster to ingest these metrics and display them on application dashboards. The graphite cluster is built from this docker image running in ECS https://github.com/hopsoft/docker-graphite-statsd. The grafana dashboard is prebuilt software bought in the AWS marketplace.

Elastic Container Registry

This is used to hold all revisions of API docker images.

Elastic Container Service

Used to run our REST Api and sidekiq worker. Environment files for these containers can be found in their respective task definitions.

Staging

The staging environment for the mustard application is identical to that of production, except for instance sizes being significantly smaller due to lower throughput requirements.