Ayima Docs

Appearance

Managed project build & deployment process ​

Projects that are managed and controlled by this build & deployment process. ​

Build ​

When ever any of the systems under deployment control receive a commit to their github repository, Buddy picks up the commit and triggers the build pipeline for the relevant project. You will need to look at the pipeline in Buddy for the exact details, but generally the pipeline will build the required docker images for the project as defined by the Dockerfile(s) in the repo for that project.

The docker images receive a few tags, most importantly the github revision id (for e.g. 63ec5c0) which is used in the deployment step. The resulting built image(s) are then pushed to Amazon AWS's docker registry (ECR) in the eu-west-2 region and no more action is taken.

Deployment ​

When the staging or production deployment endpoint is run in Buddy a number of steps generally happen, although check the pipeline for the exact details. Broadly, the first step will fetch the "secrets" for the project (more on that later) from AWS and written to the relevant kubernetes config directory for the repo. Most often this is kubernetes/production or kubernetes/staging. Example configs can be found here.

The Kubernetes configuration is then updated with some deployment specific variables, most importantly the github revision docker tag is added to the config so Kubernetes knows which image to fetch from the AWS docker registry.

The kubernetes apply command is then run using the configs. Finally, if defined, a notification will be sent to a slack channel to inform team members of a deployment.

Secrets ​

Secrets in this context are configuration variables used by the application, but that are too sensetive to be stored directly in the github repo. This includes things like database connection details or API keys.

A blank secrets file is kept in the github repo as a form of reference. A populated version of that file is then stored in AWS Secrets manager.

The most important thing to note is secret values are stored in a base64 format, so if you need to edit or change a value you will need to convert it from base64 to plaintext or vice versa. There are a number of online tools to do this, or plugins for your IDE/editor of choice to do this.

Kubernetes ​

A quick primer on Kubernetes and how it's used for our projects. Kubernetes is a cluster "orchestration" platform used to run docker images built from our projects. The Kubernetes cluster is formed of a number of servers, currently formed of two servers or "nodes". Kubernetes will then deal with moving docker images or "pods" around on the cluster to optomise the workloads to make best use of the cluster. Kubernetes also handles things like routing traffic from the Load balancer to the pods, known as the Ingress.

Ayima's kubernetes cluster is a hosted Digital Ocean service, we just provide the configuration for our apps and other aspects of our setup.

Ingress ​

Outside of the configuration for the app in question, the main configs we supply can be found in the devops-k8s-cluster github repo and can be executed with the install.sh file in that folder. This config sets up the "ingress controller" which is basically just an instant of NGINX that runs inside Kubernetes and deals with routing traffic around the cluster. The other thing set up here is the Lets Encrypt certificate system which issues SSL certificates automatically and deals with keeping them up to date. Finally this ingress controller config also deploys a digital ocean load balancer and connects it to the cluster to allow routing of web traffic.

Each application also has specific configs to control ingress endpoints and the certificate issuance, for example this config for Trieste V2 API is here.

Docker Registry authentication ​

In order to pull docker images from the Amazon container registry as part of the deployment process the kubernetes cluster needs to maintain valid AWS credentials that have access to the container registry. These credentials only last a few hours, so we have an automatic pipeline in Buddy that refreshes these credentials. For deployments to continue to function its important this pipeline is not disrupted.