We are at an age where a single pre-production environment is just not enough. Here’s the what, why, how and the reasons I want to make this feature more accessible with FullStacked.
What is Review Apps
Review Apps is the concept of deploying a branch of your app’s repository to allow user tests and visual reviews. It’s usually triggered by an automated jobs when the branch is in merge/pull request state. It creates a temporary version of your web app so you can actually view and review the modifications applied from the branch in question.
Why Review Apps is a Must in Web App Development
1. It battle tests your deployment process.
Unit, end-to-end (e2e) and integration tests are meant to test your app’s code. They rarely test your deployment and unless you have a good dry-run process, deploying your latest release stays an uncertain and stressful event. The more you repeat the task of deploying, the more resilient you’ll update your process and gain better confidence towards it.
2. It shows the actual rendered UI.
As reviewers, we can always try to figure out the rendered UI through reading lines of code. With margin, padding, flex-box, font-size, etc. It’s quite hard to be sure about how it actually looks like. Worse, reviewing responsiveness is barely impossible. Being able to see it and try the current state of the app is just such a nice experience.
3. It allows UI/UX designers to participate in merge/pull requests.
Since it’s possible to view a live version of the modified app. Anyone can access it and give feedback on how it feels and looks. THIS IS HUGE. Back-and-forth between developers and designers can feel like a painful waste of time when happening at the wrong moment. Sometimes too early: way before the development of the feature gets started; sometime too late: after the feature is already deployed to production. Now we can debate on all of this during the merging process. Amazing!
4. It makes the deployments more understandable for non-devops developers.
Since every single push to the remote repository during a merge/pull request will go through the deployment pipeline, you better make the CI job simple and debuggable. Since it won’t only be triggered by the DevOps team members, everyone has an insight of what’s going on there!
5. It can be setup for almost no cost
Disclaimer: whether you have a pro-Kubernetes in the team or you will have to mess around a little to make it functional. GitLab has some very good documentation on how to set it up, but their example mainly uses Kubernetes.
For my part, I’ve never used Kubernetes and to this day, it still feels like it’s too much overhead for the projects I’m working on. But I wanted Review Apps more than anything, so I managed to make it work without it.
How to set up Review Apps without any Kubernetes knowledge
Our goal here is to start your forked app and route a subdomain to it. To achieve that, we will have to find an available port in our Review Apps environment, then startup our app and make it listen to that port. Finally, we’ll make sure we have a subdomain pointing to it. Piece of cake!
Steps to get there :
- Environments
- DNS Initial Setup (with CloudFlare)
- Nginx #1
- Docker Compose
- shell scripts (sh)
- Nginx #2
- Dynamically Updating DNS (again with CloudFlare)
Environments
Right now, you probably already have a server/virtual machine/VPS instance for your live app, you’ll need a new environment for your Review Apps because you don’t want to mess around to close of your production environment or steal resources from it. You might even get something less expensive since it is for development purposes.
DNS Initial Setup
For my part, I use the services of CloudFlare — Free Tier for the DNS management. I started with pointing an A record to my production environment static IP (e.g., example.com => 195.246.173.113). I also really like CloudFlare for the Edge Certificates and especially the Always Use HTTPS feature. With these, we will assume we always enter in our server at port 443 with a valid SSL certificate.
We’re passing through our production environment nginx because CloudFlare — Free Tier SSL Certificate doesn’t cover sub-subdomains (e.g., *.review.example.com and more about it here). Also, we don’t want to add a wildcard subdomain DNS record (e.g., *.example.com => 192.246.173.113) because it’s bad practice and it’s a bit insecure (any subdomains will reach something in your server…). So we will automate the DNS update every time we want to route a new subdomain. You will understand this more clearly at the last step.
If you have a pro/business account, you can request a certificate for something like *.review.example.com and route this to your Review Apps environment
(e.g., *.review.example.com => 140.211.48.232). With this, you can skip the nginx #1 and the last Update DNS steps.
Nginx #1
By reverse-proxying, a first Nginx splits our running production app for the root domain and our Review Apps environment for any subdomain starting with review-.
# nginx #1 configserver {
listen 443 ssl;
server_name "~^example\.com$";
ssl_certificate fullchain.pem;
ssl_certificate_key privkey.pem;location / { proxy_pass http://localhost:8000 # <-- our main app proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}server {
listen 443 ssl;
server_name "~^review-.*\.example\.com$";
ssl_certificate fullchain.pem;
ssl_certificate_key privkey.pem;location / { proxy_pass http://140.211.48.232:80; # <-- Review Apps VM proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
Docker-Compose
For the sake of simplicity, I decided to use docker compose. This way, I don’t need to keep track of NodeJS versions nor to install it with nvm in the Review Apps environment. Only docker and docker-compose are necessary. Plus, if you need any other services for your project like a database or some debugging tool, a docker-compose setup makes it all simpler.
# docker-compose.ymlversion: '3.7'services:
node:
image: node:15-alpine
restart: always
working_dir: /app
command: [ "node", "index.js" ]
ports:
- "${PORT}:8000"
volumes:
- .:/app
A configuration like this allows to start your app with a single command :
docker-compose -p my-awesome-webapp up -dShell scripts
This script pings your localhost from port 9000 and up until it finds a port that responds nothing (meaning it is available).
# /bin/sh
# ports.sh# start at port 9000 and increment by 1
INCREMENT=1
PORT=$((9000 - INCREMENT))# loop until empty response
RESPONSE=""
while [ -z "$RESPONSE" ]; do
PORT=$((PORT + INCREMENT))
URL=http://localhost:$PORT # source: https://unix.stackexchange.com/a/84818
RESPONSE="$(curl -sSf ${URL} 2>&1 > /dev/null)"
done# write PORT={AVAILABLE_PORT} in .env file
echo PORT=$PORT\ >> .env
At the end, it writes the result to a .env. This way, docker-compose can pick it up and replace the ${PORT} we seen in our docker-compose.yml file before (more about this here).
Now we will create a nginx.conf file from a template to reverse-proxy the subdomain we want to our localhost’s port chosen by the port availability script.
# /bin/sh
# nginx.sh# argv $1 is our subdomain name
# e.g., sh ./nginx.sh review-foo# pick up the PORT value in .env
. ./.env# copy the template
cp nginx-template.conf nginx.conf# update PORT and SUBDOMAIN values
perl -i -pe"s/{PORT}/$PORT/g" $NGINX_FILE
perl -i -pe"s/{SUBDOMAIN}/$1/g" $NGINX_FILE
This is the nginx configuration template and how it should results, we’ll talk more about it in the next section.
# nginx-template.confserver {
listen 80;
server_name {SUBDOMAIN}.example.com;
location / {
proxy_pass http://localhost:{PORT};
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
The result :
# nginx.confserver {
listen 80;
server_name review-foo.example.com;
location / {
proxy_pass http://localhost:9000;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
Nginx #2
At this point, we found an available port, we can start our app with docker-compose while listening to this port and we’ve setup a nginx config file that reverse-proxies our subdomain to this instance of our app. This last step explains how nginx can pick up that new .conf file.
Edit your main nginx config file (usually at /etc/nginx/nginx.conf) like this :
# nginx #2
# /etc/nginx/nginx.conf...http {
... include /etc/nginx/conf.d/*.conf;
include /example/*/nginx.conf; <-- this line is new
}
This shows that we will put our project files for each Review Apps instances at /example/{SANITIZED_BRANCH_NAME} (I use CI_COMMIT_REF_SLUG from GitLab predefined variables). The filesystem would then look like this :
/
|_ example
|_ foo
|_ .env <-- PORT=9000
|_ docker-compose.yml
|_ nginx.conf <-- review-foo.example.com => localhost:9000
|_ nginx.sh
|_ ports.sh
|_ index.js
|_ bar
|_ .env <-- PORT=9001
|_ docker-compose.yml
|_ nginx.conf <-- review-bar.example.com => localhost:9001
|_ nginx.sh
|_ ports.sh
|_ index.js
|_ xyz
|_ .env <-- PORT=9002
|_ docker-compose.yml
|_ nginx.conf <-- review-xyz.example.com => localhost:9002
|_ nginx.sh
|_ ports.sh
|_ index.jsNow, with a simple reload of nginx, all those nginx.conf will get picked up.
sudo systemctl reload nginxDynamically Updating DNS
As I explained before about CloudFlare — Free Tier issue with having to update the DNS records for SSL certificate reasons, all we have to do now is POST their Rest API to update our records. Easy.
# /bin/sh
# dns.sh# argv $1 is our subdomain name
# e.g., sh ./dns.sh review-foocurl -X POST "https://api.cloudflare.com/client/v4/zones/{ZONE ID}/dns_records" \
-H "Authorization: Bearer {YOUR API KEY}" \
-H "Content-Type: application/json" \
--data '{"type":"CNAME","name":"'"$1"'.example.com","content":"example.com","ttl":1,"proxied":true}'
Get your Zone ID and API Key on your domain overview dashboard.
After all this, here’s the path if we hit https://review-foo.example.com :
THERE WE GO! We managed to make it work! I added an example of a functional .gitlab-ci.yml job that automates everything.
Functional GitLab CI Job example
Using sshpass :
Review:Deploy:
stage: Review
only:
- merge_request
environment:
name: review/$CI_COMMIT_REF_SLUG
url: https://review-$CI_COMMIT_REF_SLUG.example.com
auto_stop_in: 1 day
on_stop: Review:Stop
script: # Build your app
- npm ci
- npm run build # Setup sshpass
- export SSHPASS=$SSH_PASS # Clear previous folder (if existant)
- sshpass ssh [...] rm -rf /example/$CI_COMMIT_REF_SLUG
# (re)Create the directory
- sshpass ssh [...] mkdir -p /example/$CI_COMMIT_REF_SLUG
# Ship the project files
- sshpass scp -r[...] "${PWD}"[...]:/example/$CI_COMMIT_REF_SLUG # Run the shell scripts described before
- sshpass ssh [...] sh /example/$CI_COMMIT_REF_SLUG/ports.sh
- sshpass ssh [...] sh /example/$CI_COMMIT_REF_SLUG/nginx.sh review-$CI_COMMIT_REF_SLUG # Start the docker compose
- sshpass ssh [...] docker-compose -p review-$CI_COMMIT_REF_SLUG -f /example/$CI_COMMIT_REF_SLUG/docker-compose.yml up -d # Reload gracefully nginx to pickup the new conf
- sshpass ssh [...] systemctl reload nginx # Update the DNS
- sh dns.sh review-$CI_COMMIT_REF_SLUG
Here a full sshpass line (I cropped for readability in the example) :
sshpass -e ssh -o StrictHostKeyChecking=no root@140.211.48.232 [...]Stopping the temporary environment
When the review process is done, we want to delete the temporary environment. I’m pretty sure if you got to this point, you can figure how to achieve that, but here’s a few hints.
- Down the docker-compose
docker-compose -p review-$CI_COMMIT_REF_SLUG -f /example/$CI_COMMIT_REF_SLUG/docker-compose.yml down2. Delete the directory
rm -rf /example/$CI_COMMIT_REF_SLUG3. Remove the DNS records
Use this snippet to help you
Making it all simpler
Wow, at this point, I’m wondering if it would not be easier to learn Kubernetes… Nah! I love Docker too much ❤. Although, I’m conscious that it is quite the puzzle and most of us don’t want to get into that much trouble just for a DevOps feature. I really want to abstract all this process through my latest project: FullStacked. It’s definitely in my long-term roadmap to unlock this wonderful and collaborative feature to more teams and communities. I want anyone to be able to deploy a branch with a single command from their local machine or from a simple automated CI Job/Action.
I hope I could help or inspire you into trying to set up Review Apps! Feel free to ask questions in the post responses. I’ll try to answer with the best of my knowledge.
Thanks for reading!
