AWS Deployment Guide
This notebook will guide you step-by-step on how to configure your AWS instance and docker container in order to deploy your backend. Thank you to Azeem Khan, Samit Poojary, Ethan Tran, Sophie Park, Edwin Abraham, Mr. Mortensen and Jeffrey Fonseca for resources and instructions.
- Prerequisites
- Accessing AWS EC2
- Local Setup
- AWS Setup
- DNS & NGINX Setup
- Certbot Config
- Changing Code will require Deployment Updates
- Congratulations!
Prerequisites
Fullfil these requirements prior to moving forward in this guide.
- Must have a backend repository on GitHub.
- Verify Backend is running on localhost. If it doesn't run on localhost, then it won't run on the deployment server!!!
- You must have a configured Domain Name pointing to the Public IP of your deployment server. See Duck DNS Guide for a guide on how to do this.
Accessing AWS EC2
Development Operations (DevOps) requires server access.
Amazon Web Services (AWS) Management Console
- Login to AWS Console using your account. Navigate to "EC2" and the "Instances" dropdown.
- From here, a variety of instances will show up. For this project, depending on which teacher you have, select either "NCS.cf Yeung CSP" or "NCS.gq Mort CSP"
Unrestricted Gateway to AWS EC2 Terminal
At school access theway.nighthawkcodingsociety.com and log in. Follow the prompt to go to your appropriate server. (ex. type in
gocf
orgogq
to match your Teachers server).- Username is `ubuntu`. Password hint is 3 Musketeers
Local Setup
Finding a Port
In AWS EC2;
- Run
docker ps
review list and find a port number starting in 8--- that is not in use. Valid ports are between 1024-49150, but we are asking that you stick to 8---.
Setting up Docker using VSCode
-
Open VSCode and navigate to your Flask repository (backend)
-
Make sure your
Dockerfile
matches the template below.
NOTE: If you have not changed your default python version for pip to python3, then you must use "pip3" instead of "pip".
FROM docker.io/python:3.10
WORKDIR /
# --- [Install python and pip] ---
RUN apt-get update && apt-get upgrade -y && \
apt-get install -y python3 python3-pip git
COPY . /
RUN pip install --no-cache-dir -r requirements.txt
RUN pip install gunicorn
ENV GUNICORN_CMD_ARGS="--workers=3 --bind=0.0.0.0:8080"
EXPOSE 8080
CMD [ "gunicorn", "main:app" ]
- Edit your
docker-compose.yml
according to the template below and comments.
version: '3'
services:
web:
image: flask_port_v1 # Change the image name to something unique to your project, aka my_unique_name_v1
build: .
ports:
- "8---:8080" # Edit the number on the left to match the port you selected
volumes:
- ./volumes:/volumes
- ./instance:/instance
restart: unless-stopped
-
Test
docker-compose up
orsudo docker-compose up
in your VSCode terminal (don't forget to 'cd' into the right repo.) Errors will be shown in Terminal, be sure NOT to type -d. -
After it's done building, type in
http://localhost:8---
in your browser (replace '8---' with your port number you've chosen) -
If all runs smoothly, push your changes to Github and continue to AWS setup
AWS Setup
In the AWS EC2 terminal;
-
cd ~
-
Clone your backend repo:
git clone github.com/server/project.git my_unique_name
-
Navigate to your repo:
cd my_unique_name
-
Build your site:
docker-compose up -d --build
-
Test your site:
curl localhost:8---
(replace '8---' with your port number)This should show you all the html content of your home page. If this provides 500 error you need to check your site on localhost. If it produces broken pipe error you need to check your ports between docker-compose.yml and Docker files. If the page does not have your content, you need to check
docker ps
as someone is using your port number.
If all runs smooth, continue to DNS & NGINX Setup
DNS & NGINX Setup
Follow Jeffrey's guide to duckdns (Public IP can be found below the AWS terminal or by running
ifconfig.me
)
Nginx setup in AWS terminal
-
Navigate to nginx:
cd /etc/nginx/sites-available
-
Create an nginx config file:
sudo nano myUniqueName
-
Use the format below to write into your config file, make updates according to comments
server {
listen 80;
listen [::]:80;
server_name -----.duckdns.org; # CHANGE SERVER NAME TO YOUR REGISTERED DOMAIN
location / {
proxy_pass http://localhost:8---; # CHANGE PORT TO YOUR UNIQUE PORT
# Simple requests
if ($request_method ~* "(GET|POST|PUT|DELETE)") { # Customize Request methods based on your needs
add_header "Access-Control-Allow-Origin" *;
}
# Preflighted requests
if ($request_method = OPTIONS ) {
add_header "Access-Control-Allow-Origin" *;
add_header "Access-Control-Allow-Methods" "GET, POST, PUT, DELETE, OPTIONS, HEAD"; # Make sure the request methods above match here
add_header "Access-Control-Allow-Headers" "Authorization, Origin, X-Requested-With, Content-Type, Accept";
return 200;
}
}
}
-
To save changes,
ctl X
orcmd X
, theny
, thenenter
-
Create a symbolic link:
cd /etc/nginx/sites-enabled
, thensudo ln -s /etc/nginx/sites-available/myUniqueName /etc/nginx/sites-enabled
(change myUniqueName to your nginx config file name) -
Validate by running:
sudo nginx -t
-
Restart nginx by running
sudo systemctl restart nginx
-
Test your domain name on your desktop browser now (only http://, not https://)
If all runs smoothly, continue to Certbot config
sudo certbot --nginx
Ideal outcome is shown below
- There are two outcomes during Certbot
- Success, test your domain name on your desktop browser now using https://
- Failure, follow guidance provided by CertBot
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Plugins selected: Authenticator nginx, Installer nginx
Which names would you like to activate HTTPS for?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: coolcodersjava.pw
2: www.coolcodersjava.pw
3: ajarcade.duckdns.org
4: flowhealth.duckdns.org
5: goatedgroup.duckdns.org
6: jasj-inventory.duckdns.org
7: recipies.duckdns.org
8: ssvgcars.duckdns.org
9: userapi.duckdns.org
10: fr0st.ml
11: www.fr0st.ml
12: agenda.nighthawkcodescrums.gq
13: coolcoders.nighthawkcodescrums.gq
14: escaperoom.nighthawkcodescrums.gq
15: frost.nighthawkcodescrums.gq
16: jame.nighthawkcodescrums.gq
17: lawnmowers.nighthawkcodescrums.gq
18: loopholegames.nighthawkcodescrums.gq
19: musicmania.nighthawkcodescrums.gq
20: nba.nighthawkcodescrums.gq
21: sadv.nighthawkcodescrums.gq
22: ssjn.nighthawkcodescrums.gq
23: stocks.nighthawkcodescrums.gq
24: striver.nighthawkcodescrums.gq
25: tngc.nighthawkcodescrums.gq
26: white.nighthawkcodescrums.gq
27: workwatch.nighthawkcodescrums.gq
28: cars.nighthawkcodingsociety.com
29: dolphin.nighthawkcodingsociety.com
30: saakd.nighthawkcodingsociety.com
31: pythonalflask.tk
32: www.pythonalflask.tk
33: teambrobro.tk
34: www.teambrobro.tk
35: teamcheeseatimetime.tk
36: www.teamcheeseatimetime.tk
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel): # ENTER YOUR CORRESPONDING NUMBER
Cert not yet due for renewal
You have an existing certificate that has exactly the same domains or certificate name you requested and isn't close to expiry.
(ref: /etc/letsencrypt/renewal/nighthawkcodingsociety.com-0001.conf)
What would you like to do?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: Attempt to reinstall this existing certificate
2: Renew & replace the cert (limit ~5 per 7 days)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for nighthawkcodingsociety.com
http-01 challenge for csa.nighthawkcodingsociety.com
http-01 challenge for cso.nighthawkcodingsociety.com
http-01 challenge for flm.nighthawkcodingsociety.com
Waiting for verification...
Cleaning up challenges
Deploying Certificate to VirtualHost /etc/nginx/sites-enabled/nighthawk_society
Deploying Certificate to VirtualHost /etc/nginx/sites-enabled/nighthawk_csa
Deploying Certificate to VirtualHost /etc/nginx/sites-enabled/nighthawk_csp
Deploying Certificate to VirtualHost /etc/nginx/sites-enabled/nighthawk_flm
Please choose whether or not to redirect HTTP traffic to HTTPS, removing HTTP access.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1: No redirect - Make no further changes to the webserver configuration.
2: Redirect - Make all requests redirect to secure HTTPS access. Choose this for
new sites, or if you're confident your site works on HTTPS. You can undo this
change by editing your web server's configuration.
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Select the appropriate number [1-2] then [enter] (press 'c' to cancel): 2
Traffic on port 80 already redirecting to ssl in /etc/nginx/sites-enabled/nighthawk_society
Traffic on port 80 already redirecting to ssl in /etc/nginx/sites-enabled/nighthawk_csa
Traffic on port 80 already redirecting to ssl in /etc/nginx/sites-enabled/nighthawk_csp
Traffic on port 80 already redirecting to ssl in /etc/nginx/sites-enabled/nighthawk_flm
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Your existing certificate has been successfully renewed, and the new certificate
has been installed.
The new certificate covers the following domains:
https://nighthawkcodingsociety.com,
https://csa.nighthawkcodingsociety.com,
https://csp.nighthawkcodingsociety.com, and
https://flm.nighthawkcodingsociety.com,
You should test your configuration at:
https://www.ssllabs.com/ssltest/analyze.html?d=nighthawkcodingsociety.com
https://www.ssllabs.com/ssltest/analyze.html?d=csa.nighthawkcodingsociety.com
https://www.ssllabs.com/ssltest/analyze.html?d=csp.nighthawkcodingsociety.com
https://www.ssllabs.com/ssltest/analyze.html?d=flm.nighthawkcodingsociety.com
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at:
/etc/letsencrypt/live/nighthawkcodingsociety.com-0001/fullchain.pem
Your key file has been saved at:
/etc/letsencrypt/live/nighthawkcodingsociety.com-0001/privkey.pem
Your cert will expire on 2022-03-06. To obtain a new or tweaked
version of this certificate in the future, simply run certbot again
with the "certonly" option. To non-interactively renew *all* of
your certificates, run "certbot renew"
- If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le
Changing Code will require Deployment Updates
Changing Code in VSCode
To keep deployment working, good practices in your coding process with verifications prior to pushing code to GitHub will save a lot of troubleshooting.
-
Make sure to
git pull
before making changesThis will make sure that you pull any changes made by your team, and prevents merge conflicts
-
Open terminal in VSCode and run
python3 main.py
(Make sure you cd into your repo)This should give you a local address where your flask is running. Open this in your browser to see your changes live
-
Make changes that are needed
Refer to your running site often to see changes as you develop
-
Commit your changes locally
Do not
Sync
change from UI orgit push
from terminal yet, justCommit
. It is great practice to Commit Often with sensible comments. Anytime after you commit you can pull team members changes for additional verifications. -
Before updating deployment start Docker Desktop app and test your Web Application.
Test
docker-compose up
orsudo docker-compose up
in your VSCode terminal (don't forget to 'cd' into the right repo). -
After Docker is done building, type in
http://localhost:8---
in your browser.> Replace '8---' with your port number. Review your personal changes and team members changes in running site. As long as Docker Desktop and Docker App is running, you can make little changes and save, they should refresh in site within a few seconds. Any errors, runtime errors, will appear in browser of VSCode terminal, read messages thoroughly and debug errors. Docker Desktop may consume a lot of CPU resources, if you are unplugged you may want to close it after you are done testing. -
If all goes well,
Sync
change from UI orgit push
from terminal.If you can't push review
git status
from terminal. Resolve all open filesgit restore
orgit commit
, thengit pull
and repeat steps 5 to 7.
Pulling Changes into AWS EC2 deployment
Updates should be quick and easy, as long as your teams verifies problems on localhost prior to these steps.
In your AWS EC2 terminal;
- Navigate to your repo:
cd ~/my_unique_name
-
docker-compose down
Test Server in browser using https://, it should be down (502 Bad Gateway in browser)
-
git pull
-
Rebuild your docker container:
docker-compose up -d --build
Test Server in browser using https://, sll updates should be up and running on internet.
Optional, Troubleshooting checks on AWS EC2
These commands let you see status of your running web application on AWS EC2
-
Try to curl:
curl localhost:8---
(replace '8---' with your port number)> Verify home pages is yours -
Run
docker-compose ps
Perform check on your container, verify docker is up
-
Run
docker ps
Perform checks on all containers and all images