Websites: https://experiments.jcsda.org/ and https://skylab.jcsda.org/
| Table of Contents |
|---|
Our web applications, https://experiments.jcsda.org and https://skylab.jcsda.org run on two individual AWS EC2 instances in us-east-1. They are Voila applications running on port 8866 on an nginx web server on ports 80/443. The web apps are simply python notebooks in the diag-plots repository with r2d2 and solo as the only dependencies. These packages are located in /home/ubuntu. Python3 virtual environments, e.i. "venv" are not used for the web apps. All python code is installed to the system python. It was the only way we could figure out to get the notebooks to work with Viola and nginx over https.
The Viola middleware service strips the cells from each notebook at runtime and displays only the user interface graphics required for the drop down menus and image viewing. Every time someone visits one of these web pages, a new Python3 kernel is started in memory. Over time, this leads to OutOfMemoryErrors from the OS and the web apps freeze up and the instance will have to be rebooted as detailed below.
The error logs are located at /var/log/nginx/error.log and can be accessed by executing
journalctl -u voila.service |
The https nginx configurations are at /etc/nginx/sites-enabled/experiments.jcsda.org and /etc/nginx/sites-enabled/skylab.jcsda.org.
WARNING: Do not change the https nginx configurations unless you know what you are doing.
server {
server_name experiments.jcsda.org;
proxy_buffering off;
location / {
proxy_pass http://localhost:8866;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 86400;
}
client_max_body_size 100M;
error_log /var/log/nginx/error.log;
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/experiments.jcsda.org/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/experiments.jcsda.org/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
if ($host = experiments.jcsda.org) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
server_name experiments.jcsda.org;
return 301 https://$host$request_uri;
# return 404; # managed by Certbot
} |
Restart the following services and run:
sudo systemctl restart nginx.service sudo systemclt daemon-reload sudo systemclt restart voila.service |
Verify the website is back up and running.
The certificate should be automatically updated and managed via CertBot and Let's Encrypt. If you need to manually update the certificate you can follow this procedure.
Run:
sudo certbot --nginx |
Instructions from https://docs.google.com/document/d/1vqK-qAxBGt_I9j6VG1SSLtPhQp89PzV4Y32yuWLWbUg/edit
Voila documentation: https://voila.readthedocs.io/en/stable/deploy.html#running-voila-on-a-private-server
WARNING: These instructions are old and some steps might be outdated or simply not correct for the current web apps.
Start a new EC2 instance using the console:
Create nginx Service:
Install nginx (enter in SSH’d terminal):
sudo apt install nginx sudo systemctl status nginx |
Create the file /etc/nginx/sites-enabled/{website_name}.jcsda.org with the following content:
server {
listen 80;
server_name {website_name}.jcsda.org;
proxy_buffering off;
location / {
proxy_pass http://localhost:8866;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_read_timeout 86400;
}
client_max_body_size 100M;
error_log /var/log/nginx/error.log;
} |
Enable and start the nginx service.
sudo systemctl enable nginx.service sudo systemctl start nginx.service |
Install pip
sudo apt update && sudo apt install python3-pip |
Clone Git Repo and Install Dependencies:
ssh-keygen -t ed25519 -C "{email}@ucar.edu" |
cat /home/ubuntu/.ssh/id_ed25519.pub |
From instance terminal, clone feature/voila branch. Note: the branch will change to “develop” in the future - unsure if this is the case right now.
git clone -b feature/voila git@github.com:JCSDA-internal/jtd.git |
Install dependencies
sudo python3 -m pip install --no-cache-dir -r jtd/voila/requirements.txt |
Create Voila Service:
Create /usr/lib/systemd/system/voila.service with the contents:
[Unit] Description=Voila [Service] Type=simple PIDFile=/run/voila.pid ExecStart=voila --no-browser --show_traceback=False jtd/voila/app.ipynb User=ubuntu WorkingDirectory=/home/ubuntu/ Restart=always RestartSec=10 [Install] WantedBy=multi-user.target |
Enable and start the voila.service:
sudo systemctl enable voila.service sudo systemctl start voila.service |
Restart nginx
sudo systemctl restart nginx.service |
Enable access to https:
sudo snap install core; sudo snap refresh core sudo snap install --classic certbot sudo ln -s /snap/bin/certbot /usr/bin/certbot sudo certbot --nginx |
import ipywidgets as widgets
out = widgets.Output(layout={'border': '1px solid black'})
display(out)
from jedi_data_viewer_experiments import JediDataViewerExperiments
jedi_data_viewer_experiments = JediDataViewerExperiments(out)
with self._out: print("")
Issue: the web pages will fail to load/ timeout
Explanation: Each time the website is hit a new python kernel is opened and will stay open forever. Therefore, when the instance memory is full of these python kernels the page will go down.
Solution: Reboot the EC2 Instance (see procedure above)