Page History

...

• Application Load Balancer - API Server routing, load balancing, and SSL termination. 
• EC2 Compute Hosts - R2D2 API server execution environment.
• AWS Systems Manager - shell access and administration.
• AWS Relational Database Service - R2D2 database.

Common Tasks

Deploy Updated R2D2 API Server Binaries

WARNING: Manually downing or upgrading R2D2 servers without first draining the instance from our load balancer !

The standard process for updating servers involves draining them from the load balancer one at a time, updating the service, and then re-enabling them. This approach ensures continuous network availability and prevents the loss of inbound or queued API calls. A Bash script automates this process while maintaining service stability. However, updates should be performed during low-traffic periods, as serving capacity is temporarily reduced during the update.

1. Build and publish binaries with the prod tag as described later in this guide.

2. From a developer machine with admin AWS credentials (members of the JEDI-infra team) clone the r2d2 repository.

git clone https://github.com/JCSDA-internal/r2d2.git && cd r2d2

3. Run the following command and wait about 10 minutes for the full rollout to complete.

./server/scripts/update_prod_servers.sh --operation update 

Note: If the rollout fails for any reason, refer to the update_prod_servers.sh script, which contains the manual update process and extensive debugging information. This script serves as the central resource for detailed rollout procedures and documentation.

Deploy Cloud Formation Updates

1. Copy the updated prod.yaml config to our infrastructure-as-code bucket and get the versioned URL for the file as shown here. 

   
   

   aws s3 cp server/cfn/prod.yaml s3://jcsda-usaf-iac-artifacts/r2d2/prod.yaml
   file_version=$(aws s3api list-object-versions \
                      --bucket jcsda-usaf-iac-artifacts \
                      --prefix "r2d2/prod.yaml"  \
                  | jq -r '.Versions[] | select(.IsLatest==true) | .VersionId')
   echo "https://jcsda-usaf-iac-artifacts.s3.us-east-2.amazonaws.com/r2d2/prod.yaml?versionId=${file_version}"

2. Go to the CloudFormation stack "r2d2-api-prod" in the AWS console (or use quick-ref link above) and click "Update".
3. In the update menu, select "Replace existing template" and enter the URL generated in the first step. Change any parameters and review deploy options.
   1. Review the change log and ensure that EC2 instances and RDS instances are not replaced.
4. Monitor the CloudFormation rollout in console and ensure it executes to completion.

...