If you need immediate JEDI infrastructure support, please send a chat to the #jedi-infra-support Slack channel. As we build our knowledge base, we are going to try our best to document past issues and common questions in this wiki. It is recommended to search you issue here, in the slack channel, and also in the JEDI Documentation before reaching out for help.
Troubleshooting tips:
- Make sure your code is up to date.
- Try deleting your old venv and starting with fresh installs of solo, r2d2, ewok, and simobs.
- Rebuild jedi-bundle using the scripts available in jedi-tools' build_skylab.sh.
- Make sure your environment is set up correctly. Protip: use jedi-tools' setup.sh. We keep the HPC setup scripts up to date with the most recent release of spack-stack.
Previous Issues
skylab.jcsda.org is not responding
Occasionally skylab.jcsda.org might not respond, if that is the case the easiest solution is to reboot the machine this runs from via the AWS Console. You can ask a member of the infrastructure team to do this for you.
...
The current "lifetime" values are "debug" set to 14 days, "science" set to 180 days (6 months), "publication" set to 1825 days (5 years), and "release" set to indefinite days. More information can be found at R2D2's tutorial document.
Experiment scrubber
There is a scrubber in place for cleaning up old experiments and based on R2D2's "lifetime" key. Some common questions around this scrubber are addressed below.
Missing experiments and plots at https://experiments.jcsda.org/: If it is over 2 weeks since running the experiment, it is likely "lifetime" was set to the default and R2D2's cleanup worked as expected. If you need experiments to stay around longer see the above section "Updating R2D2 file lifetime".
Older experiments are showing up: The scrubber is working hard but has a big backlog to get to. If you see older experiments then it could be a sign that the scrubber is still weeding through these.
R2D2 install protobuf error
If you are using spack-stack 1.7.0, you might get the following error message when installing R2D2:
| Code Block |
|---|
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
cylc-flow 8.2.3 requires protobuf<4.22.0,>=4.21.2, but you have protobuf 3.20.1 which is incompatible.
Successfully installed protobuf-3.20.1 r2d2-2.3.0 |
This message is safe to ignore for now as it says "Successfully installed protobuf-3.20.1 r2d2-2.3.0".
MySQL exceptions and errors
Errors such as:
| Code Block |
|---|
raise get_mysql_exception(
mysql.connector.errors.ProgrammingError: 1146 (42S02): Table 'r2d2.item' doesn't exist |
indicate that your local R2D2 MySQL database is out of date. In order to update your local database, follow the instructions in R2D2's tutorial document.