18.1. Advanced Tutorial

• Learn how to install CMAQ software and underlying libraries, copy input data, and run CMAQ.

• 18.1.1. Modify Cyclecloud CMAQ Cluster
  • If you make changes to the nodes you must run the following commands
• 18.1.2. Install CMAQ and pre-requisite libraries on linux on Alinux
  • Login to updated cluster
  • Change shell to use .tcsh
  • Log out and then log back in to activate the tcsh shell
  • Optional Step to allow multiple users to run on the CycleCloud Cluster
  • Check to see if the group is added to your user ID
  • Make the /shared/build directory
  • Change ownership to your username
  • Make the /shared/cyclecloud-cmaq directory
  • Change ownership to your username
  • Install git
    • Install the cluster-cmaq git repo to the /shared directory
    • Optional - Change the group to cmaq recursively for the /shared directory/build
    • Check what modules are available on the cluster
    • Load the openmpi module
    • Verify the gcc compiler version is greater than 8.0
    • Change directories to install and build the libraries and CMAQ
    • Build netcdf C and netcdf F libraries - these scripts work for the gcc 8+ compiler
    • A .cshrc script with LD_LIBRARY_PATH was copied to your home directory, enter the shell again and check environment variables that were set using
    • If the .cshrc wasn’t created use the following command to create it
    • Execute the shell to activate it
    • Verify that you see the following setting
    • Build I/O API library
    • Build CMAQ
• 18.1.3. Install CMAQ and pre-requisite libraries on linux on SUSE Linux
  • Login to updated cluster
  • Change shell to use .tcsh
  • Log out and then log back in to activate the tcsh shell
  • Optional Step to allow multiple users to run on the CycleCloud Cluster
  • Check to see if the group is added to your user ID
  • Make the /shared/build directory
  • Change ownership to your username
  • Make the /shared/cyclecloud-cmaq directory
  • Change ownership to your username
  • Install git
    • Install the cluster-cmaq git repo to the /shared directory
    • Optional - Change the group to cmaq recursively for the /shared directory/build
    • Check what modules are available on the cluster
    • Install openmpi
    • Install gnu-compilers
    • Load the gcc copiler - note, this may have been automatically loaded by the openmpi module
    • Verify the gcc compiler version is greater than 8.0
    • Change directories to install and build the libraries and CMAQ
    • Build netcdf C and netcdf F libraries - these scripts work for the gcc 8+ compiler
    • A .cshrc script with LD_LIBRARY_PATH was copied to your home directory, enter the shell again and check environment variables that were set using
    • If the .cshrc wasn’t created use the following command to create it
    • Execute the shell to activate it
    • Verify that you see the following setting
    • Build I/O API library
    • Build CMAQ
• 18.1.4. Configuring selected storage and obtaining input data
  • Install AWS CLI to obtain data from AWS S3 Bucket
  • edit .cshrc file to add /usr/local/bin to path
    • Verify you can run the aws command
  • Azure Cyclecloud install input on the /shared/data directory
  • Change ownership
  • Use the following aws cp command to copy the CONUS input data from the CMAS s3 bucket
• 18.1.5. Copy the run scripts from the CycleCloud repo to run on HBv120
• 18.1.6. Edit the run script to run on 192 pes
• 18.1.7. Check the status in the queue
• 18.1.8. check the timings while the job is still running using the following command
• 18.1.9. When the job has completed, use tail to view the timing from the log file.
• 18.1.10. Check whether the scheduler thinks there are cpus or vcpus
• 18.1.11. Edit the run script to run on 96 pes
• 18.1.12. Check the timing after run completed
• 18.1.13. Copy the run scripts from the CycleCloud repo
• 18.1.14. Run the CONUS Domain on 176 pes
• 18.1.15. Check the status in the queue
• 18.1.16. check the timings while the job is still running using the following command
• 18.1.17. When the job has completed, use tail to view the timing from the log file.
• 18.1.18. Check whether the scheduler thinks there are cpus or vcpus