4.4. Configuring selected storage and obtaining input data#
4.4.1. Install AWS CLI to obtain data from AWS S3 Bucket#
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
4.4.2. edit .cshrc file to add /usr/local/bin to path#
add /usr/local/bin to the set path line
Run csh at the command line
Verify you can run the aws command#
If not, you may need to logout and back in.
Set up your credentials for using s3 copy (you can skip this if you do not have credentials)
4.4.4. Change ownership#
sudo chown azureuser /shared/data
/dev/mapper/vg_cyclecloud_builtinshared-lv0 1000G 66G 935G 7% /shared
4.4.5. Use the S3 script to copy the CONUS input data from the CMAS s3 bucket#
Modify the script if you want to change where the data is saved to. Script currently uses /shared/data
check that the resulting directory structure matches the run script
Note, this input data requires 44 GB of disk space (if you use the yaml file to import the data to the lustre file system rather than copying the data you save this space)
CMAQ Cycle Cloud is configured to have 1 Terrabytes of space on the /shared filesystem, to allow multiple output runs to be stored.