ParameterSweepDocumentation

Parameter Sweeps

Setting up Beocat to run a parameter sweep

1. First, you need to copy the project with all of its files into Beocat.
2. This can be done several ways, but make sure that you have them in the same folder, which needs to be named proj.
3. If you want to name it differently, you need to update run.sh to have the path to the new folder on line 15, where it has the command python ~/proj/Main_P.py
4. The project folders should be structured correctly if you kept them as given, but make sure that the project is structured as follows:
5. proj folder containing: all python and .sh files, a references folder containing ASTMA1096.txt, wireproperties.txt, and a geo_points folder containing all geometric point files
6. Make all .sh files in your proj folder executable, using the command chmod +x *.sh in your proj folder. This should make .sh files appear in green now, as shown below (note that not all files are shown)
7. 
8. Set up Beocat to run the Python files.
9. You can run the set_up_beocat.sh file to do this automatically. Run it by going to the folder that the file is in and entering the command ./set_up_beocat.sh
10. If set_up_beocat.sh doesn’t work, what you need to do is create a ‘virtual environment’ for running python. You can look at [help page] to see a different guide for doing so, but you can also look at the commands in set_up_beocat.sh to see an example of setting up the virtual environment.

Running a parameter sweep on Beocat

There are two ways to schedule the parameter sweep: directly or through the manager application.

Option 1: run directly

1. Log into Beocat and navigate to your proj folder
2. Run the following command to run the parameter sweep script by itself: ./sched_run.sh
3. If you’d like to run a whole parameter sweep: enter yes when prompted
4. Otherwise, enter no and then the number of jobs you do want to run
5. Wait for the program to finish, it will create all the input files and schedule as many jobs at once as it can.
6. Note: Do not delete the file remaining_jobs.csv unless you want to stop your current parameter sweep! It is used for keeping track of which chunks we have already submitted.
7. Follow the Run the next batch of jobs section below

Option 2: run through the manager application

1. Log into Beocat and navigate to your proj folder
2. Run the following command to run the manager program by itself: ./manager.sh
3. Enter s when prompted by the program
4. If you’d like to run a whole parameter sweep: enter yes when prompted
5. Otherwise, enter no and then the number of jobs you do want to run
6. Wait for the program to finish, it will create all the input files and schedule as many jobs at once as it can.
7. Complete any other operations you want to do and then exit the program.
8. Note: Do not delete the file remaining_jobs.csv unless you want to stop your current parameter sweep! It is used for keeping track of which chunks we have already submitted.
9. Follow the Run the next batch of jobs section below

Run the next batch of jobs/Resubmit failed jobs

Due to the limit on the number of jobs that can be submitted to Beocat, you will need to schedule the jobs in batches. There are two ways to handle doing this, as shown below. Please be sure that you wait for the majority of each batch to finish before scheduling the next batch

Option 1: run directly

1. Log into Beocat and navigate to your proj folder
2. Run the following command to run the script to submit the next jobs directly: ./manage_jobs.sh
3. Repeat this process every couple of days until you see the message No jobs remaining to be run!

Option 2: run through the manager application

1. Log into Beocat and navigate to your proj folder
2. Run the following command to run the manager program: ./manage_jobs.sh
3. Enter m when prompted by the program
4. Repeat this process every couple of days until you see the message No jobs remaining to be run!