SSH to a compute node
Sometimes you may want to log in to a compute node via SSH to run a job (e.g., R script) while keeping your master node free.
For example, say my R script uses
parallel::mclapply. This function forks processes without using the grid, so it's typically used to parallelize work on the master node. However, you may want to run this work on a compute node to leave your master node free to do other work.
- SSH in to your master node
- If you do not have any compute nodes active, bring one up:
echo "sleep 10" | qsub
- Find the IP address of the compute node when it comes up:
sudo su - -c "qconf -sh"
- SSH in to that compute node
ssh <one of the IP addresses>
- Navigate to /data, start a
screensession, and run your script
screen cd /data/<projDir> R CMD BATCH <file.R>
Because you started a screen session, feel free to close the terminal as the job will run in the background.