Rightsizing Workflows


In newer versions of metworx, many CPU and RAM combinations are available for master and worker nodes.

new-metworx

As a simple rule-of-thumb - the following guidance can be followed:

If you do not need to do any computational expensive activities on the master node (for example, just data assembly, postprocessing results, and submitting jobs to the grid), 2 vCPU's should be computational sufficient.

Note: One consideration is the installation of packages at project kickoff. Given pkgr can use multiple cores, one suggestion is to use a 4 or 8 core workflow when you know you will need to do a large installation of packages (such as a new project, or QC)

If you are doing heavy processing, to make sure the computational thread is completely clear as best as possible from other system activities, a 4 vCPU workflow may be selected. In general, if you do not have large simulations or many datasets that are 100's of megabytes or gigabytes, 16 GB of RAM should be sufficient. You can also check the https://<YOURWORKFLOW-ID>.metworx.com/rstudio/admin

admin-panel

This will show a dashboard of historical RAM and CPU usage to identify if you may need additional RAM or CPU.

Video tutorial