A Distributed Worker Launcher Framework
Crew assertion
Local asynchronous client object.
R6
async class.
R6
client class.
Controller group class
Controller class
Local process launcher class
Launcher abstract class
Local monitor class
R6
relay class.
R6
throttle class.
R6
TLS class.
Terminate dispatchers and/or workers
Create a client object.
Create a controller group.
Create a controller with a local process launcher.
Create a controller object from a client and launcher.
Deprecate a crew
feature.
Run an asynchronous task in the crew launcher.
Evaluate an R command and return results as a monad.
Create a launcher with local process workers.
Create an abstract launcher.
Create a local monitor object.
Local crew
launcher options.
Options for logging resource usage metrics.
Random name
Create a crew
relay object.
Retry code.
Manually terminate a local process.
Get the termination signal.
Create a stateful throttling object.
Configure TLS.
Crew worker.
crew: a distributed worker launcher framework
Objects exported from other packages
In computationally demanding analysis projects, statisticians and data scientists asynchronously deploy long-running tasks to distributed systems, ranging from traditional clusters to cloud services. The 'NNG'-powered 'mirai' R package by Gao (2023) <doi:10.5281/zenodo.7912722> is a sleek and sophisticated scheduler that efficiently processes these intense workloads. The 'crew' package extends 'mirai' with a unifying interface for third-party worker launchers. Inspiration also comes from packages. 'future' by Bengtsson (2021) <doi:10.32614/RJ-2021-048>, 'rrq' by FitzJohn and Ashton (2023) <https://github.com/mrc-ide/rrq>, 'clustermq' by Schubert (2019) <doi:10.1093/bioinformatics/btz284>), and 'batchtools' by Lang, Bischel, and Surmann (2017) <doi:10.21105/joss.00135>.
Useful links