A Distributed Worker Launcher Framework
R6
client class.
Controller group class
Sequential controller class
Controller class
Local process launcher class
Launcher abstract class
Local monitor class
R6
relay class.
R6
throttle class.
R6
TLS class.
Deprecated: terminate dispatchers and/or workers
Create a client object.
Create a controller group.
Create a controller with a local process launcher.
Crew assertion
Options for logging resource usage metrics.
Random name
Create a crew
relay object.
Create a sequential controller.
Create a controller object from a client and launcher.
Deprecate a crew
feature.
Evaluate an R command and return results as a monad.
Create a launcher with local process workers.
Create an abstract launcher.
Create a local monitor object.
Local crew
launcher options.
Retry code.
Manually terminate a local process.
Get the termination signal.
Create a stateful throttling object.
Configure TLS.
Crew worker.
crew: a distributed worker launcher framework
Objects exported from other packages
In computationally demanding analysis projects, statisticians and data scientists asynchronously deploy long-running tasks to distributed systems, ranging from traditional clusters to cloud services. The 'NNG'-powered 'mirai' R package by Gao (2023) <doi:10.5281/zenodo.7912722> is a sleek and sophisticated scheduler that efficiently processes these intense workloads. The 'crew' package extends 'mirai' with a unifying interface for third-party worker launchers. Inspiration also comes from packages. 'future' by Bengtsson (2021) <doi:10.32614/RJ-2021-048>, 'rrq' by FitzJohn and Ashton (2023) <https://github.com/mrc-ide/rrq>, 'clustermq' by Schubert (2019) <doi:10.1093/bioinformatics/btz284>), and 'batchtools' by Lang, Bischel, and Surmann (2017) <doi:10.21105/joss.00135>.
Useful links