cluster function

Use a multi-core cluster

beginCluster creates, and endCluster deletes a 'snow' cluster object. This object can be used for multi-core computing with those 'raster' functions that support it.

beginCluster determines the number of nodes (cores) that are available and uses all of them (unless the argument n is used).

NOTE: beginCluster may fail when the package 'nws' is installed. You can fix that by removing the 'nws' package, or by setting the cluster type manually, e.g. beginCluster(type="SOCK")

endCluster closes the cluster and removes the object.

The use of the cluster is automatic in these functions: projectRaster, resample and in extract when using polygons.

clusterR is a flexible interface for using cluster with other functions. This function only works with functions that have a Raster* object as first argument and that operate on a cell by cell basis (i.e., there is no effect of neigboring cells) and return an object with the same number of cells as the input raster object. The first argument of the function called must be a Raster* object. There can only be one Raster* object argument. For example, it works with calc and it also works with overlay as long as you provide a single RasterStack or RasterBrick as the first argument.

This function is particularly useful to speed up computations in functions like predict, interpolate, and perhaps calc.

Among other functions, it does not work with merge, crop, mosaic, (dis)aggregate, resample, projectRaster, focal, distance, buffer, direction. But note that projectRaster has a build-in capacity for clustering that is automatically used if beginCluster() has been called.

beginCluster(n, type='SOCK', nice, exclude)
endCluster()
clusterR(x, fun, args=NULL, export=NULL, filename='', cl=NULL, m=2, ...)

Arguments

• n: Integer. The number of nodes to be used (optional)
• type: Character. The cluster type to be used
• nice: Integer. To set the prioirty for the workers, between -20 and 20 (UNIX like platforms only)
• exclude: Character. Packages to exclude from loading on the nodes (because they may fail there) but are required/loaded on the master
• x: Raster* object
• fun: function that takes x as its first argument
• args: list with the arguments for the function (excluding x, which should always be the first argument
• export: character. Vector of variable names to export to the cluster nodes such that the are visible to fun (e.g. a parameter that is not passed as an argument)
• filename: character. Output filename (optional)
• cl: cluster object (do not use it if beginCluster() has been called
• m: tuning parameter to determine how many blocks should be used. The number is rounded and multiplied with the number of nodes.
• ...: additional arguments as for writeRaster

Note

If you want to write your own cluster-enabled functions see getCluster, returnCluster, and the vignette about writing functions.

Returns

beginCluster and endCluster: None. The side effect is to create or delete a cluster object.

clusterR: as for the function called with argument fun

Examples

## Not run:

# set up the cluster object for parallel computing
beginCluster()

r <- raster()
values(r) <- 1:ncell(r)

x <- clusterR(r, sqrt, verbose=T)

f1 <- function(x) calc(x, sqrt)
y <- clusterR(r, f1)

s <- stack(r, r*2, r*3)
f2 <- function(d,e,f) (d + e) / (f * param)
param <- 122
ov <- clusterR(s, overlay, args=list(fun=f2), export='param')

pts <- matrix(c(0,0, 45,45), ncol=2, byrow=T)
d <- clusterR(r, distanceFromPoints, args=list(xy=pts))

values(r) <- runif(ncell(r))
m <- c(0, 0.25, 1,  0.25, 0.5, 2,  0.5, 1, 3)
m <- matrix(m, ncol=3, byrow=TRUE)
rc1 <- clusterR(r, reclassify, args=list(rcl=m, right=FALSE), 
               filename=rasterTmpFile(), datatype='INT2S', overwrite=TRUE)

# equivalent to:
rc2 <- reclassify(r, rcl=m, right=FALSE, filename=rasterTmpFile(), datatype='INT2S', overwrite=TRUE)

# example with the calc function
a <- 10
f3 <- function(x) sum(x)+a

z1 <- clusterR(s, calc, args=list(fun=f3), export='a')

# for some raster functions that use another function as an argument 
# you can write your own parallel function instead of using clusterR
# get cluster object created with beginCluster
cl <- getCluster()  

library(parallel)
clusterExport(cl, "a")
z2 <- calc(s, fun=function(x){ parApply(cl, x, 1, f3)} )
# set flag that cluster is available again
returnCluster()
#

# done with cluster object              
endCluster()
## End(Not run)

Author(s)

Matteo Mattiuzzi and Robert J. Hijmans