gar_batch_walk function

Walk data through batches

Convenience function for walking through data in batches

gar_batch_walk(
  f,
  walk_vector,
  gar_pars = NULL,
  gar_paths = NULL,
  the_body = NULL,
  pars_walk = NULL,
  path_walk = NULL,
  body_walk = NULL,
  batch_size = 10,
  batch_function = NULL,
  data_frame_output = TRUE,
  ...,
  batch_endpoint = getOption("googleAuthR.batch_endpoint", default =
    "https://www.googleapis.com/batch")
)

Arguments

• f: a function from gar_api_generator
• walk_vector: a vector of the parameter or path to change
• gar_pars: a list of parameter arguments for f
• gar_paths: a list of path arguments for f
• the_body: a list of body arguments for f
• pars_walk: a character vector of the parameter(s) to modify for each walk of f
• path_walk: a character vector of the path(s) to modify for each walk of f
• body_walk: a character vector of the body(s) to modify for each walk of f
• batch_size: size of each request to Google /batch API
• batch_function: a function that will act on the result list of each batch API call
• data_frame_output: if the list of lists are dataframes, you can bind them all by setting to TRUE
• ...: further arguments passed to the data parse function of f
• batch_endpoint: the batch API endpoint to send

Returns

if data_frame_output is FALSE : A list of lists. Outer list the length of number of batches required, inner lists the results from the calls

if data_frame_output is TRUE : The list of lists will attempt to rbind all the results

Details

You can modify more than one parameter or path arg, but it must be the same walked vector e.g. start = end = x

Many Google APIs have batch_size limits greater than 10, 1000 is common.

The f function needs to be a gar_api_generator() function that uses one of path_args, pars_args or body_args to construct the URL (rather than say using sprintf() to create the API URL).

You don't need to set the headers in the Google docs for batching API functions - those are done for you.

The argument walk_vector needs to be a vector of the values of the arguments to walk over, which you indicate will walk over the pars/path or body arguments on the function via on of the *_walk arguments e.g. if walking over id=1, id=2, for a path argument then it would be path_walk="id" and walk_vector=c(1,2,3,4)

The gar_* parameter is required to pass intended for other arguments to the function f you may need to pass through.

gar_batch_walk() only supports changing one value at a time, for one or multiple arguments (I think only changing the start-date, end-date example would be the case when you walk through more than one per call)

batch_size should be over 1 for batching to be of any benefit at all

The batch_function argument gives you a way to operate on the parsed output of each call

Examples

## Not run:

                               

# get a webproperty per account 
getAccountInfo <- gar_api_generator(
  "https://www.googleapis.com/analytics/v3/management/accounts",
  "GET", data_parse_function = function(x) unique(x$items$id))

getWebpropertyInfo <- gar_api_generator(
  "https://www.googleapis.com/analytics/v3/management/", # don't use sprintf to construct this
  "GET",
  path_args = list(accounts = "default", webproperties = ""),
  data_parse_function = function(x) x$items)

walkData <- function(){

  # here due to R lazy evaluation  
  accs <- getAccountInfo()
  gar_batch_walk(getWebpropertyInfo, 
                 walk_vector = accs,
                 gar_paths = list("webproperties" = ""),
                 path_walk = "accounts",
                 batch_size = 100, data_frame_output = FALSE)
                 }
                 
# do the walk
walkData()

# to walk body data, be careful to modify a top level body name:
changed_emails <- lapply(email, function(x){userRef = list(email = x)})

batched <- gar_batch_walk(users, 
                       walk_vector = changed_emails, 
                       the_body = list(
                           permissions = list(
                             local = list(permissions)
                           ),
                           userRef = list(
                             email = email[[1]]
                         )
                        ),
                        body_walk = "userRef",
                        batch_size = 300,
                        data_frame_output = FALSE)
## End(Not run)

See Also

Other batch functions: gar_batch()