copy_to function

Copy a (very small) local data frame to Impala

copy_to inserts the contents of a local data frame into a new Impala table. copy_to is intended to be used only with very small data frames. It uses the SQL INSERT ... VALUES() technique, which is not suitable for loading large amounts of data. By default, this function will throw an error if you attempt to copy a data frame with more than 1000 row/column positions. You can increase this limit at your own risk by setting the option implyr.copy_to_size_limit to a higher number.

This package does not provide tools for loading larger amounts of local data into Impala tables. This is because Impala can query data stored in several different filesystems and storage systems (HDFS, Apache Kudu, Apache HBase, Amazon S3, Microsoft ADLS, and Dell EMC Isilon) and Impala does not include built-in capability for loading local data into these systems.

## S3 method for class 'src_impala'
copy_to(
  dest,
  df,
  name = deparse(substitute(df)),
  overwrite = FALSE,
  types = NULL,
  temporary = TRUE,
  unique_indexes = NULL,
  indexes = NULL,
  analyze = FALSE,
  external = FALSE,
  force = FALSE,
  field_terminator = NULL,
  line_terminator = NULL,
  file_format = NULL,
  ...
)

Arguments

• dest: an object with class with class src_impala

• df: a (very small) local data frame

• name: name for the new Impala table

• overwrite: whether to overwrite existing table data (currently ignored)

• types: a character vector giving variable types to use for the columns

• temporary: must be set to FALSE

• unique_indexes: not used

• indexes: not used

• analyze: whether to run COMPUTE STATS after adding data to the new table

• external: whether the new table will be externally managed

• force: whether to silently continue if the table already exists

• field_terminator: the deliminter to use between fields in text file data. Defaults to the ASCII control-A (hex 01) character

• line_terminator: the line terminator. Defaults to "\n"

• file_format: the storage format to use. Options are "TEXTFILE"

  (default) and "PARQUET"

• ...: other arguments passed on to methods

Returns

An object with class tbl_impala, tbl_sql, tbl_lazy, tbl

Note

Impala does not support temporary tables. When using copy_to()

to insert local data into an Impala table, you must set temporary = FALSE.

Examples

library(nycflights13)
dim(airlines) # airlines data frame is very small
# [1] 16  2

## Not run:

copy_to(impala, airlines, temporary = FALSE)
## End(Not run)