stream_in function

Stream in & flatten an ndjson file into a data.table

Given a file of streaming JSON (ndjson) this function reads in the records and creates a flat data.table / tbl from it.

stream_in(path, cls = c("dt", "tbl"))

Arguments

• path: path to file (supports "gz" files)
• cls: the package uses data.table::rbindlist for speed but that's not always the best return type for everyone, so you have option of keeping it a data.table or converting it to a tbl

Returns

data.table or tbl

Examples

f <- system.file("extdata", "test.json", package="ndjson")
nrow(stream_in(f))

gzf <- system.file("extdata", "testgz.json.gz", package="ndjson")
nrow(stream_in(gzf))

References

https://ndjson.org/