Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,8 @@

21. GForce grouping is again faster when running single-threaded, e.g. after `setDTthreads(1)`, [#3330](https://github.com/Rdatatable/data.table/issues/3330). The batching used to parallelize `gsum()` and `gmean()` is skipped when only one thread is available. Also, `mean()` of an `integer` or `logical` column by group no longer coerces the whole column to `double` first, which made it slower than in v1.12.0 regardless of the number of threads. Thanks @jangorecki for the benchmarks, @mattdowle for the report, and @ben-schwen for the fix.

22. `fread(text=, yaml=TRUE)` and `fread(input=, yaml=TRUE)` now work when the data is a single character string containing newlines, rather than failing with `cannot open the connection`, [#7896](https://github.com/Rdatatable/data.table/issues/7896). Thanks to @Noskario for the report and @AnKiT-GaRG2 for the fix.

### Notes

1. {data.table} now depends on R 3.5.0 (2018).
Expand Down
7 changes: 4 additions & 3 deletions R/fread.R
Original file line number Diff line number Diff line change
Expand Up @@ -276,7 +276,7 @@ yaml=FALSE, tmpdir=tempdir(), tz="UTC")
call_args = names(match.call())
if (is.character(skip))
warningf("Combining a search string as 'skip' and reading a YAML header may not work as expected -- currently, reading will proceed to search for 'skip' from the beginning of the file, NOT from the end of the metadata; please file an issue on GitHub if you'd like to see more intuitive behavior supported.")
yaml_res = .read_yaml_header(input, skip, verbose)
yaml_res = .read_yaml_header(input, skip, verbose, input_is_data = !identical(input, file)) # file is NULL when input is the data itself, #7896
yaml_header = yaml_res$yaml_header
n_read = yaml_res$n_read
yaml_names = names(yaml_header)
Expand Down Expand Up @@ -450,10 +450,11 @@ yaml=FALSE, tmpdir=tempdir(), tz="UTC")
}

# nocov start. Covered only in other.Rraw
.read_yaml_header = function(f, skip, verbose) {
.read_yaml_header = function(f, skip, verbose, input_is_data=FALSE) {
# create connection to stream header lines from file:
# https://stackoverflow.com/questions/9871307
con = base::file(f, 'r')
# f can also be the data itself (text= or input= with \n or \r). Decided by the caller rather than by looking for \n in f, since a file name may contain a newline, #7896
con = if (input_is_data) textConnection(f) else base::file(f, 'r')
# NB: close() won't be right if 'f' itself can be a file connection (#561)
on.exit(close(con))

Expand Down
5 changes: 5 additions & 0 deletions inst/tests/other.Rraw
Original file line number Diff line number Diff line change
Expand Up @@ -395,6 +395,11 @@ if (loaded[["yaml"]]) { # csvy; #1701. Was 2032-2033 in tests.Rraw, #5516
test(16.22, fread(f, skip = 'var1,', yaml = TRUE),
DT_yaml, warning = 'Combining a search.*YAML.*')

# yaml=TRUE with text= a single string containing newlines, #7896
txt = paste(readLines(f), collapse="\n")
test(16.23, fread(text=txt, yaml=TRUE), DT_yaml)
test(16.24, fread(txt, yaml=TRUE), DT_yaml)

# fwrite csvy: #3534
tmp = tempfile()
DT = data.table(a = 1:5, b = c(pi, 1:4), c = letters[1:5])
Expand Down
Loading