JuliaData / DataFrames.jl / 3233

93%

push

travis-ci

Improve performance of by() using NamedTuples (#1520)

Remove GroupApplied and introduce combine(::Function, ::GroupedDataFrame). This avoids
storing a copy of the per-group data returned by the user-provided function. Take advantage of
this by allowing that function to return a NamedTuple. Introduce two completely different
code paths depending on whether the first returned object is DataFrame or a NamedTuple, as
the latter allows for more efficient operation by assuming that it represents a single row.
Use the same progressive eltype widening approach as Base.map so that we fill column vectors
whose types are known inside the kernel functions. This does not eliminate the type unstability
due to the fact that the user-provided function takes a DataFrame, but ensuring type stability for
half of the operations still improves performance significantly.

Also add map(::Function, ::GroupedDataFrame), returning a GroupedDataFrame. Parameterize GroupedDataFrame on the type of data frame it wraps, and make its column index
have a concrete type. Deprecate an old map method for SubDataFrame. Fix a type unstability in hcat!.

126 of 126 new or added lines in 4 files covered. (100.0%)

1477 of 2357 relevant lines covered (62.66%)

6556.07 hits per line