uwescience / myria
59%
master: 27%

LAST BUILD BRANCH: blob_expr_UDF
DEFAULT BRANCH: master
Repo Added
Files 0
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH uda-rewrite-script-eval
branch: uda-rewrite-script-eval
CHANGE BRANCH
x

396

push

travis-ci

dhalperi 
UserDefinedAggregate: combine expressions into a script

In many cases, the updaters for user-defined aggregates share code. E.g., for
an argmax, you may do something like this:

def higher(max1, max2, val1, val2):
   case when max1 > max2 then val1 else val2 end;

And then if I have a table Student(name, gpa), I may define argmax using this
updater to pick the {name,gpa} of the student with the highest gpa.

update = [higher(s.gpa, state.gpa, s.name, state.name),
          higher(s.gpa, state.gpa, s.gpa, state.gpa)]

Currently, we compile each of the update expressions individually and then
execute them in series. Unless Java's JIT is really awesome, this likely leads
to redundant execution. (Performance results indicate that the JIT does not
optimize this redundancy away.)

Instead, we should generate the entire updater script as a single block of
code, and compile it as a single method. The execution code gets simpler and we
expose more optimization opportunities to the compiler. In my experiments, the
time of UDA execution decreases by ~20% or better.

- Add a new ScriptEvalInterface for compiled script objects, and clean up the
  name of the old EvalInterface -> ExpressionEvalInterface
- Rename Evaluator.getJavaExpression() to reflect the fact that it always
  includes code to append to an input column.
- Refactor UserDefinedAggregator (and associated Factory) to use the new script
  interface.
- Add the AppendableTable interface to Tuple so that we can use it with the new
  ScriptEvalInterface.

55 of 55 new or added lines in 5 files covered. (100.0%)

13647 of 23051 relevant lines covered (59.2%)

1.74 hits per line

Relevant lines Covered
23051 RELEVANT LINES 13647 COVERED LINES
1.74 HITS PER LINE
Source Files on uda-rewrite-script-eval
Coverage ∆ File Lines Relevant Covered Missed Hits/Line

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
396 uda-rewrite-script-eval UserDefinedAggregate: combine expressions into a script In many cases, the updaters for user-defined aggregates share code. E.g., for an argmax, you may do something like this: def higher(max1, max2, val1, val2): case when max1 > max2 then va... push dhalperi travis-ci pending completion  
See All Builds (1778)