mrocklin / multipledispatch / 231
0%

DEFAULT BRANCH: master
Ran
Jobs 7
Files 0
Run time –
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
231

push

travis-ci

mrocklin 
ENH: optimize import times (#66)

It is currently very expensive to import modules that use multiple dispatch because of all of the ordering that happens at module scope.

This change creates a notion of an unresolved dispatcher which will lazily resolve the order when needed. Dispatchers begin as regular dispatchers with no functions. When `add` is called, the cache is cleared and the object moves into the unresolved state. When the `order` attribute is looked up, the order is computed and the dispatcher moves back to the resolved state. This means that exact matches (resolved in `self.funcs`) will not trigger a reorder. Reordering can be explicitly triggered by calling the `reorder` method. This same machinery applies to the `MethodDispatcher`.

This is implemented by swapping the type under the dispatcher instance to avoid a branch on every call. One simple implementation would be to have a `self._unresolved` flag on the instance; however, this would be slower than the current solution. Most multupledispatch code starts with a large amount of dispatch definitions followed by a considerably larger amount of calls to the dispatcher. I wanted to keep the per-call overhead as low as possible. This implementation also helps by batching sequences of additions and only triggers a single reorder when used.

As a small benchmark, this takes the import time of one component of our code from 7.502 seconds down to 3.210 seconds. Using the existing mutiple dispatch the top calls by total time were:
```
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
  1742368    0.763    0.000    1.541    0.000 multipledispatch/conflict.py:7(supercedes)
   845865    0.588    0.000    1.185    0.000 multipledispatch/conflict.py:12(consistent)
      175    0.530    0.003    2.206    0.013 multipledispatch/conflict.py:24(ambiguities)
   815514    0.395    0.000    0.565    0.000 {map}
  1695752    0.343    0.000    1.847    0.000 multipledispat... (continued)
Jobs
ID Job ID Ran Files Coverage
1 231.1 0
Travis Job 231.1
2 231.2 0
Travis Job 231.2
3 231.3 0
Travis Job 231.3
4 231.4 0
Travis Job 231.4
5 231.5 0
Travis Job 231.5
6 231.6 0
Travis Job 231.6
8 231.8 0
Travis Job 231.8
Source Files on build 231
Coverage ∆ File Lines Relevant Covered Missed Hits/Line