1
99%
master: 99%

Ran 03 Sep 2015 02:51AM UTC

Files 121

Run time 3s

Badge

Embed ▾

Committed 03 Sep 2015 02:20AM UTC coverage: 99.43% (+0.003%) from 99.427%

Job # 2.2.2

Build Type

push

travis-ci

Committed by

monfresh

Commit Message

Optimize import with no_touching and SmarterCSV

In order to dynamically update the full-text search index when
entities change, the various models include callbacks to update their
parents via `touch`. This unnecessarily slows down the import because
touching in Rails 4 has a lot of overhead. Touching is not needed
during the import. Locations can be updated separately after the
import.

To bypass the touching, Rails provides a `no_touching` method, which
is what I've used here. It significantly sped up the import of a
medium-sized dataset from 31 minutes to 11 minutes.

The import script was also consuming more memory than necessary
because it was reading the CSV files with `File.read`, then loading
the contents in an array with `CVS.new.entries`.

The `File.read` part is not necessary since you can pass in a path
directly to `CSV.new` or `SmarterCSV.process`. SmarterCSV also provides
a way to load the contents in chunks to consume less memory.

Run Details

1745 of 1755 relevant lines covered (99.43%)

66.15 hits per line

codeforamerica / ohana-api / 1064 / 1
99%
master: 99%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 1064.1 (2.2.2)

codeforamerica / ohana-api / 1064 / 1 99% master: 99%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Source Files on job 1064.1 (2.2.2)

codeforamerica / ohana-api / 1064 / 1
99%
master: 99%

README BADGES
x