1064
99%
master: 99%

Ran 03 Sep 2015 02:51AM UTC

Jobs 1

Files 121

Run time 3s

Badge

Embed ▾

pending completion

Build # 1064

Build Type

push

travis-ci

Committed by

monfresh

Commit Message

Optimize import with no_touching and SmarterCSV

In order to dynamically update the full-text search index when
entities change, the various models include callbacks to update their
parents via `touch`. This unnecessarily slows down the import because
touching in Rails 4 has a lot of overhead. Touching is not needed
during the import. Locations can be updated separately after the
import.

To bypass the touching, Rails provides a `no_touching` method, which
is what I've used here. It significantly sped up the import of a
medium-sized dataset from 31 minutes to 11 minutes.

The import script was also consuming more memory than necessary
because it was reading the CSV files with `File.read`, then loading
the contents in an array with `CVS.new.entries`.

The `File.read` part is not necessary since you can pass in a path
directly to `CSV.new` or `SmarterCSV.process`. SmarterCSV also provides
a way to load the contents in chunks to consume less memory.

Run Details

1745 of 1755 relevant lines covered (99.43%)

66.15 hits per line

Jobs

ID	Job ID	Ran	Files	Coverage
1	1064.1 (2.2.2)	03 Sep 2015 02:51AM UTC	0	99.43	Travis Job 1064.1

Source Files on build 1064

Detailed source file information is not available for this build.

codeforamerica / ohana-api / 1064
99%
master: 99%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 1064

codeforamerica / ohana-api / 1064 99% master: 99%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 1064

codeforamerica / ohana-api / 1064
99%
master: 99%

README BADGES
x