UrbanCCD-UChicago / plenario2 / 931

76%

master: 76%

Pull #345

travis-ci

Ingest Bug Fix

We were having an issue consuming a few data sets. After some
investigation, I discovered that it was due to quotes being used in
header rows:

```
One,Two,"Three, or three",Four
```

This would then be translated into the Copy SQL as:

```
COPY ... ("One", "Two", ""Three, or three"", "Four") ...
```

which obviously isn't going to work.

The solution is to _regex_ split the header line rather than the naive
_string_ split. But you may ask, "Why do splits at all? Can't we just
use the CSV module and pull the keys from the first map row?". The
unfortunate answer to this is that the CSV module does not preserve
order -- if you have a header row `a,c,b` then you will get a map with
keys ordered as `a,b,c`. To preserve the order of the headers, you need
to split the line.

Fixes #344

12 of 12 new or added lines in 3 files covered. (100.0%)

1368 of 1803 relevant lines covered (75.87%)

65.74 hits per line