24
59%

Ran 28 Jul 2017 12:16PM UTC

Jobs 1

Files 0

Run time –

Badge

Embed ▾

pending completion

Build # 24

Build Type

push

travis-ci

Committed by

marblestation

Commit Message

Added celery task to update master pipeline (#42)

* add celery task to update master pipeline

python run.py nonbibToMasterPipeline # sends all nonbib records to master pipeline
as before, python run.py runRowViewPipeline # reads column files into Postgres
also removed nonbibUpdater and metricsUpdater, they copied data betweeen Postgres tables.
currently, the master pipeline holds the reference data rather than this repo so these updaters are no longer used.

* add ability to send changed nonbib records to master pipeline

during daily processing the nonbib data for roughly 10% of bibcodes changes.
to send only this data to the master pipeline, use the nonbibDeltaToMasterPipeline command.
also fixed a bug in the delta table, it was defined as materialize view but it needs to be a
table so we can insert new bibcodes to it.

* added and fixed documentation for column data file reader

* add diagnose switch

python run.py nonbibToMasterPipeline --diagnose sends a single hardcoded record of nonbib data to the master pipeline
and prints out configured connection parameters

Run Details

Jobs

ID	Job ID	Ran	Files	Coverage
1	24.1	28 Jul 2017 12:16PM UTC	0		Travis Job 24.1

Source Files on build 24

Detailed source file information is not available for this build.

adsabs / AdsDataSqlSync / 24
59%

README BADGES
x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 24

adsabs / AdsDataSqlSync / 24 59%

README BADGES x

Markdown

Textile

RDoc

HTML

Rst

Jobs

Source Files on build 24

adsabs / AdsDataSqlSync / 24
59%

README BADGES
x