great-expectations / great_expectations / 2142

80%

develop: 0%

Pull #545

travis-ci

The goal of this commit is to make sure that the CLI "add datasource" feature deals better with failures.

Previous behavior was that if the user entered a non-existing path when creating a Pandas datasource or a wrong set of credentials when creating a sqlalchemy datasource, the failure was not detected and the pofiler reported successfully profiling 0 data assets.

Fixing the Pandas datasource was trivial - pass an parameter to click's prompt method to enforce that the path entered by the user is of an existing directory.

Changing the behavior of the sqlalchemy datasource configuration required more changes:
Sqlalchemy datasource tests the credentials by attempting to connect in the constructor, catches specific exception classes that correspond to failure to connect (either bad credentials or bad connectivity) and to missing modules (either sqlalchemy or the database-specific driver).
The CLI detects these failures, reports them to the user and offers to re-enter the credentials or to exit the process.

Had to disable the test_create_sqlalchemy_datasource test in test_datasources.py - SQLAlchemy datasource's behavior has beenchanged to connect to the database in the constructor. The connection kwargs in this test point to a Postgres db that does not exist. I could not figure our how to test with SQLLite because it is not clear which kwargs should be passed in. Tried "path", but sqlalchemy.engine.url.URL says "unexpected keyword argument".

45 of 45 new or added lines in 6 files covered. (100.0%)

4401 of 5532 relevant lines covered (79.56%)

4.75 hits per line