OpenTreeMap / otm-core / 2948
84%
develop: 84%

LAST BUILD BRANCH: dependabot/npm_and_yarn/sockjs-0.3.21
DEFAULT BRANCH: develop
Ran
Jobs 1
Files 216
Run time 12s
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
2948

push

travis-ci

RobinIsTheBird 
upgrade HStore

Changes
-------
- Replace UDF specific GeoHStoreUDFManager with generic
  HStoreGeoManager in treemap and stormwater models
- Migration and settings to introduce
  django.contrib.postgres.fields.HStoreField
- Migration to demote Species from UDFModel
- `search` refactor and documentation
- `audit` minor flake8 cleanup
- `udf`
  - Instrumentation
  - Eliminate UDF specific manager
  - Move the former `UDFQuery.build_filter` logic
    to `UDFModel.transform_filter_arg`
  - Eliminate
    - `quotesingle`
    - `UDF_ORDER_PATTERN`
    - `UDFBaseContains(Lookup)`
    - `UDFContains` and `UDFIcontains`
    - `UDFQuery(Query)`
    - `UDFQuerySet(HStoreGeoQuerySet)`
    - `GeoHStoreUDFManager(HStoreGeoManager)`
- Replace model references to HStoreGeoManager with
  references to the standard models.GeoManager
- `UserDefinedCollectionValue.data` now based on
  `django.contrib.postgres.fields.HStoreField`
- Successfully dynamically replace `UDFFIeld` with `UDFDictionary`
- `UDFDictionary` obtains the correct model instance obj
- Can retrieve and set all custom fields on a model instance,
  whether scalar or collection (detail page works)
- Can search for anything but custom fields, including
  Anonymous Boundary (map page mostly works)
- More documentation
- Less instrumentation
- Keeps the audit trail
- Search now works for both collection and scalar UDFs
- Removed 'IN' and 'ISNULL' predicates from collection search
  because they are never used in production
- Removed 'WITHIN_RADIUS' predicate from scalar search
  because it is never used in production
- More small refactors, especially adding _lookup_key()
- Tiny search refactor
- Begin fixing search tests
- Improved documentation
- Removed more unused code
- Removed instrumentation
- Fix a bug in `manage/udfs` JavaScript handling response from UDF creation
- Fix lookups on `udfs__contains` to use `udfs__has_key`
- `treemap.util.make_udf_lookup_from_key`
- `UDFDictionary.lookup_name`
- If a collection field choice is removed,
  delete all UDCVs that name that choice,
  not just the choice field in the UDCV
- Replace the old `HStoreDict` `hupdate` with a loop of updates
- Cleaning enhancements
- Both scalar and collection UDFs can be assigned to trees
  and plots and updated
- Removing a choice from either a scalar or collection UDF
  removes the field or UDCV respectively from trees and plots
  to which it was assigned
- Search works on either scalar or collection UDFs for
  tree and plot
- `reverse_clean` to check for value not None, not falsey
- remove incorrect space
- Fix the `UDFDescriptor` to get and set the dict correctly
  when the dict has not already been initialized
- Remove unused `UDFDictionary._fields` attribute
- Fix form_extras to tolerate a missing UDF key
- Correct interfaces in the tests
- Instrumentation and some correction in udf.py prompted by tests
- A much simplified temporary test, which passes
- Correction to `treemap.udf.UDFDescriptor` to account for
  the test creating a Plot with a None `udfs` value
- Remove tests for scalar UDF ordering
- Fix test lookups to use `HStoreField` lookup syntax
- Remove the temporary simple test
- Remove instrumentation from `treemap.udf`
- Add comments to `treemap.udf`
- Setup psycopg2 hstore first
- Remove the irreversible `HStoreExtension` migration,
  which is unnecessary thanks to vagrant services provisioning
- Fix `UDFModel.as_dict`
- Remove pending-audit tests
- Fix `UDFModel.save_with_user` to clean scalar fields
  without error, if none are on the model instance
- Pad out collection methods on `UDFDictionary` because
  they are used by the tests directly or indirectly
- Remove the unused 'user' datatype
- test_udfs.CollectionUDFTest.test_can_delete fails downwind
  of the deepest `save_with_user` trying to do a
  `self.cursor.execute(sql, params), with a
  `DataError: invalid input syntax for integer: "water"`
  where there is an `ARRAY['water', 42]`.
- Converting 42 to a string first had no effect on the error
- `UdfCreateTest` and `UdfDeleteTest` pass
- `UdfCRUTestCase` is a base class, pass/fail irrelevant
- Fix a scalar search parse bug
- Update tests and a comment to new search interface
- Remove django_hstore from default settings
- Remove obsolete comment
- Remove last references to the udf 'user' type
- Fix flake8 complaint
- Remove vestigial 'user' data type from `test_udfs`
- Add `clean` args to `UDFDictionary` methods with defaults
- Only clean values when `clean` is True.
- Enhance treemap.udf top comment
- Radical simplification and robustness change to `UDFModel`
- In `treemap.udf`
  - Remove `UDFDescriptor`
  - Alias `UDFField` to `HStoreField` for the sake of earlier migrations
  - Rename the `HStoreField` to `hstore_udfs` with a column param
    to point it at the `udfs` column in `treemap_mapfeature` table
  - Dynamically make a `udfs` model instance attribute in `UDFModel.__init__`
  - Migration to support the above
- Small code aesthetic improvement in `treemap.audit`
- Fix `test_udf.CollectionUDFTest.test_can_delete_choice_option
  to stop looking for a hanging collection UDF, which now gets removed
- Break initialization circular dependencies by pushing responsibility
  for calling `UserTrackable.populate_previous_state` off to leaf class
  init methods
- Remove some extra calls and add some necessary calls to it
- Fix filter lookup in `treemap.test_udfs`
- Fix all `treemap.test_udfs` filter look-ups on udfs to use the
  right lookup
- Fix `UdfsProxy.__setitem__` to create a `hstore_udfs` dict if it is `None`
- Skipped two tests in `treemap.test_udfs` until I figure out what to do
- Modified the same tests so that when they run, they fail,
  to demonstrate the problem
- Lookup 'udfs__' becomes 'hstore_udfs__' everywhere
- `UDFModel.hstore_udfs` remove `null=True`, add `default=''`
- Fixes & simplifications to `treemap.udf.UDFModel`:
  - Most importantly, added `UDFModel.from_db` in order to force
    django to instantiate models with kwargs instead of just a list
    of field values, by use of the `_deferred` flag.
  - Removed the unused `UdfsProxy()._state` and constants
  - Removed the unneeded `UdfsProxy().convert()`
  - Encapsulated `udfs` attribute setup
- Comments in `treemap.tests.test_udfs` explaining what has to happen in the
  SQL in order to do numeric comparison.
- Added extensions to `Func` to `treemap.utils` that _might_ help
  with constructing ORM style queries on scalar udf fields
- Use `feature_type` attribute if it exists, because the only classname
  available might be the generic `MapFeature`.
- Lookups on HStoreField in `treemap.udf`
- Funcs in `treemap.util`, probably going to be removed,
  but handy for experiments right now
- Removed useless Func subclasses from `treemap.util`
- Added lookups to `treemap.udf`
- Eliminate `from_db`, as `_deferred` was causing initialization problems
- Fix dependency in the species migration
- Migrate HStoreFields for both UDFModel and UserDefinedCollectionValue
- The auto-generated migration used `RemoveField` and `AddField`,
  which lost all the data
- The migration here has been hacked to use `RenameField` and `AlterField`,
  and tested for data retention.
- Include scalar int udf test in `test_udfs`
- Improve udf reverse clean, although it still fails on
  treemap.tests.test_udfs.ScalarUDFFilterTest.test_has_key
- Fixes to `UserDefinedFieldDefinition.reverse_clean` for date udfs
- Fix both `treemap.udf.UDFModel.hstore_udfs` and the associated migration
  to have a callable `default`, in order to be reentrant.
- Fix the migration that gave the old `UDFField` a `default` to be
  reentrant
- Fix `UserDefinedFieldDefinition._update_choice_scalar` so that the
  `UdfsProxy` cleans and reverse cleans against the right datatype
- Add a `'multichoice'` test to `test_udfs`
- Test for a multichoice option with double quotes in the search string
- Fix `test_udfs.ScalarUDFFilterTest.test_float_gt_and_lte_constraints`
  to use the `float__` Transform
- Still removing references to the 'user' datatype from `test_udfs`
- `treemap.udf.UDFModel.UdfsProxy.__delitem__`
- `treemap.models.Species` has to `populate_previous_state()`
- `treemap.search` casts to `float` for numeric min max
- `treemap.udf.UDFModel` implements `__getattr__` to lookup attributes
  corresponding to custom fields
- test custom field attribute in `test_udfs`
- Remove pending tests from `test_audit`
- `treemap.models.ITreeCodeOverride` has to `populate_previous_state`
- Make getting udfs safe for `__getattr__`
- Remove TODOs about restoring the `readonly` field in the importer
- Test value `is None` instead of relying on falsy values, which
  could be legitimate
- Cache the name used in `udf_defs` queries in the model `__dict__`
  to avoid `__getattr__` recursion
- Fix `otm1_migrator` to initialize models with the `instance`
- Refine `otm1_migrator` test for whether to pass instance to
  model creation
- Correct `api.tests` to use the correct form of a `multichoice` udf
- `UserDefinedFieldDefinition.clean_value` now raises `ValidationError`
  exceptions that contain a well formed `message_dict`, aka `error_dict`
- Adapt `treemap.views.map_feature.update_map_feature` to include
  `ValidationError` `message_dict`s from setting a udf to an invalid value
  in its `errors`.
- Refine error return value from `importer.models.trees` to match test
- Change `form_extras` to accept `AttributeError` as well as
  `ObjectDoesNotExist`, as the former is more accurate in this case
- Fix `UDFModel._setup_udf_model_type`

Caveats
-------
- Unclear why the JavaScript is sending Stewardship
  after saving a change only to a scalar udf -
  did it always do that?
- Will need major updates to `test_search`
- Not tested on any polygonal map features
- Scalar UDFs on stormwater map features don't work,
  no idea why, neither polygonal nor RainBarrels
- Collection UDFs on stormwater map features not tested
- Can now set and get scalar UDFs on a UDFModel that has never had
  its `udfs` set before in the shell
- but when `form_extras` generates a faux `Tree` for a `SearchNode`,
  it fails with a KeyError in trying fetch its custom fields
- Can now set and get scalar UDFs on a UDFModel that has never had
  its `udfs` set before in the shell
- but when `form_extras` generates a faux `Tree` for a `SearchNode`,
  it fails with a KeyError in trying fetch its custom fields
- Have a stackoverflow reference in case ordering
  by an hstore key ever becomes important
- The following `treemap.tests.test_udfs` pass:
  - ScalarUDFTest
  - ScalarUDFInstanceIsolationTest
  - UDFDefTest
  - UDFAuditTest
  - ScalarUDFFilterTest
- CollectionUDFTest fails
- Have not tried:
  - UdfDeleteTest
  - UdfCRUTestCase
  - UdfCreateTest
- `SearchTests.test_udf_date_search` gets DataError in
  `create_tree_and_plot`
- Perhaps it's a design flaw that `udfs` perpetually cleans and
  `hstore_udfs` perpetually reverse cleans, and both should be delayed to
  `save_with_user`, but there are disadvantages to allowing dirty values, too.

Testing Notes
-------------
- Search for Stewardship combined with Anonymous Boundary works
- Only tested so far with trees and plots
- Regression: updating a scalar udf produces new audit records
  for Stewardship, even though Stewardship is untouched
- Combines well with other searches
- ... unless you are bitten by #3080

--

Connects to #1968

17181 of 20596 relevant lines covered (83.42%)

0.83 hits per line

Jobs
ID Job ID Ran Files Coverage
1 2948.1 0
83.42
 Travis Job 2948.1
Source Files on build 2948
Detailed source file information is not available for this build.