JBKahn / django-sharding / 67
90%

DEFAULT BRANCH: master
Ran
Jobs 12
Files 23
Run time 3min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
67

push

travis-ci

JBKahn 
Add postgres-specific id generator (#26)

* Added manager and queryset for automatic shard selection in the router

* Updated custom ShardQueryset to call super() functions to help handle django updates to the base method rather than returning on their own

* Updated model_config decorator to take an optional "sharded_by_field" argument that stores the field as an attribute and sets the default manager for the model to ShardManager

* Updated the router to use the new optional sharded by field on the model to get the correct shard based on the arguments passed to filter,create,get,etc. The default behavior is unchanged so existing models do not need any changes

* Fixed a bug with staticmethod not working properly in the manager

* Fixed a bug with staticmethod not working properly in the manager

* Updated logic for grabbing the shard in the router when the instance does not know what shard it should be on but there in a sharded by field defined

* Updated the settings helper database_configs to put a shard id property on each primary shard. Added and updated unit tests to check for valid shard ids.

* Fixed a test bug when using SQLite where the BigAutoField would fail with a too large for sqlite error

* Fixed another BigAutoField bug with SQLite being too large to store

* Added common local dev files to gitignore

* Added custom postgres id field that will automatically generate shard-safe IDs. Added utils for migrations. Updated model_config decorator to enforce an ID strategy if the postgres shard id field does not exist on the model

* Added unit tests for custom postgres shard id field

* Fixed a syntax error with bit manipulation

* Fixed another syntax error with bit manipulation

* Added logging to troubleshoot CI

* More logging for CI

* Fixed formatting issue with psycopg2. Removed logging. Updated unit tests.

* Added another test to verify the id coming back from the postgres id generator field is valid

* Fixed multiple syntax errors in the router where it would look up the wrong values on accident

* Created a base model for sharding to go along with the decorator. All the base model does is set the manager. Unfortunately, this cannot be done in the decorator for some reason (I cannot figure out why, it simply doesnt work correctly)

* Updated the model instace create function to copy out the kwargs before instantiating the object with them, just in case the parent class messes with the kwargs later

* Updated tests for filter() and create() shard auto detection (no more using())

* Fixed a bug in a test where the user created does not get pk=1

* Updated constants with all of the backend for each vendor. Updated references in code to check "in" rather than equality

* Updated decorator to raise an error if a person tries to use the postgres field without postgres being the engine. Updated test model and fields to reflect the change

* Fixed a bug that would sometimes attempt to run postgres specific migration on non-postgres db

* Updated docs with Postgres id generator feature sections

* Updated the decorators function with comments from github: updated exception messages, simplified logic for checking postgres field requirements

* Refactored the common logic for getting the shard in the router per the github comment. Removed the deletes from the unit tests after adding the unit test settings for multiple DBs

* Removed the test case settings from github comment, they did not work with CI environment

* Removed the requirement for inheriting from ShardedModel. Added checks + manager assignment to decorator function to deal with the sharded_by_field.

* Updated docs to include the `sharded_by_field` information

* Updated error messaging when dealing with an abstract base class and the automatic routing manager

* Updated test name

* Removed todo lines for docs that have been completed

* Fixed a typo

* Removed extra import that is no longer needed

* Removed some commented out code in the manager

* Updated tests to make sure exceptions are being rased correctly when dealing with abstrat base classes and the sharded_by_field

* Added more decorator tests

* Added test db settings

* Removed delete()'s from tests, travis did not error with the test db settings like circle'

* Fixed a an import bug related to Django 1.8

* Commented out test settings again, pretty sure its breaking the CI

* Fixed an issue with tests locally. Moved an import to top of file in decorators.py

* Removed a duplicate function declaration in the docs

* Removed TestCase settings overrides in runtests. The changes are not needed for Travis and do not appear to affect anything locally

* Removed duplicate function declaration in docs

* Removed TestCase import from runtests: not needed for testing with Travis

* Added Postgres specific ID field functionality:
- New Field for Postgres ID, along with migrations
- Updated decorators.py to account for new postgres field
- Updated settings helpers and constants with new utils, and fixed
  referencs to the same

* Added tests for Postgres-specific ID field
- Check that field works as intended
- Ensure stored procedure and sequence are created
- Added migration to support field tests

* Updated docs with postgres-specific ID information

* Added more common development files for git to ignore

* Updated runtests.py with required settings for postgres ID field testing

* Added a foreign key field that can handle Postgres generated ID fields: the Django default errors out if the FK goes out of integer range, as the FK field in Django is (by default), a standard integer rather than a bigint

* Added one-to-one field for Django to work correctly with

* Added rel_db_type function to postgres shard ID field

* Fixed formatting in migration command for 1.10

564 of 613 relevant lines covered (92.01%)

10.96 hits per line

Jobs
ID Job ID Ran Files Coverage
1 67.1 (TOX_ENV=py27-dj18) 0
91.35
 Travis Job 67.1
2 67.2 (TOX_ENV=py27-dj19) 0
91.03
 Travis Job 67.2
3 67.3 (TOX_ENV=py27-dj110) 0
90.86
 Travis Job 67.3
4 67.4 (TOX_ENV=py27-djdev) 0
90.86
 Travis Job 67.4
5 67.5 (TOX_ENV=py34-dj18) 0
91.84
 Travis Job 67.5
6 67.6 (TOX_ENV=py34-dj19) 0
91.52
 Travis Job 67.6
7 67.7 (TOX_ENV=py34-dj110) 0
91.35
 Travis Job 67.7
8 67.8 (TOX_ENV=py34-djdev) 0
91.35
 Travis Job 67.8
9 67.9 (TOX_ENV=py35-dj18) 0
91.84
 Travis Job 67.9
10 67.10 (TOX_ENV=py35-dj19) 0
91.52
 Travis Job 67.10
11 67.11 (TOX_ENV=py35-dj110) 0
91.35
 Travis Job 67.11
12 67.12 (TOX_ENV=py35-djdev) 0
91.35
 Travis Job 67.12
Source Files on build 67
Detailed source file information is not available for this build.