• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

alphagov / backdrop
90%
master: 89%

Build:
Build:
LAST BUILD BRANCH: upgrade_paas_stack
DEFAULT BRANCH: master
Repo Added 25 Jun 2013 09:22PM UTC
Files 69
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH step_5_test
branch: step_5_test
CHANGE BRANCH
x
Reset
  • step_5_test
  • 159387416-redis
  • 404-for-bucket-not-found
  • 80723696-iterate-the-flattening
  • abort-on-error
  • accept-excel-files
  • add-ack-exclude
  • add-build-to-ackrc
  • add-deprecation-warning
  • add-docs-on-emptying-dataset
  • add-flatten-param
  • add-not-authorized
  • add-ops-manual-documentation
  • add-patch-to-write-api
  • add-procfile-worker
  • add-production-config
  • add-quality
  • add-quarter
  • add-redirect-test
  • add-request-details-to-response-log
  • add-service-id
  • add-service-id-to-dataset
  • add-statsd-prefix
  • add-test-perf-dir-to-ackrc
  • add-warning-to-upload
  • add_config_for_electronic_vehicle_licensing_monitoring
  • add_dataset_name_to_error_messages
  • add_flower
  • add_rediss_support
  • add_tags_to_json_logger
  • add_timers
  • admin-caching
  • admin-improve-ux
  • allow-collecting-with-period-queries
  • allow-expected-time-to-be-updated
  • allow-saving-none
  • allow-special-field-names
  • allow-travis-fail
  • allow-unknown-data-set-fields
  • always-utc-out
  • always_add_a_default_upload_filter
  • audit-logging
  • auth_headers_the_second
  • authentication-responses
  • automate-creation-of-buckets-55989156
  • automate-user-creation
  • back-to-bucket-list
  • backdrop-errors
  • backdrop-stagecraft-integration
  • better-error-handling
  • better-upload-error-messages
  • browser-screenshot-on-failure
  • bucket_recency
  • bugfix-evl-dodgy-spreadsheet
  • bugfix-flat-data-shifting
  • bugfix-redirect-root-to-admin
  • bugfix-transformer-rate
  • bugfix/add-test-for-empty-data-with-duration-parameter
  • bugfix/no-data-transforms
  • bugfix/spotlight-cors-request-headers
  • bugfix/user-sat-transform
  • bump-client-version
  • bumping-selenium
  • cache_aff
  • change-dev-to-match-other-envs
  • change-flattened-response-format
  • change-logstash-library
  • change-production-log-location
  • changes-for-govuk-devvm
  • check_upload_mimetypes
  • clamav-errors
  • clamdscan
  • clearer-content-type-error
  • click-jacking
  • consistent-errors-single-commit
  • consistent-write-errors
  • consistent-write-errors-refactor
  • convert-excel-float-to-int
  • convert-timestamp-to-utc-in-id
  • cors-on-error
  • create-ackrc
  • create-collection-endpoint
  • create-transform-data-set
  • crossdomain
  • data-migration
  • day-and-hour-period
  • decouple-collection-create
  • decrease-graphite-disk-usage
  • delete-data-set-record
  • delete-returns-200
  • delete_datasets
  • dependency-updates
  • deployed-to-dev
  • deployed-to-integration
  • deployed-to-preview
  • deployed-to-production
  • deployed-to-staging
  • disable-file-upload-scenario
  • disable-travis-emails
  • do-not-verify-stagcraft-host
  • do_not_go_gentle_into_that_rewrite
  • docker-compose
  • document-write-api
  • dont-overwrite-virtualenv
  • drop-collections-on-replicate
  • dump-buckets-tool
  • dump_help
  • emit-write-messages
  • empty-data-set-endpoint
  • encode-unicode-strings-in-record
  • ensure-only-full-satisfaction-weeks-transformed
  • evl-channel-volumetrics-totals
  • evl-customer-satisfaction-upload
  • evl-data-upload
  • evl-satisfaction-raw-queries
  • evl-volumetrics
  • evl-volumetrics-upload
  • explain_how_to_run_a_manual_transform
  • feature-flags
  • feature/add-prefix-filter
  • feature/add-regex-filter
  • feature/aggregate-data-parity
  • feature/architecture-notes
  • feature/autopep
  • feature/better-ranges
  • feature/cache-control
  • feature/data-for-transforms
  • feature/delete-unused-code
  • feature/dependency-upgrades
  • feature/dry-auth-checks
  • feature/dry-request-id
  • feature/dry-staleness-check
  • feature/exclude-tagged-branches-from-travis
  • feature/gzip-bomb
  • feature/informative-adr-names
  • feature/log-request-id
  • feature/mapping-transform
  • feature/more-detailed-error-message
  • feature/perf-decorators
  • feature/rate-limiting
  • feature/rate-transform
  • feature/required-fields-for-transforms
  • feature/signon-client-id
  • feature/time-all-the-things
  • feature/transform-improvements
  • feature/update-splinter
  • feature/upgrade-dependencies
  • feature/upgrade-requests
  • feature/write-start-end-at
  • fill-data-perf-improvments
  • filter-latest-by-record-id
  • filth
  • fix-bearer-token-handling
  • fix-behave-new-phantom
  • fix-curl-in-readme
  • fix-decompressing-request
  • fix-dev-stagecraft-token
  • fix-evl-upload-failure
  • fix-field-names
  • fix-flattened-shifting
  • fix-ga-bucket-timezones
  • fix-ids
  • fix-integer-auto-id-error
  • fix-migrations
  • fix-missing-day-dvsa-takeup
  • fix-race-condition
  • fix-replicate-db
  • fix-replicate-script
  • fix-route-name
  • fix-stagecraft-port
  • fix-status-error
  • fix-tests-for-emptying
  • fix-the-build
  • fix-urls
  • fix-valid_start_at
  • fix/audit-log-token
  • fix/audit-nonempty-appends
  • fix/latest-data-query-params
  • fix_auto_ids_on_timestamps
  • fix_bucket_access
  • fix_celery_config_write_api
  • fix_replicate_script
  • fixing-file-upload
  • flaskutils-pep8
  • flatten-response
  • force-settings-update
  • force-utc-timestamp-in-ids
  • format-push-api-date-timestamps-utc
  • get-bucket-config-from-stagecraft
  • get-clamdscans-stderr
  • get-last-data-even-if-none
  • get-latest-tx-data
  • get_latest_but_go_back_in_time_on_nulls
  • gitignore-swap-files
  • go-back-in-time-for-data-again
  • hack-timestamps
  • handle-multiple-dashboards-for-service-id
  • http-put-to-empty-data-set
  • idempotent-migration
  • ignore-collections-in-replicate
  • ignore-specified-tx-data-types
  • improve-db-migrations-instructions
  • improve-delete-test
  • improve-error-messages-1
  • improve-error-messages-4
  • improve-error-messages-5
  • improve-naming-convention
  • instrument-transforms
  • integrate-with-coveralls
  • integrate-with-coveralls-clean
  • invalidTokenMessage
  • is_travis_selenium_slow
  • jenkinsfile
  • json-to-csv-convert-script
  • link-to-manual-entry
  • listen-to-signonotron
  • load-seeds-from-app-directory
  • log-to-stdout-in-development-mode
  • logstash_json
  • master
  • master-stagecraft
  • mattbostock-patch-1
  • max_age_expected
  • migrate-buckets-to-data-sets
  • migrations-by-pattern
  • migrations-use-repository
  • monitor-gcloud-sales
  • more-helpful-error-message
  • more-robust-start-app
  • more-robust-subclassing
  • more_log_response_messages
  • move-past-incomplete-periods
  • move-query-response-to-core
  • move-tools-into-tools
  • multiple-group_by-params
  • new-data-set
  • no-pep-on-build
  • no-users-on-write
  • nocd-on-replicate-script
  • nosetests-xml-ackrc-ignore
  • only-allow-utc-timestamp-in-ids
  • only-run-migration-if-not-capped
  • paas-migration
  • parse-csv-numbers
  • pep8
  • perf/data-set-status
  • pipper
  • point_dev_uploader_to_new_url
  • postgres-storage
  • production
  • production-log
  • profile-writes
  • protect-against-empty-dashboard-list
  • push-data-script
  • py26-compliance
  • quickstart
  • quieten-pip
  • ratty-http
  • read-api-collect-maths
  • read-data-set-format-csv
  • read2core
  • really_reliable_replicated_replicas
  • recent-last-api
  • refactor-behave-tests
  • refactor-data-layer-step-1
  • refactor-data-layer-step-2
  • refactor-data-layer-step-3
  • refactor-data-layer-step-4
  • refactor-data-layer-step-5
  • refactor-run-tests
  • refactor-use-oauth-tokens
  • refactor-write-api-functions
  • relative-time-api
  • release-backup
  • release_1012
  • release_1013
  • release_1014
  • release_1015
  • release_1016
  • release_1017
  • release_1019
  • release_1021
  • release_1022
  • release_1023
  • release_1026
  • release_1027
  • release_1028
  • release_1029
  • release_1030
  • release_1031
  • release_1032
  • release_1033
  • release_1034
  • release_1035
  • release_1036
  • release_1037
  • release_1038
  • release_1039
  • release_1040
  • release_1041
  • release_1042
  • release_1043
  • release_1046
  • release_1047
  • release_1048
  • release_1050
  • release_1051
  • release_1052
  • release_1053
  • release_1054
  • release_1055
  • release_1056
  • release_1057
  • release_1058
  • release_1059
  • release_1060
  • release_1061
  • release_1062
  • release_1064
  • release_1065
  • release_1066
  • release_1067
  • release_1068
  • release_1069
  • release_1070
  • release_1072
  • release_1075
  • release_1077
  • release_1078
  • release_1089
  • release_1094
  • release_1095
  • release_1102
  • release_1103
  • release_1106
  • release_1108
  • release_1109
  • release_1110
  • release_1111
  • release_1112
  • release_1116
  • release_1117
  • release_1118
  • release_1119
  • release_1120
  • release_1122
  • release_1123
  • release_1124
  • release_1125
  • release_1126
  • release_1127
  • release_1128
  • release_1129
  • release_1130
  • release_1131
  • release_1132
  • release_1133
  • release_1134
  • release_1137
  • release_1138
  • release_1139
  • release_1140
  • release_1141
  • release_1142
  • release_1143
  • release_1144
  • release_1145
  • release_1146
  • release_1147
  • release_1148
  • release_1149
  • release_1150
  • release_1151
  • release_1152
  • release_1153
  • release_1154
  • release_1155
  • release_1156
  • release_1157
  • release_1158
  • release_1159
  • release_1160
  • release_1161
  • release_1162
  • release_1163
  • release_1164
  • release_1165
  • release_1166
  • release_1167
  • release_1168
  • release_1169
  • release_1170
  • release_1171
  • release_1172
  • release_1173
  • release_1174
  • release_1175
  • release_1176
  • release_1177
  • release_1178
  • remove-STAGECRAFT_DATA_SET_QUERY_TOKEN
  • remove-admin-app
  • remove-ambiguous-letters-from-tokens
  • remove-create-route
  • remove-example-csv
  • remove-latest-data-restriction
  • remove-mongo-user-repository
  • remove-nested-grouped
  • remove-old-buckets-legacy
  • remove-old-procfiles
  • remove-python-26
  • remove-redundant-code
  • remove-signon-flask-dependency
  • remove-unused-bits
  • remove-updated-at-buckets
  • remove-validation-collect-internal-fields
  • rename-buckets-to-data-sets
  • rename-hosts-var
  • rename_bucket_to_data_set
  • reordering-setup-in-docs
  • replica-set-config-for-dev-vm
  • replica_set_as_variable
  • replicate-db-fail-earlier
  • replicate-db-script-vm-option
  • request-decompression
  • request-id-govuk
  • return-user-friendly-date-error-messages
  • revert-333-status_json
  • revert-450-only-allow-utc-timestamp-in-ids
  • rotating-logs
  • run-tests-blow-up-sooner
  • run-travis-on-master
  • sanity-check-1
  • scan-files-on-upload
  • schema_validation_101
  • schema_validation_102
  • send-auth-token-to-stagecraft
  • send-auth-token-to-stagecraft2
  • separate-admin-app
  • serialize-dev-startup
  • service-page-latest-data-transform
  • set-failover
  • set-travis-mongo-rs
  • set_2weeks_capped_size
  • set_2weeks_capped_size_2
  • shorter-tokens
  • show_failing_bucket_times
  • signonotron2-integration
  • simplify-conditional
  • skip-blank-values
  • skip-splinter-tests
  • split-up-write-api-feature-tests
  • stagcraft-integration-single-test-diff
  • stagecraft-integration
  • staging
  • staging-2018-08-29
  • start-up-documentation
  • status_json
  • subgroup_keys
  • sync-to-govuk-dev-vm
  • temp-new-query-repo-automate-creation-of-buckets
  • test-behave-travis
  • test-relative-date-params
  • tidy-nested-merge
  • tidy_manifest
  • transaction-explorer-transform
  • transactions-explorer-transform-clean-hist
  • transform-endpoint-tests
  • transformations
  • transforms-endpoint
  • travis-containerise
  • travis-experiments
  • treat-blank-cell-as-null
  • try-new-splinter
  • unicode-characters-transactions-explorer
  • unicode-is-hard
  • unpublisheddisclaimer
  • update-amqp-url-for-govuk-dev-vm
  • update-db-location
  • update-dev-stagecraft-urls
  • update-dev-transform-config
  • update-documentation
  • update-last-time-expected-migration
  • update-perfplat-client
  • update-readme-rabbitmq-dev-set-up
  • update-readme-replicatedb
  • update-readme-with-worker-info
  • update-replicate-script
  • update-testrunner
  • update-upload-page-copy
  • update-url-for-govuk-dev-vm
  • update-write-validations
  • update_last_time_expected_migration
  • upgrade_paas_stack
  • upgrade_splinter
  • upload-validation
  • use-postgres-159387515
  • use-urandom-not-random
  • use-write-api
  • useful-scripts
  • user-satisfaction-task
  • users-from-stagecraft
  • volumesInSubGroups
  • wat
  • well_formed_auth_headers_bruv
  • wercker
  • write-deployed-to-dev
  • write_api_accept_type_and_group

pending completion
1504

push

travis-ci

Jacob Ashdown
Add basic querying to MongoStorageEngine

Add the ability to do the most basic form of query with no filtering,
limiting, sorting or grouping.

Ensure datetime fields are converted to UTC.

Add ability to filter to MongoStorageEngine.query

Add the ability to filter by field and by start / end times.

Add sort and limit to MongoStorageEngine.query

Add group and period to MongoStorageEngine.query

This adds the group_by and period parameters. However, this does not
implement collect fields. That will come in a later commit.

Add collect fields to MongoStorageEngine.query

Add collect fields to grouped queries. This just adds the collecting of
the values, applying collect functions to these will be implemented
outside of the storage engine for the moment as it is currently done in
python.

Integrate mongo storage engine query

This is the smallest change I could figure out to integrate the mongo
storage engine's query.
Highlights:
- No response munging is done in the mongo layer now, it's done in
  NewDataSet
- A few functions about the query have been moved into properties on the
  Query object.
- None of the flask apps depend on DataSet or Database anymore

Move DataSet tests over to NewDataSet

Rewrite some tests on DataSet to the new class NewDataSet.

Change migrations to depend on MongoStorageEngine

This is a hack and migrations should be removed or fixed properly.

This change marks all existing migrations as deprecated as they don't
really make so much sense with the new model.

Remove backdrop.core.database and DataSet

This is quite a broad change that cleans up the previous commits. It
includes:

- Remove backdrop.core.database module, it is now completely
  superseded by backdrop.core.storage.mongo
- Remove DataSet class from backdrop.code.data_set, it is now completely
  superseded by NewDataSet (renaming will happen in a later commit)
- Remove query execution methods from Query object, they are now handled
  by NewDataSet. Moving query a... (continued)

1681 of 1860 relevant lines covered (90.38%)

0.9 hits per line

Relevant lines Covered
Build:
Build:
1860 RELEVANT LINES 1681 COVERED LINES
0.9 HITS PER LINE
Source Files on step_5_test
  • List 0
  • Changed 0
  • Source Changed 0
  • Coverage Changed 0
Coverage ∆ File Lines Relevant Covered Missed Hits/Line

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
1504 step_5_test Add basic querying to MongoStorageEngine Add the ability to do the most basic form of query with no filtering, limiting, sorting or grouping. Ensure datetime fields are converted to UTC. Add ability to filter to MongoStorageEngine.query Add th... push 02 Jun 2014 11:23AM UTC Jacob Ashdown travis-ci pending completion  
See All Builds (2391)
  • Repo on GitHub
  • CI Project
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2025 Coveralls, Inc