treasure-data / perfectqueue

100%

master: 100%

LAST BUILD ON BRANCH use-CONNECTION_ID

branch: use-CONNECTION_ID

CHANGE BRANCH

x

Reset

• use-CONNECTION_ID
• Sequel-UniqueConstraintViolation
• add-TaskMonitorHook-spec
• add-a-test-for-long-waiting-queue
• add-more-specs
• add-specs
• avoid-deadlock-and-unlock-earlier
• avoid_table_metadata_lock_part1
• avoid_table_metadata_lock_part2
• check-timeout-on-acquire
• check-timeout-on-update-in-acquire
• delete-v0.8.44-finished-tasks
• disable-coveralls-outside-ci
• dont-delete-finished-tasks-of-v0.8.46
• enable-covealls
• enhance_logging
• ensure-kill-child
• explicitly-specify-services-mysql
• extend-rdb-connect-timeout
• extract_max_retry_as_config
• fix-cleanup_interval_count
• fix-random-failure
• force-index-for-acquire-query
• get_rid_of_v07_client
• include-spec-dir-for-simplecov
• initial-logger-support
• it-is-mysql2-ruby
• keep-gziped-data-as-is
• log-acquire-time
• master
• merge-v07
• mysql-compress
• parallel-acquire
• randomize-initial-cleanup_interval_count
• rdb-sslmode-support
• relax-sequel
• relax-sequel-version
• remove-cancel_request
• remove-huge-task-dump
• remove-sqlite-support
• remove-transaction
• remove-unused-code
• remove-useless-created_at-check
• require-ahead-on-time
• retry-connect
• retry-connect-v08
• retry-on-finish
• revert-v0.8.44-migration-path
• rspec3
• run-48-delete-query-in-connect_locked
• separate-delete-and-acquire-queries
• sleep-between-lock-retry
• stable-0.8.44
• stress
• stress-test-supports-multiple-queues
• stress-test-tool
• suppress-warning
• use-DO-for-RELEASE_LOCK
• use-timeout-for-44-compat
• use-v08-strategy-if-with-resource-limit
• v0.10.0
• v0.10.1
• v0.8.44
• v0.8.44.1
• v0.8.45
• v0.8.46
• v0.8.47
• v0.8.48
• v0.8.49
• v0.8.50
• v0.8.51
• v0.8.51_for_test
• v0.8.52
• v0.8.53
• v0.8.54
• v0.9.0
• v0.9.1
• v0.9.2
• v08
• v08-sleep-between-lock-retry

Build # 288

Build Type

push

travis-ci

Committed by nurse

Commit Message

UPDATE first strategy

To acquire tasks, run UPDATE and SELECT insteadof SELECT and UPDATE.

This adds a new column `owner`, whose content is expected to be CONNECTION_ID(),
 to identify which connection acquires the task.
With this column, RDBCompatBackend#acquire now doesn't get table lock nor use
GET_LOCK(), but adopt mark-and-select method.
It locks only oldest N rows and their gaps during UPDATE.

CONNECTION_ID()
---------------

64bit unsigned integer from the server's internal thread ID counter.
This is unique while the MySQL server running.
http://dev.mysql.com/doc/refman/5.6/en/information-functions.html#function_connection-id
https://bugs.mysql.com/bug.php?id=19806

References
---------------

* http://yuuki.hatenablog.com/entry/go-and-mysql-jobqueue
* https://blog.engineyard.com/2011/5-subtle-ways-youre-using-mysql-as-a-queue-and-why-itll-bite-you
* https://github.com/treasure-data/perfectqueue/pull/15#discussion_r52893022

Run Details

2081 of 2081 relevant lines covered (100.0%)

8.38 hits per line