exalearn / colmena

89%

master: 90%

LAST BUILD ON BRANCH issue-117

branch: issue-117

CHANGE BRANCH

x

Reset

• issue-117
• add-ijhpca-paper
• add-osx-support
• add_papers
• app-refactor
• better_cov
• ci-on-schedule
• doc_fix
• evict-inputs
• fix-conversion
• fix-deadlock-test
• fix-retries-tests
• fix-rtd
• fix-streaming
• fix_queue_pickle
• flake8-fix
• funcx_v1
• globus-compute
• issue-138
• master
• msg_sizes
• nicer_logs
• no-inspect
• nonredisqueues
• parsl_threads
• patch-1
• poetry
• proxystore-v0.4.0
• proxystore-v0.5.0
• proxystore-version-limit
• pypi-fix
• pythonexec
• qol_improvements
• remove-nonblock-race
• result-proxy-stats
• rtd_fix
• serialization-default
• setup_in_base
• setup_teardown
• streaming
• syn-app-updates
• synthetic-app
• timing-refactor
• upgrade-redispy
• v0.3.2
• v0.3.3
• v0.3.4
• v0.4.0
• v0.4.1
• v0.4.2
• v0.4.3
• v0.4.4
• v0.4.5
• v0.5.0
• v0.5.1
• v0.5.2
• v0.6.0
• v0.6.1
• v0.7.1
• v0.7.2

Build # 7199076175

Build Type

push

github

Committed by gpauloski

Commit Message

Avoid double serialize when proxying values (#117)

Proxied objects end up being serialized twice: first when the object is
serialized with Colmena to check the size of the serialized object and
then again by ProxyStore when the object is proxied.

ProxyStore supports passing custom serializer/deserializer functions so
this commit adds a shim serializer and a deserializer wrapper to
minimize serialization overheads when an object gets proxied.

Note that there is still one extra step when using Colmena's pickle
serialization method and ProxyStore. SerializationMethod.serialize
will pickle the object producing a byte string then convert those bytes
to a hex string then the shims convert that hex string back to bytes.
I.e., there's this intermediate string representation that's created
then discarded in this path. I don't really see a way of avoiding this
because its an artifact of the different serialization types between
Colmena and ProxyStore (str vs bytes).

Run Details

13 of 17 new or added lines in 1 file covered. (76.47%)

1044 of 1179 relevant lines covered (88.55%)

1.73 hits per line