exalearn / colmena / 7247778572

90%

Build Type

push

github

Committed by web-flow

Commit Message

Improve serialization efficiency when objects are proxied (#120)

* Allow ProxyStore v0.6

* Get store once when serializing Result

* Avoid double serialize when proxying values (#117)

Proxied objects end up being serialized twice: first when the object is
serialized with Colmena to check the size of the serialized object and
then again by ProxyStore when the object is proxied.

ProxyStore supports passing custom serializer/deserializer functions so
this commit adds a shim serializer and a deserializer wrapper to
minimize serialization overheads when an object gets proxied.

Note that there is still one extra step when using Colmena's pickle
serialization method and ProxyStore. SerializationMethod.serialize
will pickle the object producing a byte string then convert those bytes
to a hex string then the shims convert that hex string back to bytes.
I.e., there's this intermediate string representation that's created
then discarded in this path. I don't really see a way of avoiding this
because its an artifact of the different serialization types between
Colmena and ProxyStore (str vs bytes).

Run Details

16 of 20 new or added lines in 1 file covered. (80.0%)

25 existing lines in 1 file now uncovered.

1044 of 1179 relevant lines covered (88.55%)

1.73 hits per line