thanos / ExZarr

76%

main: 76%

LAST BUILD ON BRANCH zarr_v3_metadata_support

branch: zarr_v3_metadata_support

CHANGE BRANCH

x

Reset

Sync Branches

• zarr_v3_metadata_support
• Additional_Data_Types
• Array_Operations
• Chunking_and_Group
• Complete_Zarr_v3_Metadata_Handling
• Concurrent_RW_Support
• Documentation_and_Examples
• Google_Cloud_Storage_backend
• S3_storage_backend
• doc_cleanup
• docs_fix
• github_setup
• main
• new_documentaion
• new_notebooks
• nx_integration
• performance_and_benchmarks
• release_v1.0.0
• v0.3.0
• v2_and_v3_formats_conversions
• zarr3_extras
• zarr_v3

Build # cef22c59ffa1c60bf66222210c1e70827cb00f83-PR-145

Build Type

Pull #145

github

Committed by thanos

Commit Message

closed #144 - Metadata - Complete v3 Serialization & Caching
1. Zarr v3 Metadata JSON Serialization:
    - Jason.Encoder implementation for MetadataV3 ✓
    - from_json/1 parsing function ✓
    - Comprehensive codec and chunk grid handling ✓
  2. v2↔v3 Format Conversion:
    - ExZarr.FormatConverter module with convert/1 function ✓
    - MetadataV3.from_v2/1 and to_v2/1 conversion functions ✓
    - Documented limitations for incompatible features ✓

   New Implementations

  1. Consolidated Metadata Module (lib/ex_zarr/consolidated_metadata.ex):
    - Creates .zmetadata files bundling all array/group metadata
    - 90-99% reduction in cloud storage API calls for group operations
    - Functions: consolidate/1, read/1, exists?/1, remove/1
    - Supports both Zarr v2 and v3 formats
    - Handles nested groups with recursive/non-recursive options
    - Compatible with zarr-python consolidated metadata format
  2. Metadata Cache Module (lib/ex_zarr/metadata_cache.ex):
    - ETS-based GenServer for fast concurrent metadata access
    - Configurable TTL (default: 5 minutes) and max size (default: 1000 entries)
    - Automatic expiration cleanup every 60 seconds
    - LRU eviction when cache is full
    - Statistics tracking (hits, misses, hit rate)
    - Functions: get/1, put/2, invalidate/1, clear/0, stats/0

Pull Request Pull Request #145: closed #144 - Metadata - Complete v3 Serialization & Caching

Run Details

106 of 126 new or added lines in 2 files covered. (84.13%)

2883 of 3817 relevant lines covered (75.53%)

936.35 hits per line