thanos / ExZarr / cef22c59ffa1c60bf66222210c1e70827cb00f83-PR-145

76%

main: 76%

Pull #145

github

closed #144 - Metadata - Complete v3 Serialization & Caching
1. Zarr v3 Metadata JSON Serialization:
    - Jason.Encoder implementation for MetadataV3 ✓
    - from_json/1 parsing function ✓
    - Comprehensive codec and chunk grid handling ✓
  2. v2↔v3 Format Conversion:
    - ExZarr.FormatConverter module with convert/1 function ✓
    - MetadataV3.from_v2/1 and to_v2/1 conversion functions ✓
    - Documented limitations for incompatible features ✓

   New Implementations

  1. Consolidated Metadata Module (lib/ex_zarr/consolidated_metadata.ex):
    - Creates .zmetadata files bundling all array/group metadata
    - 90-99% reduction in cloud storage API calls for group operations
    - Functions: consolidate/1, read/1, exists?/1, remove/1
    - Supports both Zarr v2 and v3 formats
    - Handles nested groups with recursive/non-recursive options
    - Compatible with zarr-python consolidated metadata format
  2. Metadata Cache Module (lib/ex_zarr/metadata_cache.ex):
    - ETS-based GenServer for fast concurrent metadata access
    - Configurable TTL (default: 5 minutes) and max size (default: 1000 entries)
    - Automatic expiration cleanup every 60 seconds
    - LRU eviction when cache is full
    - Statistics tracking (hits, misses, hit rate)
    - Functions: get/1, put/2, invalidate/1, clear/0, stats/0

106 of 126 new or added lines in 2 files covered. (84.13%)

2883 of 3817 relevant lines covered (75.53%)

936.35 hits per line