crutcher / bimm / #32

72%

push

Refactor dataset handling and streamline training pipeline for CINIC-… (#53)

* Refactor dataset handling and streamline training pipeline for CINIC-10 integration

- Removed unused `data.rs` and migrated datasets directly into `main.rs`.
- Replaced custom `CinicBatch` and `CinicBatcher` with generic Firehose-based batch handling.
- Introduced `ImageToTensorData` operator for image-to-tensor conversion.
- Enhanced configurability with new options (`min_learning_rate`, `dim_layout`).
- Transitioned to a unified schema-based operator environment for image loading, augmentation, and batching.
- Updated associated tests, configurations, and imports for consistency.

* Rename things.

* Refactor `swin_tiny` training pipeline and cleanup dependencies

- Removed `rs-cinic-10-burn` dependency and associated references.
- Replaced hardcoded strings with constants for column names.
- Improved error handling with `anyhow` across the pipeline.
- Updated `ValueBox` method `deserializing` to `parse_as`.
- Enhanced schema-based operator usage for image loading and augmentation.
- Consolidated `FirehoseExecutorBatcher` with `batch_result` helper.
- Updated tests, imports, and configurations for consistency.

* Update dependencies and refine `swin_tiny` configuration

- Upgraded `strum` to `0.27.2` and consolidated version management with workspace alignment.
- Adjusted `swin_tiny` training parameters: reduced learning rate, minimum learning rate, and number of workers.
- Removed unused `globwalk` dependency for cleanup.

* Refactor dataset handling in `swin_tiny` pipeline

- Removed `ComposedDataset` usage and directly initialized `ShuffledDataset` with seed.
- Updated schema with a new column for `SEED_COLUMN`.

* Refactor `swin_tiny` pipeline: add tensor stacking utility, optimize tensor loading, and clean up unused functions.

5 of 52 new or added lines in 6 files covered. (9.62%)

1980 of 2296 relevant lines covered (86.24%)

86.01 hits per line