FAIRmat-NFDI / pynxtools-em

17%

LAST BUILD ON BRANCH main

branch: main

CHANGE BRANCH

x

Reset

Sync Branches

• main
• CFFChecker
• bring-in-examples
• bump-ruff
• dependabot/pip/requests-2.32.4
• dependabot/pip/setuptools-78.1.1
• dependabot/pip/urllib3-2.5.0
• dependabot/pip/uv-0.8.6
• deprecate_wording_of_subparser
• em_app
• em_app_niac_changes
• fairmat1_final_pm
• fix_build_doc
• fix_materialsproject_example
• flatdict_revert_until_replacement
• hdf5_core_inspect
• hfive_apex_add_laue_group_support
• hfive_oxford_add_partial_support_v7.0
• hot_fix_rename_nomad_test
• issue103
• issue132
• issue139_harmonize_using_cruft
• issues_74_and_149_msa_spc_spd
• jeol_case_tifonly_with_metadata
• kassel_usecase_fix
• koch_group_runthrough_dec2025
• koch_summer2026
• nion
• pint_units_for_concept_mapping
• polishing
• prepare_v0.4.2
• prepare_v0.4.3
• prepare_v0.4.4
• prepare_v0.4.5
• py3.13
• pynx070_prepare030
• runthroughs_ebsd
• spellchecker
• switch_to_src_layout
• unpin_pint
• update_nomad_eln_schema
• warn_rather_than_ignore_when_seen_previously_unknown_content
• zenodo_allversions

Build # 27344635288

Build Type

push

github

Committed by web-flow

Commit Message

Issues 74 and 149 msa spc spd (#167)

* bump versions precommit and pyproject toml

* constrain rosettasciio always to use the latest

* skeleton rsciio_msa

* tweaking atom type handling and bibliography handling for oasisb example

* iso3_city_firstauthor naming information collected, starting mtex.h5 conversion run-through

* remove NeXus/MTex parser as this one has been updated and will be maintained via the pynxtools-microstructure plugin

* proper return values when file already exists and using logger

* reorganize location of the code for examples related to data ingestion studies, skeleton code for parsing, to be modified still from pynxtools-apm and pynxtools-microstructure, spellchecking

* spellchecking minor

* fixing batch_process queue, next steps i) test and run with msa, emd, dm2, dm3, dm4, iii) go to images

* explicit chunking for all parsers were applicable

* explicit chunking for all parsers were applicable

* working version of the msa parser

* spellchecking

* fix the issue that the running long ipynb session drain the main memory as intermediate objects and jupyter objects are not deallocated and thus memory safely returned to the operating system

* configure rosettasciio so that eds-streams are included

* adding cli executable script to overcome the problem of the significant memory consumption overhead of jupyter notebooks

* adding parsing of NXuser

* Adding a deterministic sampling to select random subsets and then decompress these to compose a representative set for the use case images, i.e. tif, bmp, with and without sidecar files, the sampling generates a configuration file (ods spreadsheet automatically, is useful also for showing how to ingest per project extra metadata), sub-sampling is required as the total number of images in the EM dataset collection contains more than 300k images which we currently cannot sieve through manually and for which the adding of atom_types is plagued by file names which do not inclu... (continued)

Coverage Stats

31 of 210 new or added lines in 14 files covered. (14.76%)

6 existing lines in 6 files now uncovered.

956 of 5662 relevant lines covered (16.88%)

0.17 hits per line