elixir-gettext / gettext

93%

main: 93%

LAST BUILD ON BRANCH parallel-locale-merge

branch: parallel-locale-merge

CHANGE BRANCH

x

Reset

Sync Branches

• parallel-locale-merge
• abdelhalim97/add-put_locale!
• al/1.18
• al/ci
• al/deprecate-plural-forms
• al/expo-plural
• al/new-backend-docs
• al/no-recompilation
• al/refactor
• al/release
• al/use-gettext-backend
• ci-bump-elixir-1.19-otp-28
• code-coverage
• dependabot/github_actions/actions/checkout-5.0.0
• dependabot/github_actions/actions/checkout-6.0.0
• dependabot/github_actions/actions/checkout-6.0.1
• dependabot/github_actions/actions/checkout-6.0.2
• dependabot/github_actions/actions/checkout-6.0.3
• dependabot/github_actions/erlef/setup-beam-1.20.2
• dependabot/github_actions/erlef/setup-beam-1.20.4
• dependabot/github_actions/erlef/setup-beam-1.21.0
• dependabot/github_actions/erlef/setup-beam-1.24.0
• deprecation_message
• extract_force_compile
• fix-concatenation-of-sigils
• fix-nofile
• fix-typos
• fix/elixir-1.20-warnings
• fix/extract-timeout
• improve-docs
• introduce_initial_top_comments_via_messages_struct
• jm/ci_deps_check_locked
• jm/conflicting_plural_message_400
• jm/default_header
• jm/deps-update
• jm/dialyzer-setup
• jm/extend_plural_interface
• jm/extract_merge_always
• jm/flaky_test_mix_cache
• jm/gh-workflow-updates
• jm/legalese
• jm/mention-igniter-changelog
• jm/metadata-stripping
• jm/plural_mod_function_fix
• jm/release/v0.23.2
• jm/singular_plural_uniq
• jv-erase-manifest-unless-on-upcoming-elixir
• jv-fix-redefining
• keep_custom_flags_option
• main
• msg_sort_iodata
• previous_messages
• refs/pull/376/merge
• release/v0.22.2
• release/v0.27.0
• runtime-gettext
• sort_by_msgid_case_insensitive
• test_up_to_date
• tiny_fix

Build # 888917e1e9e6af5b0f6c79bf7e502ab5532278ef-PR-436

Build Type

Pull #436

github

Committed by oliver-kriska

Commit Message

Parallelize locale merging in mix gettext.merge

mix gettext.merge processes locales sequentially, but each locale's
merge is fully independent: every locale task writes only to its own
<pot_dir>/<locale>/LC_MESSAGES/*.po files, the POT files at the root of
pot_dir are only read, and Gettext.Merger is stateless (pure functions
over Expo structs). Switch merge_all_locale_dirs/3 to Task.async_stream
over locales, mirroring the existing Task.async_stream over POT files
within a locale in merge_dirs/5.

The return value change (list -> :ok via Stream.run/1) is safe: the
only caller is merge_messages_dir/3, whose result is discarded in
run/1. The --locale single-locale path is untouched. A crashed locale
task raises out of Stream.run/1, matching the sequential version's
failure behavior. Mix.shell() output is GenServer-backed, so concurrent
lines serialize correctly; the only observable change is that progress
lines from different locales can interleave, as already happens in
Gettext.Compiler's parallel paths.

Benchmark on a 16-locale / 11-domain Phoenix app (160 PO files, ~4,700
msgids, Elixir 1.20.1 / OTP 29, Apple Silicon):

    mix gettext.merge priv/gettext --no-fuzzy

    before: 9.4s wall (55% CPU)
    after:  1.9-2.4s wall (341-381% CPU)

~4.9x speedup with byte-identical output files (clean git status over
committed PO state after both runs). The win scales with locale count.

Pull Request Pull Request #436: Parallelize locale merging in mix gettext.merge

Coverage Stats

3 of 3 new or added lines in 1 file covered. (100.0%)

647 of 698 relevant lines covered (92.69%)

26.7 hits per line