umputun / tg-spam

82%

master: 83%

LAST BUILD ON BRANCH fix-stopword-space-normalization

branch: fix-stopword-space-normalization

CHANGE BRANCH

x

Reset

• fix-stopword-space-normalization
• 12-provide-flag-to-not-remove-user-after-unban
• abnormal-spacing
• add-from-detected-spam
• add-slog
• admin-info
• alkk/master
• approved-users-ui
• audio-check
• autoadd-ham
• ban-user-instead-of-restriction
• callback
• cas-retry-fix
• check-by-id
• classifier-unlearn
• clean-invisible-chars
• clean-msg-fix
• db-backup
• dedup-samples
• dependabot
• dependabot/github_actions/github-actions-updates-20c91dd2a4
• dependabot/github_actions/github-actions-updates-fcf4e41b5c
• dependabot/go_modules/_examples/simplechat/github.com/umputun/tg-spam-1.18.0
• dependabot/go_modules/_examples/simplechat/go-modules-updates-0c570b14c2
• dependabot/go_modules/_examples/simplechat/go-modules-updates-28ef041b99
• dependabot/go_modules/_examples/simplechat/go-modules-updates-2c4077a24b
• dependabot/go_modules/_examples/simplechat/go-modules-updates-4af97a05b0
• dependabot/go_modules/_examples/simplechat/go-modules-updates-75e783174d
• dependabot/go_modules/_examples/simplechat/go-modules-updates-796ea05e18
• dependabot/go_modules/_examples/simplechat/go-modules-updates-8d185506e8
• dependabot/go_modules/_examples/simplechat/go-modules-updates-9e39e63601
• dependabot/go_modules/_examples/simplechat/go-modules-updates-aed0a436bf
• dependabot/go_modules/_examples/simplechat/go-modules-updates-b841ea8733
• dependabot/go_modules/_examples/simplechat/go-modules-updates-d503fb43d4
• dependabot/go_modules/_examples/simplechat/go-modules-updates-dd111468ba
• dependabot/go_modules/_examples/simplechat/go-modules-updates-e06798f191
• dependabot/go_modules/_examples/simplechat/modernc.org/sqlite-1.37.0
• dependabot/go_modules/github.com/go-pkgz/testutils-0.4.2
• dependabot/go_modules/go-modules-updates-589f0c7bc4
• dependabot/go_modules/go-modules-updates-75e783174d
• dependabot/go_modules/go-modules-updates-796ea05e18
• dependabot/go_modules/go-modules-updates-94abde7b92
• dependabot/go_modules/go-modules-updates-db89e520f1
• dependabot/go_modules/go_modules-232a611e2d
• dependabot/go_modules/go_modules-5a9c29dde4
• dependabot/go_modules/go_modules-dd7da38a6b
• detect-story
• detector-persist
• download-spam-button
• drop-chi
• dyn-update
• easy-install-docs
• eliminate-obsolete-flag
• emoji-optimization
• empty-text-locator
• exact-match-stopwords
• extra-delete-ids
• extract-lib
• faster-removal
• feat/dictionary-management-ui
• feat/e2e-ui-tests
• feat/handle-edited-messages
• feat/meta-contact-only-354
• feat/sqlite-to-postgres-converter
• feature-297-openai-thinking
• feature-custom-prompts
• feature-db-configuration
• feature-lua-plugins
• feature/add-spam-filter
• feature/aggressive-cleanup
• feature/auto-ban-threshold
• feature/bot-no-sound
• feature/delete-system-messages
• feature/digitalocean-deployment
• feature/duplicate-detection
• feature/duplicate-detection-aggressive-cleanup
• feature/info-on-unban
• feature/issue92
• feature/keyboard-detection
• feature/mentions-limit
• feature/openai-check-short-messages
• feature/report-enhancements
• feature/resend-orig-msg
• feature/silent-messages
• feature/user-spam-reporting
• feature/username-symbols-check
• feature/username-userid-stopwords
• fix-174
• fix-183
• fix-295-giveaway-forwards
• fix-304-reasoning-effort
• fix-admin-error-propagation
• fix-ci-caching
• fix-ci-disk-space
• fix-firstmgs-count
• fix-forward-detection
• fix-gpt5-max-tokens-support
• fix-info-button-markdown-escaping
• fix-issue-223
• fix-openai-short-msg-veto-335
• fix-optional-user-id
• fix-orphaned-report-deletion-343
• fix-samples-migration
• fix-settings-formatting
• fix-video-listener-event
• fix/anonymous-admin-posts-skip-spam-check
• fix/caption-link-entities
• fix/duplicate-detection-message-edits
• fix/image-caption-mentions
• fix/image-only-spam-reporting
• fix/issue-348-validate-bot-username
• fix/issue107
• fix/media-short-text-detection
• fix/rest-writeheader-renderjson
• fix/short-message-approval
• forward-check
• hashed-web-passwd
• ignore-classifier
• imporve-hist-openai
• improved-settings-page
• letterlikе
• lib-example
• links-only
• linter-v2-migration
• loading-indicator-delete
• master
• meta-checks
• min-spam-probability
• more-commands
• more-ui
• multi-first
• multi-lang
• multi-storage
• negative-time
• no-giveaway
• normalize-spam-text
• opeani-ignore
• open-ai
• openai-apibase
• openai-veto
• optimize-exclude-match
• optimize-sample-loader
• paskal/bump_modules
• paskal/claude
• paskal/dependabot-disable-updates
• paskal/dependabot-security-updates
• paskal/dockerfile_comment
• paskal/fix_ci
• paskal/fix_golangci_lint
• paskal/fix_typos
• paskal/go-flags
• paskal/openai_doc
• paskal/remove-unused-code
• paskal/remove_unused_moq
• paskal/small-fixes
• paskal/switch_to_newer_tb_lib
• paskal/test_examples
• paskal/update_doc_and_readme
• paskal/update_modules
• paskal/use_ghcrio
• patch-1
• persistent-locator
• persitent-ids
• pic-caption
• refactor-admin
• refs/tags/v0.1.0
• refs/tags/v0.1.1
• refs/tags/v0.2.0
• refs/tags/v0.2.1
• refs/tags/v0.2.2
• refs/tags/v0.2.3
• refs/tags/v0.2.4
• refs/tags/v1.0.0
• refs/tags/v1.0.1
• refs/tags/v1.1.0
• refs/tags/v1.1.2
• refs/tags/v1.10.0
• refs/tags/v1.10.1
• refs/tags/v1.10.2
• refs/tags/v1.10.3
• refs/tags/v1.11.0
• refs/tags/v1.11.1
• refs/tags/v1.11.2
• refs/tags/v1.11.3
• refs/tags/v1.11.4
• refs/tags/v1.11.5
• refs/tags/v1.11.6
• refs/tags/v1.12.0
• refs/tags/v1.12.1
• refs/tags/v1.12.2
• refs/tags/v1.13.0
• refs/tags/v1.13.1
• refs/tags/v1.13.2
• refs/tags/v1.13.3
• refs/tags/v1.13.4
• refs/tags/v1.13.5
• refs/tags/v1.13.6
• refs/tags/v1.14.0
• refs/tags/v1.14.1
• refs/tags/v1.14.2
• refs/tags/v1.14.3
• refs/tags/v1.15.0
• refs/tags/v1.16.0
• refs/tags/v1.16.1
• refs/tags/v1.16.2
• refs/tags/v1.16.3
• refs/tags/v1.17.0
• refs/tags/v1.18.0
• refs/tags/v1.18.1
• refs/tags/v1.19.0
• refs/tags/v1.19.2
• refs/tags/v1.2.0
• refs/tags/v1.2.1
• refs/tags/v1.2.2
• refs/tags/v1.2.3
• refs/tags/v1.2.4
• refs/tags/v1.2.5
• refs/tags/v1.20.0
• refs/tags/v1.20.1
• refs/tags/v1.21.0
• refs/tags/v1.22.0
• refs/tags/v1.23.0
• refs/tags/v1.23.1
• refs/tags/v1.23.2
• refs/tags/v1.23.3
• refs/tags/v1.3.0
• refs/tags/v1.3.1
• refs/tags/v1.4.0
• refs/tags/v1.4.1
• refs/tags/v1.4.2
• refs/tags/v1.5.0
• refs/tags/v1.5.1
• refs/tags/v1.5.2
• refs/tags/v1.5.3
• refs/tags/v1.6.0
• refs/tags/v1.6.1
• refs/tags/v1.7.0
• refs/tags/v1.8.0
• refs/tags/v1.8.1
• refs/tags/v1.8.2
• refs/tags/v1.8.3
• refs/tags/v1.9.0
• remove-from-aproved
• reply-ban
• req-history
• retry-openai
• retry-plain
• rt-aproved-save
• settings-ui
• small-screen-styles
• soft-ban
• softmax_float64_underflow
• spam-storage
• sqlite-all
• supers-by-id
• training-ban
• training-mode
• unban-confirm
• universal-multilang
• update-go-1.25
• v1.4.2
• video-check
• volume-warns
• web-checker
• webapi
• would

Build # 19289208180

Build Type

Pull #334

github

Committed by umputun

Commit Message

fix: normalize multiple spaces in stopword matching

Stopword detection was failing when messages contained multiple consecutive spaces between words. For example, 'дам денег' wouldn't match 'Дам  денег' (with two spaces) even though case-insensitive matching was working.

Changes:
- Add normalizeSpaces() function to collapse multiple spaces into single spaces
- Update isStopWord() to normalize both message text and stopwords before comparison
- Apply normalization to username/userid stopword checks as well
- Add comprehensive test coverage for extra spaces and case variations

Pull Request Pull Request #334: fix: normalize multiple spaces in stopword matching

Run Details

9 of 9 new or added lines in 1 file covered. (100.0%)

6171 of 7537 relevant lines covered (81.88%)

273.45 hits per line