umputun / tg-spam

79%

master: 82%

LAST BUILD ON BRANCH optimize-exclude-match

branch: optimize-exclude-match

CHANGE BRANCH

x

Reset

• optimize-exclude-match
• 12-provide-flag-to-not-remove-user-after-unban
• abnormal-spacing
• add-from-detected-spam
• add-slog
• admin-info
• alkk/master
• approved-users-ui
• audio-check
• autoadd-ham
• ban-user-instead-of-restriction
• callback
• cas-retry-fix
• check-by-id
• classifier-unlearn
• clean-invisible-chars
• clean-msg-fix
• db-backup
• dedup-samples
• dependabot
• dependabot/github_actions/github-actions-updates-20c91dd2a4
• dependabot/github_actions/github-actions-updates-fcf4e41b5c
• dependabot/go_modules/_examples/simplechat/github.com/umputun/tg-spam-1.18.0
• dependabot/go_modules/_examples/simplechat/go-modules-updates-0c570b14c2
• dependabot/go_modules/_examples/simplechat/go-modules-updates-28ef041b99
• dependabot/go_modules/_examples/simplechat/go-modules-updates-2c4077a24b
• dependabot/go_modules/_examples/simplechat/go-modules-updates-4af97a05b0
• dependabot/go_modules/_examples/simplechat/go-modules-updates-75e783174d
• dependabot/go_modules/_examples/simplechat/go-modules-updates-796ea05e18
• dependabot/go_modules/_examples/simplechat/go-modules-updates-8d185506e8
• dependabot/go_modules/_examples/simplechat/go-modules-updates-9e39e63601
• dependabot/go_modules/_examples/simplechat/go-modules-updates-aed0a436bf
• dependabot/go_modules/_examples/simplechat/go-modules-updates-b841ea8733
• dependabot/go_modules/_examples/simplechat/go-modules-updates-d503fb43d4
• dependabot/go_modules/_examples/simplechat/go-modules-updates-dd111468ba
• dependabot/go_modules/_examples/simplechat/go-modules-updates-e06798f191
• dependabot/go_modules/_examples/simplechat/modernc.org/sqlite-1.37.0
• dependabot/go_modules/github.com/go-pkgz/testutils-0.4.2
• dependabot/go_modules/go-modules-updates-589f0c7bc4
• dependabot/go_modules/go-modules-updates-75e783174d
• dependabot/go_modules/go-modules-updates-796ea05e18
• dependabot/go_modules/go-modules-updates-94abde7b92
• dependabot/go_modules/go-modules-updates-db89e520f1
• dependabot/go_modules/go_modules-232a611e2d
• dependabot/go_modules/go_modules-5a9c29dde4
• detect-story
• detector-persist
• download-spam-button
• drop-chi
• dyn-update
• easy-install-docs
• eliminate-obsolete-flag
• emoji-optimization
• empty-text-locator
• extra-delete-ids
• extract-lib
• faster-removal
• feat/dictionary-management-ui
• feat/handle-edited-messages
• feat/sqlite-to-postgres-converter
• feature-297-openai-thinking
• feature-custom-prompts
• feature-db-configuration
• feature-lua-plugins
• feature/add-spam-filter
• feature/aggressive-cleanup
• feature/bot-no-sound
• feature/delete-system-messages
• feature/digitalocean-deployment
• feature/duplicate-detection
• feature/duplicate-detection-aggressive-cleanup
• feature/info-on-unban
• feature/issue92
• feature/keyboard-detection
• feature/mentions-limit
• feature/openai-check-short-messages
• feature/resend-orig-msg
• feature/silent-messages
• feature/user-spam-reporting
• feature/username-symbols-check
• feature/username-userid-stopwords
• fix-174
• fix-183
• fix-295-giveaway-forwards
• fix-304-reasoning-effort
• fix-admin-error-propagation
• fix-firstmgs-count
• fix-forward-detection
• fix-info-button-markdown-escaping
• fix-issue-223
• fix-optional-user-id
• fix-samples-migration
• fix-settings-formatting
• fix-video-listener-event
• fix/image-only-spam-reporting
• fix/issue107
• fix/short-message-approval
• forward-check
• hashed-web-passwd
• ignore-classifier
• imporve-hist-openai
• improved-settings-page
• letterlikе
• lib-example
• links-only
• linter-v2-migration
• loading-indicator-delete
• master
• meta-checks
• min-spam-probability
• more-commands
• more-ui
• multi-first
• multi-lang
• multi-storage
• negative-time
• normalize-spam-text
• opeani-ignore
• open-ai
• openai-apibase
• openai-veto
• optimize-sample-loader
• paskal/bump_modules
• paskal/claude
• paskal/dependabot-disable-updates
• paskal/dependabot-security-updates
• paskal/dockerfile_comment
• paskal/fix_ci
• paskal/fix_golangci_lint
• paskal/fix_typos
• paskal/go-flags
• paskal/openai_doc
• paskal/remove-unused-code
• paskal/remove_unused_moq
• paskal/small-fixes
• paskal/switch_to_newer_tb_lib
• paskal/test_examples
• paskal/update_doc_and_readme
• paskal/update_modules
• paskal/use_ghcrio
• patch-1
• persistent-locator
• persitent-ids
• pic-caption
• refactor-admin
• refs/tags/v0.1.0
• refs/tags/v0.1.1
• refs/tags/v0.2.0
• refs/tags/v0.2.1
• refs/tags/v0.2.2
• refs/tags/v0.2.3
• refs/tags/v0.2.4
• refs/tags/v1.0.0
• refs/tags/v1.0.1
• refs/tags/v1.1.0
• refs/tags/v1.1.2
• refs/tags/v1.10.0
• refs/tags/v1.10.1
• refs/tags/v1.10.2
• refs/tags/v1.10.3
• refs/tags/v1.11.0
• refs/tags/v1.11.1
• refs/tags/v1.11.2
• refs/tags/v1.11.3
• refs/tags/v1.11.4
• refs/tags/v1.11.5
• refs/tags/v1.11.6
• refs/tags/v1.12.0
• refs/tags/v1.12.1
• refs/tags/v1.12.2
• refs/tags/v1.13.0
• refs/tags/v1.13.1
• refs/tags/v1.13.2
• refs/tags/v1.13.3
• refs/tags/v1.13.4
• refs/tags/v1.13.5
• refs/tags/v1.13.6
• refs/tags/v1.14.0
• refs/tags/v1.14.1
• refs/tags/v1.14.2
• refs/tags/v1.14.3
• refs/tags/v1.15.0
• refs/tags/v1.16.0
• refs/tags/v1.16.1
• refs/tags/v1.16.2
• refs/tags/v1.16.3
• refs/tags/v1.17.0
• refs/tags/v1.18.0
• refs/tags/v1.18.1
• refs/tags/v1.19.0
• refs/tags/v1.19.2
• refs/tags/v1.2.0
• refs/tags/v1.2.1
• refs/tags/v1.2.2
• refs/tags/v1.2.3
• refs/tags/v1.2.4
• refs/tags/v1.2.5
• refs/tags/v1.20.0
• refs/tags/v1.20.1
• refs/tags/v1.3.0
• refs/tags/v1.3.1
• refs/tags/v1.4.0
• refs/tags/v1.4.1
• refs/tags/v1.4.2
• refs/tags/v1.5.0
• refs/tags/v1.5.1
• refs/tags/v1.5.2
• refs/tags/v1.5.3
• refs/tags/v1.6.0
• refs/tags/v1.6.1
• refs/tags/v1.7.0
• refs/tags/v1.8.0
• refs/tags/v1.8.1
• refs/tags/v1.8.2
• refs/tags/v1.8.3
• refs/tags/v1.9.0
• remove-from-aproved
• reply-ban
• req-history
• retry-openai
• retry-plain
• rt-aproved-save
• settings-ui
• small-screen-styles
• soft-ban
• softmax_float64_underflow
• spam-storage
• sqlite-all
• supers-by-id
• training-ban
• training-mode
• unban-confirm
• universal-multilang
• v1.4.2
• video-check
• volume-warns
• web-checker
• webapi
• would

Build # 12678723087

Build Type

Pull #219

github

Committed by umputun

Commit Message

Optimize excluded tokens storage and lookup

This commit changes the storage of excluded tokens from a slice to a map
for faster lookup. It also updates related tests and benchmarks to reflect
this change. The tokenize function is modified to use the new map structure
for more efficient token exclusion checks.

Pull Request Pull Request #219: Optimize excluded tokens storage and lookup

Run Details

3164 of 3987 relevant lines covered (79.36%)

83.93 hits per line