radio-t / super-bot

75%

master: 75%

LAST BUILD ON BRANCH feature/markdown-new-content-extraction

branch: SELECT

CHANGE BRANCH

x

Sync Branches

• No branch selected
• 22-fix-extend
• 22-issue
• 50-issue-fix
• add-history-support-to-openai
• add-pin-to-cut-message
• add-podcast-link-to-MsgBroadcastStarted-message
• agent-one
• allow-other-groups
• ban-by-multibot
• banhammer-ban-by-userid
• case_insensitive_openai
• context_openai
• cov-1
• dependabot/go_modules/github.com/redis/go-redis/v9-9.7.3
• dependabot/go_modules/go-modules-updates-1313cd4f86
• dependabot/go_modules/go-modules-updates-1fa0d48fe6
• dependabot/go_modules/go-modules-updates-46cd2b41fd
• dependabot/go_modules/go-modules-updates-4eb7ca3412
• dependabot/go_modules/go-modules-updates-6d46581adb
• dependabot/go_modules/go-modules-updates-83cb7119c5
• dependabot/go_modules/go-modules-updates-a3d3f3c0fb
• dependabot/go_modules/go-modules-updates-b97c974778
• dependabot/go_modules/go-modules-updates-d9f444d3b5
• dependabot/go_modules/go-modules-updates-ef4ba45f0b
• dependabot/go_modules/go-modules-updates-ffef3402cf
• dependabot/go_modules/golang.org/x/net-0.17.0
• dependabot/go_modules/golang.org/x/net-0.33.0
• dependabot/go_modules/golang.org/x/net-0.36.0
• dependabot/go_modules/golang.org/x/net-0.7.0
• dependabot/go_modules/golang.org/x/text-0.24.0
• dependabot/go_modules/golang.org/x/text-0.3.8
• enable_auto_response_fix
• export-broadcast-partial-match
• export-ignore-empty-decorations
• export-small-improvements
• feature/markdown-new-content-extraction
• feature/new-f-case
• fix-ban-no-username
• fix-exporter-panic
• fix-exporter-panic-2
• fix-mrkdown-parse-error
• fix-spammer-display-name
• fix-username
• idle
• is_reply
• kazhuravlev/calculate-time-to-stream
• master
• master-1
• mention_with_underscore
• more_w
• no-ban-old-messages
• only_wtf_check
• openai_wtf_ban
• parse_mode_enum
• paskal/add_dependabot
• paskal/channel_ban
• paskal/dependabot-disable-updates
• paskal/dependabot-security-updates
• paskal/error_handling
• paskal/facelift
• paskal/fix_golangci_lint
• paskal/fix_telegram_msgs
• paskal/moq
• paskal/offline_tests
• paskal/telegram_v5
• paskal/update-packages
• paskal/update_modules
• patch-1
• pincut
• preppost
• prevent_shadowing
• readme
• refs/tags/v0.2.0
• refs/tags/v0.3.0
• reply-to
• reporter-handle-deleted-messages
• revert-64-va/moar_wtfs
• spam-filter
• spam-local
• summary_for_user_links
• telegram
• tests-update
• unpin-messages
• update-to-go-1.24
• vendor_up
• wtf
• wtf-bot-message-formatting
• wtf_case
• wtf_human_duration
• wtfchecker1.1
• wtffix
• wttbot

Build # 23087426892

Build Type

Pull #156

github

Committed by grayodesa

Commit Message

replace uReadability with markdown.new for content extraction

uReadability often fails to extract content from websites that block
server-side requests. markdown.new (Cloudflare-powered) provides a
three-tier fallback pipeline (content negotiation → Workers AI →
browser rendering) that handles JS-heavy and bot-protected sites.

- add MarkdownNewClient implementing uKeeperGetter interface
- parse markdown.new response format (Title header + YAML frontmatter)
- replace UKeeperClient wiring in main.go
- remove --ur-api/--ur-token flags, add --mdnew-api flag
- add unit tests for client and response parser

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

Pull Request Pull Request #156: Replace uReadability with markdown.new for content extraction

Coverage Stats

76 of 83 new or added lines in 2 files covered. (91.57%)

2684 of 3569 relevant lines covered (75.2%)

54.52 hits per line