umputun / feed-master
75%

DEFAULT BRANCH: master
Repo Added
Files 16
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

LAST BUILD ON BRANCH master
branch: SELECT
CHANGE BRANCH
x

coverage: 75.338% (+0.2%) from 75.174%
41242272240

push

github

web-flow 
Date finished YouTube live streams by download time (#184)

A YouTube live stream gets its `published` time when it is scheduled, not when it airs. `update()` moves an entry's date to the download time only when `published` is under 24h old, so a stream scheduled more than a day ahead kept the scheduling time and showed up in feeds back-dated, below newer episodes. Real case: Echo's "Один с Дмитрием Быковым" was scheduled 27 Oct 19:11 UTC, failed with "This live event will begin in 23 hours" until it ended, downloaded 28 Oct 21:21 UTC, and entered the feed dated 27 Oct.

The worker now records when it first reaches the download step for an entry (`first_seen` bucket in the youtube store, written before every download attempt, never overwritten, dropped by `SetProcessed`). `update()` resets the date when that first sighting was within 24h of `published`. A stream is first seen right after it is scheduled, so it gets the download time once it finally downloads. A new channel's backlog is first seen days after publishing and keeps its dates, as before.

`updated` from the YouTube feed is not used: it moves on any metadata edit, so recently edited old videos would jump to the top when a channel is added.

Behavior change: a regular video held back by a download outage longer than 24h also gets the recovery time as its date, the same way outages under 24h already did. Entries with no `first_seen` record, including everything pending at deploy, behave exactly as before.

48 of 66 new or added lines in 2 files covered. (72.73%)

1558 of 2068 relevant lines covered (75.34%)

20.94 hits per line

Relevant lines Covered
2068 RELEVANT LINES 1558 COVERED LINES
20.94 HITS PER LINE
Source Files on master

Recent builds

Builds Branch Commit Type Ran Committer Via Coverage
41242272240 master Date finished YouTube live streams by download time (#184) A YouTube live stream gets its `published` time when it is scheduled, not when it airs. `update()` moves an entry's date to the download time only when `published` is under 24h old, so a ... push web-flow github
75.34
41242082103 fix-live-stream-pubdate Merge remote-tracking branch 'origin/master' into fix-live-stream-pubdate lint conflicts in app/api, app/proc/twitter.go and execYtdlpUpdate resolved to master's versions, which carry the same fixes. Pull #184 umputun github
75.34
41241927365 fix-live-stream-pubdate Merge remote-tracking branch 'origin/master' into fix-live-stream-pubdate lint conflicts in app/api, app/proc/twitter.go and execYtdlpUpdate resolved to master's versions, which carry the same fixes. push umputun github
75.34
40638120975 refs/tags/v2.13.1 fix: skip re-dated items whose guid is already stored An Atom entry with only <updated> gets a new store key every time it is edited, so it was saved and posted to Telegram and Twitter again. Save now treats a non-empty GUID already present in th... push umputun github
75.17
40636505964 master fix: skip re-dated items whose guid is already stored An Atom entry with only <updated> gets a new store key every time it is edited, so it was saved and posted to Telegram and Twitter again. Save now treats a non-empty GUID already present in th... push umputun github
75.17
40636038833 master fix: preserve Atom entry dates and IDs Atom entries reach the processor with a zero timestamp and get skipped. Keep their IDs and publication dates, with updated as the fallback, and accept RFC3339 dates during normalization. push umputun github
75.05
37587336035 fix-atom-entry-dates fix: preserve Atom entry dates and IDs Atom entries reach the processor with a zero timestamp and get skipped. Keep their IDs and publication dates, with updated as the fallback, and accept RFC3339 dates during normalization. Pull #183 bensynapse github
75.05
35485058058 master docs: record how own PRs get merged past the review gate master requires one approving review that the owner cannot give on his own PRs, and Copilot posts COMMENTED rather than APPROVED, so every PR arrives blocked. The merge has been done with -... push umputun github
74.95
35484824921 master docs: correct what the youtube item limits actually control max_per_channel was documented as the per-channel download limit, which it is not. setDefaults gives every channel Keep from system.max_per_feed when the channel sets none, and for a con... push umputun github
74.95
35484671912 docs-youtube-item-limits docs: correct what the youtube item limits actually control max_per_channel was documented as the per-channel download limit, which it is not. setDefaults gives every channel Keep from system.max_per_feed when the channel sets none, and for a con... Pull #181 umputun github
74.95
See All Builds (524)