mindersec / minder / 37385833874

63%

push

github

fix(gitlab): paginate project listing (#6751)

* Paginate GitLab project listing

ListAllRepositories issued a single request to the projects endpoint,
and GitLab caps unpaginated responses at 20 items, so any account with
more than 20 accessible projects silently lost the rest during repo
registration. Add a paginated variant of the REST helper that follows
the X-Next-Page response header at GitLab's maximum page size and use
it for the project listing. An empty page stops the loop even when the
server keeps reporting a next page.

* Bound GitLab pagination and share the GET helper

Review feedback on the pagination change:

- Merge the single-page fetch back into the existing GET helper,
  renamed restGet, which now returns the decoded value and the
  response headers. restGetPaginated builds on it, so the request,
  status check, 404 mapping and error strings exist once.
- Bound the listing at 1000 pages and treat reaching the bound as an
  error. X-Next-Page is parsed as an integer and must advance; a
  missing, malformed, repeated or backwards next page ends the listing,
  so a misbehaving server cannot keep us looping.
- Parse the path once and set page and per_page on its query; the doc
  comment now says those two are overwritten.
- When a page fails, return the repositories fetched so far with the
  error, the same contract the GitHub delegates use; the caller decides
  whether a partial listing is usable.
- Drop the gl prefix from the new identifiers, and split the long call
  that tripped lll.
- Fold the pagination cases into an end-to-end TestListAllRepositories,
  adding non-advancing and backwards next-page cases and the page limit.

* Paginate the stale GitLab webhook cleanup

cleanUpStaleWebhooks listed a project's hooks with a single request,
so with more than 20 hooks any stale Minder hook beyond the first page
was never found or deleted. Use the paginated helper, and add a test
that places the stale hook on the second page.

43 of 47 new or added lines in 3 files covered. (91.49%)

22398 of 35584 relevant lines covered (62.94%)

51.82 hits per line