• Home
  • Features
  • Pricing
  • Docs
  • Announcements
  • Sign In

SwissDataScienceCenter / renku-data-services / 16521588698
86%

Build:
DEFAULT BRANCH: main
Ran 25 Jul 2025 12:19PM UTC
Jobs 2
Files 325
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst

25 Jul 2025 12:08PM UTC coverage: 87.21% (-0.003%) from 87.213%
16521588698

push

github

web-flow
fix: refactor get_all_project_copies for performance reasons (#940)

The implementation had some issues that make it slow:

- It doesn't implement any paging, so we are potentially transporting
  a lot of data. This is not fixed in this PR.

- At very first, it retrieves *all* copied projects for a given
  template_id and then filters it in memory to retain only the
  accessible ones of the current user.

  This is done by next retrieving all project ids the user has access
  to, including all public ones. This is potentially returning a lot
  of data.

  The filtering then continues to loop through the database objects
  and checks each id for membership in the `project_ids` list. A
  python list has O(n) time for membership tests, so depending on the
  number of results this gets slow quite quickly.

Measuring this call shows that by far the longest time is spent
filtering:
```python
[p for p in project_orms if p.id in project_ids]
```
It can quickly get to multi-seconds response time, if the data is a
bit larger.

The improvements involve the following:

- Exchange order of retrieval: first get all accessible project ids,
  except those for public projects. This can reduce the resulting set of
  ids to check against as it doesn't contain public entities.

- With these ids, the SQL query is constructed to only return projects
  whose ids are in this set. If the scope is read, then the condition
  includes all public ones and the ones the user has more specific
  access to.

In my measurements on my machine with a generated data set of 2000
project copies, these changes resulted in 10x faster response times.

9 of 9 new or added lines in 1 file covered. (100.0%)

3 existing lines in 2 files now uncovered.

21643 of 24817 relevant lines covered (87.21%)

1.53 hits per line

Uncovered Existing Lines

Lines Coverage ∆ File
1
83.64
-0.45% components/renku_data_services/crc/models.py
2
92.68
0.0% components/renku_data_services/storage/blueprints.py
Jobs
ID Job ID Ran Files Coverage
1 run-main-tests - 16521588698.1 25 Jul 2025 12:27PM UTC 325
86.69
GitHub Action Run
2 run-schemathesis-tests - 16521588698.2 25 Jul 2025 12:19PM UTC 325
66.33
GitHub Action Run
Source Files on build 16521588698
  • Tree
  • List 325
  • Changed 9
  • Source Changed 1
  • Coverage Changed 9
Coverage ∆ File Lines Relevant Covered Missed Hits/Line
  • Back to Repo
  • Github Actions Build #16521588698
  • dec877aa on github
  • Prev Build on main (#16502224130)
  • Next Build on main (#16725247450)
STATUS · Troubleshooting · Open an Issue · Sales · Support · CAREERS · ENTERPRISE · START FREE · SCHEDULE DEMO
ANNOUNCEMENTS · TWITTER · TOS & SLA · Supported CI Services · What's a CI service? · Automated Testing

© 2025 Coveralls, Inc