kubeflow / trainer / 39498019641
77%

DEFAULT BRANCH: master
Ran
Jobs 1
Files 45
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 75.012%. Remained the same
39498019641

push

github

web-flow 
fix(initializer): download only the objects under the S3 prefix (#4116)

* fix(initializer): download only the objects under the S3 prefix

S3Storage.download listed the bucket with the storage URI path as the prefix
and downloaded everything that came back. OpenDAL, like S3 ListObjects,
matches the prefix as a plain string, so s3://bucket/models/llama also
returned models/llama-2/... and models/llama2-7b/..., and the relative path
computed by slicing off the prefix placed them at -2/config.json and
2-7b/config.json inside the destination. The documented URI form has no
trailing slash, so this was the default outcome for any bucket with sibling
prefixes.

Keep only the object the prefix names and the objects below it, and compute
the relative path from the prefix without its trailing slash so both forms
produce the same layout.

Signed-off-by: Vidyansh <ezboi2312@gmail.com>

* fix(initializer): resolve S3 keys by exact object or directory

Follow the OpenDAL convention instead of filtering a prefix listing: a storage URI without a trailing slash is tried as an exact object first and falls back to the directory with the same name, and a URI with a trailing slash lists that directory. The directory is listed with a trailing slash, so sibling keys such as models/llama-2 are never returned for models/llama. A URI that matches no object now fails with a clear error instead of succeeding with an empty download, and a download where every object is skipped by the ignore patterns logs a warning. Tests cover the exact-object precedence, the bucket root, partial object names, and the empty result.

Fixes #4115

Signed-off-by: Vidyansh <ezboi2312@gmail.com>

---------

Signed-off-by: Vidyansh <ezboi2312@gmail.com>

3176 of 4234 relevant lines covered (75.01%)

0.75 hits per line

Jobs
ID Job ID Ran Files Coverage
1 39498019641.1 45
75.01
GitHub Action Run
Source Files on build 39498019641