kubeflow / sdk / 32730151414
86%

DEFAULT BRANCH: main
Ran
Jobs 2
Files 95
Run time 1min
Badge
Embed ▾
README BADGES
x

If you need to use a raster PNG badge, change the '.svg' to '.png' in the link

Markdown

Textile

RDoc

HTML

Rst
coverage: 85.588% (+0.01%) from 85.574%
32730151414

push

github

web-flow 
fix(spark): require host in Spark Connect URL validation (#661)

* fix(spark): require host in Spark Connect URL validation

validate_spark_connect_url() checked the URL scheme and port but never
the host, so a URL such as "sc://:15002" passed validation and returned
True. urlparse() reports hostname=None for that input while still
reporting port=15002, so both existing checks succeeded.

SparkClient.connect() calls this validator and then passes base_url to
SparkSession.builder.remote(). Spark Connect creates the session lazily,
so nothing fails at that point and the first operation retries against an
unreachable address. That retry behaviour is the same for any unreachable
host, so it is not caused by the missing host. The difference is that an
unreachable hostname cannot be detected up front while a URL with no host
at all can be, which is what connect() already documents as
"ValueError: If base_url is invalid".

Add a host check between the scheme and port checks. It is ordered before
the port check because parsed.port raises its own ValueError on a
malformed port, which would otherwise mask a missing host. This also
improves the message for "sc://:abc", which previously reported
"Port could not be cast to integer value as abc".

Add two unit tests following the existing parametrized TestCase pattern
in utils_test.py: the missing-host case, and a missing host combined with
a malformed port to cover the check ordering.

A differential check over 26 URLs confirms only inputs with
hostname=None change behaviour. IPv4, IPv6, cluster-local FQDNs and the
output of build_service_url() are unaffected.

Assisted-by: Claude
Signed-off-by: Yusuf Emre Yilmaz <yusufemreyllmz@gmail.com>

* test(spark): cover missing-host URLs in backend_test as well

Signed-off-by: Yusuf Emre Yilmaz <yusufemreyllmz@gmail.com>

---------

Signed-off-by: Yusuf Emre Yilmaz <yusufemreyllmz@gmail.com>

6366 of 7438 relevant lines covered (85.59%)

1.71 hits per line

Jobs
ID Job ID Ran Files Coverage
1 python-3.10 - 32730151414.1 95
85.59
 GitHub Action Run
2 python-3.11 - 32730151414.2 95
85.59
 GitHub Action Run
Source Files on build 32730151414