21571

Committed 16 Jun 2026 07:26AM UTC coverage: 70.816% (-0.02%) from 70.836%

Build # 21571

Build Type

push

circleci

Committed by

web-flow

Commit Message

spatial: nearest-neighbour lookups via spatial server; drop grid (#764)

* feat(spatial): nearest-neighbour lookups via spatial server; drop grid

Replace the per-table expanding-bounding-box MySQL nearest searches with the
iznik-spatial-go KNN index, and remove the vestigial locations_grids approach.

- spatial-go: new PostcodesDataset (point, ~2M rows, Extra=nil) + registered.
- Location::closestPostcode and IncomingMailService::findClosestPostcodeId
  (a duplicate) collapse to one SpatialQueryService->nearestIds('postcodes')
  call; expanding bbox removed.
- apiv2 location.ClosestPostcode -> spatial.KNN("postcodes"); closestPostcodeMySQL removed.
- Job::nearLocation -> spatial 'jobs' KNN; digest cpc floor 0.02 -> 0.10
  (matches the spatial index and the public jobs page).
- NewsfeedDigestService -> spatial 'newsfeed' KNN (nearest-N, re-filtered in
  MySQL); grow-radius getNearbyDistance/boxSql removed.
- Grid: DoogalService stops writing gridid; gridid dropped from Location
  PUBLIC_ATTS and UserLocationRemapService (no remaining consumers).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* feat(spatial): add admin /v1/{dataset}/upsert for test seeding + postcodes dataset test

The upsert endpoint inserts items by WKT (point or polygon, auto-detected) into
a live dataset index, decoupled from the MySQL rebuild — so integration tests can
seed a known geometry, exercise the real KNN path, then remove it. Smoke-tested:
upsert 2 postcode points -> KNN returns the nearest -> remove.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(spatial): point PHP config at SPATIAL_KNN_URL; integration-test closestPostcode

- config/freegle.php: spatial_server_url now reads SPATIAL_KNN_URL (the canonical
  finder var that batch already sets); SPATIAL_SERVER_URL is taken by the routing
  server, so relying on it resolved to localhost on batch and (with the fallback
  removed) would have broken closestPostcode/PostcodeRemapService in productio... (continued)

Coverage Stats

10980 of 14579 branches covered (75.31%)

Branch coverage included in aggregate %.

60 of 67 new or added lines in 7 files covered. (89.55%)

42 existing lines in 6 files now uncovered.

118917 of 168850 relevant lines covered (70.43%)

36.29 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

72.73

/iznik-batch/app/Services/SpatialQueryService.php

<?php

namespace App\Services;

use Illuminate\Support\Facades\Http;
use Illuminate\Support\Facades\Log;

/**
 * Read-side client for the iznik-spatial-go "finder" service.
 *
 * Replaces the per-table expanding-bounding-box MySQL nearest-neighbour loops
 * with a single call to the spatial server's R-tree KNN, which is exact and
 * needs one round-trip instead of up to a dozen MySQL queries.
 *
 * @see SpatialAdminService for the write/remove side.
 */
class SpatialQueryService
{
    private string $url;

    public function __construct()
    {
        $this->url = rtrim(config('freegle.spatial_server_url', 'http://localhost:8194'), '/');
    }

    /**
     * Return the external IDs of the nearest records in $dataset to (lat, lng),
     * nearest first, up to $limit. Returns [] if the spatial server errors or is
     * unreachable (logged) — callers treat that as "nothing nearby".
     *
     * @return int[]
     */
    public function nearestIds(string $dataset, float $lat, float $lng, int $limit = 1, ?string $type = null): array
    {
        $params = ['lat' => $lat, 'lng' => $lng, 'limit' => $limit];
        if ($type !== null) {
            $params['type'] = $type;
        }

        try {
            $response = Http::timeout(5)->get("{$this->url}/v1/{$dataset}/knn", $params);

            if ($response->successful()) {
                return array_map(
                    static fn ($r) => (int) $r['id'],
                    $response->json('results', []) ?? []
                );
            }

            Log::warning("SpatialQuery: {$dataset} knn HTTP {$response->status()}", [
                'lat' => $lat,
                'lng' => $lng,
            ]);
        } catch (\Throwable $e) {
            Log::warning("SpatialQuery: {$dataset} knn failed: {$e->getMessage()}", [
                'lat' => $lat,
                'lng' => $lng,
            ]);
        }

        return [];
    }
}

1	<?php
2
3	namespace App\Services;
4
5	use Illuminate\Support\Facades\Http;
6	use Illuminate\Support\Facades\Log;
7
8	/**
9	* Read-side client for the iznik-spatial-go "finder" service.
10	*
11	* Replaces the per-table expanding-bounding-box MySQL nearest-neighbour loops
12	* with a single call to the spatial server's R-tree KNN, which is exact and
13	* needs one round-trip instead of up to a dozen MySQL queries.
14	*
15	* @see SpatialAdminService for the write/remove side.
16	*/
17	class SpatialQueryService
18	{
19	private string $url;
20
21	public function __construct()	32✔
22	{
23	$this->url = rtrim(config('freegle.spatial_server_url', 'http://localhost:8194'), '/');	32✔
24	}
25
26	/**
27	* Return the external IDs of the nearest records in $dataset to (lat, lng),
28	* nearest first, up to $limit. Returns [] if the spatial server errors or is
29	* unreachable (logged) — callers treat that as "nothing nearby".
30	*
31	* @return int[]
32	*/
33	public function nearestIds(string $dataset, float $lat, float $lng, int $limit = 1, ?string $type = null): array	32✔
34	{
35	$params = ['lat' => $lat, 'lng' => $lng, 'limit' => $limit];	32✔
36	if ($type !== null) {	32✔
NEW 37	$params['type'] = $type;	×
38	}
39
40	try {
41	$response = Http::timeout(5)->get("{$this->url}/v1/{$dataset}/knn", $params);	32✔
42
43	if ($response->successful()) {	32✔
44	return array_map(	30✔
45	static fn ($r) => (int) $r['id'],	30✔
46	$response->json('results', []) ?? []	30✔
47	);	30✔
48	}
49
50	Log::warning("SpatialQuery: {$dataset} knn HTTP {$response->status()}", [	2✔
51	'lat' => $lat,	2✔
52	'lng' => $lng,	2✔
53	]);	2✔
NEW 54	} catch (\Throwable $e) {	×
NEW 55	Log::warning("SpatialQuery: {$dataset} knn failed: {$e->getMessage()}", [	×
NEW 56	'lat' => $lat,	×
NEW 57	'lng' => $lng,	×
NEW 58	]);	×
59	}
60
61	return [];	2✔
62	}
63	}

Freegle / Iznik / 21571

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous