14998737191

Committed 13 May 2025 02:05PM UTC coverage: 69.071% (-0.09%) from 69.16%

Build # 14998737191

Build Type

push

github

Committed by

web-flow

Commit Message

[BUGFIX] Fix indexing of cached pages when "disableFrontendIndexing" … (#1142)

* [BUGFIX] Fix indexing of cached pages when "disableFrontendIndexing" is active

When pages are cached in TYPO3v12, the indexing process is not
triggered when crawler fetches the pages.
This becomes a problem when the "Disable Indexing in Frontend"
(basic.disableFrontendIndexing) indexed_search flag is enabled,
because no indexing is happening then.

The solution is to tell TYPO3 to not use the cache when the page
is requested by the crawler process.

See the ShouldUseCachedPageDataIfAvailableEvent documentation at
https://docs.typo3.org/c/typo3/cms-core/main/en-us/Changelog/12.0/Feature-96968-PSR-14EventForAvoidLoadingFrontendPagesFromCache.html

Resolves: https://github.com/tomasnorre/crawler/issues/1029

* [BUGFIX] Remove unneeded initialization of queueRepository

phpStan complained:
> Property AOE\Crawler\Domain\Model\Process::$queueRepository on left
> side of ??= is not nullable nor uninitialized.
>  nullCoalesce.initializedProperty

The variable gets initialized in the constructor already.

Run Details

0 of 3 new or added lines in 1 file covered. (0.0%)

1867 of 2703 relevant lines covered (69.07%)

3.29 hits per line

Source File
Press 'n' to go to next uncovered line, 'b' for previous

0.0

/Classes/EventListener/ShouldUseCachedPageDataIfAvailableEventListener.php

<?php

declare(strict_types=1);

namespace AOE\Crawler\EventListener;

use TYPO3\CMS\Frontend\Event\ShouldUseCachedPageDataIfAvailableEvent;

/**
 * Disable frontend cache when the HTTP request has crawler parameters,
 * so that page indexing hooks are triggered.
 */
class ShouldUseCachedPageDataIfAvailableEventListener
{
    public function __invoke(ShouldUseCachedPageDataIfAvailableEvent $event): void
    {
        if ($event->getRequest()->getAttribute('tx_crawler') === null) {
            return;
        }
        $event->setShouldUseCachedPageData(false);
    }
}

1	<?php
2
3	declare(strict_types=1);
4
5	namespace AOE\Crawler\EventListener;
6
7	use TYPO3\CMS\Frontend\Event\ShouldUseCachedPageDataIfAvailableEvent;
8
9	/**
10	* Disable frontend cache when the HTTP request has crawler parameters,
11	* so that page indexing hooks are triggered.
12	*/
13	class ShouldUseCachedPageDataIfAvailableEventListener
14	{
15	public function __invoke(ShouldUseCachedPageDataIfAvailableEvent $event): void
16	{
NEW 17	if ($event->getRequest()->getAttribute('tx_crawler') === null) {	×
NEW 18	return;	×
19	}
NEW 20	$event->setShouldUseCachedPageData(false);	×
21	}
22	}

tomasnorre / crawler / 14998737191

Source File Press 'n' to go to next uncovered line, 'b' for previous

Source File
Press 'n' to go to next uncovered line, 'b' for previous