feat: not equal range filter

[divine]

Q	A
Branch?	main for features
Tickets	#4546
License	MIT
Doc PR	api-platform/docs#... (awaiting initial review)

This is a follow-up for #4546 with the added test.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[divine]

Not stale...

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[divine]

Not stale....

[nikophil]

LGTM 👌

[nikophil]

@divine actually I'm wondering if RangeFilter is the right filter to modify: it would feel weird to configure a range filter on a string property, for instance 🤔

[nesl247]

@divine actually I'm wondering if RangeFilter is the right filter to modify: it would feel weird to configure a range filter on a string property, for instance 🤔

Agreed. This belongs to the SearchFilter IMO. I didn't test this out, but

/*
 * This file is part of the API Platform project.
 *
 * (c) Kévin Dunglas <dunglas@gmail.com>
 *
 * For the full copyright and license information, please view the LICENSE
 * file that was distributed with this source code.
 */

declare(strict_types=1);

namespace ApiPlatform\Doctrine\Orm\Filter;

use ApiPlatform\Api\IdentifiersExtractorInterface;
use ApiPlatform\Api\IriConverterInterface;
use ApiPlatform\Doctrine\Common\Filter\SearchFilterInterface;
use ApiPlatform\Doctrine\Common\Filter\SearchFilterTrait;
use ApiPlatform\Doctrine\Orm\Util\QueryBuilderHelper;
use ApiPlatform\Doctrine\Orm\Util\QueryNameGeneratorInterface;
use ApiPlatform\Exception\InvalidArgumentException;
use ApiPlatform\Metadata\Operation;
use Doctrine\DBAL\Types\Types;
use Doctrine\ORM\Query\Expr\Join;
use Doctrine\ORM\QueryBuilder;
use Doctrine\Persistence\ManagerRegistry;
use Psr\Log\LoggerInterface;
use Symfony\Component\PropertyAccess\PropertyAccess;
use Symfony\Component\PropertyAccess\PropertyAccessorInterface;
use Symfony\Component\Serializer\NameConverter\NameConverterInterface;

/**
 * The search filter allows to filter a collection by given properties.
 *
 * The search filter supports `exact`, `partial`, `start`, `end`, and `word_start` matching strategies:
 * - `exact` strategy searches for fields that exactly match the value
 * - `partial` strategy uses `LIKE %value%` to search for fields that contain the value
 * - `start` strategy uses `LIKE value%` to search for fields that start with the value
 * - `end` strategy uses `LIKE %value` to search for fields that end with the value
 * - `word_start` strategy uses `LIKE value% OR LIKE % value%` to search for fields that contain words starting with the value
 *
 * Note: it is possible to filter on properties and relations too.
 *
 * Prepend the letter `i` to the filter if you want it to be case-insensitive. For example `ipartial` or `iexact`.
 * Note that this will use the `LOWER` function and *will* impact performance if there is no proper index.
 *
 * Case insensitivity may already be enforced at the database level depending on the [collation](https://en.wikipedia.org/wiki/Collation) used.
 * If you are using MySQL, note that the commonly used `utf8_unicode_ci` collation (and its sibling `utf8mb4_unicode_ci`)
 * are already case-insensitive, as indicated by the `_ci` part in their names.
 *
 * Note: Search filters with the `exact` strategy can have multiple values for the same property (in this case the
 * condition will be similar to a SQL IN clause).
 *
 * Syntax: `?property[]=foo&property[]=bar`.
 *
 * <CodeSelector>
 * ```php
 * <?php
 * // api/src/Entity/Book.php
 * use ApiPlatform\Metadata\ApiFilter;
 * use ApiPlatform\Metadata\ApiResource;
 * use ApiPlatform\Doctrine\Orm\Filter\SearchFilter;
 *
 * #[ApiResource]
 * #[ApiFilter(SearchFilter::class, properties: ['isbn' => 'exact', 'description' => 'partial'])]
 * class Book
 * {
 *     // ...
 * }
 * ```
 *
 * ```yaml
 * # config/services.yaml
 * services:
 *     book.search_filter:
 *         parent: 'api_platform.doctrine.orm.search_filter'
 *         arguments: [ { isbn: 'exact', description: 'partial' } ]
 *         tags:  [ 'api_platform.filter' ]
 *         # The following are mandatory only if a _defaults section is defined with inverted values.
 *         # You may want to isolate filters in a dedicated file to avoid adding the following lines (by adding them in the defaults section)
 *         autowire: false
 *         autoconfigure: false
 *         public: false
 *
 * # api/config/api_platform/resources.yaml
 * resources:
 *     App\Entity\Book:
 *         - operations:
 *               ApiPlatform\Metadata\GetCollection:
 *                   filters: ['book.search_filter']
 * ```
 *
 * ```xml
 * <?xml version="1.0" encoding="UTF-8" ?>
 * <!-- api/config/services.xml -->
 * <?xml version="1.0" encoding="UTF-8" ?>
 * <container
 *         xmlns="http://symfony.com/schema/dic/services"
 *         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 *         xsi:schemaLocation="http://symfony.com/schema/dic/services
 *         https://symfony.com/schema/dic/services/services-1.0.xsd">
 *     <services>
 *         <service id="book.search_filter" parent="api_platform.doctrine.orm.search_filter">
 *             <argument type="collection">
 *                 <argument key="isbn">exact</argument>
 *                 <argument key="description">partial</argument>
 *             </argument>
 *             <tag name="api_platform.filter"/>
 *         </service>
 *     </services>
 * </container>
 * <!-- api/config/api_platform/resources.xml -->
 * <resources
 *         xmlns="https://api-platform.com/schema/metadata/resources-3.0"
 *         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 *         xsi:schemaLocation="https://api-platform.com/schema/metadata/resources-3.0
 *         https://api-platform.com/schema/metadata/resources-3.0.xsd">
 *     <resource class="App\Entity\Book">
 *         <operations>
 *             <operation class="ApiPlatform\Metadata\GetCollection">
 *                 <filters>
 *                     <filter>book.search_filter</filter>
 *                 </filters>
 *             </operation>
 *         </operations>
 *     </resource>
 * </resources>
 * ```
 * </CodeSelector>
 *
 * @author Kévin Dunglas <dunglas@gmail.com>
 */
final class SearchFilter extends AbstractFilter implements SearchFilterInterface
{
    use SearchFilterTrait;

    public const DOCTRINE_INTEGER_TYPE = Types::INTEGER;

    public function __construct(ManagerRegistry $managerRegistry, IriConverterInterface $iriConverter, PropertyAccessorInterface $propertyAccessor = null, LoggerInterface $logger = null, array $properties = null, IdentifiersExtractorInterface $identifiersExtractor = null, NameConverterInterface $nameConverter = null)
    {
        parent::__construct($managerRegistry, $logger, $properties, $nameConverter);

        $this->iriConverter = $iriConverter;
        $this->identifiersExtractor = $identifiersExtractor;
        $this->propertyAccessor = $propertyAccessor ?: PropertyAccess::createPropertyAccessor();
    }

    protected function getIriConverter(): IriConverterInterface
    {
        return $this->iriConverter;
    }

    protected function getPropertyAccessor(): PropertyAccessorInterface
    {
        return $this->propertyAccessor;
    }

    /**
     * {@inheritdoc}
     */
    protected function filterProperty(string $property, $value, QueryBuilder $queryBuilder, QueryNameGeneratorInterface $queryNameGenerator, string $resourceClass, Operation $operation = null, array $context = []): void
    {
        $notEquals = false;

        if (str_ends_with($property, '!')) {
            $property = str_replace('!', '', $property);
            $notEquals = true;
        }

        if (
            null === $value
            || !$this->isPropertyEnabled($property, $resourceClass)
            || !$this->isPropertyMapped($property, $resourceClass, true)
        ) {
            return;
        }

        $alias = $queryBuilder->getRootAliases()[0];
        $field = $property;

        $values = $this->normalizeValues((array) $value, $property);
        if (null === $values) {
            return;
        }

        $associations = [];
        if ($this->isPropertyNested($property, $resourceClass)) {
            [$alias, $field, $associations] = $this->addJoinsForNestedProperty($property, $alias, $queryBuilder, $queryNameGenerator, $resourceClass, Join::INNER_JOIN);
        }

        $caseSensitive = true;
        $strategy = $this->properties[$property] ?? self::STRATEGY_EXACT;

        // prefixing the strategy with i makes it case insensitive
        if (str_starts_with($strategy, 'i')) {
            $strategy = substr($strategy, 1);
            $caseSensitive = false;
        }

        $metadata = $this->getNestedMetadata($resourceClass, $associations);

        if ($metadata->hasField($field)) {
            if ('id' === $field) {
                $values = array_map($this->getIdFromValue(...), $values);
            }

            if (!$this->hasValidValues($values, $this->getDoctrineFieldType($property, $resourceClass))) {
                $this->logger->notice('Invalid filter ignored', [
                    'exception' => new InvalidArgumentException(sprintf('Values for field "%s" are not valid according to the doctrine type.', $field)),
                ]);

                return;
            }

            $this->addWhereByStrategy($strategy, $queryBuilder, $queryNameGenerator, $alias, $field, $values, $caseSensitive);

            return;
        }

        // metadata doesn't have the field, nor an association on the field
        if (!$metadata->hasAssociation($field)) {
            return;
        }

        $values = array_map($this->getIdFromValue(...), $values);

        $associationResourceClass = $metadata->getAssociationTargetClass($field);
        $associationFieldIdentifier = $metadata->getIdentifierFieldNames()[0];
        $doctrineTypeField = $this->getDoctrineFieldType($associationFieldIdentifier, $associationResourceClass);

        if (!$this->hasValidValues($values, $doctrineTypeField)) {
            $this->logger->notice('Invalid filter ignored', [
                'exception' => new InvalidArgumentException(sprintf('Values for field "%s" are not valid according to the doctrine type.', $field)),
            ]);

            return;
        }

        $associationAlias = $alias;
        $associationField = $field;
        if ($metadata->isCollectionValuedAssociation($associationField) || $metadata->isAssociationInverseSide($field)) {
            $associationAlias = QueryBuilderHelper::addJoinOnce($queryBuilder, $queryNameGenerator, $alias, $associationField);
            $associationField = $associationFieldIdentifier;
        }

        $this->addWhereByStrategy($strategy, $queryBuilder, $queryNameGenerator, $associationAlias, $associationField, $values, $caseSensitive, $notEquals);
    }

    /**
     * Adds where clause according to the strategy.
     *
     * @throws InvalidArgumentException If strategy does not exist
     */
    protected function addWhereByStrategy(string $strategy, QueryBuilder $queryBuilder, QueryNameGeneratorInterface $queryNameGenerator, string $alias, string $field, mixed $values, bool $caseSensitive, bool $notEquals): void
    {
        if (!\is_array($values)) {
            $values = [$values];
        }

        $wrapCase = $this->createWrapCase($caseSensitive);
        $valueParameter = ':'.$queryNameGenerator->generateParameterName($field);
        $aliasedField = sprintf('%s.%s', $alias, $field);

        if (!$strategy || self::STRATEGY_EXACT === $strategy) {
            if (!$notEquals && 1 === \count($values)) {
                $queryBuilder
                    ->andWhere($queryBuilder->expr()->eq($wrapCase($aliasedField), $wrapCase($valueParameter)))
                    ->setParameter($valueParameter, $values[0]);

                return;
            }

            if ($notEquals && 1 === \count($values)) {
                $queryBuilder
                    ->andWhere($queryBuilder->expr()->neq($wrapCase($aliasedField), $wrapCase($valueParameter)))
                    ->setParameter($valueParameter, $values[0]);

                return;
            }

            if ($notEquals) {
                $queryBuilder
                    ->andWhere($queryBuilder->expr()->notIn($wrapCase($aliasedField), $valueParameter))
                    ->setParameter($valueParameter, $caseSensitive ? $values : array_map('strtolower', $values));
            }

            return;
        }

        $ors = [];
        $parameters = [];
        foreach ($values as $key => $value) {
            $keyValueParameter = sprintf('%s_%s', $valueParameter, $key);
            $parameters[] = [$caseSensitive ? $value : strtolower($value), $keyValueParameter];

            $ors[] = match ($strategy) {
                self::STRATEGY_PARTIAL => match($notEquals) {
                    false => $queryBuilder->expr()->like(
                        $wrapCase($aliasedField),
                        $wrapCase((string) $queryBuilder->expr()->concat("'%'", $keyValueParameter, "'%'"))
                    ),
                    true => $queryBuilder->expr()->notLike(
                        $wrapCase($aliasedField),
                        $wrapCase((string) $queryBuilder->expr()->concat("'%'", $keyValueParameter, "'%'"))
                    )},
                    self::STRATEGY_START => match($notEquals) {
                        false => $queryBuilder->expr()->like(
                            $wrapCase($aliasedField),
                            $wrapCase((string) $queryBuilder->expr()->concat($keyValueParameter, "'%'"))
                        ),
                    true => $queryBuilder->expr()->notLike(
                        $wrapCase($aliasedField),
                        $wrapCase((string) $queryBuilder->expr()->concat($keyValueParameter, "'%'"))
                    )},
                self::STRATEGY_END => match($notEquals) {
                    false => $queryBuilder->expr()->like(
                        $wrapCase($aliasedField),
                        $wrapCase((string) $queryBuilder->expr()->concat("'%'", $keyValueParameter))
                    ),
                    true => $queryBuilder->expr()->notLike(
                        $wrapCase($aliasedField),
                        $wrapCase((string) $queryBuilder->expr()->concat("'%'", $keyValueParameter))
                    )
                },
                self::STRATEGY_WORD_START => match($notEquals) {
                    false => $queryBuilder->expr()->orX(
                        $queryBuilder->expr()->like(
                            $wrapCase($aliasedField),
                            $wrapCase((string) $queryBuilder->expr()->concat($keyValueParameter, "'%'"))
                        ),
                        $queryBuilder->expr()->like(
                            $wrapCase($aliasedField),
                            $wrapCase((string) $queryBuilder->expr()->concat("'% '", $keyValueParameter, "'%'"))
                        )
                    ),
                    true => $queryBuilder->expr()->orX(
                        $queryBuilder->expr()->notLike(
                            $wrapCase($aliasedField),
                            $wrapCase((string) $queryBuilder->expr()->concat($keyValueParameter, "'%'"))
                        ),
                        $queryBuilder->expr()->notLike(
                            $wrapCase($aliasedField),
                            $wrapCase((string) $queryBuilder->expr()->concat("'% '", $keyValueParameter, "'%'"))
                        )
                    ),
                },
                default => throw new InvalidArgumentException(sprintf('strategy %s does not exist.', $strategy)),
            };
        }

        $queryBuilder->andWhere($queryBuilder->expr()->orX(...$ors));
        foreach ($parameters as $parameter) {
            $queryBuilder->setParameter($parameter[1], $parameter[0]);
        }
    }

    /**
     * Creates a function that will wrap a Doctrine expression according to the
     * specified case sensitivity.
     *
     * For example, "o.name" will get wrapped into "LOWER(o.name)" when $caseSensitive
     * is false.
     */
    protected function createWrapCase(bool $caseSensitive): \Closure
    {
        return static function (string $expr) use ($caseSensitive): string {
            if ($caseSensitive) {
                return $expr;
            }

            return sprintf('LOWER(%s)', $expr);
        };
    }

    /**
     * {@inheritdoc}
     */
    protected function getType(string $doctrineType): string
    {
        return match ($doctrineType) {
            Types::ARRAY => 'array',
            Types::BIGINT, Types::INTEGER, Types::SMALLINT => 'int',
            Types::BOOLEAN => 'bool',
            Types::DATE_MUTABLE, Types::TIME_MUTABLE, Types::DATETIME_MUTABLE, Types::DATETIMETZ_MUTABLE, Types::DATE_IMMUTABLE, Types::TIME_IMMUTABLE, Types::DATETIME_IMMUTABLE, Types::DATETIMETZ_IMMUTABLE => \DateTimeInterface::class,
            Types::FLOAT => 'float',
            default => 'string',
        };
    }
}

should probably work. I ported the changes we did for our own explicit NotEquals filter which is a copy of the SearchFilter.

[divine]

Hello,

I can't entirely agree if this should be in SearchFilterInterface as in RangefilterInterface as we have this parameter names:

core/src/Doctrine/Common/Filter/RangeFilterInterface.php

Lines 22 to 29 in d85884d

	interface RangeFilterInterface
	{
	public const PARAMETER_BETWEEN = 'between';
	public const PARAMETER_GREATER_THAN = 'gt';
	public const PARAMETER_GREATER_THAN_OR_EQUAL = 'gte';
	public const PARAMETER_LESS_THAN = 'lt';
	public const PARAMETER_LESS_THAN_OR_EQUAL = 'lte';
	}

"ne" is better sits in Rangefilter IMO even though it doesn't look right until you see the parameter names in both files.

core/src/Doctrine/Common/Filter/SearchFilterInterface.php

Lines 22 to 48 in d85884d

	interface SearchFilterInterface
	{
	/**
	* @var string Exact matching
	*/
	public const STRATEGY_EXACT = 'exact';
	/**
	* @var string The value must be contained in the field
	*/
	public const STRATEGY_PARTIAL = 'partial';
	/**
	* @var string Finds fields that are starting with the value
	*/
	public const STRATEGY_START = 'start';
	/**
	* @var string Finds fields that are ending with the value
	*/
	public const STRATEGY_END = 'end';
	/**
	* @var string Finds fields that are starting with the word
	*/
	public const STRATEGY_WORD_START = 'word_start';
	}

I'm not the person to decide so let's give a voice from @api-platform/core team.

Thanks!

[VincentLanglet]

@divine actually I'm wondering if RangeFilter is the right filter to modify: it would feel weird to configure a range filter on a string property, for instance 🤔

To me this using it this way would be a hacky undocumented feature with no BC promise.

At first I thought that the NotEqual type didn't make sens in a range filter but that could be considered as a range ]-INF, value[ U ] value, +INF[. A similar feature would be a "not between" param.

Are you still interested by this PR @divine ? If yes could you solve the conflict ?

Thanks

[soyuka]

@VincentLanglet this is part of the work around the range refactoring #4689

[VincentLanglet]

@VincentLanglet this is part of the work around the range refactoring #4689

I'm not sure to understand ; the #4689 PR is talking about UuidRangeFilter, this does not seems incompatible with the fact to add a "Not equal range" filter. Which seems a easier PR/step to finish than the big refacto of #4689.