A lightning-fast search API that fits effortlessly into your apps, websites, and workflow
Meilisearch v1.9 includes performance improvements for hybrid search and the addition/updating of settings. This version benefits from multiple requested features, such as the new frequency matching strategy and the ability to retrieve similar documents.
🧰 All official Meilisearch integrations (including SDKs, clients, and other tools) are compatible with this Meilisearch release. Integration deployment happens between 4 to 48 hours after a new version becomes available.
Some SDKs might not include all new features. Consult the project repository for detailed information. Is a feature you need missing from your chosen SDK? Create an issue letting us know you need it, or, for open-source karma points, open a PR implementing it (we'll love you for that ❤️).
This release introduces multiple hybrid search updates.
Done by @dureuill and @irevoire in #4633 and #4649
_vectors.embedder arraysEmpty _vectors.embedder arrays are now interpreted as having no vector embedding.
Before v1.9, Meilisearch interpreted these as a single embedding of dimension 0. This change follows user feedback that the previous behavior was unexpected and unhelpful.
_vectors field no longer present in search resultsWhen the experimental vectorStore feature is enabled, Meilisearch no longer includes _vectors in returned search results by default. This will considerably improve performance.
Use the new retrieveVectors search parameter to display the _vectors field:
curl \
-X POST 'http://localhost:7700/indexes/INDEX_NAME/search' \
-H 'Content-Type: application/json' \
--data-binary '{
"q": "SEARCH QUERY",
"retrieveVectors": true
}'
_vectorsIn order to save storage and run faster, Meilisearch is no longer storing your vector "as-is". Meilisearch now returns the float in a canonicalized representation rather than the user-provided representation.
For example, 3 may be represented as 3.0
_vectors accepts object valuesThe document _vectors field now accepts objects in addition to embedding arrays:
{
"id": 42,
"_vectors": {
"default": [0.1, 0.2 ],
"text": {
"embeddings": [[0.1, 0.2, 0.3], [0.4, 0.5, 0.6]],
"regenerate": false
},
"translation": {
"embeddings": [0.1, 0.2, 0.3, 0.4],
"regenerate": true
}
}
}
The _vectors object may contain two fields: embeddings and regenerate.
If present, embeddings will replace this document's embeddings.
regenerate must be either true or false. If regenerate: true, Meilisearch will overwrite the document embeddings each time the document is updated in the future. If regenerate: false, Meilisearch will keep the last provided or generated embeddings even if the document is updated in the future.
This change allows importing embeddings to autoembedders as a one-shot process, by setting them as regenerate: true. This change also ensures embeddings are not regenerated when importing a dump created with Meilisearch v1.9.
Meilisearch v1.9.0 also improves performance when indexing and using hybrid search, avoiding useless operations and optimizing the important ones.
Use rankingScoreThreshold to exclude search results with low ranking scores:
curl \
-X POST 'http://localhost:7700/indexes/movies/search' \
-H 'Content-Type: application/json' \
--data-binary '{
"q": "Badman dark returns 1",
"showRankingScore": true,
"limit": 5,
"rankingScoreThreshold": 0.2
}'
Meilisearch does not return any documents below the configured threshold. Excluded results do not count towards estimatedTotalHits, totalHits, and facet distribution.
⚠️ For performance reasons, if the number of documents above rankingScoreThreshold is higher than limit, Meilisearch does not evaluate the ranking score of the remaining documents. Results ranking below the threshold are not immediately removed from the set of candidates. In this case, Meilisearch may overestimate the count of estimatedTotalHits, totalHits and facet distribution.
Done by @dureuill in #4666
This release introduces a new AI-powered search feature allowing you to send a document to Meilisearch and receive a list of similar documents in return.
Use the /indexes/{indexUid}/similar endpoint to query Meilisearch for related documents:
curl \
-X POST /indexes/:indexUid/similar
-H 'Content-Type: application/json' \
--data-binary '{
"id": "23",
"offset": 0,
"limit": 2,
"filter": "release_date > 1521763199",
"embedder": "default",
"attributesToRetrieve": [],
"showRankingScore": false,
"showRankingScoreDetails": false
}'
id: string indicating the document needing similar results, requiredoffset: number of results to skip when paginating, optional, defaults to 0
limit: number of results to display, optional, defaults to 20
filter: string with a filter expression Meilisearch should apply to the results, optional, defaults to null
embedder: string indicating the embedder Meilisearch should use to retrieve similar documents, optional, defaults to "default"
attributesToRetrieve: array of strings indicating which fields Meilisearch will include in the response, optional, defaults to ["*"]
showRankingScore: boolean indicating if results should include ranking score information, optional, defaults to false
showRankingScoreDetails: boolean indicating if results should include detailed ranking score information, optional, defaults to false
rankingScoreThreshold: Excludes search results with a ranking score lower than the defined number, optional, defaults to null./indexes/{indexUid}/similar supports GET and POST routes. Use URL query parameters to configure your GET request, or include your parameters in the request body if using the POST route. Both offer identical functionality.
Done by @dureuill in #4647
frequency matching strategyThis release adds a new matching strategy, frequency. Use it to prioritize results containing the least frequent query terms:
curl \
-X POST 'http://localhost:7700/indexes/{index_uid}/search' \
-H 'Content-Type: application/json' \
--data-binary '{
"q": "cheval blanc",
"matchingStrategy": "frequency"
}'
Done by @ManyTheFish in #4667
distinctAttribute at search timeThis release introduces a new search parameter: distinct which you can use to specify the distinct attribute at search time:
curl \
-X POST 'http://localhost:7700/indexes/{index_uid}/search' \
-H 'Content-Type: application/json' \
--data-binary '{
"q": "kefir le double poney",
"distinct": "book.isbn"
}'
If a distinct attribute is already defined in the settings it'll be ignored in favor of the one defined at search time.
Done by @Kerollmops in #4693
Meilisearch now limits operations when importing settings by avoiding unnecessary writing operations in its internal database and reducing disk usage.
Additionally, when changing embedding settings, Meilisearch will now only regenerate the embeddings for the embedders whose settings have been modified, instead of for all embedders. When only the documentTemplate is modified, embeddings will only be regenerated for documents where this modification leads to a different text to embed.
Done by @irevoire, @Kerollmops, @ManyTheFish and @dureuill in #4646, #4680, #4631 and #4649
chinese-normalization-pinyin feature flag compilationexportPuffinReport experimental feature. Use logs routes and logs modes instead (#4655) @KerollmopssearchableAttributes: ["*"]. Consult the GitHub issue for a detailed breakdown of these changes (#4631) @irevoiresearchableAttributes behavior when handling nested fields. Consult the GitHub issue for more information (#4631) @irevoiresource of an embedder. This prevents misleading error messages when configuring the embedders (#4649) @dureuilloffset and limit parameters when returning keyword results early (#4746) @dureuill❤️ Thanks again to our external contributors:
⚠️ Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.
embeddings optional and improve error message for regenerate by @irevoire in https://github.com/meilisearch/meilisearch/pull/4740
:heart: Thanks to @inventor123 for first reporting https://github.com/meilisearch/meilisearch/issues/4745
⚠️ Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.
:heart: Thanks to @sam-ulrich1 for reporting the panic in hybrid search in https://github.com/meilisearch/meilisearch/issues/4588
:heart: Thanks @irevoire for reproducing the issue and checking fixes
:heart: Thanks to Tater, Hannsr, Martin from the Discord thread, and @Doutatsu on GH for the report and helping with the investigation
⚠️ Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.
_vectors field is not returned anymore when retrieving documents; you must use the retrieveVector parameter instead_vectors field with the retrieveVector parameter, their embeddings are not returned "as-is"; they'll always be returned with the maximum precisionuserProvided field has been removed in favor of a new regenerate field that better represents your intent. When set to true it means the embeddings will be regenerated on every change to the document (default behavior). If set to false the embeddings will never be updated by the engine.Full Changelog: https://github.com/meilisearch/meilisearch/compare/v1.9.0-rc.2...v1.9.0-rc.3
Thanks to @savikko for first reporting the issue :heart:
⚠️ Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.
Meilisearch v1.9 includes performance improvements for hybrid search and the addition/updating of settings. This version benefits from multiple requested features, such as the new frequency matching strategy and the ability to retrieve similar documents.
When adding new fields in the searchableAttributes setting, the engine will only index the additional attributes instead of recomputing all the searchable attributes.
The words containing œ or æ will be retrieved using oe or ae, like Daemon <=> Dæmon.
Fix: Test CI failing when enabling/disabling some features #4629
⚠️ Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.
Meilisearch v1.9 includes performance improvements for hybrid search and the addition/updating of settings. This version benefits from multiple requested features, such as the new frequency matching strategy and the ability to retrieve similar documents.
To filter returned documents by their ranking score, a new rankingScoreThreshold parameter has been added to the search and similar routes.
When a rankingScoreThreshold is provided, the results of the search/similar request are modified in the following way:
_rankingScore is under the rankingScoreThreshold is returnedestimatedTotalHits, totalHits and the facet distribution.request without score threshold:
POST /indexes/movies/search
{
"q": "Badman dark returns 1",
"showRankingScore": true,
"limit": 5
}
results:
{
"hits": [
{
"title": "Batman the dark knight returns: Part 1",
"id": "A",
"_rankingScore": 0.93430081300813
},
{
"title": "Batman the dark knight returns: Part 2",
"id": "B",
"_rankingScore": 0.6685627880184332
},
{
"title": "Badman",
"id": "E",
"_rankingScore": 0.25
},
{
"title": "Batman Returns",
"id": "C",
"_rankingScore": 0.11553030303030302
},
{
"title": "Batman",
"id": "D",
"_rankingScore": 0.11553030303030302
}
],
"query": "Badman dark returns 1",
"processingTimeMs": 11,
"limit": 5,
"offset": 0,
"estimatedTotalHits": 62
}
request with score threshold:
POST /indexes/movies/search
{
"q": "Badman dark returns 1",
"showRankingScore": true,
"limit": 5
"rankingScoreThreshold": 0.2
}
results:
{
"hits": [
{
"title": "Batman the dark knight returns: Part 1",
"id": "A",
"_rankingScore": 0.93430081300813
},
{
"title": "Batman the dark knight returns: Part 2",
"id": "B",
"_rankingScore": 0.6685627880184332
},
{
"title": "Badman",
"id": "E",
"_rankingScore": 0.25
}
],
"query": "Badman dark returns 1",
"processingTimeMs": 11,
"limit": 5,
"offset": 0,
"estimatedTotalHits": 3
}
⚠️ For performance reasons, if Meilisearch finds limit hits above the rankingScoreThreshold, then the ranking score of the remaining documents is not evaluated, and so they are not removed from the set of candidates, even if their ranking score would be below the threshold.
As a result, in this configuration the estimatedTotalHits, totalHits and the facet distribution may be overapproximation of their values.
Done by @dureuill in https://github.com/meilisearch/meilisearch/pull/4666
See also the changelog for v1.9.0-rc.0
⚠️ Since this is a release candidate (RC), we do NOT recommend using it in a production environment. Is something not working as expected? We welcome bug reports and feedback about new features.
Meilisearch v1.9 includes performance improvements for hybrid search and the addition/updating of settings. This version benefits from multiple requested features, such as the new frequency matching strategy and the ability to retrieve similar documents.
Since we're focusing on AI innovation, this version introduces multiple improvements and changes related to hybrid search. More detailed changelog here.
Done by @dureuill and @irevoire in #4633 and #4649
_vectors.embedder used to be interpreted as a single embedding of dimension 0 when specifying embeddings in documents. In v1.9 it is now interpreted as 0 embedding. The previous behavior was surprising and not useful.Meilisearch v1.9.0 improves performance when indexing and using hybrid search, avoiding useless operations and optimizing the important ones.
To retrieve similar documents in your datasets, two new routes have been introduced
POST /indexes/:indexUid/similar using parameters in the request body.GET /indexes/:indexUid/similar, using query URL parameters.POST /indexes/:indexUid/similar
{
// Mandatory: the external id of the target document
"id": "23",
// Optional, defaults to 0: how many results to skip
"offset": 0,
// Optional, defaults to 20: how many results to display
"limit": 2,
// Optional, defaults to null: an additional filter for the returned documents
"filter": "release_date > 1521763199",
// Optional, defaults to the default embedder: name of the embedder to use
// for computing recommendations.
"embedder": "default",
// Optional, defaults to null: same as the search query parameter of the same name
"attributesToRetrieve": [],
// Both optional, defaults to false: allow displaying the ranking score
// (resp. detailed ranking score)
"showRankingScore": false,
"showRankingScoreDetails": false
}
Done by @dureuill in #4647
frequency matching strategy when searchingA frequency variant to the matchingStrategy search parameter has been added. This favors the least frequent query words when retrieving the documents.
curl \
-X POST 'http://localhost:7700/indexes/movies/search' \
-H 'Content-Type: application/json' \
--data-binary '{
"q": "chaval blanc",
"matchingStrategy": "frequency"
}'
Previous existing values for matchingStrategy are last and all (last is the default value).
Done by @ManyTheFish in #4667
Meilisearch now limits operations when importing settings by avoiding useless writing operations in its internal database and by reducing disk usage.
Done by @irevoire and @Kerollmops in #4646, #4656 and #4631
exportPuffinReport experimental feature. Use logs routes and logs modes instead (#4655) @KerollmopssearchableAttributes behavior with nested fields when they were not explicitly defined. More information here (#4631) @irevoire❤️ Thanks again to our external contributors: