Lucene Versions Save

Apache Lucene open-source search software

1 week ago

Bug Fixes

Fix performance regression in NumericComparator.
Remove intra-merge parallelism for everything except HNSW graph merges.
Fix bug that prevented adding a parent field to an index with no fields.
Fix IndexOutOfBoundsException thrown in DefaultPassageFormatter by unordered matches.
StringValueFacetCounts stops throwing NPE when faceting over an empty match-set.

4 weeks ago

Add support for posix_madvise to MMapDirectory: If running on Linux/macOS and Java 21 or later, MMapDirectory uses IOContext to pass suitable MADV flags to kernel of operating system. This may improve paging logic especially when working with large indexes under memory pressure.
Expand support for new scalar bit levels for HNSW vectors. This includes 4-bit vectors and an option to compress them to gain a 50% reduction in memory usage.
Recursive graph bisection is now supported on indexes that have blocks

MergeScheduler can now provide an executor for intra-merge parallelism. The first implementation is the ConcurrentMergeScheduler.
Upgrade icu4j to version 74.2.

Use RWLock to access LRUQueryCache to reduce contention.
Speedup multi-segment HNSW graph search for diversifying child kNN queries.
Add a MemorySegment Vector scorer - for scoring without copying on-heap. This can improve search latency by almost 2x for byte vectors.
Switch to using optimized, primitive collections where possible to improve performance and heap utilization.

Full Changelog: https://github.com/apache/lucene/compare/releases/lucene/9.10.0...releases/lucene/9.11.0

4 months ago

Support for similarity-based vector searches, ie. finding all nearest neighbors whose similarity is greater than a configured threshold from a query vector. See [Byte|Float]VectorSimilarityQuery.
Index sorting is now compatible with block joins. See IndexWriterConfig#setParentField.
MMapDirectory now takes advantage of the now finalized JDK foreign memory API internally when running on Java 22 (or later). This was only supported with Java 19 to 21 until now.
SIMD vectorization now takes advantage of JDK vector incubator on Java 22. This was only supported with Java 20 or 21 until now.

Tail postings are now encoded using group-varint. This yielded speedups on queries that match lots of terms that have short postings lists in Lucene's nightly benchmarks.
Range queries on points now exit earlier when evaluating a segment that has no matches. This will improve performance when intersected with other queries that have a high up-front cost such as multi-term queries.
BooleanQueries that mix SHOULD and FILTER clauses now propagate minimum competitive scores to the SHOULD clauses, yielding significant speedups for top-k queries sorted by descending score.
IndexSearcher#count has been optimized on pure disjunctions of two term queries.