OpenSearch Lucene Study Group Meeting - Monday, March 4th, 2024

Sign up to join the meeting at Meetup:

Link to previous meeting’s post: OpenSearch Lucene Study Group Meeting - Monday, February 19th, 2024

Welcome to the OpenSearch Lucene Study Group!

Apache Lucene is the open-sourced search library that powers OpenSearch and many search applications large and small.

We start the meeting with a Lucene learning topic or Q&A session. In the second half of the meeting, we review recent developments in Apache Lucene and discuss their potential impact to OpenSearch, with a particular focus on new and exciting Lucene features that we can (and should) expose through OpenSearch. Since some changes require a deep dive to fully understand, we sometimes ask participants to volunteer for “homework” to dig deeper into changes and report back for the next meeting.

Standing Agenda:

  • Welcome / introduction (5 minutes)
  • Lucene learning series - someone will either present a Lucene-related talk or we will do Lucene Q&A (20 minutes, recorded)
  • Review assigned issues from last time (10 minutes)
  • Review new Lucene changes and assign homework (20 minutes)

By joining the OpenSearch Lucene Study Group Meeting, you grant OpenSearch, and our affiliates the right to record, film, photograph, and capture your voice and image during the OpenSearch Community Meeting (the “Recordings”). You grant to us an irrevocable, nonexclusive, perpetual, worldwide, royalty-free right and license to use, reproduce, modify, distribute, and translate, for any purpose, all or any part of the Recordings and Your Materials. For example, we may distribute Recordings or snippets of Recordings via our social media outlets.

VersionCategoryDescriptionLink
Lucene 10.0.0API ChangesGITHUB#13148: Remove ByteBufferIndexInput and only use MemorySegment APIs for MMapDirectory.https://github.com/apache/lucene/issues/13146,
Lucene 10.0.0OtherBump minimum required Java version to 21https://github.com/apache/lucene/issues/12753
Lucene 9.11.0API ChangesDeprecate ByteBufferIndexInput as it will be removed in Lucene 10.0.https://github.com/apache/lucene/issues/13145
Lucene 9.11.0ImprovementsChanged `static final Set` constants to be immutable. Among others it affected ScandinavianNormalizer.ALL_FOLDINGS set with public access.https://github.com/apache/lucene/issues/13087
Lucene 9.11.0OptimizationsReduce ArrayUtil#grow in decompress.https://github.com/apache/lucene/issues/12996
Lucene 9.11.0OptimizationsShort circuit queued flush check when flush on update is disabledhttps://github.com/apache/lucene/issues/13115
Lucene 9.11.0OptimizationsRemove unnecessary toString() / substring() calls to save some String allocationshttps://github.com/apache/lucene/issues/13085
Lucene 9.11.0Bug FixesFix ByteKnnVectorFieldSource & FloatKnnVectorFieldSource to work correctly when a segment does not contain any docs with vectorshttps://github.com/apache/lucene/issues/13105
Lucene 9.11.0Bug FixesFix DV update files referenced by merge will be deleted by concurrent flush.https://github.com/apache/lucene/issues/13017
Lucene 9.11.0Bug FixesDetect MemorySegmentIndexInput correctly in NRTSuggester.https://github.com/apache/lucene/issues/13145
Lucene 9.11.0OtherAdd public getter for SynonymQuery#fieldhttps://github.com/apache/lucene/issues/13077

Other items: