support support for ngram indexer #197
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Packaging
Fix compilation under FreeBSD
- Should close all but the first SkipBuffer.
Thanks to alian555.
Alain Barbet <alian@amisw.com>
Improved the README file's build instructions
Update for boost-1.5
- Thanks to kojik1010.
Commit e3f8992 ported the code to use Boost.Filesystem V3 API, so the warning about not being able to use Boost 1.50+ is no longer true.
Remove outdated note about Boost incompatibility from the README.
I've observed crashes where one thread is in the middle of initializing ZZ_CMAP and other is trying to use the partially initialized array and crashes. Use boost::once to ensure that only one thread handles the initialization and no thread uses the data until it is fully initialized.
Fix races when initializing static arrays in StandardTokenizerImpl
The argument should be named startOffset, not endOffset, otherwise the function is a no-op.
Use StringUtils::toString() before trying to concatenate or use operator<< which is more type-safe.
Fix some issues identified using clang's static analysis
The code previously used the deprecated (and with bost 1.87.0 removed) `boost::asio::io_service`, which used to be an alias to `io_context`. The new version heavily changes the `io_context` API and therefore is no the old interface was removed. Fixes #208 Signed-off-by: Christian Heusel <christian@heusel.eu>
…o_context Migrate to boost::asio::io_context
Fix typos in Set<T>::swap() and Collection<T>::swap(). Apparently, no compiler ever tried to compile these methods before.
Fix compilation with clang-17
This commit addresses critical crashes that occur when trying to read certain Lucene index files where field data may be corrupted or have incompatible format. Root causes: 1. Assertion failures in FieldsReader::doc() when bits flag or compressed field format is incompatible with expected values 2. Null pointer dereference in boost::shared_ptr when trying to add fields to a document with potentially corrupted data The fixes include: - Replace assertions with safe handling for invalid bit flags and format incompatibilities - Add null checks before dereferencing shared pointers in critical paths - Create field objects separately before adding them to documents - Add proper error handling for decompression failures - Ensure uncompress and string conversion methods never return null values These minimal changes maintain the original logic but make the code more robust when dealing with unexpected or corrupt index data. Instead of crashing, the code now gracefully handles these edge cases and continues processing where possible.
Fix crash when reading corrupted or incompatible Lucene index files
- If `boost::filesystem::directory_entry::status` fails, return false instead of throwing a `boost::filesystem::filesystem_error` exception. - During file enumeration, if an `ENOENT` error occurs, re-enumerate all directory entries to increase robustness.
Boost.System has been header only since Boost 1.69.0
Handle file enumeration exceptions in FileUtils::listDirectory
Fix build new cmake
Fix typo in MAX_VARINT32_LENGTH constant in BufferedIndexInput.cpp
Update DefaultSimilarity.cpp
Fix old comment about C++ standard
BitSet: Partial fix for Boost 1.90
Use conditional compilation to support both old and new Boost.Bind API: - Boost >= 1.73.0: Use boost/bind/bind.hpp - Boost < 1.73.0: Use boost/bind.hpp This approach maintains backward compatibility while fixing deprecation warnings in newer Boost versions.
Use new Boost.Bind API to fix deprecation warnings
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
i hope if you support support for ngram indexer