-
Notifications
You must be signed in to change notification settings - Fork 74
MLE-26918 Added fromView support for incremental write #1897
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Copyright Validation Results ⏭️ Skipped (Excluded) Files
✅ Valid Files
✅ All files have valid copyright headers! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Adds support for incremental write hashing via an Optic fromView query (TDE view) as an alternative to lexicon-based retrieval, along with tests and a sample TDE template.
Changes:
- Added a new
IncrementalWriteViewFilterimplementation that queries existing hashes from a TDE view viaop.fromView. - Extended
IncrementalWriteFilter.BuilderwithfromView(schemaName, viewName)and wiring inbuild(). - Added test coverage for successful
fromViewusage and for a helpful failure when the view is invalid; added a sample TDE template for the view.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| test-app/src/main/ml-schemas/tde/incrementalWriteHash.json | Adds a TDE template to expose URI/hash via a view used by fromView. |
| marklogic-client-api/src/test/java/com/marklogic/client/datamovement/filter/IncrementalWriteTest.java | Adds tests for fromView and invalid view failure messaging. |
| marklogic-client-api/src/main/java/com/marklogic/client/datamovement/filter/IncrementalWriteViewFilter.java | Implements view-based hash lookup using RowTemplate + Optic fromView. |
| marklogic-client-api/src/main/java/com/marklogic/client/datamovement/filter/IncrementalWriteFilter.java | Adds builder support for configuring and constructing the view-based filter. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| return filterDocuments(context, uri -> existingHashes.get(uri)); | ||
| } catch (FailedRequestException e) { | ||
| String message = "Unable to query for existing incremental write hashes from view " + schemaName + "." + viewName + "; cause: " + e.getMessage(); | ||
| throw new FailedRequestException(message, e.getFailedRequest()); |
Copilot
AI
Feb 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The rethrown FailedRequestException drops the original exception as the cause, which makes debugging harder (stack trace/cause chain is lost). Prefer a constructor overload that accepts the cause, or initialize the cause on the newly created exception so the original e is preserved.
| throw new FailedRequestException(message, e.getFailedRequest()); | |
| FailedRequestException fre = new FailedRequestException(message, e.getFailedRequest()); | |
| fre.initCause(e); | |
| throw fre; |
| "template": { | ||
| "description": "For incremental write that uses op.fromView instead of op.fromLexicons", | ||
| "context": "/doc", | ||
| "rows": [ | ||
| { | ||
| "schemaName": "javaClient", | ||
| "viewName": "incrementalWriteHash", | ||
| "columns": [ | ||
| { | ||
| "name": "uri", | ||
| "scalarType": "string", | ||
| "val": "xdmp:node-uri(.)" | ||
| }, | ||
| { | ||
| "name": "hash", | ||
| "scalarType": "string", | ||
| "val": "xdmp:node-metadata-value(., 'incrementalWriteHash')", | ||
| "nullable": true | ||
| } | ||
| ] | ||
| } | ||
| ] | ||
| } |
Copilot
AI
Feb 10, 2026
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This JSON uses tab indentation. If the repo’s JSON/style conventions prefer spaces (common for JSON), consider normalizing indentation to match the rest of the project to reduce noisy diffs and keep formatting consistent.
| "template": { | |
| "description": "For incremental write that uses op.fromView instead of op.fromLexicons", | |
| "context": "/doc", | |
| "rows": [ | |
| { | |
| "schemaName": "javaClient", | |
| "viewName": "incrementalWriteHash", | |
| "columns": [ | |
| { | |
| "name": "uri", | |
| "scalarType": "string", | |
| "val": "xdmp:node-uri(.)" | |
| }, | |
| { | |
| "name": "hash", | |
| "scalarType": "string", | |
| "val": "xdmp:node-metadata-value(., 'incrementalWriteHash')", | |
| "nullable": true | |
| } | |
| ] | |
| } | |
| ] | |
| } | |
| "template": { | |
| "description": "For incremental write that uses op.fromView instead of op.fromLexicons", | |
| "context": "/doc", | |
| "rows": [ | |
| { | |
| "schemaName": "javaClient", | |
| "viewName": "incrementalWriteHash", | |
| "columns": [ | |
| { | |
| "name": "uri", | |
| "scalarType": "string", | |
| "val": "xdmp:node-uri(.)" | |
| }, | |
| { | |
| "name": "hash", | |
| "scalarType": "string", | |
| "val": "xdmp:node-metadata-value(., 'incrementalWriteHash')", | |
| "nullable": true | |
| } | |
| ] | |
| } | |
| ] | |
| } |
7fc9d40 to
4103e44
Compare
The plan is to dump eval support and just offer fromLexicons and fromView, but need to test out fromView a bit first. Did some refactoring too because the constructors had gotten so ugly - there's now an IncrementalWriteConfig class that holds all the inputs from the Builder, so that filter constructors only need that as an arg.
4103e44 to
130ba68
Compare
The plan is to dump eval support and just offer fromLexicons and fromView, but need to test out fromView a bit first.