Skip to content

Conversation

@vraulji567
Copy link

Motivation

Replication worker fails to replicate ledgers with thousands of entries when auditorLedgerVerificationPercentage is set to 100. The verification process makes too many read requests at once exceeding maxPendingReadRequestsPerThread and fails with Too many requests to the same Bookie while reading. The verification process should not make too many simultaneous reads and throttle the reads based on inFlightReadEntryNumInLedgerChecker.

Changes

Pass the config to the verification process(LedgerChecker) while initializing it as part of the Replication worker to throttle the number of simultaneous reads to the bookie preventing Too many requests to the same Bookie while reading error and replication of large ledgers with 100% verification possible.

Added a test to verify the throttling logic.

@vraulji567
Copy link
Author

@rdhabalia Can you help review this pull request?

@vraulji567
Copy link
Author

@ivankelly @merlimat Can someone help to review the pull request?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant