-
Notifications
You must be signed in to change notification settings - Fork 791
SOLR-18042 Manage MDC Logging Context setup/teardown in a servlet filter. #3985
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
gus-asf
wants to merge
2
commits into
apache:main
Choose a base branch
from
gus-asf:enhancement/SOLR-18042
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+162
−52
Open
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,8 @@ | ||
| # See https://github.com/apache/solr/blob/main/dev-docs/changelog.adoc | ||
| title: SOLR 18042 - MDC Logging context lifecycle is now managed by a servlet filter | ||
| type: other # added, changed, fixed, deprecated, removed, dependency_update, security, other | ||
| authors: | ||
| - name: Gus Heck | ||
| links: | ||
| - name: SOLR-18042 | ||
| url: https://issues.apache.org/jira/browse/SOLR-18042 |
59 changes: 59 additions & 0 deletions
59
solr/core/src/java/org/apache/solr/servlet/CoreContainerAwareHttpFilter.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,59 @@ | ||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one or more | ||
| * contributor license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright ownership. | ||
| * The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| * (the "License"); you may not use this file except in compliance with | ||
| * the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
| package org.apache.solr.servlet; | ||
|
|
||
| import jakarta.servlet.FilterConfig; | ||
| import jakarta.servlet.ServletException; | ||
| import jakarta.servlet.UnavailableException; | ||
| import jakarta.servlet.http.HttpFilter; | ||
| import java.lang.invoke.MethodHandles; | ||
| import org.apache.solr.core.CoreContainer; | ||
| import org.slf4j.Logger; | ||
| import org.slf4j.LoggerFactory; | ||
|
|
||
| /** A superclass for filters that will need to interact with the CoreContainer. */ | ||
| public abstract class CoreContainerAwareHttpFilter extends HttpFilter { | ||
| private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); | ||
|
|
||
| private CoreContainerProvider containerProvider; | ||
|
|
||
| @Override | ||
| public void init(FilterConfig config) throws ServletException { | ||
| super.init(config); | ||
| containerProvider = CoreContainerProvider.serviceForContext(config.getServletContext()); | ||
| if (log.isTraceEnabled()) { | ||
| log.trace("{}.init(): {}", this.getClass().getName(), this.getClass().getClassLoader()); | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * The CoreContainer. It's guaranteed to be constructed before this filter is initialized, but | ||
| * could have been shut down. Never null. | ||
| */ | ||
| public CoreContainer getCores() throws UnavailableException { | ||
| return containerProvider.getCoreContainer(); | ||
| } | ||
|
|
||
| // TODO: not fond of having these here, but RateLimiter initialization can be sorted out later | ||
| RateLimitManager getRateLimitManager() { | ||
| return containerProvider.getRateLimitManager(); | ||
| } | ||
|
|
||
| void replaceRateLimitManager(RateLimitManager rlm) { | ||
| containerProvider.setRateLimitManager(rlm); | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
57 changes: 57 additions & 0 deletions
57
solr/core/src/java/org/apache/solr/servlet/MdcLoggingFilter.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one or more | ||
| * contributor license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright ownership. | ||
| * The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| * (the "License"); you may not use this file except in compliance with | ||
| * the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
| package org.apache.solr.servlet; | ||
|
|
||
| import jakarta.servlet.FilterChain; | ||
| import jakarta.servlet.ServletException; | ||
| import jakarta.servlet.http.HttpServletRequest; | ||
| import jakarta.servlet.http.HttpServletResponse; | ||
| import java.io.IOException; | ||
| import java.lang.invoke.MethodHandles; | ||
| import org.apache.solr.common.util.SuppressForbidden; | ||
| import org.apache.solr.logging.MDCLoggingContext; | ||
| import org.apache.solr.logging.MDCSnapshot; | ||
| import org.slf4j.Logger; | ||
| import org.slf4j.LoggerFactory; | ||
|
|
||
| /** Servlet Filter to set up and tear down MDC Logging context for each reqeust. */ | ||
| public class MdcLoggingFilter extends CoreContainerAwareHttpFilter { | ||
|
|
||
| private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); | ||
|
|
||
| @Override | ||
| @SuppressForbidden( | ||
| reason = | ||
| "Set the thread contextClassLoader for all 3rd party dependencies that we cannot control") | ||
| protected void doFilter(HttpServletRequest req, HttpServletResponse res, FilterChain chain) | ||
| throws IOException, ServletException { | ||
| ClassLoader contextClassLoader = Thread.currentThread().getContextClassLoader(); | ||
| // this autocloseable is here to invoke MDCSnapshot.close() which restores captured state | ||
| try (var mdcSnapshot = MDCSnapshot.create()) { | ||
| log.trace("MDC snapshot recorded {}", mdcSnapshot); // avoid both compiler and ide warning. | ||
| MDCLoggingContext.reset(); | ||
| MDCLoggingContext.setNode(getCores()); | ||
| // This doesn't belong here, but for the moment it is here to preserve it's relative | ||
| // timing of execution for now. Probably I will break this out in a subsequent change | ||
| Thread.currentThread().setContextClassLoader(getCores().getResourceLoader().getClassLoader()); | ||
| chain.doFilter(req, res); | ||
| } finally { | ||
| MDCLoggingContext.reset(); | ||
| Thread.currentThread().setContextClassLoader(contextClassLoader); | ||
| } | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -20,31 +20,23 @@ | |
| import static org.apache.solr.util.tracing.TraceUtils.getSpan; | ||
| import static org.apache.solr.util.tracing.TraceUtils.setTracer; | ||
|
|
||
| import com.google.common.annotations.VisibleForTesting; | ||
| import jakarta.servlet.FilterChain; | ||
| import jakarta.servlet.FilterConfig; | ||
| import jakarta.servlet.ServletException; | ||
| import jakarta.servlet.UnavailableException; | ||
| import jakarta.servlet.http.HttpFilter; | ||
| import jakarta.servlet.http.HttpServletRequest; | ||
| import jakarta.servlet.http.HttpServletResponse; | ||
| import java.io.IOException; | ||
| import java.lang.invoke.MethodHandles; | ||
| import java.util.List; | ||
| import java.util.concurrent.CountDownLatch; | ||
| import java.util.concurrent.atomic.AtomicBoolean; | ||
| import java.util.concurrent.atomic.AtomicReference; | ||
| import java.util.regex.Pattern; | ||
| import org.apache.solr.api.V2HttpCall; | ||
| import org.apache.solr.common.SolrException; | ||
| import org.apache.solr.common.SolrException.ErrorCode; | ||
| import org.apache.solr.common.util.ExecutorUtil; | ||
| import org.apache.solr.common.util.SuppressForbidden; | ||
| import org.apache.solr.core.CoreContainer; | ||
| import org.apache.solr.core.NodeRoles; | ||
| import org.apache.solr.handler.api.V2ApiUtils; | ||
| import org.apache.solr.logging.MDCLoggingContext; | ||
| import org.apache.solr.logging.MDCSnapshot; | ||
| import org.apache.solr.request.SolrRequestInfo; | ||
| import org.apache.solr.security.AuditEvent; | ||
| import org.apache.solr.security.AuditEvent.EventType; | ||
|
|
@@ -65,19 +57,13 @@ | |
| // servlets that are more focused in scope. This should become possible now that we have a | ||
| // ServletContextListener for startup/shutdown of CoreContainer that sets up a service from which | ||
| // things like CoreContainer can be requested. (or better yet injected) | ||
| public class SolrDispatchFilter extends HttpFilter { | ||
| public class SolrDispatchFilter extends CoreContainerAwareHttpFilter { | ||
| private static final Logger log = LoggerFactory.getLogger(MethodHandles.lookup().lookupClass()); | ||
|
|
||
| private CoreContainerProvider containerProvider; | ||
|
|
||
| protected final CountDownLatch init = new CountDownLatch(1); | ||
|
|
||
| protected String abortErrorMessage = null; | ||
|
|
||
| private HttpSolrCallFactory solrCallFactory; | ||
|
|
||
| private List<Pattern> excludePatterns; | ||
|
|
||
| public final boolean isV2Enabled = V2ApiUtils.isEnabled(); | ||
|
|
||
| /** | ||
|
|
@@ -115,9 +101,8 @@ public SolrDispatchFilter() {} | |
|
|
||
| @Override | ||
| public void init(FilterConfig config) throws ServletException { | ||
| super.init(config); | ||
| try { | ||
| containerProvider = CoreContainerProvider.serviceForContext(config.getServletContext()); | ||
| super.init(config); | ||
| boolean isCoordinator = | ||
| NodeRoles.MODE_ON.equals(getCores().nodeRoles.getRoleMode(NodeRoles.Role.COORDINATOR)); | ||
| solrCallFactory = | ||
|
|
@@ -134,37 +119,28 @@ public void init(FilterConfig config) throws ServletException { | |
| } | ||
| } finally { | ||
| log.trace("SolrDispatchFilter.init() done"); | ||
| init.countDown(); | ||
| } | ||
| } | ||
|
|
||
| /** The CoreContainer. It's ready for use, albeit could shut down whenever. Never null. */ | ||
| public CoreContainer getCores() throws UnavailableException { | ||
| return containerProvider.getCoreContainer(); | ||
| } | ||
|
|
||
| @Override | ||
| @SuppressForbidden( | ||
| reason = | ||
| "Set the thread contextClassLoader for all 3rd party dependencies that we cannot control") | ||
| public void doFilter(HttpServletRequest request, HttpServletResponse response, FilterChain chain) | ||
| throws IOException, ServletException { | ||
| // internal version of doFilter that tracks if we are in a retry | ||
| doFilterRetry(closeShield(request), closeShield(response), chain, false); | ||
| } | ||
|
|
||
| try (var mdcSnapshot = MDCSnapshot.create()) { | ||
| assert null != mdcSnapshot; // prevent compiler warning | ||
| MDCLoggingContext.reset(); | ||
| MDCLoggingContext.setNode(getCores()); | ||
| Thread.currentThread().setContextClassLoader(getCores().getResourceLoader().getClassLoader()); | ||
| /* | ||
|
|
||
| doFilterRetry(closeShield(request), closeShield(response), chain, false); | ||
| } | ||
| } | ||
| Wait? Where did X go??? (I hear you ask). First: look for a servlet filter to which request wrapping | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I appreciate the comment but I think it could be worded more directly to the point. |
||
| operations may have been moved (see web.xml for list) | ||
|
|
||
| */ | ||
|
|
||
| protected void doFilterRetry( | ||
| private void doFilterRetry( | ||
| HttpServletRequest request, HttpServletResponse response, FilterChain chain, boolean retry) | ||
| throws IOException, ServletException { | ||
| setTracer(request, getCores().getTracer()); | ||
| RateLimitManager rateLimitManager = containerProvider.getRateLimitManager(); | ||
| RateLimitManager rateLimitManager = getRateLimitManager(); | ||
| try { | ||
| ServletUtils.rateLimitRequest( | ||
| rateLimitManager, | ||
|
|
@@ -369,11 +345,6 @@ private boolean shouldAudit(AuditEvent.EventType eventType) { | |
| && cores.getAuditLoggerPlugin().shouldLog(eventType); | ||
| } | ||
|
|
||
| @VisibleForTesting | ||
| void replaceRateLimitManager(RateLimitManager rateLimitManager) { | ||
| containerProvider.setRateLimitManager(rateLimitManager); | ||
| } | ||
|
|
||
| /** internal API */ | ||
| public interface HttpSolrCallFactory { | ||
| default HttpSolrCall createInstance( | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hopefully not a future Filter for just one line.
I sympathize with your characterization of SolrDispatchFilter as a kitchen sink but a few lines for certain topics at the critical Solr entrypoint is understandable IMO.