最近作的一個項目由於安全審計須要,須要作安全改造。其中天然就包括XSS和CSRF漏洞安全整改。關於這兩個網絡安全漏洞的詳細說明,能夠參照我本篇博客最後的參考連接。固然,我這裏並非想寫一篇安全方面的專題。我要講的是在作了XSS漏洞修復以後引起的一系列事件。java
本地測試的時候隨便點了些頁面,而後debug跟了下代碼未發現任何問題。上線以後用戶反饋有的頁面打不開,本身去線上體驗發現大部分頁面正常,可是存在部分客戶反饋的頁面打開直接超時報錯。ios
XSS這個漏洞修復開始不是通過我處理的,上線以後因爲編碼規則太嚴格(後面我會講咱們使用的解決方案),致使前臺傳入的JSON字符串中的引號全被轉碼,形成後臺解析報錯。web
而我成爲了那個救(bei)火(guo)英(xia)雄,須要立馬解決這個問題補丁升級。我看了下之前實現的代碼,有考慮到經過一個XML白名單文件,配置忽略XSS編碼的請求URI。最直接的辦法,是直接把我這個請求的URI加入XML白名單配置,而後補丁替換白名單文件重啓服務。spring
可是當時我在修改的時候,考慮到可能不止這一個須要過濾的白名單,若是純粹啓動時加載的XML白名單列表。到時候還有別的URI須要忽略,那我豈不是還要再發增量補丁......數據庫
因而當時我修改的時候順便增長了一個能力,白名單能夠直接在界面配置,而且每次獲取白名單列表的時候動態從數據庫獲取(考慮到實時請求較大,我調用的系統已有接口提供的支持緩存的查詢方法)。正由於有這個「後門」我直接在線上配置了這個參數,先暫且把線上問題解決。express
解決問題第一步,天然是分析線上日誌。發現線上日誌的確對請求的URI中的參數作了XSS編碼處理。編程
那問題就回到咱們XSS漏洞修復的實現方式:AntiSamy(見參考連接)。其中咱們的XssRequestWrapper
源碼以下:json
public class XssRequestWrapper extends HttpServletRequestWrapper { private static Logger log = LoggerFactory.getLogger(XssRequestWrapper.class); private static Policy policy = null; static { String path = XssRequestWrapper.class.getClassLoader().getResource("security/antisamy-tinymce.xml").getFile(); log.info("policy_filepath:" + path); if (path.startsWith("file")) { //以file:開頭 path = path.substring(6); } try { policy = Policy.getInstance(path); } catch (PolicyException e) { e.printStackTrace(); } } public XssRequestWrapper(HttpServletRequest request) { super(request); } // 隊請求參數進行安全轉碼 public String getParameter(String paramString) { String str = super.getParameter(paramString); if (StringUtils.isBlank(str)) { return null; } return xssClean(str, paramString); } // 隊請求頭進行安全轉碼 public String getHeader(String paramString) { String str = super.getHeader(paramString); if (StringUtils.isBlank(str)) return null; return xssClean(str, paramString); } @SuppressWarnings({"rawtypes","unchecked"}) public Map<String, String[]> getParameterMap() { Map<String, String[]> request_map = super.getParameterMap(); Iterator iterator = request_map.entrySet().iterator(); log.debug("getParameterMap size:{}", request_map.size()); while (iterator.hasNext()) { Map.Entry me = (Map.Entry) iterator.next(); String paramsKey = (String) me.getKey(); String[] values = (String[]) me.getValue(); for (int i = 0; i < values.length; i++) { values[i] = xssClean(values[i], paramsKey); } } return request_map; } public String[] getParameterValues(String paramString) { String[] arrayOfString1 = super.getParameterValues(paramString); if (arrayOfString1 == null) return null; int i = arrayOfString1.length; String[] arrayOfString2 = new String[i]; for (int j = 0; j < i; j++) arrayOfString2[j] = xssClean(arrayOfString1[j], paramString); return arrayOfString2; } public final static String KEY_FILTER_STR = "'"; public final static String SUFFIX = "value"; //須要過濾的地方 private String xssClean(String value, String paramsKey) { String keyFilterStr = KEY_FILTER_STR; String param = paramsKey.toLowerCase(); if (param.endsWith(SUFFIX)) { // 若是參數名 name="xxxvalue" if (value.contains(keyFilterStr)) { value = value.replace(keyFilterStr, "‘"); } } AntiSamy antiSamy = new AntiSamy(); try { final CleanResults cr = antiSamy.scan(value, policy); // 安全的HTML輸出 return cr.getCleanHTML(); } catch (ScanException e) { log.error("antiSamy scan error", e); } catch (PolicyException e) { log.error("antiSamy policy error", e); } return value; } }
能夠看到了咱們項目組最終採用的策略配置文件是:antisamy-tinymce.xml
,這種策略只容許傳送純文本到後臺(這樣作真的好嗎?我的以爲這個規則太過嚴格),而且對請求頭和請求參數都作了XSS轉碼。請注意這裏,咱們相對於參考連接中源碼不一樣的處理方式在於:咱們對請求頭也進行了編碼處理。跨域
那麼看來問題就在於編碼致使的效率低下,因而我在getHeader
和getParameter
方法中都打了斷點。在揭曉結果以前,我說說我當時的猜想:由於當時用戶反饋的有問題的頁面是有不少查詢條件的,我開始的猜想是應該是傳入後臺的參數過多致使編碼影響效率。然而,現實老是無情地打臉,無論你天真不天真。緩存
Debug發現getParameter
調用的此時算正常,而getHeader
處的斷點沒完沒了的進來(最終結果證實,進來了幾千次)......
仍是那句話,沒有什麼是源碼解決不了的,若是有,那麼請Debug源碼。😄
咱們項目是傳統的SpringMVC項目,那麼固然咱們要從org.springframework.web.servlet.DispatcherServlet
入手了。DispatcherServlet
其實也是一個Servlet,他的繼承關係以下:
DispatcherServlet extends FrameworkServlet FrameworkServlet extends HttpServletBean implements ApplicationContextAware HttpServletBean extends HttpServlet implements EnvironmentCapable, EnvironmentAware HttpServlet extends GenericServlet GenericServlet implements Servlet, ServletConfig, java.io.Serializable
能夠看到,實際上SpringMVC的DispatcherServlet
最終也是經過doGet
和doPost
來對請求進行轉發,而最終其實都到了DispatcherServlet
的doService
.該方法的源碼以下:
/** * Exposes the DispatcherServlet-specific request attributes and delegates to {@link #doDispatch} * for the actual dispatching. */ @Override protected void doService(HttpServletRequest request, HttpServletResponse response) throws Exception { if (logger.isDebugEnabled()) { String resumed = WebAsyncUtils.getAsyncManager(request).hasConcurrentResult() ? " resumed" : ""; logger.debug("DispatcherServlet with name '" + getServletName() + "'" + resumed + " processing " + request.getMethod() + " request for [" + getRequestUri(request) + "]"); } // Keep a snapshot of the request attributes in case of an include, // to be able to restore the original attributes after the include. Map<String, Object> attributesSnapshot = null; if (WebUtils.isIncludeRequest(request)) { attributesSnapshot = new HashMap<String, Object>(); Enumeration<?> attrNames = request.getAttributeNames(); while (attrNames.hasMoreElements()) { String attrName = (String) attrNames.nextElement(); if (this.cleanupAfterInclude || attrName.startsWith("org.springframework.web.servlet")) { attributesSnapshot.put(attrName, request.getAttribute(attrName)); } } } // Make framework objects available to handlers and view objects. request.setAttribute(WEB_APPLICATION_CONTEXT_ATTRIBUTE, getWebApplicationContext()); request.setAttribute(LOCALE_RESOLVER_ATTRIBUTE, this.localeResolver); request.setAttribute(THEME_RESOLVER_ATTRIBUTE, this.themeResolver); request.setAttribute(THEME_SOURCE_ATTRIBUTE, getThemeSource()); FlashMap inputFlashMap = this.flashMapManager.retrieveAndUpdate(request, response); if (inputFlashMap != null) { request.setAttribute(INPUT_FLASH_MAP_ATTRIBUTE, Collections.unmodifiableMap(inputFlashMap)); } request.setAttribute(OUTPUT_FLASH_MAP_ATTRIBUTE, new FlashMap()); request.setAttribute(FLASH_MAP_MANAGER_ATTRIBUTE, this.flashMapManager); try { // 重點在這,都會進入doDispatch方法 doDispatch(request, response); } finally { if (!WebAsyncUtils.getAsyncManager(request).isConcurrentHandlingStarted()) { // Restore the original attribute snapshot, in case of an include. if (attributesSnapshot != null) { restoreAttributesAfterInclude(request, attributesSnapshot); } } } }
能夠看到doService
方法,最終仍是會進入到doDispatch
中,該方法的源碼以下:
/** * Process the actual dispatching to the handler. * <p>The handler will be obtained by applying the servlet's HandlerMappings in order. * The HandlerAdapter will be obtained by querying the servlet's installed HandlerAdapters * to find the first that supports the handler class. * <p>All HTTP methods are handled by this method. It's up to HandlerAdapters or handlers * themselves to decide which methods are acceptable. * @param request current HTTP request * @param response current HTTP response * @throws Exception in case of any kind of processing failure */ protected void doDispatch(HttpServletRequest request, HttpServletResponse response) throws Exception { HttpServletRequest processedRequest = request; HandlerExecutionChain mappedHandler = null; boolean multipartRequestParsed = false; WebAsyncManager asyncManager = WebAsyncUtils.getAsyncManager(request); try { ModelAndView mv = null; Exception dispatchException = null; try { processedRequest = checkMultipart(request); multipartRequestParsed = (processedRequest != request); // Determine handler for the current request. // 根據請求獲取處理的Handler mappedHandler = getHandler(processedRequest); if (mappedHandler == null || mappedHandler.getHandler() == null) { noHandlerFound(processedRequest, response); return; } // Determine handler adapter for the current request. HandlerAdapter ha = getHandlerAdapter(mappedHandler.getHandler()); // Process last-modified header, if supported by the handler. String method = request.getMethod(); boolean isGet = "GET".equals(method); if (isGet || "HEAD".equals(method)) { long lastModified = ha.getLastModified(request, mappedHandler.getHandler()); if (logger.isDebugEnabled()) { logger.debug("Last-Modified value for [" + getRequestUri(request) + "] is: " + lastModified); } if (new ServletWebRequest(request, response).checkNotModified(lastModified) && isGet) { return; } } if (!mappedHandler.applyPreHandle(processedRequest, response)) { return; } // Actually invoke the handler. mv = ha.handle(processedRequest, response, mappedHandler.getHandler()); if (asyncManager.isConcurrentHandlingStarted()) { return; } applyDefaultViewName(processedRequest, mv); mappedHandler.applyPostHandle(processedRequest, response, mv); } catch (Exception ex) { dispatchException = ex; } catch (Throwable err) { // As of 4.3, we're processing Errors thrown from handler methods as well, // making them available for @ExceptionHandler methods and other scenarios. dispatchException = new NestedServletException("Handler dispatch failed", err); } processDispatchResult(processedRequest, response, mappedHandler, mv, dispatchException); } catch (Exception ex) { triggerAfterCompletion(processedRequest, response, mappedHandler, ex); } catch (Throwable err) { triggerAfterCompletion(processedRequest, response, mappedHandler, new NestedServletException("Handler processing failed", err)); } finally { if (asyncManager.isConcurrentHandlingStarted()) { // Instead of postHandle and afterCompletion if (mappedHandler != null) { mappedHandler.applyAfterConcurrentHandlingStarted(processedRequest, response); } } else { // Clean up any resources used by a multipart request. if (multipartRequestParsed) { cleanupMultipart(processedRequest); } } } }
這段處理的重點也在我中文註釋的地方,咱們繼續跟進getHandler
方法:
protected HandlerExecutionChain getHandler(HttpServletRequest request) throws Exception { // 遍歷HandlerMapping,找到請求對應的Handler具體信息 for (HandlerMapping hm : this.handlerMappings) { if (logger.isTraceEnabled()) { logger.trace( "Testing handler map [" + hm + "] in DispatcherServlet with name '" + getServletName() + "'"); } HandlerExecutionChain handler = hm.getHandler(request); if (handler != null) { return handler; } } return null; } // 最終會到AbstractHandlerMethodMapping#getHandlerInternal protected HandlerMethod getHandlerInternal(HttpServletRequest request) throws Exception { // 分析請求URI (去掉Context) String lookupPath = getUrlPathHelper().getLookupPathForRequest(request); if (logger.isDebugEnabled()) { logger.debug("Looking up handler method for path " + lookupPath); } this.mappingRegistry.acquireReadLock(); try { // 重頭戲: 根據請求URI去找對應的處理器 (具體到類和方法) HandlerMethod handlerMethod = lookupHandlerMethod(lookupPath, request); if (logger.isDebugEnabled()) { if (handlerMethod != null) { logger.debug("Returning handler method [" + handlerMethod + "]"); } else { logger.debug("Did not find handler method for [" + lookupPath + "]"); } } return (handlerMethod != null ? handlerMethod.createWithResolvedBean() : null); } finally { this.mappingRegistry.releaseReadLock(); } }
注意其中的中文註釋,咱們經過解析請求中的URI,而後根據請求URI去查找對應的處理器,也就是進行適配的關鍵一步:
/** * Look up the best-matching handler method for the current request. * If multiple matches are found, the best match is selected. * @param lookupPath mapping lookup path within the current servlet mapping * @param request the current request * @return the best-matching handler method, or {@code null} if no match * @see #handleMatch(Object, String, HttpServletRequest) * @see #handleNoMatch(Set, String, HttpServletRequest) */ protected HandlerMethod lookupHandlerMethod(String lookupPath, HttpServletRequest request) throws Exception { List<Match> matches = new ArrayList<Match>(); // 直接根據請求的URI去找對應的處理類(此時前臺請求URI必須與後臺註解配置的RequestMapping徹底一致) List<T> directPathMatches = this.mappingRegistry.getMappingsByUrl(lookupPath); if (directPathMatches != null) { addMatchingMappings(directPathMatches, matches, request); } if (matches.isEmpty()) { // 注意這裏: SpringMVC作了個無奈的容錯處理,若是沒有徹底匹配的話,就遍歷全部請求URI找到大體匹配的 --- 這也是本次問題出現的緣由 // No choice but to go through all mappings... addMatchingMappings(this.mappingRegistry.getMappings().keySet(), matches, request); } if (!matches.isEmpty()) { Comparator<Match> comparator = new MatchComparator(getMappingComparator(request)); Collections.sort(matches, comparator); if (logger.isTraceEnabled()) { logger.trace("Found " + matches.size() + " matching mapping(s) for [" + lookupPath + "] : " + matches); } Match bestMatch = matches.get(0); if (matches.size() > 1) { if (CorsUtils.isPreFlightRequest(request)) { return PREFLIGHT_AMBIGUOUS_MATCH; } Match secondBestMatch = matches.get(1); if (comparator.compare(bestMatch, secondBestMatch) == 0) { Method m1 = bestMatch.handlerMethod.getMethod(); Method m2 = secondBestMatch.handlerMethod.getMethod(); throw new IllegalStateException("Ambiguous handler methods mapped for HTTP path '" + request.getRequestURL() + "': {" + m1 + ", " + m2 + "}"); } } handleMatch(bestMatch.mapping, lookupPath, request); return bestMatch.handlerMethod; } else { return handleNoMatch(this.mappingRegistry.getMappings().keySet(), lookupPath, request); } }
其實到這裏,咱們已經找到真相了,可是爲何循環遍歷請求URI會致使getHeader
方法超頻調用呢?咱們繼續跟進:
private void addMatchingMappings(Collection<T> mappings, List<Match> matches, HttpServletRequest request) { for (T mapping : mappings) { T match = getMatchingMapping(mapping, request); if (match != null) { matches.add(new Match(match, this.mappingRegistry.getMappings().get(mapping))); } } } // RequestMappingInfoHandlerMapping /** * Check if the given RequestMappingInfo matches the current request and * return a (potentially new) instance with conditions that match the * current request -- for example with a subset of URL patterns. * @return an info in case of a match; or {@code null} otherwise. */ @Override protected RequestMappingInfo getMatchingMapping(RequestMappingInfo info, HttpServletRequest request) { return info.getMatchingCondition(request); } // RequestMappingInfo /** * Checks if all conditions in this request mapping info match the provided request and returns * a potentially new request mapping info with conditions tailored to the current request. * <p>For example the returned instance may contain the subset of URL patterns that match to * the current request, sorted with best matching patterns on top. * @return a new instance in case all conditions match; or {@code null} otherwise */ @Override public RequestMappingInfo getMatchingCondition(HttpServletRequest request) { RequestMethodsRequestCondition methods = this.methodsCondition.getMatchingCondition(request); ParamsRequestCondition params = this.paramsCondition.getMatchingCondition(request); // 問題就在於headersCondition的 getMatchingCondition 方法的調用 HeadersRequestCondition headers = this.headersCondition.getMatchingCondition(request); ConsumesRequestCondition consumes = this.consumesCondition.getMatchingCondition(request); ProducesRequestCondition produces = this.producesCondition.getMatchingCondition(request); if (methods == null || params == null || headers == null || consumes == null || produces == null) { return null; } PatternsRequestCondition patterns = this.patternsCondition.getMatchingCondition(request); if (patterns == null) { return null; } RequestConditionHolder custom = this.customConditionHolder.getMatchingCondition(request); if (custom == null) { return null; } return new RequestMappingInfo(this.name, patterns, methods, params, headers, consumes, produces, custom.getCondition()); } // HeadersRequestCondition public HeadersRequestCondition getMatchingCondition(HttpServletRequest request) { // 最終調用的是CorsUtils#isCorsRequest if (CorsUtils.isPreFlightRequest(request)) { return PRE_FLIGHT_MATCH; } for (HeaderExpression expression : expressions) { if (!expression.match(request)) { return null; } } return this; } // CorsUtils /** * Returns {@code true} if the request is a valid CORS one. */ public static boolean isCorsRequest(HttpServletRequest request) { // XSS的編碼是經過Filter對請求中的Header進行編碼的,因此每次遍歷URI都會調用一次請求頭編碼 return (request.getHeader(HttpHeaders.ORIGIN) != null); } /** * Returns {@code true} if the request is a valid CORS pre-flight one. */ public static boolean isPreFlightRequest(HttpServletRequest request) { return (isCorsRequest(request) && HttpMethod.OPTIONS.matches(request.getMethod()) && request.getHeader(HttpHeaders.ACCESS_CONTROL_REQUEST_METHOD) != null); }
好了,真相終於浮出水面了。就是由於若是前臺請求URI沒有徹底匹配後臺配置的話會致使每次跨域請求校驗都會對請求頭中參數進行編碼。而項目中請求的URI有幾千個,假設每一個請求頭平均有3個參數。那麼,一次請求編碼可能上萬次...... 你說超時不超時?
最終和領導討論確認去掉對header的XSS編碼處理。咱們業務上不存在將Header的參數入庫的狀況。
善於思考的小夥伴必定會問了: 爲何會有請求URI不匹配呢?若是不匹配之前爲何能正常請求到呢?
哈哈,咱們繼續看下文:
經過最後的getMatchingCondition
方法,咱們能夠看到要想最終能找到一個匹配的請求的URI。上面幾個condition必須至少要知足一個。經過debug我發現,最終在咱們項目中匹配的是patternsCondition
。那麼這condition的具體實現是咋樣的呢?直接見源碼:
/** * Checks if any of the patterns match the given request and returns an instance * that is guaranteed to contain matching patterns, sorted via * {@link PathMatcher#getPatternComparator(String)}. * <p>A matching pattern is obtained by making checks in the following order: * <ul> * <li>Direct match * <li>Pattern match with ".*" appended if the pattern doesn't already contain a "." * <li>Pattern match * <li>Pattern match with "/" appended if the pattern doesn't already end in "/" * </ul> * @param request the current request * @return the same instance if the condition contains no patterns; * or a new condition with sorted matching patterns; * or {@code null} if no patterns match. */ public PatternsRequestCondition getMatchingCondition(HttpServletRequest request) { if (this.patterns.isEmpty()) { return this; } String lookupPath = this.pathHelper.getLookupPathForRequest(request); List<String> matches = getMatchingPatterns(lookupPath); return matches.isEmpty() ? null : new PatternsRequestCondition(matches, this.pathHelper, this.pathMatcher, this.useSuffixPatternMatch, this.useTrailingSlashMatch, this.fileExtensions); } /** * Find the patterns matching the given lookup path. Invoking this method should * yield results equivalent to those of calling * {@link #getMatchingCondition(javax.servlet.http.HttpServletRequest)}. * This method is provided as an alternative to be used if no request is available * (e.g. introspection, tooling, etc). * @param lookupPath the lookup path to match to existing patterns * @return a collection of matching patterns sorted with the closest match at the top */ public List<String> getMatchingPatterns(String lookupPath) { List<String> matches = new ArrayList<String>(); for (String pattern : this.patterns) { String match = getMatchingPattern(pattern, lookupPath); if (match != null) { matches.add(match); } } Collections.sort(matches, this.pathMatcher.getPatternComparator(lookupPath)); return matches; } private String getMatchingPattern(String pattern, String lookupPath) { if (pattern.equals(lookupPath)) { return pattern; } // 若是使用後綴匹配模式 且後臺配置的URI沒有後綴(不包含.),且前臺請求的URI中包含. 則在後臺配置的URI原來的匹配模式上加上 .* 再與前臺請求URI進行匹配進行匹配 if (this.useSuffixPatternMatch) { if (!this.fileExtensions.isEmpty() && lookupPath.indexOf('.') != -1) { for (String extension : this.fileExtensions) { if (this.pathMatcher.match(pattern + extension, lookupPath)) { return pattern + extension; } } } else { boolean hasSuffix = pattern.indexOf('.') != -1; if (!hasSuffix && this.pathMatcher.match(pattern + ".*", lookupPath)) { return pattern + ".*"; } } } if (this.pathMatcher.match(pattern, lookupPath)) { return pattern; } if (this.useTrailingSlashMatch) { if (!pattern.endsWith("/") && this.pathMatcher.match(pattern + "/", lookupPath)) { return pattern +"/"; } } return null; }
注意中文註釋的部分。咱們項目中這種不匹配的狀況是後臺只配置了/aaa/aaaa
而前臺配置的是/aaa/aaaa.json
,天然符合前面的模式匹配。天然,也就不會匹配不到後臺請求。
每一次採坑,都可以讓人前進。經過此次XSS事件,我這邊仍是有很多收穫的:
- 多想一想後門: 實現功能的時候要多考慮可能出現的意外狀況,實現功能很簡單,解決功能可能致使的問題倒是一個值的深刻思考的方向
- 源碼debug的重要性: 當有必定的編程經驗以後,不少問題須要耐心debug才能解決。經過這個debug的過程,不只可以瞭解底層的實現流程,也熟悉了大神寫的代碼。 何樂而不爲呢?
- SpringMVC前臺請求最好與後臺配置徹底匹配: 經過這篇文章相信你們也看到了,不匹配的後果。SpringMVC源碼中的省略號已經充分展現了他對你這種行爲的無語 😢
此次的博客就到這裏,一是篇幅太長了。怕吃太多消化不良,二來留點懸念,咱們下回分解(雖然不知道下回是啥時候了,哈哈哈 我儘快.....)
AntiSamy: https://blog.csdn.net/qq_35946990/article/details/74982760
XSS與CSRF: https://www.jianshu.com/p/64a413ada155