這篇文章,咱們將從Ocelot的中間件源碼分析,目前Ocelot已經實現那些功能,還有那些功能在咱們實際項目中暫時還未實現,若是咱們要使用這些功能,應該如何改造等方面來講明。html
在使用一個組件前,最好咱們要了解其中的一些原理,不然在使用過程當中遇到問題,也無從下手,今天我帶着你們一塊兒來解讀下Ocelot源碼,並梳理出具體實現的原理和流程,便於咱們根據需求擴展應用。
Ocelot源碼地址[https://github.com/ThreeMammals/Ocelot],
Ocelot文檔地址[https://ocelot.readthedocs.io/en/latest/]git
查看.NETCORE
相關中間件源碼,咱們優先找到入口方法,好比Ocelot中間件使用的是app.UseOcelot()
,咱們直接搜索UserOcelot,咱們會找到OcelotMiddlewareExtensions
方法,裏面是Ocelot中間件實際運行的方式和流程。
github
而後繼續順藤摸瓜,查看詳細的實現,咱們會發現以下代碼web
public static async Task<IApplicationBuilder> UseOcelot(this IApplicationBuilder builder, OcelotPipelineConfiguration pipelineConfiguration) { //建立配置信息 var configuration = await CreateConfiguration(builder); //監聽配置信息 ConfigureDiagnosticListener(builder); //建立執行管道 return CreateOcelotPipeline(builder, pipelineConfiguration); }
而後咱們繼續跟蹤到建立管道方法,能夠發現Ocelot的執行流程已經被找到,如今問題變的簡單了,直接查看數據庫
private static IApplicationBuilder CreateOcelotPipeline(IApplicationBuilder builder, OcelotPipelineConfiguration pipelineConfiguration) { var pipelineBuilder = new OcelotPipelineBuilder(builder.ApplicationServices); //詳細建立的管道順序在此方法 pipelineBuilder.BuildOcelotPipeline(pipelineConfiguration); var firstDelegate = pipelineBuilder.Build(); /* inject first delegate into first piece of asp.net middleware..maybe not like this then because we are updating the http context in ocelot it comes out correct for rest of asp.net.. */ builder.Properties["analysis.NextMiddlewareName"] = "TransitionToOcelotMiddleware"; builder.Use(async (context, task) => { var downstreamContext = new DownstreamContext(context); await firstDelegate.Invoke(downstreamContext); }); return builder; }
管道建立流程及實現,會不會感受到摸到大動脈了,核心的功能及原理基本找到了,那之後動手術也就能夠避開一些坑了,咱們能夠對着這個執行順序,再查看詳細的源碼,按照這個執行順序查看源碼,您就會發現整個思路很是清晰,每一步的實現一目瞭然。爲了更直觀的介紹源碼的解讀方式,這裏咱們就拿咱們後續要操刀的中間件來說解下中間件的具體實現。c#
public static class OcelotPipelineExtensions { public static OcelotRequestDelegate BuildOcelotPipeline(this IOcelotPipelineBuilder builder, OcelotPipelineConfiguration pipelineConfiguration) { // This is registered to catch any global exceptions that are not handled // It also sets the Request Id if anything is set globally builder.UseExceptionHandlerMiddleware(); // If the request is for websockets upgrade we fork into a different pipeline builder.MapWhen(context => context.HttpContext.WebSockets.IsWebSocketRequest, app => { app.UseDownstreamRouteFinderMiddleware(); app.UseDownstreamRequestInitialiser(); app.UseLoadBalancingMiddleware(); app.UseDownstreamUrlCreatorMiddleware(); app.UseWebSocketsProxyMiddleware(); }); // Allow the user to respond with absolutely anything they want. builder.UseIfNotNull(pipelineConfiguration.PreErrorResponderMiddleware); // This is registered first so it can catch any errors and issue an appropriate response builder.UseResponderMiddleware(); // Then we get the downstream route information builder.UseDownstreamRouteFinderMiddleware(); // This security module, IP whitelist blacklist, extended security mechanism builder.UseSecurityMiddleware(); //Expand other branch pipes if (pipelineConfiguration.MapWhenOcelotPipeline != null) { foreach (var pipeline in pipelineConfiguration.MapWhenOcelotPipeline) { builder.MapWhen(pipeline); } } // Now we have the ds route we can transform headers and stuff? builder.UseHttpHeadersTransformationMiddleware(); // Initialises downstream request builder.UseDownstreamRequestInitialiser(); // We check whether the request is ratelimit, and if there is no continue processing builder.UseRateLimiting(); // This adds or updates the request id (initally we try and set this based on global config in the error handling middleware) // If anything was set at global level and we have a different setting at re route level the global stuff will be overwritten // This means you can get a scenario where you have a different request id from the first piece of middleware to the request id middleware. builder.UseRequestIdMiddleware(); // Allow pre authentication logic. The idea being people might want to run something custom before what is built in. builder.UseIfNotNull(pipelineConfiguration.PreAuthenticationMiddleware); // Now we know where the client is going to go we can authenticate them. // We allow the ocelot middleware to be overriden by whatever the // user wants if (pipelineConfiguration.AuthenticationMiddleware == null) { builder.UseAuthenticationMiddleware(); } else { builder.Use(pipelineConfiguration.AuthenticationMiddleware); } // The next thing we do is look at any claims transforms in case this is important for authorisation builder.UseClaimsToClaimsMiddleware(); // Allow pre authorisation logic. The idea being people might want to run something custom before what is built in. builder.UseIfNotNull(pipelineConfiguration.PreAuthorisationMiddleware); // Now we have authenticated and done any claims transformation we // can authorise the request // We allow the ocelot middleware to be overriden by whatever the // user wants if (pipelineConfiguration.AuthorisationMiddleware == null) {//使用自定義認證,移除默認的認證方式 //builder.UseAuthorisationMiddleware(); } else { builder.Use(pipelineConfiguration.AuthorisationMiddleware); } // Now we can run the claims to headers transformation middleware builder.UseClaimsToHeadersMiddleware(); // Allow the user to implement their own query string manipulation logic builder.UseIfNotNull(pipelineConfiguration.PreQueryStringBuilderMiddleware); // Now we can run any claims to query string transformation middleware builder.UseClaimsToQueryStringMiddleware(); // Get the load balancer for this request builder.UseLoadBalancingMiddleware(); // This takes the downstream route we retrieved earlier and replaces any placeholders with the variables that should be used builder.UseDownstreamUrlCreatorMiddleware(); // Not sure if this is the best place for this but we use the downstream url // as the basis for our cache key. builder.UseOutputCacheMiddleware(); //We fire off the request and set the response on the scoped data repo builder.UseHttpRequesterMiddleware(); return builder.Build(); } private static void UseIfNotNull(this IOcelotPipelineBuilder builder, Func<DownstreamContext, Func<Task>, Task> middleware) { if (middleware != null) { builder.Use(middleware); } } }
限流中間件實現解析緩存
實現代碼以下builder.UseRateLimiting();
,咱們轉到定義,獲得以下代碼,詳細的實現邏輯在ClientRateLimitMiddleware
方法裏,繼續轉定義到這個方法,我把方法裏用到的內容註釋了下。websocket
public static class RateLimitMiddlewareExtensions { public static IOcelotPipelineBuilder UseRateLimiting(this IOcelotPipelineBuilder builder) { return builder.UseMiddleware<ClientRateLimitMiddleware>(); } } public class ClientRateLimitMiddleware : OcelotMiddleware { private readonly OcelotRequestDelegate _next; private readonly IRateLimitCounterHandler _counterHandler; private readonly ClientRateLimitProcessor _processor; public ClientRateLimitMiddleware(OcelotRequestDelegate next, IOcelotLoggerFactory loggerFactory, IRateLimitCounterHandler counterHandler) :base(loggerFactory.CreateLogger<ClientRateLimitMiddleware>()) { _next = next; _counterHandler = counterHandler; _processor = new ClientRateLimitProcessor(counterHandler); } //熟悉的Tnvoke方法,全部的邏輯都在此方法裏。 public async Task Invoke(DownstreamContext context) { var options = context.DownstreamReRoute.RateLimitOptions; // 校驗是否啓用限流配置 if (!context.DownstreamReRoute.EnableEndpointEndpointRateLimiting) {//未啓用直接進入下一個中間件 Logger.LogInformation($"EndpointRateLimiting is not enabled for {context.DownstreamReRoute.DownstreamPathTemplate.Value}"); await _next.Invoke(context); return; } // 獲取配置的校驗客戶端的方式 var identity = SetIdentity(context.HttpContext, options); // 校驗是否爲白名單 if (IsWhitelisted(identity, options)) {//白名單直接放行 Logger.LogInformation($"{context.DownstreamReRoute.DownstreamPathTemplate.Value} is white listed from rate limiting"); await _next.Invoke(context); return; } var rule = options.RateLimitRule; if (rule.Limit > 0) {//限流數是否大於0 // 獲取當前客戶端請求狀況,這裏須要注意_processor是從哪裏注入的,後續重 var counter = _processor.ProcessRequest(identity, options); // 校驗請求數是否大於限流數 if (counter.TotalRequests > rule.Limit) { //獲取下次有效請求的時間,就是避免每次請求,都校驗一次 var retryAfter = _processor.RetryAfterFrom(counter.Timestamp, rule); // 寫入日誌 LogBlockedRequest(context.HttpContext, identity, counter, rule, context.DownstreamReRoute); var retrystring = retryAfter.ToString(System.Globalization.CultureInfo.InvariantCulture); // 拋出超出限流異常並把下次可請求時間寫入header裏。 await ReturnQuotaExceededResponse(context.HttpContext, options, retrystring); return; } } //若是啓用了限流頭部 if (!options.DisableRateLimitHeaders) { var headers = _processor.GetRateLimitHeaders(context.HttpContext, identity, options); context.HttpContext.Response.OnStarting(SetRateLimitHeaders, state: headers); } //進入下一個中間件 await _next.Invoke(context); } public virtual ClientRequestIdentity SetIdentity(HttpContext httpContext, RateLimitOptions option) { var clientId = "client"; if (httpContext.Request.Headers.Keys.Contains(option.ClientIdHeader)) { clientId = httpContext.Request.Headers[option.ClientIdHeader].First(); } return new ClientRequestIdentity( clientId, httpContext.Request.Path.ToString().ToLowerInvariant(), httpContext.Request.Method.ToLowerInvariant() ); } public bool IsWhitelisted(ClientRequestIdentity requestIdentity, RateLimitOptions option) { if (option.ClientWhitelist.Contains(requestIdentity.ClientId)) { return true; } return false; } public virtual void LogBlockedRequest(HttpContext httpContext, ClientRequestIdentity identity, RateLimitCounter counter, RateLimitRule rule, DownstreamReRoute downstreamReRoute) { Logger.LogInformation( $"Request {identity.HttpVerb}:{identity.Path} from ClientId {identity.ClientId} has been blocked, quota {rule.Limit}/{rule.Period} exceeded by {counter.TotalRequests}. Blocked by rule { downstreamReRoute.UpstreamPathTemplate.OriginalValue }, TraceIdentifier {httpContext.TraceIdentifier}."); } public virtual Task ReturnQuotaExceededResponse(HttpContext httpContext, RateLimitOptions option, string retryAfter) { var message = string.IsNullOrEmpty(option.QuotaExceededMessage) ? $"API calls quota exceeded! maximum admitted {option.RateLimitRule.Limit} per {option.RateLimitRule.Period}." : option.QuotaExceededMessage; if (!option.DisableRateLimitHeaders) { httpContext.Response.Headers["Retry-After"] = retryAfter; } httpContext.Response.StatusCode = option.HttpStatusCode; return httpContext.Response.WriteAsync(message); } private Task SetRateLimitHeaders(object rateLimitHeaders) { var headers = (RateLimitHeaders)rateLimitHeaders; headers.Context.Response.Headers["X-Rate-Limit-Limit"] = headers.Limit; headers.Context.Response.Headers["X-Rate-Limit-Remaining"] = headers.Remaining; headers.Context.Response.Headers["X-Rate-Limit-Reset"] = headers.Reset; return Task.CompletedTask; } }
經過源碼解析,發現實現一個限流仍是很簡單的嗎!再進一步解析,IRateLimitCounterHandler
ClientRateLimitProcessor裏的相關接口
又是怎麼實現的呢?這時候咱們就須要瞭解下.NETCORE 的運行原理,其中ConfigureServices
方法實現了依賴注入(DI)的配置。這時候咱們看下Ocelot
是在哪裏進行注入的呢?app
services.AddOcelot()
是否是印象深入呢?原來全部的注入信息都寫在這裏,那麼問題簡單了,Ctrl+F
查找AddOcelot
方法,立刻就能定位到ServiceCollectionExtensions
方法,而後再轉到定義OcelotBuilder
asp.net
public static class ServiceCollectionExtensions { public static IOcelotBuilder AddOcelot(this IServiceCollection services) { var service = services.First(x => x.ServiceType == typeof(IConfiguration)); var configuration = (IConfiguration)service.ImplementationInstance; return new OcelotBuilder(services, configuration); } public static IOcelotBuilder AddOcelot(this IServiceCollection services, IConfiguration configuration) { return new OcelotBuilder(services, configuration); } }
又摸到大動脈啦,如今問題迎刃而解,原來全部的注入都寫在這裏,從這裏能夠找下咱們熟悉的幾個接口注入。
public OcelotBuilder(IServiceCollection services, IConfiguration configurationRoot) { Configuration = configurationRoot; Services = services; Services.Configure<FileConfiguration>(configurationRoot); Services.TryAddSingleton<IOcelotCache<FileConfiguration>, InMemoryCache<FileConfiguration>>(); Services.TryAddSingleton<IOcelotCache<CachedResponse>, InMemoryCache<CachedResponse>>(); Services.TryAddSingleton<IHttpResponseHeaderReplacer, HttpResponseHeaderReplacer>(); Services.TryAddSingleton<IHttpContextRequestHeaderReplacer, HttpContextRequestHeaderReplacer>(); Services.TryAddSingleton<IHeaderFindAndReplaceCreator, HeaderFindAndReplaceCreator>(); Services.TryAddSingleton<IInternalConfigurationCreator, FileInternalConfigurationCreator>(); Services.TryAddSingleton<IInternalConfigurationRepository, InMemoryInternalConfigurationRepository>(); Services.TryAddSingleton<IConfigurationValidator, FileConfigurationFluentValidator>(); Services.TryAddSingleton<HostAndPortValidator>(); Services.TryAddSingleton<IReRoutesCreator, ReRoutesCreator>(); Services.TryAddSingleton<IAggregatesCreator, AggregatesCreator>(); Services.TryAddSingleton<IReRouteKeyCreator, ReRouteKeyCreator>(); Services.TryAddSingleton<IConfigurationCreator, ConfigurationCreator>(); Services.TryAddSingleton<IDynamicsCreator, DynamicsCreator>(); Services.TryAddSingleton<ILoadBalancerOptionsCreator, LoadBalancerOptionsCreator>(); Services.TryAddSingleton<ReRouteFluentValidator>(); Services.TryAddSingleton<FileGlobalConfigurationFluentValidator>(); Services.TryAddSingleton<FileQoSOptionsFluentValidator>(); Services.TryAddSingleton<IClaimsToThingCreator, ClaimsToThingCreator>(); Services.TryAddSingleton<IAuthenticationOptionsCreator, AuthenticationOptionsCreator>(); Services.TryAddSingleton<IUpstreamTemplatePatternCreator, UpstreamTemplatePatternCreator>(); Services.TryAddSingleton<IRequestIdKeyCreator, RequestIdKeyCreator>(); Services.TryAddSingleton<IServiceProviderConfigurationCreator,ServiceProviderConfigurationCreator>(); Services.TryAddSingleton<IQoSOptionsCreator, QoSOptionsCreator>(); Services.TryAddSingleton<IReRouteOptionsCreator, ReRouteOptionsCreator>(); Services.TryAddSingleton<IRateLimitOptionsCreator, RateLimitOptionsCreator>(); Services.TryAddSingleton<IBaseUrlFinder, BaseUrlFinder>(); Services.TryAddSingleton<IRegionCreator, RegionCreator>(); Services.TryAddSingleton<IFileConfigurationRepository, DiskFileConfigurationRepository>(); Services.TryAddSingleton<IFileConfigurationSetter, FileAndInternalConfigurationSetter>(); Services.TryAddSingleton<IServiceDiscoveryProviderFactory, ServiceDiscoveryProviderFactory>(); Services.TryAddSingleton<ILoadBalancerFactory, LoadBalancerFactory>(); Services.TryAddSingleton<ILoadBalancerHouse, LoadBalancerHouse>(); Services.TryAddSingleton<IOcelotLoggerFactory, AspDotNetLoggerFactory>(); Services.TryAddSingleton<IRemoveOutputHeaders, RemoveOutputHeaders>(); Services.TryAddSingleton<IClaimToThingConfigurationParser, ClaimToThingConfigurationParser>(); Services.TryAddSingleton<IClaimsAuthoriser, ClaimsAuthoriser>(); Services.TryAddSingleton<IScopesAuthoriser, ScopesAuthoriser>(); Services.TryAddSingleton<IAddClaimsToRequest, AddClaimsToRequest>(); Services.TryAddSingleton<IAddHeadersToRequest, AddHeadersToRequest>(); Services.TryAddSingleton<IAddQueriesToRequest, AddQueriesToRequest>(); Services.TryAddSingleton<IClaimsParser, ClaimsParser>(); Services.TryAddSingleton<IUrlPathToUrlTemplateMatcher, RegExUrlMatcher>(); Services.TryAddSingleton<IPlaceholderNameAndValueFinder, UrlPathPlaceholderNameAndValueFinder>(); Services.TryAddSingleton<IDownstreamPathPlaceholderReplacer, DownstreamTemplatePathPlaceholderReplacer>(); Services.TryAddSingleton<IDownstreamRouteProvider, DownstreamRouteFinder>(); Services.TryAddSingleton<IDownstreamRouteProvider, DownstreamRouteCreator>(); Services.TryAddSingleton<IDownstreamRouteProviderFactory, DownstreamRouteProviderFactory>(); Services.TryAddSingleton<IHttpRequester, HttpClientHttpRequester>(); Services.TryAddSingleton<IHttpResponder, HttpContextResponder>(); Services.TryAddSingleton<IErrorsToHttpStatusCodeMapper, ErrorsToHttpStatusCodeMapper>(); Services.TryAddSingleton<IRateLimitCounterHandler, MemoryCacheRateLimitCounterHandler>(); Services.TryAddSingleton<IHttpClientCache, MemoryHttpClientCache>(); Services.TryAddSingleton<IRequestMapper, RequestMapper>(); Services.TryAddSingleton<IHttpHandlerOptionsCreator, HttpHandlerOptionsCreator>(); Services.TryAddSingleton<IDownstreamAddressesCreator, DownstreamAddressesCreator>(); Services.TryAddSingleton<IDelegatingHandlerHandlerFactory, DelegatingHandlerHandlerFactory>(); Services.TryAddSingleton<IHttpRequester, HttpClientHttpRequester>(); // see this for why we register this as singleton http://stackoverflow.com/questions/37371264/invalidoperationexception-unable-to-resolve-service-for-type-microsoft-aspnetc // could maybe use a scoped data repository Services.TryAddSingleton<IHttpContextAccessor, HttpContextAccessor>(); Services.TryAddSingleton<IRequestScopedDataRepository, HttpDataRepository>(); Services.AddMemoryCache(); Services.TryAddSingleton<OcelotDiagnosticListener>(); Services.TryAddSingleton<IMultiplexer, Multiplexer>(); Services.TryAddSingleton<IResponseAggregator, SimpleJsonResponseAggregator>(); Services.TryAddSingleton<ITracingHandlerFactory, TracingHandlerFactory>(); Services.TryAddSingleton<IFileConfigurationPollerOptions, InMemoryFileConfigurationPollerOptions>(); Services.TryAddSingleton<IAddHeadersToResponse, AddHeadersToResponse>(); Services.TryAddSingleton<IPlaceholders, Placeholders>(); Services.TryAddSingleton<IResponseAggregatorFactory, InMemoryResponseAggregatorFactory>(); Services.TryAddSingleton<IDefinedAggregatorProvider, ServiceLocatorDefinedAggregatorProvider>(); Services.TryAddSingleton<IDownstreamRequestCreator, DownstreamRequestCreator>(); Services.TryAddSingleton<IFrameworkDescription, FrameworkDescription>(); Services.TryAddSingleton<IQoSFactory, QoSFactory>(); Services.TryAddSingleton<IExceptionToErrorMapper, HttpExeptionToErrorMapper>(); //add security this.AddSecurity(); //add asp.net services.. var assembly = typeof(FileConfigurationController).GetTypeInfo().Assembly; Services.AddMvcCore() .AddApplicationPart(assembly) .AddControllersAsServices() .AddAuthorization() .AddJsonFormatters(); Services.AddLogging(); Services.AddMiddlewareAnalysis(); Services.AddWebEncoders(); }
至此Ocelot
源碼解析就到這裏了,其餘的具體實現代碼就根據流程一個一個查看便可,這裏就不詳細講解了,由於咱們已經掌握整個Ocelot代碼的運行原理和實現方式及流程,項目裏其餘的一大堆的代碼都是圍繞這個流程去一步一步實現的。
有沒有感受添加一箇中間件不是很複雜呢,是否是都躍躍欲試,準備嘗試開發本身的自定義中間件啦,本篇就不介紹中間件的具體開發流程了,後續實戰中會包含部分項目中須要用到的中間件,到時候會詳細講解如何規劃和開發一個知足本身項目需求的中間件。
在完整學習完Ocelot文檔和源碼後,咱們基本掌握了Ocelot目前已經實現的功能,再結合咱們實際項目需求,咱們梳理下還有哪些功能可能須要本身擴展實現。
項目設計網關基本需求包括路由、認證、受權、限流、緩存,仔細學習文檔和源碼後發現功能都已經存在,那是否是咱們就能夠直接拿來使用呢?這時候咱們須要拿出一些複雜業務場景來對號入座,看可否實現複雜場景的一些應用。
一、受權
可否爲每個客戶端設置獨立的訪問權限,若是客戶端A能夠訪問服務A、服務B,客戶端B只能訪問服務A,從網關層面直接受權,不知足需求不路由到具體服務。從文檔和代碼分析後發現暫時未實現。
二、限流
可否爲每個客戶端設置不能限流規則,例如客戶端A爲咱們內容應用,我但願對服務A不啓用限流,客戶端B爲第三方接入應用,我須要B訪問服務A訪問進行單獨限流(30次/分鐘),看可否經過配置實現自定義限流。從文檔和代碼分析後發現暫時未實現。
三、緩存
經過代碼發現目前緩存實現的只是Dictionary方式實現的緩存,不能實現分佈式結構的應用。
經過分析咱們發現列舉的5個基本需求,盡然有3個在咱們實際項目應用中可能會存在問題,若是不解決這些問題,很難直接拿這個完美的網關項目應用到正式項目,因此咱們到經過擴展Ocelot方法來實現咱們的目的。
如何擴展呢
爲了知足咱們項目應用的須要,咱們須要爲每個路由進行單獨設置,若是還採用配置文件的方式,確定沒法知足需求,且後續網關動態增長路由、受權、限流等沒法控制,因此咱們須要把網關配置信息從配置文件中移到數據庫中,由數據庫中的路由表、限流表、受權表等方式記錄當前網關的應用,且後續擴展直接在數據庫中增長或減小相關配置,而後動態更新網關配置實現網關的高可用。
想想是否是有點小激動,原來只要稍微改造下寶駿瞬間變寶馬,那接下來的課程就是網關改造之旅,我會從設計、思想、編碼等方面講解下如何實現咱們的第一輛寶馬。
本系列文章我也是邊想邊寫邊實現,若是發現中間有任何描述或實現不當的地方,也請各位大神批評指正,我會第一時間整理並修正,避免讓後續學習的人走彎路。