Microservices internal communication via InternalGateway #1653

0

gterdem created 3 years ago

Senior .NET Developer

We have two microservices; Microservice A and Microservice B, in Microservice A, we want to call some application service at Microservice B via sending HTTP request to InternalGateway.

You can not make direct request to InternalGateway to reach Microservice X, you make request to Microservice X through InternalGateway. Gateway works as a service locator to redirect your requests to related microservice and it does it automatically.

1- Is there any way to get token from Microservice A (from comming request or anywhere else) for use at http request to InternalGateway?Or, what is the best practice for this purpose?

You don't get any token from microservice. Token provider is AuthServer and most of the flow is automatically done by identityServer. Add the MicroserviceB.Application.Contracts project reference to MicroserviceA.Application.Contracts so that MicroserviceA application services become available in MicroserviceB.

2- Another question; If we gain Token with the way described in this link, How we can store this Token related to user session(User can log in into multi device then we can't store in Dictionary or something like that)

You can not store any token in any user session because there is no session. Server to server communication doesn't involve any user.

You don't need to make http request to AuthServer manually to get access token. Add Volo.Abp.Http.Client.IdentityModel.Web nuget package to your MicroserviceA.HttpApi.Host project. This package will automatically make request to get access token and add it to request header as bearer token. Make appsettings configurations for this package in like:

"IdentityClients": {
  "Default": {
    "GrantType": "client_credentials", // Don't change this
    "ClientId": "MicroserviceA",       // Caller
    "ClientSecret": "1q2w3e*",         // Secret you used when creating this client in IdentityServerDataSeeder
    "Authority": "https://myidentityserver.com", // AuthServer domain:port
    "Scope": "MicroserviceB"           // The resource you want to make request to 
  }
},

To make all of this work, you need to add MicroserviceA (makes the http request) as a client and MicroserviceB (the one you make the request to) as Api-Resource and Api-Scope. These configurations are done in IdentityServerDataSeeder file.

AdministrationService is also a client that makes request to IdentityService. You can check more about it in docs and in your template project. Here is how AdministrationService is configured as a client in IdentityServerDataSeeder just as example:

//Administration Service Client
await CreateClientAsync(
    name: "AdministrationService", // Your client Id.
    scopes: commonScopes.Union(new[]
    {
        "IdentityService"    //Allowed API scope
    }),
    grantTypes: new[] {"client_credentials"},
    secret: "1q2w3e*".Sha256(),     // The secret you use when making client_credential request
    permissions: new[] {IdentityPermissions.Users.Default}    // The permission you have over the method or appservice that you want to allow
);

Most of these subjects are related with IdentityServer and I suggest reading identity-server docs so that you can learn more about server-to-server (client_credential) communication and what abp helps about it.

0

yashar.aliabbasi created 3 years ago

I added the MicroserviceB.Application.Contracts reference into MicroserviceA.Application.Contracts Injected an application service of MicroserviceB into MicroserviceA's AppService but I got following error: Thank you.

0

ilker.sungur created 3 years ago

Is this thread still open? If it is; Can you please describe intercommunication best practice between microservices? In the documents of abp commercial; it is explained as "If you want OrderService to be able call the other services, you need to add the OrderService as a client under CreateClientsAsync as well. Then, update appsettings.json of the OrderService with IdentityClients section with the ClientId and granted scopes you have defined in CreateClientAsync method for client credential flow. Also, check microservice intercommunication docs for more information (TODO)." (link: https://docs.abp.io/en/commercial/latest/startup-templates/microservice/add-microservice#identityserver-configuration)

We are using commercial microservice template, and trying to access from MicroserviceA to MicroserviceB at application layer. "gterdem" explained that, "Add the MicroserviceB.Application.Contracts project reference to MicroserviceA.Application.Contracts so that MicroserviceA application services become available in MicroserviceB.". We did it, but how can we send request (or access) to MicroserviceB application service method?

It would be great if you could share an example code block or sample test project on this topic.

Thanks.

0

gterdem created 3 years ago

Senior .NET Developer

Can you please describe intercommunication best practice between microservices?

As far as I know, there is no best practice in microservice inter-communication since it depends on your needs. But if there is, it is probably loosely coupled, event based communication. Using distributed event bus (message-broker) is much more favored and meaningfull in most of the cases.

We are using commercial microservice template, and trying to access from MicroserviceA to MicroserviceB at application layer. "gterdem" explained that, "Add the MicroserviceB.Application.Contracts project reference to MicroserviceA.Application.Contracts so that MicroserviceA application services become available in MicroserviceB.". We did it, but how can we send request (or access) to MicroserviceB application service method?

That is correct. ApplicationService located in MicroserviceA.Application.Contracts is looking for implementation. Since its implementation is hosted remotely, the missing piece is adding the MicroserviceA.HttpApi.Client project reference to MicroserviceB.HttpApi.Host. It will make your application make a remote service request configured in your application service like:

"RemoteServices": {
  "Default": { 
    "BaseUrl": "https://localhost:44302/", //This must be Internalgateway endpoint
    "UseCurrentAccessToken": "false"
  }
},

You can observe the logs of your HttpApi.Host project to see if you have any problems related with IdentityServer.

To make this internal request work, you should have:

Added MicroserviceB as a client that allows which api you are making a request to as shown in this answer
Added Volo.Abp.Http.Client.IdentityModel.Web nuget package to your MicroserviceA.HttpApi.Host project as shown in this answer
Added Ocelot configuration in internal gateway appsettings

I'll prepare a detailed guide or a community article about this soon. Don't forget to check the log if you come across any errors.

0

ilker.sungur created 3 years ago

Thanks for your response gterdem. We added required references which you described. Then, how can we use MicroserviceA.HttpApi.Client from MicroserviceB.Application layer (like MicroserviceB.Application.OrderAppService) for requesting MicroserviceA application service (like MicroserviceA.Application.ProductAppService)?

It would be great if you describe with some psuedo code. ScreenShot from code attached below:

Thanks.

0

gterdem created 3 years ago

Senior .NET Developer

Thanks for your response gterdem. We added required references which you described. Then, how can we use MicroserviceA.HttpApi.Client from MicroserviceB.Application layer (like MicroserviceB.Application.OrderAppService) for requesting MicroserviceA application service (like MicroserviceA.Application.ProductAppService)?

It would be great if you describe with some psuedo code. ScreenShot from code attached below:

Thanks.

Inject IContentService located inside ContentService.Application.Contracts that you have already referenced and use it like your own. Only caveat here is:

Add ContentService.Http.Api.Client project reference as well (since IContentService implementation doesn't exist and it will be a remote service call) and it's appsettings configuration.
If your ContentService is authorized with permission, grant that permission from IdentityServer Management UI or from IdentityServerDataSeeder like below:

await CreateClientAsync(
    name: "FinanceService", // Your client Id.
    scopes: commonScopes.Union(new[]
    {
        "ContentService"    //Allowed API scope
    }),
    grantTypes: new[] {"client_credentials"},
    secret: "1q2w3e*".Sha256(),     // The secret you use when making client_credential request
    permissions: new[] {ContentServicePermissions.XXX.Default}    // The permission you have over the method or appservice that you want to allow
);

The permissions parameters is the permission list you allow for this client.

0

ilker.sungur created 3 years ago

Note: Requirement is accessing from FinanceService to ContentService over internal gateway We produced the steps you described as below:

1- Added ContentService.Application.Contracts reference to FinanceService.Application.Contracts project

2- Added ContentService.HttpApi.Client reference to FinanceService.HttpApi.host project

3- Created "FinanceService_Internal" client at IdentityDataSeeder and ran DbMigrator project (client added to db with required scope and permission)

4- Added required settings to appsetting.json for FinanceService.HttpApi.host project

5- Changed [RemoteService(IsEnabled = true)] for CategoryAppService at ContentService.Application layer

6- Injected required service interface at FinanceService.Application layer and trying to use it

7- After theese modifications, ran solution with tye

8- When testing FinanceService.PackagePriceAppService, (remote service interface injected application service), errors occured with "Autofac exceptions"

0

maliming created 3 years ago

Support Team Fullstack Developer

hi

1- Added ContentService.Application.Contracts reference to FinanceService.Application.Contracts project 2- Added ContentService.HttpApi.Client reference to FinanceService.HttpApi.host project

Are you add the module depend?

https://docs.abp.io/en/abp/latest/Module-Development-Basics#module-dependencies

0

ilker.sungur created 3 years ago

hi

1- Added ContentService.Application.Contracts reference to FinanceService.Application.Contracts project 2- Added ContentService.HttpApi.Client reference to FinanceService.HttpApi.host project

Are you add the module depend?

https://docs.abp.io/en/abp/latest/Module-Development-Basics#module-dependencies

Which module?

For testing, added typeof(ContentServiceApplicationContractsModule) dependency to FinanceServiceApplicationContractsModule but result is same, Autofac exception occured.

Is it required adding ContentServiceApplicationModule dependency to FinanceServiceServiceApplicationModule ? If it is, theese two microservices become tightly coupled to eachother, i think it is not suitable for microservice architecture.

We expected that, when requesting a ContentService's application service from FinanceService, FinanceService send http request to InternalGateway for accessing ContentService.

I think adding DLL or module referance is same as combining two microservice project in one project like monolith ?

Thanks

0

maliming created 3 years ago

Support Team Fullstack Developer

2- Added ContentService.HttpApi.Client reference to FinanceService.HttpApi.host project

Can you try to Depends the ContentServiceHttpApiClientModule On the FinanceServiceHttpApiHostModule

0

ilker.sungur created 3 years ago

It worked thank you. But one fresh error like, Authorization failed for RemoteService. Screen shot:

I have created the client for FinanceService at IdentityServerDataSeeder like (also gave all permission to newly created client at backoffice application Identity Server screens) :

await CreateClientAsync(
                name: "FinanceService_Internal", 
                scopes: commonScopes.Union(new[]
                {
                    "ContentService"
                }),
                grantTypes: new[] { "client_credentials" },
                secret: "secret".Sha256(),
                permissions: new[] { ContentServicePermissions.Categories.Default }
            );

And at FinanceService appsettings used like below:
  "RemoteServices": {
    "Default": {
      "BaseUrl": "https://localhost:44211/",
      "UseCurrentAccessToken": "false"
    }
  },
  "IdentityClients": {
    "Default": {
      "GrantType": "client_credentials", 
      "ClientId": "FinanceService_Internal",
      "ClientSecret": "secret",
      "Authority": "https://localhost:44111", //auth server url
      "Scope": "ContentService" 
    }
  },

0

maliming created 3 years ago

Support Team Fullstack Developer

hi

The Client need to grant the permissions.

0

ilker.sungur created 3 years ago

Hi maliming,

For testing purpose, all permission have been granted for related client, screen shot below:

For detailed debug, i have excluded Authserver, FinanceService, ContentService and InternalGateway projects from tye and ran all the projects. Below i am sharing some screen shots for debug logs

FinanceService (calling service) logs

ContentService (called service) logs

InternalGateway logs

Thanks

0

maliming created 3 years ago

Support Team Fullstack Developer

hi

Can you check the logs of Authserver?

Has it issued a token? Is the token forwarded by the gateway?

0

ilker.sungur created 3 years ago

hi maliming,

Checked Authserver logs, no request sent to Authserver.

Detailed check for InternalGateway logs, there seem to be some inconsistencies: First "no authentication needed for /api/content-service/categories" and "/api/content-service/{everything} route does not require user to be authorized" then "401 (Unauthorized) status code, request uri: https://localhost:44341/api/content-service/categories?SkipCount=0&MaxResultCount=10&api-version=1.0" Marked below:

[11:55:29 INF] Request starting HTTP/1.1 GET https://localhost:44211/api/content-service/categories?SkipCount=0&MaxResultCount=10&api-version=1.0 - -
[11:55:29 DBG] requestId: 0HMB24LT00SJK:00000003, previousRequestId: no previous request id, message: ocelot pipeline started
[11:55:29 DBG] requestId: 0HMB24LT00SJK:00000003, previousRequestId: no previous request id, message: Upstream url path is /api/content-service/categories
[11:55:29 DBG] requestId: 0HMB24LT00SJK:00000003, previousRequestId: no previous request id, message: downstream templates are /api/content-service/{everything}
[11:55:29 INF] requestId: 0HMB24LT00SJK:00000003, previousRequestId: no previous request id, message: EndpointRateLimiting is not enabled for /api/content-service/{everything}
> **[11:55:29 INF] requestId: 0HMB24LT00SJK:00000003, previousRequestId: no previous request id, message: No authentication needed for /api/content-service/categories**
> **[11:55:29 INF] requestId: 0HMB24LT00SJK:00000003, previousRequestId: no previous request id, message: /api/content-service/{everything} route does not require user to be authorized**
[11:55:29 DBG] requestId: 0HMB24LT00SJK:00000003, previousRequestId: no previous request id, message: Downstream url is https://localhost:44341/api/content-service/categories?SkipCount=0&MaxResultCount=10&api-version=1.0
> **[11:55:54 WRN] requestId: 0HMB24LT00SJK:00000003, previousRequestId: no previous request id, message: 401 (Unauthorized) status code, request uri: https://localhost:44341/api/content-service/categories?SkipCount=0&MaxResultCount=10&api-version=1.0**
[11:55:54 DBG] requestId: 0HMB24LT00SJK:00000003, previousRequestId: no previous request id, message: setting http response message
[11:55:54 DBG] requestId: 0HMB24LT00SJK:00000003, previousRequestId: no previous request id, message: no pipeline errors, setting and returning completed response
[11:55:54 DBG] requestId: 0HMB24LT00SJK:00000003, previousRequestId: no previous request id, message: ocelot pipeline finished
[11:55:54 INF] Request finished HTTP/1.1 GET https://localhost:44211/api/content-service/categories?SkipCount=0&MaxResultCount=10&api-version=1.0 - - - 401 159 application/json;+charset=utf-8 25041.3619ms

All steps have been done as gterdem explained but, Is there a missing or error in the appsettings files?

Thanks

0

maliming created 3 years ago

Support Team Fullstack Developer

hi

Can you use the template ms project to reproduce this?

You can share me the code and steps. liming.ma@volosoft.com

0

ilker.sungur created 3 years ago

hi,

i did not understand what you mean for "use the template ms project to reproduce this" ?

Project/solution structure is too big, min. 14 .sln file, I have shared part of realated documents (code, setting, debug logs,...) and steps at previus answers for this ticket. Is there any other solution than sharing all project structure?

Thanks

0

maliming created 3 years ago

Support Team Fullstack Developer

hi

I mean you can create a new microservice project and add some code to reproduce the problem.

abp new BookStoreMS -t microservice-pro

0

ilker.sungur created 3 years ago

hi

I mean you can create a new microservice project and add some code to reproduce the problem.
abp new BookStoreMS -t microservice-pro 

hi maliming,

i have created fresh microservice-pro project. The steps that followed: 1- Completed microservice solution settings for successful run and test. 2- Added new microservice (OrderService) to the solution, edited settings and controlled successful run 3- Added new entity (OrderInfo) to OrderService 4- Completed settings for accessing from OrderService to ProductService via InternalGateway (like described previuos answers) 5- Added required references to projects (like described previuos answers) 6- Added code to OrderService.OrderInfoAppService.GetListAsync() method for accessing ProductService.ProductAppService.GetListAsync() like below: var list = await _productAppService.GetListAsync(new GetProductsInput { }); 7- Run tye without AuthServer, InternalGateway, OrderService and ProductService 8- Run AuthServer, InternalGateway, OrderService and ProductService in debug mode 9- At web project, login and then clicked Order Infos menu item 10- Results were same ! .../api/product-service/products request sent from OrderService to InternalGateway InternalGateway logs: first: No authorization required for .../api/product-service/products, then: 401 unauthorized

I shared wetransfer link for solution zipFile to you via mail.

Thanks

0

maliming created 3 years ago

Support Team Fullstack Developer

hi I will check it assp.

0

maliming created 3 years ago

Support Team Fullstack Developer

hi

Please try to add AbpHttpClientIdentityModelModule to your OrderServiceHttpApiHostModule

0

ilker.sungur created 3 years ago

hi maliming,

Thank you for help, it worked perfectly.

I also strongly recommend that add all these information to the documentation for microservice inter-communication.

0

maliming created 3 years ago

Support Team Fullstack Developer

You're welcome, we will continue to improve the documentation.

0

ServiceBot created 3 years ago

Support Team Automatic process manager

This question has been automatically marked as stale because it has not had recent activity.

24 Answer(s)