Ends in:
2 DAYS
5 HRS
28 MIN
14 SEC
Ends in:
2 D
5 H
28 M
14 S
Open Closed

Microservices on Kubernetes for Production Sites #3271


User avatar
0
thedatacrew created

Hi,

Is their some guidance on the configuration of the Microservices Templates when moved to a production environment i.e. with real domain names. What apps/services/gateways need external ingress and an external domains and which chart values need changing?

I can't find anything in the documentation regarding making and deploying a production version of the software.

Thanks


20 Answer(s)
  • User Avatar
    0
    gterdem created
    Senior .NET Developer

    Hello, We provide helm charts for microservice templates. You can also check https://github.com/abpframework/eShopOnAbp/tree/main/etc/k8s/eshoponabp sample that has both publish scripts, domain names and values.azure.yaml with the deployed azure configurations.

  • User Avatar
    0
    thedatacrew created

    Thank you, that's very helpful, you should reference that in the README.MD file under "Deploying in Production". How are you managing the Certificate is it using Let's Encrypt Cert Manager or is it a full-blown SAN / Wildcard Cert?

    Thanks.

  • User Avatar
    0
    gterdem created
    Senior .NET Developer

    Thank you, that's very helpful, you should reference that in the README.MD file under "Deploying in Production". How are you managing the Certificate is it using Let's Encrypt Cert Manager or is it a full-blown SAN / Wildcard Cert?

    Thanks.

    We use Let's Encrypt in the sample. It is declared in each ingress as you can check https://github.com/abpframework/eShopOnAbp/blob/d261dd9c4f36ce68790458980d9c7b4fbe2fb268/etc/k8s/eshoponabp/charts/administration/templates/administration-ingress.yaml#L10

  • User Avatar
    0
    thedatacrew created

    Great - I'm using Traefik as an Ingress Controller and it's also using Let's Encrypt and it all seems to be working with the changes. When I browse to the admin app (Blazor Server), I'm not seeing the Login Screen, it's completely blank.

    I'm getting The cookie 'XSRF-TOKEN' has set 'SameSite=None' and must also set 'Secure'.

    Do I need to implement https://community.abp.io/posts/patch-for-chrome-login-issue-identityserver4-samesite-cookie-problem-weypwp3n ? Or was this fixed in 5.2 ?

    Thanks

  • User Avatar
    0
    gterdem created
    Senior .NET Developer

    Do I need to implement https://community.abp.io/posts/patch-for-chrome-login-issue-identityserver4-samesite-cookie-problem-weypwp3n ? Or was this fixed in 5.2 ?

    It is not related to ABP so It is not something we can fix.

    We also had problems with it so we implemented it: https://github.com/abpframework/eShopOnAbp/blob/d261dd9c4f36ce68790458980d9c7b4fbe2fb268/apps/auth-server/src/EShopOnAbp.AuthServer/SameSiteCookiesServiceCollectionExtensions.cs

  • User Avatar
    0
    thedatacrew created

    I implemented this, but when I click login, it tries to redirect https://app.mydomain.com/Account/Login to http://auth.mydomain.com instead of https://auth.mydomain.com - I cannot find anywhere where is is configured to not use HTTPS.

  • User Avatar
    0
    gterdem created
    Senior .NET Developer

    You can check the IdentityServerDataSeeder file. It is located under both the DbMigrator and the IdentityService.HttpApi.Host projects

    Whichever you are using to seed the data, check the appsettings.json file for IdentityServer initial data. You can examine the IdentityServerDataSeeder to learn where they are used and set.

  • User Avatar
    0
    thedatacrew created

    Ok, thanks

    In the dbmigrator appsettings.json there is this

    In the Chart the Environments Variables maps don't match

    I'm assuming these need to match for the container to override the config and seed the correct values.

  • User Avatar
    0
    gterdem created
    Senior .NET Developer

    If you are using Kubernetes, yes.

    You need to check the IdentityService values to override.

  • User Avatar
    0
    thedatacrew created

    Hi,

    I implemented the SameSiteCookiesServiceCollectionExtensions.cs in the AuthServer using teh eshop examples - I'm assuming that this is the only place it is required?

    Browsing to https://auth.mycompany.net I still get The cookie 'XSRF-TOKEN' has set 'SameSite=None' and must also set 'Secure'.

    Also, both the https://www.mycompany.net and https://app.mycompany.net redirect the login to http://auth.mycompany.net not https which gives a 404.

    I have check the configs and the databases and the seeded data - it all looks good.

    I'm a bit flummoxed now. What can I look at next? Is their an opportunity to open a paid support ticket?

    [09:41:30 INF] Bundled __bundles/Lepton.Global.9A9449B4A1BEC7DF689B1E3C3552F66F.js (736917 bytes)
    [09:41:30 INF] Executed page /Account/Login in 3456.6196ms
    [09:41:30 INF] Executed endpoint '/Account/Login'
    [09:41:30 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/Account/Login - - - 200 - text/html;+charset=utf-8 3513.3771ms
    [09:41:30 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/__bundles/Lepton.Global.ADD5F01D11E6ABD793872CD20AFE07ED.css?_v=637920924901301785 - -
    [09:41:30 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/Abp/ApplicationConfigurationScript - -
    [09:41:30 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/__bundles/Lepton.Global.9A9449B4A1BEC7DF689B1E3C3552F66F.js?_v=637920924909157339 - -
    [09:41:30 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/Abp/ServiceProxyScript - -
    [09:41:30 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/libs/timeago/locales/jquery.timeago.en.js?_v=637920493210000000 - -
    [09:41:31 INF] Executing endpoint 'Volo.Abp.AspNetCore.Mvc.ProxyScripting.AbpServiceProxyScriptController.GetAll (Volo.Abp.AspNetCore.Mvc)'
    [09:41:31 INF] Executing endpoint 'Volo.Abp.AspNetCore.Mvc.ApplicationConfigurations.AbpApplicationConfigurationScriptController.Get (Volo.Abp.AspNetCore.Mvc)'
    [09:41:31 INF] Sending file. Request path: '/libs/timeago/locales/jquery.timeago.en.js'. Physical path: '/app/wwwroot/libs/timeago/locales/jquery.timeago.en.js'
    [09:41:31 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/libs/timeago/locales/jquery.timeago.en.js?_v=637920493210000000 - - - 200 778 application/javascript 28.2478ms
    [09:41:31 INF] Route matched with {area = "Abp", action = "GetAll", controller = "AbpServiceProxyScript", page = ""}. Executing controller action with signature Microsoft.AspNetCore.Mvc.ActionResult GetAll(Volo.Abp.AspNetCore.Mvc.ProxyScripting.ServiceProxyGenerationModel) on controller Volo.Abp.AspNetCore.Mvc.ProxyScripting.AbpServiceProxyScriptController (Volo.Abp.AspNetCore.Mvc).
    [09:41:31 INF] Route matched with {area = "Abp", action = "Get", controller = "AbpApplicationConfigurationScript", page = ""}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.ActionResult] Get() on controller Volo.Abp.AspNetCore.Mvc.ApplicationConfigurations.AbpApplicationConfigurationScriptController (Volo.Abp.AspNetCore.Mvc).
    [09:41:31 INF] Sending file. Request path: '/__bundles/Lepton.Global.ADD5F01D11E6ABD793872CD20AFE07ED.css'. Physical path: 'N/A'
    [09:41:31 INF] Sending file. Request path: '/__bundles/Lepton.Global.9A9449B4A1BEC7DF689B1E3C3552F66F.js'. Physical path: 'N/A'
    [09:41:31 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/__bundles/Lepton.Global.ADD5F01D11E6ABD793872CD20AFE07ED.css?_v=637920924901301785 - - - 200 507556 text/css 67.9968ms
    [09:41:31 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/__bundles/Lepton.Global.9A9449B4A1BEC7DF689B1E3C3552F66F.js?_v=637920924909157339 - - - 200 738613 application/javascript 58.8700ms
    [09:41:31 INF] Executing ContentResult with HTTP Response ContentType of application/javascript
    [09:41:31 INF] Executed action Volo.Abp.AspNetCore.Mvc.ProxyScripting.AbpServiceProxyScriptController.GetAll (Volo.Abp.AspNetCore.Mvc) in 243.8057ms
    [09:41:31 INF] Executed endpoint 'Volo.Abp.AspNetCore.Mvc.ProxyScripting.AbpServiceProxyScriptController.GetAll (Volo.Abp.AspNetCore.Mvc)'
    [09:41:31 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/Abp/ServiceProxyScript - - - 200 1154 application/javascript 288.5070ms
    [09:41:31 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/images/logo/logo-dark.png - -
    [09:41:31 INF] Sending file. Request path: '/images/logo/logo-dark.png'. Physical path: '/app/wwwroot/images/logo/logo-dark.png'
    [09:41:31 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/images/logo/logo-dark.png - - - 200 1386 image/png 7.1821ms
    [09:41:31 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/libs/flag-icon-css/flags/1x1/gb.svg - -
    [09:41:31 INF] Sending file. Request path: '/libs/flag-icon-css/flags/1x1/gb.svg'. Physical path: '/app/wwwroot/libs/flag-icon-css/flags/1x1/gb.svg'
    [09:41:31 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/libs/flag-icon-css/flags/1x1/gb.svg - - - 200 538 image/svg+xml 0.5844ms
    [09:41:33 WRN] The cookie 'XSRF-TOKEN' has set 'SameSite=None' and must also set 'Secure'.
    
  • User Avatar
    0
    gterdem created
    Senior .NET Developer

    Also, both the https://www.mycompany.net and https://app.mycompany.net redirect the login to http://auth.mycompany.net not https which gives a 404.

    Setting auto-redirect in webserver from HTTP to HTTPS should fix this problem.

    Browsing to https://auth.mycompany.net I still get The cookie 'XSRF-TOKEN' has set 'SameSite=None' and must also set 'Secure'.

    What is this application? Angular back-office application or mvc application?

  • User Avatar
    0
    thedatacrew created

    It's a Blazor Server for the APP (https://app.mycompany.net) and MVC for the Public Website (https://www.mycompany.net)

  • User Avatar
    0
    gterdem created
    Senior .NET Developer

    Browsing to https://auth.mycompany.net I still get The cookie 'XSRF-TOKEN' has set 'SameSite=None' and must also set 'Secure'.

    It seems related to HTTPS redirection.

    You can update your cookie configurations in your authserver application Module ServiceConfiguration:

    ...
    .AddCookie("Cookies", options =>
    {
        options.ExpireTimeSpan = TimeSpan.FromDays(365);
    
        options.Cookie.HttpOnly = true;
        options.Cookie.SameSite = SameSiteMode.None;
        options.Cookie.SecurePolicy = CookieSecurePolicy.Always;
    })
    ...
    

    See more about working with sameSite cookies.

    However it is recommended to use the authentication server on https.

    Redirection from http to https in authserver web-server configuration should fix this problem.

  • User Avatar
    0
    thedatacrew created

    The Auth Server is set to use HTTPS in the configs and chart vaules. The redirect from the web apps is going to http. It's configured the same as the eShop example.

    How does eShop auto redirect to HTTPS?

  • User Avatar
    0
    gterdem created
    Senior .NET Developer

    Https redirection is based on the webserver you are using. eShop is hosted on azure kubernetes cluster. You can check forced ssl-redirection: https://github.com/abpframework/eShopOnAbp/blob/d261dd9c4f36ce68790458980d9c7b4fbe2fb268/etc/k8s/eshoponabp/charts/authserver/templates/authserver-ingress.yaml#L7

    You can google it if you are using IIS or Nginx for more accurate information.

  • User Avatar
    0
    thedatacrew created

    Still stuck, I have gone back to basics and deploying a clean 5.3.1 site to K8S on Azure. Do these gateway reRoutes look correct to you or should they be https and have the FQDN public name?

  • User Avatar
    0
    alper created
    Support Team Director

    @thedatacrew what's the recent situation in your issue?

  • User Avatar
    0
    thedatacrew created

    Pretty much just going around in circles not able to get even the basic demo app into a production K8s. Works fine with docker & Tye. There is just not enough information to get this working.

  • User Avatar
    0
    thedatacrew created

    I reached out to support to get some consultancy hours booked in, although it should be documented well enough to work and the charts should be of good enough quality to be able to to deploy the solution to Azure/AWS/GKE.

    I understand that the Microservices demo has moved to eShop but that doesn't help the OOTB Microservices templates which seem to have been forgotten and we don't need an ecommerce solution just a simple starting point.

  • User Avatar
    0
    alper created
    Support Team Director

    Microservices demo has moved to eShop

    yes

    we created eShopOnContainers to demonstrate how to deploy your services to cloud. as deployment process is custom to every customer, it's hard to cover all the could platforms with different deployment configurations. because each time we check, we see that the issues are more related to the 3rd party environments rather than the framework code itself. consultancy will work for you because they'll understand your requirements in deep.

Made with ❤️ on ABP v9.1.0-preview. Updated on December 02, 2024, 12:35