Microservices on Kubernetes for Production Sites #3271

0

gterdem created 2 years ago

Senior .NET Developer

Hello, We provide helm charts for microservice templates. You can also check https://github.com/abpframework/eShopOnAbp/tree/main/etc/k8s/eshoponabp sample that has both publish scripts, domain names and values.azure.yaml with the deployed azure configurations.

0

thedatacrew created 2 years ago

Thank you, that's very helpful, you should reference that in the README.MD file under "Deploying in Production". How are you managing the Certificate is it using Let's Encrypt Cert Manager or is it a full-blown SAN / Wildcard Cert?

Thanks.

0

gterdem created 2 years ago

Senior .NET Developer

Thank you, that's very helpful, you should reference that in the README.MD file under "Deploying in Production". How are you managing the Certificate is it using Let's Encrypt Cert Manager or is it a full-blown SAN / Wildcard Cert?

Thanks.

We use Let's Encrypt in the sample. It is declared in each ingress as you can check https://github.com/abpframework/eShopOnAbp/blob/d261dd9c4f36ce68790458980d9c7b4fbe2fb268/etc/k8s/eshoponabp/charts/administration/templates/administration-ingress.yaml#L10

0

thedatacrew created 2 years ago

Great - I'm using Traefik as an Ingress Controller and it's also using Let's Encrypt and it all seems to be working with the changes. When I browse to the admin app (Blazor Server), I'm not seeing the Login Screen, it's completely blank.

I'm getting The cookie 'XSRF-TOKEN' has set 'SameSite=None' and must also set 'Secure'.

Do I need to implement https://community.abp.io/posts/patch-for-chrome-login-issue-identityserver4-samesite-cookie-problem-weypwp3n ? Or was this fixed in 5.2 ?

Thanks

0

gterdem created 2 years ago

Senior .NET Developer

Do I need to implement https://community.abp.io/posts/patch-for-chrome-login-issue-identityserver4-samesite-cookie-problem-weypwp3n ? Or was this fixed in 5.2 ?

It is not related to ABP so It is not something we can fix.

We also had problems with it so we implemented it: https://github.com/abpframework/eShopOnAbp/blob/d261dd9c4f36ce68790458980d9c7b4fbe2fb268/apps/auth-server/src/EShopOnAbp.AuthServer/SameSiteCookiesServiceCollectionExtensions.cs

0

thedatacrew created 2 years ago

I implemented this, but when I click login, it tries to redirect https://app.mydomain.com/Account/Login to http://auth.mydomain.com instead of https://auth.mydomain.com - I cannot find anywhere where is is configured to not use HTTPS.

0

gterdem created 2 years ago

Senior .NET Developer

You can check the IdentityServerDataSeeder file. It is located under both the DbMigrator and the IdentityService.HttpApi.Host projects

Whichever you are using to seed the data, check the appsettings.json file for IdentityServer initial data. You can examine the IdentityServerDataSeeder to learn where they are used and set.

0

thedatacrew created 2 years ago

Ok, thanks

In the dbmigrator appsettings.json there is this

In the Chart the Environments Variables maps don't match

I'm assuming these need to match for the container to override the config and seed the correct values.

0

gterdem created 2 years ago

Senior .NET Developer

If you are using Kubernetes, yes.

You need to check the IdentityService values to override.

0

thedatacrew created 2 years ago

Hi,

I implemented the SameSiteCookiesServiceCollectionExtensions.cs in the AuthServer using teh eshop examples - I'm assuming that this is the only place it is required?

Browsing to https://auth.mycompany.net I still get The cookie 'XSRF-TOKEN' has set 'SameSite=None' and must also set 'Secure'.

Also, both the https://www.mycompany.net and https://app.mycompany.net redirect the login to http://auth.mycompany.net not https which gives a 404.

I have check the configs and the databases and the seeded data - it all looks good.

I'm a bit flummoxed now. What can I look at next? Is their an opportunity to open a paid support ticket?

[09:41:30 INF] Bundled __bundles/Lepton.Global.9A9449B4A1BEC7DF689B1E3C3552F66F.js (736917 bytes)
[09:41:30 INF] Executed page /Account/Login in 3456.6196ms
[09:41:30 INF] Executed endpoint '/Account/Login'
[09:41:30 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/Account/Login - - - 200 - text/html;+charset=utf-8 3513.3771ms
[09:41:30 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/__bundles/Lepton.Global.ADD5F01D11E6ABD793872CD20AFE07ED.css?_v=637920924901301785 - -
[09:41:30 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/Abp/ApplicationConfigurationScript - -
[09:41:30 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/__bundles/Lepton.Global.9A9449B4A1BEC7DF689B1E3C3552F66F.js?_v=637920924909157339 - -
[09:41:30 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/Abp/ServiceProxyScript - -
[09:41:30 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/libs/timeago/locales/jquery.timeago.en.js?_v=637920493210000000 - -
[09:41:31 INF] Executing endpoint 'Volo.Abp.AspNetCore.Mvc.ProxyScripting.AbpServiceProxyScriptController.GetAll (Volo.Abp.AspNetCore.Mvc)'
[09:41:31 INF] Executing endpoint 'Volo.Abp.AspNetCore.Mvc.ApplicationConfigurations.AbpApplicationConfigurationScriptController.Get (Volo.Abp.AspNetCore.Mvc)'
[09:41:31 INF] Sending file. Request path: '/libs/timeago/locales/jquery.timeago.en.js'. Physical path: '/app/wwwroot/libs/timeago/locales/jquery.timeago.en.js'
[09:41:31 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/libs/timeago/locales/jquery.timeago.en.js?_v=637920493210000000 - - - 200 778 application/javascript 28.2478ms
[09:41:31 INF] Route matched with {area = "Abp", action = "GetAll", controller = "AbpServiceProxyScript", page = ""}. Executing controller action with signature Microsoft.AspNetCore.Mvc.ActionResult GetAll(Volo.Abp.AspNetCore.Mvc.ProxyScripting.ServiceProxyGenerationModel) on controller Volo.Abp.AspNetCore.Mvc.ProxyScripting.AbpServiceProxyScriptController (Volo.Abp.AspNetCore.Mvc).
[09:41:31 INF] Route matched with {area = "Abp", action = "Get", controller = "AbpApplicationConfigurationScript", page = ""}. Executing controller action with signature System.Threading.Tasks.Task`1[Microsoft.AspNetCore.Mvc.ActionResult] Get() on controller Volo.Abp.AspNetCore.Mvc.ApplicationConfigurations.AbpApplicationConfigurationScriptController (Volo.Abp.AspNetCore.Mvc).
[09:41:31 INF] Sending file. Request path: '/__bundles/Lepton.Global.ADD5F01D11E6ABD793872CD20AFE07ED.css'. Physical path: 'N/A'
[09:41:31 INF] Sending file. Request path: '/__bundles/Lepton.Global.9A9449B4A1BEC7DF689B1E3C3552F66F.js'. Physical path: 'N/A'
[09:41:31 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/__bundles/Lepton.Global.ADD5F01D11E6ABD793872CD20AFE07ED.css?_v=637920924901301785 - - - 200 507556 text/css 67.9968ms
[09:41:31 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/__bundles/Lepton.Global.9A9449B4A1BEC7DF689B1E3C3552F66F.js?_v=637920924909157339 - - - 200 738613 application/javascript 58.8700ms
[09:41:31 INF] Executing ContentResult with HTTP Response ContentType of application/javascript
[09:41:31 INF] Executed action Volo.Abp.AspNetCore.Mvc.ProxyScripting.AbpServiceProxyScriptController.GetAll (Volo.Abp.AspNetCore.Mvc) in 243.8057ms
[09:41:31 INF] Executed endpoint 'Volo.Abp.AspNetCore.Mvc.ProxyScripting.AbpServiceProxyScriptController.GetAll (Volo.Abp.AspNetCore.Mvc)'
[09:41:31 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/Abp/ServiceProxyScript - - - 200 1154 application/javascript 288.5070ms
[09:41:31 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/images/logo/logo-dark.png - -
[09:41:31 INF] Sending file. Request path: '/images/logo/logo-dark.png'. Physical path: '/app/wwwroot/images/logo/logo-dark.png'
[09:41:31 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/images/logo/logo-dark.png - - - 200 1386 image/png 7.1821ms
[09:41:31 INF] Request starting HTTP/1.1 GET http://auth.mycompany.net/libs/flag-icon-css/flags/1x1/gb.svg - -
[09:41:31 INF] Sending file. Request path: '/libs/flag-icon-css/flags/1x1/gb.svg'. Physical path: '/app/wwwroot/libs/flag-icon-css/flags/1x1/gb.svg'
[09:41:31 INF] Request finished HTTP/1.1 GET http://auth.mycompany.net/libs/flag-icon-css/flags/1x1/gb.svg - - - 200 538 image/svg+xml 0.5844ms
[09:41:33 WRN] The cookie 'XSRF-TOKEN' has set 'SameSite=None' and must also set 'Secure'.

0

gterdem created 2 years ago

Senior .NET Developer

Also, both the https://www.mycompany.net and https://app.mycompany.net redirect the login to http://auth.mycompany.net not https which gives a 404.

Setting auto-redirect in webserver from HTTP to HTTPS should fix this problem.

Browsing to https://auth.mycompany.net I still get The cookie 'XSRF-TOKEN' has set 'SameSite=None' and must also set 'Secure'.

What is this application? Angular back-office application or mvc application?

0

thedatacrew created 2 years ago

It's a Blazor Server for the APP (https://app.mycompany.net) and MVC for the Public Website (https://www.mycompany.net)

0

gterdem created 2 years ago

Senior .NET Developer

Browsing to https://auth.mycompany.net I still get The cookie 'XSRF-TOKEN' has set 'SameSite=None' and must also set 'Secure'.

It seems related to HTTPS redirection.

You can update your cookie configurations in your authserver application Module ServiceConfiguration:

...
.AddCookie("Cookies", options =>
{
    options.ExpireTimeSpan = TimeSpan.FromDays(365);

    options.Cookie.HttpOnly = true;
    options.Cookie.SameSite = SameSiteMode.None;
    options.Cookie.SecurePolicy = CookieSecurePolicy.Always;
})
...

See more about working with sameSite cookies.

However it is recommended to use the authentication server on https.

Redirection from http to https in authserver web-server configuration should fix this problem.

0

thedatacrew created 2 years ago

The Auth Server is set to use HTTPS in the configs and chart vaules. The redirect from the web apps is going to http. It's configured the same as the eShop example.

How does eShop auto redirect to HTTPS?

0

gterdem created 2 years ago

Senior .NET Developer

Https redirection is based on the webserver you are using. eShop is hosted on azure kubernetes cluster. You can check forced ssl-redirection: https://github.com/abpframework/eShopOnAbp/blob/d261dd9c4f36ce68790458980d9c7b4fbe2fb268/etc/k8s/eshoponabp/charts/authserver/templates/authserver-ingress.yaml#L7

You can google it if you are using IIS or Nginx for more accurate information.

0

thedatacrew created 2 years ago

Still stuck, I have gone back to basics and deploying a clean 5.3.1 site to K8S on Azure. Do these gateway reRoutes look correct to you or should they be https and have the FQDN public name?

0

alper created 2 years ago

Support Team Director

@thedatacrew what's the recent situation in your issue?

0

thedatacrew created 2 years ago

Pretty much just going around in circles not able to get even the basic demo app into a production K8s. Works fine with docker & Tye. There is just not enough information to get this working.

0

thedatacrew created 2 years ago

I reached out to support to get some consultancy hours booked in, although it should be documented well enough to work and the charts should be of good enough quality to be able to to deploy the solution to Azure/AWS/GKE.

I understand that the Microservices demo has moved to eShop but that doesn't help the OOTB Microservices templates which seem to have been forgotten and we don't need an ecommerce solution just a simple starting point.

0

alper created 2 years ago

Support Team Director

Microservices demo has moved to eShop

yes

we created eShopOnContainers to demonstrate how to deploy your services to cloud. as deployment process is custom to every customer, it's hard to cover all the could platforms with different deployment configurations. because each time we check, we see that the issues are more related to the 3rd party environments rather than the framework code itself. consultancy will work for you because they'll understand your requirements in deep.

20 Answer(s)