Multi-tenant Domain Resolving in Microservice Solution
This documentation introduces guidance for adding multi-tenant domain resolver to your microservice solution. This guide will use mystore as the main project name and you can simply replace it with your own project name to adept your solution. This guide will be using existing helm charts for deploying this solution on local kubernetes cluster with mystore domain name.
Configuring TenantResolver
DomainTenantResolver
is added via AbpTenantResolveOptions
. Since all the applications, gateways and the microservices should resolve the subdomain as tenant, it can be configured it in Shared.Hosting.AspNetCore
module class to make it available for all of them instead of repeating the same configuration.
Navigate to the MyStoreSharedHostingAspNetCoreModule
and add the following configuration to ConfigureServices method:
var configuration = context.Services.GetConfiguration();
Configure<AbpTenantResolveOptions>(options =>
{
options.AddDomainTenantResolver(configuration["TenantDomain"]);
});
This configuration will allow to add a overridable TenantDomain key to the all applications, gateways and microservices' appsettings which will resolve the tenant based on the configuration.
If the
AbpTenantResolveOptions
namespace can not be found, add theVolo.Abp.AspNetCore.MultiTenancy
package to your project. You can simply useabp add-package Volo.Abp.AspNetCore.MultiTenancy
under the MyStore.SharedHosting.AspNetCore project.
Now add this configuration to all the application, gateway and microservice appsettings:
"TenantDomain": "https://{0}.authserver.mystore.dev" // in AuthServer appsettings.json
"TenantDomain": "https://{0}.mystore.dev" // in PublicWeb appsettings.json
"TenantDomain": "https://{0}.gateway-public-web.mystore.dev" // in PublicWebGateway appsettings.json
"TenantDomain": "https://{0}.gateway-web.mystore.dev" // in WebGateway appsettings.json
"TenantDomain": "https://{0}.administration.mystore.dev" // in AdministrationService appsettings.json
"TenantDomain": "https://{0}.identity.mystore.dev" // in IdentityService appsettings.json
"TenantDomain": "https://{0}.saas.mystore.dev" // in SaasService appsettings.json
"TenantDomain": "https://{0}.product.mystore.dev" // in ProductService appsettings.json
This configuration will allow subdomain tenant resolving. Ex, if you have a tenant named Volosoft, the applications will resolve this tenant when you navigate to https://volosoft.authserver.mystore.dev
(or any other app/gateway/microservice).
If you have more microservices, you need to add this configurations to your other microservice's appsettings.json aswell.
For angular application, you don't need to add anything since we will be overriding the angular environment via kubernetes values file.
Configuring AuthServer
When the tenant try to login from an application (Ex https://volosoft.angular.mystore.dev
) it will be redirected to AuthServer (https://volosoft.authserver.mystore.dev
) and you will be seeing a HTTP 400 error related to invalid redirect_uri
. If you check the authserver application logs (under Logs/logs.txt file or console logs), you will notice that the https://volosoft.angular.mystore.dev
is not a valid redirect_uri since it is not been seeded by the OpenIddictDataSeeder. Only the host applications, gateways and microservice URLs are seeded (like https://angular.mystore.dev
).
Instead of trying to manually add all the tenant domains, you can use enable AbpOpenIddict wildcard domain support. That will enable the subdomain support, saving you from manually entering all the redirect and post_logout redirect URIs.
To enable it, navigate to MyStoreAuthServerModule and update the PreConfigureServices
method:
var hostingEnvironment = context.Services.GetHostingEnvironment();
var configuration = context.Services.GetConfiguration();
PreConfigure<AbpOpenIddictWildcardDomainOptions>(options =>
{
options.EnableWildcardDomainSupport = true;
options.WildcardDomainsFormat.Add(configuration["WildCardDomains:AuthServer"]);
options.WildcardDomainsFormat.Add(configuration["WildCardDomains:Angular"]);
options.WildcardDomainsFormat.Add(configuration["WildCardDomains:PublicWeb"]);
options.WildcardDomainsFormat.Add(configuration["WildCardDomains:WebGateway"]);
options.WildcardDomainsFormat.Add(configuration["WildCardDomains:PublicWebGateway"]);
options.WildcardDomainsFormat.Add(configuration["WildCardDomains:IdentityService"]);
options.WildcardDomainsFormat.Add(configuration["WildCardDomains:AdministrationService"]);
options.WildcardDomainsFormat.Add(configuration["WildCardDomains:SaasService"]);
options.WildcardDomainsFormat.Add(configuration["WildCardDomains:ProductService"]);
});
This configuration will automatically handle the redirect and post_logout redirect URIs.
Add the configurations to the appsettings.json
file of the AuthServer project:
"WildCardDomains": {
"AuthServer": "https://{0}.authserver.mystore.dev",
"Angular": "https://{0}.angular.mystore.dev",
"PublicWeb": "https://{0}.mystore.dev",
"WebGateway": "https://{0}.gateway-web.mystore.dev",
"PublicWebGateway": "https://{0}.gateway-public-web.mystore.dev",
"IdentityService": "https://{0}.identity.mystore.dev",
"AdministrationService": "https://{0}.administration.mystore.dev",
"SaasService": "https://{0}.saas.mystore.dev",
"ProductService": "https://{0}.product.mystore.dev"
},
If you have more microservices or applications, you need to add them to handle the redirect and post_logout redirect URIs of them aswell.
Configuring PublicWeb/Web/Blazor Server
The server-side rendering applications like public-web, web and blazor-server apps are using hybrid flow authorization flow. Which contains additional configuration on the module when hosted on containerized environment:
if (Convert.ToBoolean(configuration["AuthServer:IsOnK8s"]))
{
context.Services.Configure<OpenIdConnectOptions>("oidc", options =>
{
options.MetadataAddress = configuration["AuthServer:MetaAddress"]!.EnsureEndsWith('/') +
".well-known/openid-configuration";
var previousOnRedirectToIdentityProvider = options.Events.OnRedirectToIdentityProvider;
options.Events.OnRedirectToIdentityProvider = async ctx =>
{
// Intercept the redirection so the browser navigates to the right URL in your host
ctx.ProtocolMessage.IssuerAddress = configuration["AuthServer:Authority"]!.EnsureEndsWith('/') + "connect/authorize";
if (previousOnRedirectToIdentityProvider != null)
{
await previousOnRedirectToIdentityProvider(ctx);
}
};
var previousOnRedirectToIdentityProviderForSignOut = options.Events.OnRedirectToIdentityProviderForSignOut;
options.Events.OnRedirectToIdentityProviderForSignOut = async ctx =>
{
// Intercept the redirection for signout so the browser navigates to the right URL in your host
ctx.ProtocolMessage.IssuerAddress = configuration["AuthServer:Authority"]!.EnsureEndsWith('/') + "connect/logout";
if (previousOnRedirectToIdentityProviderForSignOut != null)
{
await previousOnRedirectToIdentityProviderForSignOut(ctx);
}
};
});
}
This configuration contains the real DNS of the AuthServer (configuration["AuthServer:Authority"]) and the .well-known endpoint (configuration["AuthServer:MetaAddress"]) used to obtain the tokens fromthe internal network. The configuration intercepts the login and logout requests from the browser to redirect to real DNS. When the tenant try to login from the one of these application (Ex https://volosoft.mystore.dev
) it should be redirected to tenant's AuthServer (https://volosoft.authserver.mystore.dev
) instead of the host's AuthServer (https://authserver.mystore.dev
).
You need to update this configuration as below to support this functionality:
if (Convert.ToBoolean(configuration["AuthServer:IsOnK8s"]))
{
context.Services.Configure<OpenIdConnectOptions>("oidc", options =>
{
options.MetadataAddress = configuration["AuthServer:MetaAddress"]!.EnsureEndsWith('/') +
".well-known/openid-configuration";
var previousOnRedirectToIdentityProvider = options.Events.OnRedirectToIdentityProvider;
options.Events.OnRedirectToIdentityProvider = async ctx =>
{
// Intercept the redirection so the browser navigates to the right URL in your host
ctx.ProtocolMessage.IssuerAddress = configuration["AuthServer:Authority"]!.EnsureEndsWith('/') + "connect/authorize";
// Resolve the current tenant
var currentTenant = ctx.HttpContext.RequestServices.GetRequiredService<ICurrentTenant>();
var tenantDomain = configuration["TenantDomain"];
// Check if the current tenant is available and the solution is using domain tenant resolver
if (currentTenant.IsAvailable && !string.IsNullOrEmpty(tenantDomain))
{
// Replace "{0}"" string for the tenant authserver. The authority value should be replaced from
// https://{0}.authserver.mystore.dev to https://tenantName.authserver.mystore.dev
ctx.ProtocolMessage.IssuerAddress = ctx.ProtocolMessage.IssuerAddress.Replace("{0}", $"{currentTenant.Name}");
}
else
{
// Keep using the host authserver if there is no tenant
ctx.ProtocolMessage.IssuerAddress = ctx.ProtocolMessage.IssuerAddress.Replace("{0}.", string.Empty);
}
if (previousOnRedirectToIdentityProvider != null)
{
await previousOnRedirectToIdentityProvider(ctx);
}
};
var previousOnRedirectToIdentityProviderForSignOut = options.Events.OnRedirectToIdentityProviderForSignOut;
// Similar configuration for Logout request
options.Events.OnRedirectToIdentityProviderForSignOut = async ctx =>
{
// Intercept the redirection for signout so the browser navigates to the right URL in your host
ctx.ProtocolMessage.IssuerAddress = configuration["AuthServer:Authority"]!.EnsureEndsWith('/') + "connect/logout";
var currentTenant = ctx.HttpContext.RequestServices.GetRequiredService<ICurrentTenant>();
var tenantDomain = configuration["TenantDomain"];
if (currentTenant.IsAvailable && !string.IsNullOrEmpty(tenantDomain))
{
ctx.ProtocolMessage.IssuerAddress = ctx.ProtocolMessage.IssuerAddress.Replace("{0}", $"{currentTenant.Name}");
}
else
{
ctx.ProtocolMessage.IssuerAddress = ctx.ProtocolMessage.IssuerAddress.Replace("{0}.", string.Empty);
}
if (previousOnRedirectToIdentityProviderForSignOut != null)
{
await previousOnRedirectToIdentityProviderForSignOut(ctx);
}
};
});
}
Now you can override the [AuthServer:Authority]
configuration of your public-web, mvc or blazor-server applications by using https://{0}.authserver.mystore.dev
.
Configuring the Microservices for SwaggerUI
The SwaggerUI for the gateways and the microservices are also being configured to resolve the domain as below. Hence, when navigated to the tenant's gateway (or any microservice) SwaggerUI, the Authorization disovery endpoint should be pointing to the tenant's AuthServer instead of the Host's.
Configure the MetadataAddress
of the SwaggerUI AuthServer configuration to use https://{0}.authserver.mystore.dev
so the discovery endpoint can be resolved correctly.
This feature is available after ABP v7.4.3
Configuring Helm
This configuration will be related to local Kubernetes cluster deployment to help you understand how to override the related domain values for the production environment. Some of the steps are optional since there are other ways to handle the problem.
Configuring the Host File
Problem: I created a tenant with the name Volosoft but when I navigate to tenant application (https://volosoft.angular.mystore.dev
) the DNS is not resolved and I get the error Can't reach this page DNS_PROBE_FINISHED_NXDOMAIN
.
Solution: You need to map the connection between the local kubernetes IP address and the domain name server. Simply update your Windows/system32/drivers/etc/hosts
(or /etc/hosts
in linux and macos) file assuming you have will use a tenant with name Volosoft:
127.0.0.1 volosoft.angular.mystore.dev
127.0.0.1 volosoft.mystore.dev
127.0.0.1 volosoft.authserver.mystore.dev
127.0.0.1 volosoft.identity.mystore.dev
127.0.0.1 volosoft.administration.mystore.dev
127.0.0.1 volosoft.product.mystore.dev
127.0.0.1 volosoft.saas.mystore.dev
127.0.0.1 volosoft.gateway-web.mystore.dev
127.0.0.1 volosoft.gateway-public.mystore.dev
Configuring SSL Certificate
Problem: The default self-signed certificate generation powershell script in the template is only for the host application:
mkcert "mystore.dev" "*.mystore.dev"
When you navigate to a subdomain like https://volosoft.authserver.mystore.dev
, the certificate doesn't cover this domain since it doesn't support 2nd depth of sub-domains. Hence, when you navigate https://volosoft.authserver.mystore.dev
in your local staging environment, there is an untrusted SSL error.
You can create a different self-signed SSL certificate or expand the extending certificate to be used for sub-domains aswell. To expand it, simply update the create-tls-secrets.ps1
script (under the k8s/Mystore folder) with:
mkcert "mystore.dev" "*.mystore.dev" "*.angular.mystore.dev" "*.mystore.dev" "*.authserver.mystore.dev" "*.identity.mystore.dev" "*.administration.mystore.dev" "*.saas.mystore.dev" "*.product.mystore.dev" "*.gateway-web.mystore.dev" "*.gateway-public.mystore.dev"
After you run the create-tls-secrets.ps1
script, now it should create kubernetes tls secret for the subdomains aswell.
Configuring Ingress
Your request from the browser to the subdomain will be accepted by the ingress-controller. The ingress-controller will redirect to the chart ingress based on your request.
Update your charts ingress.yaml files. Ex for administration-ingress.yaml:
spec:
tls:
- hosts:
- {{ .Values.ingress.host }}
secretName: {{ .Values.ingress.tlsSecret }}
# An other host with wildcard which will be using the same tls secret created above
- hosts:
- "*.{{ .Values.ingress.host }}"
secretName: {{ .Values.ingress.tlsSecret }}
rules:
- host: "{{ .Values.ingress.host }}"
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: {{ .Release.Name }}-{{ .Chart.Name }}
port:
number: 80
# An other host with wildcard which will be responding to `volosoft.administration.mystore.dev`
- host: "*.{{ .Values.ingress.host }}"
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: {{ .Release.Name }}-{{ .Chart.Name }}
port:
number: 80
Update all the application, gateway and microservice ingress.yaml files. Eventually, when deploy the application, you will be seeing:
Overriding Environment Variables
Navigate to applications, gateways and microservices' x-deployment.yaml file and override the newly introduced TenantDomain
key:
... Removed for brevity
- name: "TenantDomain" # Add this key
value: "{{ .Values.config.tenantDomain }}" # Override value under the .Values.yaml file
- name: "AuthServer__Authority"
value: "{{ .Values.config.authServer.authority }}"
...
Update all the application, gateway and microservice deployment.yaml files.
For AuthServer, also add the WildCardDomains that is used to handle subdomain redirect and post_logout redirect URIs to the authserver-deployment.yaml file:
... Removed for brevity
- name: "TenantDomain"
value: "{{ .Values.config.tenantDomain }}"
- name: "WildCardDomains__AuthServer"
value: "{{ .Values.wildCardDomains.authServer }}"
- name: "WildCardDomains__Angular"
value: "{{ .Values.wildCardDomains.angular }}"
- name: "WildCardDomains__PublicWeb"
value: "{{ .Values.wildCardDomains.publicWeb }}"
- name: "WildCardDomains__WebGateway"
value: "{{ .Values.wildCardDomains.webGateway }}"
- name: "WildCardDomains__PublicWebGateway"
value: "{{ .Values.wildCardDomains.publicWebGateway }}"
- name: "WildCardDomains__IdentityService"
value: "{{ .Values.wildCardDomains.identityService }}"
- name: "WildCardDomains__AdministrationService"
value: "{{ .Values.wildCardDomains.administrationService }}"
- name: "WildCardDomains__SaasService"
value: "{{ .Values.wildCardDomains.saasService }}"
- name: "WildCardDomains__ProductService"
value: "{{ .Values.wildCardDomains.productService }}"
...
Afterwards, update the values.yaml file for all the sub-charts (administration, authserver etc):
config:
... Removed for brevity
tenantDomain: #
You can either enter the domain values here (Ex, for administrationService: "https://{0}.administration.mystore.dev"
) or override these values aswell on the main chart values.yaml file.
For AuthServer, also update the authserver .values.yaml file:
config:
... Removed for brevity
tenantDomain: #
wildCardDomains:
authServer: #
angular: #
publicWeb: #
webGateway:
publicWebGateway: #
identityService: #
administrationService: #
saasService: #
productService: #
Overriding Tenant Domain Values
After updating the deployment.yaml and values.yaml files of all the application, gateway and microservices' navigate to the mystore chart values.yaml file located under the k8s/Mystore/values.yaml that overrides all the sub-charts:
AuthServer:
# auth-server sub-chart override
authserver:
config:
... Removed for brevity
corsOrigins: https://*.angular.mystore.dev,https://angular.mystore.dev,https://gateway-web.mystore.dev,https://*.gateway-web.mystore.dev,https://gateway-public.mystore.dev,https://*.gateway-public.mystore.dev,https://identity.mystore.dev,https://*.identity.mystore.dev,https://administration.mystore.dev,https://*.administration.mystore.dev,https://saas.mystore.dev,https://*.saas.mystore.dev,https://product.mystore.dev,https://*.product.mystore.dev
tenantDomain: "https://{0}.authserver.mystore.dev"
wildCardDomains:
authServer: "https://{0}.authserver.mystore.dev"
angular: "https://{0}.angular.mystore.dev"
publicWeb: "https://{0}.mystore.dev"
webGateway: "https://{0}.gateway-web.mystore.dev"
publicWebGateway: "https://{0}.gateway-public-web.mystore.dev"
identityService: "https://{0}.identity.mystore.dev"
administrationService: "https://{0}.administration.mystore.dev"
saasService: "https://{0}.saas.mystore.dev"
productService: "https://{0}.product.mystore.dev"
You may also get CORS error when authenticating SwaggerUI of your gateways or microservices. Add Override the AuthServer CORS values with the subdomain to solve this problem:
identityService:
# identity-service sub-chart override
identity:
config:
authServer:
metadataAddress: https://{0}.authserver.mystore.dev
... Removed for brevity
tenantDomain: "https://{0}.identity.mystore.dev"
administrationService:
# administration-service sub-chart override
administration:
config:
authServer:
metadataAddress: https://{0}.authserver.mystore.dev
... Removed for brevity
tenantDomain: "https://{0}.administration.mystore.dev"
saasService:
# saas-service sub-chart override
saas:
config:
authServer:
metadataAddress: https://{0}.authserver.mystore.dev
... Removed for brevity
tenantDomain: "https://{0}.saas.mystore.dev"
productService:
# product-service sub-chart override
product:
config:
authServer:
metadataAddress: https://{0}.authserver.mystore.dev
... Removed for brevity
tenantDomain: "https://{0}.product.mystore.dev"
gateway-web:
# saas-service sub-chart override
gateway-web:
config:
authServer:
metadataAddress: https://{0}.authserver.mystore.dev
... Removed for brevity
tenantDomain: "https://{0}.gateway-web.mystore.dev"
gateway-web-public:
# gateway-web-public sub-chart override
gateway-web-public:
config:
authServer:
metadataAddress: https://{0}.authserver.mystore.dev
... Removed for brevity
tenantDomain: "https://{0}.gateway-public.mystore.dev"
publicweb:
# Public Web application sub-chart override
publicweb:
config:
... Removed for brevity
authServer:
authority: https://{0}.authserver.mystore.dev # should be the domain name (https://auth.mydomain.com)
... Removed for brevity
tenantDomain: "https://{0}.mystore.dev"
angular:
# Angular back-office application sub-chart override
angular:
config:
selfUrl: https://{0}.angular.mystore.dev
gatewayUrl: https://{0}.gateway-web.mystore.dev
authServer:
authority: https://{0}.authserver.mystore.dev
requireHttpsMetadata: "false"
responseType: "code"
strictDiscoveryDocumentValidation: false
skipIssuerCheck: true
If you are using Web or BlazorServer application for back-office, it is similar configuration with the public-web application
Now you can use the deploy-staging.ps1
script to deploy your solution.
Result
After creating a new tenant named Volosoft from the angular back-office application: