Open Closed

Deployment production tutorial Kubernetes / Let's encrypt #5891


User avatar
0
carl.hallqvist created
  • ABP Framework version: v7.3.3
  • UI Type: Blazor WASM
  • Database System: EF Core (SQL Server)
  • Tiered (for MVC) or Auth Server Separated (for Angular): yes
  • Exception message and full stack trace: N/A
  • Steps to reproduce the issue: N/A

Can you provide any short guide / tutorial of how to deploy the Microservice template in a Kubernetes cluster with real domain names?

Prerequisites:

  • ABP commecial license
  • A Kubernetes cluster with a minimum specification to host the microservice template pods.
  • A domain name with access to add A records for managing sub-domains (e.g., abp-microservice-tutorial.io, auth.abp-microservice-tutorial.io, ..)

Thanks, Carl


9 Answer(s)
  • User Avatar
    0
    gterdem created
    Senior .NET Developer

    We don't have a full guide about how to deploy Microservice template to Azure yet. But we have a public repository with build scripts and azure helm chart values at https://github.com/abpframework/eShopOnAbp/tree/main/etc/k8s/azure

    Let us know if you come across any problems.

  • User Avatar
    0
    carl.hallqvist created

    Thanks gterdem,

    I don't think that is related to Azure? I am no expert, but when looking at the k8s directory, it seems that most of it is in place? I guess its the installation of cert-manager and nginx ingress (for instance) and the issuer manifests. Also adding ingress part for all services.

    I will try to make it work by my self, but the abp really need some deployment guides. We was failing to deploy abp framework too, and now we are trying the commercial one.

  • User Avatar
    0
    carl.hallqvist created

    Hi,

    I have almost managed to deploy everything. However, got some configuration failure from public web app:

    [19:46:55 INF] TRIAL-INFO - You are using the trial version of ABP and your expiry date is 2023-10-26 [19:46:55 INF] Now listening on: http://[::]:80 [19:46:55 INF] Application started. Press Ctrl+C to shut down. [19:46:55 INF] Hosting environment: Production [19:46:55 INF] Content root path: /app [19:48:58 INF] Request starting HTTP/1.1 GET http://public.staging.microservice.io/ - - [19:48:58 WRN] Could not find IdentityClientConfiguration for AbpMvcClient. Either define a configuration for AbpMvcClient or set a default configuration. [19:48:58 INF] Start processing HTTP request GET https://microservice-gateway-web-public/api/abp/application-configuration?IncludeLocalizationResources=False&api-version=1.0 [19:48:58 INF] Sending HTTP request GET https://microservice-gateway-web-public/api/abp/application-configuration?IncludeLocalizationResources=False&api-version=1.0 [19:49:00 INF] Sending HTTP request GET https://microservice-gateway-web-public/api/abp/application-configuration?IncludeLocalizationResources=False&api-version=1.0 [19:49:04 INF] Sending HTTP request GET https://microservice-gateway-web-public/api/abp/application-configuration?IncludeLocalizationResources=False&api-version=1.0 [19:49:12 INF] Sending HTTP request GET https://microservice-gateway-web-public/api/abp/application-configuration?IncludeLocalizationResources=False&api-version=1.0 [19:49:28 INF] Sending HTTP request GET https://microservice-gateway-web-public/api/abp/application-configuration?IncludeLocalizationResources=False&api-version=1.0 [19:49:28 ERR] Connection id "0HMU9033L8A7L", Request id "0HMU9033L8A7L:00000001": An unhandled exception was thrown by the application. Volo.Abp.Http.Client.AbpRemoteCallException: An error occurred during the ABP remote HTTP request. (Connection refused (microservice-gateway-web-public:443)) See the inner exception for details. ---> System.Net.Http.HttpRequestException: Connection refused (microservice-gateway-web-public:443) ---> System.Net.Sockets.SocketException (111): Connection refused

    This is my value file:

    # auth-server sub-chart override
    authserver:
      config:
        selfUrl: https://auth.staging.microservice.io   
        corsOrigins: https://gateway.staging.microservice.io,https://gateway-public.staging.microservice.io,https://identity.staging.microservice.io,https://administration.staging.microservice.io,https://basket.staging.microservice.io,https://catalog.staging.microservice.io,https://order.staging.microservice.io,https://cmskit.staging.microservice.io,https://payment.staging.microservice.io,https://admin.staging.microservice.io,https://www.staging.microservice.io
        allowedRedirectUrls: https://admin.staging.microservice.io
        disablePii: "false"
        identityService:
          connString: "Server=microservice-sqldb,1433;Database=Microservice_Identity;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
        administrationService:
          connString: "Server=microservice-sqldb,1433;Database=Microservice_Administration;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
        saasService:
          connString: "Server=microservice-sqldb,1433;Database=Microservice_Saas;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
        authServer:
          authority: http://microservice-authserver
          requireHttpsMetadata: "false"
          swaggerClientId: WebGateway_Swagger
        dotnetEnv: Production
        redisHost: microservice-redis
        rabbitmqHost: microservice-rabbitmq
        elasticsearchUrl: http://microservice-elasticsearch
      ingress:
        host: auth.staging.microservice.io
        tlsSecret: auth-staging-microservice-io-tls
      image:
        repository: "registry.digitalocean.com/microservice/microservice-app-authserver"
        tag: latest
    
    # public-web sub-chart override
    public-web:
      config:
        selfUrl: https://www.staging.microservice.io
        gatewayUrl: https://microservice-gateway-web-public
        authServer:
          authority: https://auth.staging.microservice.io
          requireHttpsMetadata: "false"
          isOnProd: "true"
          metaAddress: http://microservice-authserver
        dotnetEnv: Production
        redisHost: microservice-redis
        rabbitmqHost: microservice-rabbitmq
        elasticsearchHost: http://microservice-elasticsearch
      ingress:
        host: public.staging.microservice.io
        tlsSecret: public-staging-microservice-io-tls
      image:
        repository: "registry.digitalocean.com/microservice/microservice-app-public-web"
        tag: latest
    
    blazor:
      config:
        selfUrl: https://blazor.staging.microservice.io
        gatewayUrl: https://microservice-gateway-web
        authServer:
          authority: https://microservice-st-authserver
          requireHttpsMetadata: "false"
          responseType: "code"
      ingress:
        host: blazor.staging.microservice.io
        tlsSecret: blazor-staging-microservice-io-tls
      image:
        repository: registry.digitalocean.com/microservice/microservice-app-blazor
        tag: latest
        pullPolicy: IfNotPresent
    
    # identity-service sub-chart override
    identity:
      config:
        selfUrl: https://identity.staging.microservice.io
        corsOrigins: https://gateway.staging.microservice.io,https://gateway-public.staging.microservice.io
        connectionStrings:
          identityService: "Server=microservice-sqldb,1433;Database=Microservice_Identity;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
          administrationService: "Server=microservice-sqldb,1433;Database=Microservice_Administration;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
          saasService: "Server=microservice-sqldb,1433;Database=Microservice_Saas;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
        authServer:
          authority: http://microservice-authserver
          requireHttpsMetadata: "false"
          swaggerClientId: WebGateway_Swagger
          swaggerClientSecret: "1q2w3e*"
        dotnetEnv: Production
        redisHost: microservice-redis
        rabbitmqHost: microservice-rabbitmq
        elasticsearchHost: http://microservice-elasticsearch  
      identityServerClients: # Seeded Clients
        webRootUrl: https://admin.staging.microservice.io/  
        publicWebRootUrl: https://www.staging.microservice.io/
        webGatewayRootUrl: https://gateway.staging.microservice.io/
        publicWebGatewayRootUrl: https://gateway-public.staging.microservice.io/
        identityServiceRootUrl: https://identity.staging.microservice.io/
        administrationServiceRootUrl: https://administration.staging.microservice.io/
        accountServiceRootUrl: https://auth.staging.microservice.io
      ingress:
        host: identity.staging.microservice.io
        tlsSecret: identity-staging-microservice-io-tls
      image:
        repository: "registry.digitalocean.com/microservice/microservice-service-identity"
        tag: latest
    
    # administration sub-chart override
    administration:
      config:
        selfUrl: https://administration.staging.microservice.io
        corsOrigins: https://microservice-gateway-web,https://microservice-st-gateway-web-public
        connectionStrings:
          administrationService: "Server=microservice-sqldb,1433;Database=Microservice_Administration;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
          saasService: "Server=microservice-sqldb,1433;Database=Microservice_Saas;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"    
        authServer:
          authority: http://microservice-st-authserver
          requireHttpsMetadata: "false"
          swaggerClientId: WebGateway_Swagger
          swaggerClientSecret: "1q2w3e*"
        remoteServices:
          abpIdentityBaseUrl: https://microservice-st-identity
          useCurrentToken: "false"   
        dotnetEnv: Production
        redisHost: microservice-redis
        rabbitmqHost: microservice-rabbitmq
        elasticsearchHost: http://microservice-elasticsearch
      synchedCommunication: # Used for server-to-server (client-credentials) communication with identityService for user permissions
        authority: http://microservice-authserver
        requireHttps: "false"
        validateIssuerName: "false"
        validateEndpoints: "false"
      ingress:
        host: administration.staging.microservice.io
        tlsSecret: administration-staging-microservice-io-tls
      image:
        repository: registry.digitalocean.com/microservice/microservice-service-administration
        tag: latest
    
    # gateway-web sub-chart override
    gateway-web:
      config:
        selfUrl: https://gateway.staging.microservice.io
        corsOrigins: https://admin.staging.microservice.io
        globalConfigurationBaseUrl: http://microservice-gateway-public
        authServer:
          authority: http://microservice-authserver
          requireHttpsMetadata: "false"
          swaggerClientId: WebGateway_Swagger
          swaggerClientSecret: "1q2w3e*"
        dotnetEnv: Production
        redisHost: microservice-redis
        rabbitmqHost: microservice-rabbitmq
        elasticsearchHost: http://microservice-elasticsearch
      ingress:
        host: gateway.staging.microservice.io
        tlsSecret: gateway-staging-microservice-io-tls
      image:
        repository: "registry.digitalocean.com/microservice/microservice-gateway-web"
        tag: latest
      reRoutes:
        accountService:
          url: http://microservice-authserver
        identityService:
          url: http://microservice-identity
        administrationService:
          url: http://microservice-administration
    
    # gateway-web-public sub-chart override
    gateway-web-public:
      config:
        selfUrl: https://gateway-public.staging.microservice.io
        authServer:
          authority: http://microservice-authserver
          requireHttpsMetadata: "false"
          swaggerClientId: WebGateway_Swagger
          swaggerClientSecret: "1q2w3e*"
        dotnetEnv: Production
        redisHost: microservice-redis
        rabbitmqHost: microservice-rabbitmq
        elasticsearchHost: http://microservice-elasticsearch
      ingress:
        host: gateway-public.staging.microservice.io
        tlsSecret: gateway-public-staging-microservice-io-tls
      image:
        repository: "registry.digitalocean.com/microservice/microservice-gateway-web-public"
        tag: latest
      reRoutes:
        accountService:
          url: http://microservice-authserver
        identityService:
          url: http://microservice-identity
        administrationService:
          url: http://microservice-administration
    
    # dbmigrator
    dbmigrator:
      config:
        connectionStrings:
          identityService: "Server=microservice-sqldb,1433;Database=Microservice_Identity;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
          administrationService: "Server=microservice-sqldb,1433;Database=Microservice_Administration;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
          saasService: "Server=microservice-sqldb,1433;Database=Microservice_Saas;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
          productService: "Server=microservice-sqldb,1433;Database=Microservice_ProductService;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
        dotnetEnv: Production
        elasticsearchUrl: http://microservice-elasticsearch
        redisHost: microservice-redis
      openIdDict:
        applications:
          authServerRootUrl: https://microservice-authserver/
          angularRootUrl: https://microservice-angular/
          webRootUrl: https://microservice-web/
          blazorRootUrl: https://microservice-blazor/
          blazorServerRootUrl: https://microservice-blazor-server/
          publicWebRootUrl: https://microservice-public-web/
          webGatewayRootUrl: https://microservice-gateway-web/
          publicWebGatewayRootUrl: https://microservice-gateway-web-public/ 
        resources:  
          accountServiceRootUrl: https://microservice-authserver/
          administrationServiceRootUrl: https://microservice-administration/
          identityServiceRootUrl: https://microservice-identity/
          saasServiceRootUrl: https://microservice-saas/
          productServiceRootUrl: https://microservice-product/
      image:
        repository: registry.digitalocean.com/microservice/microservice-db-migrator
        tag: latest
        pullPolicy: IfNotPresent  
    
    kibana:
      config:
        elasticsearchUrl: http://microservice-elasticsearch:9200
      ingress:
        host: kibana.staging.microservice.io
        tlsSecret: kibana-staging-microservice-io-tls
    
    # Default values for microservice.
    # This is a YAML-formatted file.
    # Declare variables to be passed into your templates.
    
    replicaCount: 1
    
    image:
      repository: nginx
      pullPolicy: IfNotPresent
      # Overrides the image tag whose default is the chart appVersion.
      tag: ""
    
    imagePullSecrets: []
    nameOverride: ""
    fullnameOverride: ""
    
    serviceAccount:
      # Specifies whether a service account should be created
      create: true
      # Annotations to add to the service account
      annotations: {}
      # The name of the service account to use.
      # If not set and create is true, a name is generated using the fullname template
      name: ""
    
    podAnnotations: {}
    
    podSecurityContext: {}
      # fsGroup: 2000
    
    securityContext: {}
      # capabilities:
      #   drop:
      #   - ALL
      # readOnlyRootFilesystem: true
      # runAsNonRoot: true
      # runAsUser: 1000
    
    service:
      type: ClusterIP
      port: 80
    
    # ingress:
    #   enabled: false
    #   className: ""
    #   annotations: {}
    #     # kubernetes.io/ingress.class: nginx
    #     # kubernetes.io/tls-acme: "true"
    #   hosts:
    #     - host: chart-example.local
    #       paths:
    #         - path: /
    #           pathType: ImplementationSpecific
    #   tls: []
    #   #  - secretName: chart-example-tls
    #   #    hosts:
    #   #      - chart-example.local
    
    resources: {}
      # We usually recommend not to specify default resources and to leave this as a conscious
      # choice for the user. This also increases chances charts run on environments with little
      # resources, such as Minikube. If you do want to specify resources, uncomment the following
      # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
      # limits:
      #   cpu: 100m
      #   memory: 128Mi
      # requests:
      #   cpu: 100m
      #   memory: 128Mi
    
    autoscaling:
      enabled: false
      minReplicas: 1
      maxReplicas: 100
      targetCPUUtilizationPercentage: 80
      # targetMemoryUtilizationPercentage: 80
    
    tolerations: []
    
    affinity: {}
    
  • User Avatar
    0
    gterdem created
    Senior .NET Developer

    public-web sub-chart override

    public-web: config: selfUrl: https://www.staging.microservice.io gatewayUrl: https://microservice-gateway-web-public ---> should be real dns like www.myapp.gateway.com

    The error is related to the gateway DNS couldn't be reached over the internet since you have provided internal network service name. Please try real DNS for it and let me know if it solves your problem.

  • User Avatar
    0
    carl.hallqvist created

    Thanks gterdem,

    Unfortuntely, I got same error. I think that I need to get each app / service / gateway working.

    auth.staging.microservice.io service seems to be ok, the login / logout is working fine.

    Log from microservice-public-web

    [20:23:22 INF] Sending HTTP request GET https://gateway-public.staging.microservice.io/api/abp/application-configuration?IncludeLocalizationResources=False&api-version=1.0
    [20:23:22 INF] Received HTTP response headers after 13.0113ms - 502
    

    So, the response from the gateway is 502, and when looking into the log in the gateway I got:

    Log from geoticket-gateway-web-public:

    [20:23:22 INF] Request starting HTTP/1.1 GET http://gateway-public.staging.microservice.io/api/abp/application-configuration?IncludeLocalizationResources=False&api-version=1.0 - -
    [20:23:22 INF] requestId: 0HMUBBQVGSBOD:00000004, previousRequestId: no previous request id, message: EndpointRateLimiting is not enabled for /api/abp/{everything}
    [20:23:22 INF] requestId: 0HMUBBQVGSBOD:00000004, previousRequestId: no previous request id, message: No authentication needed for /api/abp/application-configuration
    [20:23:22 INF] requestId: 0HMUBBQVGSBOD:00000004, previousRequestId: no previous request id, message: /api/abp/{everything} route does not require user to be authorized
    [20:23:22 WRN] requestId: 0HMUBBQVGSBOD:00000004, previousRequestId: no previous request id, message: Error Code: ConnectionToDownstreamServiceError Message: Error connecting to downstream service, exception: System.Net.Http.HttpRequestException: Connection refused (localhost:44367)
     ---> System.Net.Sockets.SocketException (111): Connection refused
       at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
       at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
       at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|281_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
       at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
       --- End of inner exception stack trace ---
       at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
       at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
       at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
       at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem)
       at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
       at System.Net.Http.HttpConnectionPool.HttpConnectionWaiter`1.WaitForConnectionAsync(Boolean async, CancellationToken requestCancellationToken)
       at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
       at System.Net.Http.DiagnosticsHandler.SendAsyncCore(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
       at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
       at Ocelot.Requester.HttpClientHttpRequester.GetResponse(HttpContext httpContext) errors found in ResponderMiddleware. Setting error response for request path:/api/abp/application-configuration, request method: GET
    [20:23:22 INF] Request finished HTTP/1.1 GET http://gateway-public.staging.microservice.io/api/abp/application-configuration?IncludeLocalizationResources=False&api-version=1.0 - - - 502 0 - 6.7145ms
    

    This is my value file:

    # public-web sub-chart override
    public-web:
      config:
        selfUrl: https://public.staging.microservice.io
        gatewayUrl: https://gateway-public.staging.microservice.io
        authServer:
          authority: https://auth.staging.microservice.io
          requireHttpsMetadata: "false"
          isOnProd: "true"
          metaAddress: http://microservice-authserver
        dotnetEnv: Production
        redisHost: microservice-redis
        rabbitmqHost: microservice-rabbitmq
        elasticsearchHost: http://microservice-elasticsearch
      ingress:
        host: public.staging.microservice.io
        tlsSecret: public-staging-microservice-io-tls
      image:
        repository: "registry.digitalocean.com/microservice/microservice-app-public-web"
        tag: latest
    	
    ###
    
    # gateway-web-public sub-chart override
    gateway-web-public:
      config:
        selfUrl: https://gateway-public.staging.microservice.io
        authServer:
          authority: http://microservice-authserver
          requireHttpsMetadata: "false"
          swaggerClientId: WebGateway_Swagger
          swaggerClientSecret: "1q2w3e*"
        dotnetEnv: Production
        redisHost: microservice-redis
        rabbitmqHost: microservice-rabbitmq
        elasticsearchHost: http://microservice-elasticsearch
      ingress:
        host: gateway-public.staging.microservice.io
        tlsSecret: gateway-public-staging-microservice-io-tls
      image:
        repository: "registry.digitalocean.com/microservice/microservice-gateway-web-public"
        tag: latest
      reRoutes:
        accountService:
          url: http://microservice-authserver
        identityService:
          url: http://microservice-identity
        administrationService:
          url: http://microservice-administration
    
    

    Kind regards, Carl

  • User Avatar
    0
    gterdem created
    Senior .NET Developer
    [20:23:22 WRN] requestId: 0HMUBBQVGSBOD:00000004, previousRequestId: no previous request id, message: Error Code: ConnectionToDownstreamServiceError Message: Error connecting to downstream service, exception: System.Net.Http.HttpRequestException: Connection refused (localhost:44367)
     ---> System.Net.Sockets.SocketException (111): Connection refused
    

    The error indicates that the problem is related to your gateway downstream. It seems you may have forgotten to override the re-routing for the gateways.

  • User Avatar
    0
    carl.hallqvist created

    Thanks gterdem,

    I did a complete reset of the helm values. This is what I got:

    # auth-server sub-chart override
    authserver:
      config:
        selfUrl: https://auth.staging.microservice.io
        corsOrigins: http://microservice-blazor,https://blazor.staging.microservice.io,https://gateway-web.staging.microservice.io,https://microservice-gateway-web-public,https://microservice-identity,https://microservice-administration,https://microservice-saas,https://microservice-product
        allowedRedirectUrls: http://blazor.staging.microservice.io,https://blazor.staging.microservice.io
        disablePii: "false"
        identityService:
          connString: "Server=microservice-sqldb,1433;Database=Microservice_Identity;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
        administrationService:
          connString: "Server=microservice-sqldb,1433;Database=Microservice_Administration;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
        saasService:
          connString: "Server=microservice-sqldb,1433;Database=Microservice_Saas;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
        authServer:
          authority: http://microservice-authserver
          requireHttpsMetadata: "false"
          swaggerClientId: WebGateway_Swagger
        dotnetEnv: Production
        redisHost: microservice-redis
        rabbitmqHost: microservice-rabbitmq
        elasticsearchUrl: http://microservice-elasticsearch
      ingress:
        host: auth.staging.microservice.io
        tlsSecret: staging.microservice.io
      image:
        repository: registry.digitalocean.com/microservice/microservice-app-authserver
        tag: latest
    
    # identity-service sub-chart override
    identity:
      config:
        selfUrl: https://microservice-identity
        corsOrigins: https://microservice-gateway-web
        connectionStrings:
          identityService: "Server=microservice-sqldb,1433;Database=Microservice_Identity;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
          administrationService: "Server=microservice-sqldb,1433;Database=Microservice_Administration;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
          saasService: "Server=microservice-sqldb,1433;Database=Microservice_Saas;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
        authServer:
          authority: http://microservice-authserver
          requireHttpsMetadata: "false"
        dotnetEnv: Production
        redisHost: microservice-redis
        rabbitmqHost: microservice-rabbitmq
        elasticsearchUrl: http://microservice-elasticsearch
      openIdDict: #Seed applications and resources
        applications:
          authServerRootUrl: https://microservice-authserver/
          blazorRootUrl: https://blazor.staging.microservice.io/
          publicWebRootUrl: https://microservice-public-web/
          webGatewayRootUrl: https://microservice-gateway-web/
          publicWebGatewayRootUrl: https://microservice-gateway-web-public/ 
        resources:  
          accountServiceRootUrl: https://microservice-authserver/
          administrationServiceRootUrl: https://microservice-administration/
          identityServiceRootUrl: https://microservice-identity/
          saasServiceRootUrl: https://microservice-saas/
          productServiceRootUrl: https://microservice-product/
      ingress:
        host: microservice-identity
      image:
        repository: registry.digitalocean.com/microservice/microservice-service-identity
        tag: latest
    
    # administration-service sub-chart override
    administration:
      config:
        selfUrl: https://microservice-administration
        corsOrigins: https://microservice-gateway-web,https://microservice-gateway-web-public
        connectionStrings:
          administrationService: "Server=microservice-sqldb,1433;Database=Microservice_Administration;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
          saasService: "Server=microservice-sqldb,1433;Database=Microservice_Saas;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"    
        authServer:
          authority: http://microservice-authserver
          requireHttpsMetadata: "false"
          swaggerClientId: WebGateway_Swagger
        remoteServices:
          abpIdentityBaseUrl: https://microservice-identity
          useCurrentToken: "false"   
        dotnetEnv: Production
        redisHost: microservice-redis
        rabbitmqHost: microservice-rabbitmq
        elasticsearchUrl: http://microservice-elasticsearch
      synchedCommunication: # Used for server-to-server (client-credentials) communication with identityService for user permissions
        authority: https://microservice-authserver
        requireHttps: "false"
        validateIssuerName: "false"
        validateEndpoints: "false"
      ingress:
        host: microservice-administration
      image:
        repository: registry.digitalocean.com/microservice/microservice-service-administration
        tag: latest
    
    # saas-service sub-chart override
    saas:
      config:
        selfUrl: https://microservice-saas
        corsOrigins: https://microservice-gateway-web
        connectionStrings:
          administrationService: "Server=microservice-sqldb,1433;Database=Microservice_Administration;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
          saasService: "Server=microservice-sqldb,1433;Database=Microservice_Saas;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
        authServer:
          authority: http://microservice-authserver
          requireHttpsMetadata: "false"
          swaggerClientId: WebGateway_Swagger
        dotnetEnv: Production
        redisHost: microservice-redis
        rabbitmqHost: microservice-rabbitmq
        elasticsearchUrl: http://microservice-elasticsearch
      ingress:
        host: microservice-saas
      image:
        repository: registry.digitalocean.com/microservice/microservice-service-saas
        tag: latest  
    
    # product-service sub-chart override
    product:
      config:
        selfUrl: https://microservice-product
        corsOrigins: https://microservice-gateway-web,https://microservice-gateway-web-public
        connectionStrings:
          productService: "Server=microservice-sqldb,1433;Database=Microservice_ProductService;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
          administrationService: "Server=microservice-sqldb,1433;Database=Microservice_Administration;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
          saasService: "Server=microservice-sqldb,1433;Database=Microservice_Saas;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
        authServer:
          authority: http://microservice-authserver
          requireHttpsMetadata: "false"
          swaggerClientId: WebGateway_Swagger
        dotnetEnv: Production
        redisHost: microservice-redis
        rabbitmqHost: microservice-rabbitmq
        elasticsearchUrl: http://microservice-elasticsearch
      ingress:
        host: microservice-product
      image:
        repository: registry.digitalocean.com/microservice/microservice-service-product
        tag: latest
    
    # gateway-web sub-chart override
    gateway-web:
      config:
        selfUrl: https://gateway-web.staging.microservice.io
        corsOrigins: https://blazor.staging.microservice.io
        globalConfigurationBaseUrl: http://microservice-gateway-web
        authServer:
          authority: http://microservice-authserver
          requireHttpsMetadata: "false"
          swaggerClientId: WebGateway_Swagger
        dotnetEnv: Production
        redisHost: microservice-redis
        rabbitmqHost: microservice-rabbitmq
        elasticsearchUrl: http://microservice-elasticsearch
      reRoutes:
        accountService:
          dns: https://auth.staging.microservice.io
          schema: http
          host: microservice-authserver
          port: 80
        identityService:
          dns: https://microservice-identity
          schema: http
          host: microservice-identity
          port: 80
        administrationService:
          dns: https://microservice-administration
          schema: http
          host: microservice-administration
          port: 80
        saasService:
          dns: https://microservice-saas
          schema: http
          host: microservice-saas
          port: 80
        productService:
          dns: https://microservice-product
          schema: http
          host: microservice-product
          port: 80  
      ingress:
        host: gateway-web.staging.microservice.io
        tlsSecret: staging.microservice.io
      image:
        repository: registry.digitalocean.com/microservice/microservice-gateway-web
        tag: latest
        pullPolicy: IfNotPresent
    
    # gateway-web-public sub-chart override
    gateway-web-public:
      config:
        selfUrl: https://microservice-gateway-web-public
        globalConfigurationBaseUrl: http://microservice-gateway-web-public
        authServer:
          authority: http://microservice-authserver
          requireHttpsMetadata: "false"
          swaggerClientId: WebGateway_Swagger
          swaggerClientSecret: "1q2w3e*"
        dotnetEnv: Production 
        redisHost: microservice-redis
        rabbitmqHost: microservice-rabbitmq
        elasticsearchUrl: http://microservice-elasticsearch
      reRoutes:
        accountService:
          dns: https://microservice-authserver
          schema: http
          host: microservice-authserver
          port: 80
        saasService:
          dns: https://microservice-saas
          schema: http
          host: microservice-saas
          port: 80
        administrationService:
          dns: https://microservice-administration
          schema: http
          host: microservice-administration
          port: 80
        productService:
          dns: https://microservice-product
          schema: http
          host: microservice-product
          port: 80  
      ingress:
        host: microservice-gateway-web-public
        # tlsSecret: microservice-tls
      image:
        repository: registry.digitalocean.com/microservice/microservice-gateway-web-public
        tag: latest
        pullPolicy: IfNotPresent
    
    kibana:
      config:
        elasticsearchUrl: http://microservice-elasticsearch:9200
    
    # dbmigrator sub-chart override
    dbmigrator:
      config:
        connectionStrings:
          identityService: "Server=microservice-sqldb,1433;Database=Microservice_Identity;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
          administrationService: "Server=microservice-sqldb,1433;Database=Microservice_Administration;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
          saasService: "Server=microservice-sqldb,1433;Database=Microservice_Saas;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
          productService: "Server=microservice-sqldb,1433;Database=Microservice_ProductService;User Id=sa;password=myPassw@rd;TrustServerCertificate=True"
        dotnetEnv: Production
        elasticsearchUrl: http://microservice-elasticsearch
        redisHost: microservice-redis
      openIdDict:
        applications:
          authServerRootUrl: https://microservice-authserver/
          angularRootUrl: https://microservice-angular/
          webRootUrl: https://microservice-web/
          blazorRootUrl: https://blazor.staging.microservice.io/
          blazorServerRootUrl: https://microservice-blazor-server/
          publicWebRootUrl: https://microservice-public-web/
          webGatewayRootUrl: https://microservice-gateway-web/
          publicWebGatewayRootUrl: https://microservice-gateway-web-public/ 
        resources:  
          accountServiceRootUrl: https://microservice-authserver/
          administrationServiceRootUrl: https://microservice-administration/
          identityServiceRootUrl: https://microservice-identity/
          saasServiceRootUrl: https://microservice-saas/
          productServiceRootUrl: https://microservice-product/
      image:
        repository: registry.digitalocean.com/microservice/microservice-db-migrator
        tag: latest
        pullPolicy: IfNotPresent  
    
    # Public Web application sub-chart override
    publicweb:
      config:
        selfUrl: https://public-web.staging.microservice.io
        gatewayUrl: http://microservice-gateway-web-public
        authServer:
          authority: https://auth.staging.microservice.io # should be domain on production (https://auth.mydomain.com)
          requireHttpsMetadata: "false"
          isOnK8s: "true"
          metaAddress: http://microservice-authserver # should be container on production (http://microservice-az-authserver)
        dotnetEnv: Production 
        redisHost: microservice-redis
        rabbitmqHost: microservice-rabbitmq
        elasticsearchUrl: http://microservice-elasticsearch
      ingress:
        host: public-web.staging.microservice.io
        tlsSecret: staging.microservice.io
      image:
        repository: registry.digitalocean.com/microservice/microservice-app-public-web
        tag: latest
        pullPolicy: IfNotPresent
    
    blazor:
      config:
        selfUrl: https://blazor.staging.microservice.io
        gatewayUrl: https://gateway-web.staging.microservice.io
        authServer:
          authority: https://auth.staging.microservice.io
          requireHttpsMetadata: "false"
          responseType: "code"
      ingress:
        host: blazor.staging.microservice.io
        tlsSecret: staging.microservice.io
      image:
        repository: registry.digitalocean.com/microservice/microservice-app-blazor
        tag: latest
        pullPolicy: IfNotPresent
    # Default values for Microservice.
    # This is a YAML-formatted file.
    # Declare variables to be passed into your templates.
    
    replicaCount: 1
    

    It starts to working now, but I got following question:

    In the dbmigrator log I got successful report:

    [19:37:09 INF] Migrating Host side...
    [19:37:10 INF] Migrating SaasService database...
    [19:37:19 INF] Migrating AdministrationService database...
    [19:37:25 INF] Migrating IdentityService database...
    [19:37:29 INF] Migrating ProductService database...
    [19:37:29 INF] All databases have been successfully migrated (HOST).
    [19:37:36 INF] Migrating tenants...
    [19:37:36 INF] Migration completed!
    

    But the identity log says:

    [19:44:43 INF] Saving external localizations...
    [19:44:43 INF] Lock is acquired for db migration and seeding on database named: IdentityService...
    [19:44:46 INF] Completed to save external localizations.
    [19:44:47 INF] Lock is released for db migration and seeding on database named: IdentityService...
    [19:44:47 INF] Seeding IdentityServer data...
    [19:44:49 ERR] Exception of type 'Volo.Abp.BusinessException' was thrown.
    [19:44:49 WRN] BusinessException has been thrown. The operation will be tried 2 times more. Exception:
    Exception of type 'Volo.Abp.BusinessException' was thrown.
    [19:45:01 INF] Seeding IdentityServer data...
    [19:45:01 ERR] Exception of type 'Volo.Abp.BusinessException' was thrown.
    [19:45:01 WRN] BusinessException has been thrown. The operation will be tried 1 times more. Exception:
    Exception of type 'Volo.Abp.BusinessException' was thrown.
    [19:45:12 INF] Seeding IdentityServer data...
    [19:45:12 ERR] Exception of type 'Volo.Abp.BusinessException' was thrown.
    [19:45:12 FTL] Microservice.IdentityService.HttpApi.Host terminated unexpectedly!
    Volo.Abp.AbpInitializationException: An error occurred during the initialize Volo.Abp.Modularity.OnPostApplicationInitializationModuleLifecycleContributor phase of the module Microservice.IdentityService.IdentityServiceHttpApiHostModule, Microservice.IdentityService.HttpApi.Host, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null: Exception of type 'Volo.Abp.BusinessException' was thrown.. See the inner exception for details.
     ---> Volo.Abp.BusinessException: Exception of type 'Volo.Abp.BusinessException' was thrown.
       at Microservice.IdentityService.DbMigrations.OpenIddictDataSeeder.CreateApplicationAsync(String name, String type, String consentType, String displayName, String secret, List`1 grantTypes, List`1 scopes, List`1 redirectUris, List`1 postLogoutRedirectUris, List`1 permissions, String clientUri, String logoUri) in /src/services/identity/src/Microservice.IdentityService.HttpApi.Host/DbMigrations/OpenIddictDataSeeder.cs:line 422
    

    Any ideas?

    Kind regards, Carl

  • User Avatar
    0
    gterdem created
    Senior .NET Developer

    The problem seems to be related to your OpenIddictDataSeeder located under the IdentityService.

    Normally, you can choose either running the DbMigrator to migrate all the databases or let each database migrate themselves.

    If you want to fix the issue, it seems related to the CreateApplicationAsync of the OpenIddictDataSeeder. You may have forgotten to override the OpenIddict values of IdentityService appsettings.json.

  • User Avatar
    0
    carl.hallqvist created

    Thanks gterdem,

    I think that I got most of if working now. Great support! :-)

Made with ❤️ on ABP v9.1.0-preview. Updated on December 13, 2024, 06:09