Error connecting to redis server on Azure Kubernetes in Auth server MVC #5010 | Support

rogercprops created one year ago

Check the docs before asking a question: https://docs.abp.io/en/commercial/latest/ Check the samples, to see the basic tasks: https://docs.abp.io/en/commercial/latest/samples/index The exact solution to your question may have been answered before, please use the search on the homepage.

If you're creating a bug/problem report, please include followings:

ABP Framework version: v7.0 - Commercial
UI type: MVC
DB provider: EF Core
Tiered (MVC) or Identity Server Separated (Angular): yes
Exception message and stack trace:
- Volo.Abp.AbpInitializationException: An error occurred during ConfigureServicesAsync phase of the module AuthServer.CprOnAbpAuthServerModule, AuthServer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null. See the inner exception for details. ---> StackExchange.Redis.RedisConnectionException: It was not possible to connect to the redis server(s). UnableToConnect on redis:6379/Interactive, Initializing/NotStarted, last: NONE, origin: BeginConnectAsync, outstanding: 0, last-read: 0s ago, last-write: 0s ago, keep-alive: 60s, state: Connecting, mgr: 10 of 10 available, last-heartbeat: never, global: 0s ago, v: 2.2.4.27433 at StackExchange.Redis.ConnectionMultiplexer.ConnectImpl(ConfigurationOptions configuration, TextWriter log) in //src/StackExchange.Redis/ConnectionMultiplexer.cs:line 1163at StackExchange.Redis.ConnectionMultiplexer.Connect(ConfigurationOptions configuration, TextWriter log) in //src/StackExchange.Redis/ConnectionMultiplexer.cs:line 1032 at StackExchange.Redis.ConnectionMultiplexer.Connect(String configuration, TextWriter log) in /_/src/StackExchange.Redis/ConnectionMultiplexer.cs:line 1015 at AuthServer.CprOnAbpAuthServerModule.ConfigureServices(ServiceConfigurationContext context) in /src/AuthServerModule.cs:line 141 at Volo.Abp.Modularity.AbpModule.ConfigureServicesAsync(ServiceConfigurationContext context) at Volo.Abp.AbpApplicationBase.ConfigureServicesAsync() --- End of inner exception stack trace --- at Volo.Abp.AbpApplicationBase.ConfigureServicesAsync() at Volo.Abp.AbpApplicationFactory.CreateAsync[TStartupModule](IServiceCollection services, Action1 optionsAction) at Microsoft.Extensions.DependencyInjection.ServiceCollectionApplicationExtensions.AddApplicationAsync[TStartupModule](IServiceCollection services, Action1 optionsAction) at Microsoft.Extensions.DependencyInjection.WebApplicationBuilderExtensions.AddApplicationAsync[TStartupModule](WebApplicationBuilder builder, Action`1 optionsAction) at AuthServer.Program.Main(String[] args) in /src/Program.cs:line 27
Steps to reproduce the issue:" I've deployed the auth server MVC app, elasticsearch and kibana on Azure Kubernetes using the helm chart templates in the eshopOnAbp demo solution as a guide. All are running in the same AKS cluster.

This is line 141 from AuthserverModule.cs

var redis = ConnectionMultiplexer.Connect(configuration["Redis:Configuration"]);

This is my auth server deployment.yaml

# Source: authserver/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: authserver
  namespace:  abp-app
spec:
  selector:
    matchLabels:
      app: authserver
  replicas: 1
  template:
    metadata:
      labels:
        app: authserver
    spec:
      containers:
      - name: authserver
        image: cprtestregistry.azurecr.io/authserver:v1.1
        imagePullPolicy: Always
        ports:
        - name: http
          containerPort: 80
        - name: https
          containerPort: 443
        env:
          - name: StringEncryption__DefaultPassPhrase
            value: ert134134t1qqerg
          - name: App__CorsOrigins
            value: https://public-gateway.cloverleafcms.dev,https://web-gateway.cloverleafcms.dev
          - name: App__DisablePII
            value: "true"
          - name: App__RedirectAllowedUrls
            value: https://admin.cloverleafcms.dev,https://admin.cloverleafcms.dev,
          - name: App__SelfUrl
            value: https://auth.cloverleafcms.dev
          - name: AuthServer__Authority
            value: https://auth.cloverleafcms.dev
          - name: AuthServer__RequireHttpsMetadata
            value: "false"
          - name: AuthServer__SwaggerClientId
            value: WebGateway_Swagger
          - name: ConnectionStrings__AdministrationService
            value: Server=***
          - name: ConnectionStrings__IdentityService
            value: Server=***
          - name: ConnectionStrings__SaasService
            value: Server=***
          - name: DOTNET_ENVIRONMENT
            value: Staging
          - name: ElasticSearch__Url
            value: http://elasticsearch
          - name: Redis__Configuration
            value: http://redis

This is my redis service.yaml file:

apiVersion: v1
kind: Service
metadata:
  labels:
    name: redis
  name: redis
spec:
  type: ClusterIP
  ports:
    - name: redis
      port: 6379
  selector:
    app: redis

I've verified that the redis service is running via kubectl get services redis ClusterIP 10.2.92.53

I've also tried using value: redis for the Redis__Configuration env and get this in the pod log: It was not possible to connect to the redis server(s). UnableToConnect on redis:6379/Interactive, Initializing/NotStarted, last: NONE, origin: BeginConnectAsync, outstanding: 0, last-read: 0s ago, last-write: 0s ago, keep-alive: 60s, state: Connecting, mgr: 10 of 10 available, last-heartbeat: never, global: 0s ago, v: 2.2.4.27433 at StackExchange.Redis.ConnectionMultiplexer.ConnectImpl(ConfigurationOptions configuration, TextWriter log) in /_/src/StackExchange.Redis/ConnectionMultiplexer.cs:line 1163

9 Answer(s)

0

gterdem created one year ago
Senior .NET Developer

Do you have redis deployment file? You may not be exposing redis at all.
0

rogercprops created one year ago

This is the redis deployment file and it is running: apiVersion: apps/v1 kind: Deployment metadata: name: redis spec: replicas: 1 selector: matchLabels: app: redis template: metadata: labels: app: redis spec: containers: - image: redis:alpine name: redis ports: - name: redis containerPort: 6379

Here's the output from the kubectl describe Name: redis-696dfdfbfc-nvd2m Namespace: abp-infrastructure Priority: 0 Node: aks-agentpool-19446568-vmss000002/10.0.0.6 Start Time: Thu, 04 May 2023 13:18:54 -0500 Labels: app=redis pod-template-hash=696dfdfbfc Annotations: <none> Status: Running IP: 10.0.0.113 IPs: IP: 10.0.0.113 Controlled By: ReplicaSet/redis-696dfdfbfc Containers: redis: Container ID: containerd://8882f84973aea2ea652dd43e5a2430971cc9d69caab5b67b7ec8855cd221276b Image: redis:alpine Image ID: docker.io/library/redis@sha256:cbcf5bfbc3eaa232b1fa99e539459f46915a41334d46b54bf894f8837a7f071e Port: 6379/TCP Host Port: 0/TCP State: Running Started: Thu, 04 May 2023 13:18:57 -0500 Ready: True Restart Count: 0 Environment: <none> Mounts: /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-jsmn9 (ro) Conditions: Type Status Initialized True Ready True ContainersReady True PodScheduled True Volumes: kube-api-access-jsmn9: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: <nil> DownwardAPI: true QoS Class: BestEffort Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: <none>
0
gterdem created one year ago
Senior .NET Developer
Can you try using

- name: Redis__Configuration value: redis (without http)
0

rogercprops created one year ago

As you can see from my original question I have tried that and got this error: It was not possible to connect to the redis server(s). UnableToConnect on redis:6379/Interactive, Initializing/NotStarted, last: NONE, origin: BeginConnectAsync, outstanding: 0, last-read: 0s ago, last-write: 0s ago, keep-alive: 60s, state: Connecting, mgr: 10 of 10 available, last-heartbeat: never, global: 0s ago, v: 2.2.4.27433 at StackExchange.Redis.ConnectionMultiplexer.ConnectImpl(ConfigurationOptions configuration, TextWriter log) in /_/src/StackExchange.Redis/ConnectionMultiplexer.cs:line 1163
0

gterdem created one year ago
Senior .NET Developer

The problem is about reaching the redis in the kubernetes network for sure.

Your configuration for authserver overrides the redis connection to http://redis. Since redis is running on TCP. As far as I have checked in our deployment scenarios, we configured it as the service name itself without the http protocol.

Other than that, this is not really related to ABP. You can ask it to StackOverflow with kubernetes tags that can be more helpful since I have limited knowledge on kubernetes.
1
rogercprops created one year ago
Please take this up the chain. We're getting increasingly frustrated with the lack of documentation and support from abp.io.

We paid for the commercial license thinking that it would greatly accelerate our deployment of a dotnet microservices architecture. However, we've spent countless hours digging through the code and documentation to get just a basic micro-service to work.

Everything works fine if you run the template solution locally using Docker desktop and Tye exactly as it's delivered in the template. Getting it deployed in the cloud or debugging issues with any modifications or a new service is a different story.

I would think that most companies who are deploying dotnet microservices are going to be deploying them on Azure. Others might use AWS or Google cloud.

If abp.io is going to provide a microservices template that includes the helm charts for deployment:

The helm charts and scripts for the services and web apps that are in the template should work with limited customization

Much better documentation that details how the infrastructure services are being used in the template. One or two sentences don't cut it.

I would challenge one of the engineers to deploy the microservices template into a cloud Kubernetes service and get it to work. Azure would probably be a good place to start. If it doesn't work, fix it and included it in the next update.

This is the 2nd support ticket I've posted without resolution from abp.io. The first one I figured out on my own by digging into the source code on the framework Github repository and finding a demo of what I was trying to do.

Thank you.
0

gterdem created one year ago
Senior .NET Developer

Thanks for your feedback!

The ABP is a development framework as you know. Since we have limited resources to focus on, seamless development experience is our top priority.

We are trying to help with the deployment aswell even if costs lots of time to learn different devops techs and stacks. Although, you can see that ABP has lots of varient for application development such as tiered, non-tiered; using mongodb, postgres, microservices etc. It is not possible to cover all the deployment scenarios.

Even though, we are trying to cover most common deployment scenarios and also the community shares their experiences on specific deployment scenarios in community posts.

For microservice azure specific deployment, we already have an eShopOnAbp sample which is deployed to Azure already; even to artifact.io.

You can also examine the scripts and the values for helm templates at https://github.com/abpframework/eShopOnAbp/tree/main/etc/k8s.

Moreover we are working on documentation for different deployment scenarios one by one but I can not give time for a relase date yet unfortunatelly.

Thank you!
1
rogercprops created one year ago
Thank you for response but let me point out a couple of points since I was trying to use the eshopOnAbp demo source code as a reference:

I followed the instructions in the root folder readme.md and when I ran tye-run it got to line 66 of the tye.yaml file it aborted with the message Error parsing tye.yaml: (66, 7): Path "abpTemplates/eShopOnAbp/.env" was not found.

Because of course there is no .env in the root directory. I'm assuming it's because there are sensitive values in your .env file that you don't want on GitHub. So it would be nice to either have something in the readme.md file with what needs to be included or a .env template with the sensitive values left out.

There is no elasticsearch directory in the helm charts of the eshopOnAbp and in the authserver values.yaml file this is what's under the config property: elasticsearchHost: #

In the microservices template /etc/k8s helm charts all of the values are #, it leaves the developer to figure out what each property is used for and then figure out what value to insert. A lot of that can be extrapolated from the appsettings.json files but it would be nice to have comments in the values file with some examples/instructions for filling out. For example this is from the authserver values.yaml file: config: selfUrl: # corsOrigins: # allowedRedirectUrls: # disablePii: # identityService: connString: # administrationService: connString: # saasService: connString: # authServer: authority: # requireHttpsMetadata: # swaggerClientId: # dotnetEnv: # redisHost: # rabbitmqHost: # elasticsearchUrl: # I would only ask if you're going to provide a template based on a solution you have running in a cloud Kubernetes environment that a) The documentation is complete and accurate. b) If the documentation is complete and accurate it actually works when you follow the instructions

Sorry to be negative, but I wanted to provide some feedback on what would help my team and other companies as well.
0

gterdem created one year ago
Senior .NET Developer

Hi,

We have .env file explanation where it is used for paypal information at https://github.com/abpframework/eShopOnAbp#instructions but maybe we should make it more clear.

I've created internal issue about documenting the k8s deployment especially the differences between individual helm chart deployment and the whole application deployment.

Thank you for your feedback!

Have an answer to this question? Log in and write your answer.