Activities of "alexander.nikonov"

@liangshiwei - ok, if you are sure that this is solely EF Core / Oracle issue, please close the ticket and restore the points. Thanks.

• ABP Framework version: v7.0.1.
• UI Type: Angular
• Database System: EF Core (Oracle)
• Auth Server Separated

I'm trying to insert about 400 entries to the table, the table is simple. The operation is very slow (maybe it's due to the communication between localhost where the app server is and Azure DB server, but I just want to find out before jumping to the conclusions).

Here is the code (chunkSize is 100):

    private async Task TryCreateChunkAsync(List<GeoPoint> geoPoints, int chunkSize)
    {
        var dbContext = await _geoPointRepository.GetDbContextAsync();
        dbContext.ChangeTracker.AutoDetectChangesEnabled = false;
        for (var skip = 0; skip < geoPoints.Count; skip += chunkSize)
        {
            using (var uow = _unitOfWorkManager.Begin(requiresNew: true, isTransactional: false))
            {
                await _geoPointRepository.InsertManyAsync(geoPoints.Skip(skip).Take(chunkSize), autoSave: false);
                await uow.CompleteAsync();
            }
        }
        dbContext.ChangeTracker.AutoDetectChangesEnabled = true;
    }

I don't know if this approach makes sense - I also tried a different approach, using DbContext, prior to that set BatchSize to 100 for our database:

var dbContext = await _geoPointRepository.GetDbContextAsync();
((MyDbContext)dbContext).AddRange(geoPoints);
await dbContext.SaveChangesAsync();

No difference. It is still very slow. There is no third-party tools for bulk operations for Oracle, so this is not an option in any case. Besides, according to some article online, the AddRange approach on 100 entries per DB round-trip is only 1.5 times slower than using bulk insert on such amount of data.

Another thing - I am not sure that ABP logging makes the insertion much slower, but I was unable to turn off these things:

despite applying [DisableAuditing] attribute to Controller or AppService method. Please note, that I don't want to use IgnoreUrls, since the Url might change in future - only the method body is relevant. I also don't want to apply [DisableAuditing] to the entity, because in some other methods I DO want the logging as usual.

What puzzles me is: >

Executed DbCommand (XXXms)

all the DbCommand take more or less appropriate amount of time (each DbCommand takes less than 1 second).

But look at this:

Executed action YYY.Controllers.GeoPoints.GeoPointController.UploadAsync (YYY.HttpApi) in 53197.6736ms

I can't explain this. I turned off the auditing by putting the attribute [DisableAuditing] on the entity class.The time for all the operations does not sum up. Why the total time is so huge, what most of this total time is spent for?? It is somehow related to the fact I'm inserting many records. Because when I do some elementary DB operation - the API request time is normal.

I have identified the root cause of the problem. There were indeed like two caches.

By chance I have noticed that IDistributedCache has IgnoreMultiTenancy property. It is false by default. It should be true, since my cache item stores data for all tenants of the apps. ApplicationService works under specific tenant (of the current user), but not RabbitMqReceiver.

I even was not aware about this feature of the cache. So when I set it to true - the cache started working as expected.

Could you please reimburse the points for this ticket and close it? Thank you.

I have created the example. But I was unable to reproduce the problem which exists in our application: in the test app, a memory cache is shared between application layers (as expected) and the data is not re-read from DB, even if the item has already been put to cache from by a call from different tier. So I don't know what to do next. I cannot share the source code.

I have a bunch of cache instances, which store various parameters - with long expiration:

        var cockpitCacheNotificationItem = await cockpitNotificationCache.GetOrAddAsync
        (
            CockpitNotificationCacheItem.Key,
            GetCockpitNotificationCacheAsync,
            () => new DistributedCacheEntryOptions
            {
                AbsoluteExpirationRelativeToNow = TimeSpan.FromDays(365)
            }
        );

All of such are singleton DI, residing - as well as CacheManager - on Application tier. And all are missed (re-read from DB) when they actually should be present in the cache. But ONLY if I invoke the cache operations via CacheManager instance which is retrieved on HttpApi.Host tier (no matter this way: serviceProvider.GetRequiredService<CacheManager>() or this way: serviceProvider.GetRequiredService<IHost>.Services.GetRequiredService<CacheManager>())

I understand that it has nothing to do with RabbitMQ. I just don't know what is the simplest way to invoke some action in HttpApi.Host project, if it's not an API call initiated action (as in scenario 1)

You got me wrong. Event received from another app via RabbitMQ. But it's received in THIS app. I just need a simple scenario to imitate this. Do I need RabbitMQ at all or can reproduce it in other way? After all, I can even send the message from THIS app too... I just didn't want to install RabbitMQ, hope could do it in an easier way.

I still cannot wrap my head around it - how to reproduce this scenario in the test project...

One part is clear:

But another part - not. In the original project, RabbitMq event is received from another running app. Receiver of this event resides in HttpApi.Host project and tries to update the cache. Could you please suggest me a simple example - how to emulate this in the test project? I don't want to make it too complex. Thanks.

Alas, I cannot do that. I have a test project which used to work a while ago. Now it does not - I don't remember why, need to start bugfixing it. I can see there is no Test DB anymore. I run Migration project, but it fails...

Seems like I need to create a brand-new test project. This does not work. I need to recall how to do that.. Probably via AbpSuite...

I thought I have dealt with #1, but now all of the sudden it does not work again!

The framework will register all IDistributedCache as Singleton. You dont need to register it again.

Thank you, I've removed excessive registrations.

Memory cache cannot cross project/application process

This is an important note. But I want to clarify. Let's say I have IDistributedCache<CacheItem> and CacheManager, which is also registered as a singleton. CacheManager reads and updates the CacheItem. Both classes - CacheItem and CacheManager reside in Application tier. So when I use CacheManager DI from Application tier - all cache operations work predictable. However, when I invoke CacheManager DI to update the cache from HttpApi.Host tier - I have the feeling that it updates a different cache! So - are you implying that it's a normal behavior and these are in fact two different caches - on two different solution tiers despite the fact that HttpApi.Host layer has project reference to Application?

Sorry - I am lack of time for building the scenario in a test project, considering this described issue is not the only weird thing, there are more coming (this all is related):

1. I have managed to overcome the described problem: instead of retrieving cockpitCacheManager instance using _serviceProvider.GetRequiredService<CockpitCacheManager>() inside MyRabbitMqReceiver I retrieved it using _serviceProvider.GetRequiredService<IHost>().Services.GetRequiredService<CockpitCacheManager>(); - this way the cache item was not null anymore; to me it looked like using a common IServiceProvider for both singletons;

2. however updating the cache still causes me the troubles - I don't know why, just thinking aloud: probably it also has something to do with DI...

What is a proper way to update the cache? Am I missing something important here or doing it in a wrong way? Trying to keep the code simple and schematic as much as possible (as always, I am not allowed to share the full code):

    public class CockpitCacheManager
    {
        ...
        public async Task<bool> UpdateAndNotifyClientsAsync(CockpitCacheUpdateDto input)
        {
            using (await _cockpitCacheUpdateSemaphore.LockAsync())
            {
                (var cockpitCacheItem, var isCacheHit) = await GetItemAsync(isForceRefresh); //reading the cache using GetOrAddAsync under the hood

                if (cockpitCacheItem == null)
                {
                    return false;
                }

                using (var scope = _host.Services.CreateScope()) //should the scope look like this?
                {
                    var caseRepository = scope.ServiceProvider.GetRequiredService<ICaseRepository>();
                    ...
                    //updating cache item:
                    if (value1 != null) //input parameter 1
                    {
                        cockpitCacheItem.Field1 = value1;
                    }
                    ...
                    if (valueN != null) //input parameter N
                    {
                        cockpitCacheItem.FieldN = valueN;
                    }
                    ...
                    if (cacheChanged) //old and new cache hash is calculated
                    {
                        await _cockpitCache.SetAsync(CockpitCacheItem.Key, cockpitCacheItem);
                    }
                }
            }
        }
    }