Static template synchronization accumulates duplicate content records in Text Template Management (ABP 10.7.1) #10958

3 Answer(s)

• 

  0

  AI-Bot created 16 hours ago

  Support Team ABP Agent

  • Your suspicion is consistent with the behavior you described: if UpdateChangedTemplatesAsync calls DeleteByDefinitionIdAsync(record.Key.Id) where record.Key.Id is the newly generated (in-memory/serialized) definition id, then no rows will be deleted for an already-persisted definition, and the subsequent insert (after the incoming content is re-pointed to templateRecordInDatabase.Id) will create additional rows with the same persisted DefinitionId + FileName. That exactly explains “1 → 2 → 3” growth across repeated synchronizations.

  • I can’t directly confirm the exact line in Volo.Abp.TextTemplateManagement.Domain 10.7.1 from source here because the code-search repository tool is currently unavailable in this environment, so I can’t quote the implementation verbatim. However, the fix you proposed is the correct one if the method indeed uses the new definition id:

  await TextTemplateContentRepository.DeleteByDefinitionIdAsync(templateRecordInDatabase.Id);
  

  (i.e., delete by the persisted definition record id, not the incoming/new id).

  • Safe cleanup guidance for existing duplicates in SQL Server (keep one row per (DefinitionId, FileName) and delete the rest):

    1. Inspect duplicates

  SELECT DefinitionId, FileName, COUNT(*) AS Cnt
    FROM AbpTextTemplateDefinitionContentRecords
    GROUP BY DefinitionId, FileName
    HAVING COUNT(*) > 1;
  

  2. Delete duplicates (keeping the newest by CreationTime; if you don’t have CreationTime, you can use Id ordering)

  ;WITH Dups AS
    (
        SELECT
            Id,
            DefinitionId,
            FileName,
            ROW_NUMBER() OVER (
                PARTITION BY DefinitionId, FileName
                ORDER BY CreationTime DESC, Id DESC
            ) AS rn
        FROM AbpTextTemplateDefinitionContentRecords
    )
    DELETE FROM AbpTextTemplateDefinitionContentRecords
    WHERE Id IN (SELECT Id FROM Dups WHERE rn > 1);
  

  3. Prevent re-occurrence at the database level (recommended)
     Add a unique index to enforce one row per (DefinitionId, FileName):

  CREATE UNIQUE INDEX UX_AbpTTDCR_DefinitionId_FileName
    ON AbpTextTemplateDefinitionContentRecords (DefinitionId, FileName);
  

  If ABP legitimately needs multiple rows per (DefinitionId, FileName) for your scenario, don’t add this; but based on your expectation (“one current record per definition and filename”), this constraint matches the intended model.

  • Applying a product fix:
    • If you need an immediate workaround without waiting for a patched NuGet: you can replace/override the service responsible for static template saving in your app (via DI decoration/replacement) and apply the corrected delete behavior before inserting. (Exact class/interface names should be taken from the module source, but I can’t verify them with the unavailable code search tool in this session.)

  Sources (validated)

  • https://abp.io/docs/10.7/modules/text-template-management
  • https://docs.abp.io/en/abp/latest/Text-Templating

  ---

  This response was generated by AI. If it didn’t solve your problem, feel free to post a new answer — our support team will jump in to help you soon.

  Markdown supported.

  Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post)

  Save Cancel
• 

  0

  s.beckers created 16 hours ago

  I think this needs to be checked in the commercial source code.

  Markdown supported.

  Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post)

  Save Cancel
• 

  0

  maliming created 13 hours ago

  Support Team Fullstack Developer

  Hi,

  The sync deletes the old content with the wrong definition id, so each new template adds another copy of the existing content. Changing only that line isn't enough. The content would then be deleted without being added back.

  You can replace the saver with the class below for now. Add it to your *.Domain project:

  using System;
  using System.Collections.Generic;
  using System.Linq;
  using System.Text;
  using System.Text.Json;
  using System.Text.Json.Serialization.Metadata;
  using System.Threading.Tasks;
  using Microsoft.Extensions.Caching.Distributed;
  using Microsoft.Extensions.Options;
  using Volo.Abp;
  using Volo.Abp.Caching;
  using Volo.Abp.DependencyInjection;
  using Volo.Abp.DistributedLocking;
  using Volo.Abp.Json.SystemTextJson.Modifiers;
  using Volo.Abp.TextTemplateManagement.TextTemplates;
  using Volo.Abp.TextTemplating;
  using Volo.Abp.Threading;
  using Volo.Abp.Uow;
  
  namespace MyCompanyName.MyProjectName.TextTemplates;
  
  [Dependency(ReplaceServices = true)]
  [ExposeServices(typeof(IStaticTextTemplateSaver))]
  public class MyStaticTextTemplateSaver : IStaticTextTemplateSaver, ITransientDependency
  {
      protected IStaticTemplateDefinitionStore StaticStore { get; }
      protected ITextTemplateDefinitionRecordRepository TextTemplateRepository { get; }
      protected ITextTemplateDefinitionContentRecordRepository TextTemplateContentRepository { get; }
      protected ITextTemplateDefinitionSerializer TextTemplateSerializer { get; }
      protected IDistributedCache Cache { get; }
      protected IApplicationInfoAccessor ApplicationInfoAccessor { get; }
      protected IAbpDistributedLock DistributedLock { get; }
      protected AbpTextTemplatingOptions TemplateOptions { get; }
      protected ICancellationTokenProvider CancellationTokenProvider { get; }
      protected AbpDistributedCacheOptions CacheOptions { get; }
      protected IUnitOfWorkManager UnitOfWorkManager { get; }
  
      public MyStaticTextTemplateSaver(
          IStaticTemplateDefinitionStore staticStore,
          ITextTemplateDefinitionRecordRepository textTemplateRepository,
          ITextTemplateDefinitionContentRecordRepository textTemplateContentRepository,
          ITextTemplateDefinitionSerializer textTemplateSerializer,
          IDistributedCache cache,
          IOptions<AbpDistributedCacheOptions> cacheOptions,
          IApplicationInfoAccessor applicationInfoAccessor,
          IAbpDistributedLock distributedLock,
          IOptions<AbpTextTemplatingOptions> templateOptions,
          ICancellationTokenProvider cancellationTokenProvider,
          IUnitOfWorkManager unitOfWorkManager)
      {
          StaticStore = staticStore;
          TextTemplateRepository = textTemplateRepository;
          TextTemplateContentRepository = textTemplateContentRepository;
          TextTemplateSerializer = textTemplateSerializer;
          Cache = cache;
          CacheOptions = cacheOptions.Value;
          ApplicationInfoAccessor = applicationInfoAccessor;
          DistributedLock = distributedLock;
          TemplateOptions = templateOptions.Value;
          CancellationTokenProvider = cancellationTokenProvider;
          UnitOfWorkManager = unitOfWorkManager;
      }
  
      [UnitOfWork]
      public virtual async Task SaveAsync()
      {
          await using var applicationLockHandle = await DistributedLock.TryAcquireAsync(
              $"{CacheOptions.KeyPrefix}_{ApplicationInfoAccessor.ApplicationName}_AbpTextTemplateUpdateLock");
  
          if (applicationLockHandle == null)
          {
              return;
          }
  
          var hashCacheKey = $"{CacheOptions.KeyPrefix}_{ApplicationInfoAccessor.ApplicationName}_AbpTextTemplatesHash";
          var cachedHash = await Cache.GetStringAsync(hashCacheKey, CancellationTokenProvider.Token);
  
          var templateRecords = await TextTemplateSerializer.SerializeAsync(await StaticStore.GetAllAsync());
          var currentHash = CalculateHash(templateRecords, TemplateOptions.DeletedTemplates);
  
          if (cachedHash == currentHash)
          {
              return;
          }
  
          await using (var commonLockHandle = await DistributedLock.TryAcquireAsync(
                           $"{CacheOptions.KeyPrefix}_Common_AbpTextTemplateUpdateLock",
                           TimeSpan.FromMinutes(5)))
          {
              if (commonLockHandle == null)
              {
                  throw new AbpException("Could not acquire distributed lock for saving static templates!");
              }
  
              using (var unitOfWork = UnitOfWorkManager.Begin(requiresNew: true, isTransactional: true))
              {
                  try
                  {
                      if (await UpdateChangedTemplatesAsync(templateRecords))
                      {
                          await Cache.SetStringAsync(
                              $"{CacheOptions.KeyPrefix}_AbpInMemoryTextTemplateCacheStamp",
                              Guid.NewGuid().ToString(),
                              new DistributedCacheEntryOptions { SlidingExpiration = TimeSpan.FromDays(30) },
                              CancellationTokenProvider.Token);
                      }
                  }
                  catch
                  {
                      try
                      {
                          await unitOfWork.RollbackAsync();
                      }
                      catch
                      {
                          /* ignored */
                      }
  
                      throw;
                  }
  
                  await unitOfWork.CompleteAsync();
              }
          }
  
          await Cache.SetStringAsync(
              hashCacheKey,
              currentHash,
              new DistributedCacheEntryOptions { SlidingExpiration = TimeSpan.FromDays(30) },
              CancellationTokenProvider.Token);
      }
  
      protected virtual async Task<bool> UpdateChangedTemplatesAsync(
          Dictionary<TextTemplateDefinitionRecord, List<TextTemplateDefinitionContentRecord>> templateRecords)
      {
          var newRecords = new List<TextTemplateDefinitionRecord>();
          var newContentRecords = new List<TextTemplateDefinitionContentRecord>();
          var changedRecords = new List<TextTemplateDefinitionRecord>();
  
          var templateRecordsInDatabase = (await TextTemplateRepository.GetListAsync()).ToDictionary(x => x.Name);
  
          foreach (var record in templateRecords)
          {
              var templateRecordInDatabase = templateRecordsInDatabase.GetOrDefault(record.Key.Name);
              if (templateRecordInDatabase == null)
              {
                  newRecords.Add(record.Key);
                  newContentRecords.AddRange(record.Value);
                  continue;
              }
  
              await TextTemplateContentRepository.DeleteByDefinitionIdAsync(templateRecordInDatabase.Id);
              newContentRecords.AddRange(record.Value.Select(x => new TextTemplateDefinitionContentRecord(
                  x.Id,
                  templateRecordInDatabase.Id,
                  x.FileName,
                  x.FileContent)));
  
              if (record.Key.HasSameData(templateRecordInDatabase))
              {
                  continue;
              }
  
              templateRecordInDatabase.Patch(record.Key);
              changedRecords.Add(templateRecordInDatabase);
          }
  
          var deletedRecords = templateRecordsInDatabase.Values
              .Where(x => TemplateOptions.DeletedTemplates.Contains(x.Name))
              .ToList();
  
          if (newRecords.Any())
          {
              await TextTemplateRepository.InsertManyAsync(newRecords);
          }
  
          if (newContentRecords.Any())
          {
              await TextTemplateContentRepository.InsertManyAsync(newContentRecords);
          }
  
          if (changedRecords.Any())
          {
              await TextTemplateRepository.UpdateManyAsync(changedRecords);
          }
  
          if (deletedRecords.Any())
          {
              await TextTemplateContentRepository.DeleteByDefinitionIdAsync(deletedRecords.Select(x => x.Id).ToArray());
              await TextTemplateRepository.DeleteManyAsync(deletedRecords);
          }
  
          return newRecords.Any() || changedRecords.Any() || deletedRecords.Any();
      }
  
      protected virtual string CalculateHash(
          Dictionary<TextTemplateDefinitionRecord, List<TextTemplateDefinitionContentRecord>> templateRecords,
          IEnumerable<string> deletedTemplates)
      {
          var jsonSerializerOptions = new JsonSerializerOptions
          {
              TypeInfoResolver = new DefaultJsonTypeInfoResolver
              {
                  Modifiers =
                  {
                      new AbpIgnorePropertiesModifiers<TextTemplateDefinitionRecord, Guid>().CreateModifyAction(x => x.Id),
                      new AbpIgnorePropertiesModifiers<TextTemplateDefinitionContentRecord, Guid>().CreateModifyAction(x => x.Id),
                      new AbpIgnorePropertiesModifiers<TextTemplateDefinitionContentRecord, Guid>().CreateModifyAction(x => x.DefinitionId)
                  }
              }
          };
  
          var stringBuilder = new StringBuilder();
          stringBuilder.Append("TemplateRecords:");
          stringBuilder.AppendLine(JsonSerializer.Serialize(templateRecords.Keys, jsonSerializerOptions));
          stringBuilder.AppendLine(JsonSerializer.Serialize(templateRecords.Values, jsonSerializerOptions));
          stringBuilder.Append("DeletedTemplate:");
          stringBuilder.Append(deletedTemplates.JoinAsString(","));
  
          return stringBuilder.ToString().ToMd5();
      }
  }
  

  On the next startup it deletes the old content of each template and inserts the current files once, so the existing duplicates are removed. You don't need to clean them up manually.

  We'll fix this in the next patch release. Your ticket has been refunded.

  Thanks

  Markdown supported.

  Copy, paste, or drag & drop images and files (max 100 MB per file, 100 MB total per post)

  Save Cancel