Skip to content

Long fw-headless syncs result in premature LcmCache eviction #2050

@myieye

Description

@myieye

Describe the bug
Here's a sync that ran for over 2 hours: (trace)
(Yes...that's pretty wild: large project (only ~10,000 entries, but 460MB, so probably loads of media), multiple "migrations" (patching missing translation IDs, maybe even adding morph type and/or publications for the first time) and it was the first time media has been synced. Anyway...)

Image

I'm 90% sure that the null-pointer-exception it eventually ran into, was a result of the LcmCache having been prematurely disposed:

System.NullReferenceException: Object reference not set to an instance of an object. at FwDataMiniLcmBridge.Api.FwDataMiniLcmApi.SetSensePartOfSpeech(Guid senseId, Nullable`1 partOfSpeechId) in /FwLite/FwDataMiniLcmBridge/Api/FwDataMiniLcmApi.cs:line 1512

After disposing LcmCache its ServiceLocator becomes null.

The log file was unfortunately too truncated to see when the cache was evicted and disposed, but there was a "Explicitly Closing project" log that should have been followed by a "Evicting project {ProjectFileName} from cache" and "FW Data Project {ProjectFileName} disposed". The fact that those logs were missing indicates that the cache had already been evicted and disposed.

We use a sliding-expiration of 30 minutes. But (1) "sliding" doesn't happen by simply using the cache, but by getting it, which doesn't happen often (it's designed for UI usage, which news up an FwDataApi on every "request") and (2) potentially a lot of time was spent only on MediaFileService.SyncMediaFiles(Guid projectId, LcmMediaService lcmMediaService), which does not utilise the LcmCache. (1) seems more important.

Ideas for improvement:

  • Somehow make usages of the LcmCache "bump" the sliding eviction time
  • Maybe explicitly check if the LcmCache has been disposed (so errors are more explicit than a NullPointerException)
  • What we really want is for syncs to go way faster, even for projects like this one.

The good news for the affected project is that the sync accomplished a lot. So, it's likely it'll work on the next attempt.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions