APM profile merger crashes on cleared versioned fields ([ts, None] in metadata.fields) #904

Summary

validate_list_values() in tracardi/service/merging/merger.py:25 rejects None values in lists, but Tracardi's own metadata.fields tracker produces [timestamp, None] tuples whenever a versioned field is cleared or was never set. The result: profile merging crashes for any profile that ever had a null-valued versioned field.

Reproduction

Run staging Tracardi 1.1.6 + APM 1.1.27. When an event arrives that clears a profile field (e.g. a logout event that nulls traits.lastFcmToken), the profile's metadata.fields document gets an entry like:

"metadata": {
  "fields": {
    "traits.lastFcmToken": [1776420649.007557, null]
  }
}

On the next APM cycle, _deduplicate() calls deduplicate_profile → ProfileMerger.compute_one_profile → merger._merge_traits_and_data → dict_merge → append → validate_list_values. None is not in (str, int, float, bool), so the function raises:

ValueError: Invalid value in list `[1776420649.007557, None]`

APM catches the exception in main.py:73 (logger.error(...)) and moves on, leaves the profile marked for merging, and repeats every ~18s forever.

Impact

On a production-like staging with ~2 000 profiles, 19 profiles had null-valued versioned fields. The APM worker produced ~3 000 identical error log lines per 18s cycle, growing the container log from empty to 30 GB in roughly a few days (~500 MB/hour). Any of these traits is affected — the ones we observed on staging:

• traits.clientId (14 profiles)
• traits.lastFcmToken (6)
• traits.kycProvider (4)
• traits.lastPushDeliveredTitle / lastPushDeliveredAt (2 each)
• traits.vendorGroup*, traits.lastPushClicked*, etc.

Root cause

The merger as a whole already supports None values elsewhere — test/unit/test_merger.py:14-32 asserts dict_merge handles "e": None, and test/unit/test_merger.py:261 asserts "d": [None, "d"] is a valid merge result. Only validate_list_values is the outlier.

Fix proposal

One-line change:

def validate_list_values(values):
    for value in values:
-        if not isinstance(value, (str, int, float, bool)):
+        if value is not None and not isinstance(value, (str, int, float, bool)):
            raise ValueError("Invalid value in list `{}`".format(values))

Same bug present on 1.1.x, 1.2.x, 2.0.x branches (same file, same line). Happy to port after the initial PR is reviewed.

PR

Submitting a PR with the fix + tests against 1.1.x shortly (verified against Tracardi 1.1.6 / APM 1.1.27 on our staging).