[Security Solution][Detection Engine] fixes IM rule failure when frozen tier node is not available (#200621)

## Summary

- addresses https://github.com/elastic/security-team/issues/11117

### How to test


1. Create a deployment with cold and frozen data tiers and use following
commands to create index and ILM

<details>
<summary>Data tiers commands</summary>

```JSON

PUT /_cluster/settings
{
  "persistent": {
    "indices.lifecycle.poll_interval": "10s"
  }
}


PUT /_ilm/policy/filtering_data_tiers
{
  "policy": {
    "phases": {
        "frozen": {
          "min_age": "10s",
          "actions": {
            "searchable_snapshot": {
              "snapshot_repository": "found-snapshots",
              "force_merge_index": true
            }
          }
        },
        "hot": {
          "min_age": "0ms",
          "actions": {
            "set_priority": {
              "priority": 100
            }
          }
        }
    }
  }
}


PUT /_index_template/filtering_data_tiers_template
{
  "index_patterns": [
    "filtering_data_tiers*"
  ],
  "template": {
    "settings": {
      "index.lifecycle.name": "filtering_data_tiers",
      "index.lifecycle.rollover_alias": "test-filtering_data_tiers"
    },
    "mappings": {
      "_meta": {
        "version": "1.6.0"
      },
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "host": {
          "properties": {
            "name": {
              "type": "keyword",
              "ignore_above": 1024
            }
          }
        }
      }
    }
  }
}

PUT /filtering_data_tiers-000001
{
  "aliases": {
    "filtering_data_tiers": {
      "is_write_index": true
    }
  }
}


POST filtering_data_tiers/_doc
{
  "@timestamp": "2024-07-08T17:00:01.000Z",
  "host.name": "test-0"
}


```

</details>

2. Wait until document moves to frozen tier
3. Run another set of commands to persist document in hot tier

<details>
<summary>Data tiers commands</summary>


```JSON

PUT /_ilm/policy/filtering_data_tiers
{
  "policy": {
    "phases": {
        "frozen": {
          "min_age": "100h",
          "actions": {
            "searchable_snapshot": {
              "snapshot_repository": "found-snapshots",
              "force_merge_index": true
            }
          }
        },
        "hot": {
          "min_age": "0ms",
          "actions": {
            "set_priority": {
              "priority": 100
            }
          }
        }
    }
  }
}


PUT /filtering_data_tiers-000002
{
  "aliases": {
    "filtering_data_tiers": {
      "is_write_index": true
    }
  }
}


POST filtering_data_tiers/_doc
{
  "@timestamp": "2024-11-08T17:00:01.000Z",
  "host.name": "test-1"
}


```
</details>

4. Pause frozen tier node (admin permissions needed for this) or
increase memory of it, forcing node to become unavailable for short
period of time.
5. Run IM rule with [advanced
setting](https://www.elastic.co/guide/en/security/current/advanced-settings.html#exclude-cold-frozen-data-rule-executions)
filtering out frozen data tier
6. Rule should not fail and generate an alert from document in a hot
tier

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This commit is contained in:
Vitalii Dmyterko 2024-11-20 10:00:16 +00:00 committed by GitHub
parent b320a37d8b
commit ee397d66b8
No known key found for this signature in database
GPG key ID: B5690EEEBB952194

View file

@ -142,6 +142,10 @@ export const createThreatSignals = async ({
await services.scopedClusterClient.asCurrentUser.openPointInTime({
index: threatIndex,
keep_alive: THREAT_PIT_KEEP_ALIVE,
// @ts-expect-error client support this option, but it is not documented and typed yet, but we need this fix in 8.16.2.
// once support added we should remove this expected type error
// https://github.com/elastic/elasticsearch-specification/issues/3144
allow_partial_search_results: true,
})
).id;
const reassignThreatPitId = (newPitId: OpenPointInTimeResponse['id'] | undefined) => {