diff --git a/modules/apm/NAMING.md b/modules/apm/NAMING.md index 8e8d1bf2463e..31cad34d0470 100644 --- a/modules/apm/NAMING.md +++ b/modules/apm/NAMING.md @@ -17,13 +17,13 @@ The **hierarchy** should be built by putting "more common" elements at the begin Example: * prefer `es.indices.docs.deleted.total `to `es.indices.total.deleted.docs` -* This way you can later add` es.indices.docs.count, es.indices.docs.ingested.total`, etc.) +* This way you can later add` es.indices.docs.total, es.indices.docs.ingested.total`, etc.) Prefix metrics: * Always use `es` as our root application name: this will give us a separate namespace and avoid any possibility of clashes with other metrics, and quick identification of Elasticsearch metrics on a dashboard. * Follow the root prefix with a simple module name, team or area of code. E.g. `snapshot, repositories, indices, threadpool`. Notice the mix of singular and plural - here this is intentional, to reflect closely the existing names in the codebase (e.g. `reindex` and `indices`) -* In building a metric name, look for existing prefixes (e.g. module name and/or area of code, e.g. `blob_cache`) and for existing sub-elements as well (e.g. `error`) to build a good, consistent name. E.g. prefer the consistent use of `error.count` rather than introducing `failures`, `failed.count` or `errors`.` ` -* Avoid having sub-metrics under a name that is also a metric (e.g. do not create names like `es.repositories.elements`,` es.repositories.elements.utilization`; use` es.repositories.element.count` and` es.repositories.element.utilization `instead). Such metrics are hard to handle well in Elasticsearch, or in some internal structures (e.g. nested maps). +* In building a metric name, look for existing prefixes (e.g. module name and/or area of code, e.g. `blob_cache`) and for existing sub-elements as well (e.g. `error`) to build a good, consistent name. E.g. prefer the consistent use of `error.total` rather than introducing `failures`, `failed.total` or `errors`.` ` +* Avoid having sub-metrics under a name that is also a metric (e.g. do not create names like `es.repositories.elements`,` es.repositories.elements.utilization`; use` es.repositories.element.total` and` es.repositories.element.utilization `instead). Such metrics are hard to handle well in Elasticsearch, or in some internal structures (e.g. nested maps). Keep the hierarchy compact: do not add elements if you don’t need to. There is a description field when registering a metric, prefer using that as an explanation. \ For example, if emitting existing metrics from node stats, do not use the whole “object path”, but choose the most significant terms. @@ -35,7 +35,7 @@ The metric name can be generated but there should be no dynamic or variable cont * Rule of thumb: you should be able to do aggregations (e.g. sum, avg) across a dimension of a given metric (without the need to aggregate over different metric names); on the other hand, any aggregation across any dimension of a given metric should be meaningful. * There might be exceptions of course. For example: * When similar metrics have significantly different implementations/related metrics. \ - If we have only common metrics like `es.repositories.element.count, es.repositories.element.utilization, es.repositories.writes.total` for every blob storage implementation, then `s3,azure` should be an attribute. \ + If we have only common metrics like `es.repositories.element.total, es.repositories.element.utilization, es.repositories.writes.total` for every blob storage implementation, then `s3,azure` should be an attribute. \ If we have specific metrics, e.g. for s3 storage classes, prefer using prefixed metric names for the specific metrics: es.repositories.s3.deep_archive_access.total (but keep `es.repositories.elements`) * When you have a finite and fixed set of names it might be OK to have them in the name (e.g. "`young`" and "`old`" for GC generations). @@ -47,12 +47,19 @@ Examples : * es.indices.storage.write.io, instead of es.indices.storage.write.bytes_per_sec * These can all be composed with the suffixes below, e.g. es.process.jvm.collection.time.total, es.indices.storage.write.total to represent the monotonic sum of time spent in GC and the total number of bytes written to indices respectively. -**Pluralization** and **suffixes**: -* If the metric is unit-less, use plural: `es.threadpool.activethreads`, `es.indices.docs` -* Use `total` as a suffix for monotonic sums (e.g. es.indices.docs.deleted.total) -* Use `count` to represent the count of "things" in the metric name/namespace (e.g. if we have `es.process.jvm.classes.loaded`, we will express the number of classes currently loaded by the JVM as es.process.jvm.classes.loaded.count, and the total number of classes loaded since the JVM started as es.process.jvm.classes.loaded.total +**Suffixes**: +* Use `total` as a suffix for monotonic metrics (always increasing counter) (e.g. es.indices.docs.deleted.total) + * Note: even though async counter is reporting a total cumulative value, it is till monotonic. +* Use `current` to represent the non-monotonic metrics (like gauges, upDownCounters) + * e.g. `current` vs `total` We can have es.process.jvm.classes.loaded.current to express the number of classes currently loaded by the JVM, and the total number of classes loaded since the JVM started as es.process.jvm.classes.loaded.total * Use `ratio` to represent the ratio of two measures with identical unit (or unit-less) or measures that represent a fraction in the range [0, 1]. Examples: * Exception: consider using utilization when the ratio is between a usage and its limit, e.g. the ratio between es.process.jvm.heap.usage and es.process.jvm.heap.limit should be es.process.jvm.heap.utilization +* Use `status` to represent enum like gauges. example es.health.overall.red.status have values 1/0 to represent true/false +* Use `usage` to represent the amount used ouf of the known resource size +* Use `size` to represent the overall size of the resource measured +* Use `utilisation` to represent a fraction of usage out of the overall size of a resource measured +* Use `histogram` to represent instruments of type histogram +* Use `time` to represent passage of time * If it has a unit of measure, then it should not be plural (and also not include the unit of measure, see above). Examples: es.process.jvm.collection.time, es.process.mem.virtual.usage, es.indices.storage.utilization ### Attributes diff --git a/modules/apm/src/main/java/org/elasticsearch/telemetry/apm/AbstractInstrument.java b/modules/apm/src/main/java/org/elasticsearch/telemetry/apm/AbstractInstrument.java index 72c6ccf90587..9329556ff0f3 100644 --- a/modules/apm/src/main/java/org/elasticsearch/telemetry/apm/AbstractInstrument.java +++ b/modules/apm/src/main/java/org/elasticsearch/telemetry/apm/AbstractInstrument.java @@ -11,6 +11,7 @@ package org.elasticsearch.telemetry.apm; import io.opentelemetry.api.metrics.Meter; import org.elasticsearch.core.Nullable; +import org.elasticsearch.telemetry.apm.internal.MetricNameValidator; import org.elasticsearch.telemetry.metric.Instrument; import java.security.AccessController; @@ -23,6 +24,7 @@ import java.util.function.Function; * An instrument that contains the name, description and unit. The delegate may be replaced when * the provider is updated. * Subclasses should implement the builder, which is used on initialization and provider updates. + * * @param delegated instrument */ public abstract class AbstractInstrument implements Instrument { @@ -50,19 +52,13 @@ public abstract class AbstractInstrument implements Instrument { } protected abstract static class Builder { - private static final int MAX_NAME_LENGTH = 255; protected final String name; protected final String description; protected final String unit; public Builder(String name, String description, String unit) { - if (name.length() > MAX_NAME_LENGTH) { - throw new IllegalArgumentException( - "Instrument name [" + name + "] with length [" + name.length() + "] exceeds maximum length [" + MAX_NAME_LENGTH + "]" - ); - } - this.name = Objects.requireNonNull(name); + this.name = MetricNameValidator.validate(name); this.description = Objects.requireNonNull(description); this.unit = Objects.requireNonNull(unit); } diff --git a/modules/apm/src/main/java/org/elasticsearch/telemetry/apm/internal/MetricNameValidator.java b/modules/apm/src/main/java/org/elasticsearch/telemetry/apm/internal/MetricNameValidator.java new file mode 100644 index 000000000000..1a698b778687 --- /dev/null +++ b/modules/apm/src/main/java/org/elasticsearch/telemetry/apm/internal/MetricNameValidator.java @@ -0,0 +1,142 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0 and the Server Side Public License, v 1; you may not use this file except + * in compliance with, at your election, the Elastic License 2.0 or the Server + * Side Public License, v 1. + */ + +package org.elasticsearch.telemetry.apm.internal; + +import java.util.Objects; +import java.util.Set; +import java.util.regex.Matcher; +import java.util.regex.Pattern; +import java.util.stream.Collectors; + +public class MetricNameValidator { + private static final Pattern ALLOWED_CHARACTERS = Pattern.compile("[a-z][a-z0-9_]*"); + static final Set ALLOWED_SUFFIXES = Set.of( + "total", + "current", + "ratio", + "status" /*a workaround for enums */, + "usage", + "size", + "utilization", + "histogram", + "time" + ); + static final int MAX_METRIC_NAME_LENGTH = 255; + + static final int MAX_ELEMENT_LENGTH = 30; + static final int MAX_NUMBER_OF_ELEMENTS = 10; + + private MetricNameValidator() {} + + /** + * Validates a metric name as per guidelines in Naming.md + * + * @param metricName metric name to be validated + * @throws IllegalArgumentException an exception indicating an incorrect metric name + */ + public static String validate(String metricName) { + Objects.requireNonNull(metricName); + validateMaxMetricNameLength(metricName); + + String[] elements = metricName.split("\\."); + hasESPrefix(elements, metricName); + hasAtLeast3Elements(elements, metricName); + hasNotBreachNumberOfElementsLimit(elements, metricName); + lastElementIsFromAllowList(elements, metricName); + perElementValidations(elements, metricName); + return metricName; + } + + private static void validateMaxMetricNameLength(String metricName) { + if (metricName.length() > MAX_METRIC_NAME_LENGTH) { + throw new IllegalArgumentException( + "Metric name length " + + metricName.length() + + "is longer than max metric name length:" + + MAX_METRIC_NAME_LENGTH + + " Name was: " + + metricName + ); + } + } + + private static void lastElementIsFromAllowList(String[] elements, String name) { + String lastElement = elements[elements.length - 1]; + if (ALLOWED_SUFFIXES.contains(lastElement) == false) { + throw new IllegalArgumentException( + "Metric name should end with one of [" + + ALLOWED_SUFFIXES.stream().collect(Collectors.joining(",")) + + "] " + + "Last element was: " + + lastElement + + ". " + + "Name was: " + + name + ); + } + } + + private static void hasNotBreachNumberOfElementsLimit(String[] elements, String name) { + if (elements.length > MAX_NUMBER_OF_ELEMENTS) { + throw new IllegalArgumentException( + "Metric name should have at most 10 elements. It had: " + elements.length + ". The name was: " + name + ); + } + } + + private static void hasAtLeast3Elements(String[] elements, String name) { + if (elements.length < 3) { + throw new IllegalArgumentException( + "Metric name consist of at least 3 elements. An es. prefix, group and a name. The name was: " + name + ); + } + } + + private static void hasESPrefix(String[] elements, String name) { + if (elements[0].equals("es") == false) { + throw new IllegalArgumentException( + "Metric name should start with \"es.\" prefix and use \".\" as a separator. Name was: " + name + ); + } + } + + private static void perElementValidations(String[] elements, String name) { + for (String element : elements) { + hasOnlyAllowedCharacters(element, name); + hasNotBreachLengthLimit(element, name); + } + } + + private static void hasNotBreachLengthLimit(String element, String name) { + if (element.length() > MAX_ELEMENT_LENGTH) { + throw new IllegalArgumentException( + "Metric name's element should not be longer than " + + MAX_ELEMENT_LENGTH + + " characters. Was: " + + element.length() + + ". Name was: " + + name + ); + } + } + + private static void hasOnlyAllowedCharacters(String element, String name) { + Matcher matcher = ALLOWED_CHARACTERS.matcher(element); + if (matcher.matches() == false) { + throw new IllegalArgumentException( + "Metric name should only use [a-z][a-z0-9_]* characters. " + + "Element does not match: \"" + + element + + "\". " + + "Name was: " + + name + ); + } + } +} diff --git a/modules/apm/src/test/java/org/elasticsearch/telemetry/apm/APMMeterRegistryTests.java b/modules/apm/src/test/java/org/elasticsearch/telemetry/apm/APMMeterRegistryTests.java index 778ca108dc5f..8144b8f9a33b 100644 --- a/modules/apm/src/test/java/org/elasticsearch/telemetry/apm/APMMeterRegistryTests.java +++ b/modules/apm/src/test/java/org/elasticsearch/telemetry/apm/APMMeterRegistryTests.java @@ -35,10 +35,8 @@ import java.util.Collections; import java.util.List; import java.util.function.Supplier; -import static org.hamcrest.Matchers.containsString; import static org.hamcrest.Matchers.equalTo; import static org.hamcrest.Matchers.hasSize; -import static org.hamcrest.Matchers.instanceOf; import static org.hamcrest.Matchers.sameInstance; public class APMMeterRegistryTests extends ESTestCase { @@ -84,8 +82,8 @@ public class APMMeterRegistryTests extends ESTestCase { public void testLookupByName() { var apmMeter = new APMMeterService(TELEMETRY_ENABLED, () -> testOtel, () -> noopOtel).getMeterRegistry(); - DoubleCounter registeredCounter = apmMeter.registerDoubleCounter("name", "desc", "unit"); - DoubleCounter lookedUpCounter = apmMeter.getDoubleCounter("name"); + DoubleCounter registeredCounter = apmMeter.registerDoubleCounter("es.test.name.total", "desc", "unit"); + DoubleCounter lookedUpCounter = apmMeter.getDoubleCounter("es.test.name.total"); assertThat(lookedUpCounter, sameInstance(registeredCounter)); } @@ -103,19 +101,6 @@ public class APMMeterRegistryTests extends ESTestCase { assertThat(meter, sameInstance(noopOtel)); } - public void testMaxNameLength() { - APMMeterService apmMeter = new APMMeterService(TELEMETRY_ENABLED, () -> testOtel, () -> noopOtel); - apmMeter.start(); - int max_length = 255; - var counter = apmMeter.getMeterRegistry().registerLongCounter("a".repeat(max_length), "desc", "count"); - assertThat(counter, instanceOf(LongCounter.class)); - IllegalArgumentException iae = expectThrows( - IllegalArgumentException.class, - () -> apmMeter.getMeterRegistry().registerLongCounter("a".repeat(max_length + 1), "desc", "count") - ); - assertThat(iae.getMessage(), containsString("exceeds maximum length [255]")); - } - public void testAllInstrumentsSwitchProviders() { TestAPMMeterService apmMeter = new TestAPMMeterService( Settings.builder().put(APMAgentSettings.TELEMETRY_METRICS_ENABLED_SETTING.getKey(), false).build(), @@ -125,18 +110,18 @@ public class APMMeterRegistryTests extends ESTestCase { APMMeterRegistry registry = apmMeter.getMeterRegistry(); Supplier doubleObserver = () -> new DoubleWithAttributes(1.5, Collections.emptyMap()); - DoubleCounter dc = registry.registerDoubleCounter("dc", "", ""); - DoubleUpDownCounter dudc = registry.registerDoubleUpDownCounter("dudc", "", ""); - DoubleHistogram dh = registry.registerDoubleHistogram("dh", "", ""); - DoubleAsyncCounter dac = registry.registerDoubleAsyncCounter("dac", "", "", doubleObserver); - DoubleGauge dg = registry.registerDoubleGauge("dg", "", "", doubleObserver); + DoubleCounter dc = registry.registerDoubleCounter("es.test.dc.total", "", ""); + DoubleUpDownCounter dudc = registry.registerDoubleUpDownCounter("es.test.dudc.current", "", ""); + DoubleHistogram dh = registry.registerDoubleHistogram("es.test.dh.histogram", "", ""); + DoubleAsyncCounter dac = registry.registerDoubleAsyncCounter("es.test.dac.total", "", "", doubleObserver); + DoubleGauge dg = registry.registerDoubleGauge("es.test.dg.current", "", "", doubleObserver); Supplier longObserver = () -> new LongWithAttributes(100, Collections.emptyMap()); - LongCounter lc = registry.registerLongCounter("lc", "", ""); - LongUpDownCounter ludc = registry.registerLongUpDownCounter("ludc", "", ""); - LongHistogram lh = registry.registerLongHistogram("lh", "", ""); - LongAsyncCounter lac = registry.registerLongAsyncCounter("lac", "", "", longObserver); - LongGauge lg = registry.registerLongGauge("lg", "", "", longObserver); + LongCounter lc = registry.registerLongCounter("es.test.lc.total", "", ""); + LongUpDownCounter ludc = registry.registerLongUpDownCounter("es.test.ludc.total", "", ""); + LongHistogram lh = registry.registerLongHistogram("es.test.lh.histogram", "", ""); + LongAsyncCounter lac = registry.registerLongAsyncCounter("es.test.lac.total", "", "", longObserver); + LongGauge lg = registry.registerLongGauge("es.test.lg.current", "", "", longObserver); apmMeter.setEnabled(true); diff --git a/modules/apm/src/test/java/org/elasticsearch/telemetry/apm/MeterRegistryConcurrencyTests.java b/modules/apm/src/test/java/org/elasticsearch/telemetry/apm/MeterRegistryConcurrencyTests.java index e8dafd996f5b..11951a9bf107 100644 --- a/modules/apm/src/test/java/org/elasticsearch/telemetry/apm/MeterRegistryConcurrencyTests.java +++ b/modules/apm/src/test/java/org/elasticsearch/telemetry/apm/MeterRegistryConcurrencyTests.java @@ -28,7 +28,7 @@ import static org.hamcrest.Matchers.equalTo; import static org.hamcrest.Matchers.sameInstance; public class MeterRegistryConcurrencyTests extends ESTestCase { - private final String name = "name"; + private final String name = "es.test.name.total"; private final String description = "desc"; private final String unit = "kg"; private final Meter noopMeter = OpenTelemetry.noop().getMeter("noop"); diff --git a/modules/apm/src/test/java/org/elasticsearch/telemetry/apm/internal/MetricNameValidatorTests.java b/modules/apm/src/test/java/org/elasticsearch/telemetry/apm/internal/MetricNameValidatorTests.java new file mode 100644 index 000000000000..64f78d0af494 --- /dev/null +++ b/modules/apm/src/test/java/org/elasticsearch/telemetry/apm/internal/MetricNameValidatorTests.java @@ -0,0 +1,102 @@ +/* + * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one + * or more contributor license agreements. Licensed under the Elastic License + * 2.0 and the Server Side Public License, v 1; you may not use this file except + * in compliance with, at your election, the Elastic License 2.0 or the Server + * Side Public License, v 1. + */ + +package org.elasticsearch.telemetry.apm.internal; + +import org.elasticsearch.test.ESTestCase; + +import static org.hamcrest.Matchers.equalTo; + +public class MetricNameValidatorTests extends ESTestCase { + public void testMetricNameNotNull() { + String metricName = "es.somemodule.somemetric.total"; + assertThat(MetricNameValidator.validate(metricName), equalTo(metricName)); + + expectThrows(NullPointerException.class, () -> MetricNameValidator.validate(null)); + } + + public void testMaxMetricNameLength() { + MetricNameValidator.validate(metricNameWithLength(255)); + + expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate(metricNameWithLength(256))); + } + + public void testESPrefixAndDotSeparator() { + MetricNameValidator.validate("es.somemodule.somemetric.total"); + + expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("somemodule.somemetric.total")); + // verify . is a separator + expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es_somemodule_somemetric_total")); + expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es_somemodule.somemetric.total")); + } + + public void testNameElementRegex() { + MetricNameValidator.validate("es.somemodulename0.somemetric.total"); + MetricNameValidator.validate("es.some_module_name0.somemetric.total"); + MetricNameValidator.validate("es.s.somemetric.total"); + + expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es.someModuleName0.somemetric.total")); + expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es.SomeModuleName.somemetric.total")); + expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es.0some_module_name0.somemetric.total")); + expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es.some_#_name0.somemetric.total")); + expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es.some-name0.somemetric.total")); + } + + public void testNameHas3Elements() { + MetricNameValidator.validate("es.group.total"); + MetricNameValidator.validate("es.group.subgroup.total"); + + expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es")); + expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es.")); + expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es.sth")); + } + + public void testNumberOfElementsLimit() { + MetricNameValidator.validate("es.a2.a3.a4.a5.a6.a7.a8.a9.total"); + + expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es.a2.a3.a4.a5.a6.a7.a8.a9.a10.total")); + } + + public void testElementLengthLimit() { + MetricNameValidator.validate("es." + "a".repeat(MetricNameValidator.MAX_ELEMENT_LENGTH) + ".total"); + + expectThrows( + IllegalArgumentException.class, + () -> MetricNameValidator.validate("es." + "a".repeat(MetricNameValidator.MAX_ELEMENT_LENGTH + 1) + ".total") + ); + } + + public void testLastElementAllowList() { + for (String suffix : MetricNameValidator.ALLOWED_SUFFIXES) { + MetricNameValidator.validate("es.somemodule.somemetric." + suffix); + } + expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es.somemodule.somemetric.some_other_suffix")); + } + + public static String metricNameWithLength(int length) { + int prefixAndSuffix = "es.".length() + ".utilization".length(); + assert length > prefixAndSuffix : "length too short"; + + var remainingChars = length - prefixAndSuffix; + StringBuilder metricName = new StringBuilder("es."); + var i = 0; + while (i < remainingChars) { + metricName.append("a"); + i++; + for (int j = 0; j < MetricNameValidator.MAX_ELEMENT_LENGTH - 1 && i < remainingChars; j++) { + metricName.append("x"); + i++; + } + metricName.append("."); + i++; + + } + metricName.append("utilization"); + return metricName.toString(); + } +} diff --git a/modules/apm/src/test/java/org/elasticsearch/telemetry/apm/internal/metrics/AsyncCountersAdapterTests.java b/modules/apm/src/test/java/org/elasticsearch/telemetry/apm/internal/metrics/AsyncCountersAdapterTests.java index 3e23b741e01e..24b40063cd63 100644 --- a/modules/apm/src/test/java/org/elasticsearch/telemetry/apm/internal/metrics/AsyncCountersAdapterTests.java +++ b/modules/apm/src/test/java/org/elasticsearch/telemetry/apm/internal/metrics/AsyncCountersAdapterTests.java @@ -38,7 +38,7 @@ public class AsyncCountersAdapterTests extends ESTestCase { // testing that a value reported is then used in a callback public void testLongAsyncCounter() throws Exception { AtomicReference attrs = new AtomicReference<>(); - LongAsyncCounter longAsyncCounter = registry.registerLongAsyncCounter("name", "desc", "unit", attrs::get); + LongAsyncCounter longAsyncCounter = registry.registerLongAsyncCounter("es.test.name.total", "desc", "unit", attrs::get); attrs.set(new LongWithAttributes(1L, Map.of("k", 1L))); @@ -70,7 +70,7 @@ public class AsyncCountersAdapterTests extends ESTestCase { public void testDoubleAsyncAdapter() throws Exception { AtomicReference attrs = new AtomicReference<>(); - DoubleAsyncCounter doubleAsyncCounter = registry.registerDoubleAsyncCounter("name", "desc", "unit", attrs::get); + DoubleAsyncCounter doubleAsyncCounter = registry.registerDoubleAsyncCounter("es.test.name.total", "desc", "unit", attrs::get); attrs.set(new DoubleWithAttributes(1.0, Map.of("k", 1.0))); @@ -102,7 +102,7 @@ public class AsyncCountersAdapterTests extends ESTestCase { public void testNullGaugeRecord() throws Exception { DoubleAsyncCounter dcounter = registry.registerDoubleAsyncCounter( - "name", + "es.test.name.total", "desc", "unit", new AtomicReference()::get @@ -112,7 +112,7 @@ public class AsyncCountersAdapterTests extends ESTestCase { assertThat(metrics, hasSize(0)); LongAsyncCounter lcounter = registry.registerLongAsyncCounter( - "name", + "es.test.name.total", "desc", "unit", new AtomicReference()::get diff --git a/modules/apm/src/test/java/org/elasticsearch/telemetry/apm/internal/metrics/GaugeAdapterTests.java b/modules/apm/src/test/java/org/elasticsearch/telemetry/apm/internal/metrics/GaugeAdapterTests.java index 10f2d58768d4..d5e605df1d09 100644 --- a/modules/apm/src/test/java/org/elasticsearch/telemetry/apm/internal/metrics/GaugeAdapterTests.java +++ b/modules/apm/src/test/java/org/elasticsearch/telemetry/apm/internal/metrics/GaugeAdapterTests.java @@ -38,7 +38,7 @@ public class GaugeAdapterTests extends ESTestCase { // testing that a value reported is then used in a callback public void testLongGaugeRecord() throws Exception { AtomicReference attrs = new AtomicReference<>(); - LongGauge gauge = registry.registerLongGauge("name", "desc", "unit", attrs::get); + LongGauge gauge = registry.registerLongGauge("es.test.name.total", "desc", "unit", attrs::get); attrs.set(new LongWithAttributes(1L, Map.of("k", 1L))); @@ -71,7 +71,7 @@ public class GaugeAdapterTests extends ESTestCase { // testing that a value reported is then used in a callback public void testDoubleGaugeRecord() throws Exception { AtomicReference attrs = new AtomicReference<>(); - DoubleGauge gauge = registry.registerDoubleGauge("name", "desc", "unit", attrs::get); + DoubleGauge gauge = registry.registerDoubleGauge("es.test.name.total", "desc", "unit", attrs::get); attrs.set(new DoubleWithAttributes(1.0d, Map.of("k", 1L))); @@ -102,12 +102,17 @@ public class GaugeAdapterTests extends ESTestCase { } public void testNullGaugeRecord() throws Exception { - DoubleGauge dgauge = registry.registerDoubleGauge("name", "desc", "unit", new AtomicReference()::get); + DoubleGauge dgauge = registry.registerDoubleGauge( + "es.test.name.total", + "desc", + "unit", + new AtomicReference()::get + ); otelMeter.collectMetrics(); List metrics = otelMeter.getRecorder().getMeasurements(dgauge); assertThat(metrics, hasSize(0)); - LongGauge lgauge = registry.registerLongGauge("name", "desc", "unit", new AtomicReference()::get); + LongGauge lgauge = registry.registerLongGauge("es.test.name.total", "desc", "unit", new AtomicReference()::get); otelMeter.collectMetrics(); metrics = otelMeter.getRecorder().getMeasurements(lgauge); assertThat(metrics, hasSize(0)); diff --git a/modules/repository-s3/src/internalClusterTest/java/org/elasticsearch/repositories/s3/S3BlobStoreRepositoryMetricsTests.java b/modules/repository-s3/src/internalClusterTest/java/org/elasticsearch/repositories/s3/S3BlobStoreRepositoryMetricsTests.java index 59f65032272d..f8503bca3ec6 100644 --- a/modules/repository-s3/src/internalClusterTest/java/org/elasticsearch/repositories/s3/S3BlobStoreRepositoryMetricsTests.java +++ b/modules/repository-s3/src/internalClusterTest/java/org/elasticsearch/repositories/s3/S3BlobStoreRepositoryMetricsTests.java @@ -38,13 +38,13 @@ import java.util.Queue; import java.util.concurrent.LinkedBlockingQueue; import static org.elasticsearch.repositories.RepositoriesMetrics.HTTP_REQUEST_TIME_IN_MICROS_HISTOGRAM; -import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_EXCEPTIONS_COUNT; import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_EXCEPTIONS_HISTOGRAM; -import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_OPERATIONS_COUNT; -import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_REQUESTS_COUNT; -import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_THROTTLES_COUNT; +import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_EXCEPTIONS_TOTAL; +import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_OPERATIONS_TOTAL; +import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_REQUESTS_TOTAL; import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_THROTTLES_HISTOGRAM; -import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_UNSUCCESSFUL_OPERATIONS_COUNT; +import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_THROTTLES_TOTAL; +import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_UNSUCCESSFUL_OPERATIONS_TOTAL; import static org.elasticsearch.rest.RestStatus.INTERNAL_SERVER_ERROR; import static org.elasticsearch.rest.RestStatus.NOT_FOUND; import static org.elasticsearch.rest.RestStatus.TOO_MANY_REQUESTS; @@ -104,11 +104,11 @@ public class S3BlobStoreRepositoryMetricsTests extends S3BlobStoreRepositoryTest final long batch = i + 1; addErrorStatus(INTERNAL_SERVER_ERROR, TOO_MANY_REQUESTS, TOO_MANY_REQUESTS); blobContainer.writeBlob(purpose, blobName, new BytesArray("blob"), false); - assertThat(getLongCounterValue(plugin, METRIC_REQUESTS_COUNT, Operation.PUT_OBJECT), equalTo(4L * batch)); - assertThat(getLongCounterValue(plugin, METRIC_OPERATIONS_COUNT, Operation.PUT_OBJECT), equalTo(batch)); - assertThat(getLongCounterValue(plugin, METRIC_UNSUCCESSFUL_OPERATIONS_COUNT, Operation.PUT_OBJECT), equalTo(0L)); - assertThat(getLongCounterValue(plugin, METRIC_EXCEPTIONS_COUNT, Operation.PUT_OBJECT), equalTo(batch)); - assertThat(getLongCounterValue(plugin, METRIC_THROTTLES_COUNT, Operation.PUT_OBJECT), equalTo(2L * batch)); + assertThat(getLongCounterValue(plugin, METRIC_REQUESTS_TOTAL, Operation.PUT_OBJECT), equalTo(4L * batch)); + assertThat(getLongCounterValue(plugin, METRIC_OPERATIONS_TOTAL, Operation.PUT_OBJECT), equalTo(batch)); + assertThat(getLongCounterValue(plugin, METRIC_UNSUCCESSFUL_OPERATIONS_TOTAL, Operation.PUT_OBJECT), equalTo(0L)); + assertThat(getLongCounterValue(plugin, METRIC_EXCEPTIONS_TOTAL, Operation.PUT_OBJECT), equalTo(batch)); + assertThat(getLongCounterValue(plugin, METRIC_THROTTLES_TOTAL, Operation.PUT_OBJECT), equalTo(2L * batch)); assertThat(getLongHistogramValue(plugin, METRIC_EXCEPTIONS_HISTOGRAM, Operation.PUT_OBJECT), equalTo(batch)); assertThat(getLongHistogramValue(plugin, METRIC_THROTTLES_HISTOGRAM, Operation.PUT_OBJECT), equalTo(2L * batch)); assertThat(getNumberOfMeasurements(plugin, HTTP_REQUEST_TIME_IN_MICROS_HISTOGRAM, Operation.PUT_OBJECT), equalTo(batch)); @@ -124,11 +124,11 @@ public class S3BlobStoreRepositoryMetricsTests extends S3BlobStoreRepositoryTest } catch (Exception e) { // intentional failure } - assertThat(getLongCounterValue(plugin, METRIC_REQUESTS_COUNT, Operation.GET_OBJECT), equalTo(2L * batch)); - assertThat(getLongCounterValue(plugin, METRIC_OPERATIONS_COUNT, Operation.GET_OBJECT), equalTo(batch)); - assertThat(getLongCounterValue(plugin, METRIC_UNSUCCESSFUL_OPERATIONS_COUNT, Operation.GET_OBJECT), equalTo(batch)); - assertThat(getLongCounterValue(plugin, METRIC_EXCEPTIONS_COUNT, Operation.GET_OBJECT), equalTo(batch)); - assertThat(getLongCounterValue(plugin, METRIC_THROTTLES_COUNT, Operation.GET_OBJECT), equalTo(batch)); + assertThat(getLongCounterValue(plugin, METRIC_REQUESTS_TOTAL, Operation.GET_OBJECT), equalTo(2L * batch)); + assertThat(getLongCounterValue(plugin, METRIC_OPERATIONS_TOTAL, Operation.GET_OBJECT), equalTo(batch)); + assertThat(getLongCounterValue(plugin, METRIC_UNSUCCESSFUL_OPERATIONS_TOTAL, Operation.GET_OBJECT), equalTo(batch)); + assertThat(getLongCounterValue(plugin, METRIC_EXCEPTIONS_TOTAL, Operation.GET_OBJECT), equalTo(batch)); + assertThat(getLongCounterValue(plugin, METRIC_THROTTLES_TOTAL, Operation.GET_OBJECT), equalTo(batch)); assertThat(getLongHistogramValue(plugin, METRIC_EXCEPTIONS_HISTOGRAM, Operation.GET_OBJECT), equalTo(batch)); assertThat(getLongHistogramValue(plugin, METRIC_THROTTLES_HISTOGRAM, Operation.GET_OBJECT), equalTo(batch)); assertThat(getNumberOfMeasurements(plugin, HTTP_REQUEST_TIME_IN_MICROS_HISTOGRAM, Operation.GET_OBJECT), equalTo(batch)); @@ -144,11 +144,11 @@ public class S3BlobStoreRepositoryMetricsTests extends S3BlobStoreRepositoryTest } catch (Exception e) { // intentional failure } - assertThat(getLongCounterValue(plugin, METRIC_REQUESTS_COUNT, Operation.LIST_OBJECTS), equalTo(5L * batch)); - assertThat(getLongCounterValue(plugin, METRIC_OPERATIONS_COUNT, Operation.LIST_OBJECTS), equalTo(batch)); - assertThat(getLongCounterValue(plugin, METRIC_UNSUCCESSFUL_OPERATIONS_COUNT, Operation.LIST_OBJECTS), equalTo(batch)); - assertThat(getLongCounterValue(plugin, METRIC_EXCEPTIONS_COUNT, Operation.LIST_OBJECTS), equalTo(batch)); - assertThat(getLongCounterValue(plugin, METRIC_THROTTLES_COUNT, Operation.LIST_OBJECTS), equalTo(5L * batch)); + assertThat(getLongCounterValue(plugin, METRIC_REQUESTS_TOTAL, Operation.LIST_OBJECTS), equalTo(5L * batch)); + assertThat(getLongCounterValue(plugin, METRIC_OPERATIONS_TOTAL, Operation.LIST_OBJECTS), equalTo(batch)); + assertThat(getLongCounterValue(plugin, METRIC_UNSUCCESSFUL_OPERATIONS_TOTAL, Operation.LIST_OBJECTS), equalTo(batch)); + assertThat(getLongCounterValue(plugin, METRIC_EXCEPTIONS_TOTAL, Operation.LIST_OBJECTS), equalTo(batch)); + assertThat(getLongCounterValue(plugin, METRIC_THROTTLES_TOTAL, Operation.LIST_OBJECTS), equalTo(5L * batch)); assertThat(getLongHistogramValue(plugin, METRIC_EXCEPTIONS_HISTOGRAM, Operation.LIST_OBJECTS), equalTo(batch)); assertThat(getLongHistogramValue(plugin, METRIC_THROTTLES_HISTOGRAM, Operation.LIST_OBJECTS), equalTo(5L * batch)); assertThat(getNumberOfMeasurements(plugin, HTTP_REQUEST_TIME_IN_MICROS_HISTOGRAM, Operation.LIST_OBJECTS), equalTo(batch)); @@ -156,11 +156,11 @@ public class S3BlobStoreRepositoryMetricsTests extends S3BlobStoreRepositoryTest // Delete to clean up blobContainer.deleteBlobsIgnoringIfNotExists(purpose, Iterators.single(blobName)); - assertThat(getLongCounterValue(plugin, METRIC_REQUESTS_COUNT, Operation.DELETE_OBJECTS), equalTo(1L)); - assertThat(getLongCounterValue(plugin, METRIC_OPERATIONS_COUNT, Operation.DELETE_OBJECTS), equalTo(1L)); - assertThat(getLongCounterValue(plugin, METRIC_UNSUCCESSFUL_OPERATIONS_COUNT, Operation.DELETE_OBJECTS), equalTo(0L)); - assertThat(getLongCounterValue(plugin, METRIC_EXCEPTIONS_COUNT, Operation.DELETE_OBJECTS), equalTo(0L)); - assertThat(getLongCounterValue(plugin, METRIC_THROTTLES_COUNT, Operation.DELETE_OBJECTS), equalTo(0L)); + assertThat(getLongCounterValue(plugin, METRIC_REQUESTS_TOTAL, Operation.DELETE_OBJECTS), equalTo(1L)); + assertThat(getLongCounterValue(plugin, METRIC_OPERATIONS_TOTAL, Operation.DELETE_OBJECTS), equalTo(1L)); + assertThat(getLongCounterValue(plugin, METRIC_UNSUCCESSFUL_OPERATIONS_TOTAL, Operation.DELETE_OBJECTS), equalTo(0L)); + assertThat(getLongCounterValue(plugin, METRIC_EXCEPTIONS_TOTAL, Operation.DELETE_OBJECTS), equalTo(0L)); + assertThat(getLongCounterValue(plugin, METRIC_THROTTLES_TOTAL, Operation.DELETE_OBJECTS), equalTo(0L)); assertThat(getLongHistogramValue(plugin, METRIC_EXCEPTIONS_HISTOGRAM, Operation.DELETE_OBJECTS), equalTo(0L)); assertThat(getLongHistogramValue(plugin, METRIC_THROTTLES_HISTOGRAM, Operation.DELETE_OBJECTS), equalTo(0L)); assertThat(getNumberOfMeasurements(plugin, HTTP_REQUEST_TIME_IN_MICROS_HISTOGRAM, Operation.DELETE_OBJECTS), equalTo(1L)); diff --git a/modules/repository-s3/src/internalClusterTest/java/org/elasticsearch/repositories/s3/S3BlobStoreRepositoryTests.java b/modules/repository-s3/src/internalClusterTest/java/org/elasticsearch/repositories/s3/S3BlobStoreRepositoryTests.java index 29342a7f5ea9..e70151cbdf8e 100644 --- a/modules/repository-s3/src/internalClusterTest/java/org/elasticsearch/repositories/s3/S3BlobStoreRepositoryTests.java +++ b/modules/repository-s3/src/internalClusterTest/java/org/elasticsearch/repositories/s3/S3BlobStoreRepositoryTests.java @@ -75,7 +75,7 @@ import java.util.concurrent.atomic.AtomicBoolean; import java.util.stream.Collectors; import java.util.stream.StreamSupport; -import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_REQUESTS_COUNT; +import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_REQUESTS_TOTAL; import static org.elasticsearch.repositories.blobstore.BlobStoreTestUtil.randomNonDataPurpose; import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertAcked; import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertHitCount; @@ -267,7 +267,7 @@ public class S3BlobStoreRepositoryTests extends ESMockAPIBasedRepositoryIntegTes .filterPlugins(TestTelemetryPlugin.class) .toList(); assertThat(plugins, hasSize(1)); - final List metrics = Measurement.combine(plugins.get(0).getLongCounterMeasurement(METRIC_REQUESTS_COUNT)); + final List metrics = Measurement.combine(plugins.get(0).getLongCounterMeasurement(METRIC_REQUESTS_TOTAL)); assertThat( statsCollectors.keySet().stream().map(S3BlobStore.StatsKey::operation).collect(Collectors.toSet()), diff --git a/server/src/main/java/org/elasticsearch/cluster/routing/allocation/allocator/DesiredBalanceReconciler.java b/server/src/main/java/org/elasticsearch/cluster/routing/allocation/allocator/DesiredBalanceReconciler.java index dc3cbfa8b5ae..95b0d23b564a 100644 --- a/server/src/main/java/org/elasticsearch/cluster/routing/allocation/allocator/DesiredBalanceReconciler.java +++ b/server/src/main/java/org/elasticsearch/cluster/routing/allocation/allocator/DesiredBalanceReconciler.java @@ -98,24 +98,24 @@ public class DesiredBalanceReconciler { unassignedShards = LongGaugeMetric.create( meterRegistry, - "es.allocator.desired_balance.shards.unassigned", - "Unassigned shards count", + "es.allocator.desired_balance.shards.unassigned.current", + "Current number of unassigned shards", "{shard}" ); totalAllocations = LongGaugeMetric.create( meterRegistry, - "es.allocator.desired_balance.shards.count", - "Total shards count", + "es.allocator.desired_balance.shards.current", + "Total number of shards", "{shard}" ); undesiredAllocations = LongGaugeMetric.create( meterRegistry, - "es.allocator.desired_balance.allocations.undesired", - "Count of shards allocated on undesired nodes", + "es.allocator.desired_balance.allocations.undesired.current", + "Total number of shards allocated on undesired nodes", "{shard}" ); undesiredAllocationsRatio = meterRegistry.registerDoubleGauge( - "es.allocator.desired_balance.allocations.undesired_ratio", + "es.allocator.desired_balance.allocations.undesired.ratio", "Ratio of undesired allocations to shard count", "1", () -> { diff --git a/server/src/main/java/org/elasticsearch/health/HealthPeriodicLogger.java b/server/src/main/java/org/elasticsearch/health/HealthPeriodicLogger.java index 878930c2962d..288837fb3c80 100644 --- a/server/src/main/java/org/elasticsearch/health/HealthPeriodicLogger.java +++ b/server/src/main/java/org/elasticsearch/health/HealthPeriodicLogger.java @@ -204,7 +204,10 @@ public class HealthPeriodicLogger implements ClusterStateListener, Closeable, Sc this.logWriter = logWriter == null ? logger::info : logWriter; // create metric for overall level metrics - this.redMetrics.put("overall", LongGaugeMetric.create(this.meterRegistry, "es.health.overall.red", "Overall: Red", "{cluster}")); + this.redMetrics.put( + "overall", + LongGaugeMetric.create(this.meterRegistry, "es.health.overall.red.status", "Overall: Red", "{cluster}") + ); } private void registerListeners() { diff --git a/server/src/main/java/org/elasticsearch/monitor/metrics/NodeMetrics.java b/server/src/main/java/org/elasticsearch/monitor/metrics/NodeMetrics.java index a7d93ec7e7d8..476ad516aab8 100644 --- a/server/src/main/java/org/elasticsearch/monitor/metrics/NodeMetrics.java +++ b/server/src/main/java/org/elasticsearch/monitor/metrics/NodeMetrics.java @@ -116,7 +116,7 @@ public class NodeMetrics extends AbstractLifecycleComponent { metrics.add( registry.registerLongGauge( - "es.translog.operations.count", + "es.translog.operations.total", "Number of transaction log operations.", "operation", () -> new LongWithAttributes(stats.getOrRefresh().getIndices().getTranslog().estimatedNumberOfOperations()) @@ -134,7 +134,7 @@ public class NodeMetrics extends AbstractLifecycleComponent { metrics.add( registry.registerLongGauge( - "es.translog.uncommitted_operations.count", + "es.translog.uncommitted_operations.total", "Number of uncommitted transaction log operations.", "operations", () -> new LongWithAttributes(stats.getOrRefresh().getIndices().getTranslog().getUncommittedOperations()) @@ -224,7 +224,7 @@ public class NodeMetrics extends AbstractLifecycleComponent { metrics.add( registry.registerLongGauge( - "es.indexing.docs.count", + "es.indexing.docs.total", "Current number of indexing documents", "documents", () -> new LongWithAttributes(stats.getOrRefresh().getIndices().getIndexing().getTotal().getIndexCurrent()) @@ -251,7 +251,7 @@ public class NodeMetrics extends AbstractLifecycleComponent { metrics.add( registry.registerLongGauge( - "es.indices.deletion.docs.count", + "es.indices.deletion.docs.total", "Current number of deleting documents", "documents", () -> new LongWithAttributes(stats.getOrRefresh().getIndices().getIndexing().getTotal().getDeleteCurrent()) @@ -323,7 +323,7 @@ public class NodeMetrics extends AbstractLifecycleComponent { metrics.add( registry.registerLongGauge( - "es.indexing.coordinating_operations.count", + "es.indexing.coordinating_operations.total", "Current number of coordinating operations", "operations", () -> new LongWithAttributes(stats.getOrRefresh().getIndexingPressureStats().getCurrentCoordinatingOps()) @@ -368,7 +368,7 @@ public class NodeMetrics extends AbstractLifecycleComponent { metrics.add( registry.registerLongGauge( - "es.indexing.primary_operations.count", + "es.indexing.primary_operations.total", "Current number of primary operations", "operations", () -> new LongWithAttributes(stats.getOrRefresh().getIndexingPressureStats().getCurrentPrimaryOps()) diff --git a/server/src/main/java/org/elasticsearch/repositories/RepositoriesMetrics.java b/server/src/main/java/org/elasticsearch/repositories/RepositoriesMetrics.java index 8442cf8c4a34..b4d79d89ec4c 100644 --- a/server/src/main/java/org/elasticsearch/repositories/RepositoriesMetrics.java +++ b/server/src/main/java/org/elasticsearch/repositories/RepositoriesMetrics.java @@ -25,22 +25,22 @@ public record RepositoriesMetrics( public static RepositoriesMetrics NOOP = new RepositoriesMetrics(MeterRegistry.NOOP); - public static final String METRIC_REQUESTS_COUNT = "es.repositories.requests.count"; - public static final String METRIC_EXCEPTIONS_COUNT = "es.repositories.exceptions.count"; - public static final String METRIC_THROTTLES_COUNT = "es.repositories.throttles.count"; - public static final String METRIC_OPERATIONS_COUNT = "es.repositories.operations.count"; - public static final String METRIC_UNSUCCESSFUL_OPERATIONS_COUNT = "es.repositories.operations.unsuccessful.count"; + public static final String METRIC_REQUESTS_TOTAL = "es.repositories.requests.total"; + public static final String METRIC_EXCEPTIONS_TOTAL = "es.repositories.exceptions.total"; + public static final String METRIC_THROTTLES_TOTAL = "es.repositories.throttles.total"; + public static final String METRIC_OPERATIONS_TOTAL = "es.repositories.operations.total"; + public static final String METRIC_UNSUCCESSFUL_OPERATIONS_TOTAL = "es.repositories.operations.unsuccessful.total"; public static final String METRIC_EXCEPTIONS_HISTOGRAM = "es.repositories.exceptions.histogram"; public static final String METRIC_THROTTLES_HISTOGRAM = "es.repositories.throttles.histogram"; public static final String HTTP_REQUEST_TIME_IN_MICROS_HISTOGRAM = "es.repositories.requests.http_request_time.histogram"; public RepositoriesMetrics(MeterRegistry meterRegistry) { this( - meterRegistry.registerLongCounter(METRIC_REQUESTS_COUNT, "repository request counter", "unit"), - meterRegistry.registerLongCounter(METRIC_EXCEPTIONS_COUNT, "repository request exception counter", "unit"), - meterRegistry.registerLongCounter(METRIC_THROTTLES_COUNT, "repository request throttle counter", "unit"), - meterRegistry.registerLongCounter(METRIC_OPERATIONS_COUNT, "repository operation counter", "unit"), - meterRegistry.registerLongCounter(METRIC_UNSUCCESSFUL_OPERATIONS_COUNT, "repository unsuccessful operation counter", "unit"), + meterRegistry.registerLongCounter(METRIC_REQUESTS_TOTAL, "repository request counter", "unit"), + meterRegistry.registerLongCounter(METRIC_EXCEPTIONS_TOTAL, "repository request exception counter", "unit"), + meterRegistry.registerLongCounter(METRIC_THROTTLES_TOTAL, "repository request throttle counter", "unit"), + meterRegistry.registerLongCounter(METRIC_OPERATIONS_TOTAL, "repository operation counter", "unit"), + meterRegistry.registerLongCounter(METRIC_UNSUCCESSFUL_OPERATIONS_TOTAL, "repository unsuccessful operation counter", "unit"), meterRegistry.registerLongHistogram(METRIC_EXCEPTIONS_HISTOGRAM, "repository request exception histogram", "unit"), meterRegistry.registerLongHistogram(METRIC_THROTTLES_HISTOGRAM, "repository request throttle histogram", "unit"), meterRegistry.registerLongHistogram( diff --git a/test/external-modules/apm-integration/src/javaRestTest/java/org/elasticsearch/test/apmintegration/MetricsApmIT.java b/test/external-modules/apm-integration/src/javaRestTest/java/org/elasticsearch/test/apmintegration/MetricsApmIT.java index 93d08fbccd37..ddcd667b9cbe 100644 --- a/test/external-modules/apm-integration/src/javaRestTest/java/org/elasticsearch/test/apmintegration/MetricsApmIT.java +++ b/test/external-modules/apm-integration/src/javaRestTest/java/org/elasticsearch/test/apmintegration/MetricsApmIT.java @@ -60,19 +60,19 @@ public class MetricsApmIT extends ESRestTestCase { public void testApmIntegration() throws Exception { Map>> sampleAssertions = new HashMap<>( Map.ofEntries( - assertion(TestMeterUsages.VERY_LONG_NAME, m -> (Double) m.get("value"), closeTo(1.0, 0.001)), - assertion("testLongCounter", m -> (Double) m.get("value"), closeTo(1.0, 0.001)), - assertion("testAsyncDoubleCounter", m -> (Double) m.get("value"), closeTo(1.0, 0.001)), - assertion("testAsyncLongCounter", m -> (Integer) m.get("value"), equalTo(1)), - assertion("testDoubleGauge", m -> (Double) m.get("value"), closeTo(1.0, 0.001)), - assertion("testLongGauge", m -> (Integer) m.get("value"), equalTo(1)), + assertion("es.test.long_counter.total", m -> (Double) m.get("value"), closeTo(1.0, 0.001)), + assertion("es.test.double_counter.total", m -> (Double) m.get("value"), closeTo(1.0, 0.001)), + assertion("es.test.async_double_counter.total", m -> (Double) m.get("value"), closeTo(1.0, 0.001)), + assertion("es.test.async_long_counter.total", m -> (Integer) m.get("value"), equalTo(1)), + assertion("es.test.double_gauge.current", m -> (Double) m.get("value"), closeTo(1.0, 0.001)), + assertion("es.test.long_gauge.current", m -> (Integer) m.get("value"), equalTo(1)), assertion( - "testDoubleHistogram", + "es.test.double_histogram.histogram", m -> ((Collection) m.get("counts")).stream().mapToInt(Integer::intValue).sum(), equalTo(2) ), assertion( - "testLongHistogram", + "es.test.long_histogram.histogram", m -> ((Collection) m.get("counts")).stream().mapToInt(Integer::intValue).sum(), equalTo(2) ) diff --git a/test/external-modules/apm-integration/src/main/java/org/elasticsearch/test/apmintegration/TestMeterUsages.java b/test/external-modules/apm-integration/src/main/java/org/elasticsearch/test/apmintegration/TestMeterUsages.java index 8a71738a0b42..9c23ce371e04 100644 --- a/test/external-modules/apm-integration/src/main/java/org/elasticsearch/test/apmintegration/TestMeterUsages.java +++ b/test/external-modules/apm-integration/src/main/java/org/elasticsearch/test/apmintegration/TestMeterUsages.java @@ -26,18 +26,17 @@ public class TestMeterUsages { private final LongHistogram longHistogram; private final AtomicReference doubleWithAttributes = new AtomicReference<>(); private final AtomicReference longWithAttributes = new AtomicReference<>(); - public static String VERY_LONG_NAME = "a1234567890123456789012345678901234567890123456789012345678901234567890"; public TestMeterUsages(MeterRegistry meterRegistry) { - this.doubleCounter = meterRegistry.registerDoubleCounter(VERY_LONG_NAME, "test", "unit"); - this.longCounter = meterRegistry.registerDoubleCounter("testLongCounter", "test", "unit"); - this.doubleHistogram = meterRegistry.registerDoubleHistogram("testDoubleHistogram", "test", "unit"); - this.longHistogram = meterRegistry.registerLongHistogram("testLongHistogram", "test", "unit"); - meterRegistry.registerDoubleGauge("testDoubleGauge", "test", "unit", doubleWithAttributes::get); - meterRegistry.registerLongGauge("testLongGauge", "test", "unit", longWithAttributes::get); + this.doubleCounter = meterRegistry.registerDoubleCounter("es.test.long_counter.total", "test", "unit"); + this.longCounter = meterRegistry.registerDoubleCounter("es.test.double_counter.total", "test", "unit"); + this.doubleHistogram = meterRegistry.registerDoubleHistogram("es.test.double_histogram.histogram", "test", "unit"); + this.longHistogram = meterRegistry.registerLongHistogram("es.test.long_histogram.histogram", "test", "unit"); + meterRegistry.registerDoubleGauge("es.test.double_gauge.current", "test", "unit", doubleWithAttributes::get); + meterRegistry.registerLongGauge("es.test.long_gauge.current", "test", "unit", longWithAttributes::get); - meterRegistry.registerLongAsyncCounter("testAsyncLongCounter", "test", "unit", longWithAttributes::get); - meterRegistry.registerDoubleAsyncCounter("testAsyncDoubleCounter", "test", "unit", doubleWithAttributes::get); + meterRegistry.registerLongAsyncCounter("es.test.async_long_counter.total", "test", "unit", longWithAttributes::get); + meterRegistry.registerDoubleAsyncCounter("es.test.async_double_counter.total", "test", "unit", doubleWithAttributes::get); } public void testUponRequest() { diff --git a/x b/x new file mode 100644 index 000000000000..444b7eef651c --- /dev/null +++ b/x @@ -0,0 +1,320 @@ +➜ elasticsearch git:(metric_name_validation) ./gradlew run -Dtests.es.logger.org.elasticsearch.telemetry.apm=debug + +> Configure project :x-pack:plugin:searchable-snapshots:qa:hdfs +hdfsFixture unsupported, please set HADOOP_HOME and put HADOOP_HOME\bin in PATH +======================================= +Elasticsearch Build Hamster says Hello! + Gradle Version : 8.5 + OS Info : Mac OS X 14.1.2 (aarch64) + JDK Version : 17.0.2+8-LTS-86 (Oracle) + JAVA_HOME : /Library/Java/JavaVirtualMachines/jdk-17.0.2.jdk/Contents/Home + Random Testing Seed : B705DEF03AA4BF36 + In FIPS 140 mode : false +======================================= + +> Task :run +[2023-12-18T13:31:10.882688Z] [BUILD] Copying additional config files from distro [/Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.13.0-SNAPSHOT/config/jvm.options.d, /Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.13.0-SNAPSHOT/config/users_roles, /Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.13.0-SNAPSHOT/config/role_mapping.yml, /Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.13.0-SNAPSHOT/config/users, /Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.13.0-SNAPSHOT/config/elasticsearch.yml, /Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.13.0-SNAPSHOT/config/roles.yml, /Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.13.0-SNAPSHOT/config/log4j2.properties, /Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.13.0-SNAPSHOT/config/elasticsearch-plugins.example.yml, /Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.13.0-SNAPSHOT/config/jvm.options] +[2023-12-18T13:31:10.885813Z] [BUILD] Creating elasticsearch keystore with password set to [] +[2023-12-18T13:31:11.630988Z] [BUILD] Adding 1 keystore settings and 0 keystore files +[2023-12-18T13:31:12.293546Z] [BUILD] Installing 0 modules +[2023-12-18T13:31:12.293769Z] [BUILD] Setting up 1 users +[2023-12-18T13:31:13.096132Z] [BUILD] Setting up roles.yml +[2023-12-18T13:31:13.096494Z] [BUILD] Starting Elasticsearch process +CompileCommand: exclude org/apache/lucene/util/MSBRadixSorter.computeCommonPrefixLengthAndBuildHistogram bool exclude = true +CompileCommand: exclude org/apache/lucene/util/RadixSelector.computeCommonPrefixLengthAndBuildHistogram bool exclude = true +Dec 18, 2023 2:31:14 PM sun.util.locale.provider.LocaleProviderAdapter +WARNING: COMPAT locale provider will be removed in a future release +[2023-12-18T14:31:15,025][INFO ][o.a.l.i.v.PanamaVectorizationProvider] [runTask-0] Java vector incubator API enabled; uses preferredBitSize=128 +[2023-12-18T14:31:15,437][INFO ][o.e.n.Node ] [runTask-0] version[8.13.0-SNAPSHOT], pid[43801], build[tar/25d9bbbb5327023f7f1896fb3044fb4c9d231342/2023-12-18T13:27:39.871866Z], OS[Mac OS X/14.1.2/aarch64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/21/21+35-2513] +[2023-12-18T14:31:15,438][INFO ][o.e.n.Node ] [runTask-0] JVM home [/Users/przemyslawgomulka/.gradle/jdks/oracle_corporation-21-aarch64-os_x/jdk-21.jdk/Contents/Home], using bundled JDK [false] +[2023-12-18T14:31:15,438][INFO ][o.e.n.Node ] [runTask-0] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=org.elasticsearch.preallocate, -Des.distribution.type=tar, -XX:+UseG1GC, -Djava.io.tmpdir=/Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/build/testclusters/runTask-0/tmp, --add-modules=jdk.incubator.vector, -XX:CompileCommand=exclude,org.apache.lucene.util.MSBRadixSorter::computeCommonPrefixLengthAndBuildHistogram, -XX:CompileCommand=exclude,org.apache.lucene.util.RadixSelector::computeCommonPrefixLengthAndBuildHistogram, -XX:+HeapDumpOnOutOfMemoryError, -XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=/Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/build/testclusters/runTask-0/logs, -XX:ErrorFile=/Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/build/testclusters/runTask-0/logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=/Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/build/testclusters/runTask-0/logs/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m, -Xms512m, -Xmx512m, -ea, -esa, -Dingest.geoip.downloader.enabled.default=true, -Dio.netty.leakDetection.level=paranoid, -XX:MaxDirectMemorySize=268435456, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, --module-path=/Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.13.0-SNAPSHOT/lib, --add-modules=jdk.net, --add-modules=ALL-MODULE-PATH, -Djdk.module.main=org.elasticsearch.server] +[2023-12-18T14:31:15,438][WARN ][o.e.n.Node ] [runTask-0] version [8.13.0-SNAPSHOT] is a pre-release version of Elasticsearch and is not suitable for production +[2023-12-18T14:31:17,172][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [repository-url] +[2023-12-18T14:31:17,172][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [rest-root] +[2023-12-18T14:31:17,172][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-core] +[2023-12-18T14:31:17,172][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-redact] +[2023-12-18T14:31:17,173][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [ingest-user-agent] +[2023-12-18T14:31:17,173][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-async-search] +[2023-12-18T14:31:17,173][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [test-error-query] +[2023-12-18T14:31:17,173][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-monitoring] +[2023-12-18T14:31:17,173][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [repository-s3] +[2023-12-18T14:31:17,173][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-analytics] +[2023-12-18T14:31:17,173][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-ent-search] +[2023-12-18T14:31:17,173][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-autoscaling] +[2023-12-18T14:31:17,173][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [lang-painless] +[2023-12-18T14:31:17,173][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-ml] +[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [test-die-with-dignity] +[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [legacy-geo] +[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [lang-mustache] +[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-ql] +[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [rank-rrf] +[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [analysis-common] +[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [test-seek-tracking-directory] +[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [health-shards-availability] +[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [transport-netty4] +[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [aggregations] +[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [ingest-common] +[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-identity-provider] +[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [frozen-indices] +[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-text-structure] +[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-shutdown] +[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [snapshot-repo-test-kit] +[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [ml-package-loader] +[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [test-delayed-aggs] +[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [kibana] +[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [constant-keyword] +[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-logstash] +[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-graph] +[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-ccr] +[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-esql] +[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [parent-join] +[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [counted-keyword] +[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-enrich] +[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [repositories-metering-api] +[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [transform] +[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [repository-azure] +[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [repository-gcs] +[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [spatial] +[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [apm] +[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [mapper-extras] +[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [mapper-version] +[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-rollup] +[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [percolator] +[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [data-streams] +[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-stack] +[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [rank-eval] +[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [reindex] +[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [test-apm-integration] +[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-security] +[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [blob-cache] +[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [searchable-snapshots] +[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-slm] +[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [test-latency-simulating-directory] +[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [snapshot-based-recoveries] +[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-watcher] +[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [old-lucene-versions] +[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-ilm] +[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-inference] +[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-voting-only-node] +[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-deprecation] +[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-fleet] +[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-profiling] +[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-aggregate-metric] +[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-downsample] +[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [ingest-geoip] +[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-write-load-forecaster] +[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [search-business-rules] +[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [wildcard] +[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [ingest-attachment] +[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-apm-data] +[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [unsigned-long] +[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-sql] +[2023-12-18T14:31:17,179][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [runtime-fields-common] +[2023-12-18T14:31:17,179][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-async] +[2023-12-18T14:31:17,179][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [vector-tile] +[2023-12-18T14:31:17,179][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [lang-expression] +[2023-12-18T14:31:17,179][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-eql] +[2023-12-18T14:31:17,224][INFO ][o.e.c.u.FeatureFlag ] [runTask-0] The current build is a snapshot, feature flag [failure_store] is enabled +[2023-12-18T14:31:17,520][INFO ][o.e.e.NodeEnvironment ] [runTask-0] using [1] data paths, mounts [[/System/Volumes/Data (/dev/disk3s5)]], net usable_space [636.9gb], net total_space [926.3gb], types [apfs] +[2023-12-18T14:31:17,520][INFO ][o.e.e.NodeEnvironment ] [runTask-0] heap size [512mb], compressed ordinary object pointers [true] +[2023-12-18T14:31:17,543][INFO ][o.e.n.Node ] [runTask-0] node name [runTask-0], node ID [RpeX_621SdCZSBh8-RSQvg], cluster name [runTask], roles [ingest, data_frozen, ml, data_hot, transform, data_content, data_warm, master, remote_cluster_client, data, data_cold] +[2023-12-18T14:31:19,294][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.allocator.desired_balance.shards.unassigned.count +[2023-12-18T14:31:19,294][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.allocator.desired_balance.shards.count +[2023-12-18T14:31:19,295][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.allocator.desired_balance.allocations.undesired.count +[2023-12-18T14:31:19,295][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.allocator.desired_balance.allocations.undesired.ratio +[2023-12-18T14:31:19,309][INFO ][o.e.c.u.FeatureFlag ] [runTask-0] The current build is a snapshot, feature flag [semantic_text] is enabled +[2023-12-18T14:31:19,326][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.breaker.parent.trip.total +[2023-12-18T14:31:19,326][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.breaker.field_data.trip.total +[2023-12-18T14:31:19,326][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.breaker.request.trip.total +[2023-12-18T14:31:19,326][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.breaker.in_flight_requests.trip.total +[2023-12-18T14:31:19,329][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.breaker.model_inference.trip.total +[2023-12-18T14:31:19,329][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.breaker.eql_sequence.trip.total +[2023-12-18T14:31:19,350][INFO ][o.e.f.FeatureService ] [runTask-0] Registered local node features [features_supported, health.dsl.info, usage.data_tiers.precalculate_stats] +[2023-12-18T14:31:19,497][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [runTask-0] [controller/43802] [Main.cc@123] controller (64 bit): Version 8.13.0-SNAPSHOT (Build c9c232240dd04f) Copyright (c) 2023 Elasticsearch BV +[2023-12-18T14:31:19,659][INFO ][o.e.t.a.APM ] [runTask-0] Sending apm metrics is disabled +[2023-12-18T14:31:19,660][INFO ][o.e.t.a.APM ] [runTask-0] Sending apm tracing is disabled +[2023-12-18T14:31:19,670][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.test.long_counter.count +[2023-12-18T14:31:19,671][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.test.double_counter.count +[2023-12-18T14:31:19,671][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.test.double_histogram.histogram +[2023-12-18T14:31:19,671][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.test.long_histogram.histogram +[2023-12-18T14:31:19,672][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.test.double_gauge.total +[2023-12-18T14:31:19,672][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.test.long_gauge.total +[2023-12-18T14:31:19,672][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.test.async_long_counter.count +[2023-12-18T14:31:19,672][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.test.async_double_counter.count +[2023-12-18T14:31:19,672][INFO ][o.e.x.s.Security ] [runTask-0] Security is enabled +[2023-12-18T14:31:19,850][INFO ][o.e.x.s.a.s.FileRolesStore] [runTask-0] parsed [1] roles from file [/Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/build/testclusters/runTask-0/config/roles.yml] +[2023-12-18T14:31:20,137][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.blob_cache.miss_that_triggered_read.count +[2023-12-18T14:31:20,138][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.blob_cache.evicted_used_regions.count +[2023-12-18T14:31:20,138][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.blob_cache.cache_miss_load.time +[2023-12-18T14:31:20,231][INFO ][o.e.x.w.Watcher ] [runTask-0] Watcher initialized components at 2023-12-18T13:31:20.231Z +[2023-12-18T14:31:20,256][INFO ][o.e.x.p.ProfilingPlugin ] [runTask-0] Profiling is enabled +[2023-12-18T14:31:20,267][INFO ][o.e.x.p.ProfilingPlugin ] [runTask-0] profiling index templates will not be installed or reinstalled +[2023-12-18T14:31:20,287][INFO ][o.e.x.a.APMPlugin ] [runTask-0] APM ingest plugin is disabled +[2023-12-18T14:31:20,408][INFO ][o.e.c.u.FeatureFlag ] [runTask-0] The current build is a snapshot, feature flag [connector_api] is enabled +[2023-12-18T14:31:20,553][INFO ][o.e.t.n.NettyAllocator ] [runTask-0] creating NettyAllocator with the following configs: [name=unpooled, suggested_max_allocation_size=1mb, factors={es.unsafe.use_unpooled_allocator=null, g1gc_enabled=true, g1gc_region_size=4mb, heap_size=512mb}] +[2023-12-18T14:31:20,566][INFO ][o.e.i.r.RecoverySettings ] [runTask-0] using rate limit [40mb] with [default=40mb, read=0b, write=0b, max=0b] +[2023-12-18T14:31:20,567][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.repositories.requests.count +[2023-12-18T14:31:20,567][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.repositories.exceptions.count +[2023-12-18T14:31:20,567][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.repositories.throttles.count +[2023-12-18T14:31:20,567][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.repositories.operations.count +[2023-12-18T14:31:20,568][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.repositories.operations.unsuccessful.count +[2023-12-18T14:31:20,568][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.repositories.exceptions.histogram +[2023-12-18T14:31:20,568][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.repositories.throttles.histogram +[2023-12-18T14:31:20,568][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.repositories.requests.http_request_time.histogram +[2023-12-18T14:31:20,591][INFO ][o.e.d.DiscoveryModule ] [runTask-0] using discovery type [multi-node] and seed hosts providers [settings, file] +[2023-12-18T14:31:20,619][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.health.overall.red.status +[2023-12-18T14:31:21,154][INFO ][o.e.n.Node ] [runTask-0] initialized +[2023-12-18T14:31:21,155][INFO ][o.e.n.Node ] [runTask-0] starting ... +[2023-12-18T14:31:21,182][INFO ][o.e.x.s.c.f.PersistentCache] [runTask-0] persistent cache index loaded +[2023-12-18T14:31:21,183][INFO ][o.e.x.d.l.DeprecationIndexingComponent] [runTask-0] deprecation component started +[2023-12-18T14:31:21,232][INFO ][o.e.t.TransportService ] [runTask-0] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300} +[2023-12-18T14:31:21,322][WARN ][o.e.b.BootstrapChecks ] [runTask-0] Transport SSL must be enabled if security is enabled. Please set [xpack.security.transport.ssl.enabled] to [true] or disable security by setting [xpack.security.enabled] to [false]; for more information see [https://www.elastic.co/guide/en/elasticsearch/reference/master/bootstrap-checks-xpack.html#bootstrap-checks-tls] +[2023-12-18T14:31:21,323][INFO ][o.e.c.c.ClusterBootstrapService] [runTask-0] this node has not joined a bootstrapped cluster yet; [cluster.initial_master_nodes] is set to [runTask-0] +[2023-12-18T14:31:21,325][WARN ][o.e.d.FileBasedSeedHostsProvider] [runTask-0] expected, but did not find, a dynamic hosts list at [/Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/build/testclusters/runTask-0/config/unicast_hosts.txt] +[2023-12-18T14:31:21,325][INFO ][o.e.c.c.Coordinator ] [runTask-0] setting initial configuration to VotingConfiguration{RpeX_621SdCZSBh8-RSQvg} +[2023-12-18T14:31:21,367][INFO ][o.e.h.AbstractHttpServerTransport] [runTask-0] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200} +[2023-12-18T14:31:21,372][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.get.total +[2023-12-18T14:31:21,372][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.get.time +[2023-12-18T14:31:21,372][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.search.fetch.total +[2023-12-18T14:31:21,373][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.search.fetch.time +[2023-12-18T14:31:21,373][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.merge.total +[2023-12-18T14:31:21,373][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.merge.time +[2023-12-18T14:31:21,373][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.translog.operations.count +[2023-12-18T14:31:21,373][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.translog.size +[2023-12-18T14:31:21,373][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.translog.uncommitted_operations.count +[2023-12-18T14:31:21,373][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.translog.uncommitted.size +[2023-12-18T14:31:21,373][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.translog.earliest_last_modified_age.time +[2023-12-18T14:31:21,373][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.transport.rx.size +[2023-12-18T14:31:21,373][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.transport.tx.size +[2023-12-18T14:31:21,374][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.jvm.mem.pools.young.size +[2023-12-18T14:31:21,374][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.jvm.mem.pools.survivor.size +[2023-12-18T14:31:21,374][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.jvm.mem.pools.old.size +[2023-12-18T14:31:21,374][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.fs.io_stats.io_time.total +[2023-12-18T14:31:21,374][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.docs.total +[2023-12-18T14:31:21,374][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.docs.total +[2023-12-18T14:31:21,374][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.failed.total +[2023-12-18T14:31:21,374][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.deletion.docs.total +[2023-12-18T14:31:21,374][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.deletion.docs.total +[2023-12-18T14:31:21,375][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.time +[2023-12-18T14:31:21,375][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.deletion.time +[2023-12-18T14:31:21,375][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.throttle.time +[2023-12-18T14:31:21,375][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.noop.total +[2023-12-18T14:31:21,375][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.coordinating_operations.memory.size.total +[2023-12-18T14:31:21,375][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.coordinating_operations.count.total +[2023-12-18T14:31:21,375][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.coordinating_operations.memory.size.total +[2023-12-18T14:31:21,375][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.coordinating_operations.count +[2023-12-18T14:31:21,376][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.coordinating_operations.rejections.total +[2023-12-18T14:31:21,376][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.primary_operations.memory.size.total +[2023-12-18T14:31:21,376][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.primary_operations.count.total +[2023-12-18T14:31:21,376][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.primary_operations.memory.size.total +[2023-12-18T14:31:21,376][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.primary_operations.count.total +[2023-12-18T14:31:21,376][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.primary_operations.rejections.total +[2023-12-18T14:31:21,376][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.memory.limit.total +[2023-12-18T14:31:21,376][INFO ][o.e.n.Node ] [runTask-0] started {runTask-0}{RpeX_621SdCZSBh8-RSQvg}{2w_LQi0HTTetBZfSdpLG6Q}{runTask-0}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.13.0}{7000099-8500007}{ml.machine_memory=68719476736, ml.config_version=12.0.0, ml.max_jvm_size=536870912, ml.allocated_processors=10, xpack.installed=true, transform.config_version=10.0.0, testattr=test, ml.allocated_processors_double=10.0} +[2023-12-18T14:31:21,421][INFO ][o.e.c.s.MasterService ] [runTask-0] elected-as-master ([1] nodes joined in term 1)[_FINISH_ELECTION_, {runTask-0}{RpeX_621SdCZSBh8-RSQvg}{2w_LQi0HTTetBZfSdpLG6Q}{runTask-0}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.13.0}{7000099-8500007} completing election], term: 1, version: 1, delta: master node changed {previous [], current [{runTask-0}{RpeX_621SdCZSBh8-RSQvg}{2w_LQi0HTTetBZfSdpLG6Q}{runTask-0}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.13.0}{7000099-8500007}]} +[2023-12-18T14:31:21,459][INFO ][o.e.c.c.CoordinationState] [runTask-0] cluster UUID set to [8TjOrE8yTw-_eqrhD95JPw] +[2023-12-18T14:31:21,484][INFO ][o.e.c.s.ClusterApplierService] [runTask-0] master node changed {previous [], current [{runTask-0}{RpeX_621SdCZSBh8-RSQvg}{2w_LQi0HTTetBZfSdpLG6Q}{runTask-0}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.13.0}{7000099-8500007}]}, term: 1, version: 1, reason: Publication{term=1, version=1} +[2023-12-18T14:31:21,498][INFO ][o.e.c.f.AbstractFileWatchingService] [runTask-0] starting file watcher ... +[2023-12-18T14:31:21,500][INFO ][o.e.c.f.AbstractFileWatchingService] [runTask-0] file settings service up and running [tid=60] +[2023-12-18T14:31:21,503][INFO ][o.e.c.c.NodeJoinExecutor ] [runTask-0] node-join: [{runTask-0}{RpeX_621SdCZSBh8-RSQvg}{2w_LQi0HTTetBZfSdpLG6Q}{runTask-0}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.13.0}{7000099-8500007}] with reason [completing election] +[2023-12-18T14:31:21,560][INFO ][o.e.g.GatewayService ] [runTask-0] recovered [0] indices into cluster_state +[2023-12-18T14:31:21,638][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [behavioral_analytics-events-mappings] +[2023-12-18T14:31:21,650][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [elastic-connectors-sync-jobs-mappings] +[2023-12-18T14:31:21,652][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [elastic-connectors-settings] +[2023-12-18T14:31:21,654][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [elastic-connectors-sync-jobs-settings] +[2023-12-18T14:31:21,674][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.monitoring-ent-search-mb] for index patterns [.monitoring-ent-search-8-*] +[2023-12-18T14:31:21,681][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [elastic-connectors-mappings] +[2023-12-18T14:31:21,694][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.monitoring-logstash-mb] for index patterns [.monitoring-logstash-8-*] +[2023-12-18T14:31:21,697][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.ml-state] for index patterns [.ml-state*] +[2023-12-18T14:31:21,702][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.ml-notifications-000002] for index patterns [.ml-notifications-000002] +[2023-12-18T14:31:21,705][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [search-acl-filter] for index patterns [.search-acl-filter-*] +[2023-12-18T14:31:21,723][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.monitoring-kibana-mb] for index patterns [.monitoring-kibana-8-*] +[2023-12-18T14:31:21,728][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding template [.monitoring-kibana] for index patterns [.monitoring-kibana-7-*] +[2023-12-18T14:31:21,734][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding template [.monitoring-logstash] for index patterns [.monitoring-logstash-7-*] +[2023-12-18T14:31:21,756][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding template [.monitoring-es] for index patterns [.monitoring-es-7-*] +[2023-12-18T14:31:21,763][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding template [.monitoring-beats] for index patterns [.monitoring-beats-7-*] +[2023-12-18T14:31:21,766][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding template [.monitoring-alerts-7] for index patterns [.monitoring-alerts-7] +[2023-12-18T14:31:21,775][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.ml-anomalies-] for index patterns [.ml-anomalies-*] +[2023-12-18T14:31:21,786][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.monitoring-beats-mb] for index patterns [.monitoring-beats-8-*] +[2023-12-18T14:31:21,802][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.monitoring-es-mb] for index patterns [.monitoring-es-8-*] +[2023-12-18T14:31:21,806][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.ml-stats] for index patterns [.ml-stats-*] +[2023-12-18T14:31:21,810][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [synthetics-mappings] +[2023-12-18T14:31:21,811][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [metrics-tsdb-settings] +[2023-12-18T14:31:21,813][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [logs-mappings] +[2023-12-18T14:31:21,816][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [metrics-mappings] +[2023-12-18T14:31:21,820][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [data-streams-mappings] +[2023-12-18T14:31:21,824][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [ecs@dynamic_templates] +[2023-12-18T14:31:21,825][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [synthetics-settings] +[2023-12-18T14:31:21,826][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [metrics-settings] +[2023-12-18T14:31:21,828][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [logs@mappings] +[2023-12-18T14:31:21,829][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [synthetics@mappings] +[2023-12-18T14:31:21,831][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [metrics@settings] +[2023-12-18T14:31:21,832][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [synthetics@settings] +[2023-12-18T14:31:21,834][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [metrics@mappings] +[2023-12-18T14:31:21,836][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [data-streams@mappings] +[2023-12-18T14:31:21,839][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [ecs@mappings] +[2023-12-18T14:31:21,842][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.kibana-reporting] for index patterns [.kibana-reporting*] +[2023-12-18T14:31:21,843][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [metrics@tsdb-settings] +[2023-12-18T14:31:21,846][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.slm-history] for index patterns [.slm-history-6*] +[2023-12-18T14:31:21,849][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [ilm-history] for index patterns [ilm-history-6*] +[2023-12-18T14:31:21,851][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [.deprecation-indexing-mappings] +[2023-12-18T14:31:21,852][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [.deprecation-indexing-settings] +[2023-12-18T14:31:21,856][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.fleet-fileds-fromhost-data] for index patterns [.fleet-fileds-fromhost-data-*] +[2023-12-18T14:31:21,859][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.fleet-fileds-tohost-data] for index patterns [.fleet-fileds-tohost-data-*] +[2023-12-18T14:31:21,862][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.fleet-fileds-tohost-meta] for index patterns [.fleet-fileds-tohost-meta-*] +[2023-12-18T14:31:21,867][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.fleet-fileds-fromhost-meta] for index patterns [.fleet-fileds-fromhost-meta-*] +[2023-12-18T14:31:21,872][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.watch-history-16] for index patterns [.watcher-history-16*] +[2023-12-18T14:31:21,943][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [elastic-connectors-sync-jobs] for index patterns [.elastic-connectors-sync-jobs-v1] +[2023-12-18T14:31:21,946][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [synthetics] for index patterns [synthetics-*-*] +[2023-12-18T14:31:21,949][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [elastic-connectors] for index patterns [.elastic-connectors-v1] +[2023-12-18T14:31:21,952][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [metrics] for index patterns [metrics-*-*] +[2023-12-18T14:31:21,955][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.deprecation-indexing-template] for index patterns [.logs-deprecation.*] +[2023-12-18T14:31:21,989][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [.monitoring-8-ilm-policy] +[2023-12-18T14:31:22,028][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [ml-size-based-ilm-policy] +[2023-12-18T14:31:22,063][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [metrics] +[2023-12-18T14:31:22,091][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [synthetics] +[2023-12-18T14:31:22,119][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [30-days-default] +[2023-12-18T14:31:22,207][INFO ][o.e.x.c.t.IndexTemplateRegistry] [runTask-0] adding ingest pipeline behavioral_analytics-events-final_pipeline +[2023-12-18T14:31:22,208][INFO ][o.e.x.c.t.IndexTemplateRegistry] [runTask-0] adding ingest pipeline logs-default-pipeline +[2023-12-18T14:31:22,208][INFO ][o.e.x.c.t.IndexTemplateRegistry] [runTask-0] adding ingest pipeline logs@default-pipeline +[2023-12-18T14:31:22,208][INFO ][o.e.x.c.t.IndexTemplateRegistry] [runTask-0] adding ingest pipeline ent-search-generic-ingestion +[2023-12-18T14:31:22,208][INFO ][o.e.x.c.t.IndexTemplateRegistry] [runTask-0] adding ingest pipeline logs@json-pipeline +[2023-12-18T14:31:22,208][INFO ][o.e.x.c.t.IndexTemplateRegistry] [runTask-0] adding ingest pipeline logs@json-message +[2023-12-18T14:31:22,210][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [behavioral_analytics-events-settings] +[2023-12-18T14:31:22,211][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [logs@settings] +[2023-12-18T14:31:22,211][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [logs-settings] +[2023-12-18T14:31:22,258][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [behavioral_analytics-events-default] for index patterns [behavioral_analytics-events-*] +[2023-12-18T14:31:22,261][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [logs] for index patterns [logs-*-*] +[2023-12-18T14:31:22,293][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [logs] +[2023-12-18T14:31:22,319][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [180-days-default] +[2023-12-18T14:31:22,347][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [365-days-default] +[2023-12-18T14:31:22,378][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [90-days-default] +[2023-12-18T14:31:22,406][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [7-days-default] +[2023-12-18T14:31:22,448][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [logs@lifecycle] +[2023-12-18T14:31:22,488][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [metrics@lifecycle] +[2023-12-18T14:31:22,522][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [synthetics@lifecycle] +[2023-12-18T14:31:22,551][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [90-days@lifecycle] +[2023-12-18T14:31:22,587][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [180-days@lifecycle] +[2023-12-18T14:31:22,621][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [7-days@lifecycle] +[2023-12-18T14:31:22,648][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [365-days@lifecycle] +[2023-12-18T14:31:22,675][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [30-days@lifecycle] +[2023-12-18T14:31:22,701][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [watch-history-ilm-policy-16] +[2023-12-18T14:31:22,729][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [slm-history-ilm-policy] +[2023-12-18T14:31:22,770][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [ilm-history-ilm-policy] +[2023-12-18T14:31:22,809][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [.deprecation-indexing-ilm-policy] +[2023-12-18T14:31:22,846][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [.fleet-file-tohost-data-ilm-policy] +[2023-12-18T14:31:22,875][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [.fleet-file-fromhost-meta-ilm-policy] +[2023-12-18T14:31:22,903][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [.fleet-file-fromhost-data-ilm-policy] +[2023-12-18T14:31:22,932][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [.fleet-actions-results-ilm-policy] +[2023-12-18T14:31:22,959][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [.fleet-file-tohost-meta-ilm-policy] +[2023-12-18T14:31:23,055][INFO ][o.e.h.n.s.HealthNodeTaskExecutor] [runTask-0] Node [{runTask-0}{RpeX_621SdCZSBh8-RSQvg}] is selected as the current health node. +[2023-12-18T14:31:23,113][INFO ][o.e.x.s.a.Realms ] [runTask-0] license mode is [basic], currently licensed security realms are [reserved/reserved,file/default_file,native/default_native] +[2023-12-18T14:31:23,114][INFO ][o.e.l.ClusterStateLicenseService] [runTask-0] license [d539d324-3578-49b7-8410-aa7c09249acf] mode [basic] - valid +<============-> 99% EXECUTING [34s] +> IDLE +> IDLE +> IDLE +> :run + + + diff --git a/x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/MlMetrics.java b/x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/MlMetrics.java index f2cedd4bf0f6..35f73551c2c3 100644 --- a/x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/MlMetrics.java +++ b/x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/MlMetrics.java @@ -103,7 +103,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust private void registerMlNodeMetrics(MeterRegistry meterRegistry) { metrics.add( meterRegistry.registerLongGauge( - "es.ml.native_memory.limit", + "es.ml.native_memory.usage", "ML native memory limit on this node.", "bytes", () -> new LongWithAttributes(nativeMemLimit, Map.of()) @@ -111,7 +111,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust ); metrics.add( meterRegistry.registerLongGauge( - "es.ml.native_memory.usage.anomaly_detectors", + "es.ml.native_memory.usage.anomaly_detectors.usage", "ML native memory used by anomaly detection jobs on this node.", "bytes", () -> new LongWithAttributes(nativeMemAdUsage, Map.of()) @@ -119,7 +119,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust ); metrics.add( meterRegistry.registerLongGauge( - "es.ml.native_memory.usage.data_frame_analytics", + "es.ml.native_memory.usage.data_frame_analytics.usage", "ML native memory used by data frame analytics jobs on this node.", "bytes", () -> new LongWithAttributes(nativeMemDfaUsage, Map.of()) @@ -127,7 +127,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust ); metrics.add( meterRegistry.registerLongGauge( - "es.ml.native_memory.usage.trained_models", + "es.ml.native_memory.usage.trained_models.usage", "ML native memory used by trained models on this node.", "bytes", () -> new LongWithAttributes(nativeMemTrainedModelUsage, Map.of()) @@ -135,7 +135,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust ); metrics.add( meterRegistry.registerLongGauge( - "es.ml.native_memory.free", + "es.ml.native_memory.free.size", "Free ML native memory on this node.", "bytes", () -> new LongWithAttributes(nativeMemFree, Map.of()) @@ -146,7 +146,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust private void registerMasterNodeMetrics(MeterRegistry meterRegistry) { metrics.add( meterRegistry.registerLongGauge( - "es.ml.anomaly_detectors.opening.count", + "es.ml.anomaly_detectors.opening.current", "Count of anomaly detection jobs in the opening state cluster-wide.", "jobs", () -> new LongWithAttributes(mlTaskStatusCounts.adOpeningCount, isMasterMap) @@ -154,7 +154,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust ); metrics.add( meterRegistry.registerLongGauge( - "es.ml.anomaly_detectors.opened.count", + "es.ml.anomaly_detectors.opened.current", "Count of anomaly detection jobs in the opened state cluster-wide.", "jobs", () -> new LongWithAttributes(mlTaskStatusCounts.adOpenedCount, isMasterMap) @@ -162,7 +162,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust ); metrics.add( meterRegistry.registerLongGauge( - "es.ml.anomaly_detectors.closing.count", + "es.ml.anomaly_detectors.closing.current", "Count of anomaly detection jobs in the closing state cluster-wide.", "jobs", () -> new LongWithAttributes(mlTaskStatusCounts.adClosingCount, isMasterMap) @@ -170,7 +170,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust ); metrics.add( meterRegistry.registerLongGauge( - "es.ml.anomaly_detectors.failed.count", + "es.ml.anomaly_detectors.failed.current", "Count of anomaly detection jobs in the failed state cluster-wide.", "jobs", () -> new LongWithAttributes(mlTaskStatusCounts.adFailedCount, isMasterMap) @@ -178,7 +178,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust ); metrics.add( meterRegistry.registerLongGauge( - "es.ml.datafeeds.starting.count", + "es.ml.datafeeds.starting.current", "Count of datafeeds in the starting state cluster-wide.", "datafeeds", () -> new LongWithAttributes(mlTaskStatusCounts.datafeedStartingCount, isMasterMap) @@ -186,7 +186,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust ); metrics.add( meterRegistry.registerLongGauge( - "es.ml.datafeeds.started.count", + "es.ml.datafeeds.started.current", "Count of datafeeds in the started state cluster-wide.", "datafeeds", () -> new LongWithAttributes(mlTaskStatusCounts.datafeedStartedCount, isMasterMap) @@ -194,7 +194,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust ); metrics.add( meterRegistry.registerLongGauge( - "es.ml.datafeeds.stopping.count", + "es.ml.datafeeds.stopping.current", "Count of datafeeds in the stopping state cluster-wide.", "datafeeds", () -> new LongWithAttributes(mlTaskStatusCounts.datafeedStoppingCount, isMasterMap) @@ -202,7 +202,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust ); metrics.add( meterRegistry.registerLongGauge( - "es.ml.data_frame_analytics.starting.count", + "es.ml.data_frame_analytics.starting.current", "Count of data frame analytics jobs in the starting state cluster-wide.", "jobs", () -> new LongWithAttributes(mlTaskStatusCounts.dfaStartingCount, isMasterMap) @@ -210,7 +210,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust ); metrics.add( meterRegistry.registerLongGauge( - "es.ml.data_frame_analytics.started.count", + "es.ml.data_frame_analytics.started.current", "Count of data frame analytics jobs in the started state cluster-wide.", "jobs", () -> new LongWithAttributes(mlTaskStatusCounts.dfaStartedCount, isMasterMap) @@ -218,7 +218,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust ); metrics.add( meterRegistry.registerLongGauge( - "es.ml.data_frame_analytics.reindexing.count", + "es.ml.data_frame_analytics.reindexing.current", "Count of data frame analytics jobs in the reindexing state cluster-wide.", "jobs", () -> new LongWithAttributes(mlTaskStatusCounts.dfaReindexingCount, isMasterMap) @@ -226,7 +226,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust ); metrics.add( meterRegistry.registerLongGauge( - "es.ml.data_frame_analytics.analyzing.count", + "es.ml.data_frame_analytics.analyzing.current", "Count of data frame analytics jobs in the analyzing state cluster-wide.", "jobs", () -> new LongWithAttributes(mlTaskStatusCounts.dfaAnalyzingCount, isMasterMap) @@ -234,7 +234,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust ); metrics.add( meterRegistry.registerLongGauge( - "es.ml.data_frame_analytics.stopping.count", + "es.ml.data_frame_analytics.stopping.current", "Count of data frame analytics jobs in the stopping state cluster-wide.", "jobs", () -> new LongWithAttributes(mlTaskStatusCounts.dfaStoppingCount, isMasterMap) @@ -242,7 +242,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust ); metrics.add( meterRegistry.registerLongGauge( - "es.ml.data_frame_analytics.failed.count", + "es.ml.data_frame_analytics.failed.current", "Count of data frame analytics jobs in the failed state cluster-wide.", "jobs", () -> new LongWithAttributes(mlTaskStatusCounts.dfaFailedCount, isMasterMap) @@ -250,7 +250,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust ); metrics.add( meterRegistry.registerLongGauge( - "es.ml.trained_models.deployment.target_allocations.count", + "es.ml.trained_models.deployment.target_allocations.current", "Sum of target trained model allocations across all deployments cluster-wide.", "allocations", () -> new LongWithAttributes(trainedModelAllocationCounts.trainedModelsTargetAllocations, isMasterMap) @@ -258,7 +258,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust ); metrics.add( meterRegistry.registerLongGauge( - "es.ml.trained_models.deployment.current_allocations.count", + "es.ml.trained_models.deployment.current_allocations.current", "Sum of current trained model allocations across all deployments cluster-wide.", "allocations", () -> new LongWithAttributes(trainedModelAllocationCounts.trainedModelsCurrentAllocations, isMasterMap) @@ -266,7 +266,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust ); metrics.add( meterRegistry.registerLongGauge( - "es.ml.trained_models.deployment.failed_allocations.count", + "es.ml.trained_models.deployment.failed_allocations.current", "Sum of failed trained model allocations across all deployments cluster-wide.", "allocations", () -> new LongWithAttributes(trainedModelAllocationCounts.trainedModelsFailedAllocations, isMasterMap)