github-mirrors/elasticsearch
1
0
Fork 0
mirror of https://github.com/elastic/elasticsearch.git synced 2025-06-28 09:28:55 -04:00
Code Issues Projects Releases Packages Wiki Activity

Add metric name validation (#103388)

Browse source 
This commit adds a minimum metric name validation which checks:

metric name starts with es. prefix
metric name is using . as a separator of elements
metric name is using characters from a white list
validate min number of elements = 3 elements ( prefix, group and the suffix name)
validate max number of elements and max characters per element
validate the suffix element in a metric name to be from the enumerated allow list

It also modifies existing metric names to adhere to those rules
This commit is contained in:
Przemyslaw Gomulka 2024-01-09 11:05:00 +01:00 • committed by GitHub
parent 943b2eae70
commit 07780a8282
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
18 changed files with 699 additions and 140 deletions
Download patch file Download diff file Expand all files Collapse all files

23
modules/apm/NAMING.md
View file

@ -17,13 +17,13 @@ The **hierarchy** should be built by putting "more common" elements at the begin
Example: Example:
* prefer `es.indices.docs.deleted.total `to `es.indices.total.deleted.docs` * prefer `es.indices.docs.deleted.total `to `es.indices.total.deleted.docs`
* This way you can later add` es.indices.docs.count, es.indices.docs.ingested.total`, etc.) * This way you can later add` es.indices.docs.total, es.indices.docs.ingested.total`, etc.)
Prefix metrics: Prefix metrics:
* Always use `es` as our root application name: this will give us a separate namespace and avoid any possibility of clashes with other metrics, and quick identification of Elasticsearch metrics on a dashboard. * Always use `es` as our root application name: this will give us a separate namespace and avoid any possibility of clashes with other metrics, and quick identification of Elasticsearch metrics on a dashboard.
* Follow the root prefix with a simple module name, team or area of code. E.g. `snapshot, repositories, indices, threadpool`. Notice the mix of singular and plural - here this is intentional, to reflect closely the existing names in the codebase (e.g. `reindex` and `indices`) * Follow the root prefix with a simple module name, team or area of code. E.g. `snapshot, repositories, indices, threadpool`. Notice the mix of singular and plural - here this is intentional, to reflect closely the existing names in the codebase (e.g. `reindex` and `indices`)
* In building a metric name, look for existing prefixes (e.g. module name and/or area of code, e.g. `blob_cache`) and for existing sub-elements as well (e.g. `error`) to build a good, consistent name. E.g. prefer the consistent use of `error.count` rather than introducing `failures`, `failed.count` or `errors`.` ` * In building a metric name, look for existing prefixes (e.g. module name and/or area of code, e.g. `blob_cache`) and for existing sub-elements as well (e.g. `error`) to build a good, consistent name. E.g. prefer the consistent use of `error.total` rather than introducing `failures`, `failed.total` or `errors`.` `
* Avoid having sub-metrics under a name that is also a metric (e.g. do not create names like `es.repositories.elements`,` es.repositories.elements.utilization`; use` es.repositories.element.count` and` es.repositories.element.utilization `instead). Such metrics are hard to handle well in Elasticsearch, or in some internal structures (e.g. nested maps). * Avoid having sub-metrics under a name that is also a metric (e.g. do not create names like `es.repositories.elements`,` es.repositories.elements.utilization`; use` es.repositories.element.total` and` es.repositories.element.utilization `instead). Such metrics are hard to handle well in Elasticsearch, or in some internal structures (e.g. nested maps).
Keep the hierarchy compact: do not add elements if you don’t need to. There is a description field when registering a metric, prefer using that as an explanation. \ Keep the hierarchy compact: do not add elements if you don’t need to. There is a description field when registering a metric, prefer using that as an explanation. \
For example, if emitting existing metrics from node stats, do not use the whole “object path”, but choose the most significant terms. For example, if emitting existing metrics from node stats, do not use the whole “object path”, but choose the most significant terms.
@ -35,7 +35,7 @@ The metric name can be generated but there should be no dynamic or variable cont
* Rule of thumb: you should be able to do aggregations (e.g. sum, avg) across a dimension of a given metric (without the need to aggregate over different metric names); on the other hand, any aggregation across any dimension of a given metric should be meaningful. * Rule of thumb: you should be able to do aggregations (e.g. sum, avg) across a dimension of a given metric (without the need to aggregate over different metric names); on the other hand, any aggregation across any dimension of a given metric should be meaningful.
* There might be exceptions of course. For example: * There might be exceptions of course. For example:
* When similar metrics have significantly different implementations/related metrics. \ * When similar metrics have significantly different implementations/related metrics. \
If we have only common metrics like `es.repositories.element.count, es.repositories.element.utilization, es.repositories.writes.total` for every blob storage implementation, then `s3,azure` should be an attribute. \ If we have only common metrics like `es.repositories.element.total, es.repositories.element.utilization, es.repositories.writes.total` for every blob storage implementation, then `s3,azure` should be an attribute. \
If we have specific metrics, e.g. for s3 storage classes, prefer using prefixed metric names for the specific metrics: <code>es.repositories.<strong>s3</strong>.deep_archive_access.total</code> (but keep `es.repositories.elements`) If we have specific metrics, e.g. for s3 storage classes, prefer using prefixed metric names for the specific metrics: <code>es.repositories.<strong>s3</strong>.deep_archive_access.total</code> (but keep `es.repositories.elements`)
* When you have a finite and fixed set of names it might be OK to have them in the name (e.g. "`young`" and "`old`" for GC generations). * When you have a finite and fixed set of names it might be OK to have them in the name (e.g. "`young`" and "`old`" for GC generations).
@ -47,12 +47,19 @@ Examples :
* <code>es.indices.storage.write.<strong>io</strong></code>, instead of <code>es.indices.storage.write.<strong>bytes_per_sec</strong></code> * <code>es.indices.storage.write.<strong>io</strong></code>, instead of <code>es.indices.storage.write.<strong>bytes_per_sec</strong></code>
* These can all be composed with the suffixes below, e.g. <code>es.process.jvm.collection.<strong>time.total</strong></code>, <code>es.indices.storage.write.<strong>total</strong></code> to represent the monotonic sum of time spent in GC and the total number of bytes written to indices respectively. * These can all be composed with the suffixes below, e.g. <code>es.process.jvm.collection.<strong>time.total</strong></code>, <code>es.indices.storage.write.<strong>total</strong></code> to represent the monotonic sum of time spent in GC and the total number of bytes written to indices respectively.
**Pluralization** and **suffixes**: **Suffixes**:
* If the metric is unit-less, use plural: `es.threadpool.activethreads`, `es.indices.docs` * Use `total` as a suffix for monotonic metrics (always increasing counter) (e.g. <code>es.indices.docs.deleted.<strong>total</strong></code>)
* Use `total` as a suffix for monotonic sums (e.g. <code>es.indices.docs.deleted.<strong>total</strong></code>) * Note: even though async counter is reporting a total cumulative value, it is till monotonic.
* Use `count` to represent the count of "things" in the metric name/namespace (e.g. if we have `es.process.jvm.classes.loaded`, we will express the number of classes currently loaded by the JVM as <code>es.process.jvm.classes.loaded.<strong>count</strong></code>, and the total number of classes loaded since the JVM started as <code>es.process.jvm.classes.loaded.<strong>total</strong></code> * Use `current` to represent the non-monotonic metrics (like gauges, upDownCounters)
* e.g. `current` vs `total` We can have <code>es.process.jvm.classes.loaded.<strong>current</strong></code> to express the number of classes currently loaded by the JVM, and the total number of classes loaded since the JVM started as <code>es.process.jvm.classes.loaded.<strong>total</strong></code>
* Use `ratio` to represent the ratio of two measures with identical unit (or unit-less) or measures that represent a fraction in the range [0, 1]. Examples: * Use `ratio` to represent the ratio of two measures with identical unit (or unit-less) or measures that represent a fraction in the range [0, 1]. Examples:
* Exception: consider using utilization when the ratio is between a usage and its limit, e.g. the ratio between <code>es.process.jvm.heap.<strong>usage</strong></code> and <code>es.process.jvm.heap.<strong>limit</strong></code> should be <code>es.process.jvm.heap.<strong>utilization</strong></code> * Exception: consider using utilization when the ratio is between a usage and its limit, e.g. the ratio between <code>es.process.jvm.heap.<strong>usage</strong></code> and <code>es.process.jvm.heap.<strong>limit</strong></code> should be <code>es.process.jvm.heap.<strong>utilization</strong></code>
* Use `status` to represent enum like gauges. example <code>es.health.overall.red.status</code> have values 1/0 to represent true/false
* Use `usage` to represent the amount used ouf of the known resource size
* Use `size` to represent the overall size of the resource measured
* Use `utilisation` to represent a fraction of usage out of the overall size of a resource measured
* Use `histogram` to represent instruments of type histogram
* Use `time` to represent passage of time
* If it has a unit of measure, then it should not be plural (and also not include the unit of measure, see above). Examples: <code>es.process.jvm.collection.time, es.process.mem.virtual.usage<strong>, </strong>es.indices.storage.utilization</code> * If it has a unit of measure, then it should not be plural (and also not include the unit of measure, see above). Examples: <code>es.process.jvm.collection.time, es.process.mem.virtual.usage<strong>, </strong>es.indices.storage.utilization</code>
### Attributes ### Attributes

10
modules/apm/src/main/java/org/elasticsearch/telemetry/apm/AbstractInstrument.java
View file

@ -11,6 +11,7 @@ package org.elasticsearch.telemetry.apm;
import io.opentelemetry.api.metrics.Meter; import io.opentelemetry.api.metrics.Meter;
import org.elasticsearch.core.Nullable; import org.elasticsearch.core.Nullable;
import org.elasticsearch.telemetry.apm.internal.MetricNameValidator;
import org.elasticsearch.telemetry.metric.Instrument; import org.elasticsearch.telemetry.metric.Instrument;
import java.security.AccessController; import java.security.AccessController;
@ -23,6 +24,7 @@ import java.util.function.Function;
* An instrument that contains the name, description and unit. The delegate may be replaced when * An instrument that contains the name, description and unit. The delegate may be replaced when
* the provider is updated. * the provider is updated.
* Subclasses should implement the builder, which is used on initialization and provider updates. * Subclasses should implement the builder, which is used on initialization and provider updates.
*
* @param <T> delegated instrument * @param <T> delegated instrument
*/ */
public abstract class AbstractInstrument<T> implements Instrument { public abstract class AbstractInstrument<T> implements Instrument {
@ -50,19 +52,13 @@ public abstract class AbstractInstrument<T> implements Instrument {
} }
protected abstract static class Builder<T> { protected abstract static class Builder<T> {
private static final int MAX_NAME_LENGTH = 255;
protected final String name; protected final String name;
protected final String description; protected final String description;
protected final String unit; protected final String unit;
public Builder(String name, String description, String unit) { public Builder(String name, String description, String unit) {
if (name.length() > MAX_NAME_LENGTH) { this.name = MetricNameValidator.validate(name);
throw new IllegalArgumentException(
"Instrument name [" + name + "] with length [" + name.length() + "] exceeds maximum length [" + MAX_NAME_LENGTH + "]"
);
}
this.name = Objects.requireNonNull(name);
this.description = Objects.requireNonNull(description); this.description = Objects.requireNonNull(description);
this.unit = Objects.requireNonNull(unit); this.unit = Objects.requireNonNull(unit);
} }

142
modules/apm/src/main/java/org/elasticsearch/telemetry/apm/internal/MetricNameValidator.java Normal file
View file

@ -0,0 +1,142 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
package org.elasticsearch.telemetry.apm.internal;
import java.util.Objects;
import java.util.Set;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.stream.Collectors;
public class MetricNameValidator {
private static final Pattern ALLOWED_CHARACTERS = Pattern.compile("[a-z][a-z0-9_]*");
static final Set<String> ALLOWED_SUFFIXES = Set.of(
"total",
"current",
"ratio",
"status" /*a workaround for enums */,
"usage",
"size",
"utilization",
"histogram",
"time"
);
static final int MAX_METRIC_NAME_LENGTH = 255;
static final int MAX_ELEMENT_LENGTH = 30;
static final int MAX_NUMBER_OF_ELEMENTS = 10;
private MetricNameValidator() {}
/**
* Validates a metric name as per guidelines in Naming.md
*
* @param metricName metric name to be validated
* @throws IllegalArgumentException an exception indicating an incorrect metric name
*/
public static String validate(String metricName) {
Objects.requireNonNull(metricName);
validateMaxMetricNameLength(metricName);
String[] elements = metricName.split("\\.");
hasESPrefix(elements, metricName);
hasAtLeast3Elements(elements, metricName);
hasNotBreachNumberOfElementsLimit(elements, metricName);
lastElementIsFromAllowList(elements, metricName);
perElementValidations(elements, metricName);
return metricName;
}
private static void validateMaxMetricNameLength(String metricName) {
if (metricName.length() > MAX_METRIC_NAME_LENGTH) {
throw new IllegalArgumentException(
"Metric name length "
+ metricName.length()
+ "is longer than max metric name length:"
+ MAX_METRIC_NAME_LENGTH
+ " Name was: "
+ metricName
);
}
}
private static void lastElementIsFromAllowList(String[] elements, String name) {
String lastElement = elements[elements.length - 1];
if (ALLOWED_SUFFIXES.contains(lastElement) == false) {
throw new IllegalArgumentException(
"Metric name should end with one of ["
+ ALLOWED_SUFFIXES.stream().collect(Collectors.joining(","))
+ "] "
+ "Last element was: "
+ lastElement
+ ". "
+ "Name was: "
+ name
);
}
}
private static void hasNotBreachNumberOfElementsLimit(String[] elements, String name) {
if (elements.length > MAX_NUMBER_OF_ELEMENTS) {
throw new IllegalArgumentException(
"Metric name should have at most 10 elements. It had: " + elements.length + ". The name was: " + name
);
}
}
private static void hasAtLeast3Elements(String[] elements, String name) {
if (elements.length < 3) {
throw new IllegalArgumentException(
"Metric name consist of at least 3 elements. An es. prefix, group and a name. The name was: " + name
);
}
}
private static void hasESPrefix(String[] elements, String name) {
if (elements[0].equals("es") == false) {
throw new IllegalArgumentException(
"Metric name should start with \"es.\" prefix and use \".\" as a separator. Name was: " + name
);
}
}
private static void perElementValidations(String[] elements, String name) {
for (String element : elements) {
hasOnlyAllowedCharacters(element, name);
hasNotBreachLengthLimit(element, name);
}
}
private static void hasNotBreachLengthLimit(String element, String name) {
if (element.length() > MAX_ELEMENT_LENGTH) {
throw new IllegalArgumentException(
"Metric name's element should not be longer than "
+ MAX_ELEMENT_LENGTH
+ " characters. Was: "
+ element.length()
+ ". Name was: "
+ name
);
}
}
private static void hasOnlyAllowedCharacters(String element, String name) {
Matcher matcher = ALLOWED_CHARACTERS.matcher(element);
if (matcher.matches() == false) {
throw new IllegalArgumentException(
"Metric name should only use [a-z][a-z0-9_]* characters. "
+ "Element does not match: \""
+ element
+ "\". "
+ "Name was: "
+ name
);
}
}
}

39
modules/apm/src/test/java/org/elasticsearch/telemetry/apm/APMMeterRegistryTests.java
View file

@ -35,10 +35,8 @@ import java.util.Collections;
import java.util.List; import java.util.List;
import java.util.function.Supplier; import java.util.function.Supplier;
import static org.hamcrest.Matchers.containsString;
import static org.hamcrest.Matchers.equalTo; import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.hasSize; import static org.hamcrest.Matchers.hasSize;
import static org.hamcrest.Matchers.instanceOf;
import static org.hamcrest.Matchers.sameInstance; import static org.hamcrest.Matchers.sameInstance;
public class APMMeterRegistryTests extends ESTestCase { public class APMMeterRegistryTests extends ESTestCase {
@ -84,8 +82,8 @@ public class APMMeterRegistryTests extends ESTestCase {
public void testLookupByName() { public void testLookupByName() {
var apmMeter = new APMMeterService(TELEMETRY_ENABLED, () -> testOtel, () -> noopOtel).getMeterRegistry(); var apmMeter = new APMMeterService(TELEMETRY_ENABLED, () -> testOtel, () -> noopOtel).getMeterRegistry();
DoubleCounter registeredCounter = apmMeter.registerDoubleCounter("name", "desc", "unit"); DoubleCounter registeredCounter = apmMeter.registerDoubleCounter("es.test.name.total", "desc", "unit");
DoubleCounter lookedUpCounter = apmMeter.getDoubleCounter("name"); DoubleCounter lookedUpCounter = apmMeter.getDoubleCounter("es.test.name.total");
assertThat(lookedUpCounter, sameInstance(registeredCounter)); assertThat(lookedUpCounter, sameInstance(registeredCounter));
} }
@ -103,19 +101,6 @@ public class APMMeterRegistryTests extends ESTestCase {
assertThat(meter, sameInstance(noopOtel)); assertThat(meter, sameInstance(noopOtel));
} }
public void testMaxNameLength() {
APMMeterService apmMeter = new APMMeterService(TELEMETRY_ENABLED, () -> testOtel, () -> noopOtel);
apmMeter.start();
int max_length = 255;
var counter = apmMeter.getMeterRegistry().registerLongCounter("a".repeat(max_length), "desc", "count");
assertThat(counter, instanceOf(LongCounter.class));
IllegalArgumentException iae = expectThrows(
IllegalArgumentException.class,
() -> apmMeter.getMeterRegistry().registerLongCounter("a".repeat(max_length + 1), "desc", "count")
);
assertThat(iae.getMessage(), containsString("exceeds maximum length [255]"));
}
public void testAllInstrumentsSwitchProviders() { public void testAllInstrumentsSwitchProviders() {
TestAPMMeterService apmMeter = new TestAPMMeterService( TestAPMMeterService apmMeter = new TestAPMMeterService(
Settings.builder().put(APMAgentSettings.TELEMETRY_METRICS_ENABLED_SETTING.getKey(), false).build(), Settings.builder().put(APMAgentSettings.TELEMETRY_METRICS_ENABLED_SETTING.getKey(), false).build(),
@ -125,18 +110,18 @@ public class APMMeterRegistryTests extends ESTestCase {
APMMeterRegistry registry = apmMeter.getMeterRegistry(); APMMeterRegistry registry = apmMeter.getMeterRegistry();
Supplier<DoubleWithAttributes> doubleObserver = () -> new DoubleWithAttributes(1.5, Collections.emptyMap()); Supplier<DoubleWithAttributes> doubleObserver = () -> new DoubleWithAttributes(1.5, Collections.emptyMap());
DoubleCounter dc = registry.registerDoubleCounter("dc", "", ""); DoubleCounter dc = registry.registerDoubleCounter("es.test.dc.total", "", "");
DoubleUpDownCounter dudc = registry.registerDoubleUpDownCounter("dudc", "", ""); DoubleUpDownCounter dudc = registry.registerDoubleUpDownCounter("es.test.dudc.current", "", "");
DoubleHistogram dh = registry.registerDoubleHistogram("dh", "", ""); DoubleHistogram dh = registry.registerDoubleHistogram("es.test.dh.histogram", "", "");
DoubleAsyncCounter dac = registry.registerDoubleAsyncCounter("dac", "", "", doubleObserver); DoubleAsyncCounter dac = registry.registerDoubleAsyncCounter("es.test.dac.total", "", "", doubleObserver);
DoubleGauge dg = registry.registerDoubleGauge("dg", "", "", doubleObserver); DoubleGauge dg = registry.registerDoubleGauge("es.test.dg.current", "", "", doubleObserver);
Supplier<LongWithAttributes> longObserver = () -> new LongWithAttributes(100, Collections.emptyMap()); Supplier<LongWithAttributes> longObserver = () -> new LongWithAttributes(100, Collections.emptyMap());
LongCounter lc = registry.registerLongCounter("lc", "", ""); LongCounter lc = registry.registerLongCounter("es.test.lc.total", "", "");
LongUpDownCounter ludc = registry.registerLongUpDownCounter("ludc", "", ""); LongUpDownCounter ludc = registry.registerLongUpDownCounter("es.test.ludc.total", "", "");
LongHistogram lh = registry.registerLongHistogram("lh", "", ""); LongHistogram lh = registry.registerLongHistogram("es.test.lh.histogram", "", "");
LongAsyncCounter lac = registry.registerLongAsyncCounter("lac", "", "", longObserver); LongAsyncCounter lac = registry.registerLongAsyncCounter("es.test.lac.total", "", "", longObserver);
LongGauge lg = registry.registerLongGauge("lg", "", "", longObserver); LongGauge lg = registry.registerLongGauge("es.test.lg.current", "", "", longObserver);
apmMeter.setEnabled(true); apmMeter.setEnabled(true);

2
modules/apm/src/test/java/org/elasticsearch/telemetry/apm/MeterRegistryConcurrencyTests.java
View file

@ -28,7 +28,7 @@ import static org.hamcrest.Matchers.equalTo;
import static org.hamcrest.Matchers.sameInstance; import static org.hamcrest.Matchers.sameInstance;
public class MeterRegistryConcurrencyTests extends ESTestCase { public class MeterRegistryConcurrencyTests extends ESTestCase {
private final String name = "name"; private final String name = "es.test.name.total";
private final String description = "desc"; private final String description = "desc";
private final String unit = "kg"; private final String unit = "kg";
private final Meter noopMeter = OpenTelemetry.noop().getMeter("noop"); private final Meter noopMeter = OpenTelemetry.noop().getMeter("noop");

102
modules/apm/src/test/java/org/elasticsearch/telemetry/apm/internal/MetricNameValidatorTests.java Normal file
View file

@ -0,0 +1,102 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/
package org.elasticsearch.telemetry.apm.internal;
import org.elasticsearch.test.ESTestCase;
import static org.hamcrest.Matchers.equalTo;
public class MetricNameValidatorTests extends ESTestCase {
public void testMetricNameNotNull() {
String metricName = "es.somemodule.somemetric.total";
assertThat(MetricNameValidator.validate(metricName), equalTo(metricName));
expectThrows(NullPointerException.class, () -> MetricNameValidator.validate(null));
}
public void testMaxMetricNameLength() {
MetricNameValidator.validate(metricNameWithLength(255));
expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate(metricNameWithLength(256)));
}
public void testESPrefixAndDotSeparator() {
MetricNameValidator.validate("es.somemodule.somemetric.total");
expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("somemodule.somemetric.total"));
// verify . is a separator
expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es_somemodule_somemetric_total"));
expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es_somemodule.somemetric.total"));
}
public void testNameElementRegex() {
MetricNameValidator.validate("es.somemodulename0.somemetric.total");
MetricNameValidator.validate("es.some_module_name0.somemetric.total");
MetricNameValidator.validate("es.s.somemetric.total");
expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es.someModuleName0.somemetric.total"));
expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es.SomeModuleName.somemetric.total"));
expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es.0some_module_name0.somemetric.total"));
expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es.some_#_name0.somemetric.total"));
expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es.some-name0.somemetric.total"));
}
public void testNameHas3Elements() {
MetricNameValidator.validate("es.group.total");
MetricNameValidator.validate("es.group.subgroup.total");
expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es"));
expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es."));
expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es.sth"));
}
public void testNumberOfElementsLimit() {
MetricNameValidator.validate("es.a2.a3.a4.a5.a6.a7.a8.a9.total");
expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es.a2.a3.a4.a5.a6.a7.a8.a9.a10.total"));
}
public void testElementLengthLimit() {
MetricNameValidator.validate("es." + "a".repeat(MetricNameValidator.MAX_ELEMENT_LENGTH) + ".total");
expectThrows(
IllegalArgumentException.class,
() -> MetricNameValidator.validate("es." + "a".repeat(MetricNameValidator.MAX_ELEMENT_LENGTH + 1) + ".total")
);
}
public void testLastElementAllowList() {
for (String suffix : MetricNameValidator.ALLOWED_SUFFIXES) {
MetricNameValidator.validate("es.somemodule.somemetric." + suffix);
}
expectThrows(IllegalArgumentException.class, () -> MetricNameValidator.validate("es.somemodule.somemetric.some_other_suffix"));
}
public static String metricNameWithLength(int length) {
int prefixAndSuffix = "es.".length() + ".utilization".length();
assert length > prefixAndSuffix : "length too short";
var remainingChars = length - prefixAndSuffix;
StringBuilder metricName = new StringBuilder("es.");
var i = 0;
while (i < remainingChars) {
metricName.append("a");
i++;
for (int j = 0; j < MetricNameValidator.MAX_ELEMENT_LENGTH - 1 && i < remainingChars; j++) {
metricName.append("x");
i++;
}
metricName.append(".");
i++;
}
metricName.append("utilization");
return metricName.toString();
}
}

8
modules/apm/src/test/java/org/elasticsearch/telemetry/apm/internal/metrics/AsyncCountersAdapterTests.java
View file

@ -38,7 +38,7 @@ public class AsyncCountersAdapterTests extends ESTestCase {
// testing that a value reported is then used in a callback // testing that a value reported is then used in a callback
public void testLongAsyncCounter() throws Exception { public void testLongAsyncCounter() throws Exception {
AtomicReference<LongWithAttributes> attrs = new AtomicReference<>(); AtomicReference<LongWithAttributes> attrs = new AtomicReference<>();
LongAsyncCounter longAsyncCounter = registry.registerLongAsyncCounter("name", "desc", "unit", attrs::get); LongAsyncCounter longAsyncCounter = registry.registerLongAsyncCounter("es.test.name.total", "desc", "unit", attrs::get);
attrs.set(new LongWithAttributes(1L, Map.of("k", 1L))); attrs.set(new LongWithAttributes(1L, Map.of("k", 1L)));
@ -70,7 +70,7 @@ public class AsyncCountersAdapterTests extends ESTestCase {
public void testDoubleAsyncAdapter() throws Exception { public void testDoubleAsyncAdapter() throws Exception {
AtomicReference<DoubleWithAttributes> attrs = new AtomicReference<>(); AtomicReference<DoubleWithAttributes> attrs = new AtomicReference<>();
DoubleAsyncCounter doubleAsyncCounter = registry.registerDoubleAsyncCounter("name", "desc", "unit", attrs::get); DoubleAsyncCounter doubleAsyncCounter = registry.registerDoubleAsyncCounter("es.test.name.total", "desc", "unit", attrs::get);
attrs.set(new DoubleWithAttributes(1.0, Map.of("k", 1.0))); attrs.set(new DoubleWithAttributes(1.0, Map.of("k", 1.0)));
@ -102,7 +102,7 @@ public class AsyncCountersAdapterTests extends ESTestCase {
public void testNullGaugeRecord() throws Exception { public void testNullGaugeRecord() throws Exception {
DoubleAsyncCounter dcounter = registry.registerDoubleAsyncCounter( DoubleAsyncCounter dcounter = registry.registerDoubleAsyncCounter(
"name", "es.test.name.total",
"desc", "desc",
"unit", "unit",
new AtomicReference<DoubleWithAttributes>()::get new AtomicReference<DoubleWithAttributes>()::get
@ -112,7 +112,7 @@ public class AsyncCountersAdapterTests extends ESTestCase {
assertThat(metrics, hasSize(0)); assertThat(metrics, hasSize(0));
LongAsyncCounter lcounter = registry.registerLongAsyncCounter( LongAsyncCounter lcounter = registry.registerLongAsyncCounter(
"name", "es.test.name.total",
"desc", "desc",
"unit", "unit",
new AtomicReference<LongWithAttributes>()::get new AtomicReference<LongWithAttributes>()::get

13
modules/apm/src/test/java/org/elasticsearch/telemetry/apm/internal/metrics/GaugeAdapterTests.java
View file

@ -38,7 +38,7 @@ public class GaugeAdapterTests extends ESTestCase {
// testing that a value reported is then used in a callback // testing that a value reported is then used in a callback
public void testLongGaugeRecord() throws Exception { public void testLongGaugeRecord() throws Exception {
AtomicReference<LongWithAttributes> attrs = new AtomicReference<>(); AtomicReference<LongWithAttributes> attrs = new AtomicReference<>();
LongGauge gauge = registry.registerLongGauge("name", "desc", "unit", attrs::get); LongGauge gauge = registry.registerLongGauge("es.test.name.total", "desc", "unit", attrs::get);
attrs.set(new LongWithAttributes(1L, Map.of("k", 1L))); attrs.set(new LongWithAttributes(1L, Map.of("k", 1L)));
@ -71,7 +71,7 @@ public class GaugeAdapterTests extends ESTestCase {
// testing that a value reported is then used in a callback // testing that a value reported is then used in a callback
public void testDoubleGaugeRecord() throws Exception { public void testDoubleGaugeRecord() throws Exception {
AtomicReference<DoubleWithAttributes> attrs = new AtomicReference<>(); AtomicReference<DoubleWithAttributes> attrs = new AtomicReference<>();
DoubleGauge gauge = registry.registerDoubleGauge("name", "desc", "unit", attrs::get); DoubleGauge gauge = registry.registerDoubleGauge("es.test.name.total", "desc", "unit", attrs::get);
attrs.set(new DoubleWithAttributes(1.0d, Map.of("k", 1L))); attrs.set(new DoubleWithAttributes(1.0d, Map.of("k", 1L)));
@ -102,12 +102,17 @@ public class GaugeAdapterTests extends ESTestCase {
} }
public void testNullGaugeRecord() throws Exception { public void testNullGaugeRecord() throws Exception {
DoubleGauge dgauge = registry.registerDoubleGauge("name", "desc", "unit", new AtomicReference<DoubleWithAttributes>()::get); DoubleGauge dgauge = registry.registerDoubleGauge(
"es.test.name.total",
"desc",
"unit",
new AtomicReference<DoubleWithAttributes>()::get
);
otelMeter.collectMetrics(); otelMeter.collectMetrics();
List<Measurement> metrics = otelMeter.getRecorder().getMeasurements(dgauge); List<Measurement> metrics = otelMeter.getRecorder().getMeasurements(dgauge);
assertThat(metrics, hasSize(0)); assertThat(metrics, hasSize(0));
LongGauge lgauge = registry.registerLongGauge("name", "desc", "unit", new AtomicReference<LongWithAttributes>()::get); LongGauge lgauge = registry.registerLongGauge("es.test.name.total", "desc", "unit", new AtomicReference<LongWithAttributes>()::get);
otelMeter.collectMetrics(); otelMeter.collectMetrics();
metrics = otelMeter.getRecorder().getMeasurements(lgauge); metrics = otelMeter.getRecorder().getMeasurements(lgauge);
assertThat(metrics, hasSize(0)); assertThat(metrics, hasSize(0));

50
modules/repository-s3/src/internalClusterTest/java/org/elasticsearch/repositories/s3/S3BlobStoreRepositoryMetricsTests.java
View file

@ -38,13 +38,13 @@ import java.util.Queue;
import java.util.concurrent.LinkedBlockingQueue; import java.util.concurrent.LinkedBlockingQueue;
import static org.elasticsearch.repositories.RepositoriesMetrics.HTTP_REQUEST_TIME_IN_MICROS_HISTOGRAM; import static org.elasticsearch.repositories.RepositoriesMetrics.HTTP_REQUEST_TIME_IN_MICROS_HISTOGRAM;
import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_EXCEPTIONS_COUNT;
import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_EXCEPTIONS_HISTOGRAM; import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_EXCEPTIONS_HISTOGRAM;
import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_OPERATIONS_COUNT; import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_EXCEPTIONS_TOTAL;
import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_REQUESTS_COUNT; import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_OPERATIONS_TOTAL;
import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_THROTTLES_COUNT; import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_REQUESTS_TOTAL;
import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_THROTTLES_HISTOGRAM; import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_THROTTLES_HISTOGRAM;
import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_UNSUCCESSFUL_OPERATIONS_COUNT; import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_THROTTLES_TOTAL;
import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_UNSUCCESSFUL_OPERATIONS_TOTAL;
import static org.elasticsearch.rest.RestStatus.INTERNAL_SERVER_ERROR; import static org.elasticsearch.rest.RestStatus.INTERNAL_SERVER_ERROR;
import static org.elasticsearch.rest.RestStatus.NOT_FOUND; import static org.elasticsearch.rest.RestStatus.NOT_FOUND;
import static org.elasticsearch.rest.RestStatus.TOO_MANY_REQUESTS; import static org.elasticsearch.rest.RestStatus.TOO_MANY_REQUESTS;
@ -104,11 +104,11 @@ public class S3BlobStoreRepositoryMetricsTests extends S3BlobStoreRepositoryTest
final long batch = i + 1; final long batch = i + 1;
addErrorStatus(INTERNAL_SERVER_ERROR, TOO_MANY_REQUESTS, TOO_MANY_REQUESTS); addErrorStatus(INTERNAL_SERVER_ERROR, TOO_MANY_REQUESTS, TOO_MANY_REQUESTS);
blobContainer.writeBlob(purpose, blobName, new BytesArray("blob"), false); blobContainer.writeBlob(purpose, blobName, new BytesArray("blob"), false);
assertThat(getLongCounterValue(plugin, METRIC_REQUESTS_COUNT, Operation.PUT_OBJECT), equalTo(4L * batch)); assertThat(getLongCounterValue(plugin, METRIC_REQUESTS_TOTAL, Operation.PUT_OBJECT), equalTo(4L * batch));
assertThat(getLongCounterValue(plugin, METRIC_OPERATIONS_COUNT, Operation.PUT_OBJECT), equalTo(batch)); assertThat(getLongCounterValue(plugin, METRIC_OPERATIONS_TOTAL, Operation.PUT_OBJECT), equalTo(batch));
assertThat(getLongCounterValue(plugin, METRIC_UNSUCCESSFUL_OPERATIONS_COUNT, Operation.PUT_OBJECT), equalTo(0L)); assertThat(getLongCounterValue(plugin, METRIC_UNSUCCESSFUL_OPERATIONS_TOTAL, Operation.PUT_OBJECT), equalTo(0L));
assertThat(getLongCounterValue(plugin, METRIC_EXCEPTIONS_COUNT, Operation.PUT_OBJECT), equalTo(batch)); assertThat(getLongCounterValue(plugin, METRIC_EXCEPTIONS_TOTAL, Operation.PUT_OBJECT), equalTo(batch));
assertThat(getLongCounterValue(plugin, METRIC_THROTTLES_COUNT, Operation.PUT_OBJECT), equalTo(2L * batch)); assertThat(getLongCounterValue(plugin, METRIC_THROTTLES_TOTAL, Operation.PUT_OBJECT), equalTo(2L * batch));
assertThat(getLongHistogramValue(plugin, METRIC_EXCEPTIONS_HISTOGRAM, Operation.PUT_OBJECT), equalTo(batch)); assertThat(getLongHistogramValue(plugin, METRIC_EXCEPTIONS_HISTOGRAM, Operation.PUT_OBJECT), equalTo(batch));
assertThat(getLongHistogramValue(plugin, METRIC_THROTTLES_HISTOGRAM, Operation.PUT_OBJECT), equalTo(2L * batch)); assertThat(getLongHistogramValue(plugin, METRIC_THROTTLES_HISTOGRAM, Operation.PUT_OBJECT), equalTo(2L * batch));
assertThat(getNumberOfMeasurements(plugin, HTTP_REQUEST_TIME_IN_MICROS_HISTOGRAM, Operation.PUT_OBJECT), equalTo(batch)); assertThat(getNumberOfMeasurements(plugin, HTTP_REQUEST_TIME_IN_MICROS_HISTOGRAM, Operation.PUT_OBJECT), equalTo(batch));
@ -124,11 +124,11 @@ public class S3BlobStoreRepositoryMetricsTests extends S3BlobStoreRepositoryTest
} catch (Exception e) { } catch (Exception e) {
// intentional failure // intentional failure
} }
assertThat(getLongCounterValue(plugin, METRIC_REQUESTS_COUNT, Operation.GET_OBJECT), equalTo(2L * batch)); assertThat(getLongCounterValue(plugin, METRIC_REQUESTS_TOTAL, Operation.GET_OBJECT), equalTo(2L * batch));
assertThat(getLongCounterValue(plugin, METRIC_OPERATIONS_COUNT, Operation.GET_OBJECT), equalTo(batch)); assertThat(getLongCounterValue(plugin, METRIC_OPERATIONS_TOTAL, Operation.GET_OBJECT), equalTo(batch));
assertThat(getLongCounterValue(plugin, METRIC_UNSUCCESSFUL_OPERATIONS_COUNT, Operation.GET_OBJECT), equalTo(batch)); assertThat(getLongCounterValue(plugin, METRIC_UNSUCCESSFUL_OPERATIONS_TOTAL, Operation.GET_OBJECT), equalTo(batch));
assertThat(getLongCounterValue(plugin, METRIC_EXCEPTIONS_COUNT, Operation.GET_OBJECT), equalTo(batch)); assertThat(getLongCounterValue(plugin, METRIC_EXCEPTIONS_TOTAL, Operation.GET_OBJECT), equalTo(batch));
assertThat(getLongCounterValue(plugin, METRIC_THROTTLES_COUNT, Operation.GET_OBJECT), equalTo(batch)); assertThat(getLongCounterValue(plugin, METRIC_THROTTLES_TOTAL, Operation.GET_OBJECT), equalTo(batch));
assertThat(getLongHistogramValue(plugin, METRIC_EXCEPTIONS_HISTOGRAM, Operation.GET_OBJECT), equalTo(batch)); assertThat(getLongHistogramValue(plugin, METRIC_EXCEPTIONS_HISTOGRAM, Operation.GET_OBJECT), equalTo(batch));
assertThat(getLongHistogramValue(plugin, METRIC_THROTTLES_HISTOGRAM, Operation.GET_OBJECT), equalTo(batch)); assertThat(getLongHistogramValue(plugin, METRIC_THROTTLES_HISTOGRAM, Operation.GET_OBJECT), equalTo(batch));
assertThat(getNumberOfMeasurements(plugin, HTTP_REQUEST_TIME_IN_MICROS_HISTOGRAM, Operation.GET_OBJECT), equalTo(batch)); assertThat(getNumberOfMeasurements(plugin, HTTP_REQUEST_TIME_IN_MICROS_HISTOGRAM, Operation.GET_OBJECT), equalTo(batch));
@ -144,11 +144,11 @@ public class S3BlobStoreRepositoryMetricsTests extends S3BlobStoreRepositoryTest
} catch (Exception e) { } catch (Exception e) {
// intentional failure // intentional failure
} }
assertThat(getLongCounterValue(plugin, METRIC_REQUESTS_COUNT, Operation.LIST_OBJECTS), equalTo(5L * batch)); assertThat(getLongCounterValue(plugin, METRIC_REQUESTS_TOTAL, Operation.LIST_OBJECTS), equalTo(5L * batch));
assertThat(getLongCounterValue(plugin, METRIC_OPERATIONS_COUNT, Operation.LIST_OBJECTS), equalTo(batch)); assertThat(getLongCounterValue(plugin, METRIC_OPERATIONS_TOTAL, Operation.LIST_OBJECTS), equalTo(batch));
assertThat(getLongCounterValue(plugin, METRIC_UNSUCCESSFUL_OPERATIONS_COUNT, Operation.LIST_OBJECTS), equalTo(batch)); assertThat(getLongCounterValue(plugin, METRIC_UNSUCCESSFUL_OPERATIONS_TOTAL, Operation.LIST_OBJECTS), equalTo(batch));
assertThat(getLongCounterValue(plugin, METRIC_EXCEPTIONS_COUNT, Operation.LIST_OBJECTS), equalTo(batch)); assertThat(getLongCounterValue(plugin, METRIC_EXCEPTIONS_TOTAL, Operation.LIST_OBJECTS), equalTo(batch));
assertThat(getLongCounterValue(plugin, METRIC_THROTTLES_COUNT, Operation.LIST_OBJECTS), equalTo(5L * batch)); assertThat(getLongCounterValue(plugin, METRIC_THROTTLES_TOTAL, Operation.LIST_OBJECTS), equalTo(5L * batch));
assertThat(getLongHistogramValue(plugin, METRIC_EXCEPTIONS_HISTOGRAM, Operation.LIST_OBJECTS), equalTo(batch)); assertThat(getLongHistogramValue(plugin, METRIC_EXCEPTIONS_HISTOGRAM, Operation.LIST_OBJECTS), equalTo(batch));
assertThat(getLongHistogramValue(plugin, METRIC_THROTTLES_HISTOGRAM, Operation.LIST_OBJECTS), equalTo(5L * batch)); assertThat(getLongHistogramValue(plugin, METRIC_THROTTLES_HISTOGRAM, Operation.LIST_OBJECTS), equalTo(5L * batch));
assertThat(getNumberOfMeasurements(plugin, HTTP_REQUEST_TIME_IN_MICROS_HISTOGRAM, Operation.LIST_OBJECTS), equalTo(batch)); assertThat(getNumberOfMeasurements(plugin, HTTP_REQUEST_TIME_IN_MICROS_HISTOGRAM, Operation.LIST_OBJECTS), equalTo(batch));
@ -156,11 +156,11 @@ public class S3BlobStoreRepositoryMetricsTests extends S3BlobStoreRepositoryTest
// Delete to clean up // Delete to clean up
blobContainer.deleteBlobsIgnoringIfNotExists(purpose, Iterators.single(blobName)); blobContainer.deleteBlobsIgnoringIfNotExists(purpose, Iterators.single(blobName));
assertThat(getLongCounterValue(plugin, METRIC_REQUESTS_COUNT, Operation.DELETE_OBJECTS), equalTo(1L)); assertThat(getLongCounterValue(plugin, METRIC_REQUESTS_TOTAL, Operation.DELETE_OBJECTS), equalTo(1L));
assertThat(getLongCounterValue(plugin, METRIC_OPERATIONS_COUNT, Operation.DELETE_OBJECTS), equalTo(1L)); assertThat(getLongCounterValue(plugin, METRIC_OPERATIONS_TOTAL, Operation.DELETE_OBJECTS), equalTo(1L));
assertThat(getLongCounterValue(plugin, METRIC_UNSUCCESSFUL_OPERATIONS_COUNT, Operation.DELETE_OBJECTS), equalTo(0L)); assertThat(getLongCounterValue(plugin, METRIC_UNSUCCESSFUL_OPERATIONS_TOTAL, Operation.DELETE_OBJECTS), equalTo(0L));
assertThat(getLongCounterValue(plugin, METRIC_EXCEPTIONS_COUNT, Operation.DELETE_OBJECTS), equalTo(0L)); assertThat(getLongCounterValue(plugin, METRIC_EXCEPTIONS_TOTAL, Operation.DELETE_OBJECTS), equalTo(0L));
assertThat(getLongCounterValue(plugin, METRIC_THROTTLES_COUNT, Operation.DELETE_OBJECTS), equalTo(0L)); assertThat(getLongCounterValue(plugin, METRIC_THROTTLES_TOTAL, Operation.DELETE_OBJECTS), equalTo(0L));
assertThat(getLongHistogramValue(plugin, METRIC_EXCEPTIONS_HISTOGRAM, Operation.DELETE_OBJECTS), equalTo(0L)); assertThat(getLongHistogramValue(plugin, METRIC_EXCEPTIONS_HISTOGRAM, Operation.DELETE_OBJECTS), equalTo(0L));
assertThat(getLongHistogramValue(plugin, METRIC_THROTTLES_HISTOGRAM, Operation.DELETE_OBJECTS), equalTo(0L)); assertThat(getLongHistogramValue(plugin, METRIC_THROTTLES_HISTOGRAM, Operation.DELETE_OBJECTS), equalTo(0L));
assertThat(getNumberOfMeasurements(plugin, HTTP_REQUEST_TIME_IN_MICROS_HISTOGRAM, Operation.DELETE_OBJECTS), equalTo(1L)); assertThat(getNumberOfMeasurements(plugin, HTTP_REQUEST_TIME_IN_MICROS_HISTOGRAM, Operation.DELETE_OBJECTS), equalTo(1L));

4
modules/repository-s3/src/internalClusterTest/java/org/elasticsearch/repositories/s3/S3BlobStoreRepositoryTests.java
View file

@ -75,7 +75,7 @@ import java.util.concurrent.atomic.AtomicBoolean;
import java.util.stream.Collectors; import java.util.stream.Collectors;
import java.util.stream.StreamSupport; import java.util.stream.StreamSupport;
import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_REQUESTS_COUNT; import static org.elasticsearch.repositories.RepositoriesMetrics.METRIC_REQUESTS_TOTAL;
import static org.elasticsearch.repositories.blobstore.BlobStoreTestUtil.randomNonDataPurpose; import static org.elasticsearch.repositories.blobstore.BlobStoreTestUtil.randomNonDataPurpose;
import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertAcked; import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertAcked;
import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertHitCount; import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertHitCount;
@ -267,7 +267,7 @@ public class S3BlobStoreRepositoryTests extends ESMockAPIBasedRepositoryIntegTes
.filterPlugins(TestTelemetryPlugin.class) .filterPlugins(TestTelemetryPlugin.class)
.toList(); .toList();
assertThat(plugins, hasSize(1)); assertThat(plugins, hasSize(1));
final List<Measurement> metrics = Measurement.combine(plugins.get(0).getLongCounterMeasurement(METRIC_REQUESTS_COUNT)); final List<Measurement> metrics = Measurement.combine(plugins.get(0).getLongCounterMeasurement(METRIC_REQUESTS_TOTAL));
assertThat( assertThat(
statsCollectors.keySet().stream().map(S3BlobStore.StatsKey::operation).collect(Collectors.toSet()), statsCollectors.keySet().stream().map(S3BlobStore.StatsKey::operation).collect(Collectors.toSet()),

14
server/src/main/java/org/elasticsearch/cluster/routing/allocation/allocator/DesiredBalanceReconciler.java
View file

@ -98,24 +98,24 @@ public class DesiredBalanceReconciler {
unassignedShards = LongGaugeMetric.create( unassignedShards = LongGaugeMetric.create(
meterRegistry, meterRegistry,
"es.allocator.desired_balance.shards.unassigned", "es.allocator.desired_balance.shards.unassigned.current",
"Unassigned shards count", "Current number of unassigned shards",
"{shard}" "{shard}"
); );
totalAllocations = LongGaugeMetric.create( totalAllocations = LongGaugeMetric.create(
meterRegistry, meterRegistry,
"es.allocator.desired_balance.shards.count", "es.allocator.desired_balance.shards.current",
"Total shards count", "Total number of shards",
"{shard}" "{shard}"
); );
undesiredAllocations = LongGaugeMetric.create( undesiredAllocations = LongGaugeMetric.create(
meterRegistry, meterRegistry,
"es.allocator.desired_balance.allocations.undesired", "es.allocator.desired_balance.allocations.undesired.current",
"Count of shards allocated on undesired nodes", "Total number of shards allocated on undesired nodes",
"{shard}" "{shard}"
); );
undesiredAllocationsRatio = meterRegistry.registerDoubleGauge( undesiredAllocationsRatio = meterRegistry.registerDoubleGauge(
"es.allocator.desired_balance.allocations.undesired_ratio", "es.allocator.desired_balance.allocations.undesired.ratio",
"Ratio of undesired allocations to shard count", "Ratio of undesired allocations to shard count",
"1", "1",
() -> { () -> {

5
server/src/main/java/org/elasticsearch/health/HealthPeriodicLogger.java
View file

@ -204,7 +204,10 @@ public class HealthPeriodicLogger implements ClusterStateListener, Closeable, Sc
this.logWriter = logWriter == null ? logger::info : logWriter; this.logWriter = logWriter == null ? logger::info : logWriter;
// create metric for overall level metrics // create metric for overall level metrics
this.redMetrics.put("overall", LongGaugeMetric.create(this.meterRegistry, "es.health.overall.red", "Overall: Red", "{cluster}")); this.redMetrics.put(
"overall",
LongGaugeMetric.create(this.meterRegistry, "es.health.overall.red.status", "Overall: Red", "{cluster}")
);
} }
private void registerListeners() { private void registerListeners() {

12
server/src/main/java/org/elasticsearch/monitor/metrics/NodeMetrics.java
View file

@ -116,7 +116,7 @@ public class NodeMetrics extends AbstractLifecycleComponent {
metrics.add( metrics.add(
registry.registerLongGauge( registry.registerLongGauge(
"es.translog.operations.count", "es.translog.operations.total",
"Number of transaction log operations.", "Number of transaction log operations.",
"operation", "operation",
() -> new LongWithAttributes(stats.getOrRefresh().getIndices().getTranslog().estimatedNumberOfOperations()) () -> new LongWithAttributes(stats.getOrRefresh().getIndices().getTranslog().estimatedNumberOfOperations())
@ -134,7 +134,7 @@ public class NodeMetrics extends AbstractLifecycleComponent {
metrics.add( metrics.add(
registry.registerLongGauge( registry.registerLongGauge(
"es.translog.uncommitted_operations.count", "es.translog.uncommitted_operations.total",
"Number of uncommitted transaction log operations.", "Number of uncommitted transaction log operations.",
"operations", "operations",
() -> new LongWithAttributes(stats.getOrRefresh().getIndices().getTranslog().getUncommittedOperations()) () -> new LongWithAttributes(stats.getOrRefresh().getIndices().getTranslog().getUncommittedOperations())
@ -224,7 +224,7 @@ public class NodeMetrics extends AbstractLifecycleComponent {
metrics.add( metrics.add(
registry.registerLongGauge( registry.registerLongGauge(
"es.indexing.docs.count", "es.indexing.docs.total",
"Current number of indexing documents", "Current number of indexing documents",
"documents", "documents",
() -> new LongWithAttributes(stats.getOrRefresh().getIndices().getIndexing().getTotal().getIndexCurrent()) () -> new LongWithAttributes(stats.getOrRefresh().getIndices().getIndexing().getTotal().getIndexCurrent())
@ -251,7 +251,7 @@ public class NodeMetrics extends AbstractLifecycleComponent {
metrics.add( metrics.add(
registry.registerLongGauge( registry.registerLongGauge(
"es.indices.deletion.docs.count", "es.indices.deletion.docs.total",
"Current number of deleting documents", "Current number of deleting documents",
"documents", "documents",
() -> new LongWithAttributes(stats.getOrRefresh().getIndices().getIndexing().getTotal().getDeleteCurrent()) () -> new LongWithAttributes(stats.getOrRefresh().getIndices().getIndexing().getTotal().getDeleteCurrent())
@ -323,7 +323,7 @@ public class NodeMetrics extends AbstractLifecycleComponent {
metrics.add( metrics.add(
registry.registerLongGauge( registry.registerLongGauge(
"es.indexing.coordinating_operations.count", "es.indexing.coordinating_operations.total",
"Current number of coordinating operations", "Current number of coordinating operations",
"operations", "operations",
() -> new LongWithAttributes(stats.getOrRefresh().getIndexingPressureStats().getCurrentCoordinatingOps()) () -> new LongWithAttributes(stats.getOrRefresh().getIndexingPressureStats().getCurrentCoordinatingOps())
@ -368,7 +368,7 @@ public class NodeMetrics extends AbstractLifecycleComponent {
metrics.add( metrics.add(
registry.registerLongGauge( registry.registerLongGauge(
"es.indexing.primary_operations.count", "es.indexing.primary_operations.total",
"Current number of primary operations", "Current number of primary operations",
"operations", "operations",
() -> new LongWithAttributes(stats.getOrRefresh().getIndexingPressureStats().getCurrentPrimaryOps()) () -> new LongWithAttributes(stats.getOrRefresh().getIndexingPressureStats().getCurrentPrimaryOps())

20
server/src/main/java/org/elasticsearch/repositories/RepositoriesMetrics.java
View file

@ -25,22 +25,22 @@ public record RepositoriesMetrics(
public static RepositoriesMetrics NOOP = new RepositoriesMetrics(MeterRegistry.NOOP); public static RepositoriesMetrics NOOP = new RepositoriesMetrics(MeterRegistry.NOOP);
public static final String METRIC_REQUESTS_COUNT = "es.repositories.requests.count"; public static final String METRIC_REQUESTS_TOTAL = "es.repositories.requests.total";
public static final String METRIC_EXCEPTIONS_COUNT = "es.repositories.exceptions.count"; public static final String METRIC_EXCEPTIONS_TOTAL = "es.repositories.exceptions.total";
public static final String METRIC_THROTTLES_COUNT = "es.repositories.throttles.count"; public static final String METRIC_THROTTLES_TOTAL = "es.repositories.throttles.total";
public static final String METRIC_OPERATIONS_COUNT = "es.repositories.operations.count"; public static final String METRIC_OPERATIONS_TOTAL = "es.repositories.operations.total";
public static final String METRIC_UNSUCCESSFUL_OPERATIONS_COUNT = "es.repositories.operations.unsuccessful.count"; public static final String METRIC_UNSUCCESSFUL_OPERATIONS_TOTAL = "es.repositories.operations.unsuccessful.total";
public static final String METRIC_EXCEPTIONS_HISTOGRAM = "es.repositories.exceptions.histogram"; public static final String METRIC_EXCEPTIONS_HISTOGRAM = "es.repositories.exceptions.histogram";
public static final String METRIC_THROTTLES_HISTOGRAM = "es.repositories.throttles.histogram"; public static final String METRIC_THROTTLES_HISTOGRAM = "es.repositories.throttles.histogram";
public static final String HTTP_REQUEST_TIME_IN_MICROS_HISTOGRAM = "es.repositories.requests.http_request_time.histogram"; public static final String HTTP_REQUEST_TIME_IN_MICROS_HISTOGRAM = "es.repositories.requests.http_request_time.histogram";
public RepositoriesMetrics(MeterRegistry meterRegistry) { public RepositoriesMetrics(MeterRegistry meterRegistry) {
this( this(
meterRegistry.registerLongCounter(METRIC_REQUESTS_COUNT, "repository request counter", "unit"), meterRegistry.registerLongCounter(METRIC_REQUESTS_TOTAL, "repository request counter", "unit"),
meterRegistry.registerLongCounter(METRIC_EXCEPTIONS_COUNT, "repository request exception counter", "unit"), meterRegistry.registerLongCounter(METRIC_EXCEPTIONS_TOTAL, "repository request exception counter", "unit"),
meterRegistry.registerLongCounter(METRIC_THROTTLES_COUNT, "repository request throttle counter", "unit"), meterRegistry.registerLongCounter(METRIC_THROTTLES_TOTAL, "repository request throttle counter", "unit"),
meterRegistry.registerLongCounter(METRIC_OPERATIONS_COUNT, "repository operation counter", "unit"), meterRegistry.registerLongCounter(METRIC_OPERATIONS_TOTAL, "repository operation counter", "unit"),
meterRegistry.registerLongCounter(METRIC_UNSUCCESSFUL_OPERATIONS_COUNT, "repository unsuccessful operation counter", "unit"), meterRegistry.registerLongCounter(METRIC_UNSUCCESSFUL_OPERATIONS_TOTAL, "repository unsuccessful operation counter", "unit"),
meterRegistry.registerLongHistogram(METRIC_EXCEPTIONS_HISTOGRAM, "repository request exception histogram", "unit"), meterRegistry.registerLongHistogram(METRIC_EXCEPTIONS_HISTOGRAM, "repository request exception histogram", "unit"),
meterRegistry.registerLongHistogram(METRIC_THROTTLES_HISTOGRAM, "repository request throttle histogram", "unit"), meterRegistry.registerLongHistogram(METRIC_THROTTLES_HISTOGRAM, "repository request throttle histogram", "unit"),
meterRegistry.registerLongHistogram( meterRegistry.registerLongHistogram(

16
test/external-modules/apm-integration/src/javaRestTest/java/org/elasticsearch/test/apmintegration/MetricsApmIT.java
View file

@ -60,19 +60,19 @@ public class MetricsApmIT extends ESRestTestCase {
public void testApmIntegration() throws Exception { public void testApmIntegration() throws Exception {
Map<String, Predicate<Map<String, Object>>> sampleAssertions = new HashMap<>( Map<String, Predicate<Map<String, Object>>> sampleAssertions = new HashMap<>(
Map.ofEntries( Map.ofEntries(
assertion(TestMeterUsages.VERY_LONG_NAME, m -> (Double) m.get("value"), closeTo(1.0, 0.001)), assertion("es.test.long_counter.total", m -> (Double) m.get("value"), closeTo(1.0, 0.001)),
assertion("testLongCounter", m -> (Double) m.get("value"), closeTo(1.0, 0.001)), assertion("es.test.double_counter.total", m -> (Double) m.get("value"), closeTo(1.0, 0.001)),
assertion("testAsyncDoubleCounter", m -> (Double) m.get("value"), closeTo(1.0, 0.001)), assertion("es.test.async_double_counter.total", m -> (Double) m.get("value"), closeTo(1.0, 0.001)),
assertion("testAsyncLongCounter", m -> (Integer) m.get("value"), equalTo(1)), assertion("es.test.async_long_counter.total", m -> (Integer) m.get("value"), equalTo(1)),
assertion("testDoubleGauge", m -> (Double) m.get("value"), closeTo(1.0, 0.001)), assertion("es.test.double_gauge.current", m -> (Double) m.get("value"), closeTo(1.0, 0.001)),
assertion("testLongGauge", m -> (Integer) m.get("value"), equalTo(1)), assertion("es.test.long_gauge.current", m -> (Integer) m.get("value"), equalTo(1)),
assertion( assertion(
"testDoubleHistogram", "es.test.double_histogram.histogram",
m -> ((Collection<Integer>) m.get("counts")).stream().mapToInt(Integer::intValue).sum(), m -> ((Collection<Integer>) m.get("counts")).stream().mapToInt(Integer::intValue).sum(),
equalTo(2) equalTo(2)
), ),
assertion( assertion(
"testLongHistogram", "es.test.long_histogram.histogram",
m -> ((Collection<Integer>) m.get("counts")).stream().mapToInt(Integer::intValue).sum(), m -> ((Collection<Integer>) m.get("counts")).stream().mapToInt(Integer::intValue).sum(),
equalTo(2) equalTo(2)
) )

17
test/external-modules/apm-integration/src/main/java/org/elasticsearch/test/apmintegration/TestMeterUsages.java
View file

@ -26,18 +26,17 @@ public class TestMeterUsages {
private final LongHistogram longHistogram; private final LongHistogram longHistogram;
private final AtomicReference<DoubleWithAttributes> doubleWithAttributes = new AtomicReference<>(); private final AtomicReference<DoubleWithAttributes> doubleWithAttributes = new AtomicReference<>();
private final AtomicReference<LongWithAttributes> longWithAttributes = new AtomicReference<>(); private final AtomicReference<LongWithAttributes> longWithAttributes = new AtomicReference<>();
public static String VERY_LONG_NAME = "a1234567890123456789012345678901234567890123456789012345678901234567890";
public TestMeterUsages(MeterRegistry meterRegistry) { public TestMeterUsages(MeterRegistry meterRegistry) {
this.doubleCounter = meterRegistry.registerDoubleCounter(VERY_LONG_NAME, "test", "unit"); this.doubleCounter = meterRegistry.registerDoubleCounter("es.test.long_counter.total", "test", "unit");
this.longCounter = meterRegistry.registerDoubleCounter("testLongCounter", "test", "unit"); this.longCounter = meterRegistry.registerDoubleCounter("es.test.double_counter.total", "test", "unit");
this.doubleHistogram = meterRegistry.registerDoubleHistogram("testDoubleHistogram", "test", "unit"); this.doubleHistogram = meterRegistry.registerDoubleHistogram("es.test.double_histogram.histogram", "test", "unit");
this.longHistogram = meterRegistry.registerLongHistogram("testLongHistogram", "test", "unit"); this.longHistogram = meterRegistry.registerLongHistogram("es.test.long_histogram.histogram", "test", "unit");
meterRegistry.registerDoubleGauge("testDoubleGauge", "test", "unit", doubleWithAttributes::get); meterRegistry.registerDoubleGauge("es.test.double_gauge.current", "test", "unit", doubleWithAttributes::get);
meterRegistry.registerLongGauge("testLongGauge", "test", "unit", longWithAttributes::get); meterRegistry.registerLongGauge("es.test.long_gauge.current", "test", "unit", longWithAttributes::get);
meterRegistry.registerLongAsyncCounter("testAsyncLongCounter", "test", "unit", longWithAttributes::get); meterRegistry.registerLongAsyncCounter("es.test.async_long_counter.total", "test", "unit", longWithAttributes::get);
meterRegistry.registerDoubleAsyncCounter("testAsyncDoubleCounter", "test", "unit", doubleWithAttributes::get); meterRegistry.registerDoubleAsyncCounter("es.test.async_double_counter.total", "test", "unit", doubleWithAttributes::get);
} }
public void testUponRequest() { public void testUponRequest() {

320
x Normal file
View file

@ -0,0 +1,320 @@
➜ elasticsearch git:(metric_name_validation) ./gradlew run -Dtests.es.logger.org.elasticsearch.telemetry.apm=debug
> Configure project :x-pack:plugin:searchable-snapshots:qa:hdfs
hdfsFixture unsupported, please set HADOOP_HOME and put HADOOP_HOME\bin in PATH
=======================================
Elasticsearch Build Hamster says Hello!
Gradle Version : 8.5
OS Info : Mac OS X 14.1.2 (aarch64)
JDK Version : 17.0.2+8-LTS-86 (Oracle)
JAVA_HOME : /Library/Java/JavaVirtualMachines/jdk-17.0.2.jdk/Contents/Home
Random Testing Seed : B705DEF03AA4BF36
In FIPS 140 mode : false
=======================================
> Task :run
[2023-12-18T13:31:10.882688Z] [BUILD] Copying additional config files from distro [/Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.13.0-SNAPSHOT/config/jvm.options.d, /Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.13.0-SNAPSHOT/config/users_roles, /Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.13.0-SNAPSHOT/config/role_mapping.yml, /Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.13.0-SNAPSHOT/config/users, /Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.13.0-SNAPSHOT/config/elasticsearch.yml, /Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.13.0-SNAPSHOT/config/roles.yml, /Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.13.0-SNAPSHOT/config/log4j2.properties, /Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.13.0-SNAPSHOT/config/elasticsearch-plugins.example.yml, /Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.13.0-SNAPSHOT/config/jvm.options]
[2023-12-18T13:31:10.885813Z] [BUILD] Creating elasticsearch keystore with password set to []
[2023-12-18T13:31:11.630988Z] [BUILD] Adding 1 keystore settings and 0 keystore files
[2023-12-18T13:31:12.293546Z] [BUILD] Installing 0 modules
[2023-12-18T13:31:12.293769Z] [BUILD] Setting up 1 users
[2023-12-18T13:31:13.096132Z] [BUILD] Setting up roles.yml
[2023-12-18T13:31:13.096494Z] [BUILD] Starting Elasticsearch process
CompileCommand: exclude org/apache/lucene/util/MSBRadixSorter.computeCommonPrefixLengthAndBuildHistogram bool exclude = true
CompileCommand: exclude org/apache/lucene/util/RadixSelector.computeCommonPrefixLengthAndBuildHistogram bool exclude = true
Dec 18, 2023 2:31:14 PM sun.util.locale.provider.LocaleProviderAdapter <clinit>
WARNING: COMPAT locale provider will be removed in a future release
[2023-12-18T14:31:15,025][INFO ][o.a.l.i.v.PanamaVectorizationProvider] [runTask-0] Java vector incubator API enabled; uses preferredBitSize=128
[2023-12-18T14:31:15,437][INFO ][o.e.n.Node ] [runTask-0] version[8.13.0-SNAPSHOT], pid[43801], build[tar/25d9bbbb5327023f7f1896fb3044fb4c9d231342/2023-12-18T13:27:39.871866Z], OS[Mac OS X/14.1.2/aarch64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/21/21+35-2513]
[2023-12-18T14:31:15,438][INFO ][o.e.n.Node ] [runTask-0] JVM home [/Users/przemyslawgomulka/.gradle/jdks/oracle_corporation-21-aarch64-os_x/jdk-21.jdk/Contents/Home], using bundled JDK [false]
[2023-12-18T14:31:15,438][INFO ][o.e.n.Node ] [runTask-0] JVM arguments [-Des.networkaddress.cache.ttl=60, -Des.networkaddress.cache.negative.ttl=10, -Djava.security.manager=allow, -XX:+AlwaysPreTouch, -Xss1m, -Djava.awt.headless=true, -Dfile.encoding=UTF-8, -Djna.nosys=true, -XX:-OmitStackTraceInFastThrow, -Dio.netty.noUnsafe=true, -Dio.netty.noKeySetOptimization=true, -Dio.netty.recycler.maxCapacityPerThread=0, -Dlog4j.shutdownHookEnabled=false, -Dlog4j2.disable.jmx=true, -Dlog4j2.formatMsgNoLookups=true, -Djava.locale.providers=SPI,COMPAT, --add-opens=java.base/java.io=org.elasticsearch.preallocate, -Des.distribution.type=tar, -XX:+UseG1GC, -Djava.io.tmpdir=/Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/build/testclusters/runTask-0/tmp, --add-modules=jdk.incubator.vector, -XX:CompileCommand=exclude,org.apache.lucene.util.MSBRadixSorter::computeCommonPrefixLengthAndBuildHistogram, -XX:CompileCommand=exclude,org.apache.lucene.util.RadixSelector::computeCommonPrefixLengthAndBuildHistogram, -XX:+HeapDumpOnOutOfMemoryError, -XX:+ExitOnOutOfMemoryError, -XX:HeapDumpPath=/Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/build/testclusters/runTask-0/logs, -XX:ErrorFile=/Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/build/testclusters/runTask-0/logs/hs_err_pid%p.log, -Xlog:gc*,gc+age=trace,safepoint:file=/Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/build/testclusters/runTask-0/logs/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m, -Xms512m, -Xmx512m, -ea, -esa, -Dingest.geoip.downloader.enabled.default=true, -Dio.netty.leakDetection.level=paranoid, -XX:MaxDirectMemorySize=268435456, -XX:G1HeapRegionSize=4m, -XX:InitiatingHeapOccupancyPercent=30, -XX:G1ReservePercent=15, --module-path=/Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/distribution/archives/darwin-aarch64-tar/build/install/elasticsearch-8.13.0-SNAPSHOT/lib, --add-modules=jdk.net, --add-modules=ALL-MODULE-PATH, -Djdk.module.main=org.elasticsearch.server]
[2023-12-18T14:31:15,438][WARN ][o.e.n.Node ] [runTask-0] version [8.13.0-SNAPSHOT] is a pre-release version of Elasticsearch and is not suitable for production
[2023-12-18T14:31:17,172][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [repository-url]
[2023-12-18T14:31:17,172][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [rest-root]
[2023-12-18T14:31:17,172][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-core]
[2023-12-18T14:31:17,172][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-redact]
[2023-12-18T14:31:17,173][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [ingest-user-agent]
[2023-12-18T14:31:17,173][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-async-search]
[2023-12-18T14:31:17,173][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [test-error-query]
[2023-12-18T14:31:17,173][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-monitoring]
[2023-12-18T14:31:17,173][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [repository-s3]
[2023-12-18T14:31:17,173][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-analytics]
[2023-12-18T14:31:17,173][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-ent-search]
[2023-12-18T14:31:17,173][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-autoscaling]
[2023-12-18T14:31:17,173][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [lang-painless]
[2023-12-18T14:31:17,173][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-ml]
[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [test-die-with-dignity]
[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [legacy-geo]
[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [lang-mustache]
[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-ql]
[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [rank-rrf]
[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [analysis-common]
[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [test-seek-tracking-directory]
[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [health-shards-availability]
[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [transport-netty4]
[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [aggregations]
[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [ingest-common]
[2023-12-18T14:31:17,174][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-identity-provider]
[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [frozen-indices]
[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-text-structure]
[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-shutdown]
[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [snapshot-repo-test-kit]
[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [ml-package-loader]
[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [test-delayed-aggs]
[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [kibana]
[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [constant-keyword]
[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-logstash]
[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-graph]
[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-ccr]
[2023-12-18T14:31:17,175][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-esql]
[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [parent-join]
[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [counted-keyword]
[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-enrich]
[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [repositories-metering-api]
[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [transform]
[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [repository-azure]
[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [repository-gcs]
[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [spatial]
[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [apm]
[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [mapper-extras]
[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [mapper-version]
[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-rollup]
[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [percolator]
[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [data-streams]
[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-stack]
[2023-12-18T14:31:17,176][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [rank-eval]
[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [reindex]
[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [test-apm-integration]
[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-security]
[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [blob-cache]
[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [searchable-snapshots]
[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-slm]
[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [test-latency-simulating-directory]
[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [snapshot-based-recoveries]
[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-watcher]
[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [old-lucene-versions]
[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-ilm]
[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-inference]
[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-voting-only-node]
[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-deprecation]
[2023-12-18T14:31:17,177][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-fleet]
[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-profiling]
[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-aggregate-metric]
[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-downsample]
[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [ingest-geoip]
[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-write-load-forecaster]
[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [search-business-rules]
[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [wildcard]
[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [ingest-attachment]
[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-apm-data]
[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [unsigned-long]
[2023-12-18T14:31:17,178][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-sql]
[2023-12-18T14:31:17,179][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [runtime-fields-common]
[2023-12-18T14:31:17,179][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-async]
[2023-12-18T14:31:17,179][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [vector-tile]
[2023-12-18T14:31:17,179][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [lang-expression]
[2023-12-18T14:31:17,179][INFO ][o.e.p.PluginsService ] [runTask-0] loaded module [x-pack-eql]
[2023-12-18T14:31:17,224][INFO ][o.e.c.u.FeatureFlag ] [runTask-0] The current build is a snapshot, feature flag [failure_store] is enabled
[2023-12-18T14:31:17,520][INFO ][o.e.e.NodeEnvironment ] [runTask-0] using [1] data paths, mounts [[/System/Volumes/Data (/dev/disk3s5)]], net usable_space [636.9gb], net total_space [926.3gb], types [apfs]
[2023-12-18T14:31:17,520][INFO ][o.e.e.NodeEnvironment ] [runTask-0] heap size [512mb], compressed ordinary object pointers [true]
[2023-12-18T14:31:17,543][INFO ][o.e.n.Node ] [runTask-0] node name [runTask-0], node ID [RpeX_621SdCZSBh8-RSQvg], cluster name [runTask], roles [ingest, data_frozen, ml, data_hot, transform, data_content, data_warm, master, remote_cluster_client, data, data_cold]
[2023-12-18T14:31:19,294][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.allocator.desired_balance.shards.unassigned.count
[2023-12-18T14:31:19,294][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.allocator.desired_balance.shards.count
[2023-12-18T14:31:19,295][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.allocator.desired_balance.allocations.undesired.count
[2023-12-18T14:31:19,295][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.allocator.desired_balance.allocations.undesired.ratio
[2023-12-18T14:31:19,309][INFO ][o.e.c.u.FeatureFlag ] [runTask-0] The current build is a snapshot, feature flag [semantic_text] is enabled
[2023-12-18T14:31:19,326][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.breaker.parent.trip.total
[2023-12-18T14:31:19,326][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.breaker.field_data.trip.total
[2023-12-18T14:31:19,326][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.breaker.request.trip.total
[2023-12-18T14:31:19,326][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.breaker.in_flight_requests.trip.total
[2023-12-18T14:31:19,329][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.breaker.model_inference.trip.total
[2023-12-18T14:31:19,329][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.breaker.eql_sequence.trip.total
[2023-12-18T14:31:19,350][INFO ][o.e.f.FeatureService ] [runTask-0] Registered local node features [features_supported, health.dsl.info, usage.data_tiers.precalculate_stats]
[2023-12-18T14:31:19,497][INFO ][o.e.x.m.p.l.CppLogMessageHandler] [runTask-0] [controller/43802] [Main.cc@123] controller (64 bit): Version 8.13.0-SNAPSHOT (Build c9c232240dd04f) Copyright (c) 2023 Elasticsearch BV
[2023-12-18T14:31:19,659][INFO ][o.e.t.a.APM ] [runTask-0] Sending apm metrics is disabled
[2023-12-18T14:31:19,660][INFO ][o.e.t.a.APM ] [runTask-0] Sending apm tracing is disabled
[2023-12-18T14:31:19,670][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.test.long_counter.count
[2023-12-18T14:31:19,671][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.test.double_counter.count
[2023-12-18T14:31:19,671][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.test.double_histogram.histogram
[2023-12-18T14:31:19,671][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.test.long_histogram.histogram
[2023-12-18T14:31:19,672][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.test.double_gauge.total
[2023-12-18T14:31:19,672][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.test.long_gauge.total
[2023-12-18T14:31:19,672][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.test.async_long_counter.count
[2023-12-18T14:31:19,672][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.test.async_double_counter.count
[2023-12-18T14:31:19,672][INFO ][o.e.x.s.Security ] [runTask-0] Security is enabled
[2023-12-18T14:31:19,850][INFO ][o.e.x.s.a.s.FileRolesStore] [runTask-0] parsed [1] roles from file [/Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/build/testclusters/runTask-0/config/roles.yml]
[2023-12-18T14:31:20,137][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.blob_cache.miss_that_triggered_read.count
[2023-12-18T14:31:20,138][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.blob_cache.evicted_used_regions.count
[2023-12-18T14:31:20,138][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.blob_cache.cache_miss_load.time
[2023-12-18T14:31:20,231][INFO ][o.e.x.w.Watcher ] [runTask-0] Watcher initialized components at 2023-12-18T13:31:20.231Z
[2023-12-18T14:31:20,256][INFO ][o.e.x.p.ProfilingPlugin ] [runTask-0] Profiling is enabled
[2023-12-18T14:31:20,267][INFO ][o.e.x.p.ProfilingPlugin ] [runTask-0] profiling index templates will not be installed or reinstalled
[2023-12-18T14:31:20,287][INFO ][o.e.x.a.APMPlugin ] [runTask-0] APM ingest plugin is disabled
[2023-12-18T14:31:20,408][INFO ][o.e.c.u.FeatureFlag ] [runTask-0] The current build is a snapshot, feature flag [connector_api] is enabled
[2023-12-18T14:31:20,553][INFO ][o.e.t.n.NettyAllocator ] [runTask-0] creating NettyAllocator with the following configs: [name=unpooled, suggested_max_allocation_size=1mb, factors={es.unsafe.use_unpooled_allocator=null, g1gc_enabled=true, g1gc_region_size=4mb, heap_size=512mb}]
[2023-12-18T14:31:20,566][INFO ][o.e.i.r.RecoverySettings ] [runTask-0] using rate limit [40mb] with [default=40mb, read=0b, write=0b, max=0b]
[2023-12-18T14:31:20,567][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.repositories.requests.count
[2023-12-18T14:31:20,567][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.repositories.exceptions.count
[2023-12-18T14:31:20,567][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.repositories.throttles.count
[2023-12-18T14:31:20,567][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.repositories.operations.count
[2023-12-18T14:31:20,568][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.repositories.operations.unsuccessful.count
[2023-12-18T14:31:20,568][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.repositories.exceptions.histogram
[2023-12-18T14:31:20,568][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.repositories.throttles.histogram
[2023-12-18T14:31:20,568][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.repositories.requests.http_request_time.histogram
[2023-12-18T14:31:20,591][INFO ][o.e.d.DiscoveryModule ] [runTask-0] using discovery type [multi-node] and seed hosts providers [settings, file]
[2023-12-18T14:31:20,619][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.health.overall.red.status
[2023-12-18T14:31:21,154][INFO ][o.e.n.Node ] [runTask-0] initialized
[2023-12-18T14:31:21,155][INFO ][o.e.n.Node ] [runTask-0] starting ...
[2023-12-18T14:31:21,182][INFO ][o.e.x.s.c.f.PersistentCache] [runTask-0] persistent cache index loaded
[2023-12-18T14:31:21,183][INFO ][o.e.x.d.l.DeprecationIndexingComponent] [runTask-0] deprecation component started
[2023-12-18T14:31:21,232][INFO ][o.e.t.TransportService ] [runTask-0] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2023-12-18T14:31:21,322][WARN ][o.e.b.BootstrapChecks ] [runTask-0] Transport SSL must be enabled if security is enabled. Please set [xpack.security.transport.ssl.enabled] to [true] or disable security by setting [xpack.security.enabled] to [false]; for more information see [https://www.elastic.co/guide/en/elasticsearch/reference/master/bootstrap-checks-xpack.html#bootstrap-checks-tls]
[2023-12-18T14:31:21,323][INFO ][o.e.c.c.ClusterBootstrapService] [runTask-0] this node has not joined a bootstrapped cluster yet; [cluster.initial_master_nodes] is set to [runTask-0]
[2023-12-18T14:31:21,325][WARN ][o.e.d.FileBasedSeedHostsProvider] [runTask-0] expected, but did not find, a dynamic hosts list at [/Users/przemyslawgomulka/workspace/pgomulka/elasticsearch/build/testclusters/runTask-0/config/unicast_hosts.txt]
[2023-12-18T14:31:21,325][INFO ][o.e.c.c.Coordinator ] [runTask-0] setting initial configuration to VotingConfiguration{RpeX_621SdCZSBh8-RSQvg}
[2023-12-18T14:31:21,367][INFO ][o.e.h.AbstractHttpServerTransport] [runTask-0] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
[2023-12-18T14:31:21,372][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.get.total
[2023-12-18T14:31:21,372][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.get.time
[2023-12-18T14:31:21,372][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.search.fetch.total
[2023-12-18T14:31:21,373][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.search.fetch.time
[2023-12-18T14:31:21,373][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.merge.total
[2023-12-18T14:31:21,373][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.merge.time
[2023-12-18T14:31:21,373][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.translog.operations.count
[2023-12-18T14:31:21,373][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.translog.size
[2023-12-18T14:31:21,373][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.translog.uncommitted_operations.count
[2023-12-18T14:31:21,373][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.translog.uncommitted.size
[2023-12-18T14:31:21,373][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.translog.earliest_last_modified_age.time
[2023-12-18T14:31:21,373][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.transport.rx.size
[2023-12-18T14:31:21,373][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.transport.tx.size
[2023-12-18T14:31:21,374][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.jvm.mem.pools.young.size
[2023-12-18T14:31:21,374][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.jvm.mem.pools.survivor.size
[2023-12-18T14:31:21,374][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.jvm.mem.pools.old.size
[2023-12-18T14:31:21,374][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.fs.io_stats.io_time.total
[2023-12-18T14:31:21,374][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.docs.total
[2023-12-18T14:31:21,374][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.docs.total
[2023-12-18T14:31:21,374][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.failed.total
[2023-12-18T14:31:21,374][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.deletion.docs.total
[2023-12-18T14:31:21,374][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.deletion.docs.total
[2023-12-18T14:31:21,375][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.time
[2023-12-18T14:31:21,375][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.deletion.time
[2023-12-18T14:31:21,375][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.throttle.time
[2023-12-18T14:31:21,375][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.noop.total
[2023-12-18T14:31:21,375][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.coordinating_operations.memory.size.total
[2023-12-18T14:31:21,375][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.coordinating_operations.count.total
[2023-12-18T14:31:21,375][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.coordinating_operations.memory.size.total
[2023-12-18T14:31:21,375][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.coordinating_operations.count
[2023-12-18T14:31:21,376][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.coordinating_operations.rejections.total
[2023-12-18T14:31:21,376][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.primary_operations.memory.size.total
[2023-12-18T14:31:21,376][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.primary_operations.count.total
[2023-12-18T14:31:21,376][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.primary_operations.memory.size.total
[2023-12-18T14:31:21,376][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.primary_operations.count.total
[2023-12-18T14:31:21,376][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.primary_operations.rejections.total
[2023-12-18T14:31:21,376][DEBUG][o.e.t.a.APMMeterRegistry ] [runTask-0] Registering an instrument with name: es.node.stats.indices.indexing.memory.limit.total
[2023-12-18T14:31:21,376][INFO ][o.e.n.Node ] [runTask-0] started {runTask-0}{RpeX_621SdCZSBh8-RSQvg}{2w_LQi0HTTetBZfSdpLG6Q}{runTask-0}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.13.0}{7000099-8500007}{ml.machine_memory=68719476736, ml.config_version=12.0.0, ml.max_jvm_size=536870912, ml.allocated_processors=10, xpack.installed=true, transform.config_version=10.0.0, testattr=test, ml.allocated_processors_double=10.0}
[2023-12-18T14:31:21,421][INFO ][o.e.c.s.MasterService ] [runTask-0] elected-as-master ([1] nodes joined in term 1)[_FINISH_ELECTION_, {runTask-0}{RpeX_621SdCZSBh8-RSQvg}{2w_LQi0HTTetBZfSdpLG6Q}{runTask-0}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.13.0}{7000099-8500007} completing election], term: 1, version: 1, delta: master node changed {previous [], current [{runTask-0}{RpeX_621SdCZSBh8-RSQvg}{2w_LQi0HTTetBZfSdpLG6Q}{runTask-0}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.13.0}{7000099-8500007}]}
[2023-12-18T14:31:21,459][INFO ][o.e.c.c.CoordinationState] [runTask-0] cluster UUID set to [8TjOrE8yTw-_eqrhD95JPw]
[2023-12-18T14:31:21,484][INFO ][o.e.c.s.ClusterApplierService] [runTask-0] master node changed {previous [], current [{runTask-0}{RpeX_621SdCZSBh8-RSQvg}{2w_LQi0HTTetBZfSdpLG6Q}{runTask-0}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.13.0}{7000099-8500007}]}, term: 1, version: 1, reason: Publication{term=1, version=1}
[2023-12-18T14:31:21,498][INFO ][o.e.c.f.AbstractFileWatchingService] [runTask-0] starting file watcher ...
[2023-12-18T14:31:21,500][INFO ][o.e.c.f.AbstractFileWatchingService] [runTask-0] file settings service up and running [tid=60]
[2023-12-18T14:31:21,503][INFO ][o.e.c.c.NodeJoinExecutor ] [runTask-0] node-join: [{runTask-0}{RpeX_621SdCZSBh8-RSQvg}{2w_LQi0HTTetBZfSdpLG6Q}{runTask-0}{127.0.0.1}{127.0.0.1:9300}{cdfhilmrstw}{8.13.0}{7000099-8500007}] with reason [completing election]
[2023-12-18T14:31:21,560][INFO ][o.e.g.GatewayService ] [runTask-0] recovered [0] indices into cluster_state
[2023-12-18T14:31:21,638][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [behavioral_analytics-events-mappings]
[2023-12-18T14:31:21,650][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [elastic-connectors-sync-jobs-mappings]
[2023-12-18T14:31:21,652][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [elastic-connectors-settings]
[2023-12-18T14:31:21,654][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [elastic-connectors-sync-jobs-settings]
[2023-12-18T14:31:21,674][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.monitoring-ent-search-mb] for index patterns [.monitoring-ent-search-8-*]
[2023-12-18T14:31:21,681][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [elastic-connectors-mappings]
[2023-12-18T14:31:21,694][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.monitoring-logstash-mb] for index patterns [.monitoring-logstash-8-*]
[2023-12-18T14:31:21,697][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.ml-state] for index patterns [.ml-state*]
[2023-12-18T14:31:21,702][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.ml-notifications-000002] for index patterns [.ml-notifications-000002]
[2023-12-18T14:31:21,705][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [search-acl-filter] for index patterns [.search-acl-filter-*]
[2023-12-18T14:31:21,723][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.monitoring-kibana-mb] for index patterns [.monitoring-kibana-8-*]
[2023-12-18T14:31:21,728][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding template [.monitoring-kibana] for index patterns [.monitoring-kibana-7-*]
[2023-12-18T14:31:21,734][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding template [.monitoring-logstash] for index patterns [.monitoring-logstash-7-*]
[2023-12-18T14:31:21,756][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding template [.monitoring-es] for index patterns [.monitoring-es-7-*]
[2023-12-18T14:31:21,763][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding template [.monitoring-beats] for index patterns [.monitoring-beats-7-*]
[2023-12-18T14:31:21,766][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding template [.monitoring-alerts-7] for index patterns [.monitoring-alerts-7]
[2023-12-18T14:31:21,775][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.ml-anomalies-] for index patterns [.ml-anomalies-*]
[2023-12-18T14:31:21,786][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.monitoring-beats-mb] for index patterns [.monitoring-beats-8-*]
[2023-12-18T14:31:21,802][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.monitoring-es-mb] for index patterns [.monitoring-es-8-*]
[2023-12-18T14:31:21,806][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.ml-stats] for index patterns [.ml-stats-*]
[2023-12-18T14:31:21,810][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [synthetics-mappings]
[2023-12-18T14:31:21,811][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [metrics-tsdb-settings]
[2023-12-18T14:31:21,813][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [logs-mappings]
[2023-12-18T14:31:21,816][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [metrics-mappings]
[2023-12-18T14:31:21,820][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [data-streams-mappings]
[2023-12-18T14:31:21,824][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [ecs@dynamic_templates]
[2023-12-18T14:31:21,825][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [synthetics-settings]
[2023-12-18T14:31:21,826][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [metrics-settings]
[2023-12-18T14:31:21,828][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [logs@mappings]
[2023-12-18T14:31:21,829][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [synthetics@mappings]
[2023-12-18T14:31:21,831][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [metrics@settings]
[2023-12-18T14:31:21,832][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [synthetics@settings]
[2023-12-18T14:31:21,834][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [metrics@mappings]
[2023-12-18T14:31:21,836][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [data-streams@mappings]
[2023-12-18T14:31:21,839][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [ecs@mappings]
[2023-12-18T14:31:21,842][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.kibana-reporting] for index patterns [.kibana-reporting*]
[2023-12-18T14:31:21,843][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [metrics@tsdb-settings]
[2023-12-18T14:31:21,846][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.slm-history] for index patterns [.slm-history-6*]
[2023-12-18T14:31:21,849][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [ilm-history] for index patterns [ilm-history-6*]
[2023-12-18T14:31:21,851][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [.deprecation-indexing-mappings]
[2023-12-18T14:31:21,852][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [.deprecation-indexing-settings]
[2023-12-18T14:31:21,856][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.fleet-fileds-fromhost-data] for index patterns [.fleet-fileds-fromhost-data-*]
[2023-12-18T14:31:21,859][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.fleet-fileds-tohost-data] for index patterns [.fleet-fileds-tohost-data-*]
[2023-12-18T14:31:21,862][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.fleet-fileds-tohost-meta] for index patterns [.fleet-fileds-tohost-meta-*]
[2023-12-18T14:31:21,867][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.fleet-fileds-fromhost-meta] for index patterns [.fleet-fileds-fromhost-meta-*]
[2023-12-18T14:31:21,872][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.watch-history-16] for index patterns [.watcher-history-16*]
[2023-12-18T14:31:21,943][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [elastic-connectors-sync-jobs] for index patterns [.elastic-connectors-sync-jobs-v1]
[2023-12-18T14:31:21,946][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [synthetics] for index patterns [synthetics-*-*]
[2023-12-18T14:31:21,949][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [elastic-connectors] for index patterns [.elastic-connectors-v1]
[2023-12-18T14:31:21,952][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [metrics] for index patterns [metrics-*-*]
[2023-12-18T14:31:21,955][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [.deprecation-indexing-template] for index patterns [.logs-deprecation.*]
[2023-12-18T14:31:21,989][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [.monitoring-8-ilm-policy]
[2023-12-18T14:31:22,028][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [ml-size-based-ilm-policy]
[2023-12-18T14:31:22,063][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [metrics]
[2023-12-18T14:31:22,091][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [synthetics]
[2023-12-18T14:31:22,119][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [30-days-default]
[2023-12-18T14:31:22,207][INFO ][o.e.x.c.t.IndexTemplateRegistry] [runTask-0] adding ingest pipeline behavioral_analytics-events-final_pipeline
[2023-12-18T14:31:22,208][INFO ][o.e.x.c.t.IndexTemplateRegistry] [runTask-0] adding ingest pipeline logs-default-pipeline
[2023-12-18T14:31:22,208][INFO ][o.e.x.c.t.IndexTemplateRegistry] [runTask-0] adding ingest pipeline logs@default-pipeline
[2023-12-18T14:31:22,208][INFO ][o.e.x.c.t.IndexTemplateRegistry] [runTask-0] adding ingest pipeline ent-search-generic-ingestion
[2023-12-18T14:31:22,208][INFO ][o.e.x.c.t.IndexTemplateRegistry] [runTask-0] adding ingest pipeline logs@json-pipeline
[2023-12-18T14:31:22,208][INFO ][o.e.x.c.t.IndexTemplateRegistry] [runTask-0] adding ingest pipeline logs@json-message
[2023-12-18T14:31:22,210][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [behavioral_analytics-events-settings]
[2023-12-18T14:31:22,211][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [logs@settings]
[2023-12-18T14:31:22,211][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding component template [logs-settings]
[2023-12-18T14:31:22,258][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [behavioral_analytics-events-default] for index patterns [behavioral_analytics-events-*]
[2023-12-18T14:31:22,261][INFO ][o.e.c.m.MetadataIndexTemplateService] [runTask-0] adding index template [logs] for index patterns [logs-*-*]
[2023-12-18T14:31:22,293][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [logs]
[2023-12-18T14:31:22,319][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [180-days-default]
[2023-12-18T14:31:22,347][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [365-days-default]
[2023-12-18T14:31:22,378][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [90-days-default]
[2023-12-18T14:31:22,406][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [7-days-default]
[2023-12-18T14:31:22,448][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [logs@lifecycle]
[2023-12-18T14:31:22,488][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [metrics@lifecycle]
[2023-12-18T14:31:22,522][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [synthetics@lifecycle]
[2023-12-18T14:31:22,551][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [90-days@lifecycle]
[2023-12-18T14:31:22,587][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [180-days@lifecycle]
[2023-12-18T14:31:22,621][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [7-days@lifecycle]
[2023-12-18T14:31:22,648][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [365-days@lifecycle]
[2023-12-18T14:31:22,675][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [30-days@lifecycle]
[2023-12-18T14:31:22,701][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [watch-history-ilm-policy-16]
[2023-12-18T14:31:22,729][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [slm-history-ilm-policy]
[2023-12-18T14:31:22,770][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [ilm-history-ilm-policy]
[2023-12-18T14:31:22,809][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [.deprecation-indexing-ilm-policy]
[2023-12-18T14:31:22,846][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [.fleet-file-tohost-data-ilm-policy]
[2023-12-18T14:31:22,875][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [.fleet-file-fromhost-meta-ilm-policy]
[2023-12-18T14:31:22,903][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [.fleet-file-fromhost-data-ilm-policy]
[2023-12-18T14:31:22,932][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [.fleet-actions-results-ilm-policy]
[2023-12-18T14:31:22,959][INFO ][o.e.x.i.a.TransportPutLifecycleAction] [runTask-0] adding index lifecycle policy [.fleet-file-tohost-meta-ilm-policy]
[2023-12-18T14:31:23,055][INFO ][o.e.h.n.s.HealthNodeTaskExecutor] [runTask-0] Node [{runTask-0}{RpeX_621SdCZSBh8-RSQvg}] is selected as the current health node.
[2023-12-18T14:31:23,113][INFO ][o.e.x.s.a.Realms ] [runTask-0] license mode is [basic], currently licensed security realms are [reserved/reserved,file/default_file,native/default_native]
[2023-12-18T14:31:23,114][INFO ][o.e.l.ClusterStateLicenseService] [runTask-0] license [d539d324-3578-49b7-8410-aa7c09249acf] mode [basic] - valid
<============-> 99% EXECUTING [34s]
> IDLE
> IDLE
> IDLE
> :run

42
x-pack/plugin/ml/src/main/java/org/elasticsearch/xpack/ml/MlMetrics.java
View file

@ -103,7 +103,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
private void registerMlNodeMetrics(MeterRegistry meterRegistry) { private void registerMlNodeMetrics(MeterRegistry meterRegistry) {
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.native_memory.limit", "es.ml.native_memory.usage",
"ML native memory limit on this node.", "ML native memory limit on this node.",
"bytes", "bytes",
() -> new LongWithAttributes(nativeMemLimit, Map.of()) () -> new LongWithAttributes(nativeMemLimit, Map.of())
@ -111,7 +111,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
); );
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.native_memory.usage.anomaly_detectors", "es.ml.native_memory.usage.anomaly_detectors.usage",
"ML native memory used by anomaly detection jobs on this node.", "ML native memory used by anomaly detection jobs on this node.",
"bytes", "bytes",
() -> new LongWithAttributes(nativeMemAdUsage, Map.of()) () -> new LongWithAttributes(nativeMemAdUsage, Map.of())
@ -119,7 +119,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
); );
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.native_memory.usage.data_frame_analytics", "es.ml.native_memory.usage.data_frame_analytics.usage",
"ML native memory used by data frame analytics jobs on this node.", "ML native memory used by data frame analytics jobs on this node.",
"bytes", "bytes",
() -> new LongWithAttributes(nativeMemDfaUsage, Map.of()) () -> new LongWithAttributes(nativeMemDfaUsage, Map.of())
@ -127,7 +127,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
); );
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.native_memory.usage.trained_models", "es.ml.native_memory.usage.trained_models.usage",
"ML native memory used by trained models on this node.", "ML native memory used by trained models on this node.",
"bytes", "bytes",
() -> new LongWithAttributes(nativeMemTrainedModelUsage, Map.of()) () -> new LongWithAttributes(nativeMemTrainedModelUsage, Map.of())
@ -135,7 +135,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
); );
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.native_memory.free", "es.ml.native_memory.free.size",
"Free ML native memory on this node.", "Free ML native memory on this node.",
"bytes", "bytes",
() -> new LongWithAttributes(nativeMemFree, Map.of()) () -> new LongWithAttributes(nativeMemFree, Map.of())
@ -146,7 +146,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
private void registerMasterNodeMetrics(MeterRegistry meterRegistry) { private void registerMasterNodeMetrics(MeterRegistry meterRegistry) {
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.anomaly_detectors.opening.count", "es.ml.anomaly_detectors.opening.current",
"Count of anomaly detection jobs in the opening state cluster-wide.", "Count of anomaly detection jobs in the opening state cluster-wide.",
"jobs", "jobs",
() -> new LongWithAttributes(mlTaskStatusCounts.adOpeningCount, isMasterMap) () -> new LongWithAttributes(mlTaskStatusCounts.adOpeningCount, isMasterMap)
@ -154,7 +154,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
); );
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.anomaly_detectors.opened.count", "es.ml.anomaly_detectors.opened.current",
"Count of anomaly detection jobs in the opened state cluster-wide.", "Count of anomaly detection jobs in the opened state cluster-wide.",
"jobs", "jobs",
() -> new LongWithAttributes(mlTaskStatusCounts.adOpenedCount, isMasterMap) () -> new LongWithAttributes(mlTaskStatusCounts.adOpenedCount, isMasterMap)
@ -162,7 +162,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
); );
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.anomaly_detectors.closing.count", "es.ml.anomaly_detectors.closing.current",
"Count of anomaly detection jobs in the closing state cluster-wide.", "Count of anomaly detection jobs in the closing state cluster-wide.",
"jobs", "jobs",
() -> new LongWithAttributes(mlTaskStatusCounts.adClosingCount, isMasterMap) () -> new LongWithAttributes(mlTaskStatusCounts.adClosingCount, isMasterMap)
@ -170,7 +170,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
); );
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.anomaly_detectors.failed.count", "es.ml.anomaly_detectors.failed.current",
"Count of anomaly detection jobs in the failed state cluster-wide.", "Count of anomaly detection jobs in the failed state cluster-wide.",
"jobs", "jobs",
() -> new LongWithAttributes(mlTaskStatusCounts.adFailedCount, isMasterMap) () -> new LongWithAttributes(mlTaskStatusCounts.adFailedCount, isMasterMap)
@ -178,7 +178,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
); );
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.datafeeds.starting.count", "es.ml.datafeeds.starting.current",
"Count of datafeeds in the starting state cluster-wide.", "Count of datafeeds in the starting state cluster-wide.",
"datafeeds", "datafeeds",
() -> new LongWithAttributes(mlTaskStatusCounts.datafeedStartingCount, isMasterMap) () -> new LongWithAttributes(mlTaskStatusCounts.datafeedStartingCount, isMasterMap)
@ -186,7 +186,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
); );
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.datafeeds.started.count", "es.ml.datafeeds.started.current",
"Count of datafeeds in the started state cluster-wide.", "Count of datafeeds in the started state cluster-wide.",
"datafeeds", "datafeeds",
() -> new LongWithAttributes(mlTaskStatusCounts.datafeedStartedCount, isMasterMap) () -> new LongWithAttributes(mlTaskStatusCounts.datafeedStartedCount, isMasterMap)
@ -194,7 +194,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
); );
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.datafeeds.stopping.count", "es.ml.datafeeds.stopping.current",
"Count of datafeeds in the stopping state cluster-wide.", "Count of datafeeds in the stopping state cluster-wide.",
"datafeeds", "datafeeds",
() -> new LongWithAttributes(mlTaskStatusCounts.datafeedStoppingCount, isMasterMap) () -> new LongWithAttributes(mlTaskStatusCounts.datafeedStoppingCount, isMasterMap)
@ -202,7 +202,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
); );
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.data_frame_analytics.starting.count", "es.ml.data_frame_analytics.starting.current",
"Count of data frame analytics jobs in the starting state cluster-wide.", "Count of data frame analytics jobs in the starting state cluster-wide.",
"jobs", "jobs",
() -> new LongWithAttributes(mlTaskStatusCounts.dfaStartingCount, isMasterMap) () -> new LongWithAttributes(mlTaskStatusCounts.dfaStartingCount, isMasterMap)
@ -210,7 +210,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
); );
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.data_frame_analytics.started.count", "es.ml.data_frame_analytics.started.current",
"Count of data frame analytics jobs in the started state cluster-wide.", "Count of data frame analytics jobs in the started state cluster-wide.",
"jobs", "jobs",
() -> new LongWithAttributes(mlTaskStatusCounts.dfaStartedCount, isMasterMap) () -> new LongWithAttributes(mlTaskStatusCounts.dfaStartedCount, isMasterMap)
@ -218,7 +218,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
); );
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.data_frame_analytics.reindexing.count", "es.ml.data_frame_analytics.reindexing.current",
"Count of data frame analytics jobs in the reindexing state cluster-wide.", "Count of data frame analytics jobs in the reindexing state cluster-wide.",
"jobs", "jobs",
() -> new LongWithAttributes(mlTaskStatusCounts.dfaReindexingCount, isMasterMap) () -> new LongWithAttributes(mlTaskStatusCounts.dfaReindexingCount, isMasterMap)
@ -226,7 +226,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
); );
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.data_frame_analytics.analyzing.count", "es.ml.data_frame_analytics.analyzing.current",
"Count of data frame analytics jobs in the analyzing state cluster-wide.", "Count of data frame analytics jobs in the analyzing state cluster-wide.",
"jobs", "jobs",
() -> new LongWithAttributes(mlTaskStatusCounts.dfaAnalyzingCount, isMasterMap) () -> new LongWithAttributes(mlTaskStatusCounts.dfaAnalyzingCount, isMasterMap)
@ -234,7 +234,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
); );
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.data_frame_analytics.stopping.count", "es.ml.data_frame_analytics.stopping.current",
"Count of data frame analytics jobs in the stopping state cluster-wide.", "Count of data frame analytics jobs in the stopping state cluster-wide.",
"jobs", "jobs",
() -> new LongWithAttributes(mlTaskStatusCounts.dfaStoppingCount, isMasterMap) () -> new LongWithAttributes(mlTaskStatusCounts.dfaStoppingCount, isMasterMap)
@ -242,7 +242,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
); );
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.data_frame_analytics.failed.count", "es.ml.data_frame_analytics.failed.current",
"Count of data frame analytics jobs in the failed state cluster-wide.", "Count of data frame analytics jobs in the failed state cluster-wide.",
"jobs", "jobs",
() -> new LongWithAttributes(mlTaskStatusCounts.dfaFailedCount, isMasterMap) () -> new LongWithAttributes(mlTaskStatusCounts.dfaFailedCount, isMasterMap)
@ -250,7 +250,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
); );
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.trained_models.deployment.target_allocations.count", "es.ml.trained_models.deployment.target_allocations.current",
"Sum of target trained model allocations across all deployments cluster-wide.", "Sum of target trained model allocations across all deployments cluster-wide.",
"allocations", "allocations",
() -> new LongWithAttributes(trainedModelAllocationCounts.trainedModelsTargetAllocations, isMasterMap) () -> new LongWithAttributes(trainedModelAllocationCounts.trainedModelsTargetAllocations, isMasterMap)
@ -258,7 +258,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
); );
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.trained_models.deployment.current_allocations.count", "es.ml.trained_models.deployment.current_allocations.current",
"Sum of current trained model allocations across all deployments cluster-wide.", "Sum of current trained model allocations across all deployments cluster-wide.",
"allocations", "allocations",
() -> new LongWithAttributes(trainedModelAllocationCounts.trainedModelsCurrentAllocations, isMasterMap) () -> new LongWithAttributes(trainedModelAllocationCounts.trainedModelsCurrentAllocations, isMasterMap)
@ -266,7 +266,7 @@ public final class MlMetrics extends AbstractLifecycleComponent implements Clust
); );
metrics.add( metrics.add(
meterRegistry.registerLongGauge( meterRegistry.registerLongGauge(
"es.ml.trained_models.deployment.failed_allocations.count", "es.ml.trained_models.deployment.failed_allocations.current",
"Sum of failed trained model allocations across all deployments cluster-wide.", "Sum of failed trained model allocations across all deployments cluster-wide.",
"allocations", "allocations",
() -> new LongWithAttributes(trainedModelAllocationCounts.trainedModelsFailedAllocations, isMasterMap) () -> new LongWithAttributes(trainedModelAllocationCounts.trainedModelsFailedAllocations, isMasterMap)