So clients reported that they got stuck in the set up screen In the
Universal Profling UI. And when the set up button was clicked an error
happened:
```
An integration policy with the name elastic-universal-profiling-collector already exists. Please rename it or choose a different name.
```
This happens because when we were checking if the Collector and
Symbolizer integrations were installed we weren't taking into
consideration that the Fleet API is paginated. So if neither integration
was available on the first page we just assumed the Profiling wasn't set
up.
This PR fixes it by adding a kuery filter in the Fleet API call to only
look for out integrations. So we don't need to worry about paginating.
A bunch of tests on dashboards are customising some of the panels
settings and providing custom time ranges:
<img width="409" alt="image"
src="c869c1a3-f7db-4ccd-ad00-c5403f2b4201">
Currently, the logic is not waiting for the quick toggle animation to
complete, before proceeding to select a time range.
This can cause a flaky behavior if the logic tries to customize the
range before the button is actually available, as seen on [this failed
test](018a4c46-0e7a-4b69-9a3d-9c54c27165b0_4fcbc47e71644919129e320eea8bb3bc.html?response-content-type=text%2Fhtml&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAQPCP3C7LZWZ5UB5F%2F20230901%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230901T094837Z&X-Amz-Expires=600&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEB0aCXVzLWVhc3QtMSJGMEQCIGCyKcVLGPUawZubNzZdt5oZNb5v0saiIuPqXwI7rmwlAiAsOj%2Fiep94v%2BYZJtLY3Gw0m%2FmK5mJw2IcIBdNKFXgK%2BCr6Awjm%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F8BEAAaDDAzMjM3OTcwNTMwMyIMXOd1Hm6ks%2FNE37V0Ks4DgMUso7syv87hnPcC%2BB1soxvFFnj4JnNZc6ZgkLUe93z99iPFBUsqH%2BRbUTfSbjVOEJYBKGYuvp32xvSWsYNVPXKmcej18LC0yNi%2BBzoG2X%2Bj80g%2BbGMm6YfTncjPhOE0CHHqOWXts9nQ8WpDy8XOl0zfMtuiPjzOXHo9lvw2mgYDZIJIMV72FYB9JGg8FPbLQtD3rysLGNE0VDKgl5LCnYwhY1pwRCRHnVW41QfV0pwK%2FbjNf9HjdK31LQvMY%2FGPuB3M6O2CUZLsvLGfWBeGYHtkqb0hrL9ijO1Uo28ZSS1FytPftEdF0e1kAC9C5zD56HtYm55aktOWtaaC0XPWLdWWGUq%2FKQzhxSCiXK6ovATU3zI3yPNoZs92YBYmIPMOpEI40dCCpksjPwAMCiQd%2F9gMNKP5Qp5CbYd2Khy%2FeXaT8J7HOZCueN63O0j%2FtX1tbwfznhbr74lAcRQjueRYmwboZaGSDZUQ33lSSmyZk1V9WF9eJyt88oHvIx0q9bIjvOlW05DiNKfEFWYwfBywdGuvRU6eGMs1QcDNu33Lb%2BhymudM2JZmQKIjZOcb2l3Fzctp614owH4JcRlmF4%2BIa4xHeBdRlTMysS8bTIsgMK7axacGOqYBzIpC1wgZWJ1kZ0agLWCNaMIdUl%2B4xrr7w%2Fz0843WWMhRrvbJhDTHqk5UclF%2FSROAMe0FH2XEXiQ65ILyUPlrUMels5tfQ3Pp%2FJWPi9NsQJUQ1n9uLN%2BFPDOoMo8Uxg4%2FkG2O7yTkrIdArfA6pWN9I21gFMW%2BFZy9BMYltt5T65ZKOyYAIFGpLhgfBySIBCUMgwR1kusfDhf1%2FRTvtDKD2sJKN5a0IA%3D%3D&X-Amz-SignedHeaders=host&X-Amz-Signature=35fabe908aa7514e4a92de0ed12973af85ccfb439984fc3bdd7ef3bb8fe3419b).
(part of this [failed CI
build](https://buildkite.com/elastic/kibana-pull-request/builds/155285#018a4c46-0e7a-4b69-9a3d-9c54c27165b0))
The goal of this PR is to add a small waiting period, to make sure the
toggle animation has completed, and that the time range controls are
visible and clickable.
I used the opportunity to cleanup some "await delay millis" calls,
reusing existing logic instead.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
**Relates to:** https://github.com/elastic/kibana/issues/153645
## Summary
This PR makes refactoring of navigation utilities and constants to make it transparent and be able to specify correct ownership later on.
## Motivation
Engineers need to add new tests and maintain the older ones. Base building blocks of this are constants and utility functions in particular URL constants and `visit()` and `visitWithoutDataRange()` functions. It turned out a simple `visit()` helper function also passes `timerange` as a query param while only some tests need it. Having it as a base utility function (as the name suggests) it also gets used in the new code while `visitWithoutDateRange()` should be preferred. On top of that URL constants are combined in one file without clear ownership and contain a mix of legacy and new urls with some parts looking outdated as navigating to the url causes redirecting to a specific page.
Having only relevant URL constants in a common file will help to reduce confusion. As the next step constants should be split into files/folders with clear ownership.
Also having `visit()` adding no extra params (besides common for ALL the tests) will make the intention clear. Whenever a time range is needed `visitWithTimeRange()` can be used (ideally accepting a time range defined in a test). And the same stays true for any other pages, e.g. rule details page can have a utility `visitRuleDetailsPage(id: string)` containing some general waiting logic so the following actions operate on a loaded page.
## Details
As a step towards clearness and transparent ownership this PR performs refactoring of `x-pack/test/security_solution_cypress/cypress/urls/navigation.ts` and `x-pack/test/security_solution_cypress/cypress/tasks/login.ts` files. The following has been done
- all url constants in `x-pack/test/security_solution_cypress/cypress/urls/navigation.ts` were checked and updated to remove duplications, avoid redirections and grouped
- legacy urls were moved to the only one test using them to test compatibility (`x-pack/test/security_solution_cypress/cypress/e2e/urls/compatibility.cy.ts`)
- `visit()` was renamed to `visitWithTimeRange()`
- `visitWithoutDateRange()` was renamed to `visit()`
- `visit()` was refactored to accept a query string
## Next steps
It's expected teams decompose `x-pack/test/security_solution_cypress/cypress/urls/navigation.ts` into feature specific file(s)/folder(s) with assigned owners.
There is no 100% chance a generic wait for a page to be loaded helper function meet requirements for each page. It makes sense to consider adding per feature `visitFeatureAPage()` helper function containing assertions for the page to be loaded. `visitRuleDetailsPage(id: string)` was added for this purpose while waiting for page to be loaded functionality is omitted in this PR to reduce a number of changes.
## Summary
Fixes https://github.com/elastic/kibana/issues/166100
This PR adds a workaround fix for the new index details page when
opening for index names with special characters, for example
`test_index%`. Because of how encoding/decoding works, we can't use the
index name as a part of the url like `/indices/${indexName}` (see for
more details). Instead we have to pass the index name in a query
parameter like `/indices/index_details?indexName=${indexName}. The
downside of this workaround is that the url semantics doesn't reflect
that the index name is mandatory for the page to work. Once
https://github.com/elastic/kibana/issues/132600 is resolved, we should
revert this workaround and use the index name as a url segment again.
Note for reviewers: The jest tests for this fix are part of
https://github.com/elastic/kibana/pull/165705
### How to test
1. Add `xpack.index_management.dev.enableIndexDetailsPage: true` to the
file `config/kibana.dev.yml` to enable the new index details page
2. Navigate to Index Management and use the "create index" button
3. Type a name with special characters, for example `test%`
4. Click the new index name in the list and check that the details page
and all tabs work
5. Also reload the page completely and check that the page still loads
correctly
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Alison Goryachev <alisonmllr20@gmail.com>
## Summary
This enables the installer for SLO transforms on CI.
- For the `many_fields_transform` performance journey, it will then look
for the "Create transform" on top of the table instead of the "Create
your first transform" button. Command to test the performance journey:
`node scripts/run_performance.js --journey-path
x-pack/performance/journeys/many_fields_transform.ts --skip-warmup`
- For the functional tests that assert the empty transform list, this
adds a command to delete all transforms before running those tests.
## Summary
In ES|QL we don't have field formatters. We just have the type of the
field (number/date etc).
If you are in Discover and write a query which returns only one
column/row then the new metric is suggested.
<img width="1990" alt="image"
src="79650d4b-f0c9-476b-b050-617653ccc993">
If for whatever reason the user goes to breakdown and selects the same
number variable (try_new) the chart will fail to render and you will see
in the console an error.
This happens because EC wait the title to be string but here as we don't
have field formatters this is number and fails.
## Summary
Fix confusing labels for status alert toggle
removed the Disabled wording from toggle
<img width="1473" alt="image"
src="811cea3f-b3bd-4e65-bd48-317d259c8696">
## Summary
Resolve https://github.com/elastic/kibana/issues/164269
Some context why I picked this up now:
https://github.com/elastic/kibana/issues/162772#issuecomment-1728031080
To verify:
- Make sure 8.8+ apm package is installed
- Create data stream `PUT _data_stream/metrics-apm.app.default-default`
- Reinstall apm package from API or UI
- Check kibana info logs, expect to not see simulate template error and
rollover like below
```
[2023-09-21T15:54:36.559+02:00][INFO ][plugins.fleet] Mappings update for metrics-apm.app.default-default failed due to ResponseError: illegal_argument_exception
Root causes:
illegal_argument_exception: unable to simulate template [metrics-apm.app.default] that does not exist
[2023-09-21T15:54:36.559+02:00][INFO ][plugins.fleet] Triggering a rollover for metrics-apm.app.default-default
```
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
This PR removes the Profiling dependency from APM, introduced on `8.10`.
- Exposes a new service in profiling-data-access plugin
- Create a new APM API that calls the new service and checks if
Profiling is initialized
- Move Locators from the Profiling plugin to the Observability-shared
plugin
- Move logic to check Profiling status (has_setup/has_data...) from
Profiling server to profiling-data-access plugin
- Create API tests, testing the status services based on different
scenarios:
- When profiling hasn't been initialized and there's no data
- When profiling is initialized but has no data
- When collector integration is not installed
- When symbolized integration is not installed
- When APM server integration is not found
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes#158957
## Summary
Adds the missing `getActionErrorLog` privilege. With the updated
privileges, users with a custom Role including full access to "Actions
and Connectors", "Rule Settings" and "Stack Rules" can successfully
inspect errored actions' logs:
## To Test
- Create a Role with `All` privileges granted in `Actions and
Connectors`, `Rules Settings`, `Stack Rules` (under Kibana > Management)
and assign it to a user
- Log in with that user
- Create a rule with a failing action (i.e. an Email Connector with
wrong addresses)
- Wait for the rule to execute (or execute it manually)
- In the rule page, under `History` click the number under `Errored
actions` in one of the rows of the logs table
- Check that error logs are visible in the flyout
Resolves https://github.com/elastic/kibana/issues/162630
## Summary
Adds checks for invalid alert index names in two places:
- before updating the underlying settings/mappings of a concrete index;
attempting these updates would typically throw an error and prevent
alert resource installation from successfully completing, causing
subsequent writes to fail; in this PR ,we check for unexpected index
names and log a warning
- before updating an alert document; there is a scenario where an
existing active alert document could be in a partial or restored alert
index and trying to update that alert document would fail. to prevent
these failures, we skip the update and log a warning. we expect this
case to be more rare as most times frozen indices only contain old
alerts so the likelihood of it containing a currently active alert
should be low.
## To Verify
- Run ES with these options: `yarn es snapshot --license trial --ssl -E
path.data=../data_partial_alerts -E path.repo=<snaphot folder>-E
xpack.searchable.snapshot.shared_cache.size=100b -E
indices.lifecycle.poll_interval=1m`
- Start Kibana
- Create a snapshot repository here:
https://localhost:5601/app/management/data/snapshot_restore/add_repository.
Use `Shared File System` and use the same path as you used for
`path.repo` when running ES
- Modify the `.alerts-ilm-policy` to roll over in the hot phase with max
age of 3 minutes. Add a frozen phase that moves data into the frozen
phase after 5 minutes.
- Create some rules that generate alerts. I did both metric threshold
(uses lifecycle executor) and index threshold (uses framework).
- Wait for ILM to run and move indices to frozen. This will take a few
ILM cycles but eventually you should be able to do a `GET
.internal.alerts-stack.alerts-default-*/_alias/.alerts-stack.alerts-default`
and see a partial index name in the results
- Restart Kibana. You should see warnings logged related to the partial
indices but Kibana should successfully start and rule execution should
succeed.
## Notes
I tested what would happen if we added a bunch of new fields to a
component template and increased the total fields limit in the presence
of partial indices. Here, it works in our favor that we only allow
additive changes to our mappings, so the existing partial indices keep
the old mappings and don't need a field limit update because their
mappings don't change. Searching against both the alerts alias (that
targets partial and normal indices) works as expected and searching
directly against the partial index works as expected.
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR is updating Discover's rule to be created under the
`stackAlerts` consumer and we created an [breaking change
issue](https://github.com/elastic/dev/issues/2344) to explain the
consequences of this update.
We also fix the rule's consumer for all rule types created under the
observability rule management to use their producer instead of `alerts`.
Also, we add the ability for the ES Query and new Generic Threshold
rules type to pick the consumer associated to the rule. The
`ensureAuthorized` and the `filter` functions have modified and
simplified to support this use case please check the newest unit test
added in
`x-pack/plugins/alerting/server/authorization/alerting_authorization.test.ts`.
There is now a dropdown in the rule form to prompt the user when
creating ES Query/Generic threshold rules to select the consumer based
on their authorized consumers (we can no longer use `alerts` for these).
If there is only 1 option, then the dropdown will not be shown and the
option will be chosen automatically.
Generic threshold rules will have the following possible consumers:
- infrastructure
- logs
ES query rules will have the following possible consumers:
- infrastructure
- logs
- stackAlerts (only from the stack management rule page)
## To Test:
### Single Consumer:
1. Create a user with only `logs` feature enabled (ensuring
`stackAlerts` is not enabled).
2. Navigate to the O11Y rule management page
3. Click the create rule button
4. Assert that both ES query and generic threshold rules are available
5. Click ES query and fill out the relevant information and create the
rule
6. Assert that the rule created has `logs` set in the `consumer` field
7. Repeat 5-6 for the generic threshold rule
8. Repeat 2-7 but on the Stack Management rules page
9. Repeat 1-8 for the `infrastructure` feature.
### Multiple Consumers:
1. Create a user with `logs`, `infrastructure` and `apm` features
enabled (ensuring `stackAlerts` is not enabled).
2. Navigate to the O11Y rule management page
3. Click the create rule button
4. Assert that both ES query and generic threshold rules are available
5. Click ES query and fill out the relevant information and create the
rule
6. A dropdown should prompt the user to select between 1 of the 3
consumers, select 1
7. Assert that the rule was created with the selected consumer
8. Repeat 5-7 for the generic threshold rule
9. Repeat 2-8 but on the Stack Management rules page
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Jiawei Wu <74562234+JiaweiWu@users.noreply.github.com>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Resolves https://github.com/elastic/kibana/issues/166418
## Summary
Change from `warn` to `debug`. Was able to replicate this error when
migrating an es query rule from 8.9 to 8.10 where es query uses AAD.
