## Summary
API integration tests added in #180094 fail occasionally on main, but
pass consistently locally and in the flaky test runner. This PR adds a
retry to the `getMetrics` request in hopes of removing the flakiness.
Flake issues:
https://github.com/elastic/kibana/issues/180530https://github.com/elastic/kibana/issues/180641
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Closes#179660.
#### Changes:
- **Remove APM specific functions**. This cuts down on token cost in
some cases, and generally creates a more cohesive story for all
Observability data.
- **Add an `get_apm_dataset_info` function**. This allows the LLM to
more easily see where data is stored.
- **Introduces a `changes` function**. Returns change points for any
kind of logs (using `categorize_text`) or metrics data.
- **Increased the no of max function calls from 5 to 8**. With
`get_dataset_info` and `get_apm_dataset_info` being called more often
this becomes more necessary.
- **Improve accuracy of SORT commands**. Removes some examples from the
docs where a STATS command is being used without creating a column. This
seems to limit hallucinations from the LLM where it tries to SORT on an
expression instead of a column.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Adds option to Defend integration's Antivirus Registration card to sync
registration with Malware settings.
<img width="1220" alt="image"
src="ac7716ec-5bf6-4aa2-9fdf-3f8fa1ebfe93">
## Details:
- it adds a new field to `PolicyConfig`: `antivirus_registration.mode`,
which can be `enabled`, `disabled` or `sync_with_malware_prevent`
<img width="313" alt="image"
src="fc38799c-e620-4dbc-a727-576c5c827065">
- this field is **not** used by Endpoint: instead the existing
`antivirus_registration.enabled` field is derived from this field, so
it's compatible with older Endpoints, too
- the calculation of `antivirus_registration.enabled` happens both on
client side and on server side (in Fleet's `packagePolicyUpdate` ingest
callback)
- default value for new policy is the same: `disabled`, as in previous
version
- adds migration
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: David Sánchez <davidsansol92@gmail.com>
Co-authored-by: Joe Peeples <joe.peeples@elastic.co>
## Summary
Issue: https://github.com/elastic/kibana/issues/179476
Parent issue: https://github.com/elastic/kibana/issues/157883
This PR versions the update (`PUT '/api/alerting/rule/{id}'`) route.
Still using `config-schema` for now, even though we will eventually
switch to `zod` when core is ready with openapi doc generation support
in the versioned router.
We are now validating update data using the update data schema when
calling `rulesClient->update`. We are also validating (but not throwing)
the updated rule as well.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Updates the ES|QL docs for 8.14. Biggest change is probably the rename
of AUTO_BUCKET to BUCKET, and the fact that aggregation functions now
support expressions.
Closes [#180235](https://github.com/elastic/kibana/issues/180235).
Adds an advanced setting to enable simulated function calling. Simulated
function calling is an alternative for models/APIs that do not natively
(via the API) support function calling. Instead of using the API, we
inject functions into the system prompt, tell the LLM how to call it via
plain text, and extract the function calls from the response message.
For Bedrock this is currently always used. (We previously used XML, but
now use JSON Schema).
For OpenAI, we only use it when the advanced setting is on.
If it has been enabled, we show a warning:
We _don't_ show this warning for Bedrock if the setting is not on, even
though we always use it.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes#180531
This pull request moves entity analytics route registration and url
definition into files owned by our team.
Currently, to add a new route we require a code owners review from both
the `security-detections-response` and `security-threat-hunting` teams
unnecessarily. This is because we needed to change the following files:
- `x-pack/plugins/security_solution/common/constants.ts`
- `x-pack/plugins/security_solution/server/routes/index.ts`
As recommended by @maximpn
[here](https://github.com/elastic/kibana/pull/179930#pullrequestreview-1992231221)
I have also removed redundant feature flag checks for enabling risk
scoring and risk engine privileges routes, these feature flags are
enabled now.
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
closes [#175446](https://github.com/elastic/kibana/issues/175446)
## Summary
Adds new Metrics tab and redesigns the metrics section in the Overview
tab
766305ce-23c6-4001-ab20-2230decc02a1
### Extra changes
- Replaced usages of asset name in favor of asset id to filter asset
data across the Asset Details components - which will help when it
starts supporting more asset types
- Renamed some translation keys to use `assetDetails` instead of
`nodeDetails`
### How to test
- Start a local Kibana instance
- Navigate to Infrastructure > Hosts
- Open the flyout for a host
- Overview tab
- Check if the metrics section matches the design and AC
- Metrics tab
- Check if it matches the design the design and AC
- Check the above in the full page view.
---------
Co-authored-by: Cauê Marcondes <55978943+cauemarcondes@users.noreply.github.com>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: jennypavlova <dzheni.pavlova@elastic.co>
## Summary
Resolves https://github.com/elastic/kibana/issues/178945.
Resolves#178947.
Resolves https://github.com/elastic/kibana/issues/178952.
~Undergoing the work to export package list in a sharable way while
minimally impacting the existing implementation used by the Fleet team.~
### Update
Updated to provide the flows for all three paths of the form. Synthetics
and APM still link to their respective apps. Logs icons link to the
embedded flows.
#### Things to test
- Check that the integration cards behave in a manner you'd expect.
There's a lot of custom card creation happening and it's possible I have
made a mistake in some linking.
- Icons: there are a few icons that don't correspond to the design. I
wasn't able to find the correct `src` for the colored-in APM logo, for
example, and I'm not sure where the one logs icon lives. I've used the
`logoFilebeat` icon in its place as it's quite similar and is heavily
used in the integrations list.
- For the searchable integrations list, I was unsure if I should re-hide
it when the user changes their radio selection in the top-level
question. Right now it'll just stay as-is with their original query.
- The `Collections` feature, make sure collection buttons properly apply
the search query.
~This now works with the exported package list grid from Fleet. I've
introduced a real package list in place of the dummy grid that we had
before. I also added a searchable grid below.~
~Surely there are some things to iron out still. I also still consider
the code rough, so if you should choose to review it you will likely
find areas you want to see improved.~
---
**Note:** I will likely extract the Fleet-specific pieces of this to a
separate patch that we can merge to Kibana `main` to make it easier for
the Fleet team to review the changes. We can then merge those upstream
and pull them into our feature branch.
^^ this is still true, before we merged this I'll likely pull out the
Fleet-specific modifications we need and have the Fleet team review them
separately. We can still merge this to the feature branch in the
meantime and resolve it later.
---------
Co-authored-by: Joe Reuter <johannes.reuter@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Thom Heymann <190132+thomheymann@users.noreply.github.com>
## Summary
We could find the user's full name might not always be available from
the 1st image below, but username should always exist.
Previously, we displayed username straight away and it didn't look good.
[bug](https://github.com/elastic/kibana/issues/177204)
In this PR, we change the logic and display user's full name if it
exists, otherwise fallback to the username.
**No Full Name scenario - It displays username**
<img width="2293" alt="Screenshot 2024-04-12 at 09 58 52"
src="355e5a3d-e8f8-4204-8234-8eddd14691e1">
<img width="2559" alt="Screenshot 2024-04-12 at 09 59 18"
src="14ba8250-57cf-4fc1-9bdf-a3ac021b91c8">
**Full Name available scenario - It displays the full name**
<img width="2291" alt="Screenshot 2024-04-12 at 10 07 28"
src="57cb5aa2-ae23-4e0b-bd13-7b6d72edce40">
<img width="2557" alt="Screenshot 2024-04-12 at 10 08 24"
src="a4cbd64f-7eef-454b-a5fc-e12f25a82ea5">
### Checklist
Delete any items that are not applicable to this PR.
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR fixes https://github.com/elastic/kibana/issues/175002
Follow steps defined in the linked issue during testing, but generally
this is about adding custom filter in the timeline query tab.
- Closes https://github.com/elastic/kibana/issues/175419
## Summary
This PR enables customization for title/description we show in Inspector
flyout for ES|QL requests.
<img width="500" alt="Screenshot 2024-04-09 at 15 25 11"
src="9636458b-496f-4a87-bd1c-8d125f867752">
<img width="500" alt="Screenshot 2024-04-09 at 15 25 20"
src="3efe83c1-49ef-47e5-867f-eae3f00ca488">
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
This PR adds the contribution scores used when calculating an entity's
risk score.
These can be viewed by opening an entity's details flyout and clicking
on `View risk contributions` which will open the explainability tab.
The new tab now shows the contribution from different contexts and
individual alerts. Currently, there's only one context: Asset
Criticality.
The alert inputs shown are the top 10 (at most) alerts used as an input
at the time of the scoring calculation. If the final score used more
than 10 alerts, we display a message indicating the leftover sum
contribution.
This work also updates the server side in order to store the
contribution of each individual alert in the risk document itself. We
now query the document to retrieve the inputs and then fetch the
respective alerts on opening the tab.
Example:
### How to test
1. Generate some alerts - you can use
https://github.com/elastic/security-documents-generator
2. Enable risk scoring via `Security > Manage > Entity Risk Score`
3. Open the entity details flyout and click `View risk contributions`
Make sure to enable Asset Criticality in `Stack Management > Advanced
Settings` if you want to see the criticality contributions.
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Closes https://github.com/elastic/kibana/issues/158861
## Summary
Expose agent logging level in agent policy settings.
The new setting is added via the new settings framework and will show up
under "advanced settings". It's currently hidden until the agent
supports it.
### Testing
Enable the settings config:
https://github.com/elastic/kibana/pull/180597#discussion_r1562448034
- Go to the agent policy settings form
- Under advanced settings there is a new dropdown "Agent logging level".
Choose a value and save the policy
- The new value should be retained after saving
- Go to the agent policies tab and select action "View policy"
- The new field should be visible under `agent.logging.level`
<details>
<summary> Screenshots</summary>
</details>
### Checklist
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/kibana/issues/163417
When there was no from command in the query we were using the current
dataview. This might have the @timestamp field which is not returned by
the `ROW ...` or `Show meta` commands. So the histogram was failing.
I am solving this issue by creating a dataview based on the current
dataview but without the timeFieldName
<img width="1677" alt="image"
src="81b79634-8c2e-4346-bd34-48ae7580ab89">
I still think we should find another way to deal with these commands but
for now this is a nice way forward
**Before:**
<img width="1679" alt="image"
src="68ec6f76-6721-472b-8b49-7c719ad04208">
### Checklist
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
## Summary
Adds group by cardinality count to the synthetics availability SLO
indicator
Resolves https://github.com/elastic/kibana/issues/178409
Resolves https://github.com/elastic/kibana/issues/178140
Also, it's come to my attention that
https://github.com/elastic/kibana/issues/178341 was not fixed by a
previous PR. This PR now also resolves
https://github.com/elastic/kibana/issues/178341
### Testing
1. Create an cluster with oblt-cli and add the config to your
`kibana.dev.yml`
2. Navigate to the Synthetics app. Create at least two synthetic
monitors
3. Navigate to SLO create. Select the synthetic availability indicator
4. Check the group by cardinality callout. The cardinality should
reflect the number of monitor/location combinations
<img width="730" alt="Screenshot 2024-04-12 at 1 04 57 PM"
src="a05ffaff-c01b-4107-8f8d-2ea8362fe72e">
5. Now filter by monitor name or tag. The group by cardinality should
reflect the number of monitors that match the filters
<img width="733" alt="Screenshot 2024-04-12 at 1 05 11 PM"
src="079c74ea-dd1c-45f2-bf0e-2dbefea30f96">
### Testing https://github.com/elastic/kibana/issues/178341
To test the fix for https://github.com/elastic/kibana/issues/178341,
create a simple custom kql SLO with a group by. Add a overall filter
that would impact the overall group by count. Verify that the group by
count accurately reflects the overall filter.
Added new sub-feature to AI-Assistant which allows to grant user role
privilege to edit Anonymization fields:
<img width="761" alt="Screenshot 2024-04-11 at 7 29 50 PM"
src="d4358178-d8e9-4c68-b7d4-a19d2befa29b">
How to test:
1. Create user role, which has access to Security and Actions/Connectors
and AI Assistant.
2. Customize sub-feature privileges: Remove checkbox for Update
anonymization fields.
3. Save role.
4. Create/update existing user with the new role.
5. Go to Security AI Assistant settings and open Anonymization tab.
6. For the user role with removed privilege to edit anonymization, all
actionable buttons should be disabled. Button Save will remain enabled,
because it correspond to all Assistant settings.
7. Public API "/api/elastic_assistant/anonymization_fields/_bulk_action"
should return "Forbidden" 403 access error.
How it looks when no privilege for the user role:
<img width="771" alt="Screenshot 2024-04-12 at 10 12 34 AM"
src="7c3b6c92-12cb-46ae-8356-dc687c82726f">
@patrykkopycinski please adopt it after merging this PR to the new UX
---------
Co-authored-by: Garrett Spong <garrett.spong@elastic.co>
## Summary
On behavior alerts with Defend, the default behavior is to enrich the
alert by scanning some memory regions against Yara memory signatures.
This PR adds an advanced setting to opt-out of this behavior.
### Checklist
Delete any items that are not applicable to this PR.
- [x] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Daniel Ferullo <56368752+ferullo@users.noreply.github.com>
Closes https://github.com/elastic/kibana/issues/180225
Test instructions
1. start kibana with `yarn start --run-examples`
2. install web logs sample data set
3. create new dashboard, Click "Add panel" and select "Unified search
example"
4. Set time range to last 7 days
5. create a filter "bytes >= 15000". Ensure panel updates
6. Click reload, ensure panel re-fetches data
7. Try a bunch of other things in Unified search UI and ensure panel
fetches data as expected
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Part of https://github.com/elastic/kibana/issues/169547
View docs at [Changed
pages](https://kibana_169928.docs-preview.app.elstc.co/diff)
Add monitor api public api
### Testing
Make sure you have some monitors populated before testing this PR and
before switching to the branch
- [ ] Try editing already added monitors via API
- [ ] Test adding monitors via API, and then edit those via and
subsqeuently try editing via API the same monitor
- [ ] Test editing monitors via API
- [ ] Test deleting monitors via API
- [ ] Test getting monitors via API
- [ ] Testing private as well public locations
Basic workflow that i am interesting in testing is to make sure, you can
add/edit via both API and UI without any issues
Test each of HTTP/TCP/ICMP browser examples
<img width="1728" alt="image"
src="3575d93a-5f04-4c80-ac62-038643f466f8">
---------
Co-authored-by: Justin Kambic <jk@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Dominique Clarke <dominique.clarke@elastic.co>
