## Summary
It closes https://github.com/elastic/security-team/issues/10104
This PR addresses the necessary changes in Kibana to reflect the updated
agentless API settings in kibana.yml settings:
- **xpack.fleet.agentless.enabled: true**
Changing to use `xpack.fleet.agentless.enabled` as a Feature flag for
enabling agentless in ESS environments. (Serverless changes was out of
scope, so it is still using `xpack.fleet.enableExperimental` for now).
Also, this value will now be exposed to the client side instead of the
API URL (It closes
https://github.com/elastic/security-team/issues/10019).
- **xpack.fleet.agentless.api.url**
This setting will now include only the URL (without the basePath), so I
added a `prependAgentlessApiBasePathToEndpoint` helper function to
append the basePath with type support. Also this value will no longer be
exposed to the client side
### Additional changes
- This PR also modified the Agentless settings schema validation to be
optional (so it will only validate the types), as because of the nature
of how the `kibana.yml` will set the configuration as of
[this](https://github.com/elastic/cloud/pull/130314) PR, it might happen
that we will have partial settings, so changing the schema to optional
will prevent Kibana failing from starting due to incomplete agentless
settings. (It closes
https://github.com/elastic/security-team/issues/10189)
- Updated unit tests and FTR e2e config files to reflect the latest
changes
## Summary
This PR tries to stabilize the Discover sidebar tests which started to
be flaky when running against MKI.
### Details
The serverless Discover sidebar tests (`Common Group
5.x-pack/test_serverless/functional/test_suites/common/discover/group6/_sidebar·ts`)
started to become flaky when running against MKI. 5 out of the last 10
runs failed for the same reason:
```
discover/group6 discover sidebar renders field groups should render even when retrieving documents failed with an error
Error: expected '48 available fields. 6 empty fields. 4 meta fields.' to equal '49 available fields. 5 empty fields. 4 meta fields.'
at Assertion.assert (expect.js💯11)
at Assertion.apply (expect.js:227:8)
at Assertion.be (expect.js:69:22)
at Context. (_sidebar.ts:644:83)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at Object.apply (wrap_function.js:73:16)
```
So it seems the newly added field is sometimes listed as `empty` instead
of `available`.
Failure screenshot:
The screenshot shows an active global loading indicator, so that might
be the reason for the flakiness (loading not quite done sometimes, so
field listed as `empty`).
This PR tries to fix this flakiness by adding a wait for the global
loading right before fetching the sidebar details.
## Summary
Renames the variables from ?`start` to `?t_start` and `?end` to `?t_end`
Naming is hard so bare with us 😅 (I think this will be the last change)
## Summary
Renames the variables from `?start` to `?t_start` and `?end` to `?t_end`
Naming is hard so bare with us 😅 (I think this will be the last change)
## Summary
### This PR introduces a new type of API integration tests in FTR:
deployment-agnostic
#### Test suite is considered deployment-agnostic when it fulfils the
following criteria:
**Functionality**: It tests Kibana APIs that are **logically identical
in both stateful and serverless environments** for the same SAML roles.
**Design**: The test design is **clean and does not require additional
logic** to execute in either stateful or serverless environments.
### How It Works
Most existing stateful tests use basic authentication for API testing.
In contrast, serverless tests use SAML authentication with
project-specific role mapping.
Since stateful deployments also support SAML, deployment-agnostic tests
**configure Elasticsearch and Kibana with SAML authentication in both
cases**. For roles, stateful deployments define 'viewer', 'editor', and
'admin' roles with serverless-alike privileges.
New `samlAuth` service has `AuthProvider` interface with 2 different
implementations: depending on environment context (serverless or
stateful) appropriate implementation is used. But it remains on service
level and hidden in test suite.
test example
```
export default function ({ getService }: DeploymentAgnosticFtrProviderContext) {
const samlAuth = getService('samlAuth');
const supertestWithoutAuth = getService('supertestWithoutAuth');
let roleAuthc: RoleCredentials;
let internalHeaders: InternalRequestHeader;
describe('GET /api/console/api_server', () => {
before(async () => {
roleAuthc = await samlAuth.createM2mApiKeyWithRoleScope('admin');
internalHeaders = samlAuth.getInternalRequestHeader();
});
after(async () => {
await samlAuth.invalidateM2mApiKeyWithRoleScope(roleAuthc);
});
it('returns autocomplete definitions', async () => {
const { body } = await supertestWithoutAuth
.get('/api/console/api_server')
.set(roleAuthc.apiKeyHeader)
.set(internalHeaders)
.set('kbn-xsrf', 'true')
.expect(200);
expect(body.es).to.be.ok();
const {
es: { name, globals, endpoints },
} = body;
expect(name).to.be.ok();
expect(Object.keys(globals).length).to.be.above(0);
expect(Object.keys(endpoints).length).to.be.above(0);
});
});
}
```
Please read
[readme](966822ac87/x-pack/test/api_integration/deployment_agnostic/README.md)
for more details and step-by-step guide. It should help migrating
existing serverless tests to deployment-agnostic, assuming requirements
are met.
### Examples
Deployment-agnostic tests:
```
x-pack/test/api_integration/deployment_agnostic/apis/console/spec_definitions.ts
x-pack/test/api_integration/deployment_agnostic/apis/core/compression.ts
x-pack/test/api_integration/deployment_agnostic/apis/painless_lab/painless_lab.ts
```
Configs to run it:
```
node scripts/functional_tests --config x-pack/test/api_integration/deployment_agnostic/oblt.serverless.config.ts
node scripts/functional_tests --config x-pack/test/api_integration/deployment_agnostic/search.serverless.config.ts
node scripts/functional_tests --config x-pack/test/api_integration/deployment_agnostic/security.serverless.config.ts
node scripts/functional_tests --config x-pack/test/api_integration/deployment_agnostic/stateful.config.ts
```
PR is a compact version of #188737 with reduced changes in existing
serverless tests.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: elena-shostak <165678770+elena-shostak@users.noreply.github.com>
Co-authored-by: Aleh Zasypkin <aleh.zasypkin@gmail.com>
- Follow up for https://github.com/elastic/kibana/pull/180536
## Summary
This PR fixes an issue with `rowsPerPage` Advanced Setting: after the
refactoring it was ignored.
To test:
Change `rowsPerPage` on Advanced Setting page, navigate to Dashboard and
add a saved search panel. It should use the configured value by default.
The default value can be overwritten by custom panel settings or saved
search own settings.
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- Closes https://github.com/elastic/kibana/issues/186637
- Closes https://github.com/elastic/kibana/issues/186808
## Summary
- [x] Extend UnifiedDataTable with a new `rowAdditionalLeadingControls`
prop to render additional leading controls
- [x] In case of many actions, collapse the rest of them under "More"
button
- [x] New OneDiscover extension
- [x] Convert from `customControlColumnsConfiguration` to the new prop.
Refactor actions format in Log Explorer.
- [x] Swap the default "select" and "expand" control columns ~if custom
row actions are specified~
- [x] Add to example OneDiscover profile
- [x] Add functional and units tests
<img width="858" alt="Screenshot 2024-07-26 at 16 00 17"
src="https://github.com/user-attachments/assets/68832abc-a498-4ec6-8333-79ad5f83855b">
<img width="1139" alt="Screenshot 2024-07-26 at 16 00 47"
src="https://github.com/user-attachments/assets/8ff99ac1-21b0-4687-b548-fbf3c5517808">
### Testing
For testing the example profile:
- add `discover.experimental.enabledProfiles: ['example-root-profile',
'example-data-source-profile', 'example-document-profile']` to
kibana.dev.yml
- start kibana
- make sure to have an index with `my-example-logs` name or create an
alias to an existing index:
```
POST _aliases
{
"actions": [
{
"add": {
"index": "kibana_sample_data_logs",
"alias": "my-example-logs"
}
}
]
}
```
- create a data view for `my-example-logs` index
### Follow up for Security solution
The following items would require deprecation/refactoring in components
on Security Solution pages to have consistent UX (can result in 500+
lines of code changes):
- Convert from `externalControlColumns` to the new prop
`rowAdditionalLeadingControls`
- Convert from `trailingControlColumns` to a normal column.
`trailingControlColumns` is deprecated.
- https://github.com/elastic/kibana/issues/189294
- Use `getRowIndicator` prop on UnifiedDataTable instead of
`border-left` style
- https://github.com/elastic/kibana/issues/189295
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [x] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Data View field editor will now accept a DataView or DataViewLazy to
help in moving kibana apps over to DataViewLazy and loading fields on an
as-needed basis.
Internally, the field editor's namesNotAllowed list has been removed as
loading it can be expensive. The painless field editor still needs the
full field list for autocomplete suggestions but the list is only loaded
for that specific component.
DataViewLazy provides a method to get a sorted field list - as a result,
number of tests have been updated to accommodate a properly ordered
list.
Part of https://github.com/elastic/kibana/issues/178926
## Summary
1. Apply https://github.com/elastic/kibana/pull/185870 for Security
Solution Explore and Investigation tests.
2. Remove duplicated tests.
3. Investigation/timeline/serverless tests are skipped atm.
How to run the api integration tests (Use Investigation Timeline ESS as
an example)
```
cd x-pack
node scripts/functional_tests_server.js --config ./test/security_solution_api_integration/test_suites/investigation/timeline/trial_license_complete_tier/configs/ess.config.ts
// After server is started, open another terminal
cd x-pack
node ../scripts/functional_test_runner --config=test/security_solution_api_integration/test_suites/investigation/timeline/trial_license_complete_tier/configs/ess.config.ts
```
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Currently, if you attempt to create a data view with a index pattern
where some of the segments aren't matched, you can't select a timestamp
field. This is a common use case for security and observability. This PR
allows a timestamp selection as expected.
Closes https://github.com/elastic/kibana/issues/189024
The EBT pacakge has been moved to a separate NPM package
([@elastic/ebt](https://www.npmjs.com/package/@elastic/ebt))
The npm package is on version `0.0.x` until we finish the reviews then
i'll publish the `1.0.0` version before merging this PR.
The PR is mostly code deletes after moving the code to the public ebt
github repo https://github.com/elastic/ebt
The significant changes are:
1. removed the `packages/analytics/ebt` package from kibana
2. remove @kbn/ebt references in favor of the npm package.
3. Added a util package to provide the package with the telemetry
endpoint and headers
This was previously backed into the package but now i've rewired it be
provided from Kibana, this way we have more control over the URL and
headers we use to send EBT telemetry for our elastic endpoint, which
will probably be different between users of this package and this way
we'll also avoid republishing the package if we ever want to change
these details.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Alejandro Fernández Haro <afharo@gmail.com>
## Summary
I noticed `elastic/appex-qa` is pinged for quite many PRs.
with #188606 some test folders became packages with `"owner":
"@elastic/appex-qa",`, that autmatically updated CODEOWNERS file with
appex-qa listed for basically every test path.
https://github.com/elastic/kibana/pull/188606/files#diff-3d36a1bf06148bc6ba1ce2ed3d19de32ea708d955fed212c0d27c536f0bd4da7R878-R881
This PR removes `owner` for the following test "packages"
- x-pack/test_serverless
- test
- x-pack/test
and CODEOWNERS file keeps these paths without specific owner.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
At the moment, our package generator creates all packages with the type
`shared-common`. This means that we cannot enforce boundaries between
server-side-only code and the browser, and vice-versa.
- [x] I started fixing `packages/core/*`
- [x] It took me to fixing `src/core/` type to be identified by the
`plugin` pattern (`public` and `server` directories) vs. a package
(either common, or single-scoped)
- [x] Unsurprisingly, this extended to packages importing core packages
hitting the boundaries eslint rules. And other packages importing the
latter.
- [x] Also a bunch of `common` logic that shouldn't be so _common_ 🙃
### For maintainers
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Adding generic support for experimental features in timelines
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Similar to how APM is shown as an integration as well, show the new
OTel-based flow on the integrations page so people find it from there as
well:
<img width="991" alt="Screenshot 2024-07-25 at 11 32 46"
src="https://github.com/user-attachments/assets/4d806ed1-4b01-4ac8-985c-0e59708fa4c6">
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Closes https://github.com/elastic/kibana/issues/187587
This PR changes how the dashboard panel selection items get computed, it
had previously been computed eagerly, in this implementation panel
selection items would only be computed when the user actually clicks the
`add panel` button, with it's results cached so that subsequent
interactions with the `add panel` button leverages the already computed
data.
**Notable Mention:**
The options presented as the dashboard panel list now only comprise of
uiActions specifically registered with the uiAction trigger
`ADD_PANEL_TRIGGER` and specific dashboard visualisation types. See
https://github.com/elastic/kibana/pull/187797#discussion_r1681320456 to
follow the reasoning behind this.
That been said adding new panels to the dashboard, would be something
along the following lines;
```ts
import { ADD_PANEL_TRIGGER } from '@kbn/ui-actions-plugin/public';
uiActions.attachAction(ADD_PANEL_TRIGGER, <registredActionId>);
// alternatively
// uiActions.addTriggerAction(ADD_PANEL_TRIGGER, ...someActionDefintion);
````
### Visuals
7c029a64-2cd8-4e3e-af5a-44b6788faa45
### How to test
- Navigate to a dashboard of choice
- Slow down your network speed using your browser dev tools, refresh
your dashboard, and click on the “Add panel” button as soon as it is
available (before the panels have a chance to load).
- You should be presented with a loading indicator, that eventually is
swapped out for the list of panels available for selection.
### Checklist
Delete any items that are not applicable to this PR.
<!--
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials -->
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
<!--
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] If a plugin configuration key changed, check if it needs to be
allowlisted in the cloud and added to the [docker
list](https://github.com/elastic/kibana/blob/main/src/dev/build/tasks/os_packages/docker_generator/resources/base/bin/kibana-docker)
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
### Risk Matrix
Delete this section if it is not applicable to this PR.
Before closing this PR, invite QA, stakeholders, and other developers to
identify risks that should be tested prior to the change/feature
release.
When forming the risk matrix, consider some of the following examples
and how they may potentially impact the change:
| Risk | Probability | Severity | Mitigation/Notes |
|---------------------------|-------------|----------|-------------------------|
| Multiple Spaces—unexpected behavior in non-default Kibana Space.
| Low | High | Integration tests will verify that all features are still
supported in non-default Kibana Space and when user switches between
spaces. |
| Multiple nodes—Elasticsearch polling might have race conditions
when multiple Kibana nodes are polling for the same tasks. | High | Low
| Tasks are idempotent, so executing them multiple times will not result
in logical error, but will degrade performance. To test for this case we
add plenty of unit tests around this logic and document manual testing
procedure. |
| Code should gracefully handle cases when feature X or plugin Y are
disabled. | Medium | High | Unit tests will verify that any feature flag
or plugin combination still results in our service operational. |
| [See more potential risk
examples](https://github.com/elastic/kibana/blob/main/RISK_MATRIX.mdx) |
### For maintainers
- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
-->
