There is a bug with the way we check whether or not to start polling for
blocked jobs. We should only start polling there isn't an existing poll
running.
This causes the polling to run as possible and to refresh the full jobs
list on every check.
Now the blocked jobs polling correctly runs at the interval of 2s and
the full jobs list is only updated when there is a change in the list of
blocked jobs.
Partially fixes https://github.com/elastic/kibana/issues/171626
A more in-depth change as specified in the issue can be done in follow
up work. It's good to get this change in for the next release just in
case we run out of time for the larger change.
## Summary
This PR disables TLS mode for Kibana run in serverless.
Related to #170417 enabling serverless roles testing
Blocked by #171513
PR is created in cooperation with @azasypkin and intended to simplify
the automated testing process for serverless:
starting Kibana with TLS enabled adds unnecessary complexity to the
process of getting session cookie and overall Kibana APIs calling with
Dev certificate in the tests.
The selected approach is to disable TLS for Kibana and simply rely on
#171513 to configure mocked idp realm for Serverless ES with TLS
enabled.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
The package data isn't needed anymore, so we can remove that package. On
top, it seems like the package was causing some issues.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## 📓 Summary
Closes#170721
These changes add icons for the cloud provider and service fields in the
Log detail flyout.
Regarding the Package Icon next to the dataset field, it should require
a remote request to determine what package is part of the current
dataset. We haven't determined this part yet, so we'll keep it for an
upcoming implementation.
fc882810-6b0f-40ee-9727-762f57ad01cc
---------
Co-authored-by: Marco Antonio Ghiani <marcoantonio.ghiani@elastic.co>
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Achyut Jhunjhunwala <achyut.jhunjhunwala@elastic.co>
- Renames references to index patterns to data views in function and
variable names.
- Some inconsistent naming of schemas for data frame analytics was
cleaned up as part of this PR.
- Note this doesn't cover the whole ml owned codebase but just code
related to data frame analytics.
## Summary
Fixes the issue when the message is added as a filter, discover
component surrounds the value with the <mark> tag in order to highlight
the field. Since our message field is inside a `EuiCodeBlock` it cannot
be highlighted and hence should not support adding of this tag.
### Before
### After
## Summary
In this PR we're getting rid of the `enableExperimental` flag
`disableTimelineSaveTour` in favor of a custom localStorage flag in the
cypress tests. This will allow us to run the tests against a production
version of Kibana without setting custom config flags.
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
Part of https://github.com/elastic/kibana/issues/165928
Closes https://github.com/elastic/kibana/issues/144498
Allows the user to edit the ES|QL query from the dashboard. Also allows
the user to select one of the suggestions.
<img width="1886" alt="image"
src="9961c154-e414-4ce1-bff5-33ec5c30db69">
<img width="1883" alt="image"
src="6e8971d3-4a35-466f-804a-b8df58b09394">
### Testing
Navigate to Discover ES|QL mode and save a Lens chart to a dashboard.
Click the edit Visualization.
### Important notes
- We can very easily enable suggestions for the dataview panels but I am
going to do it on a follow up PR to keep this PR clean
- Creation is going to be added on a follow up PR
- Warnings are not rendered in the editor because I am using the limit 0
for performance reasons. We need to find another way to depict them via
the embeddable or store. It will be on a follow up PR.
- Errors are being displayed though. The user is not allowed to apply
the changes when an error occurs.
- Creating ES|QL charts from dashboard will happen to a follow up PR
### Running queries which don't return numeric fields
In these cases (i.e. `from logstash-* | keep clientip` we are returning
a table. I had to change the datatable logic for text based datasource
to not depend to isBucketed flag. This is something we had foreseen from
the [beginning of text based
languages](https://github.com/elastic/kibana/issues/144498)
<img width="1879" alt="image"
src="ca4b66fd-560d-4c1e-881d-b173458a06ae">
### Running queries which return a lot of fields
For queries with many fields Lens is going to suggest a huge table
trying to add the fields to the different dimensions. This is not
something we want:
- not performant
- user possibly will start removing fields from the dimensions
- this table is unreadable
For this reason we decided to select the first 5 fields and then the
user can easily adjust the dimensions with the fields they want.
<img width="1215" alt="image"
src="07d7ee78-0085-41b1-98a0-a77eefbc0dcd">
### Checklist
- [ ] Any text added follows [EUI's writing
guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses
sentence case text and includes [i18n
support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] Any UI touched in this PR is usable by keyboard only (learn more
about [keyboard accessibility](https://webaim.org/techniques/keyboard/))
- [ ] Any UI touched in this PR does not create any new axe failures
(run axe in browser:
[FF](https://addons.mozilla.org/en-US/firefox/addon/axe-devtools/),
[Chrome](https://chrome.google.com/webstore/detail/axe-web-accessibility-tes/lhdoppojpmngadmnindnejefpokejbdd?hl=en-US))
- [ ] This renders correctly on smaller devices using a responsive
layout. (You can test this [in your
browser](https://www.browserstack.com/guide/responsive-testing-on-local-server))
- [ ] This was checked for [cross-browser
compatibility](https://www.elastic.co/support/matrix#matrix_browsers)
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
## Summary
There was a "memory-leak" bug on the `ContractComponents` logic (Get
Started page), which was causing a React error.
<img width="814" alt="render_depth_error"
src="c6353e53-9e4a-4103-b61c-1aaf67cda590">
The bug was caused by the `pipe` call in the `getComponent$` function
returning a new observable every call and the `LandingPageComponent` was
not memorizing it.
I changed the `pipe` call by a regular `asObservable`, and created a
reusable `useContractComponents` hook to unify the implementation, which
also memorizes the observable.
I leveraged the changes to the `ContractComponents` implementation to
solve another problem, allowing the stored components to render normally
as a `<Component />`, receiving props if needed, instead of having to
render always as an `{element}`.
---------
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR adds support for outputs with secrets preconfigured in the
`kibana.yml` config file.
As Kibana needs to compare the value of the secret to manage updates, a
hash of the value is stored in the output's saved object. The
implementation follows [option 2 in Infosec's
recommendations](https://github.com/elastic/infosec/issues/14853#issuecomment-1788705606)
with the Argon2id algorithm.
See [here](https://www.npmjs.com/package/argon2) for information about
the `argon2` Node package and
[here](https://github.com/ranisalt/node-argon2/wiki/Options) for the
config options. Here, `argon2` was configured with the recommended
`m=19456 (19 MiB), t=2, p=1 ` (for some reason, `timeCost` cannot be set
to less than 2).
Closes#166360
### Testing
1. Ensure the [`outputSecretsStorage` experimental feature
](fd4fdb01bc/x-pack/plugins/fleet/common/experimental_features.ts (L26))is
enabled.
2. Add the following to your kibana config:
```
xpack.fleet.outputs:
- id: my-logstash-output-with-a-secret
name: preconfigured logstash output with a secret
type: logstash
hosts: ["localhost:9999"]
ssl:
certificate: xxxxxxxxxx
secrets:
ssl:
key: thisissecret
```
3. Verify the secret has been correctly created, e.g. by issuing a `GET
.fleet-secrets/_search` request in Dev Tools: the secret should be
listed there.
4. Change the preconfigured value and wait for kibana to restart: the
secret should be updated with the new value.
---------
Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: jillguyonnet <jill.guyonnet@gmail.com>
## Summary
New versions of Universal Profiling agent will dynamically check for a
patch of the bug. Adjust the warning text accordingly.
Signed-off-by: Florian Lehner <florian.lehner@elastic.co>
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
In order to capture Telemetry on which of the AI assistant the users are
clicking, we need an identifier so that we can use it to create a funnel
in Full Story.
What better than having an option to pass dataTestSubj from the client
Closes#159340
## Summary
Since the Custom threshold is the default aggregation type, we no longer
need to pass `aggType: CUSTOM_AGGREGATOR` to the rule creation API. I
added this as optional in the schema so the previous rules with this
field will not throw a schema validation error.
## How to test
- Everything should work as before in the custom threshold; the only
difference is that there is no need to provide aggType at the top level.
- Providing `aggType: custom` through API should be OK, and the rule
should work as expected with or without this field.
## Summary
Only set `time_series_metric` and `time_series_dimension` in data stream
mappings if tsdb is enabled.
This fixes an issue with apm package in 8.11.
Steps to reproduce the issue:
1. install apm-8.10.4 (create a 8.10 cluster or upload zip)
2. index a document that has jvm fields, so that dynamic mappings are
created
3. upgrade package to apm-8.11.0
4. bug: can't update mappings with the error `mapper_parsing_exception:
Field [jvm.memory.non_heap.pool.committed] attempted to shadow a
time_series_metric]`
5. expected with the fix: the apm package upgrade succeeds,
`time_series_metrics` is not needed if tsdb is not enabled.
This is happening because apm introduced the mapping of jvm fields in
8.11, so clusters that ingested jvm data in apm 8.10 had those fields
created as runtime fields. When mappings were updated in 8.11 with jvm
fields and `time_series_metric`, elasticsearch gave that error with the
shadowing, probably because the write index had the runtime mappings.
This fix is a change to conditionally add `time_series_metric`, it helps
with apm because they don't use tsdb.
The same issue can technically happen with other packages if they have
runtime fields and add mappings on those in a new version, and they use
tsdb.
Maybe this should be fixed on elasticsearch side.
An alternative solution would be to do a rollover first (so that runtime
fields disappear from write index), and then do the mapping update,
however this wouldn't work in all cases as there is a race condition -
there could be documents indexed after the rollover before the mapping
update.
```
// install apm 8.10.4 - downloaded from kibana 8.10.4 bundled packages
curl -XPOST -H 'content-type: application/zip' -H 'kbn-xsrf: true' http://localhost:5601/julia/api/fleet/epm/packages -u elastic:changeme --data-binary @apm-8.10.4.zip
// index doc
POST metrics-apm.internal-default/_doc
{"metricset":{"name":"app","samples":[{"name":"jvm.memory.non_heap.pool.committed","value":3.407872e+06},{"value":1.073741824e+09,"name":"jvm.memory.non_heap.pool.max"},{"name":"jvm.memory.non_heap.pool.used","value":3.12092e+06}]},"process":{"parent":{"pid":24713},"pid":24715,"title":"/usr/lib/jvm/java-11-openjdk-amd64/bin/java"},"@timestamp":"2023-11-21T16:04:51.071Z","data_stream":{"type":"metrics","dataset":"apm.internal","namespace":"default"},"host":{"os":{"platform":"Linux"},"architecture":"amd64","hostname":"carson-elastic","ip":["127.0.0.1"],"name":"carson-elastic"},"service":{"runtime":{"name":"Java","version":"11.0.20.1"},"language":{"name":"Java","version":"11.0.20.1"},"name":"hello_world","node":{"name":"carson-elastic"}},"agent":{"activation_method":"javaagent-flag","ephemeral_id":"cd44472e-a95d-402b-8c68-ccfa7e37ff93","name":"java","version":"1.43.1-SNAPSHOT.3e2ec51"},"labels":{"name":"Compressed Class Space"},"observer":{"hostname":"carson-elastic","type":"apm-server","version":"8.10.4"}}
// check runtime fields in mappings
GET metrics-apm.internal-default/_mapping
{...
"runtime": {
"jvm.memory.non_heap.pool.committed": {
"type": "double"
},
"jvm.memory.non_heap.pool.max": {
"type": "double"
},
"jvm.memory.non_heap.pool.used": {
"type": "double"
}
},
...}
// upgrade to apm 8.11
curl -XPOST -H 'content-type: application/zip' -H 'kbn-xsrf: true' http://localhost:5601/julia/api/fleet/epm/packages -u elastic:changeme --data-binary @apm-8.11.0.zip
```
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
Renames references to index pattern to data views.
- Updates references in `data-test-subj` attributes for data frame
analytics.
- Renames methods in the ` ml.testResources` service used in tests.
- `IndexPattern` references in testing code referring to single indices
were renamed to use `IndexName`. For variable names still using
`IndexPattern` they were prefixed with e.g. `esIndexPattern` to avoid
ambiguity with the legacy data view name.
Note there are still references in the state management code of the data
frame analytics creation wizard, this wasn't picked up in this PR since
it focuses mostly on test related code.
## Summary
Partially Resolves: https://github.com/elastic/kibana/issues/164255
This pull request is part 1/3 to add scoped queries to maintenance
windows. More specifically, this PR adds the new `scoped_query` field to
the `maintenanceWindow` type and schema. Also adds the `scoped_query`
field to `create/update` maintenance window APIs.
This PR only contains the schema and API component. All changes should
be backwards compatible since the `scoped_query` field is optional. So
this PR can be merged without any dependencies.
The 2 PRs that comes after will be:
- Frontend changes
- Task runner changes
### Checklist
- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
## Summary
This PR removes the api call for ILM on the Reporting management page
for serverless. SInce ILM is not used by reporting in serverless, this
api get call does not need to execute.
### BEFORE
fetch.ts:123 GET
https://localhost:5601/ugz/internal/reporting/ilm_policy_status 404 (Not
Found)
<img width="1444" alt="Screenshot 2023-09-14 at 11 12 48 AM"
src="b695df38-7e3d-451e-8245-7753dd89039f">
This error in the console disappears after these changes.
---------
Co-authored-by: Tim Sullivan <tsullivan@users.noreply.github.com>
