* Refactor es queries and associated components/endpoints.
* Add unit tests, repair broken tests.
* [Uptime] Add API functional tests for uptime graphQL (#29128)
* Add API functional tests for uptime graphQL.
* Remove obsolete code.
* Add CI group for UI functional tests.
* Delete obsolete code, rename heartbeat es archive.
* Refactor adapter methods.
* Refactor adapter methods.
* Attempt to fix ci-group tag error.
* Skip functional app tests until later PR.
* Remove unused code.
* Optimize test runs.
* Add uptime to api test index.
* Fix formatting.
* Add HB 7.0 data for API tests.
* Configure first error_list test to work with 7.x data.
* Configure error_list filtered by id to work with 7.x data.
* Configure error_list functional tests to work with 7.x data.
* Update snapshot test to work with 7.x data.
* Update snapshot down filtered test to work with 7.x data.
* Configure snapshot up test to work with 7.x data.
* Configure ping list tests to work with 7.x data.
* Configure monitor list tests to work with 7.x data.
* Configure monitor status bar tests to work with 7.x data.
* Configure filterBar tests to work with 7.x data.
* Configure docCount tests to work with 7.x data.
* Simplify code based on PR feedback.
* Add loading spinner to monitor page title based on PR feedback.
* Rename GQL type based on PR feedback.
* Remove use of 'undefined' in ES query based on PR feedback.
* Simplify code based on PR feedback.
* Add definite size/shard_size for terms agg based on PR feedback.
* Simplify ES query based on PR feedback.
* Update x-pack/plugins/uptime/server/lib/adapters/monitors/elasticsearch_monitors_adapter.ts
Implement PR feedback.
Co-Authored-By: justinkambic <justin.kambic@elastic.co>
* Increase size for ES errors query based on PR feedback.
* Fix hardcoded field in terms filter based on PR feedback.
* Simplify get code for monitors function.
* Reduce unnecessarily large size for terms agg based on PR feedback.
* Pluralize filter bar props.
* Refactor filter bar query based on PR feedback.
* Update test.
* Fix busted GQL query.
* Update functional test docs to use data without buggy values.
* Update index name in HB functional api test docs.
* Update snapshot base functional test.
* Make snapshot filter tests pass, fix associated bug.
* Configure remaining snapshot e2e tests to work with 7.x data.
* Give better variable names and comments for ugly code.
* Configure ping list query tests to work with updated 7.x data.
* Rename graphql describe block.
* Update monitor status bar query tests to work with updated 7.x data.
* Update monitor list query tests to work with updated 7.x data.
* Update filter bar query to work with updated 7.x data.
* Update error list query to work with updated 7.x data.
* Update doc count fixture to work with new 7.x data.
* Address PR feedback with filter typing to clean up code.
* Add comments based on PR feedback.
* Fix bug introduced in 8856be8e39.
* [visualize] update breadcrumbs when updating visualization
* [ftr] auto-scroll elements passed the fixed header if necessary
* [uiSettings] enable the k7design by default
* [ftr/services/flyout] add ensureClosed() and ensureAllClosed()
* [ftr/services/globalNav] implement globalNav service
* [ftr/services/appsMenu] implement service for using app menu
* [ftr/services/userMenu] add service for using user menu
* [ftr/discover+visualize] update assertions that are based on the app width
* [ftr/monitoring] pass test subjects to new breadcrumbs
* [headerGlobalNav] don't offset the app container in embed mode
* update heights for slightly smaller header
This adds the MVP of the Phase 1 version of the Maps Plugin to Kibana (https://github.com/elastic/kibana/issues/19582).
This is added as a new Stack Feature, requiring a basic license.
* Get basic scaffolding working
* Wire up cluster checkup data
* Add types for elasticsearch plugin
* Implement basics of checkup tab
* Update style of deprecations
* Add copy and reload button to checkup
* Add filtering by warning level
* Add deprecation logging tab
* Copy updates and cleanup
* Type cleanup
* Move deprecation logging to overview tab
* Make filters togglable
* Move sections into tabs and add support for grouping
* Cleanup and add clearer labels
* Use tables for message grouping
* Cleanup and small fixes
* Allow console to load relative URLs
* Add reindex in console button to reindex tasks
* Merge documentation UI and uiButtons
* Fix tests
* Filter bar tweaks
* Filter out index settings that can't be set
* Fix types
* Add tests for deprecation_logging
* Add tests for reindex templates
* Make KibanaConfig generic
* Simplify integration test
* Finish backend unit tests
* Fixup types
* Fix uiButton updating for reindex items
* Fixed background color stretching
* Pulling tabs out and re-ordering filter buttons
* Making accordions more item-list like
* Turned Healths into Badges
- Couldn’t do the conversion within the cell because it only passed color
* Fix overflow issue
* Optional filter and expand/collapse controls
* Reorganizing
- Added placeholder for moving action button up into accordion header
- Removed repetitive message name outputs
- Slightly better listing of each message when sorting by index
- Only showing number of severity when sorting by index
- Still need to allow showing all severity levels
- Added indice count when sorting by issue
* Putting `Deprecation logging` in a `EuiDescribedFormGroup`
* Added some stats, empty prompts, and all clear prompt
* Added docs link
* Cleaned up sass files
* Revert changes to fake_deprecations
* Update blacklisted settings
* wip
* Move data fetching and tab control
* Wire up overview summary
* Cleanup docs/uiButtons + move actions to index table
* Add expand/collapse all functionality
* Wire up search box
* Wire up severity indicators
* Fix types
* Round out functional tests
* Fix fake data
* Remove info deprecation level
* Fix extra space on cluster tab control bar
* Cleanup code and localize majority of UI controls
* Change overview tab to steps layout
* Update copy
* Localize overview tab
* Complete localization of checkup tabs
* Make ES version dynamic based on branch
* Add pagination to checkup tabs
* Rename checkup -> assistant
* Cleanup filter and group by bars
* WIP UI unit tests
* Copy tweaks
* Fix i18n formatting issues
* Update tests for copy
* Add tests for remaining UI
* Fix pagination w/ filter changes + table button color
* Small cleanup
* Add reindex button to old index deprecations
* Add shrunken indices setting to copy settings blacklist for #18469
* Add next steps to overview tab + update copy
* Remove usage of migration assistance API
* Use all/critical toggle for filter buttons
* Cloud upgrade copy
* Translate reindex button
* Remove hacked EUI type
* Show incomplete banner on all tabs
* Update copy for waiting for next version
* Review comments
* Update deprecation level type
* Update checkup tab snapshots
* Remove dependencies on types from #25168
* Use types from new global type defs
* Remove 'Reindex in Console' button
* Remove unused variable
* Refactor BeatsCM
* update deps
* update more deps
* update for new EUI definitions
* update import
* Revert "update deps"
This reverts commit 759a14561d.
* use _source_includes
* remove _source_includes
* work-around due to watcher UI tests
* Keep all xpack checks safe because we cant trust its there in tests for some reason
* VALIDATION. This commit is to ensure the errors in CI are coming from beats
* remove validation that this is a beats CM issue
* More try/catch to try and find where this error is
* testing another call
* revert back to dangerouslyGetActiveInjector
* ensure expire always is a number
* fix swallowed error
* Update x-pack/plugins/beats_management/public/lib/compose/kibana.ts
Co-Authored-By: mattapperson <me@mattapperson.com>
* Update x-pack/plugins/beats_management/public/utils/page_loader.test.ts
Co-Authored-By: mattapperson <me@mattapperson.com>
* Update x-pack/plugins/beats_management/public/utils/page_loader.ts
Co-Authored-By: mattapperson <me@mattapperson.com>
* fix for new webpack import
* Fix translation map
* fix URL path
* fix other link
* removing tag from beats via tag details screen now uses container
* remove debug text
* added comment/readme about routing on the client side
* enrolled beat UI now works on overview screen
* newly enrolled beat now reloads the beats table
* fix TS errors
* Make saved object client error while Kibana index is migrating
* Tidy up a bit, and refactor the way the `isMigrated` check is accessed
* Remove unused interface declaration
* Remove default migrator from saved objects repository constructor
* Fix repository migrator isComplete check
* Wrap callCluster and delay it until migrations have completed...
* Fix inaccurate comment
* Ensure migrations wait for elasticsearch to go green prior to running
* Reenabling tests
* Add tests for callCluster being wrapped in the repository, fix
the es_archiver's call to migrate index.
* Fixing esArchiver's usage of migrations
* Disabling spaces for the phanton api BWC tests
* don't throw if authorization mode is already initialized
* Adding spaces to the reporting historical archives
* Loading empty_kibana for grok debugger tests
* Enabling reporting tests
* Altering the method in which we logout users to be more fault tolerant
* Actually doing what I said before...
* Skipping Dashboard Preserve Layout, it likes to fail a lot
* Skipping dashboard view mode tests
* Putting logout back how it was, trying to make the security tests run
properly when we don't have dashboard mode tests
* Running subsection of tests that are failing
* Don't bail, run them all
* Disabling canvas, breaks logout
* Fixing spaces create legacy error assertion
* Putting comment about why we're disabling spaces for the functional
tests
### Review notes
This is generally ready for review. We are awaiting https://github.com/elastic/elasticsearch/issues/32777 to improve handling when users do not have any access to Kibana, but this should not hold up the overall review for this PR.
This PR is massive, there's no denying that. Here's what to focus on:
1) `x-pack/plugins/spaces`: This is, well, the Spaces plugin. Everything in here is brand new. The server code is arguably more important, but feel free to review whatever you see fit.
2) `x-pack/plugins/security`: There are large and significant changes here to allow Spaces to be securable. To save a bit of time, you are free to ignore changes in `x-pack/plugins/security/public`: These are the UI changes for the role management screen, which were previously reviewed by both us and the design team.
3) `x-pack/test/saved_object_api_integration` and `x-pack/test/spaces_api_integration`: These are the API test suites which verify functionality for:
a) Both security and spaces enabled
b) Only security enabled
c) Only spaces enabled
What to ignore:
1) As mentioned above, you are free to ignore changes in `x-pack/plugins/security/public`
2) Changes to `kibana/src/server/*`: These changes are part of a [different PR that we're targeting against master](https://github.com/elastic/kibana/pull/23378) for easier review.
## Saved Objects Client Extensions
A bulk of the changes to the saved objects service are in the namespaces PR, but we have a couple of important changes included here.
### Priority Queue for wrappers
We have implemented a priority queue which allows plugins to specify the order in which their SOC wrapper should be applied: `kibana/src/server/saved_objects/service/lib/priority_collection.ts`. We are leveraging this to ensure that both the security SOC wrapper and the spaces SOC wrapper are applied in the correct order (more details below).
### Spaces SOC Wrapper
This wrapper is very simple, and it is only responsible for two things:
1) Prevent users from interacting with any `space` objects (use the Spaces client instead, described below)
2) Provide a `namespace` to the underlying Saved Objects Client, and ensure that no other wrappers/callers have provided a namespace. In order to accomplish this, the Spaces wrapper uses the priority queue to ensure that it is the last wrapper invoked before calling the underlying client.
### Security SOC Wrapper
This wrapper is responsible for performing authorization checks. It uses the priority queue to ensure that it is the first wrapper invoked. To say another way, if the authorization checks fail, then no other wrappers will be called, and the base client will not be called either. This wrapper authorizes users in one of two ways: RBAC or Legacy. More details on this are below.
### Examples:
`GET /s/marketing/api/saved_objects/index-pattern/foo`
**When both Security and Spaces are enabled:**
1) Saved objects API retrieves an instance of the SOC via `savedObjects.getScopedClient()`, and invokes its `get` function
2) The Security wrapper is invoked.
a) Authorization checks are performed to ensure user can access this particular saved object at this space.
3) The Spaces wrapper is invoked.
a) Spaces applies a `namespace` to be used by the underlying client
4) The underlying client/repository are invoked to retrieve the object from ES.
**When only Spaces are enabled:**
1) Saved objects API retrieves an instance of the SOC via `savedObjects.getScopedClient()`, and invokes its `get` function
2) The Spaces wrapper is invoked.
a) Spaces applies a `namespace` to be used by the underlying client
3) The underlying client/repository are invoked to retrieve the object from ES.
**When only Security is enabled:**
(assume `/s/marketing` is no longer part of the request)
1) Saved objects API retrieves an instance of the SOC via `savedObjects.getScopedClient()`, and invokes its `get` function
2) The Security wrapper is invoked.
a) Authorization checks are performed to ensure user can access this particular saved object globally.
3) The underlying client/repository are invoked to retrieve the object from ES.
## Authorization
Authorization changes for this project are centered around Saved Objects, and builds on the work introduced in RBAC Phase 1.
### Saved objects client
#### Security without spaces
When security is enabled, but spaces is disabled, then the authorization model behaves the same way as before: If the user is taking advantage of Kibana Privileges, then we check their privileges "globally" before proceeding. A "global" privilege check specifies `resources: ['*']` when calling the [ES _has_privileges api.](https://www.elastic.co/guide/en/elasticsearch/reference/current/security-api-has-privileges.html). Legacy users (non-rbac) will continue to use the underlying index privileges for authorization.
#### Security with spaces
When both plugins are enabled, then the authorization model becomes more fine-tuned. Rather than checking privileges globally, the privileges are checked against a specific resource that matches the user's active space. In order to accomplish this, the Security plugin needs to know if Spaces is enabled, and if so, it needs to ask Spaces for the user's active space. The subsequent call to the `ES _has_privileges api` would use `resources: ['space:marketing']` to verify that the user is authorized at the `marketing` space. Legacy users (non-rbac) will continue to use the underlying index privileges for authorization. **NOTE** The legacy behavior implies that those users will have access to all spaces. The read/write restrictions are still enforced, but there is no way to restrict access to a specific space for legacy auth users.
#### Spaces without security
No authorization performed. Everyone can access everything.
### Spaces client
Spaces, when enabled, prevents saved objects of type `space` from being CRUD'd via the Saved Objects Client. Instead, the only "approved" way to work with these objects is through the new Spaces client (`kibana/x-pack/plugins/spaces/lib/spaces_client.ts`).
When security is enabled, the Spaces client performs its own set of authorization checks before allowing the request to proceed. The Spaces client knows which authorization checks need to happen for a particular request, but it doesn't know _how_ to check privileges. To accomplish this, the spaces client will delegate the check security's authorization service.
#### FAQ: Why oh why can't you used the Saved Objects Client instead!?
That's a great question! We did this primarily to simplify the authorization model (at least for our initial release). Accessing regular saved objects follows a predictible authorization pattern (described above). Spaces themselves inform the authorization model, and this interplay would have greatly increased the complexity. We are brainstorming ideas to obselete the Spaces client in favor of using the Saved Objects Client everywhere, but that's certainly out of scope for this release.
## Test Coverage
### Saved Objects API
A bulk of the changes to enable spaces are centered around saved objects, so we have spent a majority of our time automating tests against the saved objects api.
**`x-pack/test/saved_object_api_integration/`** contains the test suites for the saved objects api. There is a `common/suites` subfolder which contains a bulk of the test logic. The suites defined here are used in the following test configurations:
1) Spaces only: `./spaces_only`
2) Security and spaces: `./security_and_spaces`
3) Security only: `./security_only`
Each of these test configurations will start up ES/Kibana with the appropriate license and plugin set. Each set runs through the entire test suite described in `common/suites`. Each test with in each suite is run multiple times with different inputs, to test the various permutations of authentication, authorization type (legacy vs RBAC), space-level privileges, and the user's active space.
### Spaces API
Spaces provides an experimental public API.
**`x-pack/test/spaces_api_integration`** contains the test suites for the Spaces API. Similar to the Saved Objects API tests described above, there is a `common/suites` folder which contains a bulk of the test logic. The suites defined here are used in the following test configurations:
1) Spaces only: `./spaces_only`
2) Security and spaces: `./security_and_spaces`
### Role Management UI
We did not provide any new functional UI tests for role management, but the existing suite was updated to accomidate the screen rewrite.
We do have a decent suite of jest unit tests for the various components that make up the new role management screen. They're nested within `kibana/x-pack/plugins/security/public/views/management/edit_role`
### Spaces Management UI
We did not provide any new functional UI tests for spaces management, but the components that make up the screens are well-tested, and can be found within `kibana/x-pack/plugins/spaces/public/views/management/edit_space`
### Spaces Functional UI Tests
There are a couple of UI tests that verify _basic_ functionality. They assert that a user can login, select a space, and then choose a different space once inside: `kibana/x-pack/test/functional/apps/spaces`
## Reference
Notable child PRs are listed below for easier digesting. Note that some of these PRs are built on other PRs, so the deltas in the links below may be outdated. Cross reference with this PR when in doubt.
### UI
- Reactify Role Management Screen: https://github.com/elastic/kibana/pull/19035
- Space Aware Privileges UI: https://github.com/elastic/kibana/pull/21049
- Space Selector (in Kibana Nav): https://github.com/elastic/kibana/pull/19497
- Recently viewed Widget: https://github.com/elastic/kibana/pull/22492
- Support Space rename/delete: https://github.com/elastic/kibana/pull/22586
### Saved Objects Client
- ~~Space Aware Saved Objects: https://github.com/elastic/kibana/pull/18862~~
- ~~Add Space ID to document id: https://github.com/elastic/kibana/pull/21372~~
- Saved object namespaces (supercedes #18862 and #21372): https://github.com/elastic/kibana/pull/22357
- Securing saved objects: https://github.com/elastic/kibana/pull/21995
- Dedicated Spaces client (w/ security): https://github.com/elastic/kibana/pull/21995
### Other
- Public Spaces API (experimental): https://github.com/elastic/kibana/pull/22501
- Telemetry: https://github.com/elastic/kibana/pull/20581
- Reporting: https://github.com/elastic/kibana/pull/21457
- Spencer's original Spaces work: https://github.com/elastic/kibana/pull/18664
- Expose `spaceId` to "Add Data" tutorials: https://github.com/elastic/kibana/pull/22760Closes#18948
"Release Note: Create spaces within Kibana to organize dashboards, visualizations, and other saved objects. Secure access to each space when X-Pack Security is enabled"
* [Tests] Add http integration test setup
* Base path tests
* SSL tests
* Eslint fixes
* Remove env from config schema
* Rename folders so no_rewrite and rewrite match configs/tests
* wip
* Use self-signed cert for SSL test
* Improve basepath tests
* Run base path proxy server in dev mode for now
* Remove env from x-pack reporting config
* Remove redundant base-path tests
* Test SSL with redirectHttpFromPort set
* Test SSL with redirectHttpFromPort set
* Flesh out comments
* Remove some cruft
* Add SSL tests to CI run
* Move all reporting tests into their own folder to allow for multiple kibana.yml configuration tests, including chromium
* Add debugging and try to skip other tests an jenkins to speed things up
* More debug output
* more logging (remove other line which failed on jenkins)
* Remove no sandbox flag, it doesn't help
* Add fix for socket hangup and clean up tests
* fix path to logstash_functional
* Extend timeout for chromium, add verbose logging, add better comment, conditionally output curl command
* fix path... again
* Ah, other functional tests still need access to reporting page object, put it back
* fix sp err
* Add debug logs for screenshot stitching for png.bitblt error
* Fix tests that don't pass logger to screenshotStitcher
* Fix logger being undefined
* Add more debug output
* png has data, too much info to spit out
* Add comment with link to issue for extra debug messages so they can be left in since so many passes
* Dont use spawnSync with curl, use http.request instead, more support for it
* Comment out chromium tests for now to avoid flakiness
* Wait... lets at least make sure the other fix worked (the http.request instead of spawnsync and curl)
* New http.request code doesn't seem to work on jenkins, timing out after 10 secs maybe
* go back to spawnsync to see if it's an issue with the rxjs code or the http.request code
* I think I figured it out...
* Comment out tests to avoid flaky png error in the screenshot stitcher
* Use a const for OSS archive path
* use path.resolve correctly.
* [XpackMain] Add _xpack/usage API
* add xpack usage http api integration test
* comment
* misc test describe fixes
* fix integration test
* fix reply called twice
* enable api test
* enable kibana collection for integration test to work
* throw error comment
* Revert "[DOCS] Removes redundant index.asciidoc files (#19192)"
This reverts commit d11b5aae9a.
* Revert "[typescript] add typescript support for the server and browser (#19104)"
This reverts commit c6112067fc.
* Revert "Option to run kibana from build for CI (#19125)"
This reverts commit 5969860303.
Restructure testing with kbn-test package
- Run with multiple configs, move cli options to config
- Package-ify kbn-test
- Eventually we'll have functional_test_runner live in a package
of its own, and then this kbn-test will use that as a dependency,
probably still as a devDependency.
- Implement functional_tests_server
- Collapse single and multiple config apis into one command
Use kbn-es
Replace es_test_cluster + es_test_config with kbn/test utils
Implement new createEsTestCluster
Improve scripts, jsdocs, cli top-level tools
Lift error handling to the top level
