Commit graph

28 commits

Author SHA1 Message Date
Cristina Amico
b12f70800c
[Fleet] Add support for non-superuser access to Fleet and Integrations (#122347)
* [Fleet] Split Fleet and Integration privileges

* Update UI when Fleet has All privileges and Integrations have None

* Replace remaining superuser checks

* Updates to server/plugin

* Update getAuthzFromRequest

* Update start method in the client side

* Fix tests

* Fix functional tests

* Make changes to the UI based on new privilege system

* Further UI changes

* Make capabilities accessible to unit tests in createStartServices

* Fix failing tests

* Fix ts checks

* Address most review comments

* Introduce hook exposing authz and make UI checks more granular; address rest of comments

* Remove capabilities hook

* Get rid of useCapabilites

* Address review comments

* Other fixes

* Fix tutorial app privileges

* Address code review comments and update privileges naming

* Fix i18n failing check

* Block fleet server setup UI  when user does not have manage_service_account privilege

* Minor changes

* Use unique i18n id

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Josh Dover <doverfake@elastic.co>
2022-01-31 18:59:18 +01:00
Felix Stürmer
125c569944
[RAC] [Observability] Enable the observability alerting and cases features (#115785)
Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-10-25 15:19:59 +02:00
Xavier Mouligneau
3b958e76aa
[Security Solution] Top-level Cases feature under the Security (#112980)
* add the new top cases feature in security

* fix api intyegration and cypress

* fix api integration

* fix cypress roles test

* missing api integration

* review Joe

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
2021-09-30 20:18:25 -04:00
Felix Stürmer
238c61de07
[RAC] [Observability] Remove rac feature flags from default functional tests (#111942) 2021-09-15 11:17:52 +02:00
Uladzislau Lasitsa
70090e326c
Timelion App removal (#110255)
* Remove timelion app and stuff which related to it

* Fix CI

* Fix lint

* Fix tests

* Fix tests

* Fis tests

* Fix some comments

* Clean up

* fix CI

* fix some comments

* Fix deprecation examples

* Return `enabled` property in config for timelion vis

* Remove unused angular lib

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Alexey Antonov <alexwizp@gmail.com>
2021-09-10 14:53:07 +03:00
Patryk Kopyciński
9edcf9e71e
[Osquery] RBAC (#106669) 2021-08-10 16:36:27 +02:00
Steph Milovic
de07e98663
[Observability] [Cases] Cases in the observability app (#101487) 2021-06-10 16:39:27 -06:00
Brandon Kobel
4584a8b570
Elastic License 2.0 (#90099)
* Updating everything except the license headers themselves

* Applying ESLint rules

* Manually replacing the stragglers
2021-02-03 18:12:39 -08:00
Nicolas Chaulet
1b6cfe819d
[Fleet] Rename ingestManager plugin ID fleet (#83200) 2020-11-19 08:43:14 -05:00
Pierre Gayvallet
d4b2a5145a
SavedObjects tagging MVP (#79096)
* create xpack plugin skeleton, start to implement management section

* add tag creation modal

* first implementation of the tags table

* use InMemoryTable

* add edit modal and delete action

* update plugin list

* add tag list, fix types

* add capabilities check on client-side

* add tag combo box component

* add missing i18n keys

* fix privilege FTR tests

* add base structure for FTR tests

* fix feature ftr test

* use string literals for i18n

* create savedObjectsTaggingOss plugin, move API types to oss plugin, start to wire to SO management page.

* update plugin list

* fix types

* allow to use `_find` with multiple references

* add FTR test for _find API on references fields

* add _find integration tests

* update generated doc

* start to implement tag filtering on SO management section

* update generated docs

* wire tagging API to dashboard listing page

* fix i18n namespace

* fix type & tests

* update dashboard listing snapshots

* adapt FTR listingTable service to search for parsable queries

* wite tagging API to visualize listing

* update tagging plugin limits

* add server-side and client-side validation for tag create/edit

* rename title field to name

* fix types

* fix types bis

* add removeReferencesTo API to SOR/SOC

* update generated doc

* add server-side unit test for `savedObjectsTagging` plugin

* move tagging API types to its own file

* add savedObjectsTaggingOss mock

* add tags_cache tests

* add tests for client-side tag client

* extract uiApi to distinct files

* various API improvements

* add more tests

* add link between tag and so management sections + add connection counts

* add base functional test suite for tagging

* add more FTR tests

* improve feature control func test

* update codeowners

* update generated doc

* fix access to proxy modal

* adapt SO save modal to allow to add tag field

* add SO decorator registry and tag implementation

* add unit tests for SO tag decorator

* add functional tests for visualize integration

* add tag SO read permission for vis/dash feature

* add RBAC api integ tests

* add API integration tests

* add test for getTagConnectionsUrl

* add SOM test suite

* add dashboard integration suite

* remove test line

* add missing unit tests

* improve API types doc

* fix create modal save button label

* remove console.log

* improve doc

* self review

* add refresh interval for tag cache

* improve page object doc

* minor cleanup

* address review comments

* small layout fixes

* add initial focus

* use lazy accessor for tag request handler context

* adapt SOM export and export route to handle references

* remove icon from feature config due to master changes

* fix SO table tests

* update generated docs

* sort tags by name in filter dropdown and listing component

* wire SO tagging to dashboard save modal

* fix types

* - add 'create tag' action in tag selector
- add notifications on update/create/delete from management
- delete modal wording

* add description max length validation

* remove real-time validation

* fix i18n bundle id

* update expected size of savedObjectsTagging plugin

* use own useIfMounted

* update limit again, contract components cannot be lazy loaded atm.

* math is hard

* remove single usage of lodash for bundle size

* add async imports for create/edit modal

* add FTR test for 'create tag' action from tag selector

* allow 'create new' option to prepopulate name field

* extract savedObjectToTag

* add advancedSettings read user for security api_integ suite

* add audit login for security client wrapper

* use import type when possible

* wire SO tagging to lens visualization

* fix lens jest test

* Fix `create tag` option being selected when closing the selector dropdown

* add sorting to tag column from getTableColumnDef

* address some of restrry comments

* rename tag selector's setSelected option to onTagsSelected

* fix audit logging even type for saved_object_remove_references

* update plugin size limit to current size

* adapt maxlength validation wording

* remove selection column until we have batch action menu

* remove connections link when user lack read privilege to savedObjectManagement

* forbid registering multiple SO decorators with the same priority

* add so decorator test

* extract getTagFindReferences and create API mock

* update audit-logging ascidoc

* doc nit

* throw conflict error if update returns any failure

* use refresh=true as default

* wording nits

* export: rename `references` to `hasReference`

* update generated doc

* set description max length to 100

* do not initialize tag cache on anonymous pages

* split fetchObjectsToExport into two distinct functions

* change tag client `delete` call order

* tsdoc nits

* more nits

* add README for oss plugin

* add oss plugin start tests

* SavedObject.find: rename `references` to `hasReference`

* change section description label

* remove url prefix constants

* last nits and comments

* update generated doc
2020-11-03 10:33:18 +01:00
Thomas Watson
e31ec7eb54
Give user the option to log out if they encounter a 403 (#75538) 2020-10-06 20:40:28 +02:00
Gidi Meir Morris
25c5daa83a
Renames "Built-In Alerts" feature to "Stack Alerts" and "Actions" feature to "Actions and Connectors" (#79513)
Renames "Built-In Alerts" feature to "Stack Alerts" and "Actions" feature to "Actions and Connectors" as we've decided these names make more appropriate and better communicate what these features are.
2020-10-06 17:51:05 +01:00
Larry Gregory
2e34eb239f
Hide management sections based on cluster/index privileges (#67791)
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-09-14 09:30:47 -04:00
Gidi Meir Morris
4abe864f10
Adds Role Based Access-Control to the Alerting & Action plugins based on Kibana Feature Controls (#67157)
This PR adds _Role Based Access-Control_ to the Alerting framework & Actions feature using  Kibana Feature Controls, addressing most of the Meta issue: https://github.com/elastic/kibana/issues/43994

This also closes https://github.com/elastic/kibana/issues/62438

This PR includes the following:

1. Adds `alerting` specific Security Actions (not to be confused with Alerting Actions) to the `security` plugin which allows us to assign alerting specific privileges to users of other plugins using the `features` plugin.
2. Removes the security wrapper from the savedObjectsClient in AlertsClient and instead plugs in the new AlertsAuthorization which performs the privilege checks on each api call made to the AlertsClient.
3. Adds privileges in each plugin that is already using the Alerting Framework which mirror (as closely as possible) the existing api-level tag-based privileges and plugs them into the AlertsClient.
4. Adds feature granted privileges arounds Actions (by relying on Saved Object privileges under the hood) and plugs them into the ActionsClient
5. Removes the legacy api-level tag-based privilege system from both the Alerts and Action HTTP APIs
2020-07-22 14:45:57 +01:00
Constance
f7b5144e1d
New Enterprise Search Kibana plugin (#66922)
* Initial App Search in Kibana plugin work

- Initializes a new platform plugin that ships out of the box w/ x-pack
- Contains a very basic front-end that shows AS engines, error states, or a Setup Guide
- Contains a very basic server that remotely calls the AS internal engines API and returns results

* Update URL casing to match Kibana best practices

- URL casing appears to be snake_casing, but kibana.json casing appears to be camelCase

* Register App Search plugin in Home Feature Catalogue

* Add custom App Search in Kibana logo

- I haven't had much success in surfacing a SVG file via a server-side endpoint/URL, but then I realized EuiIcon supports passing in a ReactElement directly. Woo!

* Fix appSearch.host config setting to be optional

- instead of crashing folks on load

* Rename plugin to Enterprise Search

- per product decision, URL should be enterprise_search/app_search and Workplace Search should also eventually live here
- reorganize folder structure in anticipation for another workplace_search plugin/codebase living alongside app_search
- rename app.tsx/main.tsx to a standard top-level index.tsx (which will contain top-level routes/state)
- rename AS->ES files/vars where applicable
- TODO: React Router

* Set up React Router URL structure

* Convert showSetupGuide action/flag to a React Router link

- remove showSetupGuide flag
- add a new shared helper component for combining EuiButton/EuiLink with React Router behavior (https://github.com/elastic/eui/blob/master/wiki/react-router.md#react-router-51)

* Implement Kibana Chrome breadcrumbs

- create shared helper (WS will presumably also want this) for generating EUI breadcrumb objects with React Router links+click behavior
- create React component that calls chrome.setBreadcrumbs on page mount
- clean up type definitions - move app-wide props to IAppSearchProps and update most pages/views to simply import it instead of calling their own definitions

* Added server unit tests (#2)

* Added unit test for server

* PR Feedback

* Refactor top-level Kibana props to a global context state

- rather them passing them around verbosely as props, the components that need them should be able to call the useContext hook

+ Remove IAppSearchProps in favor of IKibanaContext

+ Also rename `appSearchUrl` to `enterpriseSearchUrl`, since this context will contained shared/Kibana-wide values/actions useful to both AS and WS

* Added unit tests for public (#4)

* application.test.ts

* Added Unit Test for EngineOverviewHeader

* Added Unit Test for generate_breadcrumbs

* Added Unit Test for set_breadcrumb.tsx

* Added a unit test for link_events

- Also changed link_events.tsx to link_events.ts since it's just TS, no
React
- Modified letBrowserHandleEvent so it will still return a false
boolean when target is blank

* Betterize these tests

Co-Authored-By: Constance <constancecchen@users.noreply.github.com>

Co-authored-by: Constance <constancecchen@users.noreply.github.com>

* Add UI telemetry tracking to AS in Kibana (#5)

* Set up Telemetry usageCollection, savedObjects, route, & shared helper

- The Kibana UsageCollection plugin handles collecting our telemetry UI data (views, clicks, errors, etc.) and pushing it to elastic's telemetry servers
- That data is stored in incremented in Kibana's savedObjects lib/plugin (as well as mapped)
- When an end-user hits a certain view or action, the shared helper will ping the app search telemetry route which increments the savedObject store

* Update client-side views/links to new shared telemetry helper

* Write tests for new telemetry files

* Implement remaining unit tests (#7)

* Write tests for React Router+EUI helper components

* Update generate_breadcrumbs test

- add test suite for generateBreadcrumb() itself (in order to cover a missing branch)
- minor lint fixes
- remove unnecessary import from set_breadcrumbs test

* Write test for get_username util

+ update test to return a more consistent falsey value (null)

* Add test for SetupGuide

* [Refactor] Pull out various Kibana context mocks into separate files

- I'm creating a reusable useContext mock for shallow()ed enzyme components
+ add more documentation comments + examples

* Write tests for empty state components

+ test new usecontext shallow mock

* Empty state components: Add extra getUserName branch test

* Write test for app search index/routes

* Write tests for engine overview table

+ fix bonus bug

* Write Engine Overview tests

+ Update EngineOverview logic to account for issues found during tests :)
  - Move http to async/await syntax instead of promise syntax (works better with existing HttpServiceMock jest.fn()s)
  - hasValidData wasn't strict enough in type checking/object nest checking and was causing the app itself to crash (no bueno)

* Refactor EngineOverviewHeader test to use shallow + to full coverage

- missed adding this test during telemetry work
- switching to shallow and beforeAll reduces the test time from 5s to 4s!

* [Refactor] Pull out React Router history mocks into a test util helper

+ minor refactors/updates

* Add small tests to increase branch coverage

- mostly testing fallbacks or removing fallbacks in favor of strict type interface
- these are slightly obsessive so I'd also be fine ditching them if they aren't terribly valuable

* Address larger tech debt/TODOs (#8)

* Fix optional chaining TODO

- turns out my local Prettier wasn't up to date, completely my bad

* Fix constants TODO

- adds a common folder/architecture for others to use in the future

* Remove TODO for eslint-disable-line and specify lint rule being skipped

- hopefully that's OK for review, I can't think of any other way to sanely do this without re-architecting the entire file or DDoSing our API

* Add server-side logging to route dependencies

+ add basic example of error catching/logging to Telemetry route
+ [extra] refactor mockResponseFactory name to something slightly easier to read

* Move more Engines Overview API logic/logging to server-side

- handle data validation in the server-side
- wrap server-side API in a try/catch to account for fetch issues
- more correctly return 2xx/4xx statuses and more correctly deal with those responses in the front-end
- Add server info/error/debug logs (addresses TODO)
- Update tests + minor refactors/cleanup
    - remove expectResponseToBe200With helper (since we're now returning multiple response types) and instead make mockResponse var name more readable
    - one-line header auth
    - update tests with example error logs
    - update schema validation for `type` to be an enum of `indexed`/`meta` (more accurately reflecting API)

* Per telemetry team feedback, rename usageCollection telemetry mapping name to simpler 'app_search'

- since their mapping already nests under 'kibana.plugins'
- note: I left the savedObjects name with the '_telemetry' suffix, as there very well may be a use case for top-level generic 'app_search' saved objects

* Update Setup Guide installation instructions (#9)

Co-authored-by: Chris Cressman <chris@chriscressman.com>

* [Refactor] DRY out route test helper

* [Refactor] Rename public/test_utils to public/__mocks__

- to better follow/use jest setups and for .mock.ts suffixes

* Add platinum licensing check to Meta Engines table/call (#11)

* Licensing plugin setup

* Add LicensingContext setup

* Update EngineOverview to not hit meta engines API on platinum license

* Add Jest test helpers for future shallow/context use

* Update plugin to use new Kibana nav + URL update (#12)

* Update new nav categories to add Enterprise Search + update plugin to use new category

- per @johnbarrierwilson and Matt Riley, Enterprise Search should be under Kibana and above Observability
- Run `node scripts/check_published_api_changes.js --accept` since this new category affects public API

* [URL UPDATE] Change '/app/enterprise_search/app_search' to '/app/app_search'

- This needs to be done because App Search and Workplace search *have* to be registered as separate plugins to have 2 distinct nav links
- Currently Kibana doesn't support nested app names (see: https://github.com/elastic/kibana/issues/59190) but potentially will in the future

- To support this change, we need to update applications/index.tsx to NOT handle '/app/enterprise_search' level routing, but instead accept an async imported app component (e.g. AppSearch, WorkplaceSearch).
- AppSearch should now treat its router as root '/' instead of '/app_search'

- (Addl) Per Josh Dover's recommendation, switch to `<Router history={params.history}>` from `<BrowserRouter basename={params.appBasePath}>` since they're deprecating appBasePath

* Update breadcrumbs helper to account for new URLs

- Remove path for Enterprise Search breadcrumb, since '/app/enterprise_search' will not link anywhere meaningful for the foreseeable future, so the Enterprise Search root should not go anywhere
- Update App Search helper to go to root path, per new React Router setup

Test changes:
- Mock custom basepath for App Search tests
- Swap enterpriseSearchBreadcrumbs and appSearchBreadcrumbs test order (since the latter overrides the default mock)

* Add create_first_engine_button telemetry tracking to EmptyState

* Switch plugin URLs back to /app/enterprise_search/app_search

Now that https://github.com/elastic/kibana/pull/66455 has been merged in 🎉

* Add i18n formatted messages / translations (#13)

* Add i18n provider and formatted/i18n translated messages

* Update tests to account for new I18nProvider context + FormattedMessage components

- Add new mountWithContext helper that provides all contexts+providers used in top-level app
- Add new shallowWithIntl helper for shallow() components that dive into FormattedMessage

* Format i18n dates and numbers

+ update some mock tests to not throw react-intl invalid date messages

* Update EngineOverviewHeader to disable button on prop

* Address review feedback (#14)

* Fix Prettier linting issues

* Escape App Search API endpoint URLs

- per PR feedback
- querystring should automatically encodeURIComponent / escape query param strings

* Update server plugin.ts to use getStartServices() rather than storing local references from start()

- Per feedback: https://github.com/elastic/kibana/blob/master/src/core/CONVENTIONS.md#applications

- Note: savedObjects.registerType needs to be outside of getStartServices, or an error is thrown

- Side update to registerTelemetryUsageCollector to simplify args

- Update/fix tests to account for changes

* E2E testing (#6)

* Wired up basics for E2E testing

* Added version with App Search

* Updated naming

* Switched configuration around

* Added concept of 'fixtures'

* Figured out how to log in as the enterprise_search user

* Refactored to use an App Search service

* Added some real tests

* Added a README

* Cleanup

* More cleanup

* Error handling + README updatre

* Removed unnecessary files

* Apply suggestions from code review

Co-authored-by: Constance <constancecchen@users.noreply.github.com>

* Update x-pack/plugins/enterprise_search/public/applications/app_search/components/engine_overview/engine_table.tsx

Co-authored-by: Constance <constancecchen@users.noreply.github.com>

* PR feedback - updated README

* Additional lint fixes

Co-authored-by: Constance <constancecchen@users.noreply.github.com>

* Add README and CODEOWNERS (#15)

* Add plugin README and CODEOWNERS

* Fix Typescript errors (#16)

* Fix public mocks

* Fix empty states types

* Fix engine table component errors

* Fix engine overview component errors

* Fix setup guide component errors

- SetBreadcrumbs will be fixed in a separate commit

* Fix App Search index errors

* Fix engine overview header component errors

* Fix applications context index errors

* Fix kibana breadcrumb helper errors

* Fix license helper errors

*  Refactor React Router EUI link/button helpers
- in order to fix typescript errors

- this changes the component logic significantly to a react render prop, so that the Link and Button components can have different types - however, end behavior should still remain the same

* Fix telemetry helper errors

* Minor unused var cleanup in plugin files

* Fix telemetry collector/savedobjects errors

* Fix MockRouter type errors and add IRouteDependencies export

- routes will use IRouteDependencies in the next few commits

* Fix engines route errors

* Fix telemetry route errors

* Remove any type from source code

- thanks to Scotty for the inspiration

* Add eslint rules for Enterprise Search plugin

- Add checks for type any, but only on non-test files
- Disable react-hooks/exhaustive-deps, since we're already disabling it in a few files and other plugins also have it turned off

* Cover uncovered lines in engines_table and telemetry tests

* Fixed TS warnings in E2E tests (#17)

* Feedback: Convert static CSS values to EUI variables where possible

* Feedback: Flatten nested CSS where possible

- Prefer setting CSS class overrides on individual EUI components, not on a top-level page

+ Change CSS class casing from kebab-case to camelCase to better match EUI/Kibana

+ Remove unnecessary .euiPageContentHeader margin-bottom override by changing the panelPaddingSize of euiPageContent

+ Decrease engine overview table padding on mobile

* Refactor out components shared with Workplace Search (#18)

* Move getUserName helper to shared

- in preparation for Workplace Search plugin also using this helper

* Move Setup Guide layout to a shared component

* Setup Guide: add extra props for standard/native auth links

Note: It's possible this commit may be unnecessary if we can publish shared Enterprise Search security mode docs

* Update copy per feedback from copy team

* Address various telemetry issues

- saved objects: removing indexing per #43673
- add schema and generate json per #64942
- move definitions over to collectors since saved objects is mostly empty at this point, and schema throws an error when it imports an obj instead of being defined inline
- istanbul ignore saved_objects file since it doesn't have anything meaningful to test but was affecting code coverage

* Disable plugin access if a normal user does not have access to App Search (#19)

* Set up new server security dependency and configs

* Set up access capabilities

* Set up checkAccess helper/caller

* Remove NoUserState component from the public UI

- Since this is now being handled by checkAccess / normal users should never see the plugin at all if they don't have an account/access, the component is no longer needed

* Update server routes to account for new changes

- Remove login redirect catch from routes, since the access helper should now handle that for most users by disabling the plugin (superusers will see a generic cannot connect/error screen)
- Refactor out new config values to a shared mock

* Refactor Enterprise Search http call to hit/return new internal API endpoint

+ pull out the http call to a separate library for upcoming public URL work (so that other files can call it directly as well)

* [Discussion] Increase timeout but add another warning timeout for slow servers

- per recommendation/convo with Brandon

* Register feature control

* Remove no_as_account from UI telemetry

- since we're no longer tracking that in the UI

* Address PR feedback - isSuperUser check

* Public URL support for Elastic Cloud (#21)

* Add server-side public URL route

- Per feedback from Kibana platform team, it's not possible to pass info from server/ to public/ without a HTTP call :[

* Update MockRouter for routes without any payload/params

* Add client-side helper for calling the new public URL API

+ API seems to return a URL a trailing slash, which we need to omit

* Update public/plugin.ts to check and set a public URL

- relies on this.hasCheckedPublicUrl to only make the call once per page load instead of on every page nav

* Fix failing feature control tests

- Split up scenario cases as needed
- Add plugin as an exception alongside ML & Monitoring

* Address PR feedback

- version: kibana
- copy edits
- Sass vars
- code cleanup

* Casing feedback: change all plugin registration IDs from snake_case to camelCase

- note: current remainng snake_case exceptions are telemetry keys
- file names and api endpoints are snake_case per conventions

* Misc security feedback

- remove set
- remove unnecessary capabilities registration
- telemetry namespace agnostic

* Security feedback: add warn logging to telemetry collector

see https://github.com/elastic/kibana/pull/66922#discussion_r451215760
- add if statement
- pass log dependency around (this is kinda medium, should maybe refactor)
- update tests
- move test file comment to the right file (was meant for telemetry route file)

* Address feedback from Pierre

- Remove unnecessary ServerConfigType
- Remove unnecessary uiCapabilities
- Move registerTelemetryRoute / SavedObjectsServiceStart workaround
- Remove unnecessary license optional chaining

* PR feedback

Address type/typos

* Fix telemetry API call returning 415 on Chrome

- I can't even?? I swear charset=utf-8 fixed the same error a few weeks ago

* Fix failing tests

* Update Enterprise Search functional tests (without host) to run on CI

- Fix incorrect navigateToApp slug (hadn't realized this was a URL, not an ID)
- Update without_host_configured tests to run without API key
- Update README

* Address PR feedback from Pierre

- remove unnecessary authz?
- remove unnecessary content-type json headers
- add loggingSystemMock.collect(mockLogger).error assertion
- reconstrcut new MockRouter on beforeEach for better sandboxing
- fix incorrect describe()s -should be it()
- pull out reusable mockDependencies helper (renamed/extended from mockConfig) for tests that don't particularly use config/log but still want to pass type definitions
- Fix comment copy

Co-authored-by: Jason Stoltzfus <jastoltz24@gmail.com>
Co-authored-by: Chris Cressman <chris@chriscressman.com>
Co-authored-by: scottybollinger <scotty.bollinger@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-07-09 13:10:31 -07:00
Xavier Mouligneau
46ed40ec07
[SECURITY] rename server plugin to siem (#69044)
* rename server plugin to siem to avoid privilege issue

* review from alerting

* missing change with rename

* fix tests

* missing api integration test

* fix api integration spaces
2020-06-15 09:16:52 -04:00
Xavier Mouligneau
1216b0f7cd
[SECURITY] Rename siem plugin to security_solution (#67902)
* rename siem to security_solution

* rename siem to security solution inside of code

* rename translation keys

* fix snapshot

* replace siem for security solution in tutorial

* missing translation to be renamed

* fix types for api test integration

* updates runner file to match the new path

* change category for kibana settings

* miss renaming in advance settings

* fixes cypress tests

* fix api integration test

* fix new translation

* fix unit test

* update translation i18n

* update translation i18n II

Co-authored-by: Gloria Hornero <snootchie.boochies@gmail.com>
2020-06-04 05:35:13 -04:00
restrry
bf04235dae apply prettier styles 2020-05-22 09:08:58 +02:00
Xavier Mouligneau
bcfc02987f
[SIEMDPOINT] Move endpoint to siem (#66907)
* adds the stuff

* keeps moving stuff

* finishes moving the stuff

* moves tests

* fix type

* try moving it all at once. BROKEN

* move endpoint to siem

* fix package coming from endpoint

* missing scripts + change url

* fix eslint

* temporary disable functional testing for endpoint

* fix api integration types

* allow api integration test + comment functional test

* fix internationalization

* fix internationalization II

* fix jest test

* fix x-pack test

* fix i18n

* fix api integration

* fix circular dependency

* add new dependency to cypress test

Co-authored-by: Davis Plumlee <davis.plumlee@elastic.co>
Co-authored-by: oatkiller <robert.austin@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
2020-05-19 11:24:53 -04:00
John Schulz
96f74e6e7a
[Ingest] Add Fleet & EPM features (#59376)
* [EPM] Documentation of HTTP routes & TS types for Ingest (#48798)

* Add beginning models and two routes for Ingest

* Update types & models per discussion w/Ruflin
Also reviewed data structures listed at https://docs.google.com/document/d/1IBR3f9dpHqJmXYEdg06WV34KSMd3g5k4aMGa4jde_Eg/edit#

* Update: /policies always returns array. /policy returns single policy

* Add pagination for /policy & /datasources. Uses per_page & page params

* Add API metadata. Standardize policy_id param name.

* Update descriptions to match Google Doc. Move use case to Policy.

Disabled the '@typescript-eslint/array-type' rule because it was going around in circles. It didn't like Datasource[] or Array<Datasource>

* Return to initial TS annotation for Arrays

Remove the line disabling @typescript-eslint/array-type now that it's behaving normally again 🤷

* [EPM] Add directory structure for server/lib. (#50469)

* Add directory structure for server/lib.

* 'tests' seems to be more common than 'test'

* Make CI happy

* [EPM] Add basic documentation directory (#50478)

* [EPM] Add basic documentation directory

Having the doc directory around allows us to easily add docs from here on to document how EPM works.

To run the docs build, use the following command from the kibana directory:

```
../docs/build_docs --doc docs/epm/index.asciidoc --open
```

The above assumes that docs (https://github.com/elastic/docs) are checked out in the same directory as Kibana.

With this change, the EPM docs build is not included yet in the overall docs build. For this adjustments to https://github.com/elastic/docs/blob/master/conf.yaml must be made.

* [EPM] Add basic index template (#50471)

This PR adds the very basic index template we will use for the packages. It contains all the basic settings and some examples. The examples will be remove as soon as we have an actual implementation with packages but for now is convenient to see if it is a valid package.

This code is put into the lib directory as it does not tie directly into any handlers.

It also adds an functional tests for loading a template. This means we have a way to check if a template is valid in Elasticsearch. Based on this we can check in the future all our generated templates for validity with Elasticsearch.

To run the functional test, go to the Kibana x-pack directory. Start the first command:

```
node scripts/functional_tests_server.js --config test/epm_api_integration/config.ts
```

Keep the above running and switch to an other Terminal. Now run:

```
node scripts/functional_test_runner.js --config x-pack/test/epm_api_integration/config.ts
```

* 40752 rewrite ingest pipeline (#50627)

* Add directory structure for server/lib.

* 'tests' seems to be more common than 'test'

* Make CI happy

* Implement pipeline rewriting.

* Add more testcases

* For posterity (comment change)

* Allow beats-style template delimiters

* Be more succinct

* Document better

* Replace AssetType enum with union type (#50696)

See https://github.com/elastic/kibana/pull/50609#discussion_r346080439

Discussed in Slack and agree to revert for now. Can track down the issues & restore later

* Remove unnecessary await if we can return the promise (#50329)

* Fix whitespace per figma comments. Closes #47348 (#47350)

* Add fix & comment for TS 3.7.2 regression

* [EPM] cleanup assets, filter assets for those currently supported (#50609)

* cleanup assets, filter assets for those currently supported

* removed unused type

* fix type

* add comment for better type

* change type name to be more descriptive

* hardcode image width for ie11 (#49796)

* hardcode image width for ie11

* eslint

* improve comment

* add maxWidth

* [EPM] useKibana hook & render in plugin.start (#50110)

* plugin.start now does reactdom.render vs returning react element

export plugin function from public/index

* Move setClient call from plugin.start to plugin.setup

* Use `useUiSetting$` from `useKibana` hooks

* Fix broken app due to bad hooks usage

Can't use useKibana outside a React component.

Reverting to prior approach since it's still NP. Can revisit context usage in a followup PR

* [EPM] Install package from detail view on button click (#50735)

* Support basic "click button -> show spinner -> installed" install flow

* Remove incorrect comments. Add TS return types to data functions.

* [EPM] Use NP feature_catalogue.register (#50108)

* Use NP feature_catalogue.register

* Use type from NP plugin

* fix linting

* fix types

* fix headers in Fleet

* skipping test due to ES param change

* Revert "skipping test due to ES param change"

This reverts commit d05f20decf.

* remove type field

* remove unused import

* [EPM] Final(?) update from integrations_manager -> EPM (#50976)

* Update (all remaining?) references from integrations_manager to EPM

* Update path in i18n file

* [EPM] update compatibility section (#50975)

* change min max version to single range value

* add elastic stack icon and change text

* remove badge from version and use code font

* remove euistyled

* add back version component lost in merge

* remove euiStyled

* remove old file

* Restore RequirementVersionRange type

* Disable test for elasticsearch username.

Temporary work around until we know how the stack wants to add the permissions we need. Either adding to the kibana user or creating a new user.

* Revert "Disable test for elasticsearch username."

This reverts commit f1020e4eab.

* Disable test for elasticsearch username.

    Temporary work around until we know how the stack wants to add the permissions we need. Either adding to the kibana user or creating a new user.

* Fix EPM typing issues in register feature

* Fix typings after master merge

* [EPM] CI fixes (#51284)

* Initialize es in test.

* Add it(), no es.init()

* Clean up.

* [EPM] add confirmation modal (#51172)

* add confirmation modal, move install state to Header

* update callout to use title

* move components only used in detail view to detail dir

* use better variable names

* update to more descriptive  variable names

* Restore prior response vaulues for install & delete package (#51252)

Discussed this with @skh https://github.com/elastic/kibana/pull/51112#commitcomment-35961413 & https://github.com/elastic/kibana/pull/51112#commitcomment-35970664 as well as in a video call

Also added some TS type annotations for data fetching functions to make the contracts more explicit

* [EPM] /package API only lists installable assets. Restore enums. (#51414)

* API only shows installable assets. Server types to own file. Restore enums

* Fix type imports

* Only return installable asset types (kibana for now)

* server/types now only has code from hapi which shouldn't go to client

* Add more restricted TS types to DisplayAssets object

* Flip order of arguments to Extract

In these cases it still works the same, but looking at https://www.typescriptlang.org/docs/handbook/advanced-types.html the signature is

`Extract<T, U>` - Extract from `T` those types that are assignable to `U`

so the larger set should be first

* [Fleet] Enrollment api key UI (#51495)

* Make button pretty in dark mode as well. (#51610)

* [EPM] Add docs entry about registryUrl config (#51697)

This documentation is at the moment mainly for internal use. I found myself searching for this URL several times in the code or PRs so I thought I rather add it to the docs for now.

* [EPM] Remove encoding of Kibana objects as not needed anymore

* [Fleet] Move agent status server side and API to get aggregated status for a policy (#51673)

* [EPM] Add basic docs around install/delete API endpoint (#51728)

This is mainly for internal usage at the moment to look up.

* Ingest/policy (#51741)

* wip policy

* tweaks

* tweaks

* FIX TYPOS

* WIP move policy => agent config conversion to fleet, WIP policy changed method

* fix tests and bugs

* updates tests and snaps

* more fixes

* use AGENT_POLLING_INTERVAL

* cleanup and fix some formatting

* Update x-pack/legacy/plugins/ingest/server/libs/datasources.ts

Co-Authored-By: John Schulz <github.com@jfsiii.org>

* Update x-pack/legacy/plugins/ingest/server/libs/datasources.ts

Co-Authored-By: John Schulz <github.com@jfsiii.org>

* Update x-pack/legacy/plugins/ingest/server/libs/outputs.ts

Co-Authored-By: John Schulz <github.com@jfsiii.org>

* Update x-pack/legacy/plugins/fleet/server/libs/policy.ts

Co-Authored-By: John Schulz <github.com@jfsiii.org>

* fix things broken by PR review suggestions

* remove unused field

* fix types

* fix mappings

* add datasource mappings

* Fix mappings and remove get full policy from checkin

* Fix ingest api integration tests

* run es-lint to fix fleet

* Fix typescript issues

* [EPM] Track package install state and add toast notification (#51734)

* add notifications from core to plugin

* add package install state hook

* fix type error

* use toMountPoint helper to add jsx to notification

* add warning notification to failed install

* make notifications dependency explicit prop

* move PackageInstall provider lower

* add comment about InstallStatus type overlapping InstallationStatus

* use InstallStatus type in InstallationButton component

* fix type

* [Ingest] Adds support for a working default output (#51841)

* aadding config

* add working settings to the default output

* remove default username and password

* update libs

* [EPM] Add basics for creating the ILM setup (#50474)

This contains the basic objects to setup ILM

* Create index and alias with a write index
* Get the policy

The code does not contain any functional tests yet as it is still open on how to do it best. I suggest to get this in as a foundation and then iterate on top of it.

* [EPM] Add datasource (ingest pipeline) from package (#51851)

## Summary

This mixes a few concerns but I think it's worth it to show the parts working together.

Take a look at the individual commits for a better separation of features.

This adds 
 - the `/datasource/install/{pkgkey}` endpoint which installs ingest pipelines from a package into ES and saves a reference to them in the EPM state Saved Object
 - Connects the "Add datasource" button in the successful installation Toast to the new API
 - Adds a toast notification to inform the user the datasource was added correctly
 - Adds a "Delete Package" button on the details page so we can uninstall a package while we're waiting for the separate view which allows deletes
 - b99eda6 Pushes logic that was in the detail view into `InstallationButton`. This consolidates the logic in one component (or one component & the existing hook) and, iiic, means we can put `<InstallationButton package={...} />` on any view and get the same behavior

I'm marking this as a normal PR so people can merge if they wish

![add-datasource-delete-package-small](https://user-images.githubusercontent.com/57655/69775686-7fb39280-1167-11ea-8d41-e2b8a02252a1.gif)

* [EPM] Add basic processing of fields.yml file (#51148)

The fields.yml is used to generate the Elasticsearch template and Kibana index pattern. This PR adds a very basic implementation of processing the fields.yml and then create an Elasticsearch template out of it. The only fields that are supported at the moment are keyword fields, more will be added as a follow up.

The testing was implemented with a golden file. The output from the method is compared to a json file. If the input is changed or the method is changed, it is possible to regenerate the files with the `-generate` flag as following:

```
node scripts/jest  ./legacy/plugins/epm/server/lib/template/template.test.ts -generate
```

This will allow us to quickly test many inputs / outputs in the future, make adjustments to the existing files and generate the new outputs. We then can compare it in the diff it the changes make sense.

* [EPM] Create basic implementation to merge input template and dataset manifest (#51803)

* [EPM] Create basic implementation to merge input template and dataset manifest

With this code it is possible to take an input template for the agent and merge it with the config variables from the dataset manifest file. Currently only the name and the default value are merged. Later on we must implement to be able to pass user configured variables to it and make decision based on OS selection.

Closing https://github.com/elastic/kibana/issues/51794

* [EPM] Refactoring of lib structure (#51885)

This refactors the structure of lib. As so far all the lib parts are related to assets in the package, it is organised the same way with the same structure. Each directory has its own tests directory if it needs one. This makes it possible to (almost) not need relative paths for tests.

* [EPM] Allow to read files from fields directory (#51958)

This change allows to also extract files from the `fields` directory. Previously this was not possible because it always assumed a service must be there.

* [EPM] Install Elasticsearch Index Template for data source (#51878)

This installs the Elasticsearch index template for each dataset in a package. For now the names are hardcoded based on package key and dataset name but will be more dynamic later on when we pass the full dataset information.

The dataset extractions is a bit "hacky" at the moment and we should get a full implementation of dataset at a later stage and replace this code.

* [Fleet] Policy list, details, create, edit UIs (#51950)

* Set up simple policies list view

* Adjust spec to return single policy

* Set up simple policy details page

* Add demo stats/chart to policy details

* Add description string

* Initial setup of policy form and create policy UI

* Policy create/edit form; integrate policy list api

* Integrate create policy api

* Integrate policy detail, agent status, and policy edit APIs; adjust policy list api integration in agent enrollment

* Fix edit policy mock meta

* Fix policy list search bar

* PR and linting fixes; use typings from ingest plugin

* Fix i18n

* [EPM] Add datasource saved object type (#51871)

## Summary

This PR makes a few assumptions, and contains a lot of refactoring. It might be beneficial to look at the resulting directory structure under `server` first to get the (new) big picture.

Assumptions:
- our API deals with several concerns, for now these are packages and datasources
- we manage our own HTTP API endpoints for these concerns (in particular, don't use the ingest plugin for that)
- we manage (for now) the Kibana saved object in which datasources are saved. importing and calling methods from the ingest plugin to do that down the road will (hopefully) be a manageable change

This led to the following decisions:
- the code is separated into subdirectories by concern, containing all the route handlers and tightly coupled code
- for now, these directories are in `server/packages` and `server/datasources`. I'm tempted to move them into `server/api/{packages,datasources}` but wanted to limit the amount of refactoring in one PR
- shared code lives in `server/lib`
- some code from `server/packages` has been almost duplicated to handle saving to Datasource saved objects, some has been refactored and is used from both places. The deduplication needs further improvement
- maybe `server/registry` should also move under `server/lib` (but see above, I'm trying to not move everything around all at once)

Testing:

* Please note that this is a breaking change because the saved object type for package information
has also been renamed. You'll need to start with a fresh `.kibana-*` index. Restarting `yarn es snapshot` (withouth specifying a data directory) should do the trick.

* Package installation should still work, e.g. with a GET request to `http://localhost:5601/api/epm/package/coredns-1.0.1`. The saved objects for packages can be inspected with a GET request to `http://localhost:5601/api/saved_objects/epm-package/$PKG_KEY`, e.g. `http://localhost:5601/api/saved_objects/epm-package/coredns-1.0.1`
* Datasource creation should still work, e.g. with a GET request to `http://localhost:5601/api/epm/datasource/install/coredns-1.0.1`. The saved objects for datasources can be inspected with a GET to `http://localhost:5601/api/saved_objects/epm-datasource/$PKG_KEY`, e.g. `http://localhost:5601/api/saved_objects/epm-datasource/coredns-1.0.1`

* [Fleet] Expose policy during agent checkin (#51968)

* [EPM] Add /epr prefix to the tar.gz download path (#51881)

The registry slightly changed the .tar.gz path because of download stats reason. This adjusts for it. See https://github.com/elastic/package-registry/pull/169

* [EPM] Move template installation to lib and add asset helper  (#52049)

* [EPM] Move template installation to lib and add asset helper

All the logic related to the installation of the templates for a package should be inside the template library folder. This moves the logic into this folder.

A few refactorings were made to simplify installation:

* Introduction of DataSet interface: This interface is needed to extract the data sets inside a package and install one template per data set.
* Pass package instead of package key to installation process: Passing the package instead of the package key means fetching of package information is decoupled from the installation process and abstracted. This separates the two concerns and should simplify testing.
* getAsssets method: The getAssets methods works on top of the package object to extract asset paths. It is inspired by get_objects methods but supports passing a package and a dataset.

Currently one problem with testing that exists is that to fetch the content of an asset is not decoupled yet.

* [EPM] Reduce data source to one type (#52061)

Between Fleet / Ingest / EPM there had been several interface definitions of Datasource and the related types. This reduces it to one place for the definition. The same applies to the policy definition.

The goal of this is that from now on we all rely on the same definition. If we make changes, we make them in all parts of the code.

In this PR is only the minimal change needed to get us all on one interface. Further changes will be needed that we all rely on the same saved objects etc.

* add export command

* revert 2 more files to rely on export

* revert imports

* Fix types for Datasource Saved Object

* merge in master

* fix type check

* Run VSCode's organize imports on EPM files (#52234)

Learned about it on Slack from https://twitter.com/ryanchenkie/status/1201883268527927301

Blog at https://code.visualstudio.com/updates/v1_23#_run-code-actions-on-save

Basically does the order we've been loosely following (3rd party, then relative) & alphabetic by location and variable name.

It's not customizable but it's reasonable and, afaict, consistent.

* [EPM] More realistic datasource SO. Error if package not installed. (#52229)

* Move cache 'hack' into getAssetsData

* p -> pkg. package is reserved. pkgkey is used in many places

* Remove unnecessary type cast

* Clarify reasons behind asset path manipulation

* Return the Datasource; not the Saved Object.

* Use real values from package in fake datasource SO

* Error if /datasource/install before /package/install

```
> curl --user elastic:changeme localhost:5601/api/epm/datasource/install/coredns-1.0.1
{
  "statusCode": 403,
  "error": "Forbidden",
  "message": "coredns-1.0.1 is not installed"
}

> curl --user elastic:changeme localhost:5601/api/epm/install/coredns-1.0.1
[
  {
    "id": "53aa1f70-443e-11e9-8548-ab7fbe04f038",
    "type": "dashboard"
  },
  {
    "id": "Metricbeat-CoreDNS-Dashboard-ecs",
    "type": "dashboard"
  },
  {
    "id": "75743f70-443c-11e9-8548-ab7fbe04f038",
    "type": "visualization"
  },
  {
    "id": "36e08510-53c4-11e9-b466-9be470bbd327-ecs",
    "type": "visualization"
  },
  {
    "id": "277fc650-67a9-11e9-a534-715561d0bf42",
    "type": "visualization"
  },
  {
    "id": "cfde7fb0-443d-11e9-8548-ab7fbe04f038",
    "type": "visualization"
  },
  {
    "id": "a19df590-53c4-11e9-b466-9be470bbd327-ecs",
    "type": "visualization"
  },
  {
    "id": "a58345f0-7298-11e9-b0d0-414c3011ddbb",
    "type": "visualization"
  },
  {
    "id": "9dc640e0-4432-11e9-8548-ab7fbe04f038",
    "type": "visualization"
  },
  {
    "id": "3ad75810-4429-11e9-8548-ab7fbe04f038",
    "type": "visualization"
  },
  {
    "id": "57c74300-7308-11e9-b0d0-414c3011ddbb",
    "type": "visualization"
  },
  {
    "id": "27da53f0-53d5-11e9-b466-9be470bbd327-ecs",
    "type": "visualization"
  },
  {
    "id": "86177430-728d-11e9-b0d0-414c3011ddbb",
    "type": "visualization"
  },
  {
    "id": "4804eaa0-7315-11e9-b0d0-414c3011ddbb",
    "type": "visualization"
  }
]

> curl --user elastic:changeme localhost:5601/api/epm/datasource/install/coredns-1.0.1
[
  {
    "id": "coredns_1_0_1_dataset_log_elasticsearch_ingest_pipeline_pipeline_plaintext_json",
    "type": "ingest-pipeline"
  },
  {
    "id": "coredns_1_0_1_dataset_log_elasticsearch_ingest_pipeline_pipeline_json_json",
    "type": "ingest-pipeline"
  },
  {
    "id": "coredns_1_0_1_dataset_log_elasticsearch_ingest_pipeline_pipeline_entry_json",
    "type": "ingest-pipeline"
  }
]
```

* fix duplicated imports

* [EPM] Move golden files generation over to jest snapshot (#52203)

* [EPM] Move golden files generation over to jest snapshot

I initially used my own implementation to write the generated files. It runs out jest has a feature to write snapshots which simplifies the code a lot.

I added a loop with an additional test file so in the future we can just keep adding test files without having to modify the test code.

To updated the snapshots, the param `-u` has to be used:

```
node scripts/jest legacy/plugins/epm/server/lib/fields/field.test.ts -u
```

* [EPM] Create metrics-* and logs-* Kibana index pattern (#52277)

This creates the very basic Kibana index patterns metrics-* and logs-* for Kibana. At the moment it is overwritten every time. We need to change this in the future to take the fields from all installed data sources and regenerate it.

* [EPM] Create helper for elasticsearch asset names (#52265)

Most of the Elasticsearch assets have the same base name. This creates a helper to get the base name for the assets. In case we decide to change the base name in the future, we can change it in one place.

* fix tests and destructing

* [EPM] Update Registry types. Prevent errors installing certain datasources. (#52285)

* Update RegistryPackage type

* Use download key from EPR to fetch archive

* Fix errors caused by correcting the Registry types.

The issues were largely that some Registry types like `title, `datasets` and `assets` where marked as required, but are actually optional. This highlighted area in the code were we relied on them always being present.

We added to the issue by wrapping Registry types in `Required` which made those items which were correctly listed as optional, required for EPM code. Updated EPM types to reflect the largely pass-through nature of the EPM types.

There are two properties which we ensure are in every EPM response, those were put into their own (unexported) type.

Confirm by trying to add a datasource to a package which has no datasources, like apache-1.0.1 or system-2.0.1. In `feature-ingest` and the earlier version of this PR, the `/datasource/install` call returns a 500. In this PR it succeeds.

* [EPM] Add setup of default ILM policies (#52272)

This creates two ILM policies: logs-default and metrics-default. These are the default ILM policies used. Currently the policy content is hardcoded in the code but should be fetched from the base package in the future. The setup happens as part of the datasource installation. When a data source is installed it is a good time to check if the assets are there but we might extract this to a better place in the future.

* [EPM] 52075 add data source first page (#52320)

* Update RegistryPackage type

* add first page of add data source

* fix for ie11 flex min width bug

* remove toDetailViewRelative

* remove unneeded spread

* Update TS type names for EPR search results (#52512)


 * `RegistryList -> RegistrySearchResults`
 * `RegistryListItem -> RegistrySearchResult`

* Restore import sort order from #52234 (#52548)

Many of the changes from #52234 were lost. Presumably due to PR(s) merging which were based on branches which had the previous unsorted order.

* [EPM] Replace wildcard export (#52554)


 * PackageNotInstalledError -> packages/index.ts
 * pkgToPkgKey -> registry/index.ts (will convert existing `${name}-${version}` instances later)

* Replace export * from packages.

There's an argument that the import sites should be updated to import from `packages/get`, `packages/install`, etc but that can wait for a later PR.

* [EPM] Reduce usage of epm-package SavedObject (#52576)

* Delete existing Installation type. Rename InstallationAttributes to Installation

* Reduce usage of EPM SO. Add getInstallation().

Replaced two calls of getInstallationObject() with getInstallation().

Two less places with knowledge of SO internals.

Lots of potential improvements for EPM TS types remain (refactoring/removing Installable, etc), but this is a good incremental step, IMO

* [EPM] Fix missing export link (#52628)

Without it, things break. I am surprised CI did not catch this.

* [EPM] Cleanup ILM loading (#52632)

Before the check for the ILM policy to exist triggered an exception. With this change it is a normal response also if the policy does not exist yet.

A follow up issue will be created in Elasticsearch to get a HEAD request for this available.

* [EPM] Switch to staging URL for registry (#52626)

The old cluster with the registry will be removed as soon as this is merged.

* [EPM] Use Dataset interface to generate template (#52255)

This will make sure we have to pass much feature params and can fully rely on the datasource object to create names for assets.

* [Fleet] Use agent events to compute agent health (#52513)

* [EPM] Data source integration tests (#52542)

* Add fixtures for data source integration test.

* Move test setup to beforeEach

* Add test for datasource creation

* Handle pipelines in yml format.

* Make integration test for adding a data source pass.

* Use EPR staging URL with CDN. (#52776)

See https://github.com/elastic/kibana/pull/52626#pullrequestreview-330622868

* [EPM] Add Data Source page updates (#52705)

* remove dupe type RegistryPackage

* change switches to checkboxes, use datasets to create checkboxes, add some local form state

* update types

* [EPM] redirect after package install (#52771)

* add callback after successful installation and redirect

* add temp data sources tab content to access add data source page

* remove assets tab for mvp

* hide data sources link and redirect from data sources tab if package not installed

* change callback name

* remove commented out assets logic

* add redirect to hook

* fix type

* Use ingest datasource api (#52964)

Incremental change. Uses HTTP API for datasource creation. Will do follow-up PR which uses JS function instead

* Remove duplicate fetchInfo & installTemplates

I think this was from a bad merge, but pretty sure we don't want these functions called twice in the same function

* WIP. Pushing so others can see

* Improve correctness/flexibility of absolute URL

* Disable datasource test & template installation

* [Ingest] Data source APIs (#52448)

* Clean up ingest imports and remove unneeded mock_spec files

* Initial pass at datasources lib and API endpoints

* Add add/remove datasource to/from policy API endpoints

* Add datasource contract tests and related policy contract tests; update snapshots

* Fix tests

* Fix tests again

* Fix tests 3

* Adjust routes, PR feedback

* modify epm createDatasource endpoint to use user data (#52971)

* change epm/datasource/install/{pkg} to POST, send user data to endpoint, install pipelines and templates based on user selected datasets

* change test to post for installing a datasource

* change some names and types around

* delete request.headers['transfer-encoding'] being passed through from epm request

* [EPM] Don't share CreateFakeDatasource type (#53068)

It's not shared between client & server so it doesn't need to be in common. Also, it imports server code which would try to bring server types to the client. It's types so they're compiled away but it's important to keep common to what's truly common. Breaking this separation is why we thought enums broke the client. A lint rule just landed in master to prevent this.

* [EPM] Index template generation fixes (#53104)

* Only add keyword type field to mappings.

* Index template installation

* Handle empty fields definition files
* Re-enable index template installation

* [Fleet] Assign/Unassign data source from policy UI (#53058)

* Add index files to export various modules; normalize imports

* Clean up unused files; extract datasources table component from policy details page

* Expose http client to frontend libs; remove unused types; import ES UI's useRequest lib

* Adjust shape of rest api adapter interface to better match with rest of kibana; remove unused node adapter; change per_page param to perPage in agent events route

* Initial pass at assign data sources flyout

* Initial pass at unassigning data sources from policy

* Make data sources table searchable by package values

* Fix enrollment key lib for rest adapter param changes

* Fix imports and types

* `yarn.lock` changes after bootstrapping

* [EPM] Implement getConfig for dataset (#53261)

* [EPM] Implement getConfig for dataset

* Implements a getConfig method on a dataset object. 
* Build the configuration for each dataset in a package.
* construct and save streams into datasource saved object

* [EPM] Fix template installation (#53272)

As dataset.package was not set, the installed templates contained undefined in the template name. This changes fixes this.

* [EM] Refactor ingest pipeline installation (#53309)

* Refactor ingest pipeline installation

* Only install index templates for requested datasets

* Add index.default_pipeline to index template

* Hook up pipeline rewriting

* Add correct types.

* change POST create datasources path (#53165)

* change POST create datasources path

* remove pkgkey from params

* Fix creation of a data source with a custom ID (#53537)

* [Ingest] Return associated policy IDs in data source info (#53350)

* Return number of policies from data source, surface in assign data source UI

* Update snapshots

* [EPM]: Assign data source to policy in UI (#53597)

* Let ES generate source ids. Refactor along the way.
* Datasource.id isn't optional. It's just missing before we send to Ingest
* Delete EPM's mapping of datasources saved object. Ingest handles that.
* Keep datasource object-related work in constructDatasource
* Move asset installation into own function. Keep entry point high-level.
* More descriptive (less ambiguous) names for these two functions
* Use enum values from Ingest instead of plain strings
* Limit the 'type' key of references to known asset types.
* Update variable names to clarify that we're merging arrays of references
* Use [].flat instead .reduce + .concat to avoid error on empty arrays.
* Pass PackageInfo value directly to component vs pulling off n properties
* Name handlers/options based on the data, not the UI element
* Populate policy combo box based on values from Ingest policy API
* Mark Dataset.vars as optional.
* Add TODOs

* Add commands to run API tests to README (#53847)

* Limit functions to 3 params max. Update those which used more (#53848)

* [EPM] Code in 'common' directories shouldn't import server code (#53854)

* [Fleet] Code in 'common' directories shouldn't import server code (#53938)

* [Fleet] Remove server code from common folder in fleet
* [Fleet] Fix typescript issues after master merge
* [EPM] Fix typescript issues after master merge

* Fix eslint issues

* Fix typescript issues after merge

* Fix merge master missing line

* Fix merge conflict

* [Fleet] Fix registration of Ingest management section (#54065)

* Fix registration of Ingest management section

* Fix i18n key

* [Fleet] Remove server code from common folder in ingest (#53969)

* [Fleet] Connect fleet to policy change update (#53201)

* [Fleet] Send created event when a policy is created
* [Fleet] updated created event when a policy is created
* [Fleet] Send deleted event when a policy is deleted

* [Fleet] Rename output.url => output.hosts (#54258)

* [Ingest] Remove policies UI (#54308)

* Remove meta field UI from policy add/edit form

* Initial pass at policy bulk+single delete UI and API

* Adjust policy links from agent list and detail pages so that links are only active if policy exists

* Add delete policy UI to policy detail page

* Disable policy delete button for default policy

* Commit updated kbn-pm artifact.

CI is failing with messages like
14:52:28  ERROR: 'yarn kbn run build -i @kbn/pm' caused changes to the following files:
14:52:28
14:52:28  packages/kbn-pm/dist/index.js

Following advice from https://elastic.slack.com/archives/C0D8P2XK5/p1570032166063400 and running/committing build

* Update kbn/pm package

Signed-off-by: Tyler Smalley <tyler.smalley@elastic.co>

* [EPM] create logs metrics index patterns (#54037)

* fixes bug in for loop returning too early and not looping through all yaml files creating incomplete index template, move loading yaml files to own function, other cleanup

* use reduce in place of for loop

* basic functionality for creating index patterns

* separate logs and metrics index patterns

* dedupe fields

* adjust flattenFields to rename nested fields with parent name path

* some tests

* use yml files for tests

* add awaits as part of installing the package

* optimize loading of yaml files

* fix typo

* change type packageName to package

* update tests to use all files from the beginning

* fix type errors

* fix test

* Use dataset.package from registry

https://github.com/elastic/kibana/pull/54037#pullrequestreview-340362812

* Form validation on add datasource page. (#53920)

* [Ingest] Add support for policy `label` field (#54413)

* Allow `label` field in policy APIs, update UIs to support `label` field

* PR review changes, typo fixes, update tests

* [Fleet] Fix api key creation (#54498)

* [Fleet] Move agent acks to his own endpoint (#54401)

* [Ingest] Fix MaxListenersExceededWarning during kibana boot (#54745)

* [Fleet] Create a default api key for the default output (#54658)

* [EPM] update index pattern fields (#54862)

* add DatasetType type

* move loadFieldsFromYaml to fields

* add logic to determine field values

* update tests

* add back accidentally removed readFromDocValues

* remove DatasetType and add IndexPatternType

* rename dedup to dedupe

* group tests

* use null coalescing operator

* Fix typing issues

* [Fleet] Support agent event subtype STOPPING (#55231)

* [EPM] Prevent double submit when creating data sources

Update installationRequested state after submit (#55100)

* [EPM] handle alias fields when creating kibana index pattern (#55254)

* add alias support, update flattenFields to handle alias copying, update and add tests

* update snapshot

* update findFieldByPath to return undefined if not leaf node

* remove temporary alias type from map

* [EPM] handle multi fields when creating index patterns (#55554)

* handle mult_fields

* move nested function out

* [Fleet] Fleet/spec docs (#55619)

* [EPM] Start to document definitions (#55361)

This is a first stab at creating a place where we define the terms we use across ingest management. This is not complete but defines a place where we can add all the future defintions.

* [EPM] Indexing strategy docs (#55301)

* [EPM] Indexing strategy docs

This documentation is to start documenting our new indexing strategy in a public place and have it versioned. This will allow us to share the current state of the indexing strategy more easily in a single place, track it when updated and also already have it ready for our future users to look it up and understand the benefits of it.

* update typos

* fix one more typo

* apply review feedback

* skip fields that are disabled (#55735)

* [Fleet] Fix fleet typing issues after merging master

* [EPM] Create fieldFormatMap in kibana index pattern (#55892)

* add support for fieldFormatMap

* output params must be camel case

* fix case

* add url_template param

* [Fleet] Use user from saved object to create apiKeys (#55458)

* [EPM] Document package upgrade behaviour (#56138)

This PR adds more detailed documentation on what should happen when a package is upgraded.

* Remove some explicit typing to pass type checks.

This abstraction will be removed in the single plugin going into master.

* remove readFromDocValues (#56227)

* Add symlink to yarn.lock to fix(?) CI

https://github.com/elastic/kibana/pull/56443/checks?check_run_id=418123781 failed saying EPM directory "MUST have a 'yarn.lock' symlink"

Seems to have originated with https://github.com/elastic/kibana/pull/55440

Following example from other legacy/plugins/* in that PR

* Like 441d9ed, but correct

* [Ingest] Convert `ingest` plugin to new platform `ingest_manager` plugin (#56262)

* Seed Ingest Manager as a new NP plugin

* Add contexts for core, deps, and config. Begin routing and nav UI

* Export NP ready request from top-level es_ui_shared/public/

* Add nav styling w/ theming, add useRequest hook

* Set up license and config server-side services; add test routes

* Move most types and constants into /common

* Initial pass at:
* data stream and agent config models
* data stream routes and schemas

* Initial pass at agent config api route handlers

* Change plugin id to camel case

* Fix circular schema dependency, add security as optional plugin

* Create appContext service, use request user info in agent config routes + libs

* Create default agent config

* Add default output host config, output typings, and create default output and its api key

* Move saved object mapping to new plugin

* Change data streams -> datasources

* Add legacy plugin to bootstrap mappings

* Adjust fleet's ingest dependencies

* Disable policies UI in Fleet

* Adjust EPM's ingest dependencies

* Adjust ingest manager base API route

* Adjust fleet's client side ingest dependencies

* Remove more ingest dependencies from fleet

* REMOVE MOST OF LEGACY INGEST PLUGIN

* Add section for agent configs in UI nav

* Allow useRequest and sendRequest consumers to specify typing for response

* Initial pass at porting over agent config list UI

* Port over agent config creation

* Port over delete agent config functionality

* Fix app routing

* Port over fleet setup routes

* Adjust fleet's ingest dependencies

* Make fleet happy path work, skip some tests (MESSY! :))

* Remove policy list UI code from fleet

* Change useRequestResponse error type

* Add missing agent config schemas and hooks

* Fix type check issues

* Register IM under management

* Fix type issues as a result of changes to use/sendRequest interfaces

* Make all ingest saved objects *not* space-aware

* Fix i18n path

* Fix app categories import

* Fix datasource package assets schema (array of asset objects)

* Seed Ingest Manager privileges to fix tests

* Change `features` to optional plugin instead of required

* Fix security privileges tests

* Fix feature test

* Fix duplicate enrollment key created for default agent config

* Fix fleet agent enrollment by catching agent config 404

* PR feedback

* [Fleet] Detailed docs of fleet <-> agents interactions (#56212)

* [EPM] update index patterns on install/uninstall of package (#56591)

* create index patterns functionality on install/uninstall of package

* update snapshots

* [Fleet] Generate an ES api key per agent per output (#56637)

* [Fleet] Remove unused enrollement rules (#56753)

* remove files related to creating data source (#56745)

* Fix typing issues after mergin master

* Fix api key authentication after master merge

* [EPM] NP Migration: Move server files and route handlers to ingest_manager (#56854)

* initial pass moving registry and categories endpoint

* moves all needed server files and gets list endpoint working

* add route list validation schema

* remove epm config

* use config to get the registryUrl

* clean up registry url

* gets file endpoint working

* add info endpoint

* get install package endpoint working

* support uninstall package endpoint

* add API response types

* move epm types to models

* move AssetType to IngestAsset type from ingest plugin

* remove redundant export

* update epm_api_integration tests to new endpoint paths in ingest manager

* fix imports

* [Ingest Management] Change indexing from {type}-{namespace}-{dataset} to {type}-{dataset}-{namespace} (#56132)

Currently we have the indexing strategy defined as `{type}-{namespace}-{dataset}`. In this PR I propose to change this to `{type}-{dataset}-{namespace}`. As all 3 fields are constant keyword fields, the orders does not matter in most cases.

The reason I propose this change is to better align the name of indices with the name of the other assets:

* ingest pipeline: {type}-{dataset}
* index template: {type}-{dataset}
  * Index pattern: {type}-{dataset}-*
* alias name: {type}-{dataset}-{namespace}

This makes it easier to remember the asset names conventions (at least for me).

It makes a difference when specifying security per namespace: To lock down security, previously it is `/(logs|metrics)-prod-$/` and becomes `/(logs|metrics)-[^-]+-prod-$/`. In any case, we should help / assist the user to get this right.

* [EPM] Remove epm plugin and directory (#57309)

* delete server files, move over epm saved object schemas and mapping, stop epm plugin loading in xpack

* updated yarn

* change to updated name

* remove epm dir, copy readme to ingest_manager

* move package.json over and update yarn.lock

* update package name

* add yarn.lock symlink

* fix yarn.lock symlink

* remove epm from security privileges map

* remove epm from feature endpoint and i18nrc

* [Fleet] Move fleet to the new platform and to ingest_manager plugin (#56803)

* [Fleet] NP migration public part (#57567)

* [EPM] EPM to new plugin, UI part (#56882)

* Move EPM home / list view over to ingest-manager

* Use react-router-dom in epm section.

* WIP: add package detail view.

* Use correct route.

* Only import needed types to public

* Remove obsolete file.

* Import type correctly

* Revert "Remove obsolete file."

This reverts commit 4b061102ebc62b49e7d1291060405ea8d23a3a8a.

* Routes are still needed, fix them.

* Import types correctly

* More type import fixes.

* update get categories hook

* remove no longer used getCategories function

* get list packages hook working

* delete routes.tsx, cleanup links

* add the usePackageInstall hook

* replace rest of api calls with use/send request

* remove tmp_routes

* bring over breadcrumbs

* remove comments and get styles working

* get ride side col loading

* temp type fix

* remove useCore

* add assets

* remove comment

* add public directory to legacy ingest_manager and update asset path

* Fix PackageInfo type. Use for API & UI vs a saved object type.

The `as PackageInfo` cast was required because the pipeline was typed as returning `Installed | NotInstalled` which are saved object response.

Updating that to PackageInfo allows the `as` to be removed but revealed an incompatibility between the `assets` properties of RegistryPackage and PackageInfo

```
Types of property 'assets' are incompatible.
  Type 'Record<"kibana", Record<KibanaAssetType, KibanaAssetParts[]>>' is missing the following properties from type 'string[]': length, pop, push, concat, and 28 more.
```

It seems the `RegistryPackage & PackageAdditions` didn't cause the PackageAdditions.assets to replace the RegistryPackage.assets property.

I changed the definition of PackageInfo to do what I initially thought it was doing. See comments in models/epm.ts for more about how the new type is constructed.

* remove comment

* fix paths

* fix public paths

* fix path

* remove ui types file

* fix types

Co-authored-by: Sandra Gonzales <neptunian@users.noreply.github.com>
Co-authored-by: John Schulz <github.com@jfsiii.org>

* [Fleet] AgentEvent change agent_id and remove data (#57818)

* Remove legacy `ingest` plugin completely (#58056)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* WIP. 1 type error (but >1 bugs) remaining.

* Add `callCluster` accessors for using ES vs appContext.getClusterClient()

* Undo (?) changes to kbn-pm/dist/index.js

* Run scripts/build_renovate_config for @types/tar

* Replace data w/ agent_id in server schema

* Different way to declare a saved object type

* Use a more specific path to the agent script

* Replace data with agent_id for agent checkin

* Restore internalSavedObjectsClient in app context

* Use project & HTTP TS types in scripts/dev_agent

* Remove ingestManager from FTR features

The plugin is disabled by default and not currently running the FTR tests. I believe we'll add this back when we restore the EPM integration tests.

* Move more variables to common/{constants,types}

* Remove ingestManager from default expected features

* Enable conditional routes. Adjust integration tests

EPM routes currently return a 500 for these tests. For now they `.expect(500)` when enabled and `.expect(404)` when disabled. We can look into the issue and get them to `.expect(200)` in later tests.

* Replace React.FC with React.FunctionComponent following new repo pattern

* Enable Fleet & Ingest FTR tests

* Remove duplicate *Response entries from server/types

* Update README instructions for CLI flags. Rearrange sections (#58363)

* Add instructions for CLI flags. Rearrange sections

Also added some more information about the plugin behavior with links to relevant code

* Add instructions for Ingest & Fleet FTR tests

* Restore search to EPM list page

* [Fleet] Allow to configure CA sha for kibana and elasticsearch (#58186)

* [Ingest] Adjust saved object mappings and rename policy -> config (#58670)

* Replace all reference to (agent) "policy"/"policies" with "config"/"configs"

* Adjust output and datasource saved object mappings

* Update schemas and types to match SO changes

* Fix type check

* Adjust default output object

* Fix property names in tests

* Move installing of index templates and ingest pipelines to package installation (#58619)

* update template and pipeline asset names, install on package install

* fix package install error handling messaging

* save references to installations to package saved object

* add epm.enabled flag for epm functional test runner

* don't add suffix to pipeline entry

* [Fleet] Add a schema of all of our saved objects (#58769)

* [Fleet] Do not use default id for saved objects that need to be encrypted (#57876)

* Adding events for index pattern generation (#58908)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* Fix yarn.lock after merge

* [Fleet] create agent config before enabling fleet user (#59166)

* support for top level elasticsearch assets installation (#58869)

* add support for ilm policy installation

* check if ilm policy exists

* handle prebuilt index templates, update tests

* cleanup

* update type in install

* fix installing index templates to create multiple ones for inheritance

* [Fleet] Create default output while creating default config (#59223)

* Use the new definitions from package-registry#176 (#59311)

* [Fleet] Remove our custom API key authentication  (#59212)

* [Ingest] Full agent config schema & API (#59262)

* Add schema for full agent config and business logic to convert SO agent config -> full agent config

* Adjust output properties in full agent config

* Whitelist full agent config output fields

* [Fleet] UI Agent enrollment flyout (#58524)

* Use POST vs GET for EPM install & remove (#59367)

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* install default packages during setup (#59330)

* install default packages during setup

* check if package is already installed

* [Ingest] Updates to Agent Configuration List UI (#59374)

- Added search bar
- Sync of columns to design
- Actions per row in popup menu
- Connect pagination and per-page count to API request
- Support for `kuery` url param

* Update docs from /api/:section to /api/ingest_manager/:section (#59422)

* [EPM] Add Streams TS type to mirror EPR's (#59446)

* Add Streams TS type in EPM to mirror EPR's

Follows the changes add in https://github.com/elastic/package-registry/pull/230/files#diff-7dea786222588c32c19238bffffee9c2

* Add RegistryPackage.datasources

* Add more detailed shape for Registry Vars

* Don't code in modify src/plugins/management

See PR convo at c47975535f (r389042975)

* [Ingest] Add agent counts to each agent config output of `/agent_configs` API  (#59552)

* Added agent counts to getAgentConfigsHandler
* Show agent counts on Agent Config List

* [Fleet] Agent list header (#59487)

* [EPM] Use /packages & /packages/{pkgkey} (#59550)

* Use /packages & /packages/{pkgkey}

* Update paths in skipped tests

* Docs use /packages vs /package/{install,delete}

Some copy & code sample changes.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* [Ingest] Design sync for Agent Configuration Create Flyout (#59479)

* Match form to design
* Support URI route param to open flyout

* [Ingest] Create data source step 1 & 2 (#59590)

* Add Error and PackageIcon components

* Add create datasource layout+navigation, and select package step

* Add description field to datasource, remove assets and description from datasource package

* Add temporary datasources typing for EPM package info

* Initial pass at configure datasources step. Stream vars only (no input vars yet)

* First pass at input vars config; separate components

* Fix issue with adding more than one datasource to a config

* Add shell review step; save datasouce

* Remove assign/unassign datasource from agent config details UI, replace with add datasource buttons

* Remove actions column from datasource table

* Initial pass at create datasource from package

* Move package to config service to /common, add tests

* Rename VarsEntry to RegistryVarsEntry, add datasets and datasources to RegistrySearchResult definition

* Add typings to create datasource flow

* Add real count of agents to select agent config list

* Ensure the necessary package is installed at time of datasource creation

* Use lowercase pkgkey for consistency

* Update EPM file path to use /packages (#59693)

* fixed header padding (#59711)

* [EPM] Use icons from packages, if present (#59765)

* [Ingest] Create data source step 3 (#59822)

* Make app setup loading state prettier

* Add review step of datasource wizard

* Change name to ID in agent config datasources field

* Fix types

* Add stored datasource to agent datasource unit tests

* Adjustment of registry typings and which registry copy fields to show to sync with elastic/package-registry#242

* Fix `multi` vars not populating with array: elastic/kibana#59724

* Account for if a stream is enabled by default from registry package definition: elastic/kibana#59724

* Adjust tests to account for last two commits

* Fix review page back link

* Fix d'oh typo

* [Ingest] Agent Config Details header and sub tabs navigation (#59783)

- Syncs Agent Config Details header to design
- Includes sub navigation tabs connected to route URL
- Agent Config List Create data source row action enabled

* [Fleet] update agent list UI (#59685)

* [Fleet] ensure default packages are added to the default config (#59759)

* [Fleet] fix output rename api_token => api_key (#60001)

* [Ingest] Address #59376 feedback (#59961)

* Disable create/destroy CTAs if no write capability

Use `core.application.capabilities.ingestManager.write` to test user permissions

* Add -all & -read tags for HTTP routes

* Update test .expect() to match description

* Add useCapabilities hook. Fix two issues with hiding/disabling CTA.

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

* Missed one in e12a8ad8a4

* Use package icon as default when no other can be found (#60025)

* Remove duplicate xpack setting

FTR tests failing to start ES with error

```
ERROR ERROR: setting [xpack.security.authc.api_key.enabled] already set, saw [true] and [true]
```

https://github.com/elastic/kibana/pull/59376/checks?check_run_id=503930031 & https://github.com/elastic/kibana/pull/59376/checks?check_run_id=503975576 etc

It appears the xpack.security.authc.api_key.enabled flag was recently added to master in another part, so remove our instance of the setting to prevent the error

* Update EPM file tests to use /packages/{pkgkey}

These should have failed when the routes were changed. Will go back and see what happened.

* [Ingest] Add `revision` to agent configs & data sources (#59848)

* Add revision to agent config and datasource saved objects, add delete datasource service and datasource

* Add revision to full agent config output

* PR feedback

Co-authored-by: Sonja Krause-Harder <sonja.krause-harder@elastic.co>
Co-authored-by: Nicolas Ruflin <spam@ruflin.com>
Co-authored-by: Sandra Gonzales <neptunian@users.noreply.github.com>
Co-authored-by: Matt Apperson <me@mattapperson.com>
Co-authored-by: Nicolas Chaulet <nicolas.chaulet@elastic.co>
Co-authored-by: Nicolas Chaulet <n.chaulet@gmail.com>
Co-authored-by: Matt Apperson <matthew.apperson@elastic.co>
Co-authored-by: Jen Huang <its.jenetic@gmail.com>
Co-authored-by: Brian Seeders <brian.seeders@elastic.co>
Co-authored-by: Tyler Smalley <tyler.smalley@elastic.co>
Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: neptunian <sandra.gonzales@elastic.co>
Co-authored-by: Jonathan Buttner <56361221+jonathan-buttner@users.noreply.github.com>
Co-authored-by: Paul Tavares <56442535+paul-tavares@users.noreply.github.com>
Co-authored-by: Henry <henry.harding@elastic.co>
2020-03-12 18:52:06 -04:00
Kevin Logan
2f3ca65886
[Endpoint] Register endpoint app (#53527)
* register app, create functional test

* formatting

* update tests

* adjust test data for endpoint

* add endpoint tests for testing spaces, app enabled, disabled, etc

* linting

* add read privileges to endpoint

* rename variable since its used now

* remove deprecated context

* remove unused variable

* fix type check

* correct test suite message

Co-Authored-By: Larry Gregory <lgregorydev@gmail.com>

Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>
Co-authored-by: Larry Gregory <lgregorydev@gmail.com>
2019-12-20 10:31:41 -05:00
Mike Côté
88ae8d07c8
Cleanup alerting / actions feature controls (#52286)
* Initial work

* Apply changes based on feedback

* Fix a broken test

* Fix failing test

* Revert monitoring changes

* Remove UI features for now
2019-12-13 10:36:46 -05:00
Spencer
b66415e416
[xpack/ftr/common] follow service provider conventions (#52236)
* [xpack/ftr/common] follow conventions, expose a config file

In order to make xpack/test/common ftr config easier to work with, this updates it to follow the conventions established by most other configs of exposing all services from the services module as an object that can be easily merges into local services modules.

* common config file and FtrProviderConfig are unused and unnecessary

* turns out FtrProviderContext was used...
2019-12-05 10:16:17 -07:00
Mike Côté
1367814a8e
Enable alerting and actions plugin by default (#51254)
* Enable alerting and actions plugin by default

* Fix test failure

* Fix features test
2019-11-28 11:02:11 -05:00
Court Ewing
0b82cfeae7
Remove experimental code app (#49404)
* Fix erroneous code plugin import in apm

* Stop running legacy code plugin

* Stop testing code app and apis

* Remove code plugin source and deprecate config

* Remove code plugin docs

* Remove xpack.code translations

* Remove code import on api_integration

* Remove code privilege from security test

* remove two more mentions of code

* remove code es_archives

* remove code doc images
2019-10-28 11:40:27 -04:00
Brandon Kobel
21841f6ff8
Removing the lens feature, using visualize instead (#48041)
* Removing the lens feature, using visualize instead

* Updating /api/features test because lens isn't a feature anymore

* Updating capability from lens to visualize in test

* Fixing api integration test
2019-10-15 12:03:12 -07:00
Joe Reuter
eed848ab2e New visualization editor Lens (#36437)
* [lens] Initial Commit (#35627)

* [visualization editor] Initial Commit

* [lens] Add more complete initial state

* [lens] Fix type issues

* [lens] Remove feature control

* [lens] Bring back feature control and add tests

* [lens] Update plugin structure and naming per comments

* replace any usage by safe casting

* [lens] Respond to review comments

* [lens] Remove unused EditorFrameState type

* [lens] Initial state for IndexPatternDatasource (#36052)

* [lens] Add first tests to indexpattern data source

* Respond to review comments

* Fix type definitions

* [lens] Editor frame initializes datasources and visualizations (#36060)

* [lens] Editor frame initializes datasources and visualizations

* Respond to review comments

* Fix build issues

* Fix state management issue

* [lens][draft] Lens/drag drop (#36268)

Add basic drag / drop component to Lens

* remove local package (#36456)

* [lens] Native renderer (#36165)

* Add nativerenderer component

* Use native renderer in app and editor frame

* [Lens] No explicit any (#36515)

* [Lens] Implement basic editor frame state handling (#36443)

* [lens] Load index patterns and render in data panel (#36463)

* [lens] Editor frame initializes datasources and visualizations

* Respond to review comments

* Fix build issues

* remove local package

* [lens] Load index patterns into data source

* Redo types for Index Pattern Datasource

* Fix one more type

* Respond to review comments

* [draft] Lens/line chart renderer (#36827)

Expression logic for the Lens xy chart.

* [lens] Index pattern data panel (initial) (#37015)

* [lens] Index pattern switcher

* Respond to review comments

* [Lens] Editor state 2 (#36513)

* [lens] Dimension panel that generates columns (#37117)

* [lens] Dimension panel that generates columns

* Update from review comments

* [lens] Generate esdocs queries from index pattern (#37361)

* [lens] Generate esdocs queries from index pattern

* Remove unused code

* Update yarn.lock from yarn kbn bootstrap

* [Lens] Add basic Lens xy chart suggestions (#37030)

Basic xy chart suggestions

* [Lens] Expression rendering (#37648)

* [Lens] Expression handling (#37876)

* [Lens] Lens/xy config panel (#37391)

Basic xy chart configuration panel

* [Lens] Xy expression building (#37967)

* [Lens] Initialize visualization with datasource api (#38142)

* [lens] Dimension panel lets users select operations and fields individually (#37573)

* [lens] Dimension panel lets users select operations and fields individually

* Split files and add tests

* Fix dimension labeling and add clear button

* Support more aggregations, aggregation nesting, rollups, and clearing

* Fix esaggs expression

* Increase top-level test coverage of dimension panel

* Update from review comments

* [Lens] Rename columns (#38278)

* [Lens] Lens/index pattern drag drop (#37711)

* Basic xy chart suggestions

* Re-apply XY config panel after force merge

* Initial integration of lens drag and drop

* Tweak naming, remove irellevant comment

* Tweaks per Wylie's feedback

* Add xy chart internationalization
Tweak types per Joe's feedback

* Update xy chart i18n implementation

* Fix i18n id

* Add drop tests to the lens index pattern

* improve tests

* [lens] Only allow aggregated dimensions (#38820)

* [lens] Only allow aggregated dimensions

* [lens] Index pattern suggest on drop

* Fully remove value

* Revert "[lens] Index pattern suggest on drop"

This reverts commit 604c6ed68c.

* Fix type errors

* [lens] Suggest on drop (#38848)

* [lens] Index pattern suggest on drop

* Add test for suggestion without date field

* fix merge

* [Lens] Parameter configurations and new dimension config flow (#38863)

* fix eslint failure

* [lens] Fix build by updating saved objects and i18n (#39391)

* [lens] Update location of saved objects code

* Update internatationalization

* Remove added file

* [lens] Fix arguments to esaggs using booleans (#39462)

* [lens] Datatable visualization plugin (#39390)

* [lens] Datatable visualization plugin

* Fix merge issues and add tests

* Update from review

* Fix file locations

* [lens] Use first suggestion when switching visualizations (#39377)

* [lens] Label each Y axis with its operation label (#39461)

* [lens] Label each Y axis with its operation label

* Remove comment

* Add link to chart issue

* [Lens] Suggestion preview rendering (#39576)

* [Lens] Popover configs (#39565)

* [Lens] Basic layouting (#39587)

* remove datasource public API in suggestions (#39772)

* [Lens] Basic save / load (#39257)

Add basic routing, save, and load to Lens

* [lens] Fix lint error

* [lens] Use node scripts/eslint.js --fix to fix errors

* [lens] Include link to lens from Visualize (#40542)

* [lens] Support stacking in xy visualization (#40546)

* [lens] Support stacking in xy visualization

* Use chart type switcher for stacked and horizontal xy charts

* Clean up remaining isStacked code

* Fix type error

* [Lens] Add xy split series support (#39726)

* Add split series to lens xy chart

* [lens] Lens Filter Ratio (#40196)

* WIP filter ratio

* Fix test issues

* Pass dependencies through plugin like new platform

* Pass props into filter ratio popover editor

* Provide mocks to filter_ratio popover test

* Add another test

* Clean up to prepare for review

* Clean up unnecessary changes

* Respond to review comments

* Fix tests

* [Lens] Terms order direction (#39884)

* fix types

* [Lens] Data panel styling and optimizations (#40787)

Style the data panel (mostly Joe Reuter's doing). Optimize a bunch of the Lens stack.

* [Lens] Optimize dimension panel flow (#41114)

* [Lens] re-introduce no-explicit-any (#41454)

* [Lens] No results marker (#41450)

* [lens] Support layers for visualizing results of multiple queries (#41290)

* [lens] WIP add support for layers

* [lens] WIP switch to nested tables

* Get basic layering to work

* Load multiple tables and render in one chart

* Fix priority ordering

* Reduce quantity of linting errors

* Ensure that new xy layer state has a split column

* Make the "add" y / split the trailing accessor

* Various fixes for datasource public API and implementation

* Unify datasource deletion and accessor removal

* Fix broken scss

* Fix xy visualization TypeScript errors?

* Build basic suggestions

* Restore save/load and fix typescript bugs

* simplify init routine

* fix save tests

* fix persistence tests

* fix state management tests

* Ensure the data table is aligned to the top

* Add layer support to Lens datatable

* Give xy chart a default layer initially

* Allow deletion of layers in xy charts

* xy: Make split accessor singular
Remove commented code blocks

* Change expression type for lens_merge_tables

* Fix XY chart rendering expression

* Fix type errors relating to `layerId` in table suggestions

* Pass around tables for suggestions with associated layerIds

* fix tests in workspace panel

* fix editor_frame tests

* Fix xy tests, skip inapplicable tests
that will be implemented in a separate PR

* add some tests for multiple datasources and layers

* Suggest that split series comes before X axis in XY chart

* Get datatable suggestion working

* Adjust how xy axis labels are computed

* Datasource suggestions should handle layers and have tests

* Fix linting in XY chart and remove commented code

* Update snapshots from earlier change

* Fix linting errors

* More cleanup

* Remove commented code

* Test the multi-column editor

* XY Visualization does not need to track datasourceId

* Fix various comments

* Remove unused xy prop
Add datasource header to datatable config

* Use operation labels for XY chart

* Adding and removing layers is reflected in the datasource

* rewrote datasource state init

* clean up editor_frame frame api implementation

* clean up editor frame

* [Lens] Embeddable (#41361)

* [lens] Move XY chart config into popover and fix layering (#41927)

* [lens] Move XY chart config into popover and fix layering

* Fix tests

* Update style

* Change wrapping of layer settings popover

* [Lens] Fix bugs in date_histogram and filter ratio (#42046)

* [Lens] Performance improvements (#41784)

* fix type error

* switch default size of terms operation to 3 (#42334)

* [lens] Improve suggestions for split series (#42052)

* [lens] Add chart switcher (#42093)

* solve merge conflicts

* fix test case

* [Lens] Allow only current visualization on field drop in workspace (#42344)

* [Lens] Remove indexpattern id on column (#42429)

* [lens] Implement app-level filtering and time picker (#42031)

* [lens] Implement app-level filtering and time picker

* More integration with filter bar

* Clean up test code and type errors

* Add frame level tests for syncing with app

* Add test coverage for app logic

* Simplify state management from app down

* Fix import errors

* Clarify whether properties are ids or titles for index pattern

* pass new saved object by ref

* add dirty state checking

* Fix tests

* [Lens] Add some tests around document handling in dimension panel (#42670)

* [Lens] Terms operation boolean support (#42817)

* [lens] Minor UX/UI improvements in Lens (#42852)

* Make dimension popover toggle when clicking button

* Without suggestions hide suggestion panel

* Add missing translations (#42921)

* [Lens] Config panel design (#42980)

* Fix up design of config panel

Does not include config popover

* Remove a couple of non-null assertions (#43013)

* Remove a couple of non-null assertions

* Remove orphaned import

* [Lens] Switch indexpattern manually (#42599)

* [Lens] Update frame to put suggestions at the bottom (#42997)

* fix type errors

* switch indexpattern on layer if there is only a single empty one (#43079)

* [Lens] Suggest reduced versions of current data table (#42537)

* [Lens] Field formatter support (#38874)

* Fix bugs

* [Lens] Add bucket nesting editor to indexpattern (#42869)

* [Lens] Remove unnecessary fields and indexing from mappings (#43285)

* [Lens] Xy scale type (#42142)

* [lens] Allow updater function to be used for updating state (#43373)

* [Lens] Lens metric visualization (#39364)

* Fix axis rotation (#43792)

* [Lens] Auto date histogram (#43775)

* Add auto date histogram

* Improve documentation and cleanup

* Add tests

* Change test name

* [Lens] Fix query bar integration (#43865)

* [Lens] Clean up operations code (#43784)

* [Lens] Functional tests (#44279)

Foundational layer for lens functional tests. Lens-specific page objects are not in this PR.

* [Lens] Add Lens visualizations to Visualize list (#43398)

* [Lens] Suggestion improvements (#43688)

* [lens] Calculate existence of fields in datasource (#44422)

* [lens] Calculate existence of fields in datasource

* Fix route registration

* Add page object and use existence in functional test

* Simplify layout of filters for index pattern

* Respond to review feedback

* Update class names

* Use new URL constant

* Fix usage of base path

* Fix lint errors

* [Lens ] Preview metric (#43755)

* format filter ratio as percentage (#44625)

* [Lens] Remove datasource suggestion id (#44495)

* [Lens] Make breadcrumbs look and feel like Visualize (#44258)

* [lens] Fix breakage from app-arch movements (#44720)

* [lens] Fix type error in test from merge

* [lens] Fix registration of embeddable (#45171)

* [Lens] Functional tests (#44814)

Basic functional tests for Lens, by no means comprehensive. This is more of a smokescreen test of some normal use cases.

* [lens] Add Lens to CODEOWNERS (#45296)

* [lens] Fix visualization alias registration

* [lens] Fix usage of EUI after typescript upgrade (#45404)

* [lens] Fix usage of EUI after typescript upgrade

* Use local fix instead of workaround

* [lens] Fix usage of expressions plugin (#45544)

* [lens] Fix usage of expressions plugin

* Use updated exports from #45538

* Fix imports and mocha tests

* Use relative instead of absolute path to fix tests

* [lens] More cleanup from QueryBar changes in master (#45687)

* [lens] Fix build and use new platform from entry points (#45834)

* [lens] Fix build and use new platform from entry points

* Fix params for existence route
2019-09-17 14:57:53 -04:00
Aleh Zasypkin
9d69b72a5f
Move base feature controls functionality from XPack Main plugin to a dedicated XPack Features plugin (#44664) 2019-09-09 19:15:58 +02:00
Renamed from x-pack/test/api_integration/apis/xpack_main/features/features.ts (Browse further)