diff --git a/docs/static/releasenotes.asciidoc b/docs/static/releasenotes.asciidoc index 1bceeab1d..57d9d13d2 100644 --- a/docs/static/releasenotes.asciidoc +++ b/docs/static/releasenotes.asciidoc @@ -17,16 +17,7 @@ coming[7.0.0] The list combines release notes from the 7.0.0-alpha1, -alpha2, -beta1, -rc1 and -rc2 releases. -[[logstash-7-0-0-rc2]] -=== Logstash 7.0.0-rc2 Release Notes - -* logstash-input-snmp - - Added support for querying SNMP tables - - Changed three error messages in the base_client to include the target address for clarity in the logs. - -[[logstash-7-0-0-rc1]] -=== Logstash 7.0.0-rc1 Release Notes - +==== Logstash core * BUGFIX: Correctly count total queued items across multiple pipelines https://github.com/elastic/logstash/pull/10564[#10564] * BUGFIX: Fix issue setting 'enable_metric => false' https://github.com/elastic/logstash/pull/10538[#10538] * BUGFIX: Prevent concurrent convergence of pipeline actions https://github.com/elastic/logstash/pull/10537[#10537] @@ -36,6 +27,208 @@ The list combines release notes from the 7.0.0-alpha1, -alpha2, -beta1, -rc1 and * Improve docs about using Filebeat modules with Logstash https://github.com/elastic/logstash/pull/10438[#10438] * Bump JRuby to 9.2.6.0 https://github.com/elastic/logstash/pull/10425[#10425] * BUGFIX: Remove exclusive lock for Ruby pipeline initialization https://github.com/elastic/logstash/pull/10462[#10462] +* Update Java dependencies https://github.com/elastic/logstash/pull/10340[#10340] +* Remove pipeline output workers setting https://github.com/elastic/logstash/pull/10358[#10358] +* Cleanup Ruby gems dependencies https://github.com/elastic/logstash/pull/10171[#10171] +* Ensure compatibility of module data with ES and Kibana 7.0 https://github.com/elastic/logstash/pull/10356[#10356] +* Rename x-pack monitoring and management config option .url and .ca to .hosts and .certificate_authority https://github.com/elastic/logstash/pull/10380[#10380] +* BUGFIX: building of deb and rpm artifacts https://github.com/elastic/logstash/pull/10396[#10396] +* Make Java execution the default https://github.com/elastic/logstash/pull/8649[#8649] +* Field-reference parsing is now strict by default https://github.com/elastic/logstash/pull/9543[#9543] +* Improvements to core Javaification +* BUGFIX: Support for Byte, Short and Date type conversions as seen in the rabbitmq input plugin https://github.com/elastic/logstash/pull/9984[#9984] + +==== Plugins +Here are the plugin changes. + +===== Codec plugins +* logstash-codec-cef + - Removed obsolete `sev` and `deprecated_v1_fields` fields + - Fixed minor doc inconsistencies (added reverse_mapping to options table, moved it to alpha order in option descriptions, fixed typo) + https://github.com/logstash-plugins/logstash-codec-cef/pull/60[#60] +* logstash-codec-es_bulk + - Add documentation about use with http input +* logstash-codec-netflow + - Fix sub-second timestamp math + - BREAKING: Added support for RFC6759 decoding of application_id. This is a breaking change to the way application_id is decoded. The format changes from e.g. 0:40567 to 0..12356..40567 + - Fixed IPFIX options template parsing for Juniper MX240 JunOS 15.1 + - Fixed incorrect parsing of zero-filled Netflow 9 packets from Palo Alto + - Added support for Netflow v9 devices with VarString fields (H3C Netstream) + - Reduced complexity of creating, persisting, loading an retrieving template caches + - Fixed issue where TTL in template registry was not being respected + - Added Cisco ACI to list of known working Netflow v9 exporters + - Added support for IXIA Packet Broker IPFIX + - Fixed issue with Procera float fields + +===== Filter plugins +* logstash-filter-aggregate + - new feature: add ability to dynamically define a custom `timeout` or `inactivity_timeout` in `code` block (fix issues https://github.com/logstash-plugins/logstash-filter-aggregate/issues/91[#91] and https://github.com/logstash-plugins/logstash-filter-aggregate/issues/92[#92]) + - new feature: add meta informations available in `code` block through `map_meta` variable + - new feature: add Logstash metrics, specific to aggregate plugin: aggregate_maps, pushed_events, task_timeouts, code_errors, timeout_code_errors + - new feature: validate at startup that `map_action` option equals to 'create', 'update' or 'create_or_update' +* logstash-filter-clone + - Make 'clones' a required option + - Added a warning when 'clones' is empty since that results in a no-op https://github.com/logstash-plugins/logstash-filter-clone/issues/14[#14] +* logstash-filter-de_dot + - fix failure of fieldnames with boolean value "false" +* logstash-filter-dns + - Fixed issue where unqualified domains would fail to resolve when running this plugin with Logstash 5.x https://github.com/logstash-plugins/logstash-filter-dns/pull/48[#48] + - Fixed crash that could occur when encountering certain classes of invalid inputs https://github.com/logstash-plugins/logstash-filter-dns/pull/49[#49] +* logstash-filter-elasticsearch + - Add support for extracting hits total from Elasticsearch 7.x responses + - Added connection check during register to avoid failures during processing + - Changed Elasticsearch Client transport to use Manticore + - Changed amount of logging details during connection failure +* logstash-filter-fingerprint + - Fixed concurrent SHA fingerprinting by making the instances thread local +* logstash-filter-geoip + - Removed obsolete lru_cache_size field +* NEW: logstash-filter-http + - Beta version of HTTP filter plugin based on @lucashenning's https://github.com/lucashenning/logstash-filter-rest[REST filter]. + - Fixed minor documentation issues https://github.com/logstash-plugins/logstash-filter-http/pull/9[#9] + - Minor documentation fixes +* logstash-filter-jdbc_static + - Added info to documentation to emphasize significance of table order https://github.com/logstash-plugins/logstash-filter-jdbc_static/pull/36[36] +* logstash-filter-jdbc_streaming + - Swap out mysql for postgresql for testing https://github.com/logstash-plugins/logstash-filter-jdbc_streaming/pull/11[#11] +* logstash-filter-json + - Updated documentation with some clarifications and fixes +* logstash-filter-kv + - Added a timeout enforcer which prevents inputs that are pathological against the generated parser from blocking + the pipeline. By default, timeout is a generous 30s, but can be configured or disabled entirely with the new + `timeout_millis` and `tag_on_timeout` directives https://github.com/logstash-plugins/logstash-filter-kv/pull/79[#79] + - Made error-handling configurable with `tag_on_failure` directive. +* NEW: logstash-filter-memcached + - Updated to 1.0.0 + - The plugin common options (e.g., `add_field`, `add_tag`, etc.) are now correctly only invoked when the plugin successfully gets one or more values from, or sets one or more values to memcached (#4) + - Fix links to argument types in documentation (#3) +* logstash-filter-metrics + - Fixed two minor typos in documentation +* logstash-filter-mutate + - Added ability to directly convert from integer and float to boolean https://github.com/logstash-plugins/logstash-filter-mutate/pull/127[#127] +* logstash-filter-split + - Fixed numeric values, optimized @target verification, cleanups and specs https://github.com/logstash-plugins/logstash-filter-split/pull/36[#36] +* logstash-filter-xml + - Fixed creation of empty arrays when xpath failed https://github.com/logstash-plugins/logstash-filter-xml/pull/59[#59] + - Fixed force_array behavior with nested elements https://github.com/logstash-plugins/logstash-filter-xml/pull/57[#57] + +===== Input plugins +* logstash-input-azure_event_hubs + - Updated Azure event hub library dependencies https://github.com/logstash-plugins/logstash-input-azure_event_hubs/pull/27[#27] +* logstash-input-beats + - Removed obsolete setting congestion_threshold and target_field_for_codec + - Changed default value of `add_hostname` to false + - Loosen jar-dependencies manager gem dependency to allow plugin to work with JRubies that include a later version + - Updated jar dependencies to reflect newer releases +* logstash-input-elasticsearch + - Added managed slice scrolling with `slices` option +* logstash-input-file + - Fixed problem in Windows where some paths would fail to return an identifier ("inode"). Make path into a C style String before encoding to UTF-16LE. https://github.com/logstash-plugins/logstash-input-file/issues/232[#232] + - Fixed issue where logs were being spammed with needless error messages https://github.com/logstash-plugins/logstash-input-file/pull/224[#224] + - Fixed problem in tail and read modes where the read loop could get stuck if an IO error occurs in the loop. + The file appears to be being read but it is not, suspected with file truncation schemes. + https://github.com/logstash-plugins/logstash-input-file/issues/205[#205] + - Fixed problem in rotation handling where the target file being rotated was + subjected to the start_position setting when it must always start from the beginning. + https://github.com/logstash-plugins/logstash-input-file/issues/214[#214] +* logstash-input-gelf + - Fixed shutdown handling, robustness in socket closing and restarting, json parsing, code DRYing and cleanups https://github.com/logstash-plugins/logstash-input-gelf/pull/62[#62] +* logstash-input-http + - Added configurable response code option https://github.com/logstash-plugins/logstash-input-http/pull/103[#103] + - Added explanation about operation order of codec and additional_codecs https://github.com/logstash-plugins/logstash-input-http/pull/104[#104] + - Added configurable response code option https://github.com/logstash-plugins/logstash-input-http/pull/103[#103] + - Added explanation about operation order of codec and additional_codecs https://github.com/logstash-plugins/logstash-input-http/pull/104[#104] + - Loosen jar-dependencies manager gem dependency to allow plugin to work with JRubies that include a later version. + - Changed jar dependencies to reflect newer versions +* logstash-input-http_poller + - Fixed minor doc and doc formatting issues https://github.com/logstash-plugins/logstash-input-http_poller/pull/107[#107] + - Removed obsolete field `interval` + - Changed `schedule` entry to show that it is required + https://github.com/logstash-plugins/logstash-input-http_poller/pull/102[#102] +* logstash-input-kafka + - Removed obsolete `ssl` option + - Added support for kafka property ssl.endpoint.identification.algorithm https://github.com/logstash-plugins/logstash-input-kafka/pull/302[#302] + - Changed Kafka client version to 2.1.0 + - Changed Kafka client version to 2.0.1 https://github.com/logstash-plugins/logstash-input-kafka/pull/295[#295] +* logstash-input-snmp + - Added no_codec condition to the documentation and bumped version https://github.com/logstash-plugins/logstash-input-snmp/pull/39[#39] + - Changed docs to improve options layout https://github.com/logstash-plugins/logstash-input-snmp/pull/38[#38] + - Added support for querying SNMP tables + - Changed three error messages in the base_client to include the target address for clarity in the logs. +* logstash-input-sqs + - Added support for multiple events inside same message from SQS https://github.com/logstash-plugins/logstash-input-sqs/pull/48[#48] +* logstash-input-tcp + - Removed obsolete `data_timeout` and `ssl_cacert` options + - Fixed race condition where data would be accepted before queue was configured + - Support multiple certificates per file https://github.com/logstash-plugins/logstash-input-tcp/pull/140[#140] + +===== Output plugins +* logstash-output-elasticsearch + - Remove support for parent child (still support join data type) since we don't support multiple document types any more + - Removed obsolete `flush_size` and `idle_flush_time` + - Added 'auto' setting for ILM with default of 'auto' https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/838[#838] + - Fixed sniffing support for 7.x https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/827[#827] + - Fixed issue with escaping index names which was causing writing aliases for ILM to fail https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/831[#831] + - Adds support for Index Lifecycle Management for Elasticsearch 6.6.0 and above, running with at least a Basic License(Beta) https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/805[#805] + - Fixed support for Elasticsearch 7.x https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/812[#812] + - Tweaked logging statements to reduce verbosity + - Fixed numerous issues relating to builds on Travis https://github.com/logstash-plugins/logstash-output-elasticsearch/pull/799[#799] +* logstash-output-file + - Removed JRuby check when using FIFOs https://github.com/logstash-plugins/logstash-output-file/pull/75[#75] +* logstash-output-http + - Relax dependency on http_client mixin since current major works on both + - Fixed handling of empty `retryable_codes` https://github.com/logstash-plugins/logstash-output-http/pull/99[#99] +* logstash-output-kafka + - Fixed issue with unnecessary sleep after retries exhausted https://github.com/logstash-plugins/logstash-output-kafka/pull/216[#216] + - Removed obsolete `block_on_buffer_full`, `ssl` and `timeout_ms` options + - Added support for kafka property `ssl.endpoint.identification.algorithm` https://github.com/logstash-plugins/logstash-output-kafka/pull/213[#213] + - Changed Kafka client to version 2.1.0 + - Changed Kafka client to version 2.0.1 https://github.com/logstash-plugins/logstash-output-kafka/pull/209[#209] +* logstash-output-pagerduty + - Update _development_ dependency webmock to latest version to prevent conflicts in logstash core's dependency matrix. +* logstash-output-redis + - Removed obsolete fields `queue` and `name` + - Changed major version of redis library dependency to 4.x +* logstash-output-s3 + - Add support for setting mutipart upload threshold https://github.com/logstash-plugins/logstash-output-s3/pull/202[#202] + - Fixed issue where on restart, 0 byte files could erroneously be uploaded to s3 https://github.com/logstash-plugins/logstash-output-s3/issues/195[#195] +* logstash-output-sqs + - Removed obsolete fields `batch` and `batch_timeout` + - Removed workaround to JRuby bug https://github.com/jruby/jruby/issues/3645[#3645] +* logstash-output-tcp + - Removed obsolete field `message_format` + - Removed requirement to have a certificate/key pair when enabling ssl + + +* logstash-mixin-http_client + - Removed obsolete ssl_certificate_verify option + + + + +[[logstash-7-0-0-rc2]] +=== Logstash 7.0.0-rc2 Release Notes + +==== Plugins +* logstash-input-snmp + - Added support for querying SNMP tables + - Changed three error messages in the base_client to include the target address for clarity in the logs. + +[[logstash-7-0-0-rc1]] +=== Logstash 7.0.0-rc1 Release Notes + +==== Logstash core +* BUGFIX: Correctly count total queued items across multiple pipelines https://github.com/elastic/logstash/pull/10564[#10564] +* BUGFIX: Fix issue setting 'enable_metric => false' https://github.com/elastic/logstash/pull/10538[#10538] +* BUGFIX: Prevent concurrent convergence of pipeline actions https://github.com/elastic/logstash/pull/10537[#10537] +* Monitoring: Change internal document type to push "_doc" instead of "doc" https://github.com/elastic/logstash/pull/10533[#10533] +* BUGFIX: Allow explicitly-specified Java codecs https://github.com/elastic/logstash/pull/10520[#10520] +* Central management typeless API https://github.com/elastic/logstash/pull/10421[#10421] +* Improve docs about using Filebeat modules with Logstash https://github.com/elastic/logstash/pull/10438[#10438] +* Bump JRuby to 9.2.6.0 https://github.com/elastic/logstash/pull/10425[#10425] +* BUGFIX: Remove exclusive lock for Ruby pipeline initialization https://github.com/elastic/logstash/pull/10462[#10462] + +==== Plugins * logstash-filter-dns - Fixed issue where unqualified domains would fail to resolve when running this plugin with Logstash 5.x https://github.com/logstash-plugins/logstash-filter-dns/pull/48[#48] - Fixed crash that could occur when encountering certain classes of invalid inputs https://github.com/logstash-plugins/logstash-filter-dns/pull/49[#49] @@ -59,12 +252,15 @@ The list combines release notes from the 7.0.0-alpha1, -alpha2, -beta1, -rc1 and [[logstash-7-0-0-beta1]] === Logstash 7.0.0-beta1 Release Notes +==== Logstash core * Update Java dependencies https://github.com/elastic/logstash/pull/10340[#10340] * Remove pipeline output workers setting https://github.com/elastic/logstash/pull/10358[#10358] * Cleanup Ruby gems dependencies https://github.com/elastic/logstash/pull/10171[#10171] * Ensure compatibility of module data with ES and Kibana 7.0 https://github.com/elastic/logstash/pull/10356[#10356] * Rename x-pack monitoring and management config option .url and .ca to .hosts and .certificate_authority https://github.com/elastic/logstash/pull/10380[#10380] * BUGFIX: building of deb and rpm artifacts https://github.com/elastic/logstash/pull/10396[#10396] + +==== Plugins * logstash-codec-cef - Removed obsolete `sev` and `deprecated_v1_fields` fields - Fixed minor doc inconsistencies (added reverse_mapping to options table, moved it to alpha order in option descriptions, fixed typo) @@ -145,6 +341,7 @@ The list combines release notes from the 7.0.0-alpha1, -alpha2, -beta1, -rc1 and [[logstash-7-0-0-alpha2]] === Logstash 7.0.0-alpha2 Release Notes +==== Plugins * logstash-filter-elasticsearch - Add support for extracting hits total from Elasticsearch 7.x responses - Added connection check during register to avoid failures during processing @@ -185,10 +382,13 @@ The list combines release notes from the 7.0.0-alpha1, -alpha2, -beta1, -rc1 and [[logstash-7-0-0-alpha1]] === Logstash 7.0.0-alpha1 Release Notes +==== Logstash core * Make Java execution the default https://github.com/elastic/logstash/pull/8649[#8649] * Field-reference parsing is now strict by default https://github.com/elastic/logstash/pull/9543[#9543] * Improvements to core Javaification * BUGFIX: Support for Byte, Short and Date type conversions as seen in the rabbitmq input plugin https://github.com/elastic/logstash/pull/9984[#9984] + +==== Plugins * logstash-codec-netflow - BREAKING: Added support for RFC6759 decoding of application_id. This is a breaking change to the way application_id is decoded. The format changes from e.g. 0:40567 to 0..12356..40567 - Fixed IPFIX options template parsing for Juniper MX240 JunOS 15.1