API: avoid starting webserver when http.enabled=false

In some workflows such as simple file manipulation, starting a webserver is
unnecessary overhead, and we should be able to avoid it.

Here we introduce a new parameter `http.enabled`, which defaults to `true` to
maintain the existing functionality.

Resolves: elastic/logstash#9408
Closes: elastic/logstash#11525

Co-authored-by: Benoit Dupont <benoit.dupont@gmail.com>

Fixes #11533

config/logstash.yml

@@ -102,6 +102,27 @@ pipeline.ordered: auto
 #
 # config.support_escapes: false
 #
+# ------------ HTTP API Settings -------------
+# Define settings related to the HTTP API here.
+#
+# The HTTP API is enabled by default. It can be disabled, but features that rely
+# on it will not work as intended.
+# http.enabled: true
+#
+# By default, the HTTP API is bound to only the host's local loopback interface,
+# ensuring that it is not accessible to the rest of the network. Because the API
+# includes neither authentication nor authorization and has not been hardened or
+# tested for use as a publicly-reachable API, binding to publicly accessible IPs
+# should be avoided where possible.
+#
+# http.host: 127.0.0.1
+#
+# The HTTP API web server will listen on an available port from the given range.
+# Values can be specified as a single port (e.g., `9600`), or an inclusive range
+# of ports (e.g., `9600-9700`).
+#
+# http.port: 9600-9700
+#
 # ------------ Module Settings ---------------
 # Define modules here.  Modules definitions must be defined as an array.
 # The simple way to see this is to prepend each `name` with a `-`, and keep

logstash-core/lib/logstash/agent.rb

@@ -112,7 +112,7 @@ class LogStash::Agent
 
     converge_state_and_update
 
-    start_webserver
+    start_webserver_if_enabled
 
     if auto_reload?
       # `sleep_then_run` instead of firing the interval right away
@@ -384,6 +384,14 @@ class LogStash::Agent
     end
   end
 
+  def start_webserver_if_enabled
+    if @settings.get_value("http.enabled")
+      start_webserver
+    else
+      @logger.info("HTTP API is disabled (`http.enabled=false`); webserver will not be started.")
+    end
+  end
+
   def start_webserver
     @webserver_control_lock.synchronize do
       options = {:http_host => @http_host, :http_ports => @http_port, :http_environment => @http_environment }

logstash-core/lib/logstash/environment.rb

@@ -69,6 +69,7 @@ module LogStash
            Setting::Boolean.new("version", false),
            Setting::Boolean.new("help", false),
             Setting::String.new("log.format", "plain", true, ["json", "plain"]),
+           Setting::Boolean.new("http.enabled", true),
             Setting::String.new("http.host", "127.0.0.1"),
          Setting::PortRange.new("http.port", 9600..9700),
             Setting::String.new("http.environment", "production"),

logstash-core/lib/logstash/runner.rb

@@ -204,6 +204,11 @@ class LogStash::Runner < Clamp::StrictCommand
     :attribute_name => "config.reload.interval",
     :default => LogStash::SETTINGS.get_default("config.reload.interval")
 
+  option ["--http.enabled"], "ENABLED",
+         I18n.t("logstash.runner.flag.http_enabled"),
+         :attribute_name => 'http.enabled',
+         :default => LogStash::SETTINGS.get_default('http.enabled')
+
   option ["--http.host"], "HTTP_HOST",
     I18n.t("logstash.runner.flag.http_host"),
     :attribute_name => "http.host",

logstash-core/locales/en.yml

@@ -282,6 +282,9 @@ en:
           e.g. 'username:<password>'
         configtest: |+
           Check configuration for valid syntax and then exit.
+        http_enabled: |+
+          Can be used to disable the Web API, which is
+          enabled by default.
         http_host: Web API binding host
         http_port: Web API http port
         pipeline-id: |+