How-To Configure a Valid (3rd Party) SSL/TLS Certificate for the Splunk HEC Input Port (8088)

HTTPS / TLS configuration on the Splunk HEC input port (default tcp/8088) differs slightly from the configuration of the web UI port (default tcp/8000) as the privKeyPath attribute is not available for configuration. Instead, the private key is appended the end of the file referenced in the serverCert attribute to create a "combo" certificate.

To configure a Splunk instance to use TLS encryption on the HEC input port, perform the following steps:

1) Upload the public key, private key, and intermediate certificate to Splunk.

Splunk, by default, uses cert.pem and privkey.pem as the file names for the public and private keys for the web UI certs. In this case, it is often simplest to combine those two files for re-use by HEC and we will assume this configuration method in this guide.

Splunk Docs recommends the following location and for standardization purposes, I recommend the same.

$SPLUNK_HOME/etc/auth/insert-org-here/

Modify "insert-org-here" to instead reference a short unique identifier for your organization.

Note that although not required, providing the intermediate certificate (excluding the root CA cert) is a best practice. The intermediate certificates are provided as a bundle from the CA and should be appended to the public key in the following order:

-----BEGIN CERTIFICATE----- # Your public key here #\
MIIFqjCCBJKgAwIBAgIQT0eb7D7qe0Fv+IPt8WRkPTANBgkqhkiG9w0BAQsFADCB
jzELMAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4G
...truncated...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE----- # Your lowest level intermediate here #
MIIFqjCCBJKgAwIBAgIQT0eb7D7qe0Fv+IPt8WRkPTANBgkqhkiG9w0BAQsFADCB
jzELMAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4G
...truncated...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE----- # highest level (closest to root CA cert) here #
MIIFqjCCBJKgAwIBAgIQT0eb7D7qe0Fv+IPt8WRkPTANBgkqhkiG9w0BAQsFADCB
jzELMAkGA1UEBhMCR0IxGzAZBgNVBAgTEkdyZWF0ZXIgTWFuY2hlc3RlcjEQMA4G
...truncated...
-----END CERTIFICATE-----

For reference, the private key looks like this:

-----BEGIN PRIVATE KEY-----
MIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQCmPQfkbtXcvaxI
...truncated...
-----END PRIVATE KEY-----

When the certs have been uploaded, we should see the following in place:

[splunk@lab01 ~]# ll $SPLUNK_HOME/etc/auth/insert-org-here/
total 8
-rw-r--r-- 1 splunk splunk 2025 Apr 23 20:29 cert.pem
-rw-r--r-- 1 splunk splunk 1704 Apr 23 20:18 privkey.pem

2) Combine the public and private key files.

cat cert.pem >> combo.pem && cat privkey.pem >> combo.pem

This will create a new combined key fiie that we will reference in step 3.

[splunk@lab01 ~]# ll $SPLUNK_HOME/etc/auth/insert-org-here/
total 20
-rw-r--r-- 1 splunk splunk 2025 Apr 23 20:29 cert.pem
-rw-rw-r-- 1 splunk splunk 7852 Jul  7 18:51 combo.pem
-rw-r--r-- 1 splunk splunk 1704 Apr 23 20:18 privkey.pem

3) Add the following attributes to your inputs.conf file:

vi $SPLUNK_HOME/etc/system/local/inputs.conf

[http]
enableSSL = 1
serverCert = /opt/splunk/etc/auth/cse-lab/combo.pem

4) Finally, restart splunkd:

via systemd:

sudo systemctl restart Splunkd

or via initd:

$SPLUNK_HOME/bin/splunk restart

5) In the event of an issue after splunkd restart (web service does not start etc.) check the following:

netstat -an | grep 8000 (insert your expected web port here)
tail (-f) $SPLUNK_HOME/var/log/splunk/web_services.log
openssl s_client -connect my.hostname.com:8088

The first command lets us know if Splunk made it to the point that it is listening on the web port at all or not. web_services.log provides a more conclusive view into the nature of the error and will call out if a certificate file could not be found or read. OpenSSL simulates a connection to the HEC port and validates the SSL handshake and configuration.

Resources

(1) Splunk Docs: Secure Splunk Web Using a Signed Certificate

https://docs.splunk.com/Documentation/Splunk/latest/Security/SecureSplunkWebusingasignedcertificate

(2) Splunk Docs: Get Third Party Certificates for Splunk Web

https://docs.splunk.com/Documentation/Splunk/latest/Security/Getthird-partycertificatesforSplunkWeb

(3) Splunk Docs: inputs.conf spec

https://docs.splunk.com/Documentation/Splunk/latest/Admin/Inputsconf