Securing remote JMX

2016-09-01 java

Like it or not, JMX is one the main tools for JVM monitoring. If you are using Tomcat, Kafka or Cassandra you’ll have to setup JMX tools to monitor them.

But JMX has several drawbacks:

  • It’s based on the obsolete RMI protocol

  • It can trigger harmful functions like garbage collection

  • It can be used for nasty exploits like invoking arbitrary code

So securing JMX is not an option.

Going remote

By default, JMX is only locally accessible and secure: It can be accessed through Unix sockets. This means you need to have access to the machine and run JMX tools with the same user as your application. It’s usually enough for development but not for production.

To enable remote JMX, the documentations tells you turn on some JVM flags:

java -Dcom.sun.management.jmxremote=true \
     -Dcom.sun.management.jmxremote.port=31419 \
     -Dcom.sun.management.jmxremote.ssl=false \
     -Dcom.sun.management.jmxremote.authenticate=false \
     Tomcat

The JVM will starting listening on 0.0.0.0:31419 for JMX requests. Anybody will be able to plug any JMX tool (JConsole, JVisualVM, Mission Control…​) from a remote machine.

If you have a firewall (IPTable or the like) to protect your server, and opened the 31419 TCP port, or connecting through a tunnel, the JMX tools may fail to connect to the server. At first, the JMX client connects to port 31419, but gets redirected to a randomly chosen port (RMI WTF!). To prevent this behaviour and force the JVM to use a single port:

java -Dcom.sun.management.jmxremote=true \
     -Dcom.sun.management.jmxremote.port=31419 \
     -Dcom.sun.management.jmxremote.rmi.port=31419 \
     -Dcom.sun.management.jmxremote.ssl=false \
     -Dcom.sun.management.jmxremote.authenticate=false \
     Tomcat

File based authentication

This configuration is nice for debugging but not secure at all. Let’s add authentication on this JMX connection.

First create a password file, similar to /etc/password, containing login/password pairs:

jmxremote.password
admin  adminpassword
user   userpassword

Then create an access file, similar to /etc/groups, containing login/group pairs:

jmxremote.access
admin readwrite (1)
user  readonly (2)
1 admin has read/write access
2 user has read-only access

These two files should have limited access rights.

Finally, tell the JVM to use these files to authenticate users:

java -Dcom.sun.management.jmxremote=true \
     -Dcom.sun.management.jmxremote.port=31419 \
     -Dcom.sun.management.jmxremote.rmi.port=31419 \
     -Dcom.sun.management.jmxremote.ssl=false \
     -Dcom.sun.management.jmxremote.password.file=/path/to/jmxremote.password \
     -Dcom.sun.management.jmxremote.access.file=/path/to/jmxremote.access \
     Tomcat

Using SSL

With this configuration, JVM tools have to provide a login and password to access to MBeans. But, the password is sent over the wire without encryption. Let’s connect to JMX though SSL.

You may already known that, in the Java land, private keys and trusted certificates are stored in wallets known as keystores, and using the JKS file format. A tool named keytool provided with the JDK is used to import/export keys and certs in the keystore.

Once the keystore (containing private key matching the server name along with certificate chain) and the truststore (containing trusted certificates) are built, just reference them:

java -Dcom.sun.management.jmxremote=true \
     -Dcom.sun.management.jmxremote.port=31419 \
     -Dcom.sun.management.jmxremote.rmi.port=31419 \
     -Dcom.sun.management.jmxremote.password.file=/path/to/jmxremote.password \
     -Dcom.sun.management.jmxremote.access.file=/path/to/jmxremote.access \
     -Dcom.sun.management.jmxremote.registry.ssl=true \
     -Djavax.net.ssl.keyStore=/path/to/keystore.jks \
     -Djavax.net.ssl.keyStorePassword=keystore_password \
     -Djavax.net.ssl.trustStore=/path/to/truststore.jks \
     -Djavax.net.ssl.trustStorePassword=truststore_password \
     Tomcat

To connect, the JMX tools using SSL, you’ll have to provide the trusted certificates. For instance, to start the JConsole:

jconsole \
     -J-Djavax.net.ssl.trustStore=/path/to/truststore.jks \
     -J-Djavax.net.ssl.trustStorePassword=truststore_password

JMX configuration file

Again, this configuration has problem:

  • Having passwords on the command line is not a good option, because it’s easy to use ps command to grab them.

  • Having some many options on the command like is not elegant

Let’s place all these options in a dedicated file:

jmxremote.properties
com.sun.management.jmxremote=true
com.sun.management.jmxremote.port=31419
com.sun.management.jmxremote.rmi.port=31419
com.sun.management.jmxremote.password.file=/path/to/jmxremote.password
com.sun.management.jmxremote.access.file=/path/to/jmxremote.access
com.sun.management.jmxremote.registry.ssl=true
com.sun.management.jmxremote.ssl.config.file=/path/to/jmxremote.properties (1)

javax.net.ssl.keyStore=/path/to/keystore.jks
javax.net.ssl.keyStorePassword=keystore_password
javax.net.ssl.trustStore=/path/to/truststore.jks
javax.net.ssl.trustStorePassword=truststore_password
1 Path to file containing javax.net.ssl.* properties

And now, the command line sums up to

java -Dcom.sun.management.config.file=/path/to/jmxremote.properties \
     Tomcat

Such a JMX configuration file already exists in your JRE, it’s named JRE/lib/management/management.properties.

Some pointers

Other posts