Apache Tomcat 8 (8.5.93-dev) - Monitoring and Managing Tomcat

## Monitoring and Managing Tomcat

### Table of Contents

- Introduction
- Enabling JMX Remote
- Manage Tomcat with JMX remote Ant Tasks
- JMXAccessorOpenTask - JMX open connection task
- JMXAccessorGetTask: get attribute value Ant task
- JMXAccessorSetTask: set attribute value Ant task
- JMXAccessorInvokeTask: invoke MBean operation Ant task
- JMXAccessorQueryTask: query MBean Ant task
- JMXAccessorCreateTask: remote create MBean Ant task
- JMXAccessorUnregisterTask: remote unregister MBean Ant task
- JMXAccessorCondition: express condition
- JMXAccessorEqualsCondition: equals MBean Ant condition
- Using the JMXProxyServlet

### Introduction

Monitoring is a key aspect of system administration. Looking inside a running server, obtaining some statistics or reconfiguring some aspects of an application are all daily administration tasks.

### Enabling JMX Remote

Note: This configuration is needed only if you are going to monitor Tomcat remotely. It is not needed if you are going to monitor it locally, using the same user that Tomcat runs with.

The Oracle website includes the list of options and how to configure JMX Remote on Java 8: http://docs.oracle.com/javase/6/docs/technotes/guides/management/agent.html.

The following is a quick configuration guide for Java 8:

Add the following parameters to `setenv.bat` script of your Tomcat (see RUNNING.txt for details).
Note: This syntax is for Microsoft Windows. The command has to be on the same line. It is wrapped to be more readable. If Tomcat is running as a Windows service, use its configuration dialog to set java options for the service. For Linux, MacOS, etc, remove `"set "` from beginning of the line.

    set CATALINA_OPTS=-Dcom.sun.management.jmxremote.port=%my.jmx.port%
      -Dcom.sun.management.jmxremote.rmi.port=%my.rmi.port%
      -Dcom.sun.management.jmxremote.ssl=false
      -Dcom.sun.management.jmxremote.authenticate=false

If you don't set `com.sun.management.jmxremote.rmi.port` then the JSR 160 JMX-Adaptor will select a port at random which will may it difficult to configure a firewall to allow access.

If you require TLS:

1. change and add this:

         -Dcom.sun.management.jmxremote.ssl=true
         -Dcom.sun.management.jmxremote.registry.ssl=true

2. to configure the protocols and/or cipher suites use:

         -Dcom.sun.management.jmxremote.ssl.enabled.protocols=%my.jmx.ssl.protocols%
         -Dcom.sun.management.jmxremote.ssl.enabled.cipher.suites=%my.jmx.cipher.suites%

3. to client certificate authentication use:

         -Dcom.sun.management.jmxremote.ssl.need.client.auth=%my.jmx.ssl.clientauth%

If you require authorization (it is strongly recommended that TLS is always used with authentication):

1. change and add this:

         -Dcom.sun.management.jmxremote.authenticate=true
         -Dcom.sun.management.jmxremote.password.file=../conf/jmxremote.password
         -Dcom.sun.management.jmxremote.access.file=../conf/jmxremote.access

2. edit the access authorization file $CATALINA_BASE/conf/jmxremote.access:

       monitorRole readonly
       controlRole readwrite

3. edit the password file $CATALINA_BASE/conf/jmxremote.password:

       monitorRole tomcat
       controlRole tomcat

   Tip: The password file should be read-only and only accessible by the operating system user Tomcat is running as.
4. Alternatively, you can configure a JAAS login module with:

         -Dcom.sun.management.jmxremote.login.config=%login.module.name%

If you need to specify a host name to be used in the RMI stubs sent to the client (e.g. because the public host name that must be used to connect is not the same as the local host name) then you can set:

    set CATALINA_OPTS=-Djava.rmi.server.hostname

If you need to specify a specific interface for the JMX service to bind to then you can set:

    set CATALINA_OPTS=-Dcom.sun.management.jmxremote.host

### Manage Tomcat with JMX remote Ant Tasks

To simplify JMX usage with Ant, a set of tasks is provided that may be used with antlib.

antlib: Copy your catalina-ant.jar from $CATALINA_HOME/lib to $ANT_HOME/lib.

The following example shows the JMX Accessor usage:
Note: The `name` attribute value was wrapped here to be more readable. It has to be all on the same line, without spaces.

    <project name="Catalina Ant JMX"
          xmlns:jmx="antlib:org.apache.catalina.ant.jmx"
          default="state"
          basedir=".">
      <property name="jmx.server.name" value="localhost" />
      <property name="jmx.server.port" value="9012" />
      <property name="cluster.server.address" value="192.168.1.75" />
      <property name="cluster.server.port" value="9025" />

      <target name="state" description="Show JMX Cluster state">
        <jmx:open
          host="${jmx.server.name}"
          port="${jmx.server.port}"
          username="controlRole"
          password="tomcat"/>
        <jmx:get
          name=
    "Catalina:type=IDataSender,host=localhost,
    senderAddress=${cluster.server.address},senderPort=${cluster.server.port}"
          attribute="connected"
          resultproperty="IDataSender.backup.connected"
          echo="false"
        />
        <jmx:get
          name="Catalina:type=ClusterSender,host=localhost"
          attribute="senderObjectNames"
          resultproperty="senderObjectNames"
          echo="false"
        />
        <!-- get current maxActiveSession from ClusterTest application
           echo it to Ant output and store at
           property <em>clustertest.maxActiveSessions.original</em>
        -->
        <jmx:get
          name="Catalina:type=Manager,context=/ClusterTest,host=localhost"
          attribute="maxActiveSessions"
          resultproperty="clustertest.maxActiveSessions.original"
          echo="true"
        />
        <!-- set maxActiveSession to 100
        -->
        <jmx:set
          name="Catalina:type=Manager,context=/ClusterTest,host=localhost"
          attribute="maxActiveSessions"
          value="100"
          type="int"
        />
        <!-- get all sessions and split result as delimiter <em>SPACE</em> for easy
           access all session ids directly with Ant property sessions.[0..n].
        -->
        <jmx:invoke
          name="Catalina:type=Manager,context=/ClusterTest,host=localhost"
          operation="listSessionIds"
          resultproperty="sessions"
          echo="false"
          delimiter=" "
        />
        <!-- Access session attribute <em>Hello</em> from first session.
        -->
        <jmx:invoke
          name="Catalina:type=Manager,context=/ClusterTest,host=localhost"
          operation="getSessionAttribute"
          resultproperty="Hello"
          echo="false"
        >
          <arg value="${sessions.0}"/>
          <arg value="Hello"/>
        </jmx:invoke>
        <!-- Query for all application manager.of the server from all hosts
           and bind all attributes from all found manager MBeans.
        -->
        <jmx:query
          name="Catalina:type=Manager,*"
          resultproperty="manager"
          echo="true"
          attributebinding="true"
        />
        <!-- echo the create properties -->
    <echo>
    senderObjectNames: ${senderObjectNames.0}
    IDataSender.backup.connected: ${IDataSender.backup.connected}
    session: ${sessions.0}
    manager.length: ${manager.length}
    manager.0.name: ${manager.0.name}
    manager.1.name: ${manager.1.name}
    hello: ${Hello}
    manager.ClusterTest.0.name: ${manager.ClusterTest.0.name}
    manager.ClusterTest.0.activeSessions: ${manager.ClusterTest.0.activeSessions}
    manager.ClusterTest.0.counterSend_EVT_SESSION_EXPIRED:
     ${manager.ClusterTest.0.counterSend_EVT_SESSION_EXPIRED}
    manager.ClusterTest.0.counterSend_EVT_GET_ALL_SESSIONS:
     ${manager.ClusterTest.0.counterSend_EVT_GET_ALL_SESSIONS}
    </echo>

      </target>

    </project>

import: Import the JMX Accessor Project with <import file="${CATALINA.HOME}/bin/catalina-tasks.xml" /> and reference the tasks with jmxOpen, jmxSet, jmxGet, jmxQuery, jmxInvoke, jmxEquals and jmxCondition.

### JMXAccessorOpenTask - JMX open connection task

List of Attributes

Attribute | Description | Default value
--- | --- | ---
url | Set JMX connection URL - service:jmx:rmi:///jndi/rmi://localhost:8050/jmxrmi |
host | Set the host, shortcut the very long URL syntax. | `localhost`
port | Set the remote connection port | `8050`
username | remote JMX connection user name. |
password | remote JMX connection password. |
ref | Name of the internal connection reference. With this attribute you can configure more the one connection inside the same Ant project. | `jmx.server`
echo | Echo the command usage (for access analysis or debugging) | `false`
if | Only execute if a property of the given name exists in the current project. |
unless | Only execute if a property of the given name not exists in the current project. |

Example to open a new JMX connection

      <jmx:open
        host="${jmx.server.name}"
        port="${jmx.server.port}"
      />

Example to open a JMX connection from URL, with authorization and store at other reference

      <jmx:open
        url="service:jmx:rmi:///jndi/rmi://localhost:9024/jmxrmi"
        ref="jmx.server.9024"
        username="controlRole"
        password="tomcat"
      />

Example to open a JMX connection from URL, with authorization and store at other reference, but only when property jmx.if exists and jmx.unless not exists

      <jmx:open
        url="service:jmx:rmi:///jndi/rmi://localhost:9024/jmxrmi"
        ref="jmx.server.9024"
        username="controlRole"
        password="tomcat"
        if="jmx.if"
        unless="jmx.unless"
      />

Note: All properties from jmxOpen task also exists at all other tasks and conditions.

### JMXAccessorGetTask: get attribute value Ant task

List of Attributes

Attribute | Description | Default value
--- | --- | ---
name | Full qualified JMX ObjectName -- Catalina:type=Server |
attribute | Existing MBean attribute (see Tomcat MBean description above) |
ref | JMX Connection reference | `jmx.server`
echo | Echo command usage (access and result) | `false`
resultproperty | Save result at this project property |
delimiter | Split result with delimiter (java.util.StringTokenizer) and use resultproperty as prefix to store tokens. |
separatearrayresults | When return value is an array, save result as property list ($resultproperty.[0..N] and $resultproperty.length) | `true`

Example to get remote MBean attribute from default JMX connection

      <jmx:get
        name="Catalina:type=Manager,context=/servlets-examples,host=localhost"
        attribute="maxActiveSessions"
        resultproperty="servlets-examples.maxActiveSessions"
      />

Example to get and result array and split it at separate properties

      <jmx:get
          name="Catalina:type=ClusterSender,host=localhost"
          attribute="senderObjectNames"
          resultproperty="senderObjectNames"
      />

Access the senderObjectNames properties with:

      ${senderObjectNames.length} give the number of returned sender list.
      ${senderObjectNames.[0..N]} found all sender object names

Example to get IDataSender attribute connected only when cluster is configured.
Note: The `name` attribute value was wrapped here to be more readable. It has to be all on the same line, without spaces.

      <jmx:query
        failonerror="false"
        name="Catalina:type=Cluster,host=${tomcat.application.host}"
        resultproperty="cluster"
      />
      <jmx:get
        name=
    "Catalina:type=IDataSender,host=${tomcat.application.host},
    senderAddress=${cluster.backup.address},senderPort=${cluster.backup.port}"
        attribute="connected"
        resultproperty="datasender.connected"
        if="cluster.0.name" />

### JMXAccessorSetTask: set attribute value Ant task

List of Attributes

Attribute | Description | Default value
--- | --- | ---
name | Full qualified JMX ObjectName -- Catalina:type=Server |
attribute | Existing MBean attribute (see Tomcat MBean description above) |
value | value that set to attribute |
type | type of the attribute. | `java.lang.String`
ref | JMX Connection reference | `jmx.server`
echo | Echo command usage (access and result) | `false`

Example to set remote MBean attribute value

      <jmx:set
        name="Catalina:type=Manager,context=/servlets-examples,host=localhost"
        attribute="maxActiveSessions"
        value="500"
        type="int"
      />

### JMXAccessorInvokeTask: invoke MBean operation Ant task

List of Attributes

Attribute | Description | Default value
--- | --- | ---
name | Full qualified JMX ObjectName -- Catalina:type=Server |
operation | Existing MBean operation |
ref | JMX Connection reference | `jmx.server`
echo | Echo command usage (access and result) | `false`
resultproperty | Save result at this project property |
delimiter | Split result with delimiter (java.util.StringTokenizer) and use resultproperty as prefix to store tokens. |
separatearrayresults | When return value is an array, save result as property list ($resultproperty.[0..N] and $resultproperty.length) | `true`

stop an application

      <jmx:invoke
        name="Catalina:type=Manager,context=/servlets-examples,host=localhost"
        operation="stop"/>

Now you can find the sessionid at ${sessions.[0..N} properties and access the count with ${sessions.length} property.

Example to get all sessionids

      <jmx:invoke
        name="Catalina:type=Manager,context=/servlets-examples,host=localhost"
        operation="listSessionIds"
        resultproperty="sessions"
        delimiter=" "
      />

Now you can find the sessionid at ${sessions.[0..N} properties and access the count with ${sessions.length} property.

Example to get remote MBean session attribute from session ${sessionid.0}

      <jmx:invoke
        name="Catalina:type=Manager,context=/ClusterTest,host=localhost"
        operation="getSessionAttribute"
        resultproperty="hello">
         <arg value="${sessionid.0}"/>
         <arg value="Hello" />
      </jmx:invoke>

Example to create a new access logger valve at vhost localhost

     <jmx:invoke
             name="Catalina:type=MBeanFactory"
             operation="createAccessLoggerValve"
             resultproperty="accessLoggerObjectName"
     >
         <arg value="Catalina:type=Host,host=localhost"/>
     </jmx:invoke>

Now you can find new MBean with name stored at ${accessLoggerObjectName} property.

### JMXAccessorQueryTask: query MBean Ant task

List of Attributes

Attribute | Description | Default value
--- | --- | ---
name | JMX ObjectName query string -- Catalina:type=Manager,* |
ref | JMX Connection reference | `jmx.server`
echo | Echo command usage (access and result) | `false`
resultproperty | Prefix project property name to all founded MBeans (mbeans.[0..N].objectname) |
attributebinding | bind ALL MBean attributes in addition to name | `false`
delimiter | Split result with delimiter (java.util.StringTokenizer) and use resultproperty as prefix to store tokens. |
separatearrayresults | When return value is an array, save result as property list ($resultproperty.[0..N] and $resultproperty.length) | `true`

Get all Manager ObjectNames from all services and Hosts

      <jmx:query
        name="Catalina:type=Manager,*
        resultproperty="manager" />

Now you can find the Session Manager at ${manager.[0..N].name} properties and access the result object counter with ${manager.length} property.

Example to get the Manager from servlet-examples application an bind all MBean properties

      <jmx:query
        name="Catalina:type=Manager,context=/servlet-examples,host=localhost*"
        attributebinding="true"
        resultproperty="manager.servletExamples" />

Now you can find the manager at ${manager.servletExamples.0.name} property and can access all properties from this manager with ${manager.servletExamples.0.[manager attribute names]}. The result object counter from MBeans is stored ad ${manager.length} property.

Example to get all MBeans from a server and store inside an external XML property file

    <project name="jmx.query"
                xmlns:jmx="antlib:org.apache.catalina.ant.jmx"
                default="query-all" basedir=".">
    <property name="jmx.host" value="localhost"/>
    <property name="jmx.port" value="8050"/>
    <property name="jmx.username" value="controlRole"/>
    <property name="jmx.password" value="tomcat"/>

    <target name="query-all" description="Query all MBeans of a server">
      <!-- Configure connection -->
      <jmx:open
        host="${jmx.host}"
        port="${jmx.port}"
        ref="jmx.server"
        username="${jmx.username}"
        password="${jmx.password}"/>

      <!-- Query MBean list -->
      <jmx:query
        name="*:*"
        resultproperty="mbeans"
        attributebinding="false"/>

      <echoproperties
        destfile="mbeans.properties"
        prefix="mbeans."
        format="xml"/>

      <!-- Print results -->
      <echo message=
        "Number of MBeans in server ${jmx.host}:${jmx.port} is ${mbeans.length}"/>
    </target>
    </project>

Now you can find all MBeans inside the file mbeans.properties.

### JMXAccessorCreateTask: remote create MBean Ant task

List of Attributes

Attribute | Description | Default value
--- | --- | ---
name | Full qualified JMX ObjectName -- Catalina:type=MBeanFactory |
className | Existing MBean full qualified class name (see Tomcat MBean description above) |
classLoader | ObjectName of server or web application classloader ( Catalina:type=ServerClassLoader,name=[server,common,shared] or Catalina:type=WebappClassLoader,context=/myapps,host=localhost) |
ref | JMX Connection reference | `jmx.server`
echo | Echo command usage (access and result) | `false`

Example to create remote MBean

      <jmx:create
        ref="${jmx.reference}"
        name="Catalina:type=MBeanFactory"
        className="org.apache.commons.modeler.BaseModelMBean"
        classLoader="Catalina:type=ServerClassLoader,name=server">
        <arg value="org.apache.catalina.mbeans.MBeanFactory" />
      </jmx:create>

Warning: Many Tomcat MBeans can't be linked to their parent once
created. The Valve, Cluster and Realm MBeans are not automatically
connected with their parent. Use the MBeanFactory create
operation instead.

### JMXAccessorUnregisterTask: remote unregister MBean Ant task

List of Attributes

Attribute | Description | Default value
--- | --- | ---
name | Full qualified JMX ObjectName -- Catalina:type=MBeanFactory |
ref | JMX Connection reference | `jmx.server`
echo | Echo command usage (access and result) | `false`

Example to unregister remote MBean

      <jmx:unregister
        name="Catalina:type=MBeanFactory"
      />

Warning: A lot of Tomcat MBeans can't be unregister.
The MBeans are not unlinked from their parent. Use MBeanFactory
remove operation instead.

### JMXAccessorCondition: express condition

List of Attributes

Attribute | Description | Default value
--- | --- | ---
url | Set JMX connection URL - service:jmx:rmi:///jndi/rmi://localhost:8050/jmxrmi |
host | Set the host, shortcut the very long URL syntax. | `localhost`
port | Set the remote connection port | `8050`
username | remote JMX connection user name. |
password | remote JMX connection password. |
ref | Name of the internal connection reference. With this attribute you can configure more the one connection inside the same Ant project. | `jmx.server`
name | Full qualified JMX ObjectName -- Catalina:type=Server |
echo | Echo condition usage (access and result) | `false`
if | Only execute if a property of the given name exists in the current project. |
unless | Only execute if a property of the given name not exists in the current project. |
value (required) | Second arg for operation |
type | Value type to express operation (support long and double) | `long`
operation | express one == equals != not equals > greater than (&gt;) >= greater than or equals (&gt;=) < lesser than (&lt;) <= lesser than or equals (&lt;=) | `==`

Wait for server connection and that cluster backup node is accessible

    <target name="wait">
      <waitfor maxwait="${maxwait}" maxwaitunit="second" timeoutproperty="server.timeout" >
        <and>
          <socket server="${server.name}" port="${server.port}"/>
          <http url="${url}"/>
          <jmx:condition
            operation="=="
            host="localhost"
            port="9014"
            username="controlRole"
            password="tomcat"
            name=
    "Catalina:type=IDataSender,host=localhost,senderAddress=192.168.111.1,senderPort=9025"
            attribute="connected"
            value="true"
          />
        </and>
      </waitfor>
      <fail if="server.timeout" message="Server ${url} don't answer inside ${maxwait} sec" />
      <echo message="Server ${url} alive" />
    </target>

### JMXAccessorEqualsCondition: equals MBean Ant condition

List of Attributes

Attribute | Description | Default value
--- | --- | ---
url | Set JMX connection URL - service:jmx:rmi:///jndi/rmi://localhost:8050/jmxrmi |
host | Set the host, shortcut the very long URL syntax. | `localhost`
port | Set the remote connection port | `8050`
username | remote JMX connection user name. |
password | remote JMX connection password. |
ref | Name of the internal connection reference. With this attribute you can configure more the one connection inside the same Ant project. | `jmx.server`
name | Full qualified JMX ObjectName -- Catalina:type=Server |
echo | Echo condition usage (access and result) | `false`

Wait for server connection and that cluster backup node is accessible

    <target name="wait">
      <waitfor maxwait="${maxwait}" maxwaitunit="second" timeoutproperty="server.timeout" >
        <and>
          <socket server="${server.name}" port="${server.port}"/>
          <http url="${url}"/>
          <jmx:equals
            host="localhost"
            port="9014"
            username="controlRole"
            password="tomcat"
            name=
    "Catalina:type=IDataSender,host=localhost,senderAddress=192.168.111.1,senderPort=9025"
            attribute="connected"
            value="true"
          />
        </and>
      </waitfor>
      <fail if="server.timeout" message="Server ${url} don't answer inside ${maxwait} sec" />
      <echo message="Server ${url} alive" />
    </target>

### Using the JMXProxyServlet

Tomcat offers an alternative to using remote (or even local) JMX connections while still giving you access to everything JMX has to offer: Tomcat's JMXProxyServlet.

The JMXProxyServlet allows a client to issue JMX queries via an HTTP interface. This technique offers the following advantages over using JMX directly from a client program:

- You don't have to launch a full JVM and make a remote JMX connection just to ask for one small piece of data from a running server
- You don't have to know how to work with JMX connections
- You don't need any of the complex configuration covered in the rest of this page
- Your client program does not have to be written in Java

A perfect example of JMX overkill can be seen in the case of popular server-monitoring software such as Nagios or Icinga: if you want to monitor 10 items via JMX, you will have to launch 10 JVMs, make 10 JMX connections, and then shut them all down every few minutes. With the JMXProxyServlet, you can make 10 HTTP connections and be done with it.

You can find out more information about the JMXProxyServlet in the documentation for the Tomcat manager.