Writing Custom Task Classes - Gradle User Guide Version 4.4

# Chapter 40. Writing Custom Task Classes

Gradle supports two types of task. One such type is the simple task, where you define the task with an action closure. We have seen these in Chapter 16, Build Script Basics. For this type of task, the action closure determines the behaviour of the task. This type of task is good for implementing one-off tasks in your build script.

The other type of task is the enhanced task, where the behaviour is built into the task, and the task provides some properties which you can use to configure the behaviour. We have seen these in Chapter 19, More about Tasks. Most Gradle plugins use enhanced tasks. With enhanced tasks, you don’t need to implement the task behaviour as you do with simple tasks. You simply declare the task and configure the task using its properties. In this way, enhanced tasks let you reuse a piece of behaviour in many different places, possibly across different builds.

The behaviour and properties of an enhanced task is defined by the task’s class. When you declare an enhanced task, you specify the type, or class of the task.

Implementing your own custom task class in Gradle is easy. You can implement a custom task class in pretty much any language you like, provided it ends up compiled to bytecode. In our examples, we are going to use Groovy as the implementation language. Groovy, Java or Kotlin are all good choices as the language to use to implement a task class, as the Gradle API has been designed to work well with these languages. In general, a task implemented using Java or Kotlin, which are statically typed, will perform better than the same task implemented using Groovy.

## 40.1. Packaging a task class

There are several places where you can put the source for the task class.

Build script

  You can include the task class directly in the build script. This has the benefit that the task class is automatically compiled and included in the classpath of the build script without you having to do anything. However, the task class is not visible outside the build script, and so you cannot reuse the task class outside the build script it is defined in.

`buildSrc` project

  You can put the source for the task class in the `rootProjectDir/buildSrc/src/main/groovy` directory. Gradle will take care of compiling and testing the task class and making it available on the classpath of the build script. The task class is visible to every build script used by the build. However, it is not visible outside the build, and so you cannot reuse the task class outside the build it is defined in. Using the `buildSrc` project approach separates the task declaration - that is, what the task should do - from the task implementation - that is, how the task does it.

  See Chapter 43, Organizing Build Logic for more details about the `buildSrc` project.

Standalone project

  You can create a separate project for your task class. This project produces and publishes a JAR which you can then use in multiple builds and share with others. Generally, this JAR might include some custom plugins, or bundle several related task classes into a single library. Or some combination of the two.

In our examples, we will start with the task class in the build script, to keep things simple. Then we will look at creating a standalone project.

## 40.2. Writing a simple task class

To implement a custom task class, you extend `DefaultTask`.

Example 40.1. Defining a custom task

`build.gradle`

    class GreetingTask extends DefaultTask {
    }

This task doesn’t do anything useful, so let’s add some behaviour. To do so, we add a method to the task and mark it with the `TaskAction` annotation. Gradle will call the method when the task executes. You don’t have to use a method to define the behaviour for the task. You could, for instance, call `doFirst()` or `doLast()` with a closure in the task constructor to add behaviour.

Example 40.2. A hello world task

`build.gradle`

    class GreetingTask extends DefaultTask {
        @TaskAction
        def greet() {
            println 'hello from GreetingTask'
        }
    }

    // Create a task using the task type
    task hello(type: GreetingTask)

Output of `gradle -q hello`

    > gradle -q hello
    hello from GreetingTask

Let’s add a property to the task, so we can customize it. Tasks are simply POGOs, and when you declare a task, you can set the properties or call methods on the task object. Here we add a `greeting` property, and set the value when we declare the `greeting` task.

Example 40.3. A customizable hello world task

`build.gradle`

    class GreetingTask extends DefaultTask {
        String greeting = 'hello from GreetingTask'

        @TaskAction
        def greet() {
            println greeting
        }
    }

    // Use the default greeting
    task hello(type: GreetingTask)

    // Customize the greeting
    task greeting(type: GreetingTask) {
        greeting = 'greetings from GreetingTask'
    }

Output of `gradle -q hello greeting`

    > gradle -q hello greeting
    hello from GreetingTask
    greetings from GreetingTask

## 40.3. A standalone project

Now we will move our task to a standalone project, so we can publish it and share it with others. This project is simply a Groovy project that produces a JAR containing the task class. Here is a simple build script for the project. It applies the Groovy plugin, and adds the Gradle API as a compile-time dependency.

Example 40.4. A build for a custom task

`build.gradle`

    apply plugin: 'groovy'

    dependencies {
        compile gradleApi()
        compile localGroovy()
    }

Note: The code for this example can be found at `samples/customPlugin/plugin` in the ‘-all’ distribution of Gradle.

We just follow the convention for where the source for the task class should go.

Example 40.5. A custom task

`src/main/groovy/org/gradle/GreetingTask.groovy`

    package org.gradle

    import org.gradle.api.DefaultTask
    import org.gradle.api.tasks.TaskAction

    class GreetingTask extends DefaultTask {
        String greeting = 'hello from GreetingTask'

        @TaskAction
        def greet() {
            println greeting
        }
    }

### 40.3.1. Using your task class in another project

To use a task class in a build script, you need to add the class to the build script’s classpath. To do this, you use a `buildscript { }` block, as described in Section 43.6, “External dependencies for the build script”. The following example shows how you might do this when the JAR containing the task class has been published to a local repository:

Example 40.6. Using a custom task in another project

`build.gradle`

    buildscript {
        repositories {
            maven {
                url uri('../repo')
            }
        }
        dependencies {
            classpath group: 'org.gradle', name: 'customPlugin',
                      version: '1.0-SNAPSHOT'
        }
    }

    task greeting(type: org.gradle.GreetingTask) {
        greeting = 'howdy!'
    }

### 40.3.2. Writing tests for your task class

You can use the `ProjectBuilder` class to create `Project` instances to use when you test your task class.

Example 40.7. Testing a custom task

`src/test/groovy/org/gradle/GreetingTaskTest.groovy`

    class GreetingTaskTest {
        @Test
        public void canAddTaskToProject() {
            Project project = ProjectBuilder.builder().build()
            def task = project.task('greeting', type: GreetingTask)
            assertTrue(task instanceof GreetingTask)
        }
    }

## 40.4. Incremental tasks

Incremental tasks are an incubating feature.

Since the introduction of the implementation described above (early in the Gradle 1.6 release cycle), discussions within the Gradle community have produced superior ideas for exposing the information about changes to task implementors to what is described below. As such, the API for this feature will almost certainly change in upcoming releases. However, please do experiment with the current implementation and share your experiences with the Gradle community.

The feature incubation process, which is part of the Gradle feature lifecycle (see Appendix C, The Feature Lifecycle), exists for this purpose of ensuring high quality final implementations through incorporation of early user feedback.

With Gradle, it’s very simple to implement a task that is skipped when all of its inputs and outputs are up to date (see Section 19.10, “Up-to-date checks (AKA Incremental Build)”). However, there are times when only a few input files have changed since the last execution, and you’d like to avoid reprocessing all of the unchanged inputs. This can be particularly useful for a transformer task, that converts input files to output files on a 1:1 basis.

If you’d like to optimise your build so that only out-of-date inputs are processed, you can do so with an incremental task.

### 40.4.1. Implementing an incremental task

For a task to process inputs incrementally, that task must contain an incremental task action. This is a task action method that contains a single `IncrementalTaskInputs` parameter, which indicates to Gradle that the action will process the changed inputs only.

The incremental task action may supply an `IncrementalTaskInputs.outOfDate(org.gradle.api.Action)` action for processing any input file that is out-of-date, and a `IncrementalTaskInputs.removed(org.gradle.api.Action)` action that executes for any input file that has been removed since the previous execution.

Example 40.8. Defining an incremental task action

`build.gradle`

    class IncrementalReverseTask extends DefaultTask {
        @InputDirectory
        def File inputDir

        @OutputDirectory
        def File outputDir

        @Input
        def inputProperty

        @TaskAction
        void execute(IncrementalTaskInputs inputs) {
            println inputs.incremental ? 'CHANGED inputs considered out of date'
                                       : 'ALL inputs considered out of date'
            if (!inputs.incremental)
                project.delete(outputDir.listFiles())

            inputs.outOfDate { change ->
                println "out of date: ${change.file.name}"
                def targetFile = new File(outputDir, change.file.name)
                targetFile.text = change.file.text.reverse()
            }

            inputs.removed { change ->
                println "removed: ${change.file.name}"
                def targetFile = new File(outputDir, change.file.name)
                targetFile.delete()
            }
        }
    }

Note: The code for this example can be found at `samples/userguide/tasks/incrementalTask` in the ‘-all’ distribution of Gradle.

If for some reason the task is not run incremental, e.g. by running with `--rerun-tasks`, only the outOfDate action is executed, even if there were deleted input files. You should consider handling this case at the beginning, as is done in the example above.

For a simple transformer task like this, the task action simply needs to generate output files for any out-of-date inputs, and delete output files for any removed inputs.

A task may only contain a single incremental task action.

### 40.4.2. Which inputs are considered out of date?

When Gradle has history of a previous task execution, and the only changes to the task execution context since that execution are to input files, then Gradle is able to determine which input files need to be reprocessed by the task. In this case, the `IncrementalTaskInputs.outOfDate(org.gradle.api.Action)` action will be executed for any input file that was added or modified, and the `IncrementalTaskInputs.removed(org.gradle.api.Action)` action will be executed for any removed input file.

However, there are many cases where Gradle is unable to determine which input files need to be reprocessed. Examples include:

- There is no history available from a previous execution.

- You are building with a different version of Gradle. Currently, Gradle does not use task history from a different version.

- An `upToDateWhen` criteria added to the task returns `false`.

- An input property has changed since the previous execution.

- One or more output files have changed since the previous execution.

In any of these cases, Gradle will consider all of the input files to be `outOfDate`. The `IncrementalTaskInputs.outOfDate(org.gradle.api.Action)` action will be executed for every input file, and the `IncrementalTaskInputs.removed(org.gradle.api.Action)` action will not be executed at all.

You can check if Gradle was able to determine the incremental changes to input files with `IncrementalTaskInputs.isIncremental()`.

### 40.4.3. An incremental task in action

Given the incremental task implementation above, we can explore the various change scenarios by example. Note that the various mutation tasks ('updateInputs', 'removeInput', etc) are only present for demonstration purposes: these would not normally be part of your build script.

First, consider the `IncrementalReverseTask` executed against a set of inputs for the first time. In this case, all inputs will be considered “out of date”:

Example 40.9. Running the incremental task for the first time

`build.gradle`

    task incrementalReverse(type: IncrementalReverseTask) {
        inputDir = file('inputs')
        outputDir = file("$buildDir/outputs")
        inputProperty = project.properties['taskInputProperty'] ?: 'original'
    }

Build layout

    incrementalTask/
      build.gradle
      inputs/
        1.txt
        2.txt
        3.txt

Output of `gradle -q incrementalReverse`

    > gradle -q incrementalReverse
    ALL inputs considered out of date
    out of date: 1.txt
    out of date: 2.txt
    out of date: 3.txt

Naturally when the task is executed again with no changes, then the entire task is up to date and no files are reported to the task action:

Example 40.10. Running the incremental task with unchanged inputs

Output of `gradle -q incrementalReverse`

    > gradle -q incrementalReverse

When an input file is modified in some way or a new input file is added, then re-executing the task results in those files being reported to `IncrementalTaskInputs.outOfDate(org.gradle.api.Action)`:

Example 40.11. Running the incremental task with updated input files

`build.gradle`

    task updateInputs() {
        doLast {
            file('inputs/1.txt').text = 'Changed content for existing file 1.'
            file('inputs/4.txt').text = 'Content for new file 4.'
        }
    }

Output of `gradle -q updateInputs incrementalReverse`

    > gradle -q updateInputs incrementalReverse
    CHANGED inputs considered out of date
    out of date: 1.txt
    out of date: 4.txt

When an existing input file is removed, then re-executing the task results in that file being reported to `IncrementalTaskInputs.removed(org.gradle.api.Action)`:

Example 40.12. Running the incremental task with an input file removed

`build.gradle`

    task removeInput() {
        doLast {
            file('inputs/3.txt').delete()
        }
    }

Output of `gradle -q removeInput incrementalReverse`

    > gradle -q removeInput incrementalReverse
    CHANGED inputs considered out of date
    removed: 3.txt

When an output file is deleted (or modified), then Gradle is unable to determine which input files are out of date. In this case, all input files are reported to the `IncrementalTaskInputs.outOfDate(org.gradle.api.Action)` action, and no input files are reported to the `IncrementalTaskInputs.removed(org.gradle.api.Action)` action:

Example 40.13. Running the incremental task with an output file removed

`build.gradle`

    task removeOutput() {
        doLast {
            file("$buildDir/outputs/1.txt").delete()
        }
    }

Output of `gradle -q removeOutput incrementalReverse`

    > gradle -q removeOutput incrementalReverse
    ALL inputs considered out of date
    out of date: 1.txt
    out of date: 2.txt
    out of date: 3.txt

When a task input property is modified, Gradle is unable to determine how this property impacted the task outputs, so all input files are assumed to be out of date. So similar to the changed output file example, all input files are reported to the `IncrementalTaskInputs.outOfDate(org.gradle.api.Action)` action, and no input files are reported to the `IncrementalTaskInputs.removed(org.gradle.api.Action)` action:

Example 40.14. Running the incremental task with an input property changed

Output of `gradle -q -PtaskInputProperty=changed incrementalReverse`

    > gradle -q -PtaskInputProperty=changed incrementalReverse
    ALL inputs considered out of date
    out of date: 1.txt
    out of date: 2.txt
    out of date: 3.txt

### 40.4.4. Storing incremental state for cached tasks

Using Gradle’s `IncrementalTaskInputs` is not the only way to create tasks that only works on changes since the last execution. Tools like the Kotlin compiler provide incrementality as a built-in feature. The way this is typically implemented is that the tool stores some analysis data about the state of the previous execution in some file. If such state files are relocatable, then they can be declared as outputs of the task. This way when the task’s results are loaded from cache, the next execution can already use the analysis data loaded from cache, too.

However, if the state files are non-relocatable, then they can’t be shared via the build cache. Indeed, when the task is loaded from cache, any such state files must be cleaned up to prevent stale state to confuse the tool during the next execution. Gradle can ensure such stale files are removed if they are declared via `task.localState.register()` or a property is marked with the `@LocalState` annotation.

## 40.5. The Worker API

The Worker API is an incubating feature.

As can be seen from the discussion of incremental tasks, the work that a task performs can be viewed as discrete units (i.e. a subset of inputs that are transformed to a certain subset of outputs). Many times, these units of work are highly independent of each other, meaning they can be performed in any order and simply aggregated together to form the overall action of the task. In a single threaded execution, these units of work would execute in sequence, however if we have multiple processors, it would be desirable to perform independent units of work concurrently. By doing so, we can fully utilize the available resources at build time and complete the activity of the task faster.

The Worker API provides a mechanism for doing exactly this. It allows for safe, concurrent execution of multiple items of work during a task action. But the benefits of the Worker API are not confined to parallelizing the work of a task. You can also configure a desired level of isolation such that work can be executed in an isolated classloader or even in an isolated process. Furthermore, the benefits extend beyond even the execution of a single task. Using the Worker API, Gradle can begin to execute tasks in parallel by default. In other words, once a task has submitted its work to be executed asynchronously, and has exited the task action, Gradle can then begin the execution of other independent tasks in parallel, even if those tasks are in the same project.

### 40.5.1. Using the Worker API

In order to submit work to the Worker API, two things must be provided: an implementation of the unit of work, and a configuration for the unit of work. The implementation is simply a class that extends `java.lang.Runnable`. This class should have a constructor that is annotated with `javax.inject.Inject` and accepts parameters that configure the class for a single unit of work. When a unit of work is submitted to the `WorkerExecutor`, an instance of this class will be created and the parameters configured for the unit of work will be passed to the constructor.

Example 40.15. Creating a unit of work implementation

`build.gradle`

    import org.gradle.workers.WorkerExecutor

    import javax.inject.Inject

    // The implementation of a single unit of work
    class ReverseFile implements Runnable {
        File fileToReverse
        File destinationFile

        @Inject
        public ReverseFile(File fileToReverse, File destinationFile) {
            this.fileToReverse = fileToReverse
            this.destinationFile = destinationFile
        }

        @Override
        public void run() {
            destinationFile.text = fileToReverse.text.reverse()
        }
    }

The configuration of the worker is represented by a `WorkerConfiguration` and is set by configuring an instance of this object at the time of submission. However, in order to submit the unit of work, it is necessary to first acquire the `WorkerExecutor`. To do this, a constructor should be provided that is annotated with `javax.inject.Inject` and accepts a `WorkerExecutor` parameter. Gradle will inject the instance of `WorkerExecutor` at runtime when the task is created.

Example 40.16. Submitting a unit of work for execution

`build.gradle`

    class ReverseFiles extends SourceTask {
        final WorkerExecutor workerExecutor

        @OutputDirectory
        File outputDir

        // The WorkerExecutor will be injected by Gradle at runtime
        @Inject
        public ReverseFiles(WorkerExecutor workerExecutor) {
            this.workerExecutor = workerExecutor
        }

        @TaskAction
        void reverseFiles() {
            // Create and submit a unit of work for each file
            source.files.each { file ->
                workerExecutor.submit(ReverseFile.class) { WorkerConfiguration config ->
                    // Use the minimum level of isolation
                    config.isolationMode = IsolationMode.NONE

                    // Constructor parameters for the unit of work implementation
                    config.params file, project.file("${outputDir}/${file.name}")
                }
            }
        }
    }

Note that one element of the `WorkerConfiguration` is the `params` property. These are the parameters passed to the constructor of the unit of work implementation for each item of work submitted. Any parameters provided to the unit of work must be `java.io.Serializable`.

Once all of the work for a task action has been submitted, it is safe to exit the task action. The work will be executed asynchronously and in parallel (up to the setting of `max-workers`). Of course, any tasks that are dependent on this task (and any subsequent task actions of this task) will not begin executing until all of the asynchronous work completes. However, other independent tasks that have no relationship to this task can begin executing immediately.

If any failures occur while executing the asynchronous work, the task will fail and a `WorkerExecutionException` will be thrown detailing the failure for each failed work item. This will be treated like any failure during task execution and will prevent any dependent tasks from executing.

In some cases, however, it might be desirable to wait for work to complete before exiting the task action. This is possible using the `WorkerExecutor.await()` method. As in the case of allowing the work to complete asynchronously, any failures that occur while executing an item of work will be surfaced as a `WorkerExecutionException` thrown from the `WorkerExecutor.await()` method.

Note that Gradle will only begin running other independent tasks in parallel when a task has exited a task action and returned control of execution to Gradle. When `WorkerExecutor.await()` is used, execution does not leave the task action. This means that Gradle will not allow other tasks to begin executing and will wait for the task action to complete before doing so.

Example 40.17. Waiting for asynchronous work to complete

`build.gradle`

    // Create and submit a unit of work for each file
    source.files.each { file ->
        workerExecutor.submit(ReverseFile.class) { config ->
            config.isolationMode = IsolationMode.NONE
            // Constructor parameters for the unit of work implementation
            config.params file, project.file("${outputDir}/${file.name}")
        }
    }

    // Wait for all asynchronous work to complete before continuing
    workerExecutor.await()
    logger.lifecycle("Created ${outputDir.listFiles().size()} reversed files in ${project.relativePath(outputDir)}")

### 40.5.2. Isolation Modes

Gradle provides three isolation modes that can be configured on a unit of work and are specified using the `IsolationMode` enum:

IsolationMode.NONE

  This states that the work should be run in a thread with a minimum of isolation. For instance, it will share the same classloader that the task is loaded from. This is the fastest level of isolation.

IsolationMode.CLASSLOADER

  This states that the work should be run in a thread with an isolated classloader. The classloader will have the classpath from the classloader that the unit of work implementation class was loaded from as well as any additional classpath entries added through `WorkerConfiguration.classpath(java.lang.Iterable)`.

IsolationMode.PROCESS

  This states that the work should be run with a maximum level of isolation by executing the work in a separate process. The classloader of the process will use the classpath from the classloader that the unit of work was loaded from as well as any additional classpath entries added through `WorkerConfiguration.classpath(java.lang.Iterable)`. Furthermore, the process will be a Worker Daemon which will stay alive and can be reused for future work items that may have the same requirements. This process can be configured with different settings than the Gradle JVM using `WorkerConfiguration.forkOptions(org.gradle.api.Action)`.

### 40.5.3. Worker Daemons

When using `IsolationMode.PROCESS`, gradle will start a long-lived Worker Daemon process that can be reused for future work items.

Example 40.18. Submitting an item of work to run in a worker daemon

`build.gradle`

    workerExecutor.submit(ReverseFile.class) { WorkerConfiguration config ->
        // Run this work in an isolated process
        config.isolationMode = IsolationMode.PROCESS

        // Configure the options for the forked process
        config.forkOptions { JavaForkOptions options ->
            options.maxHeapSize = "512m"
            options.systemProperty "org.gradle.sample.showFileSize", "true"
        }

        // Constructor parameters for the unit of work implementation
        config.params file, project.file("${outputDir}/${file.name}")
    }

When a unit of work for a Worker Daemon is submitted, Gradle will first look to see if a compatible, idle daemon already exists. If so, it will send the unit of work to the idle daemon, marking it as busy. If not, it will start a new daemon. When evaluating compatibility, Gradle looks at a number of criteria, all of which can be controlled through `WorkerConfiguration.forkOptions(org.gradle.api.Action)`.

executable

  A daemon is considered compatible only if it uses the same java executable.

classpath

  A daemon is considered compatible if its classpath contains all of the classpath entries requested. Note that a daemon is considered compatible if it has more classpath entries in addition to those requested.

heap settings

  A daemon is considered compatible if it has at least the same heap size settings as requested. In other words, a daemon that has higher heap settings than requested would be considered compatible.

jvm arguments

  A daemon is considered compatible if it has set all of the jvm arguments requested. Note that a daemon is considered compatible if it has additional jvm arguments beyond those requested (except for arguments treated specially such as heap settings, assertions, debug, etc).

system properties

  A daemon is considered compatible if it has set all of the system properties requested with the same values. Note that a daemon is considered compatible if it has additional system properties beyond those requested.

environment variables

  A daemon is considered compatible if it has set all of the environment variables requested with the same values. Note that a daemon is considered compatible if it has more environment variables in addition to those requested.

bootstrap classpath

  A daemon is considered compatible if it contains all of the bootstrap classpath entries requested. Note that a daemon is considered compatible if it has more bootstrap classpath entries in addition to those requested.

debug

  A daemon is considered compatible only if debug is set to the same value as requested (true or false).

enable assertions

  A daemon is considered compatible only if enable assertions is set to the same value as requested (true or false).

default character encoding

  A daemon is considered compatible only if the default character encoding is set to the same value as requested.

Worker daemons will remain running until either the build daemon that started them is stopped, or system memory becomes scarce. When available system memory is low, Gradle will begin stopping worker daemons in an attempt to minimize memory consumption.

## 40.6. Re-using logic between task classes

There are different ways to re-use logic between task classes. The easiest case is when you can extract the logic you want to share in a separate method or class and then use the extracted piece of code in your tasks. For example, the `Copy` task re-uses the logic of the `Project.copy(org.gradle.api.Action)` method. Another option is to add a task dependency on the task which outputs you want to re-use. Other options include using task rules or the worker API.