Working With Files - Gradle User Manual

# Working With Files

Most builds work with files. Gradle adds some concepts and APIs to help you achieve this.

## Locating files

You can locate a file relative to the project directory using the `Project.file(java.lang.Object)` method.

Example 122. Locating files

`build.gradle`

    // Using a relative path
    File configFile = file('src/config.xml')

    // Using an absolute path
    configFile = file(configFile.absolutePath)

    // Using a File object with a relative path
    configFile = file(new File('src/config.xml'))

    // Using a java.nio.file.Path object with a relative path
    configFile = file(Paths.get('src', 'config.xml'))

    // Using an absolute java.nio.file.Path object
    configFile = file(Paths.get(System.getProperty('user.home')).resolve('global-config.xml'))

You can pass any object to the `file()` method, and it will attempt to convert the value to an absolute `File` object. Usually, you would pass it a `String`, `File` or `Path` instance. If this path is an absolute path, it is used to construct a `File` instance. Otherwise, a `File` instance is constructed by prepending the project directory path to the supplied path. The `file()` method also understands URLs, such as `file:/some/path.xml`.

Using this method is a useful way to convert some user provided value into an absolute `File`. It is preferable to using `new File(somePath)`, as `file()` always evaluates the supplied path relative to the project directory, which is fixed, rather than the current working directory, which can change depending on how the user runs Gradle.

## File collections

A file collection is simply a set of files. It is represented by the `FileCollection` interface. Many objects in the Gradle API implement this interface. For example, dependency configurations implement `FileCollection`.

One way to obtain a `FileCollection` instance is to use the `Project.files(java.lang.Object[])` method. You can pass this method any number of objects, which are then converted into a set of `File` objects. The `files()` method accepts any type of object as its parameters. These are evaluated relative to the project directory, as per the `file()` method, described in the section called “Locating files”. You can also pass collections, iterables, maps and arrays to the `files()` method. These are flattened and the contents converted to `File` instances.

Example 123. Creating a file collection

`build.gradle`

    FileCollection collection = files('src/file1.txt',
                                      new File('src/file2.txt'),
                                      ['src/file3.txt', 'src/file4.txt'],
                                      Paths.get('src', 'file5.txt'))

A file collection is iterable, and can be converted to a number of other types using the `as` operator. You can also add 2 file collections together using the `+` operator, or subtract one file collection from another using the `-` operator. Here are some examples of what you can do with a file collection.

Example 124. Using a file collection

`build.gradle`

    // Iterate over the files in the collection
    collection.each { File file ->
        println file.name
    }

    // Convert the collection to various types
    Set set = collection.files
    Set set2 = collection as Set
    List list = collection as List
    String path = collection.asPath
    File file = collection.singleFile
    File file2 = collection as File

    // Add and subtract collections
    def union = collection + files('src/file3.txt')
    def different = collection - files('src/file3.txt')

You can also pass the `files()` method a closure or a `Callable` instance. This is called when the contents of the collection are queried, and its return value is converted to a set of `File` instances. The return value can be an object of any of the types supported by the `files()` method. This is a simple way to 'implement' the `FileCollection` interface.

Example 125. Implementing a file collection

`build.gradle`

    task list {
        doLast {
            File srcDir

            // Create a file collection using a closure
            collection = files { srcDir.listFiles() }

            srcDir = file('src')
            println "Contents of $srcDir.name"
            collection.collect { relativePath(it) }.sort().each { println it }

            srcDir = file('src2')
            println "Contents of $srcDir.name"
            collection.collect { relativePath(it) }.sort().each { println it }
        }
    }

Output of `gradle -q list`

    > gradle -q list
    Contents of src
    src/dir1
    src/file1.txt
    Contents of src2
    src2/dir1
    src2/dir2

Some other types of things you can pass to `files()`:

`FileCollection`

  These are flattened and the contents included in the file collection.

`Task`

  The output files of the task are included in the file collection.

`TaskOutputs`

  The output files of the TaskOutputs are included in the file collection.

It is important to note that the content of a file collection is evaluated lazily, when it is needed. This means you can, for example, create a `FileCollection` that represents files which will be created in the future by, say, some task.

## File trees

A file tree is a collection of files arranged in a hierarchy. For example, a file tree might represent a directory tree or the contents of a ZIP file. It is represented by the `FileTree` interface. The `FileTree` interface extends `FileCollection`, so you can treat a file tree exactly the same way as you would a file collection. Several objects in Gradle implement the `FileTree` interface, such as source sets.

One way to obtain a `FileTree` instance is to use the `Project.fileTree(java.util.Map)` method. This creates a `FileTree` defined with a base directory, and optionally some Ant-style include and exclude patterns.

Example 126. Creating a file tree

`build.gradle`

    // Create a file tree with a base directory
    FileTree tree = fileTree(dir: 'src/main')

    // Add include and exclude patterns to the tree
    tree.include '**/*.java'
    tree.exclude '**/Abstract*'

    // Create a tree using path
    tree = fileTree('src').include('**/*.java')

    // Create a tree using closure
    tree = fileTree('src') {
        include '**/*.java'
    }

    // Create a tree using a map
    tree = fileTree(dir: 'src', include: '**/*.java')
    tree = fileTree(dir: 'src', includes: ['**/*.java', '**/*.xml'])
    tree = fileTree(dir: 'src', include: '**/*.java', exclude: '**/*test*/**')

You use a file tree in the same way you use a file collection. You can also visit the contents of the tree, and select a sub-tree using Ant-style patterns:

Example 127. Using a file tree

`build.gradle`

    // Iterate over the contents of a tree
    tree.each {File file ->
        println file
    }

    // Filter a tree
    FileTree filtered = tree.matching {
        include 'org/gradle/api/**'
    }

    // Add trees together
    FileTree sum = tree + fileTree(dir: 'src/test')

    // Visit the elements of the tree
    tree.visit {element ->
        println "$element.relativePath => $element.file"
    }

By default, the `FileTree` instance `fileTree()` returns will apply some Ant-style default exclude patterns for convenience. For the complete default exclusion list, see Default Excludes.

## Using the contents of an archive as a file tree

You can use the contents of an archive, such as a ZIP or TAR file, as a file tree. You do this using the `Project.zipTree(java.lang.Object)` and `Project.tarTree(java.lang.Object)` methods. These methods return a `FileTree` instance which you can use like any other file tree or file collection. For example, you can use it to expand the archive by copying the contents, or to merge some archives into another.

Example 128. Using an archive as a file tree

`build.gradle`

    // Create a ZIP file tree using path
    FileTree zip = zipTree('someFile.zip')

    // Create a TAR file tree using path
    FileTree tar = tarTree('someFile.tar')

    //tar tree attempts to guess the compression based on the file extension
    //however if you must specify the compression explicitly you can:
    FileTree someTar = tarTree(resources.gzip('someTar.ext'))

## Specifying a set of input files

Many objects in Gradle have properties which accept a set of input files. For example, the `JavaCompile` task has a `source` property, which defines the source files to compile. You can set the value of this property using any of the types supported by the files() method, which was shown above. This means you can set the property using, for example, a `File`, `String`, collection, `FileCollection` or even a closure. Here are some examples:

Example 129. Specifying a set of files

`build.gradle`

    task compile(type: JavaCompile)

    // Use a File object to specify the source directory
    compile {
        source = file('src/main/java')
    }

    // Use a String path to specify the source directory
    compile {
        source = 'src/main/java'
    }

    // Use a collection to specify multiple source directories
    compile {
        source = ['src/main/java', '../shared/java']
    }

    // Use a FileCollection (or FileTree in this case) to specify the source files
    compile {
        source = fileTree(dir: 'src/main/java').matching { include 'org/gradle/api/**' }
    }

    // Using a closure to specify the source files.
    compile {
        source = {
            // Use the contents of each zip file in the src dir
            file('src').listFiles().findAll {it.name.endsWith('.zip')}.collect { zipTree(it) }
        }
    }

Usually, there is a method with the same name as the property, which appends to the set of files. Again, this method accepts any of the types supported by the files() method.

Example 130. Appending a set of files

`build.gradle`

    compile {
        // Add some source directories use String paths
        source 'src/main/java', 'src/main/groovy'

        // Add a source directory using a File object
        source file('../shared/java')

        // Add some source directories using a closure
        source { file('src/test/').listFiles() }
    }

## Copying files

You can use the `Copy` task to copy files. The copy task is very flexible, and allows you to, for example, filter the contents of the files as they are copied, and map to the file names.

To use the `Copy` task, you must provide a set of source files to copy, and a destination directory to copy the files to. You may also specify how to transform the files as they are copied. You do all this using a copy spec. A copy spec is represented by the `CopySpec` interface. The `Copy` task implements this interface. You specify the source files using the `CopySpec.from(java.lang.Object[])` method. To specify the destination directory, use the `CopySpec.into(java.lang.Object)` method.

Example 131. Copying files using the copy task

`build.gradle`

    task copyTask(type: Copy) {
        from 'src/main/webapp'
        into 'build/explodedWar'
    }

The `from()` method accepts any of the arguments that the files() method does. When an argument resolves to a directory, everything under that directory (but not the directory itself) is recursively copied into the destination directory. When an argument resolves to a file, that file is copied into the destination directory. When an argument resolves to a non-existing file, that argument is ignored. If the argument is a task, the output files (i.e. the files the task creates) of the task are copied and the task is automatically added as a dependency of the `Copy` task. The `into()` accepts any of the arguments that the file() method does. Here is another example:

Example 132. Specifying copy task source files and destination directory

`build.gradle`

    task anotherCopyTask(type: Copy) {
        // Copy everything under src/main/webapp
        from 'src/main/webapp'
        // Copy a single file
        from 'src/staging/index.html'
        // Copy the output of a task
        from copyTask
        // Copy the output of a task using Task outputs explicitly.
        from copyTaskWithPatterns.outputs
        // Copy the contents of a Zip file
        from zipTree('src/main/assets.zip')
        // Determine the destination directory later
        into { getDestDir() }
    }

You can select the files to copy using Ant-style include or exclude patterns, or using a closure:

Example 133. Selecting the files to copy

`build.gradle`

    task copyTaskWithPatterns(type: Copy) {
        from 'src/main/webapp'
        into 'build/explodedWar'
        include '**/*.html'
        include '**/*.jsp'
        exclude { details -> details.file.name.endsWith('.html') &&
                             details.file.text.contains('staging') }
    }

You can also use the `Project.copy(org.gradle.api.Action)` method to copy files. It works the same way as the task with some major limitations though. First, the `copy()` is not incremental (see the section called “Up-to-date checks (AKA Incremental Build)”).

Example 134. Copying files using the copy() method without up-to-date check

`build.gradle`

    task copyMethod {
        doLast {
            copy {
                from 'src/main/webapp'
                into 'build/explodedWar'
                include '**/*.html'
                include '**/*.jsp'
            }
        }
    }

Secondly, the `copy()` method cannot honor task dependencies when a task is used as a copy source (i.e. as an argument to `from()`) because it’s a method and not a task. As such, if you are using the `copy()` method as part of a task action, you must explicitly declare all inputs and outputs in order to get the correct behavior.

Example 135. Copying files using the copy() method with up-to-date check

`build.gradle`

    task copyMethodWithExplicitDependencies{
        // up-to-date check for inputs, plus add copyTask as dependency
        inputs.files copyTask
        outputs.dir 'some-dir' // up-to-date check for outputs
        doLast{
            copy {
                // Copy the output of copyTask
                from copyTask
                into 'some-dir'
            }
        }
    }

It is preferable to use the `Copy` task wherever possible, as it supports incremental building and task dependency inference without any extra effort on your part. The `copy()` method can be used to copy files as part of a task’s implementation. That is, the copy method is intended to be used by custom tasks (see Writing Custom Task Classes) that need to copy files as part of their function. In such a scenario, the custom task should sufficiently declare the inputs/outputs relevant to the copy action.

### Renaming files

Example 136. Renaming files as they are copied

`build.gradle`

    task rename(type: Copy) {
        from 'src/main/webapp'
        into 'build/explodedWar'
        // Use a closure to map the file name
        rename { String fileName ->
            fileName.replace('-staging-', '')
        }
        // Use a regular expression to map the file name
        rename '(.+)-staging-(.+)', '$1$2'
        rename(/(.+)-staging-(.+)/, '$1$2')
    }

### Filtering files

Example 137. Filtering files as they are copied

`build.gradle`

    import org.apache.tools.ant.filters.FixCrLfFilter
    import org.apache.tools.ant.filters.ReplaceTokens

    task filter(type: Copy) {
        from 'src/main/webapp'
        into 'build/explodedWar'
        // Substitute property tokens in files
        expand(copyright: '2009', version: '2.3.1')
        expand(project.properties)
        // Use some of the filters provided by Ant
        filter(FixCrLfFilter)
        filter(ReplaceTokens, tokens: [copyright: '2009', version: '2.3.1'])
        // Use a closure to filter each line
        filter { String line ->
            "[$line]"
        }
        // Use a closure to remove lines
        filter { String line ->
            line.startsWith('-') ? null : line
        }
        filteringCharset = 'UTF-8'
    }

When you use the `ReplaceTokens` class with the “filter” operation, the result is a template engine that replaces tokens of the form “@tokenName@” (the Apache Ant-style token) with a set of given values. The “expand” operation does the same thing except it treats the source files as Groovy templates in which tokens take the form “${tokenName}”. Be aware that you may need to escape parts of your source files when using this option, for example if it contains literal “$” or “<%” strings.

It’s a good practice to specify the charset when reading and writing the file, using the `filteringCharset` property. If not specified, the JVM default charset is used, which might not match with the actual charset of the files to filter, and might be different from one machine to another.

### Using the `CopySpec` class

Copy specs form a hierarchy. A copy spec inherits its destination path, include patterns, exclude patterns, copy actions, name mappings and filters.

Example 138. Nested copy specs

`build.gradle`

    task nestedSpecs(type: Copy) {
        into 'build/explodedWar'
        exclude '**/*staging*'
        from('src/dist') {
            include '**/*.html'
        }
        into('libs') {
            from configurations.runtime
        }
    }

## Using the `Sync` task

The `Sync` task extends the `Copy` task. When it executes, it copies the source files into the destination directory, and then removes any files from the destination directory which it did not copy. This can be useful for doing things such as installing your application, creating an exploded copy of your archives, or maintaining a copy of the project’s dependencies.

Here is an example which maintains a copy of the project’s runtime dependencies in the `build/libs` directory.

Example 139. Using the Sync task to copy dependencies

`build.gradle`

    task libs(type: Sync) {
        from configurations.runtime
        into "$buildDir/libs"
    }

## Creating archives

A project can have as many JAR archives as you want. You can also add WAR, ZIP and TAR archives to your project. Archives are created using the various archive tasks: `Zip`, `Tar`, `Jar`, `War`, and `Ear`. They all work the same way, so let’s look at how you create a ZIP file.

Example 140. Creating a ZIP archive

`build.gradle`

    apply plugin: 'java'

    task zip(type: Zip) {
        from 'src/dist'
        into('libs') {
            from configurations.runtime
        }
    }

### Why are you using the Java plugin?

The Java plugin adds a number of default values for the archive tasks. You can use the archive tasks without using the Java plugin, if you like. You will need to provide values for some additional properties.

The archive tasks all work exactly the same way as the `Copy` task, and implement the same `CopySpec` interface. As with the `Copy` task, you specify the input files using the `from()` method, and can optionally specify where they end up in the archive using the `into()` method. You can filter the contents of file, rename files, and all the other things you can do with a copy spec.

### Archive naming

The format of `projectName-version.type` is used for generated archive file names. For example:

Example 141. Creation of ZIP archive

`build.gradle`

    apply plugin: 'java'

    version = 1.0

    task myZip(type: Zip) {
        from 'somedir'
    }

    println myZip.archiveName
    println relativePath(myZip.destinationDir)
    println relativePath(myZip.archivePath)

Output of `gradle -q myZip`

    > gradle -q myZip
    zipProject-1.0.zip
    build/distributions
    build/distributions/zipProject-1.0.zip

This adds a `Zip` archive task with the name `myZip` which produces ZIP file `zipProject-1.0.zip`. It is important to distinguish between the name of the archive task and the name of the archive generated by the archive task. The default name for archives can be changed with the `archivesBaseName` project property. The name of the archive can also be changed at any time later on.

There are a number of properties which you can set on an archive task. These are listed below in Table 8. You can, for example, change the name of the archive:

Example 142. Configuration of archive task - custom archive name

`build.gradle`

    apply plugin: 'java'
    version = 1.0

    task myZip(type: Zip) {
        from 'somedir'
        baseName = 'customName'
    }

    println myZip.archiveName

Output of `gradle -q myZip`

    > gradle -q myZip
    customName-1.0.zip

You can further customize the archive names:

Example 143. Configuration of archive task - appendix & classifier

`build.gradle`

    apply plugin: 'java'
    archivesBaseName = 'gradle'
    version = 1.0

    task myZip(type: Zip) {
        appendix = 'wrapper'
        classifier = 'src'
        from 'somedir'
    }

    println myZip.archiveName

Output of `gradle -q myZip`

    > gradle -q myZip
    gradle-wrapper-1.0-src.zip

Table 8. Archive tasks - naming properties

Property name | Type | Default value | Description
`archiveName` | `String` | `baseName-appendix-version-classifier.extension` If any of these properties is empty the trailing `-` is not added to the name. | The base file name of the generated archive
`archivePath` | `File` | `destinationDir/archiveName` | The absolute path of the generated archive.
`destinationDir` | `File` | Depends on the archive type. JARs and WARs go into `project.buildDir/libraries`. ZIPs and TARs go into `project.buildDir/distributions`. | The directory to generate the archive into
`baseName` | `String` | `project.name` | The base name portion of the archive file name.
`appendix` | `String` | `null` | The appendix portion of the archive file name.
`version` | `String` | `project.version` | The version portion of the archive file name.
`classifier` | `String` | `null` | The classifier portion of the archive file name,
`extension` | `String` | Depends on the archive type, and for TAR files, the compression type as well: `zip`, `jar`, `war`, `tar`, `tgz` or `tbz2`. | The extension of the archive file name.

### Sharing content between multiple archives

You can use the `Project.copySpec(org.gradle.api.Action)` method to share content between archives.

### Reproducible archives

Sometimes it can be desirable to recreate archives in a byte for byte way on different machines. You want to be sure that building an artifact from source code produces the same result, byte for byte, no matter when and where it is built. This is necessary for projects like reproducible-builds.org.

Reproducing the same archive byte for byte poses some challenges since the order of the files in an archive is influenced by the underlying filesystem. Each time a zip, tar, jar, war or ear is built from source, the order of the files inside the archive may change. Files that only have a different timestamp also causes archives to be slightly different between builds. All `AbstractArchiveTask` (e.g. Jar, Zip) tasks shipped with Gradle include incubating support producing reproducible archives.

For example, to make a `Zip` task reproducible you need to set `Zip.isReproducibleFileOrder()` to `true` and `Zip.isPreserveFileTimestamps()` to `false`. In order to make all archive tasks in your build reproducible, consider adding the following configuration to your build file:

Example 144. Activating reproducible archives

`build.gradle`

    tasks.withType(AbstractArchiveTask) {
        preserveFileTimestamps = false
        reproducibleFileOrder = true
    }

Often you will want to publish an archive, so that it is usable from another project. This process is described in Publishing artifacts

## Properties files

Properties files are used in many places during Java development. Gradle makes it easy to create properties files as a normal part of the build. You can use the `WriteProperties` task to create properties files.

The `WriteProperties` task also fixes a well-known problem with `Properties.store()` that can reduce the usefulness of incremental builds (see the section called “Up-to-date checks (AKA Incremental Build)”). The standard Java way to write a properties file produces a unique file every time, even when the same properties and values are used, because it includes a timestamp in the comments. Gradle’s `WriteProperties` task generates exactly the same output byte-for-byte if none of the properties have changed. This is achieved by a few tweaks to how a properties file is generated:

- no timestamp comment is added to the output

- the line separator is system independent, but can be configured explicitly (it defaults to `'\n'`)

- the properties are sorted alphabetically