# Working with IO

version 2.4.14

Groovy provides a number of helper methods for working with I/O. While you could use standard Java code in Groovy to deal with those, Groovy provides much more convenient ways to handle files, streams, readers, …​

In particular, you should take a look at methods added to:

- the `java.io.File` class : http://docs.groovy-lang.org/latest/html/groovy-jdk/java/io/File.html

- the `java.io.InputStream` class: http://docs.groovy-lang.org/latest/html/groovy-jdk/java/io/InputStream.html

- the `java.io.OutputStream` class: http://docs.groovy-lang.org/latest/html/groovy-jdk/java/io/OutputStream.html

- the `java.io.Reader` class: http://docs.groovy-lang.org/latest/html/groovy-jdk/java/io/Reader.html

- the `java.io.Writer` class: http://docs.groovy-lang.org/latest/html/groovy-jdk/java/io/Writer.html

- the `java.nio.file.Path` class: http://docs.groovy-lang.org/latest/html/groovy-jdk/java/nio/file/Path.html

The following section focuses on sample idiomatic constructs using helper methods available above but is not meant to be a complete description of all available methods. For that, please read the GDK API.

## 1. Reading files

As a first example, let’s see how you would print all lines of a text file in Groovy:

    new File(baseDir, 'haiku.txt').eachLine { line ->
        println line
    }

The `eachLine` method is a method added to the `File` class automatically by Groovy and has many variants, for example if you need to know the line number, you can use this variant:

    new File(baseDir, 'haiku.txt').eachLine { line, nb ->
        println "Line $nb: $line"
    }

If for whatever reason the an exception is thrown in the `eachLine` body, the method makes sure that the resource is properly closed. This is true for all I/O resource methods that Groovy adds.

For example in some cases you will prefer to use a `Reader`, but still benefit from the automatic resource management from Groovy. In the next example, the reader will be closed even if the exception occurs:

    def count = 0, MAXSIZE = 3
    new File(baseDir,"haiku.txt").withReader { reader ->
        while (reader.readLine()) {
            if (++count > MAXSIZE) {
                throw new RuntimeException('Haiku should only have 3 verses')
            }
        }
    }

Should you need to collect the lines of a text file into a list, you can do:

    def list = new File(baseDir, 'haiku.txt').collect {it}

Or you can even leverage the `as` operator to get the contents of the file into an array of lines:

    def array = new File(baseDir, 'haiku.txt') as String[]

How many times did you have to get the contents of a file into a `byte[]` and how much code does it require? Groovy makes it very easy actually:

    byte[] contents = file.bytes

Working with I/O is not limited to dealing with files. In fact, a lot of operations rely on input/output streams, hence why Groovy adds a lot of support methods to those, as you can see in the documentation.

As an example, you can obtain an `InputStream` from a `File` very easily:

    def is = new File(baseDir,'haiku.txt').newInputStream()
    // do something ...
    is.close()

However you can see that it requires you to deal with closing the inputstream. In Groovy it is in general a better idea to use the `withInputStream` idiom that will take care of that for you:

    new File(baseDir,'haiku.txt').withInputStream { stream ->
        // do something ...
    }

## 2. Writing files

Of course in some cases you won’t want to read but write a file. One of the options is to use a `Writer`:

    new File(baseDir,'haiku.txt').withWriter('utf-8') { writer ->
        writer.writeLine 'Into the ancient pond'
        writer.writeLine 'A frog jumps'
        writer.writeLine 'Water’s sound!'
    }

But for such a simple example, using the `<<` operator would have been enough:

    new File(baseDir,'haiku.txt') << '''Into the ancient pond
    A frog jumps
    Water’s sound!'''

Of course we do not always deal with text contents, so you could use the `Writer` or directly write bytes as in this example:

    file.bytes = [66,22,11]

Of course you can also directly deal with output streams. For example, here is how you would create an output stream to write into a file:

    def os = new File(baseDir,'data.bin').newOutputStream()
    // do something ...
    os.close()

However you can see that it requires you to deal with closing the output stream. Again it is in general a better idea to use the `withOutputStream` idiom that will handle the exceptions and close the stream in any case:

    new File(baseDir,'data.bin').withOutputStream { stream ->
        // do something ...
    }

## 3. Traversing file trees

In scripting contexts it is a common task to traverse a file tree in order to find some specific files and do something with them. Groovy provides multiple methods to do this. For example you can perform something on all files of a directory:

    dir.eachFile { file ->                      (1)
        println file.name
    }
    dir.eachFileMatch(~/.*\.txt/) { file ->     (2)
        println file.name
    }

1 | executes the closure code on each file found in the directory
2 | executes the closure code on files in the directory matching the specified pattern

Often you will have to deal with a deeper hierarchy of files, in which case you can use `eachFileRecurse`:

    dir.eachFileRecurse { file ->                      (1)
        println file.name
    }

    dir.eachFileRecurse(FileType.FILES) { file ->      (2)
        println file.name
    }

1 | executes the closure code on each file or directory found in the directory, recursively
2 | executes the closure code only on files, but recursively

For more complex traversal techniques you can use the `traverse` method, which requires you to set a special flag indicating what to do with the traversal:

    dir.traverse { file ->
        if (file.directory && file.name=='bin') {
            FileVisitResult.TERMINATE                   (1)
        } else {
            println file.name
            FileVisitResult.CONTINUE                    (2)
        }

    }

1 | if the current file is a directory and its name is `bin`, stop the traversal
2 | otherwise print the file name and continue

## 4. Data and objects

In Java it is not uncommon to serialize and deserialize data using the `java.io.DataOutputStream` and `java.io.DataInputStream` classes respectively. Groovy will make it even easier to deal with them. For example, you could serialize data into a file and deserialize it using this code:

    boolean b = true
    String message = 'Hello from Groovy'
    // Serialize data into a file
    file.withDataOutputStream { out ->
        out.writeBoolean(b)
        out.writeUTF(message)
    }
    // ...
    // Then read it back
    file.withDataInputStream { input ->
        assert input.readBoolean() == b
        assert input.readUTF() == message
    }

And similarily, if the data you want to serialize implements the `Serializable` interface, you can proceed with an object output stream, as illustrated here:

    Person p = new Person(name:'Bob', age:76)
    // Serialize data into a file
    file.withObjectOutputStream { out ->
        out.writeObject(p)
    }
    // ...
    // Then read it back
    file.withObjectInputStream { input ->
        def p2 = input.readObject()
        assert p2.name == p.name
        assert p2.age == p.age
    }

## 5. Executing External Processes

The previous section described how easy it was to deal with files, readers or streams in Groovy. However in domains like system administration or devops it is often required to communicate with external processes.

Groovy provides a simple way to execute command line processes. Simply write the command line as a string and call the `execute()` method. E.g., on a *nix machine (or a windows machine with appropriate *nix commands installed), you can execute this:

    def process = "ls -l".execute()             (1)
    println "Found text ${process.text}"        (2)

1 | executes the `ls` command in an external process
2 | consume the output of the command and retrieve the text

The `execute()` method returns a `java.lang.Process` instance which will subsequently allow the in/out/err streams to be processed and the exit value from the process to be inspected etc.

e.g. here is the same command as above but we will now process the resulting stream a line at a time:

    def process = "ls -l".execute()             (1)
    process.in.eachLine { line ->               (2)
        println line                            (3)
    }

1 | executes the `ls` command in an external process
2 | for each line of the input stream of the process
3 | print the line

It is worth noting that `in` corresponds to an input stream to the standard output of the command. `out` will refer to a stream where you can send data to the process (its standard input).

Remember that many commands are shell built-ins and need special handling. So if you want a listing of files in a directory on a Windows machine and write:

    def process = "dir".execute()
    println "${process.text}"

you will receive an `IOException` saying Cannot run program "dir": CreateProcess error=2, The system cannot find the file specified.

This is because `dir` is built-in to the Windows shell (`cmd.exe`) and can’t be run as a simple executable. Instead, you will need to write:

    def process = "cmd /c dir".execute()
    println "${process.text}"

Also, because this functionality currently makes use of `java.lang.Process` undercover, the deficiencies of that class must be taken into consideration. In particular, the javadoc for this class says:

> Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock

Because of this, Groovy provides some additional helper methods which make stream handling for processes easier.

Here is how to gobble all of the output (including the error stream output) from your process:

    def p = "rm -f foo.tmp".execute([], tmpDir)
    p.consumeProcessOutput()
    p.waitFor()

There are also variations of `consumeProcessOutput` that make use of `StringBuffer`, `InputStream`, `OutputStream` etc…​ For a complete list, please read the GDK API for java.lang.Process

In addition, these is a `pipeTo` command (mapped to `|` to allow overloading) which lets the output stream of one process be fed into the input stream of another process.

Here are some examples of use:

Pipes in action

    proc1 = 'ls'.execute()
    proc2 = 'tr -d o'.execute()
    proc3 = 'tr -d e'.execute()
    proc4 = 'tr -d i'.execute()
    proc1 | proc2 | proc3 | proc4
    proc4.waitFor()
    if (proc4.exitValue()) {
        println proc4.err.text
    } else {
        println proc4.text
    }

Consuming errors

    def sout = new StringBuilder()
    def serr = new StringBuilder()
    proc2 = 'tr -d o'.execute()
    proc3 = 'tr -d e'.execute()
    proc4 = 'tr -d i'.execute()
    proc4.consumeProcessOutput(sout, serr)
    proc2 | proc3 | proc4
    [proc2, proc3].each { it.consumeProcessErrorStream(serr) }
    proc2.withWriter { writer ->
        writer << 'testfile.groovy'
    }
    proc4.waitForOrKill(1000)
    println "Standard output: $sout"
    println "Standard error: $serr"