Repository Types - Gradle User Manual

# Repository Types

## Flat directory repository

Some projects might prefer to store dependencies on a shared drive or as part of the project source code instead of a binary repository product. If you want to use a (flat) filesystem directory as a repository, simply type:

Example: Flat repository resolver

`build.gradle`

    repositories {
        flatDir {
            dirs 'lib'
        }
        flatDir {
            dirs 'lib1', 'lib2'
        }
    }

This adds repositories which look into one or more directories for finding dependencies. Note that this type of repository does not support any meta-data formats like Ivy XML or Maven POM files. Instead, Gradle will dynamically generate a module descriptor (without any dependency information) based on the presence of artifacts. However, as Gradle prefers to use modules whose descriptor has been created from real meta-data rather than being generated, flat directory repositories cannot be used to override artifacts with real meta-data from other repositories. For example, if Gradle finds only `jmxri-1.2.1.jar` in a flat directory repository, but `jmxri-1.2.1.pom` in another repository that supports meta-data, it will use the second repository to provide the module.

For the use case of overriding remote artifacts with local ones consider using an Ivy or Maven repository instead whose URL points to a local directory. If you only work with flat directory repositories you don’t need to set all attributes of a dependency.

## Maven Central repository

Maven Central is a popular repository hosting open source libraries for consumption by Java projects.

To declare the central Maven repository for your build add this to your script:

Example: Adding central Maven repository

`build.gradle`

    repositories {
        mavenCentral()
    }

## JCenter Maven repository

Bintray's JCenter is an up-to-date collection of all popular Maven OSS artifacts, including artifacts published directly to Bintray.

To declare the JCenter Maven repository add this to your build script:

Example: Adding Bintray's JCenter Maven repository

`build.gradle`

    repositories {
        jcenter()
    }

## Google Maven repository

The Google repository hosts Android-specific artifacts including the Android SDK. For usage examples, see the relevant documentation.

To declare the Google Maven repository add this to your build script:

Example: Adding Google Maven repository

`build.gradle`

    repositories {
        google()
    }

## Local Maven repository

Gradle can consume dependencies available in the local Maven repository. Declaring this repository is beneficial for teams that publish to the local Maven repository with one project and consume the artifacts by Gradle in another project.

Gradle stores resolved dependencies in its own cache. A build does not need to declare the local Maven repository even if you resolve dependencies from a Maven-based, remote repository.

To declare the local Maven cache as a repository add this to your build script:

Example: Adding the local Maven cache as a repository

`build.gradle`

    repositories {
        mavenLocal()
    }

Gradle uses the same logic as Maven to identify the location of your local Maven cache. If a local repository location is defined in a `settings.xml`, this location will be used. The `settings.xml` in `USER_HOME/.m2` takes precedence over the `settings.xml` in `M2_HOME/conf`. If no `settings.xml` is available, Gradle uses the default location `USER_HOME/.m2/repository`.

## Custom Maven repositories

Many organizations host dependencies in an in-house Maven repository only accessible within the company’s network. Gradle can declare Maven repositories by URL.

For adding a custom Maven repository you can do:

Example: Adding custom Maven repository

`build.gradle`

    repositories {
        maven {
            url "http://repo.mycompany.com/maven2"
        }
    }

Sometimes a repository will have the POMs published to one location, and the JARs and other artifacts published at another location. To define such a repository, you can do:

Example: Adding additional Maven repositories for JAR files

`build.gradle`

    repositories {
        maven {
            // Look for POMs and artifacts, such as JARs, here
            url "http://repo2.mycompany.com/maven2"
            // Look for artifacts here if not found at the above location
            artifactUrls "http://repo.mycompany.com/jars"
            artifactUrls "http://repo.mycompany.com/jars2"
        }
    }

Gradle will look at the first URL for the POM and the JAR. If the JAR can’t be found there, the artifact URLs are used to look for JARs.

### Accessing password-protected Maven repositories

You can specify credentials for Maven repositories secured by basic authentication.

Example: Accessing password-protected Maven repository

`build.gradle`

    repositories {
        maven {
            url "http://repo.mycompany.com/maven2"
            credentials {
                username "user"
                password "password"
            }
        }
    }

## Custom Ivy repositories

Organizations might decide to host dependencies in an in-house Ivy repository. Gradle can declare Ivy repositories by URL.

### Defining an Ivy repository with a standard layout

To declare an Ivy repository using the standard layout no additional customization is needed. You just declare the URL.

Example: Ivy repository

`build.gradle`

    repositories {
        ivy {
            url "http://repo.mycompany.com/repo"
        }
    }

### Defining a named layout for an Ivy repository

You can specify that your repository conforms to the Ivy or Maven default layout by using a named layout.

Example: Ivy repository with named layout

`build.gradle`

    repositories {
        ivy {
            url "http://repo.mycompany.com/repo"
            layout "maven"
        }
    }

Valid named layout values are `'gradle'` (the default), `'maven'`, `'ivy'` and `'pattern'`. See `IvyArtifactRepository.layout(java.lang.String, groovy.lang.Closure)` in the API documentation for details of these named layouts.

### Defining custom pattern layout for an Ivy repository

To define an Ivy repository with a non-standard layout, you can define a `'pattern'` layout for the repository:

Example: Ivy repository with pattern layout

`build.gradle`

    repositories {
        ivy {
            url "http://repo.mycompany.com/repo"
            layout "pattern", {
                artifact "[module]/[revision]/[type]/[artifact].[ext]"
            }
        }
    }

To define an Ivy repository which fetches Ivy files and artifacts from different locations, you can define separate patterns to use to locate the Ivy files and artifacts:

Each `artifact` or `ivy` specified for a repository adds an additional pattern to use. The patterns are used in the order that they are defined.

Example: Ivy repository with multiple custom patterns

`build.gradle`

    repositories {
        ivy {
            url "http://repo.mycompany.com/repo"
            layout "pattern", {
                artifact "3rd-party-artifacts/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"
                artifact "company-artifacts/[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"
                ivy "ivy-files/[organisation]/[module]/[revision]/ivy.xml"
            }
        }
    }

Optionally, a repository with pattern layout can have its `'organisation'` part laid out in Maven style, with forward slashes replacing dots as separators. For example, the organisation `my.company` would then be represented as `my/company`.

Example: Ivy repository with Maven compatible layout

`build.gradle`

    repositories {
        ivy {
            url "http://repo.mycompany.com/repo"
            layout "pattern", {
                artifact "[organisation]/[module]/[revision]/[artifact]-[revision].[ext]"
                m2compatible = true
            }
        }
    }

### Accessing password-protected Ivy repositories

You can specify credentials for Ivy repositories secured by basic authentication.

Example: Ivy repository with authentication

`build.gradle`

    repositories {
        ivy {
            url "http://repo.mycompany.com"
            credentials {
                username "user"
                password "password"
            }
        }
    }

## Supported metadata sources

When searching for a module in a repository, Gradle, by default, checks for supported metadata file formats in that repository. In a Maven repository, Gradle looks for a `.pom` file, in an ivy repository it looks for an `ivy.xml` file and in a flat directory repository it looks directly for `.jar` files as it does not expect any metadata. Starting with 5.0, Gradle also looks for `.module` (Gradle module metadata) files.

However, if you define a customized repository you might want to configure this behavior. For example, you can define a Maven repository without `.pom` files but only jars. To do so, you can configure metadata sources for any repository.

Example: Maven repository that supports artifacts without metadata

`build.gradle`

    repositories {
        maven {
            url "http://repo.mycompany.com/repo"
            metadataSources {
                mavenPom()
                artifact()
            }
        }
    }

You can specify multiple sources to tell Gradle to keep looking if a file was not found. In that case, the order of checking for sources is predefined.

The following metadata sources are supported:

Repository transport protocols

Metadata source | Description | Order | Maven | Ivy / flat dir
`gradleMetadata()` | Look for Gradle `.module` files | 1st | yes | yes
`mavenPom()` | Look for Maven `.pom` files | 2nd | yes | yes
`ivyDescriptor()` | Look for `ivy.xml` files | 2nd | no | yes
`artifact()` | Look directly for artifact | 3rd | yes | yes

The defaults for Ivy and Maven repositories change with Gradle 5.0. Before 5.0, `artifact()` was included in the defaults. Leading to some inefficiency when modules are missing completely. To restore this behavior, for example, for Maven central you can use `mavenCentral { mavenPom(); artifact() }`. In a similar way, you can opt into the new behavior in older Gradle verisions using `mavenCentral { mavenPom() }`

## Supported repository transport protocols

Maven and Ivy repositories support the use of various transport protocols. At the moment the following protocols are supported:

Repository transport protocols

Type | Credential types
`file` | none
`http` | username/password
`https` | username/password
`sftp` | username/password
`s3` | access key/secret key/session token or Environment variables
`gcs` | default application credentials sourced from well known files, Environment variables etc.

Username and password should never be checked in plain text into version control as part of your build file. You can store the credentials in a local `gradle.properties` file and use one of the open source Gradle plugins for encrypting and consuming credentials e.g. the credentials plugin.

The transport protocol is part of the URL definition for a repository. The following build script demonstrates how to create a HTTP-based Maven and Ivy repository:

Example: Declaring a Maven and Ivy repository

`build.gradle`

    repositories {
        maven {
            url "http://repo.mycompany.com/maven2"
        }

        ivy {
            url "http://repo.mycompany.com/repo"
        }
    }

The following example shows how to declare SFTP repositories:

Example: Using the SFTP protocol for a repository

`build.gradle`

    repositories {
        maven {
            url "sftp://repo.mycompany.com:22/maven2"
            credentials {
                username "user"
                password "password"
            }
        }

        ivy {
            url "sftp://repo.mycompany.com:22/repo"
            credentials {
                username "user"
                password "password"
            }
        }
    }

When using an AWS S3 backed repository you need to authenticate using `AwsCredentials`, providing access-key and a private-key. The following example shows how to declare a S3 backed repository and providing AWS credentials:

Example: Declaring a S3 backed Maven and Ivy repository

`build.gradle`

    repositories {
        maven {
            url "s3://myCompanyBucket/maven2"
            credentials(AwsCredentials) {
                accessKey "someKey"
                secretKey "someSecret"
                // optional
                sessionToken "someSTSToken"
            }
        }

        ivy {
            url "s3://myCompanyBucket/ivyrepo"
            credentials(AwsCredentials) {
                accessKey "someKey"
                secretKey "someSecret"
                // optional
                sessionToken "someSTSToken"
            }
        }
    }

You can also delegate all credentials to the AWS sdk by using the AwsImAuthentication. The following example shows how:

Example: Declaring a S3 backed Maven and Ivy repository using IAM

`build.gradle`

    repositories {
        maven {
            url "s3://myCompanyBucket/maven2"
            authentication {
               awsIm(AwsImAuthentication) // load from EC2 role or env var
            }
        }

        ivy {
            url "s3://myCompanyBucket/ivyrepo"
            authentication {
               awsIm(AwsImAuthentication)
            }
        }
    }

When using a Google Cloud Storage backed repository default application credentials will be used with no further configuration required:

Example: Declaring a Google Cloud Storage backed Maven and Ivy repository using default application credentials

`build.gradle`

    repositories {
        maven {
            url "gcs://myCompanyBucket/maven2"
        }

        ivy {
            url "gcs://myCompanyBucket/ivyrepo"
        }
    }

### S3 configuration properties

The following system properties can be used to configure the interactions with s3 repositories:

S3 configuration properties

Property | Description
org.gradle.s3.endpoint | Used to override the AWS S3 endpoint when using a non AWS, S3 API compatible, storage service.
org.gradle.s3.maxErrorRetry | Specifies the maximum number of times to retry a request in the event that the S3 server responds with a HTTP 5xx status code. When not specified a default value of 3 is used.

### S3 URL formats

S3 URL’s are 'virtual-hosted-style' and must be in the following format `s3://<bucketName>[.<regionSpecificEndpoint>]/<s3Key>`

e.g. `s3://myBucket.s3.eu-central-1.amazonaws.com/maven/release`

- `myBucket` is the AWS S3 bucket name.

- `s3.eu-central-1.amazonaws.com` is the optional region specific endpoint.

- `/maven/release` is the AWS S3 key (unique identifier for an object within a bucket)

### S3 proxy settings

A proxy for S3 can be configured using the following system properties:

- `https.proxyHost`

- `https.proxyPort`

- `https.proxyUser`

- `https.proxyPassword`

- `http.nonProxyHosts`

If the 'org.gradle.s3.endpoint' property has been specified with a http (not https) URI the following system proxy settings can be used:

- `http.proxyHost`

- `http.proxyPort`

- `http.proxyUser`

- `http.proxyPassword`

- `http.nonProxyHosts`

### AWS S3 V4 Signatures (AWS4-HMAC-SHA256)

Some of the AWS S3 regions (eu-central-1 - Frankfurt) require that all HTTP requests are signed in accordance with AWS’s signature version 4. It is recommended to specify S3 URL’s containing the region specific endpoint when using buckets that require V4 signatures. e.g. `s3://somebucket.s3.eu-central-1.amazonaws.com/maven/release`

When a region-specific endpoint is not specified for buckets requiring V4 Signatures, Gradle will use the default AWS region (us-east-1) and the following warning will appear on the console:

Attempting to re-send the request to …​. with AWS V4 authentication. To avoid this warning in the future, use region-specific endpoint to access buckets located in regions that require V4 signing.

Failing to specify the region-specific endpoint for buckets requiring V4 signatures means:

- `3 round-trips to AWS, as opposed to one, for every file upload and download.`

- `Depending on location - increased network latencies and slower builds.`

- `Increased likelihood of transmission failures.`

### Google Cloud Storage configuration properties

The following system properties can be used to configure the interactions with Google Cloud Storage repositories:

Google Cloud Storage configuration properties

Property | Description
org.gradle.gcs.endpoint | Used to override the Google Cloud Storage endpoint when using a non-Google Cloud Platform, Google Cloud Storage API compatible, storage service.
org.gradle.gcs.servicePath | Used to override the Google Cloud Storage root service path which the Google Cloud Storage client builds requests from, defaults to `/`.

### Google Cloud Storage URL formats

Google Cloud Storage URL’s are 'virtual-hosted-style' and must be in the following format `gcs://<bucketName>/<objectKey>`

e.g. `gcs://myBucket/maven/release`

- `myBucket` is the Google Cloud Storage bucket name.

- `/maven/release` is the Google Cloud Storage key (unique identifier for an object within a bucket)

### Configuring HTTP authentication schemes

When configuring a repository using HTTP or HTTPS transport protocols, multiple authentication schemes are available. By default, Gradle will attempt to use all schemes that are supported by the Apache HttpClient library, documented here. In some cases, it may be preferable to explicitly specify which authentication schemes should be used when exchanging credentials with a remote server. When explicitly declared, only those schemes are used when authenticating to a remote repository. The following example show how to configure a repository to use only digest authentication:

Example: Configure repository to use only digest authentication

`build.gradle`

    repositories {
        maven {
            url 'https://repo.mycompany.com/maven2'
            credentials {
                username "user"
                password "password"
            }
            authentication {
                digest(DigestAuthentication)
            }
        }
    }

Currently supported authentication schemes are:

Authentication schemes

Type | Description
`BasicAuthentication` | Basic access authentication over HTTP. When using this scheme, credentials are sent preemptively.
`DigestAuthentication` | Digest access authentication over HTTP.

### Using preemptive authentication

Gradle’s default behavior is to only submit credentials when a server responds with an authentication challenge in the form of a HTTP 401 response. In some cases, the server will respond with a different code (ex. for repositories hosted on GitHub a 404 is returned) causing dependency resolution to fail. To get around this behavior, credentials may be sent to the server preemptively. To enable preemptive authentication simply configure your repository to explicitly use the `BasicAuthentication` scheme:

Example: Configure repository to use preemptive authentication

`build.gradle`

    repositories {
        maven {
            url 'https://repo.mycompany.com/maven2'
            credentials {
                username "user"
                password "password"
            }
            authentication {
                basic(BasicAuthentication)
            }
        }
    }