4.4 The DataSource - Reference Documentation
Authors: Graeme Rocher, Peter Ledbrook, Marc Palmer, Jeff Brown, Luke Daley, Burt Beckwith, Lari Hotari
Version: 3.1.10
Table of Contents
4.4 The DataSource
Since Grails is built on Java technology setting up a data source requires some knowledge of JDBC (the technology that doesn't stand for Java Database Connectivity).If you use a database other than H2 you need a JDBC driver. For example for MySQL you would need Connector/J.Drivers typically come in the form of a JAR archive. It's best to use the dependency resolution to resolve the jar if it's available in a Maven repository, for example you could add a dependency for the MySQL driver like this:dependencies { runtime 'mysql:mysql-connector-java:5.1.29' }
lib
directory.Once you have the JAR resolved you need to get familiar with how Grails manages its database configuration. The configuration can be maintained in either grails-app/conf/application.groovy
or grails-app/conf/application.yml
. These files contain the dataSource definition which includes the following settings:
driverClassName
- The class name of the JDBC driverusername
- The username used to establish a JDBC connectionpassword
- The password used to establish a JDBC connectionurl
- The JDBC URL of the databasedbCreate
- Whether to auto-generate the database from the domain model - one of 'create-drop', 'create', 'update' or 'validate'pooled
- Whether to use a pool of connections (defaults to true)logSql
- Enable SQL logging to stdoutformatSql
- Format logged SQLdialect
- A String or Class that represents the Hibernate dialect used to communicate with the database. See the org.hibernate.dialect package for available dialects.readOnly
- Iftrue
makes the DataSource read-only, which results in the connection pool callingsetReadOnly(true)
on eachConnection
transactional
- Iffalse
leaves the DataSource's transactionManager bean outside the chained BE1PC transaction manager implementation. This only applies to additional datasources.persistenceInterceptor
- The default datasource is automatically wired up to the persistence interceptor, other datasources are not wired up automatically unless this is set totrue
properties
- Extra properties to set on the DataSource bean. See the Tomcat Pool documentation. There is also a Javadoc format documentation of the properties.jmxExport
- Iffalse
, will disable registration of JMX MBeans for all DataSources. By default JMX MBeans are added for DataSources withjmxEnabled = true
in properties.
application.groovy
may be something like:dataSource { pooled = true dbCreate = "update" url = "jdbc:mysql://localhost:3306/my_database" driverClassName = "com.mysql.jdbc.Driver" dialect = org.hibernate.dialect.MySQL5InnoDBDialect username = "username" password = "password" properties { jmxEnabled = true initialSize = 5 maxActive = 50 minIdle = 5 maxIdle = 25 maxWait = 10000 maxAge = 10 * 60000 timeBetweenEvictionRunsMillis = 5000 minEvictableIdleTimeMillis = 60000 validationQuery = "SELECT 1" validationQueryTimeout = 3 validationInterval = 15000 testOnBorrow = true testWhileIdle = true testOnReturn = false jdbcInterceptors = "ConnectionState;StatementCache(max=200)" defaultTransactionIsolation = java.sql.Connection.TRANSACTION_READ_COMMITTED } }
When configuring the DataSource do not include the type or the def keyword before any of the configuration settings as Groovy will treat these as local variable definitions and they will not be processed. For example the following is invalid:
dataSource { boolean pooled = true // type declaration results in ignored local variable … }
dataSource { pooled = true dbCreate = "update" url = "jdbc:mysql://localhost:3306/my_database" driverClassName = "com.mysql.jdbc.Driver" dialect = org.hibernate.dialect.MySQL5InnoDBDialect username = "username" password = "password" properties { // Documentation for Tomcat JDBC Pool // http://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html#Common_Attributes // https://tomcat.apache.org/tomcat-7.0-doc/api/org/apache/tomcat/jdbc/pool/PoolConfiguration.html jmxEnabled = true initialSize = 5 maxActive = 50 minIdle = 5 maxIdle = 25 maxWait = 10000 maxAge = 10 * 60000 timeBetweenEvictionRunsMillis = 5000 minEvictableIdleTimeMillis = 60000 validationQuery = "SELECT 1" validationQueryTimeout = 3 validationInterval = 15000 testOnBorrow = true testWhileIdle = true testOnReturn = false ignoreExceptionOnPreLoad = true // http://tomcat.apache.org/tomcat-7.0-doc/jdbc-pool.html#JDBC_interceptors jdbcInterceptors = "ConnectionState;StatementCache(max=200)" defaultTransactionIsolation = java.sql.Connection.TRANSACTION_READ_COMMITTED // safe default // controls for leaked connections abandonWhenPercentageFull = 100 // settings are active only when pool is full removeAbandonedTimeout = 120 removeAbandoned = true // use JMX console to change this setting at runtime logAbandoned = false // causes stacktrace recording overhead, use only for debugging // JDBC driver properties // Mysql as example dbProperties { // Mysql specific driver properties // http://dev.mysql.com/doc/connector-j/en/connector-j-reference-configuration-properties.html // let Tomcat JDBC Pool handle reconnecting autoReconnect=false // truncation behaviour jdbcCompliantTruncation=false // mysql 0-date conversion zeroDateTimeBehavior='convertToNull' // Tomcat JDBC Pool's StatementCache is used instead, so disable mysql driver's cache cachePrepStmts=false cacheCallableStmts=false // Tomcat JDBC Pool's StatementFinalizer keeps track dontTrackOpenResources=true // performance optimization: reduce number of SQLExceptions thrown in mysql driver code holdResultsOpenOverStatementClose=true // enable MySQL query cache - using server prep stmts will disable query caching useServerPrepStmts=false // metadata caching cacheServerConfiguration=true cacheResultSetMetadata=true metadataCacheSize=100 // timeouts for TCP/IP connectTimeout=15000 socketTimeout=120000 // timer tuning (disable) maintainTimeStats=false enableQueryTimeouts=false // misc tuning noDatetimeStringSync=true } } }
More on dbCreate
Hibernate can automatically create the database tables required for your domain model. You have some control over when and how it does this through thedbCreate
property, which can take these values:
- create - Drops the existing schema and creates the schema on startup, dropping existing tables, indexes, etc. first.
- create-drop - Same as create, but also drops the tables when the application shuts down cleanly.
- update - Creates missing tables and indexes, and updates the current schema without dropping any tables or data. Note that this can't properly handle many schema changes like column renames (you're left with the old column containing the existing data).
- validate - Makes no changes to your database. Compares the configuration with the existing database schema and reports warnings.
- any other value - does nothing
dbCreate
setting completely, which is recommended once your schema is relatively stable and definitely when your application and database are deployed in production. Database changes are then managed through proper migrations, either with SQL scripts or a migration tool like Liquibase (the Database Migration plugin uses Liquibase and is tightly integrated with Grails and GORM).
4.4.1 DataSources and Environments
The previous example configuration assumes you want the same config for all environments: production, test, development etc.Grails' DataSource definition is "environment aware", however, so you can do:dataSource { pooled = true driverClassName = "com.mysql.jdbc.Driver" dialect = org.hibernate.dialect.MySQL5InnoDBDialect // other common settings here }environments { production { dataSource { url = "jdbc:mysql://liveip.com/liveDb" // other environment-specific settings here } } }
4.4.2 Automatic Database Migration
ThedbCreate
property of the DataSource
definition is important as it dictates what Grails should do at runtime with regards to automatically generating the database tables from GORM classes. The options are described in the DataSource section:
create
create-drop
update
validate
- no value
dbCreate
is by default set to "create-drop", but at some point in development (and certainly once you go to production) you'll need to stop dropping and re-creating the database every time you start up your server.It's tempting to switch to update
so you retain existing data and only update the schema when your code changes, but Hibernate's update support is very conservative. It won't make any changes that could result in data loss, and doesn't detect renamed columns or tables, so you'll be left with the old one and will also have the new one.Grails supports migrations with Flyway or Liquibase using the same mechanism provided by Spring Boot.
4.4.3 Transaction-aware DataSource Proxy
The actualdataSource
bean is wrapped in a transaction-aware proxy so you will be given the connection that's being used by the current transaction or Hibernate Session
if one is active.If this were not the case, then retrieving a connection from the dataSource
would be a new connection, and you wouldn't be able to see changes that haven't been committed yet (assuming you have a sensible transaction isolation setting, e.g. READ_COMMITTED
or better).The "real" unproxied dataSource
is still available to you if you need access to it; its bean name is dataSourceUnproxied
.You can access this bean like any other Spring bean, i.e. using dependency injection:class MyService { def dataSourceUnproxied … }
ApplicationContext
:def dataSourceUnproxied = ctx.dataSourceUnproxied
4.4.4 Database Console
The H2 database console is a convenient feature of H2 that provides a web-based interface to any database that you have a JDBC driver for, and it's very useful to view the database you're developing against. It's especially useful when running against an in-memory database.You can access the console by navigating to http://localhost:8080/dbconsole in a browser. The URI can be configured using thegrails.dbconsole.urlRoot
attribute in application.groovy
and defaults to '/dbconsole'
.The console is enabled by default in development mode and can be disabled or enabled in other environments by using the grails.dbconsole.enabled
attribute in application.groovy
. For example, you could enable the console in production like this:environments { production { grails.serverURL = "http://www.changeme.com" grails.dbconsole.enabled = true grails.dbconsole.urlRoot = '/admin/dbconsole' } development { grails.serverURL = "http://localhost:8080/${appName}" } test { grails.serverURL = "http://localhost:8080/${appName}" } }
If you enable the console in production be sure to guard access to it using a trusted security framework.
Configuration
By default the console is configured for an H2 database which will work with the default settings if you haven't configured an external database - you just need to change the JDBC URL tojdbc:h2:mem:devDB
. If you've configured an external database (e.g. MySQL, Oracle, etc.) then you can use the Saved Settings dropdown to choose a settings template and fill in the url and username/password information from your application.groovy
.
4.4.5 Multiple Datasources
By default all domain classes share a singleDataSource
and a single database, but you have the option to partition your domain classes into two or more DataSource
s.Configuring Additional DataSources
The defaultDataSource
configuration in grails-app/conf/application.yml
looks something like this:--- dataSource: pooled: true jmxExport: true driverClassName: org.h2.Driver username: sa password:environments: development: dataSource: dbCreate: create-drop url: jdbc:h2:mem:devDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE test: dataSource: dbCreate: update url: jdbc:h2:mem:testDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE production: dataSource: dbCreate: update url: jdbc:h2:prodDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE properties: jmxEnabled: true initialSize: 5
DataSource
with the Spring bean named dataSource
. To configure extra DataSource
s, add a dataSources
block (at the top level, in an environment block, or both, just like the standard DataSource
definition) with a custom name. For example, this configuration adds a second DataSource
, using MySQL in the development environment and Oracle in production:--- dataSources: dataSource: pooled: true jmxExport: true driverClassName: org.h2.Driver username: sa password: lookup: dialect: org.hibernate.dialect.MySQLInnoDBDialect driverClassName: com.mysql.jdbc.Driver username: lookup password: secret url: jdbc:mysql://localhost/lookup dbCreate: updateenvironments: development: dataSources: dataSource: dbCreate: create-drop url: jdbc:h2:mem:devDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE test: dataSources: dataSource: dbCreate: update url: jdbc:h2:mem:testDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE production: dataSources: dataSource: dbCreate: update url: jdbc:h2:prodDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE properties: jmxEnabled: true initialSize: 5 … lookup: dialect: org.hibernate.dialect.Oracle10gDialect driverClassName: oracle.jdbc.driver.OracleDriver username: lookup password: secret url: jdbc:oracle:thin:@localhost:1521:lookup dbCreate: update
Configuring Domain Classes
If a domain class has noDataSource
configuration, it defaults to the standard 'dataSource'
. Set the datasource
property in the mapping
block to configure a non-default DataSource
. For example, if you want to use the ZipCode
domain to use the 'lookup'
DataSource
, configure it like this:class ZipCode { String code static mapping = { datasource 'lookup' } }
DataSource
s. Use the datasources
property with a list of names to configure more than one, for example:class ZipCode { String code static mapping = { datasources(['lookup', 'auditing']) } }
DataSource
and one or more others, use the special name 'DEFAULT'
to indicate the default DataSource
:class ZipCode { String code static mapping = { datasources(['lookup', 'DEFAULT']) } }
DataSource
s use the special value 'ALL'
:class ZipCode { String code static mapping = { datasource 'ALL' } }
Namespaces and GORM Methods
If a domain class uses more than oneDataSource
then you can use the namespace implied by each DataSource
name to make GORM calls for a particular DataSource
. For example, consider this class which uses two DataSource
s:class ZipCode { String code static mapping = { datasources(['lookup', 'auditing']) } }
DataSource
specified is the default when not using an explicit namespace, so in this case we default to 'lookup'. But you can call GORM methods on the 'auditing' DataSource
with the DataSource
name, for example:def zipCode = ZipCode.auditing.get(42) … zipCode.auditing.save()
DataSource
to the method call in both the static case and the instance case.Hibernate Mapped Domain Classes
You can also partition annotated Java classes into separate datasources. Classes using the default datasource are registered ingrails-app/conf/hibernate.cfg.xml
. To specify that an annotated class uses a non-default datasource, create a hibernate.cfg.xml
file for that datasource with the file name prefixed with the datasource name.For example if the Book
class is in the default datasource, you would register that in grails-app/conf/hibernate.cfg.xml
:<?xml version='1.0' encoding='UTF-8'?> <!DOCTYPE hibernate-configuration PUBLIC '-//Hibernate/Hibernate Configuration DTD 3.0//EN' 'http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd'> <hibernate-configuration> <session-factory> <mapping class='org.example.Book'/> </session-factory> </hibernate-configuration>
Library
class is in the "ds2" datasource, you would register that in grails-app/conf/ds2_hibernate.cfg.xml
:<?xml version='1.0' encoding='UTF-8'?> <!DOCTYPE hibernate-configuration PUBLIC '-//Hibernate/Hibernate Configuration DTD 3.0//EN' 'http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd'> <hibernate-configuration> <session-factory> <mapping class='org.example.Library'/> </session-factory> </hibernate-configuration>
Services
Like Domain classes, by default Services use the defaultDataSource
and PlatformTransactionManager
. To configure a Service to use a different DataSource
, use the static datasource
property, for example:class DataService { static datasource = 'lookup' void someMethod(...) {
…
}
}
DataSource
, so be sure to only make changes for domain classes whose DataSource
is the same as the Service.Note that the datasource specified in a service has no bearing on which datasources are used for domain classes; that's determined by their declared datasources in the domain classes themselves. It's used to declare which transaction manager to use.What you'll see is that if you have a Foo domain class in dataSource1 and a Bar domain class in dataSource2, and WahooService uses dataSource1, a service method that saves a new Foo and a new Bar will only be transactional for Foo since they share the datasource. The transaction won't affect the Bar instance. If you want both to be transactional you'd need to use two services and XA datasources for two-phase commit, e.g. with the Atomikos plugin.Transactions across multiple datasources
Grails uses the Best Efforts 1PC pattern for handling transactions across multiple datasources.The Best Efforts 1PC pattern is fairly general but can fail in some circumstances that the developer must be aware of. This is a non-XA pattern that involves a synchronized single-phase commit of a number of resources. Because the 2PC is not used, it can never be as safe as an XA transaction, but is often good enough if the participants are aware of the compromises.The basic idea is to delay the commit of all resources as late as possible in a transaction so that the only thing that can go wrong is an infrastructure failure (not a business-processing error). Systems that rely on Best Efforts 1PC reason that infrastructure failures are rare enough that they can afford to take the risk in return for higher throughput. If business-processing services are also designed to be idempotent, then little can go wrong in practice.The BE1PC implementation was added in Grails 2.3.6. . Before this change additional datasources didn't take part in transactions initiated in Grails. The transactions in additional datasources were basically in auto commit mode. In some cases this might be the wanted behavior. One reason might be performance: on the start of each new transaction, the BE1PC transaction manager creates a new transaction to each datasource. It's possible to leave an additional datasource out of the BE1PC transaction manager by settingtransactional = false
in the respective configuration block of the additional dataSource. Datasources with readOnly = true
will also be left out of the chained transaction manager (since 2.3.7).By default, the BE1PC implementation will add all beans implementing the Spring PlatformTransactionManager
interface to the chained BE1PC transaction manager. For example, a possible JMSTransactionManager
bean in the Grails application context would be added to the Grails BE1PC transaction manager's chain of transaction managers.You can exclude transaction manager beans from the BE1PC implementation with the this configuration option:
grails.transaction.chainedTransactionManagerPostProcessor.blacklistPattern = '.*'
transactional = false
or readOnly = true
will be skipped and using this configuration option is not required in that case.XA and Two-phase Commit
When the Best Efforts 1PC pattern isn't suitable for handling transactions across multiple transactional resources (not only datasources), there are several options available for adding XA/2PC support to Grails applications.The Spring transactions documentation contains information about integrating the JTA/XA transaction manager of different application servers. In this case, you can configure a bean with the nametransactionManager
manually in resources.groovy
or resources.xml
file.There is also Atomikos plugin available for XA support in Grails applications.