.. Licensed under the Apache License, Version 2.0 (the "License"); you may not
.. use this file except in compliance with the License. You may obtain a copy of
.. the License at
..
..   http://www.apache.org/licenses/LICENSE-2.0
..
.. Unless required by applicable law or agreed to in writing, software
.. distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
.. WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
.. License for the specific language governing permissions and limitations under
.. the License.

.. _cluster/setup:

======
Set Up
======

Everything you need to know to prepare the cluster for the installation of
CouchDB.

Firewall
========

If you do not have a firewall between your servers, then you can skip this.

CouchDB in cluster mode uses the port ``5984`` just as standalone, but it also
uses ``5986`` for node-local APIs.

Erlang uses TCP port ``4369`` (EPMD) to find other nodes, so all servers must be
able to speak to each other on this port. In an Erlang Cluster, all nodes are
connected to all other nodes. A mesh.

.. warning::
    If you expose the port ``4369`` to the Internet or any other untrusted
    network, then the only thing protecting you is the
    `cookie`_.

.. _cookie: http://erlang.org/doc/reference_manual/distributed.html

Every Erlang application then uses other ports for talking to each other. Yes,
this means random ports. This will obviously not work with a firewall, but it is
possible to force an Erlang application to use a specific port rage.

This documentation will use the range TCP ``9100-9200``. Open up those ports in
your firewalls and it is time to test it.

You need 2 servers with working hostnames. Let us call them server1 and server2.

On server1:

.. code-block:: bash

    erl -sname bus -setcookie 'brumbrum' -kernel inet_dist_listen_min 9100 -kernel inet_dist_listen_max 9200

Then on server2:

.. code-block:: bash

    erl -sname car -setcookie 'brumbrum' -kernel inet_dist_listen_min 9100 -kernel inet_dist_listen_max 9200

An explanation to the commands:
    * ``erl`` the Erlang shell.
    * ``-sname bus`` the name of the Erlang node.
    * ``-setcookie 'brumbrum'`` the "password" used when nodes connect to each
      other.
    * ``-kernel inet_dist_listen_min 9100`` the lowest port in the rage.
    * ``-kernel inet_dist_listen_max 9200`` the highest port in the rage.

This gives us 2 Erlang shells. shell1 on server1, shell2 on server2.
Time to connect them. The ``.`` is to Erlang what ``;`` is to C.

In shell1:

.. code-block:: erlang

    net_kernel:connect_node(car@server2).

This will connect to the node called ``car`` on the server called ``server2``.

If that returns true, then you have an Erlang cluster, and the firewalls are
open. If you get false or nothing at all, then you have a problem with the
firewall.

First time in Erlang? Time to play!
-----------------------------------

Run in both shells:

.. code-block:: erlang

    register(shell, self()).

shell1:

.. code-block:: erlang

    {shell, car@server2} ! {hello, from, self()}.

shell2:

.. code-block:: erlang

    flush().
    {shell, bus@server1} ! {"It speaks!", from, self()}.

shell1:

.. code-block:: erlang

    flush().

To close the shells, run in both:

.. code-block:: erlang

    q().

Make CouchDB use the open ports.
--------------------------------

Open ``vm.args``, on all nodes, and add ``-kernel inet_dist_listen_min 9100``
and ``-kernel inet_dist_listen_max 9200`` like below:

.. code-block:: erlang

    -name ...
    -setcookie ...
    ...
    -kernel inet_dist_listen_max 9100
    -kernel inet_dist_listen_max 9200

.. _cluster/setup/wizard:

The Cluster Setup Wizard
========================

Setting up a cluster of Erlang applications correctly can be a daunting
task. Luckily, CouchDB 2.0 comes with a convenient Cluster Setup Wizard
as part of the Fauxton web administration interface.

After installation and initial start-up, visit Fauxton at
``http://127.0.0.1:5984/_utils#setup``. You will be asked to set up
CouchDB as a single-node instance or set up a cluster.

When you click "setup cluster" you are asked for admin credentials again and
then to add nodes by IP address. To get more nodes, go through the same install
procedure on other machines. Be sure to specify the total number of nodes you
expect to add to the cluster before adding nodes.

In file etc/vm.args change the the line ``-name couchdb@127.0.0.1`` to
``-name couchdb@<this-nodes-ip-address| FQDN>`` for each node which defines
the node and must be seperate for each node. For clustered setup, each node in
system must have a unique name. Can also be a valid FQDN not necessarily the IP.

Before you can add nodes to form a cluster, you must have them listening on an
IP address accessible from the other nodes in the cluster.
Do this once per node:

.. code-block:: bash

    curl -X PUT http://127.0.0.1:5984/_node/couchdb@<this-nodes-ip-address>/_config/admins/admin -d '"password"'
    curl -X PUT http://127.0.0.1:5984/_node/couchdb@<this-nodes-ip-address>/_config/chttpd/bind_address -d '"0.0.0.0"'

Now you can enter their IP addresses in the setup screen on your first
node. And make sure to put in the admin username and password. And use
the same admin username and password on all nodes.

Once you added all nodes, click "Setup" and Fauxton will finish the
cluster configuration for you.

See http://127.0.0.1:5984/_membership to get a list of all the nodes in
your cluster.

Now your cluster is ready and available. You can send requests to any
one of the nodes and get to all the data.

For a proper production setup, you'd now set up an HTTP proxy in front
of the nodes, that does load balancing. We recommend `HAProxy`_. See
our `example configuration for HAProxy`_. All you need is to adjust the
ip addresses and ports.

.. _cluster/setup/api:

The Cluster Setup API
=====================

If you would prefer to manually configure your CouchDB cluster, CouchDB exposes
the ``_cluster_setup`` endpoint for that. After installation and initial setup,
we can set up the cluster. On each node we need to run the following command to
set up the node:

.. code-block:: bash

     curl -X POST -H "Content-Type: application/json" http://admin:password@127.0.0.1:5984/_cluster_setup -d '{"action": "enable_cluster", "bind_address":"0.0.0.0", "username": "admin", "password":"password", "node_count":"3"}'

After that we can join all the nodes together. Choose one node
as the "setup coordination node" to run all these commands on.
This is a "setup coordination node" that manages the setup and
requires all other nodes to be able to see it and vice versa.
Set up will not work with unavailable nodes.
The notion of "setup coordination node" will be gone once the setup is finished.
From then on, the cluster will no longer have a "setup coordination node".
To add a node run these commands for each node you want to add:

.. code-block:: bash

    curl -X POST -H "Content-Type: application/json" http://admin:password@127.0.0.1:5984/_cluster_setup -d '{"action": "enable_cluster", "bind_address":"0.0.0.0", "username": "admin", "password":"password", "port": 15984, "node_count": "3", "remote_node": "<remote-node-ip>", "remote_current_user": "<remote-node-username>", "remote_current_password": "<remote-node-password>" }'
    curl -X POST -H "Content-Type: application/json" http://admin:password@127.0.0.1:5984/_cluster_setup -d '{"action": "add_node", "host":"<remote-node-ip>", "port": <remote-node-port>, "username": "admin", "password":"password"}'

This will join the two nodes together.
Keep running the above commands for each
node you want to add to the cluster. Once this is done run the
following command to complete the setup and add the missing databases:

.. code-block:: bash

    curl -X POST -H "Content-Type: application/json" http://admin:password@127.0.0.1:5984/_cluster_setup -d '{"action": "finish_cluster"}'

Verify install:

.. code-block:: bash

    curl http://admin:password@127.0.0.1:5984/_cluster_setup

Response:

.. code-block:: bash

    {"state":"cluster_finished"}

Verify cluster nodes:

.. code-block:: bash

    curl http://admin:password@127.0.0.1:5984/_membership

Response:

.. code-block:: bash

    {
        "all_nodes": [
            "couchdb@couch1",
            "couchdb@couch2",
        ],
        "cluster_nodes": [
            "couchdb@couch1",
            "couchdb@couch2",
        ]
    }

You CouchDB cluster is now set up.

.. _HAProxy: http://haproxy.org/
.. _example configuration for HAProxy: https://github.com/apache/couchdb/blob/master/rel/haproxy.cfg
