Realtime Communications - Clustered Servers

[Ref: OpenBSD 4.9, ejabberd 2.1.6, XMPP | Set up clustering in ejabberd | Chapter 6 Clustering]

We have two sites where we want to have Realtime Communications. We want a separate server at each site, but want to have the same accounts, administration etc.

  • first is the host name from /bin/hostname -s on our primary node
  • second is the host name from /bin/hostname -s on the secondary node

Install your ejabberd first node, as per Chat/XMPP instructions, and install the second node without configuring user accounts, access privileges since we will be taking that from the first node.

Make sure that the domain name used for first is resolvable at node second, and likewise the domain name for second is resolvable at node first.

Magic Cookies

[Ref: 3.4 Distributed Programming]

Erlang systems which talk to each other must have the same magic cookie. The easiest way to achieve this is by having a file called .erlang.cookie in you home directory on all machines on which you are going to run Erlang systems communicating with each other.

Copy /var/db/ejabberd/.erlang.cookie from the first node to replace the cookie in the second node.

/var/db/ejabberd is the OpenBSD Package default $HOME directory for the ejabberd server user _ejabberd.

Keeping the cookie in the $HOME directory means that ejabberd on node first and second will allow RPC between the two servers. When you need to run as root, with the same privileges, you can specify the -setcookie COOKIE-CONTENT

Nodes

When you start an Erlang System which is going to talk to other Erlang systems, you must give it a name.
erl -sname mynode

ejabberdctl normally handles this for us, but as with the above $NAME definition we can normally use the following as our node-name.

export EJHOME=/var/db/ejabberd
export NODE=ejabberd
export HOST=`hostname -s`
export NAME=${NODE}@${HOST}

Second: Clean Database

We want to share the database from host first to host second so we clear out our database configuration at host second.

On the second node:

export EJHOME=/var/db/ejabberd
export NODE=ejabberd
export HOST=`hostname -s`
export NAME=${NODE}@${HOST}

rm -rf ${EJHOME}/${NODE}

Second: Node Configuration

On the second host, we need to make some configuration changes to connect the server to the first host. From details in ejabberdctl we export the following configurations:

export NAME=${NODE}@${HOST}
cd ${EJHOME}

Start

# erl -sname $NAME -setcookie `cat ${EJHOME}/.erlang.cookie` \
    -mnesia dir '"/var/db/ejabberd/ejabberd"' \
    -mnesia extra_db_nodes "['ejabberd@first']" \
    -s mnesia

This connects the current Erlang node to the Erlang node ejabberd@first and leaves you in the erl shell. The database will be created in the dir path /var/db/ejabberd/ejabberd. We use this path, as it is the default SPOOLDIR used by /usr/local/sbin/ejabberdctl

Erlang R13B04 (erts-5.7.5) [source] [rq:1] [async-threads:0] [kernel-poll:false]

Eshell V5.7.5 (abort with ^G)
1> _

If everything is working correctly, you should be able to see ejabberd@second as one of the nodes in the web interface at http://first:5280/admin/nodes/

[Ref: mnesia:info().]

Within this erl shell you can view some generic information about your current database configuration using:

1> mnesia:info().

The key output information we want to look at include:

runnind db nodes   = [ejabberd@first, ejabberd@second]

% .. stuff left out ..

ram_copies         = [schema]
disc_copies        = []

Synchronise Databases

2> mnesia:change_table_copy_type(schema, node(), disc_copies).
{atomic,ok}
3> mnesia:info().
runnind db nodes   = [ejabberd@first, ejabberd@second]

% .. stuff left out ..

ram_copies         = []
disc_copies        = [schema]

Alternatively, you can specify the synchronisation through the web interface, by setting the Second Node's Database configuration for schema to be "RAM and disc copy"

The advantage, at this point, of reviewing the database tables at the Web Interface, is getting a cleaner management view of the synchronisation settings for the tables.

Quit from Mnesia

4> q().

The above quit may take some time depending on the database synchronisation changes you select.

Permissions

Because we ran the erl command as root, we need to make sure that the ejabberd user-account (_ejabberd) has all permissions to the database directory.

chown -R _ejabberd:_ejabberd /var/db/ejabberd/ejabberd