Multi-Node Cassandra on One Machine with No Virtualization

30 Nov 2015

For testing purposes, multi-node Cassandra on one machine can be a handy thing.

$ cd ${HOME}

Let's clear out the data directory to prevent confusion:

$ rm -rf ${HOME}/apache-cassandra-3.0.0/data/*

Now let's make our node copies:

$ cp -r apache-cassandra-3.0.0 apache-cassandra-3.0.0-2
$ cp -r apache-cassandra-3.0.0 apache-cassandra-3.0.0-3

Let's be more specific in original cass installation:

$ cd ${HOME}/apache-cassandra-3.0.0/conf
$ vim cassandra.yaml

listen_address: localhost  ==>  listen_address: 127.0.0.1
rpc_address: localhost     ==>  rpc_address: 127.0.0.1

Let's set up node 2:

$ cd ${HOME}/apache-cassandra-3.0.0-2/conf
$ vim cassandra-env.sh

JMX_PORT="7199"  ==>  JMX_PORT="7200"

$ vim cassandra.yaml

listen_address: localhost  ==>  listen_address: 127.0.0.2
rpc_address: localhost     ==>  rpc_address: 127.0.0.2

Let's set up node 3:

$ cd ${HOME}/apache-cassandra-3.0.0-3/conf
$ vim cassandra-env.sh

JMX_PORT="7199"  ==>  JMX_PORT="7201"

$ vim cassandra.yaml

listen_address: localhost  ==>  listen_address: 127.0.0.3
rpc_address: localhost     ==>  rpc_address: 127.0.0.3

Start each node.

Note that you may want to wait a minute or two between starting each node to let them fully talk with each other and join the cluster. Also note that I left '- seeds: "127.0.0.1"' alone in each cassandra.yaml, so our apache cassandra running at address 127.0.0.1 has to be started first. But, for more flexibility, you could change '- seeds: "127.0.0.1"' to '- seeds: "127.0.0.1,127.0.0.2,127.0.0.3"'

Terminal 1:

$ cd ${HOME}/apache-cassandra-3.0.0/bin
$ ./cassandra -f

Terminal 2:

$ cd ${HOME}/apache-cassandra-3.0.0-2/bin
$ ./cassandra -f

Terminal 3:

$ cd ${HOME}/apache-cassandra-3.0.0-3/bin
$ ./cassandra -f

Terminal 4:

Let's check on the status of our newly-started cluster:

$ cd apache-cassandra-3.0.0/bin/
$ ./nodetool status
Datacenter: datacenter1
=======================
Status=Up/Down
|/ State=Normal/Leaving/Joining/Moving
--  Address    Load       Tokens       Owns    Host ID                               Rack
UN  127.0.0.1  111.96 KB  256          ?       2d878b9b-7e46-4402-ae24-5dd6ce4dee67  rack1
UN  127.0.0.2  73.63 KB   256          ?       64b09fd3-9b35-4e9c-b574-09949317bc32  rack1
UN  127.0.0.3  97.37 KB   256          ?       9087c06e-c129-403a-91de-19277e1a6750  rack1

Note: Non-system keyspaces don't have the same replication settings, effective ownership information is meaningless