CockroachDB is a scalable and cloud-native SQL database for building scalable cloud services. It is specially designed to store copies of data in multiple locations to deliver speedy access. It’s a distributed SQL database built on the transactional and key-value store. IT can survive disk, machine, and data center failures with near-zero latency disruption and no manual intervention.
This tutorial will show you how to set up three three-node CockroachDB cluster on an Ubuntu server.
Prerequisites
- Three server running Ubuntu 20.04.
- A root password is configured the server.
For this tutorial, we will use the following setup:
hostname IP address
node1 104.245.33.97
node2 216.98.11.175
node3 45.58.38.224
Getting Started
Before starting, you will need to update all nodes to the latest version. You can update all nodes one by one by running the following command:
apt-get update -y
Once all the nodes are updated, restart them to apply the changes.
Setup Time Synchronization
Before starting, you will need to set up a time synchronization between all nodes. You can do it using the chrony. Chrony is a flexible implementation of the Network Time Protocol (NTP). It is used to synchronize the system clock from different NTP servers
First, install chrony with the following command:
apt-get install chrony -y
Once installed, edit the chrony configuration file with the following command:
nano /etc/chrony/chrony.conf
Find the default pool and replace them with the following lines:
pool 0.id.pool.ntp.org iburst maxsources 4 pool 1.id.pool.ntp.org iburst maxsources 1 pool 2.id.pool.ntp.org iburst maxsources 1 pool 3.id.pool.ntp.org iburst maxsources 2
Save and close the file then restart the chrony service and enable it to start at system reboot with the following command:
systemctl restart chrony systemctl enable chrony
Once you have finished, you can proceed to the next step.
Install CockroachDB
First, you will need to install CockroachDB on all nodes.
You can download the latest version of CockroachDB from their official website with the following command:
wget https://binaries.cockroachdb.com/cockroach-latest.linux-amd64.tgz
Once the download is completed, extract the downloaded file with the following command:
tar -xvzf cockroach-latest.linux-amd64.tgz
Next, copy the CockroachDB binary to the /usr/local/bin directory with the following command:
cp cockroach-*/cockroach /usr/local/bin/
Next, verify the CockroachDB version using the following command:
cockroach version
You should get the following output:
Build Tag: v20.1.6 Build Time: 2020/09/24 18:16:45 Distribution: CCL Platform: linux amd64 (x86_64-unknown-linux-gnu) Go Version: go1.13.9 C Compiler: gcc 6.3.0 Build SHA-1: be8c0a720e98a147263424cc13fc9bfc75f46013 Build Type: release Note: Run all commands on all nodes.
Create Certificates
First, create a directory to store certificate files on all nodes. Run the following command on all nodes to create a certs directory:
mkdir ~/certs
Next, you will need to create a CA certificate, root certificate, and client certificate.
Create CA Cetificate
On the node1, create a CA certificate with the following command:
cockroach cert create-ca --certs-dir=certs --ca-key=certs/ca.key
This command will generate ca.key and ca.crt inside ~/certs directory.
Next, copy the generated CA to both nodes using the following command:
scp ~/certs/ca.crt ~/certs/ca.key [email protected]:~/certs/ scp ~/certs/ca.crt ~/certs/ca.key [email protected]:~/certs/
Create Client Certificate
Next, you will need to generate a client certificate to secure the communication between SQL and cluster.
Run the following command on all nodes to generate the client certificate:
cockroach cert create-client root --certs-dir=certs --ca-key=certs/ca.key
Once you are finished, you can proceed to create server certificate.
Create Server Certificates
Next, you will need to generate the server certificate to secure communication between servers on the CockroachDB cluster.
On the node1, run the following command to generate the server certificate:
cockroach cert create-node localhost $(hostname) 104.245.33.97 --certs-dir=certs --ca-key=certs/ca.key
On the node2, run the following command to generate the server certificate:
cockroach cert create-node localhost $(hostname) 216.98.11.175 --certs-dir=certs --ca-key=certs/ca.key
On the node3, run the following command to generate the server certificate:
cockroach cert create-node localhost $(hostname) 45.58.38.224 --certs-dir=certs --ca-key=certs/ca.key
This will generate the node.key and node.crt file inside ~/certs directory.
You can list all certificates with the following command:
cockroach --certs-dir=certs cert list
You should get the following output:
Certificate directory: certs Usage | Certificate File | Key File | Expires | Notes | Error ---------+------------------+-----------------+------------+----------------------------------------+-------- CA | ca.crt | | 2030/10/06 | num certs: 1 | Node | node.crt | node.key | 2025/10/02 | addresses: localhost,db1,104.245.33.97 | Client | client.root.crt | client.root.key | 2025/10/02 | user: root | (3 rows)
Once you are finished, you can proceed to the next step.
Start CockroachDB Cluster
At this point, all certificates are ready to start the cluster.
On the node1, run the following command to initialize the Secure CockroachDB Cluster:
cockroach start --background --certs-dir=certs --advertise-host=104.245.33.97 --listen-addr=104.245.33.97
You can check the status of the cluster with the following command:
cockroach node status --certs-dir=certs --host=104.245.33.97
You should get the following output:
id | address | sql_address | build | started_at | updated_at | locality | is_available | is_live -----+---------------------+---------------------+---------+----------------------------------+----------------------------------+----------+--------------+---------- 1 | 104.245.33.97:26257 | 104.245.33.97:26257 | v20.1.6 | 2020-09-28 08:34:44.939507+00:00 | 2020-09-28 08:36:10.492789+00:00 | | true | true (1 row)
Once you are finished, you can proceed to the next step.
Add Both Nodes to the Cluster
Next, you must add both nodes to the CockroachDB secure cluster.
On the node2, please run the following command to add it to the CockroachDB secure cluster.
cockroach start --background --certs-dir=certs --advertise-host=216.98.11.175 --listen-addr=216.98.11.175 --join=104.245.33.97:26257
On node 3, please run the following command to add it to the CockroachDB secure cluster.
cockroach start --background --certs-dir=certs --advertise-host=45.58.38.224 --listen-addr=45.58.38.224 --join=104.245.33.97:26257
Next, go back to the node1 and check the status of your cluster with the following command:
cockroach node status --certs-dir=certs --host=104.245.33.97
You should see that all nodes are added to the cluster.
id | address | sql_address | build | started_at | updated_at | locality | is_available | is_live -----+---------------------+---------------------+---------+----------------------------------+----------------------------------+----------+--------------+---------- 1 | 104.245.33.97:26257 | 104.245.33.97:26257 | v20.1.6 | 2020-09-28 08:34:44.939507+00:00 | 2020-09-28 08:45:42.014332+00:00 | | true | true 2 | 216.98.11.175:26257 | 216.98.11.175:26257 | v20.1.6 | 2020-09-28 08:37:12.209878+00:00 | 2020-09-28 08:45:40.747232+00:00 | | true | true 3 | 45.58.38.224:26257 | 45.58.38.224:26257 | v20.1.6 | 2020-09-28 08:39:37.913658+00:00 | 2020-09-28 08:45:37.97068+00:00 | | true | true (3 rows)
At this point, CockroachDB cluster is started and listening on port 8080.
Once you are finished, you can proceed to the next step.
Access CockroachDB Dashboard
CockroachDB provides a simple and easy-to-use web interface to monitor the cluster. Before accessing the CockroachDB web interface, you will need to create an admin user and set a password for it.
First, log into the cockroach DB SQL shell with the following command:
cockroach sql --certs-dir=certs --host=104.245.33.97
Next, create a user named hitesh and set the password with the following command:
CREATE USER hitesh WITH PASSWORD 'mypassword';
Next, exit from the SQL shell then access the CockroachDB web interface using the URL https://node1-ip-address:8080. You should see the following page:
Provide your admin username and password, and click on the LOG IN button. You should see the CockroachDB dashboard in the next page:
Verify Database Replication
Next, you will need to verify whether the database is replicated among all nodes.
On node1, log into the SQL shell with the following command:
cockroach sql --certs-dir=certs --host=104.245.33.97
Next, create a database named testdb1 and testdb2 with the following command:
create database testdb1; create database testdb2;
Next, verify the databases with the following command:
show databases;
You should see all databases in the following output:
database_name ----------------- defaultdb postgres system testdb1 testdb2 (5 rows) Time: 3.568509ms
Next, go to Node2 and verify whether the database is replicated.
On the node2, log into the SQL shell with the following command:
cockroach sql --certs-dir=certs --host=216.98.11.175
Run the following command to show the databases:
show databases;
You should see the testdb1 and testdb2 in the following output:
database_name ----------------- defaultdb postgres system testdb1 testdb2 (5 rows) Time: 19.196903ms
The above result indicates that the database replication is working on the CockroachDB Cluster.
Conclusion
Congratulations! You have successfully installed and set up a secure CockroachDB Cluster on an Ubuntu 20.04 server. You can now add more nodes to the cluster easily. Feel free to ask me if you have any questions. For more information, you can visit the CockroachDB official documentation.