User quick start guide for testing application DR

This page will help you to set up an environment for testing Regional-DR basic tests.

This video will demonstrate how to test application DR after completing the configuration.

What you’ll need

To set up a Ramen development environment you will need to run 3 Kubernetes clusters. Ramen makes this easy using minikube, but you need enough resources:

Bare metal or virtual machine with nested virtualization enabled
8 CPUs or more
20 GiB of free memory
100 GiB of free space
Internet connection
Linux - tested on Fedora 37 and 38
non-root user with sudo privileges (all instructions are for non-root user)

Setting up your machine

Fork the Ramen repository at https://github.com/RamenDR/ramen

Clone the source locally

git clone https://github.com/{your_github_id}/ramen.git
cd ramen

Create python virtual environment
The Ramen project use python tool to create and provision test environment and run tests. The create a virtual environment including the tools run:
```
make venv
```
This create a virtual environment in ~/.venv/ramen and a symbolic link venv for activating the environment.
To activate the environment use:
```
source venv
```
To exit virtual environment issue command deactivate.
Enable libvirtd service if disabled.
```
sudo systemctl enable libvirtd --now
```
Verify libvirtd service is now active with no errors.
```
sudo systemctl status libvirtd -l
```
Add yourself to the libvirt group (required for minikube kvm2 driver).
```
sudo usermod -a -G libvirt $(whoami)
```
Logout and login again for the change above to be in effect.
Install @virtualization group - on Fedora you can use:
```
sudo dnf install @virtualization
rpm -q libvirt virt-manager
```
For more information see Virtualization on Fedora .

Install minikube - on Fedora you can use::

sudo dnf install https://github.com/kubernetes/minikube/releases/latest/download/minikube-latest.x86_64.rpm
minikube version

NOTE: minikube version 1.39.0 or later is required, latest version is recommended.

Install the kubectl tool

version=$(curl --fail --silent --show-error --location https://dl.k8s.io/release/stable.txt)
curl --fail --silent --show-error --location --remote-name \
    "https://dl.k8s.io/release/$version/bin/linux/amd64/kubectl"
sudo install kubectl /usr/local/bin
rm kubectl
kubectl version --client

For more info see Install and Set Up kubectl on Linux Tested with version v1.37.0.

Validate the installation

Run the drenv-selftest to validate that we can create a test cluster:

test/scripts/drenv-selftest

Example output:

1. Activating the ramen virtual environment ...


2. Creating a test cluster ...

2023-11-12 14:53:43,321 INFO    [drenv-selftest-vm] Starting environment
2023-11-12 14:53:43,367 INFO    [drenv-selftest-cluster] Starting minikube cluster
2023-11-12 14:54:15,331 INFO    [drenv-selftest-cluster] Cluster started in 31.96 seconds
2023-11-12 14:54:15,332 INFO    [drenv-selftest-cluster/0] Running addons/example/start
2023-11-12 14:54:33,181 INFO    [drenv-selftest-cluster/0] addons/example/start completed in 17.85 seconds
2023-11-12 14:54:33,181 INFO    [drenv-selftest-cluster/0] Running addons/example/test
2023-11-12 14:54:33,381 INFO    [drenv-selftest-cluster/0] addons/example/test completed in 0.20 seconds
2023-11-12 14:54:33,381 INFO    [drenv-selftest-vm] Environment started in 50.06 seconds

3. Deleting the test cluster ...

2023-11-12 14:54:33,490 INFO    [drenv-selftest-vm] Deleting environment
2023-11-12 14:54:33,492 INFO    [drenv-selftest-cluster] Deleting cluster
2023-11-12 14:54:34,106 INFO    [drenv-selftest-cluster] Cluster deleted in 0.61 seconds
2023-11-12 14:54:34,106 INFO    [drenv-selftest-vm] Environment deleted in 0.62 seconds

drenv is set up properly

Install the clusteradm tool

bash -o pipefail -c 'curl --fail --silent --show-error --location \
    https://raw.githubusercontent.com/open-cluster-management-io/clusteradm/main/install.sh \
    | bash -s 0.11.2'
clusteradm version

For more info see Install clusteradm CLI tool .

WARNING: clusteradm 0.11.1 is not compatible

Install the subctl tool

bash -o pipefail -c 'curl --fail --silent --show-error --location https://get.submariner.io | bash'
sudo install .local/bin/subctl /usr/local/bin/
rm .local/bin/subctl
subctl version

For more info see Submariner subctl installation . Version v0.24.0 or later is required.

Install the velero tool

curl --fail --silent --show-error --location --output velero.tar.gz \
    https://github.com/vmware-tanzu/velero/releases/download/v1.14.0/velero-v1.14.0-linux-amd64.tar.gz
tar xf velero.tar.gz --strip 1 velero-v1.14.0-linux-amd64/velero
sudo install velero /usr/local/bin
rm velero.tar.gz velero
velero version

For more info see Velero Basic Install

Install the virtctl tool.

curl --fail --silent --show-error --location --output virtctl \
    https://github.com/kubevirt/kubevirt/releases/download/v1.6.0/virtctl-v1.6.0-linux-amd64
sudo install virtctl /usr/local/bin
rm virtctl
virtctl version

For more info see virtctl install

Install mc tool

curl --fail --silent --show-error --location --output mc \
    https://dl.min.io/client/mc/release/linux-amd64/mc
sudo install mc /usr/local/bin
rm mc
mc --version

For more info see MinIO Client Quickstart

Install kustomize tool

bash -o pipefail -c 'curl --fail --silent --show-error --location \
    https://raw.githubusercontent.com/kubernetes-sigs/kustomize/master/hack/install_kustomize.sh \
    | bash'
sudo install kustomize /usr/local/bin
rm kustomize
kustomize version

For more info see kustomize install

Install the argocd tool

curl --fail --silent --show-error --location --output argocd \
    https://github.com/argoproj/argo-cd/releases/download/v3.4.4/argocd-linux-amd64
sudo install argocd /usr/local/bin/
rm argocd
argocd version --client

For more info see argocd installation

Install the kubectl-gather plugin

bash -o pipefail <<'EOF'
tag="$(curl -fsSL https://api.github.com/repos/nirs/kubectl-gather/releases/latest | jq -r .tag_name)"
os="$(uname | tr '[:upper:]' '[:lower:]')"
machine="$(uname -m)"
if [ "$machine" = "aarch64" ]; then machine="arm64"; fi
if [ "$machine" = "x86_64" ]; then machine="amd64"; fi
curl -L -o kubectl-gather https://github.com/nirs/kubectl-gather/releases/download/$tag/kubectl-gather-$tag-$os-$machine
sudo install kubectl-gather /usr/local/bin
rm kubectl-gather
EOF
kubectl gather --version

kubectl-gather version 0.6.0 or later is required. Tested with version 0.8.0. For more info see kubectl-gather

Install helm tool - on Fedora you can use:

helm_version=$(curl --fail --silent --show-error --location https://get.helm.sh/helm-latest-version)
bash -o pipefail -c "curl --fail --silent --show-error --location \
    https://get.helm.sh/helm-${helm_version}-linux-amd64.tar.gz \
    | tar xz --strip 1 linux-amd64/helm"
sudo install helm /usr/local/bin
rm -f helm
helm version

See Installing Helm for other options Tested with version v4.0.1.

Install podman - on Fedora you can use:
```
sudo dnf install podman
podman --version
```
Tested with version 5.5.1.
Install golang - on Fedora you can use:
```
sudo dnf install golang
go version
```
Tested with version go1.24.4.

Starting the test environment

Before using the drenv tool to start a test environment, you need to activate the python virtual environment:

cd ramen
source venv

This environment simulates a Regional DR setup, when the managed clusters are located in different zones (e.g. US, Europe) and storage is replicated asynchronously. This environment includes 3 clusters - a hub cluster (hub), and 2 managed clusters (dr1, dr2) using Ceph storage. The connectivity between managed clusters will be configured using subctl (Submariner client). In addition, rbd-mirroring is configured using the Submariner connection for ceph image replication of block volumes.

To start the environment, run:

cd test
drenv start envs/regional-dr.yaml

Starting takes 10-15 minutes, depending on your machine and internet connection.

Example output:

2023-11-02 05:26:41,729 INFO    [rdr] Starting environment
2023-11-02 05:26:41,860 INFO    [dr1] Starting minikube cluster
2023-11-02 05:26:42,948 INFO    [dr2] Starting minikube cluster
2023-11-02 05:26:43,929 INFO    [hub] Starting minikube cluster
2023-11-02 05:28:23,677 INFO    [dr1] Cluster started in 101.82 seconds
[...]
2023-11-02 05:40:40,777 INFO    [rdr] Dumping ramen e2e config to '/home/aclewett/.config/drenv/rdr'
2023-11-02 05:40:41,129 INFO    [rdr] Environment started in 839.40 seconds

The kubconfigs for dr1, dr2 and hub can be found in $HOME/.config/drenv/rdr/kubeconfigs/.

Validate that the clusters are created.

minikube profile list

Example output:

|---------|-----------|------------|-----------------|------|---------|---------|-------|--------|
| Profile | VM Driver |  Runtime   |       IP        | Port | Version | Status  | Nodes | Active |
|---------|-----------|------------|-----------------|------|---------|---------|-------|--------|
| dr1     | kvm2      | containerd | 192.168.122.70  | 8443 | v1.27.4 | Running |     1 |        |
| dr2     | kvm2      | containerd | 192.168.122.51  | 8443 | v1.27.4 | Running |     1 |        |
| hub     | kvm2      | containerd | 192.168.122.162 | 8443 | v1.27.4 | Running |     1 |        |
|---------|-----------|------------|-----------------|------|---------|---------|-------|--------|

You can inspect the environment using these commands.

kubectl get deploy -A --context hub
kubectl get deploy -A --context dr1
kubectl get deploy -A --context dr2

Build the ramen operator image

Build the Ramen operator container image:

cd ramen
make docker-build

Select docker.io/library/golang:1.21 when prompted.

podman build -t quay.io/ramendr/ramen-operator:latest .
[1/2] STEP 1/9: FROM golang:1.21 AS builder
? Please select an image:
    registry.fedoraproject.org/golang:1.21
    registry.access.redhat.com/golang:1.21
  ▸ docker.io/library/golang:1.21
    quay.io/golang:1.21

This builds the image quay.io/ramendr/ramen-operator:latest

Deploy and Configure the ramen operator

To deploy the Ramen operator in the test environment:

ramendev deploy test/envs/regional-dr.yaml

For more info on the ramendev tool see ramendev/README.md .

Ramen now needs to be configured to know about the managed clusters and the s3 endpoints. To configure the ramen operator for test environment:

ramendev config test/envs/regional-dr.yaml

Running system tests

At this point Ramen is ready to protect workloads in your cluster, and you are ready for testing DR flows.

Please refer to the e2e testing documentation for detailed instructions on running end-to-end tests.

Undeploying the test environment

If you want to clean up your environment completely run this command. All clusters and resources will be deleted.

cd ramen/test
drenv delete envs/regional-dr.yaml