Objective

Design and build a three-server environment using VirtualBox with an internal network: a gateway/router server running iptables NAT, an application server hosting Apache and a MariaDB database, and a monitoring server running Prometheus and Grafana. The project required inter-server communication, centralized user management via NFS-mounted home directories, and a monitoring dashboard displaying CPU, memory, and disk metrics for all nodes.

Tools & Technologies

  • Ubuntu Server 22.04 — all three nodes
  • iptables / nftables — NAT and firewall on gateway node
  • Apache2 + PHP — web application server
  • MariaDB 10.6 — relational database backend
  • NFS (nfs-kernel-server) — shared home directories
  • Prometheus + Node Exporter — metrics collection
  • Grafana — metrics visualization and dashboards
  • VirtualBox internal network — isolated lab environment
  • rsyslog — centralized log forwarding

Architecture Overview

flowchart TD Internet[Host Machine / Internet] -->|NAT| GW GW[Gateway Server\n10.0.0.1\niptables NAT + UFW] GW -->|internal| APP GW -->|internal| MON APP[App Server\n10.0.0.2\nApache2 + MariaDB] MON[Monitor Server\n10.0.0.3\nPrometheus + Grafana] NFS[NFS Share\n/srv/homes →\nmounted on all nodes] APP --> NFS MON -->|scrape :9100| APP MON -->|scrape :9100| GW style Internet fill:#181818,stroke:#1e1e1e,color:#888 style GW fill:#1a1a2e,stroke:#00d4ff,color:#e0e0e0 style APP fill:#1a1a2e,stroke:#00d4ff,color:#e0e0e0 style MON fill:#1a1a2e,stroke:#00d4ff,color:#e0e0e0 style NFS fill:#181818,stroke:#1e1e1e,color:#888

Step-by-Step Process

01
Network Design & Gateway Configuration

Assigned static IPs on the VirtualBox internal network. Configured the gateway server with two NICs (NAT for internet, internal for the lab subnet) and enabled IP forwarding with iptables MASQUERADE.

# On gateway: /etc/netplan/00-installer-config.yaml
network:
  version: 2
  ethernets:
    enp0s3:          # NAT adapter (internet)
      dhcp4: true
    enp0s8:          # Internal network
      addresses: [10.0.0.1/24]
      nameservers:
        addresses: [8.8.8.8]

sudo netplan apply

# Enable IP forwarding
echo "net.ipv4.ip_forward=1" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

# NAT rule — masquerade internal subnet out enp0s3
sudo iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -o enp0s3 -j MASQUERADE
sudo iptables -A FORWARD -i enp0s8 -o enp0s3 -j ACCEPT
sudo iptables -A FORWARD -i enp0s3 -o enp0s8 -m state --state RELATED,ESTABLISHED -j ACCEPT

# Persist iptables rules
sudo apt install -y iptables-persistent
sudo netfilter-persistent save
02
NFS Home Directory Server

Configured the app server as the NFS host for shared home directories. All three nodes mount /srv/homes so users have a consistent home directory across the infrastructure.

# On app server — NFS server setup
sudo apt install -y nfs-kernel-server
sudo mkdir -p /srv/homes
sudo chown nobody:nogroup /srv/homes

# Export the share
echo "/srv/homes  10.0.0.0/24(rw,sync,no_subtree_check,no_root_squash)" \
  | sudo tee -a /etc/exports
sudo exportfs -arv
sudo systemctl enable --now nfs-kernel-server

# On gateway and monitor nodes — mount NFS
sudo apt install -y nfs-common
sudo mkdir -p /srv/homes
echo "10.0.0.2:/srv/homes  /srv/homes  nfs  defaults,_netdev  0  0" \
  | sudo tee -a /etc/fstab
sudo mount -a
df -h /srv/homes
03
Apache2 + MariaDB Application Stack

Deployed a LAMP stack on the app server, created a test database with a web-accessible PHP info page, and locked down MariaDB to localhost only.

sudo apt install -y apache2 php libapache2-mod-php php-mysql mariadb-server

# Secure MariaDB
sudo mysql_secure_installation
# Remove anon users, disallow root remote login, remove test DB

# Create app database and user
sudo mysql -u root -p <<'SQL'
CREATE DATABASE appdb CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'appuser'@'localhost' IDENTIFIED BY 'SecurePass123!';
GRANT ALL PRIVILEGES ON appdb.* TO 'appuser'@'localhost';
FLUSH PRIVILEGES;
SQL

# Test PHP-MySQL connection
cat > /var/www/html/info.php <<'EOF'
<?php phpinfo(); ?>
EOF

# Verify from gateway
curl -s http://10.0.0.2/info.php | grep -i "php version"
04
Prometheus & Node Exporter Deployment

Installed Node Exporter on all three servers and Prometheus + Grafana on the monitoring server. Configured Prometheus scrape jobs and imported a Node Exporter dashboard in Grafana.

# Install Node Exporter on ALL nodes (gateway, app, monitor)
wget https://github.com/prometheus/node_exporter/releases/download/v1.7.0/node_exporter-1.7.0.linux-amd64.tar.gz
tar xvf node_exporter-*.tar.gz
sudo cp node_exporter-*/node_exporter /usr/local/bin/

# Create systemd service
sudo tee /etc/systemd/system/node_exporter.service <<'EOF'
[Unit]
Description=Prometheus Node Exporter
[Service]
User=nobody
ExecStart=/usr/local/bin/node_exporter
[Install]
WantedBy=multi-user.target
EOF
sudo systemctl enable --now node_exporter

# On monitor server: install Prometheus
wget https://github.com/prometheus/prometheus/releases/download/v2.48.0/prometheus-2.48.0.linux-amd64.tar.gz
# ... extract and configure

# /etc/prometheus/prometheus.yml
scrape_configs:
  - job_name: 'gateway'
    static_configs:
      - targets: ['10.0.0.1:9100']
  - job_name: 'appserver'
    static_configs:
      - targets: ['10.0.0.2:9100']
  - job_name: 'monitor'
    static_configs:
      - targets: ['10.0.0.3:9100']
05
Grafana Dashboard & Alerting

Installed Grafana, added Prometheus as a data source, imported the official Node Exporter Full dashboard (ID 1860), and configured an alert rule for disk usage above 80%.

sudo apt install -y apt-transport-https
wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
echo "deb https://packages.grafana.com/oss/deb stable main" \
  | sudo tee /etc/apt/sources.list.d/grafana.list
sudo apt update && sudo apt install -y grafana
sudo systemctl enable --now grafana-server

# Grafana accessible at http://10.0.0.3:3000
# Default: admin/admin → change immediately

# Add Prometheus datasource via UI:
# Configuration → Data Sources → Add → Prometheus
# URL: http://localhost:9090

# Import dashboard ID 1860 (Node Exporter Full)
# Dashboards → Import → Enter 1860 → Load → Select Prometheus DS

# Verify all three nodes appear in dashboard dropdowns

Complete Workflow

flowchart LR A[Provision 3 VMs\nAssign static IPs] --> B[Configure Gateway\nNAT + IP Forward] B --> C[Setup NFS\nShared Homes] C --> D[Deploy LAMP\nApp Server] D --> E[Install Node Exporter\nAll Nodes] E --> F[Install Prometheus\nMonitor Server] F --> G[Install Grafana\nDashboard + Alerts] G --> H[Verify All\nServices + Metrics] style A fill:#1a1a2e,stroke:#00d4ff,color:#e0e0e0 style H fill:#1a1a2e,stroke:#00ff88,color:#e0e0e0 style B fill:#181818,stroke:#1e1e1e,color:#888 style C fill:#181818,stroke:#1e1e1e,color:#888 style D fill:#181818,stroke:#1e1e1e,color:#888 style E fill:#181818,stroke:#1e1e1e,color:#888 style F fill:#181818,stroke:#1e1e1e,color:#888 style G fill:#181818,stroke:#1e1e1e,color:#888

Challenges & Solutions

  • Internal VMs could not reach the internet — iptables FORWARD chain had no ACCEPT rule for the internal-to-NAT direction. Adding the bidirectional FORWARD rules with connection tracking fixed routing.
  • NFS mounts hanging at boot — The _netdev option in fstab was missing, causing mount attempts before networking was up. Adding it deferred mounts until the network was available.
  • Prometheus showing all targets as DOWN — UFW on each node was blocking port 9100. Added ufw allow from 10.0.0.3 to any port 9100 on each node to allow scraping only from the monitoring server.
  • Grafana dashboard showing no data — Prometheus data source URL was set to 10.0.0.3:9090 (external IP) instead of localhost:9090. Grafana queries Prometheus from the local machine, not across the network.

Key Takeaways

  • Multi-server infrastructure requires careful network planning before touching any configuration — IP addressing, routing, and firewall rules must be designed as a system, not node-by-node.
  • NFS is a simple but effective solution for shared storage in small infrastructure; the _netdev fstab option is critical for network-backed mounts to survive reboots cleanly.
  • Prometheus + Grafana is a powerful open-source observability stack — Node Exporter provides comprehensive host metrics with zero custom instrumentation.
  • Firewall rules on monitoring targets should restrict Prometheus scraping to the monitoring server IP only, not open port 9100 to all sources.