Server Infrastructure Project
Objective
Design and build a three-server environment using VirtualBox with an internal network: a gateway/router server running iptables NAT, an application server hosting Apache and a MariaDB database, and a monitoring server running Prometheus and Grafana. The project required inter-server communication, centralized user management via NFS-mounted home directories, and a monitoring dashboard displaying CPU, memory, and disk metrics for all nodes.
Tools & Technologies
Ubuntu Server 22.04— all three nodesiptables / nftables— NAT and firewall on gateway nodeApache2 + PHP— web application serverMariaDB 10.6— relational database backendNFS (nfs-kernel-server)— shared home directoriesPrometheus + Node Exporter— metrics collectionGrafana— metrics visualization and dashboardsVirtualBox internal network— isolated lab environmentrsyslog— centralized log forwarding
Architecture Overview
Step-by-Step Process
Assigned static IPs on the VirtualBox internal network. Configured the gateway server with two NICs (NAT for internet, internal for the lab subnet) and enabled IP forwarding with iptables MASQUERADE.
# On gateway: /etc/netplan/00-installer-config.yaml
network:
version: 2
ethernets:
enp0s3: # NAT adapter (internet)
dhcp4: true
enp0s8: # Internal network
addresses: [10.0.0.1/24]
nameservers:
addresses: [8.8.8.8]
sudo netplan apply
# Enable IP forwarding
echo "net.ipv4.ip_forward=1" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p
# NAT rule — masquerade internal subnet out enp0s3
sudo iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -o enp0s3 -j MASQUERADE
sudo iptables -A FORWARD -i enp0s8 -o enp0s3 -j ACCEPT
sudo iptables -A FORWARD -i enp0s3 -o enp0s8 -m state --state RELATED,ESTABLISHED -j ACCEPT
# Persist iptables rules
sudo apt install -y iptables-persistent
sudo netfilter-persistent save
Configured the app server as the NFS host for shared home directories. All three nodes mount /srv/homes so users have a consistent home directory across the infrastructure.
# On app server — NFS server setup
sudo apt install -y nfs-kernel-server
sudo mkdir -p /srv/homes
sudo chown nobody:nogroup /srv/homes
# Export the share
echo "/srv/homes 10.0.0.0/24(rw,sync,no_subtree_check,no_root_squash)" \
| sudo tee -a /etc/exports
sudo exportfs -arv
sudo systemctl enable --now nfs-kernel-server
# On gateway and monitor nodes — mount NFS
sudo apt install -y nfs-common
sudo mkdir -p /srv/homes
echo "10.0.0.2:/srv/homes /srv/homes nfs defaults,_netdev 0 0" \
| sudo tee -a /etc/fstab
sudo mount -a
df -h /srv/homes
Deployed a LAMP stack on the app server, created a test database with a web-accessible PHP info page, and locked down MariaDB to localhost only.
sudo apt install -y apache2 php libapache2-mod-php php-mysql mariadb-server
# Secure MariaDB
sudo mysql_secure_installation
# Remove anon users, disallow root remote login, remove test DB
# Create app database and user
sudo mysql -u root -p <<'SQL'
CREATE DATABASE appdb CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
CREATE USER 'appuser'@'localhost' IDENTIFIED BY 'SecurePass123!';
GRANT ALL PRIVILEGES ON appdb.* TO 'appuser'@'localhost';
FLUSH PRIVILEGES;
SQL
# Test PHP-MySQL connection
cat > /var/www/html/info.php <<'EOF'
<?php phpinfo(); ?>
EOF
# Verify from gateway
curl -s http://10.0.0.2/info.php | grep -i "php version"
Installed Node Exporter on all three servers and Prometheus + Grafana on the monitoring server. Configured Prometheus scrape jobs and imported a Node Exporter dashboard in Grafana.
# Install Node Exporter on ALL nodes (gateway, app, monitor)
wget https://github.com/prometheus/node_exporter/releases/download/v1.7.0/node_exporter-1.7.0.linux-amd64.tar.gz
tar xvf node_exporter-*.tar.gz
sudo cp node_exporter-*/node_exporter /usr/local/bin/
# Create systemd service
sudo tee /etc/systemd/system/node_exporter.service <<'EOF'
[Unit]
Description=Prometheus Node Exporter
[Service]
User=nobody
ExecStart=/usr/local/bin/node_exporter
[Install]
WantedBy=multi-user.target
EOF
sudo systemctl enable --now node_exporter
# On monitor server: install Prometheus
wget https://github.com/prometheus/prometheus/releases/download/v2.48.0/prometheus-2.48.0.linux-amd64.tar.gz
# ... extract and configure
# /etc/prometheus/prometheus.yml
scrape_configs:
- job_name: 'gateway'
static_configs:
- targets: ['10.0.0.1:9100']
- job_name: 'appserver'
static_configs:
- targets: ['10.0.0.2:9100']
- job_name: 'monitor'
static_configs:
- targets: ['10.0.0.3:9100']
Installed Grafana, added Prometheus as a data source, imported the official Node Exporter Full dashboard (ID 1860), and configured an alert rule for disk usage above 80%.
sudo apt install -y apt-transport-https
wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
echo "deb https://packages.grafana.com/oss/deb stable main" \
| sudo tee /etc/apt/sources.list.d/grafana.list
sudo apt update && sudo apt install -y grafana
sudo systemctl enable --now grafana-server
# Grafana accessible at http://10.0.0.3:3000
# Default: admin/admin → change immediately
# Add Prometheus datasource via UI:
# Configuration → Data Sources → Add → Prometheus
# URL: http://localhost:9090
# Import dashboard ID 1860 (Node Exporter Full)
# Dashboards → Import → Enter 1860 → Load → Select Prometheus DS
# Verify all three nodes appear in dashboard dropdowns
Complete Workflow
Challenges & Solutions
- Internal VMs could not reach the internet — iptables FORWARD chain had no ACCEPT rule for the internal-to-NAT direction. Adding the bidirectional FORWARD rules with connection tracking fixed routing.
- NFS mounts hanging at boot — The
_netdevoption in fstab was missing, causing mount attempts before networking was up. Adding it deferred mounts until the network was available. - Prometheus showing all targets as DOWN — UFW on each node was blocking port 9100. Added
ufw allow from 10.0.0.3 to any port 9100on each node to allow scraping only from the monitoring server. - Grafana dashboard showing no data — Prometheus data source URL was set to
10.0.0.3:9090(external IP) instead oflocalhost:9090. Grafana queries Prometheus from the local machine, not across the network.
Key Takeaways
- Multi-server infrastructure requires careful network planning before touching any configuration — IP addressing, routing, and firewall rules must be designed as a system, not node-by-node.
- NFS is a simple but effective solution for shared storage in small infrastructure; the
_netdevfstab option is critical for network-backed mounts to survive reboots cleanly. - Prometheus + Grafana is a powerful open-source observability stack — Node Exporter provides comprehensive host metrics with zero custom instrumentation.
- Firewall rules on monitoring targets should restrict Prometheus scraping to the monitoring server IP only, not open port 9100 to all sources.