Search

Server reinstall and massive configuration change

Server reinstall and massive configuration change

This week I decided that I would like to virtualise all my server functionality and segment functionality across several virtual machines.

The first decision was the frontend for the virtual machines. I chose ProxMox and installed it with ease.

My next consideration was how many virtual machines I needed. I decided on 5 as a starting point:
APSVM: Apache proxy server virtual machine. This server forwards requests for sites to the appropriate virtual machine
VM1-ME: My server. Contains my sites.
VM2-JIM: A friend’s VM. Contains his sites.
VM3-JAIL: A jailed VM. Secure server for third party use.
VM4-MIRROR: Mirror VM. All servers and software that I mirror are hosted on here.

The next step was to configure port forwarding using iptables to allow access to the individual servers and sites. This was a major hassle.
It is set up so that I have ftp, ssh, http and rsync on each virtual machine and accessible from the internet:
APSVM is the only VM that has 2 standard ports open to the internet (http/80) as all the domains must be accessible on port 80 and DNS/53 so the server can provide domain name services.

Each VM (including APSVM) has apps mapped to non-standard ports:

VM name VM port external port
APSVM FTP/21 2100
APSVM SSH/22 2200
APSVM HTTP/80 8000
APSVM RSYNC/873 8730
VM1-ME FTP/21 2101
VM1-ME SSH/22 2201
VM1-ME HTTP/80 8001
VM1-ME RSYNC/873 8731
VM2-JIM FTP/21 2102
VM2-JIM SSH/22 2202
VM2-JIM HTTP/80 8002
VM2-JIM RSYNC/873 8732
VM3-JAIL FTP/21 2103
VM3-JAIL SSH/22 2203
VM3-JAIL HTTP/80 8003
VM3-JAIL RSYNC/873 8733
VM4-MIRROR FTP/21 2104
VM4-MIRROR SSH/22 2204
VM4-MIRROR HTTP/80 8004
VM4-MIRROR RSYNC/873 8734
The eagle-eyed among you will have noticed that to calculate the external port I have multiplied the port by 10 and checked if the port is < 1000, if so multiply by 10 a second time. I then add the VM number (0-APSVM, 1-ME, 2-JIM, 3-JAIL, 4-MIRROR). Doing this means that I can connect directly with any VM immediately and ensure it is set up correctly. As far as the VM is concerned I have connected to the standard port which means no VM-side configuration change and that I could clone a VM, change the IP address information and be up and ready for stage 2 in no time.The next step (and the step I am currently taking a break from by writing this blog post) is to migrate the old sites and users to the appropriate VMs.After that all that remains is to ensure that the relevant people are notified of the new SSH and FTP ports and everything is complete.After taking 3 days so far I am looking forward to finishing this task. It is relatively straightforward but incredibly easy to make a mistake and leave a site misconfigured or the port mapping incorrect.iptables has been a major source of stress while doing this and caused no end of issues. Here is the script I used to set up iptables automatically and provide a simple mechanism for future expansion:

#!/bin/bash
sys=`which sysctl`
ipt=`which iptables`
mp=`which modprobe`
grep=`which grep`

if [[ “$sys” == “” || $ipt == “” || $mp == “” || $grep == “” ]]; then
echo “Error – needed program not found”
return -1
fi

services=/etc/services

function getport
{
local a b p
p=$1
if [ “$p” == “” ]; then
echo “noport”
return
fi
a=`$grep -m 1 -w “$p” $services`
if [ “$a” == “” ]; then
echo “no match”
return
fi
b=${a%%/*}
p=${b%%$p*}
echo -n $p
}

ipprefix=192.168.0

#the IP below ($proxyip) handles all requests for $proxiedports from the outside world
proxyip=200
#Starting IP
firstip=200

dns=53
http=80
ftp=21
ssh=22
rsync=873
proxiedports=( $dns $http )

#VM names
vm1=201
vm2=202
vm3=203
vm4=204
toips=( $proxyip $vm1 $vm2 $vm3 $vm4 )
ports=( $ftp $ssh $dns $http $rsync )

netmask=24
extif=vmbr0
intif=vmbr1

#Flush iptables
$ipt -F
$ipt -t nat -F

# Net Sharing
$mp iptable_nat

echo Enabling ipv4 forwarding
$sys net.ipv4.ip_forward=1
echo 1 > /proc/sys/net/ipv4/ip_forward

$ipt -t nat -A POSTROUTING -o eth0 -j MASQUERADE
$ipt -A INPUT -m state –state ESTABLISHED,RELATED -j ACCEPT
$ipt -A INPUT -i lo -j ACCEPT
$ipt -A INPUT -j LOG –log-level 4 –log-prefix “ATTACK”
$ipt -A INPUT -j DROP

# Add your additional rules here
$ipt -t nat -A POSTROUTING -s $ipprefix.0/$netmask -j MASQUERADE

#hardcoded rules
#default services that need remapping and must remain on a standard port:
# DNS (port $dns)
# Web server (port $http)
#both of these will be handled by the first VM

for port in ${proxiedports[@]}; do
echo Setting up $ipprefix.$proxyip to handle external calls to port $port \($(getport $port)\)
$ipt -t nat -A PREROUTING -p tcp -i $extif –dport $port -j DNAT –to-destination $ipprefix.$proxyip:$port
$ipt -t nat -A POSTROUTING -p tcp -o $extif -d $ipprefix.$proxyip –dport $port -j MASQUERADE
$ipt -t nat -A PREROUTING -p udp -i $extif –dport $port -j DNAT –to-destination $ipprefix.$proxyip:$port
$ipt -t nat -A POSTROUTING -p udp -o $extif -d $ipprefix.$proxyip –dport $port -j MASQUERADE
done

count=0
for toip in ${toips[@]}; do
echo Configuring VM $ipprefix.$toip
for port in ${ports[@]}; do
oldport=$port
if [ $(printf “%.0f” $port) -gt 99 ]; then
port=$(echo “scale=1; $port / 10″ | bc)
if [ $(printf “%.0f” $port) -gt 99 ]; then
port=$(echo “scale=1; $port / 10″ | bc)
fi
fi

mport=$(printf “%.0f” `echo “scale=0; ($port * 100) + ($toip – $firstip)” | bc`)
port=$oldport
echo mapping external port $mport to $ipprefix.$toip:$port \($(getport $port)\)
$ipt -t nat -D PREROUTING -i $extif -p tcp –dport $mport -j DNAT –to-destination $ipprefix.$toip:$port > /dev/null 2>&1
$ipt -t nat -D POSTROUTING -o $extif -p tcp -d $ipprefix.$toip –dport $port -j MASQUERADE > /dev/null 2>&1
$ipt -t nat -A PREROUTING -i $extif -p tcp –dport $mport -j DNAT –to-destination $ipprefix.$toip:$port
$ipt -t nat -A POSTROUTING -o $extif -p tcp -d $ipprefix.$toip –dport $port -j MASQUERADE
done
done
This script is run at bootup on the VM host to provide all the port forwarding to the individual VMs.

Related posts

Leave a Comment