Running Vservers on Debian

From Section6wiki

Jump to: navigation, search

Author: kz

Here's how to make a jail on debian using vserver:

Contents

Get services to listen to only one address

First, get your daemons to listen to only the ip addr of the main system. If you actually use inetd, you'll want to use xinetd instead, otherwise disable it: 

update-rc.d -f inetd remove
killall inetd

If you run xinetd put this into /etc/xinetd.conf 

defaults {   bind = 10.95.81.15 }

Don't forget sshd Edit /etc/ssh/sshd_config 

ListenAddress 10.0.0.5

Finally do: 

# netstat -l

and see if there are any other services listening on all interfaces.

Installing vserver

Configuring the Kernel

# apt-get install kernel-patch-vserver

You'll also need to install the source for your kernel. You can do 

# uname -a
Linux foo 2.6.8-2-686 #1 Thu May 19 17:53:30 JST 2005 i686 GNU/Linux

This means you'll need to get kernel source 2.6.8 if you want to run the same kernel: 

# apt-get install kernel-source-2.6.8

And for make menuconfig and make-kpkg 

# apt-get install ncurses-dev debhelper modutils kernel-package fakeroot module-init-tools

Untar your kernel and link it 

# tar -jxvf kernel-source-2.6.8.tar.bz2
# ln -s kernel-source-2.6.8 linux

Patch the kernel 

# cd /usr/src/linux
# ../kernel-patches/all/apply/vserver

And if you want quotas 

# ../kernel-patches/all/apply/vserver-quota

You should see something like: 

START applying vserver patch (Context switching virtual private servers)
Testing whether "Context switching virtual private servers" patch for 2.6.8 applies (dry run):
"Context switching virtual private servers" patch for 2.6.8 succeeded
Removing empty files:
Done.
END applying vserver patch

Now run: 

# make menuconfig

At this point vserver should already be selected. You may want to go into 

Processor type and features -> Processor family

and select your processor type, if you want SMP support you'll also want to select: 

Processor type and features -> [*] Symmetric multi-processing support

When asked "Do you wish to save your new kernel configuration?" select < Yes >

Building and Installing the Kernel

OPTIONAL: If you have 2 procs: 

# echo "CONCURRENCY_LEVEL=2" >> /etc/kernel-pkg.conf

Now make the kernel package: 

# make-kpkg clean
# fakeroot make-kpkg --append_to_version -vserver --initrd --revision=1.0 kernel_image

This will take a long time so go for a walk or read /. or something... Finally install the kernel and reboot: 

# cd /usr/src/
# dpkg -i kernel-image*.deb
# reboot

This should ideally put your vserver kernel at the top. Upon reboot make sure your new kernel is loaded: 

# uname -a
Linux foo 2.6.8-vserver #1 Thu Jun 30 10:51:20 PDT 2005 i686 GNU/Linux

Creating a new vserver for the first time

This is the current accepted way to build a vserver. It uses a crazy directory structure which is outlined in the infamous yet sadistic flower page. See references at the bottom for an important note about the flower page.

In the following examples we're going to create a sample server called www. You'll want to replace this with whatever you decide to call your vserver.

You'll want to install a few vserver packages first: 

# apt-get install util-vserver vserver-debiantools

You'll also need a symlink to your vserver home 

# rm /etc/vservers/.defaults/vdirbase
# ln -s /home/vservers /etc/vservers/.defaults/vdirbase

Now configure /etc/vservers.conf 

#Home of your vservers
VSERVERS_ROOT=/home/vservers
# When starting or entering a vserver, its /etc/mtab is generated on
# the fly so it matches the various volumes mounted inside the vserver
GENERATEMTAB=yes

Now build your vserver (this example has 2 ips): 

vserver www build -m debootstrap --hostname www --interface eth0:10.0.0.80 \ 
--interface eth0:10.0.0.81 -- -d sarge -- --exclude=$REMOVE_PACKAGES

Once you do this you may want to restrict the capabilities of your vserver. Your vserver will run fine without this step but will likely have more privileges than it needs.

Edit the file /etc/vservers/www/bcapabilites (for 2.6 kernel) or the capabilities file for 2.4 kernel: 

CAP_NET_BIND_SERVICE

These will enable a web server to work just fine. More capabilities are Running_Vservers_on_Debian#A_Review_of_S_CAPS here.

If you want this vserver to start on boot you'll have to edit /etc/vservers/www/apps/init/mark: 

echo "default" > /etc/vservers/www/apps/init/mark

Now copy your apt confs to your new vserver: 

cd /home/vservers
cp /etc/apt/sources.list /etc/apt/apt.conf www/etc/apt

If you want sshd to work you'll have to add this device 

cd /home/vservers/www/dev/
mknod ptmx c 5 2
chmod 666 ptmx

Note: A caveat about ifconfig:

You'll notice that your root server's ifconfig and the ifconfig results in your vserver will only show the first (real) interface in the list. If you want the additional aliases to show you must edit the file: 

/etc/vservers/<vserver>/interfaces/0/name

Otherwise any additional IPs you made with vserver will not show up. Also take note you must make names for any additional interfaces. This doesn't happen with the legacy method and is something you should take note of.

Creating a new vserver for the first time (old legacy method)

Note: You can skip this section if you're doing things the above way.

This is the old way to build a vserver. It still works, for now, but you'll get a warning every time you start your vserver. In the future this method is likely to be deprecated.

In the following examples we're going to create a sample server called www. You'll want to replace this with whatever you decide to call your vserver.

You'll want to install a few vserver packages first: 

# apt-get install util-vserver vserver-debiantools

Now configure /etc/vservers.conf 

#Home of your vservers
VSERVERS_ROOT=/home/vservers
# When starting or entering a vserver, its /etc/mtab is generated on
# the fly so it matches the various volumes mounted inside the vserver
GENERATEMTAB=yes

Now create a conf for the vserver you wish to create for example /etc/vservers/www.conf 

# Most easy thing is to have an own ip-address for each vserver
IPROOT=10.0.0.80
IPROOTMASK=255.255.255.0
# How shall the networkdevice be named from the view of the vserver
IPROOTDEV=eth0
# shown hostname
S_HOSTNAME=www
# lock = you are not allowed to create a new context in your context
S_FLAGS="lock nproc"
# What capabilities shall the server have? Look for explanation
S_CAPS="CAP_NET_RAW CAP_NET_BIND_SERVICE"
# Start this server on booting? This will be checked in /etc/init.d/vservers
ONBOOT=yes

Now make your vserver home and make some basics 

# mkdir -p /home/vservers/www/etc/apt

You'll also need a symlink to your vserver home 

# rm /etc/vservers/.defaults/vdirbase
# ln -s /home/vservers /etc/vservers/.defaults/vdirbase

Install the bootstrap (a list is in: /usr/lib/debootstrap/scripts/) If you need to use a proxy, be sure to edit /etc/wgetrc first... 

# cd /home/vservers
# debootstrap sarge www
# cd /home/vservers/www
# cp -rp /etc/apt* /home/vservers/www/etc/apt

Configuring the base system

Now lets enter the vserver and install the base system: If you haven't run vprocunhide yet do so 

# /etc/init.d/vprocunhide start

Now start the server 

# vserver www start
# vserver www enter

More than likely you'll want locales on your system or you'll get a lot of complaints from aptitude and other programs 

# apt-get update
# apt-get install locales
# locale-gen

OPTIONAL: you can run base config to configure your timezone and configure your mail agent, however everything else should be done for you. 

# base-config

Also keep in mind that by default, debian enables the following: 

/sbin/syslogd
/usr/sbin/inetd
/usr/sbin/atd
/usr/sbin/cron

If you don't want one or all of these you'll have to disable them one at a time with a command like: 

# update-rc.d -f inetd remove

Which is the same one we used at the beginning.

You'll also be able to save a few megs by deleting any packages you don't need: 

# apt-get --purge remove ipchains iptables pppoe pppoeconf ppp pppconfig dhcp-client nano

Finally delete any passwords in the password file you don't need with: 

vipw

Installing items on your vserver

Once inside run: 

aptitude

And install what packages you want in your vserver.

A Review of S_CAPS

SCAPS give you certain capabilities as part of your vserver. Below is a review of some commonly used ones:

Capability Description
CAP_CHOWN You can chown files
CAP_SYS_TIME You can set time (use date,ntpd)
CAP_LINUX_IMMUTABLE You can modify immutable flags (chattr -i)
CAP_NET_BROADCAST You can send broadcast packets (zeroconf, samba)
CAP_NET_BIND_SERVICE You can bind to ports below 1024
CAP_NET_RAW You can create raw sockets (ping, tcpdump, etc)
CAP_SYS_NICE Allow nice and renice of processes
CAP_SYS_MODULE You can insert and remove kernel modules
CAP_SYS_RESOURCE override resource limits;override quota and reserved space;modify data journaling, colsoles, keymap (Used by bind9)
CAP_NET_ADMIN You can do any network operations including sniffing
CAP_SYS_ADMIN You can do everything, even mount filesystems

For a complete list see [1]

Creating additional vservers

You can use vserver-copy and copy a blank vserver to another one, or you can do it from scratch.
It's much like the above. I'll show a different example now that it's all set up: 

vserver mysql build -m debootstrap --hostname mysql --interface eth0:10.0.0.50 \ 
 -- -d sarge -- --exclude=$REMOVE_PACKAGES

Edit the file /etc/vservers/mysql/bcapabilites (for 2.6 kernel) or the capabilities file for 2.4 kernel: 

CAP_NET_BIND_SERVICE

If you want it to start on boot: 

echo "default" > /etc/vservers/mysql/apps/init/mark

If you want the IPs to be visible with names 

echo "mysql" > /etc/vservers/mysql/interfaces/0/name

Now copy your apt confs to your new vserver: 

cd /home/vservers
cp /etc/apt/sources.list /etc/apt/apt.conf mysql/etc/apt

If you want sshd to work you'll have to add this device 

cd /home/vservers/mysql/dev/
mknod ptmx c 5 2
chmod 666 ptmx

Now start the server 

# vserver mysql start
# vserver mysql enter

More than likely you'll want locales on your system or you'll get a lot of complaints from aptitude and other programs 

# apt-get update
# apt-get install locales
# locale-gen

Use base-config to configure your timezone and mail agent: 

# base-config

Remove any packages you don't need: 

# apt-get --purge remove ipchains iptables pppoe pppoeconf ppp pppconfig dhcp-client nano

Remove any unwanted services running by default with: 

# update-rc.d -f inetd remove

Finally delete any passwords in the password file you don't need with: 

vipw

Some Notes on vservers

While vservers may start automatically, there's no way to tell what processes are running inside them unless you enter them and run ps or do: 

vserver servername exec ps auxww

Below are other useful vserver commands:

Command Description
vserver service stop|start Stop or start a vserver
vserver service enter Enter the specified vserver
vserver-stat Give ps like stats of running vservers
vserver-info Give info about the vserver program itself
vserver-copy service newservice Copy or move a vserver.
vtop A top command that shows the top of *all* vservers
vpstree A tree ps view of processes of all vservers
vps A ps that shows processes of all vservers

Debian notes for Dell 1850

The 2.6.8 kernel that sarge comes with doesn't come with the appropriate megaraid module. Thus you must get a newer kernel from sid.

So update /etc/apt/sources.list 

deb http://debian.oregonstate.edu/debian/ unstable main
deb-src http://debian.oregonstate.edu/debian/ unstable main

And install the sources for 2.6.11

When doing make menuconfig be sure to add: 

Device Drivers ->
  SCSI device support --->
     SCSI low-level drivers --->
        [*] LSI Login New Generation RAID Device Drivers
        [M] LSI Logic Management Module (New Driver)
        [M] LSI Logic MegaRAID Driver (New Driver)

This will add the modules megaraid_mm and megaraid_mbox after you make the kernel package and install it you'll need to mess with the initrd image because the debian initrd-tools puts the wrong drivers in the loadmodules file (what it initally modprobes before loading the rest of the system).

So go to your home directory and do: 

cp /boot/initrd.img-2.6.11-vserver .
mount -t cramfs initrd.img-2.6.11-vserver /mnt -o loop
mkdir initrd
cd initrd
mkdir bin bin2 dev dev2 devfs etc keyscripts lib mnt proc sbin scripts sys tmp usr var
cp -ra /mnt/* .

You may get warnings when you copy the files from /mnt but these can safely be ignored.

Now we'll want to edit the loadmodules file so that it says: 

modprobe -k  unix 2> /dev/null
modprobe -k  megaraid_mm
modprobe -k  megaraid_mbox
modprobe -k  sd_mod

And now we make a new initrd image: 

cd ..
mkcramfs initrd initrd.img-2.6.11-vserver
cp initrd.img-2.6.11-vserver /boot

Once you have your new initrd image you can also use it between other Dells with the same kernel image package. Just be sure to install the modules mentioned above (including module-init-tools) before you install the kernel.

Once you're done be sure to comment the lines out in /etc/apt/sources.list so you don't end up sidifying your sarge distribution.

References

Retrieved from "http://www.section6.net/wiki/index.php/Running_Vservers_on_Debian"
Personal tools