 |
|
Recommended hardware
|
|
Capabilities & Requirements
- Boots from PXE prompt on EPIA-M BIOS, powerup -> POST -> Linux and initrd loaded and booted in 30 seconds
- Uses standard Fedora Core Kernel RPMs with slightly
modified initrd
- Kernel Network Block Device mounts a "network hard drive" from the server machine as / and runs Fedora from that
- "network hard drive" on the server contains an ext3
filesystem with a Fedora Core 3 install on it, that can include any
Fedora Core 3 stuff like KDE, etc
- Supports 5.1 AC3 sound, widescreen video
- Really NO storage device needed on the EPIA end, no HDD or
flash... all done over the network. "Network Hard Drive" is as
fast or faster (depending on server cache) than a local HDD.
- Requires a Windows or Linux server providing DHCP, TFTP, NBD and Samba (or a Windows Share).
- No NFS!
Detailed description of the network booting process
Step 1: BIOS Network Boot
 |
The EPIA BIOS can be configured to request an IP address from a DHCP server. Along with the IP address and other network settings, the DHCP server is configured to tell the EPIA to download a particular bootloader file from a particular TFTP server (often the same server is running all the needed services, DHCP, TFTP and NBD). In this case the DHCP server is configured to tell the EPIA to fetch a file called pxelinux.0 from the TFTP server |
Step 2: PXELINUX
| The pxelinux.0 file is
fetched from the TFTP server and the EPIA runs
it. Pxelinux.0 is a network-aware bootloader, it goes back to the
TFTP
server and looks for a file called pxelinux.cfg/01-<your MAC
address>. This is a text file describing the filename of the
appropriate Linux kernel on the TFTP server, and the filename of the
matching Initrd, which contains a filesystem with /bin, /etc, /lib and
so on). Pxelinux then goes back to the TFTP server and
downloads the
first the Linux kernel file and then the Initrd file. (The Initrd
file
is deliberately kept on a strict diet to minimize startup time; we add
some stuff into the Redhat initrd to allow our network boot and NBD
capability) |
 |
3) Linux
| The last act of Pxelinux
is to start the downloaded kernel, this begins
the booting process. When the kernel initialization is over, the
kernel decompresses the Initrd file and runs a program called init.
This program then runs various scripts and programs that complete
the
booting action, in the same way as a normal Linux boot. |
4) Mounting the Network Block Device filesystem
 |
The Initrd just contains enough programs to start Linux up and get a network connection, the rest of the Linux distro packages and applications are kept in a file on the server and accessed over the Network Block Device kernel subsystem. This makes the remote file seem to be a mountable partition called /dev/nda on the EPIA. One of the actions of the Initrd scripts is to connect to the server machine using this method and mount /dev/nda as the root filesystem on the EPIA. This then appears as a normal ext3 mounted partition on the EPIA, it has been prepared with a bunch of Fedora Core 3 RPMs to contain a usable distro. There is no limit to the size of the network hard drive except the storage space on your server, but you have to set the size when creating the filesystem. 1GB is plenty for a media server type use, the media files themselves are accessed over a samba share. |
5) Running user apps!
| The picture shows mplayer
running, decoding AC3 in realtime and displaying at 16:9, at around 80%
CPU. The whole setup is completely silent (except for the
six-channel sound output directly by the EPIA) -- Gah! Look out,
Frodo! |
 |
Initializing the remote filesystem
Initially the remote filesystem is prepared with a minimal set of RPMs
from Fedora Core 3 to get you a text-mode login, but with yum and rpm
installed. You can then boot the EPIA over the network and use
yum to download further packages for you automatically to provide
x.org, KDE and so on.Step by step install instructions
0) Download and unpack the silentcat tarball
Unpack the silentcat tarball into /usr/srccd /usr/src
tar zxf /path/to/silentcat-0.2.tar.gz
1) Find out your EPIA MAC address
This can be found on a sticker at the back of the EPIA network/USB connector. If the sticker is missing, you can also find this out by setting the EPIA board for network booting in the BIOS, and it shows the MAC address on the monitor. Either way it will look like 00:40:xx:xx:xx:xx make a careful note of it because we will be needing it a lot. To simplify matters, run the following once you have the MAC, replacing the xx with your MAC numbers (NOTE that the numbers are separated by - here)export EPIAMAC=00-40-xx-xx-xx-xx
2) Setting up the server services
Make sure you have your DHCP and TFTP servers installed. For a Fedora server that means installing the following packages:
- dhcp
- tftp
- tftp-server
yum install dhcp tftp tftp-server
On Redhat TFTP operates under xinetd. The scripts assume that your TFTP base directory is /tftpboot, since this is the default for Redhat. You also need to edit /etc/xinetd.d/tftp to set disable to no; by default the service is disabled. Then to ensure that xinetd runs after a reboot, and runs now:
chkconfig xinetd on
service xinetd restart
TFTP requires a hole in your server firewall for UDP port 69. You can do this with (as root)
iptables -I INPUT -p udp --dport 69 -j ACCEPT
also copy this line to /etc/rc.local to ensure it works after the next boot.
A note on security - an external firewall, like a NAT router or a hardware firewall, must protect your server. Its an unnecessary risk to have people from the Internet being able to touch the NBD or TFTP ports. Be safe!
The DHCP server is controlled by /etc/dhcpd.conf, which you need to create. Here is an example of one set up for TFTP booting... cut and paste it, but you will need to change the IP addresses shown in bold to match your network setup.
allow booting; |
- If you are not on 192.168.0.* then you need to edit the subnet, netmask and broadcast-address lines to match your subnet
- You should change the IP addresses in "option
domain-name-servers 192.168.0.1;"
to match your ISP nameserver(s), or as in the case shown here, your
internal DNS server.
- "option routers 192.168.0.1;" should be changed to reflect your gateway IP out to the Internet.
- "next-server 192.168.0.31;" should be changed to your DHCP/TFTP server machine IP address.
- "hardware ethernet 00:40:63:ca:76:ed;"
should be changed to reflect the Ethernet MAC address of your EPIA board.
- lastly change the IP in "fixed-address 192.168.0.30;" to the IP address that you want the EPIA board to be allocated on your network.
Now we need to set the dhcp server to run every boot, and set it running now
chkconfig dhcpd on
service dhcpd restart
2a) Running the prepserver script
The rest of the steps have been automated into a script. The
steps carried out by the script are documented below for completeness,
but in fact running the prepserver script will jump you to section 12
in one step. Here is how to do that:cd /usr/src/silentcat-0.2
./prepserver 00-40-xx-xx-xx-xx
where you replace 00-40-xx-xx-xx-xx with your actual EPIA MAC address. If the script completed happily, you can now jump over most of the rest of this page to step 12.
Note -
these steps have been automated into the prepserver script described
above. This section is for documentation only. After
running prepserver, you can skip to section 12
|
| libattr module-init-tools sed zlib dev86 gmp libgcc ncurses setup device-mapper grep libselinux net-tools glibc-common |
hwdata libtermcap iproute lvm2 psmisc udev chkconfig findutils iputils MAKEDEV python usbutils coreutils |
shadow-utils e2fsprogs gzip libsepol openssl sysklogd elfutils hotplug libstdc++ pam SysVinit basesystem elfutils-libelf |
gawk kernel mingetty readline util-linux cpio gdbm krb5-libs mkinitrd rpm wget cracklib glib2 |
tzdata bzip2-libs filesystem less mktemp rpm-libs words cracklib-dicts glibc libacl rpm-python yum db4 |
pcre tar bash ethtool info libxml2 popt termcap beecrypt fedora-release initscripts libxml2-python procps |
To make life easier, I provide in the silentcat tarball a "getfiles" script which takes this list of packages (found in packagelist) and downloads them from a Fedora mirror. To set it going,
cd /usr/src/silentcat-0.2After some time you should have a set of the latest RPMs for the list of packages shown above in the current directory. What we need to do now is to install these RPMs into the ext3 filesystem file we created earlier - NOT on the server itself. First we need to mount our ext3 filesystem file somewhere so we can access the contents. As root...
./getfiles packagelist .
mkdir /tmp/mntThese commands mount our ext3 filesystem file so that it appears at /tmp/mnt. If you do an ls /tmp/mnt you won't see much in there, just a lost+found directory created by the ext3 format earlier. The thing is completely empty at the moment. In fact it is a little too empty, we have to generate a couple of device nodes in there so everything will work smoothly when we come to install the RPMs:
mount -o loop /tftpboot/silentcat/filesystem-$EPIAMAC /tmp/mnt
mkdir /tmp/mnt/devThere is also no /etc/fstab in there, so we must create one. The first line represents the fact that our root filesystem is coming from /dev/nda, the NBD device, and the second one is necessary to get virtual terminals working, eg, xterm, konsole and so on.
mknod /tmp/mnt/dev/null c 1 3
mknod /tmp/mnt/dev/nda b 43 0
echo "/dev/nda / ext3 defaults 0 0" > /tmp/mnt/etc/fstab
echo "none /dev/pts devpts gid=5,mode=620 0 0" >> /tmp/mnt/etc/fstab
We also need to import the Redhat GPG signatures so RPM can recognize good packages
rpm -r /tmp/mnt --import /usr/src/silentcat-0.2/RPM-GPG-KEY
rpm -r /tmp/mnt --import /usr/src/silentcat-0.2/RPM-GPG-KEY-fedora
So our next move is to install the RPMs we just downloaded into this filesystem, which we do with this:
rpm -r /tmp/mnt -i *.rpm
The -r part is VERY important: this tells RPM not to install to your main filesystem as it normally would, but to install into /tmp/mnt instead. This is the difference between installing the RPMs on to the server itself and installing them into our filesystem file. This should complete after a minute or two. Now if you do an ls /tmp/mnt you will see a goodlooking root filesystem in there, with /bin, /lib and so on all generated and populated. Basically we installed a minimal Fedora Core 3 into the filesystem file.
8) Copy out the kernel and initrd and modify the initrd
You will see the Redhat kernel and initrd installed to /tmp/mnt/boot from when the kernel RPM was installed. This is of intense interest to us because we need a copy of the kernel and initrd so we can send it over the network at boot-time. First we will copy the kernel image into /tftpboot/silentcat where it can be accessed by the booting client over TFTP, then we do the same for the initrd:cp /tmp/mnt/boot/vmlinuz* /tftpboot/silentcat/vmlinuz-$EPIAMAC
cp /tmp/mnt/boot/initrd* /tftpboot/silentcat/initrd-$EPIAMAC.gz
The kernel image is ready to use for network boot and needs no fiddling with. You'll notice that the initrd is renamed to end with .gz, this is because current Redhat initrds are actually gzip-compressed cpio archives, and unfortunately that is of more than intellectual interest: because we have to mess with the initrd in order to insert the things we need to complete network boot. Terrifying as the prospect sounds, in fact the mods are fairly lightweight and it is simple to extract and repack the initrd.
cd /tftpboot/silentcat
mkdir unpacked
gunzip initrd-$EPIAMAC.gz
cd unpacked
cpio -i < ../initrd-$EPIAMAC
If you do an ls from inside /tftpboot/silentcat/unpacked you will see the unpacked initrd filesystem spread out before you. This is a bit of a cut down filesystem deliberately created along very minimalist lines. The job of the initrd is to get enough of the system up so that it can replace itself with the real root (ie, mounted at / ) filesystem, usually an ext3 filesystem on a local hard drive. For example, there is no bash interpreter in there, but a specialized and tiny "nash" shell which is superficially bash-like but includes hardly any of the bash features. The file ./init is a nash script that controls the actions of the initrd, and it is this we will have to mess with in a moment.
But first we need to add some kernel modules to the initrd which we need it to have for our purposes. The kernel modules we need are mii.ko and via-rhine.ko to get the Ethernet working, and nbd.ko to get the kernel to understand Network Block Device protocol for our remote hard drive. We can pick these modules out from the ext3 filesystem we prepared earlier with the RPMs (particularly the kernel RPM which provides these modules) and which we still have mounted at /tmp/mnt.
cp /tmp/mnt/lib/modules/*/kernel/drivers/block/nbd.ko /tftpboot/silentcat/unpacked/libIn addition, we need to add two specialized usermode programs into the initrd. First is dhcpcd, the DHCP Client Daemon, which we will use to reassign an IP address to eth0 from the DHCP server. We need to do this because we will be bringing up the Ethernet inside Linux now, which needs the IP setting anew; previously only the pxelinux.0 bootloader had been talking to the DHCP server and got its IP address assigned before Linux was even started. That is lost now Linux has started, and Linux has no idea what IP address to assign to eth0 until we run dhcpcd to contact the DHCP server again. The second specialized program is nbd-client, which does the magic to connect our /dev/nda to the remote NBD server's filesystem file to get us our "network hard drive". We need to copy both of these apps, which have been statically linked (and compiled with i386 arch NOT i686!) from the silentcat archive to /tftpboot/silentcat/unpacked/bin
cp /tmp/mnt/lib/modules/*/kernel/drivers/net/via-rhine.ko /tftpboot/silentcat/unpacked/lib
cp /tmp/mnt/lib/modules/*/kernel/drivers/net/mii.ko /tftpboot/silentcat/unpacked/lib
#!/bin/nash |
cp /usr/src/silentcat-0.2/dhcp*/dhcpcd /tftpboot/silentcat/unpacked/bin
cp /usr/src/silentcat-0.2/nbd*/nbd-client /tftpboot/silentcat/unpacked/bin
Alright, we have all our assets in place inside the unpacked initrd, now we need to edit /tftpboot/silentcat/unpacked/init and get it to do some extra tasks to allow network boot. Here is a copy of my modified init script showing the additions.
The one line you need to modify to match your NBD server IP is /bin/nbd-client 192.168.0.31 1078 /dev/nda, change this to match your NBD server that is exporting the filesystem file. Don't miss the last change,
mount -o defaults --ro -t ext3 /dev/nda /sysroot
which is easy to overlook.9) Repack and recompress the initrd
The changes to the initrd completed, the last step is to regenerate the cpio archive and zip it back upcd /tftpboot/silentcat/unpacked
find . | cpio --quiet -c -o > ../initrd-$EPIAMAC
cd ..
gzip -9 ./initrd-$EPIAMAC
This regenerates ./initrd-$EPIAMAC.gz
10) Final touches to the exported filesystem
/etc/resolv.conf is not set up, that is, DNS will not work. You need to replace 192.168.0.1 below with your or your ISP's DNS server IP addressecho "nameserver 192.168.0.1" > /tmp/mnt/etc/resolv.conf
Lastly, adding the DAG yum repository will be helpful when we populate the system with more packages like mplayer.
echo "[Dag]" >/tmp/mnt/etc/yum.repos.d/dag.repo
echo "baseurl=http://apt.sw.be/fedora/3/en/$basearch/dag" >>/tmp/mnt/etc/yum.repos.d/dag.repo
echo "enabled=1" >>/tmp/mnt/etc/yum.repos.d/dag.repo
echo "gpgcheck=0" >>/tmp/mnt/etc/yum.repos.d/dag.repo
11) Unmount the exported filesystem
We are done fiddling with the exported filesystem on the server. It's important that a filesystem is not mounted multiple times simultaneously, so now we need to unmount it from /tmp/mnt because we are about to try to boot from itumount /tmp/mnt
12) Network Boot time!
Okay, now you should be able to network boot your EPIA board into a runlevel 3 login prompt. Just login as root, no password is needed. Once you are logged in, you can set the root password using passwd.13) Using Yum to add more packages
Now that you are up on the EPIA with a minimal set of packages, you can use yum from the EPIA to download and install the package set needed to make a nice media player. The following are recommended
# hard to edit things without an editor!
yum install vim-minimal
# allows remote login over ssh
yum install openssh-server
chkconfig sshd on
service sshd start
# allows connection to samba servers
yum install samba
# required to get X working (generates /etc/X11/xorg.conf)
yum install system-config-display
# installs X and Konqueror
yum install kdebase
# configure X
system-config-display
# start X
telinit 5
yum install alsa-utils
# set audio levels
alsamixer
alsactl store
yum install mplayer
To boot into a graphical login, you need to edit /etc/inittab and change the line
id:3:initdefault:
to
id:5:initdefault:
Getting Auto-on from the EPIA
By default the EPIA sits there after powerup waiting for the power button to be pressed before it will do anything. You can change this behaviour to automatic startup on powerup simply by adding a 22uF capacitor between pins 6 (+ side of the cap) and 8 (- side of the cap) of the "Front Panel" connector on the EPIA. This behaviour is a lot more "appliance like" and allows the EPIA to be available whenever the TV is powered, for example. We also use the ext3 journalling to allow uncontrolled poweroffs without any problems.FAQ
Is YALD (Yet Another Linux Distro) needed for this?
It's not really another distro, it is a light modification to Fedora Core to allow Network Booting over NBD.Isn't the "network hard drive" slow?
No, its real fast. I can't tell the difference between a local HDD and the network one, except for the silence. In some cases its actually noticeably faster, since the "network hard drive" NBD server has a lot more memory for cache than the EPIA, and the EPIA IDE interfaces are not that great.Do I have to shutdown, or can I just turn the thing off?
In my experience, the ext3 formatted "network hard drive" is fully able to manage having the power just turned off without a controlled shutdown, that's what we have been doing for months without trouble. At the next boot it may have to replay a couple of journal entries which takes a couple of seconds, but we have seen no problems from this.What are the security implications of running this?
The main source of potential danger is that you will be running dhcpd, tftpd and nbd-server on your server machine. Many people are already running dhcpd locally for normal DHCP duties. I suggest that you use iptables on your server machine to limit the accessability of these services to local IP addresses. If you keep your server up to date with security updates are are otherwise secure, it should not present any problems.For the client machine, there is nothing really stopping a local attacker with access to the local network spoofing your EPIA MAC and IP and contaminating the "network hard drive" used by the EPIA. So you should not store really valuable secrets on the "network hard drive". However, if the attacker has control of a machine on your local network already, suffient to change its MAC address and IP, you have worse problems that anything silentcat can cause.
For remote attacks the EPIA, the silentcat distro is really aggressively firewalled, everything is dropped by default from before the network interface is active in Linux. Only the connection to the specific NBD server machine on the specific port is allowed.
How does the EPIA know what to load at boot-time?
The sequence is this:| EPIA
ETHERNET BROADCAST: Hey! Any DHCP servers out there? My MAC
address is 00-40-63-xx-xx-xx... what should I use as my IP? |
|||
| DHCP Server at 192.168.0.50 | Right, I heard
about you from my /etc/dhcpd.conf, you're meant to be 192.168.0.199 |
||
| EPIA 192.168.0.199: Alright, thanks -- does it
say I should TFTP-download anything too? |
|||
| DHCP Server at 192.168.0.50 | Yeah, pxelinux.0, and you
need to use 192.168.0.50 to get it |
||
| EPIA 192.168.0.199: Thanks - okay.... Hey! TFTP
server at 192.168.0.50, Can I get pxelinux.0 please? |
|||
| TFTP Server at 192.168.0.50: | Here you go, sending
my /tftpboot/pxelinux.0 |
||
| EPIA 192.168.0.199: Okay,
running that... alright, TFTP server at 192.168.0.50, now I need a file
called pxelinux.cfg/01-<my MAC ads>? |
|||
| TFTP Server at 192.168.0.50 | Here you go, sending /tftpboot/pxelinux.cfg/01-<your MAC address> | ||
| EPIA 192.168.0.199: Lets
see.... that config file is telling me to get a kernel
silentcat/vmlinuz.... and an initrdfile called silentcat/initrd-tv.gz
you got them? |
|||
| TFTP Server at 192.168.0.50 | Here you go, sending
/tftpboot/silentcat/vmlinuz and /tftpboot/silentcat/initrd-tv.gz |
||
| EPIA 192.168.0.199: (boots
kernel and runs stuff in initrd) - Whoa, I feel much better now - hey
this initrd has a script in it telling me to connect to an NBD "network
hard drive" at 192.168.0.50 port 1077... you there? |
|||
| NBD Server at 192.168.0.50 | Yep, you're connected, I've
been told to serve /tftpboot/silentcat/filesystem-xx-xx-xx-xx-xx-xx on
that port |
||
| EPIA 192.168.0.199:
(mounts the remote filesystem over NBD and starts booting up X, KDE and
so on that is in there) Okay, one more thing, the /etc/fstab in
this network hard drive is telling me to mount a Samba share
from //192.168.0.50/media? |
|
||
| Samba SERVER at 192.168.0.50 | |
Here you go |
|
| EPIA 192.168.0.199: (completes boot and starts
playing movies) Thanks guys! |
|||
- PXE in the EPIA BIOS uses DHCP (the information is in /etc/dhcpd.conf on the DHCP server) to find the right TFTP server to get pxelinux.0 from, given the EPIA MAC address
- pxelinux.0 looks in that TFTP server's ./pxelinux.cfg/<MAC address> to find the right kernel and initrd
- pxelinux.0 fetches the right kernel and initrd from the TFTP server into the EPIA's memory, and boots that kernel with the initrd as the initial root filesystem. This gets us most of the way through the boot process.
Contact me at silentcat@warmcat.com if you have any questions or comments