PXEboot

PXEboot allows you to boot a system from its network card. It loads an operating system from a PXEboot server.

I use it when I want to install a Linux distrubution on a system that does not have a CDROM drive available such as my Via EPIA 800. I set it up to run Debian using this method.

I also use it as a rescue/recovery method for remote servers. Instead of having a support person sit on the phone with me while I give them endlessly detailed instructions spelling out the commands one letter at a time, I can say this: "Please push the reset button on server #4. Thanks!"

What happens when you PXEboot

1. The client starts a loader from the network card boot rom.
2. The boot rom broadcasts a request for network information via DHCP.
3. The DHCP information instructs the boot rom to load and execute a file via TFTP, typically called "pxelinux.0"
4. The pxelinux.0 file instructs the client what to do next; typically to load a file via tftp from pxelinux.cfg/* where * is either a piece of its MAC address or 'default' (which is what I use). It tries the MAC approach first and if it fails to find any matches, it uses 'default'.
5. Typically the 'default' file starts a bootloader (such as syslinux) and gives you a menu. You select an option or hit Enter to accept the default. 'Default' is a text file you can easily customize.
6. Default tells the system which kernel to use, and kernel options. This initiates a standard Linux boot.

Hardware requirements for PXEboot

The system has to support network booting at the BIOS level.

Recent PC's (especially with the network interface on the motherboard) support PXEboot right out of the box. Mamy of computers I work with are not 'recent. They have ethernet on a PCI card. Furthermore, most of the ethernet cards that I own don't have support for PXEboot, either.

3COM cards support PXEboot. Look at yours. Is there a big empty socket? Is there a n empty set of pads where a chip could be soldered on? Most of the cards I have from 3COM fall into one of these two categories. It means that there is no boot rom on board and the board cannot be used for PXEboot. I have one 3COM card that works. In theory you can obtain the chip and plug it in, In practice it's not worth the effort because replacement cards are so cheap.

Everyone is currently upgrading from 100BT to 1000BT so there is a huge supply of cheap perfectly usable 100BT equipment on Ebay.

I bought a lot of 5 Intel EtherPro 100 cards from Ebay for $15 including shipping. The bad news is that it turns out that they don't have flash memory either. The good news is they have PXEboot code built in. The bad news is that these cards have broken firmware. I looked at installing Etherboot but when I loaded up DOS and the Intel IBAutil.exe program I discovered these cards don't have flash memory. The final good news is that there is a workaround for them. All I had to do was apply a couple fixes from the PXElinux hardware page.

Fixes for Intel Pro/100 cards

The fixes are

1. add this line "net/ipv4/ip_no_pmtu_disc=1" to /etc/sysctl.conf
2. add "-r blksize" to OPTIONS in /etc/default/tftpd-hpa

Now my Intel cards are happily PXEbooting my RIP image.

BIOS Setup

Some systems have only a boot option called "LAN" or "Network". One of my systems (Intel SE440BX-2) probes for the boot rom on any network cards and they show up as options like "LANDesk (R)"... for an Intel card or "UNDI"... for a 3COM.

Put the appropriate boot option at the top of the list so that PXEboot takes precedence over the hard drive and CDROM drive.

On my ancient testbed system, each time I swap in a new network card the network boot option drops to the end of the list! Boo! Keep in mind that a network card replacement might require a trip into BIOS on your system.

Put a separate setting for each client system into dhcpd.conf. Enable PXEboot in DHCP for the client when you need it to boot from PXE and disable when you need to boot from hard drive or CDROM. When the system comes up, it will try PXEboot and then continue on. This makes booting take a few second longer, but gives you the option to get it to boot even when it is headless or remotely located.

This is what I see at boot time.

Intel LANDesk (R) Service Agent, version 0.99b
Copyright (C) 1997 Intel Corporation. All rights reserved.

DHCP MAC ADDR: 00 08 C7 07 11 2F
PXE-EA1: No PXE server found, using standard boot file.
IP ADDR: 192.168.123.248
TFTP...

An alternative: Etherboot

http://www.etherboot.org

The Intel cards aren't PXEbooting, so now I have to reflash them.

Installing etherboot in Intel Pro/100 cards

PXEboot server on Debian

Install and configure these packages

dhcp server: dhcp3-server

tftp server: tftpd or tftpd-hpa
The version in the tftpd package runs only from inetd, so configure it in /etc/inetd.conf The tftpd-hpa version runs either as inetd or as a standalone daemon. I use the former as I have had problems with tftpd-hpa serving a file only once and then refusing connections.

tftp client: tftp-hpa useful for testing the server

Configuration

TFTP

I run tftpd from inetd. After installing it, you must uncomment the appropriate line in /etc/inetd.conf, and then restart inetd so that tftpd becomes available. My entry looks like this:

tftp           dgram   udp     wait    root  /usr/sbin/in.tftpd /usr/sbin/in.tftpd -s /home/diskless/tftpboot

DHCP

Generally I put a configuration for each machine into /etc/dhcp3/dhcpd.conf so that I have control over what happens when a given machine connects. The critical lines to add to dhcpd.conf are

host clientname {
  hardware ethernet mac-address;
  fixed-address ip address of your choosing;

  # Uncomment next two lines to boot from network.
  next-server 192.168.123.130; # Ip address of TFTP server
  filename "pxelinux.0"; # name of file to load from TFTP server
}

This tells the client what file to grab via tftp. The files for tftp should be installed in /var/lib/tftpboot (see comment above on symlinks)

NFS

The disk image of the rescue disk has to live somewhere. I keep it in the /var/lib/tftpboot directory. A line in /etc/exports makes the directory available:

/var/lib/tftpboot       *(no_root_squash,no_subtree_check)

I have a feeling I could export the image read-only but I have not tried it. And of course you have to have your NFS server running.

Network boot installer

Many Linux distributions (including Redhat, Debian, Trustix, Ubuntu, and SuSe) have network install support. You just have to poke around to find the files.

For example, to install Debian on the client, you poke around in the Debian FTP archives until you find netboot.tgz. Download it and unpack it in your /var/lib/tftpboot directory. Then make symlinks from /var/lib/tftpboot/pxelinux.0 and pxelinux.cfg to the real ones lurk in the debian_installer tree. That's it.

When you boot a client, it will get an IP address from the DHCP server and find the other TFTP settings. Then it will use those settings to download the pxelinux.0 file and run it. That file will tell the client to get and load a boot loader which in turn will load and start Linux on the client.

Client setup

Set the client to boot from LAN or Network. Then boot it. If you are near the client and can look at the console,

• you should see the "UNDI" boot stuff. Notably you should see the client get the IP address that you set up for it in dhcpd.conf.
• next you should see it connect to the tftp server. It will show a series lines as it tries to find a pxelinux.cfg file; it will uses its MAC address piece by piece and then give up and load pxelinux.cfg/default
• if the pxelinux.cfg loads then it should fire up the installer and from then on, the screens are what you'd see if you booted from a cdrom.

Serial PXElinux

To control kernel parameters, edit PXElinux.cfg/default (or other files therein if you have set up files for individual machines based on MAC numbers.)

Refer to the [PXElinux] and [SYSlinux] documentation.

System Rescue mode

But wait! I want to do a remote bare metal rescue mode!

Currently I have to instruct someone to push the "magic cd" into the computer and reboot it. This taxes everyone's patience because later I have to call them and ask them to remove it and stand by to see if the system will reboot correctly. The "magic cd" is a customized version of SystemRescueCD with ssh enabled on it and a root password of my choosing stored in it.

The SystemRescueCD project has died by the way. So it's only a matter of time until we cannot use the magic disk anymore. Look at FrozenTech for alternatives.

Olive looks like fun.

I want to PXEboot into a system rescue tool kit and I want the result to have a serial console. I just want to ask someone to push the reset button (or automate that too someday) and have the system come up. In theory I can just ssh in once the system pxeboots; I know it has a functional network card if it pxebooted, right? But I want it anyway. I am not guaranteed that the operating system will be able to find the network card after the kernel loads, now am I?

Then I can control whether it boots from network or not by modifying my dhcp server settings. If I uncomment the 'deny booting' line it netboots, otherwise it boots from its own hard drive. If this works well enough I might quit having anything boot from a hard drive ever again.

Which distribution?

I'd sort of like a very minimal Debian-based installation so that it is small but so that I can easily install relevant packages. I'd like to be able to scale it from a rescue/ramdisk version up to a RAID mirror full server installation.

FloppyFW looks interesting. It is a single floppy firewall router.

"RIP" = "Recovery is Possible" is a good rescue image. Appears to have what I need for this application. The only change I'd like to make is to enable the console on the serial port. RIP Notes

"Linux from Scratch" I had the idea I could use the LiveCD image but starting with the images LiveCD does not work because after booting the system looks for the CD.

"Damn Small Linux" is another Live CD in the Knoppix tradition: it comes up in a graphical environment.

Trinity Rescue Kit

Lately I have been using the Trinity Rescue Kit with my Magnia SG 20, Otter.

Directions on how to enabled ssh login are described here: http://trinityhome.org/Home/index.php?wpid=33&front_id=12

Basically you create a directory and put an ssh public key into it. When the system boots it sees the file and starts its ssh server.

More information

General information on PXEboot for Linux

Remote Network Boot via PXE

Installing Debian via PXE

H. Peter Anvin's PXELINUX

Related systems

LinBox is a complete system for backup and bare metal restores of systems

Damn Small Linux is based on Knoppix and can be upgraded to Debian

[1] aka "Recovery Is Possible" is based on Slackware. There are CD and PXE versions. It uses the latest kernel.

Linux From Scratch can be thought of as a book on how to create a Linux system from scratch (as opposed to one of the 350 distributions out there.) But you can download the 40 MB bootable CDROM image, which has all the source code on it, as a starting point.