PXEboot: Difference between revisions

From Wildsong
Jump to navigationJump to search
Brian Wilson (talk | contribs)
mNo edit summary
 
Brian Wilson (talk | contribs)
 
(42 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Using PXEboot with Debian ==
[[Category: Toshiba Magnia]]


# Client requests network information from BOOTP / DHCP server
PXEboot allows you to boot a system from its network card. It loads an operating system from a PXEboot server.
# Client requests kernel via TFTP


== On the Debian server ==
I use it when I want to install a Linux distrubution on a system that does not have a CDROM drive available such as my [[Via EPIA 800]]. I set it up to run Debian using this method.


Install these packages
I also use it as a rescue/recovery method for remote servers.
Instead of having a support person sit on the phone with me while I give
them endlessly detailed instructions spelling out the commands one letter
at a time, I can say this: "Please push the reset button on server #4. Thanks!"
 
== What happens when you PXEboot ==
 
# The client starts a loader from the network card boot rom.
# The boot rom broadcasts a request for network information via DHCP.
# The DHCP information instructs the boot rom to load and execute a file via TFTP, typically called "pxelinux.0"
# The pxelinux.0 file instructs the client what to do next; typically to load a file via tftp from pxelinux.cfg/* where * is either a piece of its MAC address or 'default' (which is what I use). It tries the MAC approach first and if it fails to find any matches, it uses 'default'.
# Typically the 'default' file starts a bootloader (such as syslinux) and gives you a menu. You select an option or hit Enter to accept the default. 'Default' is a text file you can easily customize.
# Default tells the system which kernel to use, and kernel options. This initiates a standard Linux boot.
 
== Hardware requirements for PXEboot ==
 
The system has to support network booting at the BIOS level.
 
Recent PC's (especially with the network interface on the motherboard) support
PXEboot right out of the box. Mamy of computers I work with are not 'recent. They have ethernet on a PCI card. Furthermore, most of the ethernet cards that I own don't have support for PXEboot, either.
 
3COM cards support PXEboot. Look at yours. Is there a big empty socket? Is there a n empty set of pads where a chip could be soldered on? Most of the cards I have from 3COM fall into one of these two categories. It means that there is no boot rom on board and the board cannot be used for PXEboot. I have one 3COM card that works. In theory you can obtain the chip and plug it in, In practice it's not worth the effort because replacement cards are so cheap.
 
Everyone is currently upgrading from 100BT to 1000BT so there is a huge supply of cheap perfectly usable 100BT equipment on Ebay.
 
I bought a lot of 5 Intel EtherPro 100 cards from Ebay for $15 including shipping. The bad news is that it turns out that they don't have flash memory either. The good news is they have PXEboot code built in. The bad news is that
these cards have broken firmware. I looked at installing [[Etherboot]] but when I loaded up DOS and the Intel IBAutil.exe program I discovered these cards don't have flash memory. The final good news is that there is a workaround for them. All I had to do was apply a couple fixes from the [http://syslinux.zytor.com/hardware.php PXElinux hardware page.]
 
=== Fixes for Intel Pro/100 cards ===
 
The fixes are
 
# add this line "net/ipv4/ip_no_pmtu_disc=1" to /etc/sysctl.conf
# add "-r blksize" to OPTIONS in /etc/default/tftpd-hpa
 
Now my Intel cards are happily PXEbooting my RIP image.
 
== BIOS Setup ==
 
Some systems have only a boot option called "LAN" or "Network".
One of my systems (Intel SE440BX-2) probes for the boot rom on any network cards and they show up as options like "LANDesk (R)"... for an Intel card or "UNDI"... for a 3COM.
 
Put the appropriate boot option at the top of the list so that PXEboot takes precedence over the hard drive and CDROM drive.
 
On my ancient testbed system, each time I swap in a new network card the network boot option drops to the end of the list! Boo! Keep in mind that a network card replacement might require a trip into BIOS on your system.
 
Put a separate setting for each client system into dhcpd.conf. Enable PXEboot in DHCP for the client when you need it to boot from PXE and disable when you need to boot from hard drive or CDROM. When the system comes up, it will try PXEboot and then continue on. This makes booting take a few second longer, but gives you the option to get it to boot even when it is headless or remotely located.
 
This is what I see at boot time.
 
Intel LANDesk (R) Service Agent, version 0.99b
Copyright (C) 1997 Intel Corporation. All rights reserved.
 
DHCP MAC ADDR: 00 08 C7 07 11 2F
PXE-EA1: No PXE server found, using standard boot file.
IP ADDR: 192.168.123.248
TFTP...
 
== An alternative: Etherboot ==
 
http://www.etherboot.org
 
 
The Intel cards aren't PXEbooting, so now I have to reflash them.
 
[[Installing etherboot in Intel Pro/100 cards]]
 
== PXEboot server on Debian ==
 
Install and configure these packages


dhcp server:  '''dhcp3-server'''
dhcp server:  '''dhcp3-server'''


tftp server:  '''tftpd-hpa'''<br>
tftp server:  '''tftpd''' or '''tftpd-hpa'''<br>
When the installer asks "Should it be started from inet?", say 'yes'
The version in the tftpd package runs only from inetd, so configure it in ''/etc/inetd.conf'' The tftpd-hpa version runs either as inetd or as a standalone daemon. I use the former as I have had problems with tftpd-hpa serving a file only once and then refusing connections.


tftp client: (useful for testing the server) '''tftp-hpa'''
tftp client: '''tftp-hpa''' useful for testing the server


=== Configuration ===
=== Configuration ===
==== TFTP ====
I run tftpd from inetd. After installing it, you must uncomment the appropriate line in /etc/inetd.conf, and then restart inetd so that tftpd becomes available.
My entry looks like this:
tftp          dgram  udp    wait    root  /usr/sbin/in.tftpd /usr/sbin/in.tftpd -s /home/diskless/tftpboot
==== DHCP ====


Generally I put a configuration for each machine into /etc/dhcp3/dhcpd.conf
Generally I put a configuration for each machine into /etc/dhcp3/dhcpd.conf
Line 23: Line 100:
  host ''clientname'' {
  host ''clientname'' {
   hardware ethernet ''mac-address'';
   hardware ethernet ''mac-address'';
   fixed-address ''ip address of your chosing'';
   fixed-address ''ip address of your choosing'';
   filename "pxelinux.0";
 
  # Uncomment next two lines to boot from network.
  next-server 192.168.123.130; # Ip address of TFTP server
   filename "pxelinux.0"; # name of file to load from TFTP server
  }
  }


This tells the client what file to grab via tftp.
This tells the client what file to grab via tftp.
The files for tftp live in '''/var/lib/tftpboot'''.
The files for tftp should be installed in '''/var/lib/tftpboot''' (see comment above on symlinks)
 
==== NFS ====
 
The disk image of the rescue disk has to live somewhere.
I keep it in the /var/lib/tftpboot directory.
A line in /etc/exports makes the directory available:
 
/var/lib/tftpboot      *(no_root_squash,no_subtree_check)
 
I have a feeling I could export the image read-only but I have not tried it.
And of course you have to have your NFS server running.
 
=== Network boot installer ===
 
Many Linux distributions (including Redhat, Debian, Trustix, Ubuntu, and SuSe) have network install support. You just have to poke around to find the files.
 
For example, to install Debian on the client, you poke around in the Debian FTP archives until you find netboot.tgz. Download it and unpack it in your /var/lib/tftpboot directory. Then make symlinks from /var/lib/tftpboot/pxelinux.0 and pxelinux.cfg to the real ones lurk in the debian_installer tree. That's it.
 
When you boot a client, it will get an IP address from the DHCP server and find the  other TFTP settings. Then it will use those settings to download the pxelinux.0 file and run it. That file will tell the client to get and load a boot loader which in turn will load and start Linux on the client.
 
== Client setup ==
 
Set the client to boot from LAN or Network. Then boot it. If you are near the client and can look at the console,
* you should see the "UNDI" boot stuff. Notably you should see the client get the IP address that you set up for it in dhcpd.conf.
* next you should see it connect to the tftp server. It will show a series lines as it tries to find a pxelinux.cfg file; it will uses its MAC address piece by piece and then give up and load pxelinux.cfg/default
* if the pxelinux.cfg loads then it should fire up the installer and from then on, the screens are what you'd see if you booted from a cdrom.
 
== Serial PXElinux ==
 
To control kernel parameters, edit PXElinux.cfg/default (or other files therein if you have set up files for individual machines based on MAC numbers.)
 
Refer to the [PXElinux] and [SYSlinux] documentation.
 
== System Rescue mode ==
 
But wait! I want to do a remote bare metal rescue mode!
 
Currently I have to instruct someone to push the "magic cd" into the
computer and reboot it. This taxes everyone's patience because later
I have to call them and ask them to remove it and stand by to see if
the system will reboot correctly. The "magic cd" is a customized version of
[http://www.sysresccd.org/Main_Page SystemRescueCD] with ssh enabled on it
and a root password of my choosing stored in it.
 
The [http://www.sysresccd.org/forums/viewtopic.php?t=719 SystemRescueCD project has died] by the way. So it's only a matter of time until we cannot use the magic disk anymore. Look at [http://www.frozentech.com/content/livecd.php FrozenTech] for alternatives.
 
[http://web.isteve.bofh.cz/olive/ Olive] looks like fun.
 
I want to PXEboot into a system rescue tool kit and I want the result to have a serial console. I just want to ask someone to push the reset button (or automate that too someday) and have the system come up. In theory I can just ssh in once the system pxeboots; I know it has a functional network card if it pxebooted, right? But I want it anyway. I am not guaranteed that the operating system will be able to find the network card after the kernel loads, now am I?
 
Then I can control whether it boots from network or not by modifying my dhcp server settings. If I uncomment the 'deny booting' line it netboots, otherwise it boots from its own hard drive. If this works well enough I might quit having anything boot from a hard drive ever again.
 
=== Which distribution? ===
 
I'd sort of like a very minimal Debian-based installation so that it is small but so that I can easily install relevant packages. I'd like to be able to scale it from a rescue/ramdisk version up to a RAID mirror full server installation.
 
[http://www.zelow.no/floppyfw/ FloppyFW] looks interesting. It is a single floppy firewall router.
 
[http://www.tux.org/pub/people/kent-robotti/looplinux/rip/ "RIP"] = "Recovery is Possible" is a good rescue image. Appears to have what I need for this application. The only change I'd like to make is to enable the console on the serial port. [[RIP Notes]]


Many Linux distributions have network install support. You just have to poke around to find the files.  
[http://linuxfromscratch.org "Linux from Scratch"] I had the idea I could use the LiveCD image but starting with the images LiveCD does not work because after booting the system looks for the CD.


== Booting the client ==
[http://damnsmalllinux.org "Damn Small Linux"] is another Live CD in the Knoppix tradition: it comes up in a graphical environment.


Set the client to boot from LAN or Network. Then boot it.
=== Trinity Rescue Kit ===
* You should see the "UNDI" boot stuff. Notably you should see the client
get the IP address that you set up for it in dhcpd.conf.
* You should see it connect to the tftp server.


=== Customizing for serial console ===
Lately I have been using the [http://trinityhome.org/ Trinity Rescue Kit] with my Magnia SG 20, [[Otter]].


Directions on how to enabled ssh login are described here:
http://trinityhome.org/Home/index.php?wpid=33&front_id=12


Basically you create a directory and put an ssh public key into it. When the system boots it sees the file and starts its ssh server.


== More information ==
== More information ==


[http://www.kegel.com/linux/pxe.html Remote Network Boot via PXE]<br>
=== General information on PXEboot for Linux ===
 
[http://www.kegel.com/linux/pxe.html Remote Network Boot via PXE]
 
[http://www.debianplanet.com/node.php?id=818 Installing Debian via PXE]
[http://www.debianplanet.com/node.php?id=818 Installing Debian via PXE]
H. Peter Anvin's [http://syslinux.zytor.com/pxe.php PXELINUX]
=== Related systems ===
[http://www.linbox.com/en/ppart.html LinBox] is a complete system for backup and bare metal restores of systems <br>
[http://damnsmalllinux.org/ Damn Small Linux] is based on [http://www.knoppix.org Knoppix] and can be upgraded to [http://debian.org Debian]
[http://www.tux.org/pub/people/kent-robotti/looplinux/rip/RIP] aka "Recovery Is Possible" is based on [http://www.slackware.org/ Slackware]. There are CD and PXE versions. It uses the latest kernel.
[http://linuxfromscratch.org/ Linux From Scratch] can be thought of as a book on how to create a Linux system from scratch (as opposed to one of the 350 distributions out there.) But you can download the 40 MB bootable CDROM image, which has all the source code on it, as a starting point.

Latest revision as of 05:23, 12 August 2010


PXEboot allows you to boot a system from its network card. It loads an operating system from a PXEboot server.

I use it when I want to install a Linux distrubution on a system that does not have a CDROM drive available such as my Via EPIA 800. I set it up to run Debian using this method.

I also use it as a rescue/recovery method for remote servers. Instead of having a support person sit on the phone with me while I give them endlessly detailed instructions spelling out the commands one letter at a time, I can say this: "Please push the reset button on server #4. Thanks!"

What happens when you PXEboot

  1. The client starts a loader from the network card boot rom.
  2. The boot rom broadcasts a request for network information via DHCP.
  3. The DHCP information instructs the boot rom to load and execute a file via TFTP, typically called "pxelinux.0"
  4. The pxelinux.0 file instructs the client what to do next; typically to load a file via tftp from pxelinux.cfg/* where * is either a piece of its MAC address or 'default' (which is what I use). It tries the MAC approach first and if it fails to find any matches, it uses 'default'.
  5. Typically the 'default' file starts a bootloader (such as syslinux) and gives you a menu. You select an option or hit Enter to accept the default. 'Default' is a text file you can easily customize.
  6. Default tells the system which kernel to use, and kernel options. This initiates a standard Linux boot.

Hardware requirements for PXEboot

The system has to support network booting at the BIOS level.

Recent PC's (especially with the network interface on the motherboard) support PXEboot right out of the box. Mamy of computers I work with are not 'recent. They have ethernet on a PCI card. Furthermore, most of the ethernet cards that I own don't have support for PXEboot, either.

3COM cards support PXEboot. Look at yours. Is there a big empty socket? Is there a n empty set of pads where a chip could be soldered on? Most of the cards I have from 3COM fall into one of these two categories. It means that there is no boot rom on board and the board cannot be used for PXEboot. I have one 3COM card that works. In theory you can obtain the chip and plug it in, In practice it's not worth the effort because replacement cards are so cheap.

Everyone is currently upgrading from 100BT to 1000BT so there is a huge supply of cheap perfectly usable 100BT equipment on Ebay.

I bought a lot of 5 Intel EtherPro 100 cards from Ebay for $15 including shipping. The bad news is that it turns out that they don't have flash memory either. The good news is they have PXEboot code built in. The bad news is that these cards have broken firmware. I looked at installing Etherboot but when I loaded up DOS and the Intel IBAutil.exe program I discovered these cards don't have flash memory. The final good news is that there is a workaround for them. All I had to do was apply a couple fixes from the PXElinux hardware page.

Fixes for Intel Pro/100 cards

The fixes are

  1. add this line "net/ipv4/ip_no_pmtu_disc=1" to /etc/sysctl.conf
  2. add "-r blksize" to OPTIONS in /etc/default/tftpd-hpa

Now my Intel cards are happily PXEbooting my RIP image.

BIOS Setup

Some systems have only a boot option called "LAN" or "Network". One of my systems (Intel SE440BX-2) probes for the boot rom on any network cards and they show up as options like "LANDesk (R)"... for an Intel card or "UNDI"... for a 3COM.

Put the appropriate boot option at the top of the list so that PXEboot takes precedence over the hard drive and CDROM drive.

On my ancient testbed system, each time I swap in a new network card the network boot option drops to the end of the list! Boo! Keep in mind that a network card replacement might require a trip into BIOS on your system.

Put a separate setting for each client system into dhcpd.conf. Enable PXEboot in DHCP for the client when you need it to boot from PXE and disable when you need to boot from hard drive or CDROM. When the system comes up, it will try PXEboot and then continue on. This makes booting take a few second longer, but gives you the option to get it to boot even when it is headless or remotely located.

This is what I see at boot time.

Intel LANDesk (R) Service Agent, version 0.99b
Copyright (C) 1997 Intel Corporation. All rights reserved.
DHCP MAC ADDR: 00 08 C7 07 11 2F
PXE-EA1: No PXE server found, using standard boot file.
IP ADDR: 192.168.123.248
TFTP...

An alternative: Etherboot

http://www.etherboot.org


The Intel cards aren't PXEbooting, so now I have to reflash them.

Installing etherboot in Intel Pro/100 cards

PXEboot server on Debian

Install and configure these packages

dhcp server: dhcp3-server

tftp server: tftpd or tftpd-hpa
The version in the tftpd package runs only from inetd, so configure it in /etc/inetd.conf The tftpd-hpa version runs either as inetd or as a standalone daemon. I use the former as I have had problems with tftpd-hpa serving a file only once and then refusing connections.

tftp client: tftp-hpa useful for testing the server

Configuration

TFTP

I run tftpd from inetd. After installing it, you must uncomment the appropriate line in /etc/inetd.conf, and then restart inetd so that tftpd becomes available. My entry looks like this:

tftp           dgram   udp     wait    root  /usr/sbin/in.tftpd /usr/sbin/in.tftpd -s /home/diskless/tftpboot

DHCP

Generally I put a configuration for each machine into /etc/dhcp3/dhcpd.conf so that I have control over what happens when a given machine connects. The critical lines to add to dhcpd.conf are

host clientname {
  hardware ethernet mac-address;
  fixed-address ip address of your choosing;
  # Uncomment next two lines to boot from network.
  next-server 192.168.123.130; # Ip address of TFTP server
  filename "pxelinux.0"; # name of file to load from TFTP server
}

This tells the client what file to grab via tftp. The files for tftp should be installed in /var/lib/tftpboot (see comment above on symlinks)

NFS

The disk image of the rescue disk has to live somewhere. I keep it in the /var/lib/tftpboot directory. A line in /etc/exports makes the directory available:

/var/lib/tftpboot       *(no_root_squash,no_subtree_check)

I have a feeling I could export the image read-only but I have not tried it. And of course you have to have your NFS server running.

Network boot installer

Many Linux distributions (including Redhat, Debian, Trustix, Ubuntu, and SuSe) have network install support. You just have to poke around to find the files.

For example, to install Debian on the client, you poke around in the Debian FTP archives until you find netboot.tgz. Download it and unpack it in your /var/lib/tftpboot directory. Then make symlinks from /var/lib/tftpboot/pxelinux.0 and pxelinux.cfg to the real ones lurk in the debian_installer tree. That's it.

When you boot a client, it will get an IP address from the DHCP server and find the other TFTP settings. Then it will use those settings to download the pxelinux.0 file and run it. That file will tell the client to get and load a boot loader which in turn will load and start Linux on the client.

Client setup

Set the client to boot from LAN or Network. Then boot it. If you are near the client and can look at the console,

  • you should see the "UNDI" boot stuff. Notably you should see the client get the IP address that you set up for it in dhcpd.conf.
  • next you should see it connect to the tftp server. It will show a series lines as it tries to find a pxelinux.cfg file; it will uses its MAC address piece by piece and then give up and load pxelinux.cfg/default
  • if the pxelinux.cfg loads then it should fire up the installer and from then on, the screens are what you'd see if you booted from a cdrom.

Serial PXElinux

To control kernel parameters, edit PXElinux.cfg/default (or other files therein if you have set up files for individual machines based on MAC numbers.)

Refer to the [PXElinux] and [SYSlinux] documentation.

System Rescue mode

But wait! I want to do a remote bare metal rescue mode!

Currently I have to instruct someone to push the "magic cd" into the computer and reboot it. This taxes everyone's patience because later I have to call them and ask them to remove it and stand by to see if the system will reboot correctly. The "magic cd" is a customized version of SystemRescueCD with ssh enabled on it and a root password of my choosing stored in it.

The SystemRescueCD project has died by the way. So it's only a matter of time until we cannot use the magic disk anymore. Look at FrozenTech for alternatives.

Olive looks like fun.

I want to PXEboot into a system rescue tool kit and I want the result to have a serial console. I just want to ask someone to push the reset button (or automate that too someday) and have the system come up. In theory I can just ssh in once the system pxeboots; I know it has a functional network card if it pxebooted, right? But I want it anyway. I am not guaranteed that the operating system will be able to find the network card after the kernel loads, now am I?

Then I can control whether it boots from network or not by modifying my dhcp server settings. If I uncomment the 'deny booting' line it netboots, otherwise it boots from its own hard drive. If this works well enough I might quit having anything boot from a hard drive ever again.

Which distribution?

I'd sort of like a very minimal Debian-based installation so that it is small but so that I can easily install relevant packages. I'd like to be able to scale it from a rescue/ramdisk version up to a RAID mirror full server installation.

FloppyFW looks interesting. It is a single floppy firewall router.

"RIP" = "Recovery is Possible" is a good rescue image. Appears to have what I need for this application. The only change I'd like to make is to enable the console on the serial port. RIP Notes

"Linux from Scratch" I had the idea I could use the LiveCD image but starting with the images LiveCD does not work because after booting the system looks for the CD.

"Damn Small Linux" is another Live CD in the Knoppix tradition: it comes up in a graphical environment.

Trinity Rescue Kit

Lately I have been using the Trinity Rescue Kit with my Magnia SG 20, Otter.

Directions on how to enabled ssh login are described here: http://trinityhome.org/Home/index.php?wpid=33&front_id=12

Basically you create a directory and put an ssh public key into it. When the system boots it sees the file and starts its ssh server.

More information

General information on PXEboot for Linux

Remote Network Boot via PXE

Installing Debian via PXE

H. Peter Anvin's PXELINUX

Related systems

LinBox is a complete system for backup and bare metal restores of systems

Damn Small Linux is based on Knoppix and can be upgraded to Debian

[1] aka "Recovery Is Possible" is based on Slackware. There are CD and PXE versions. It uses the latest kernel.

Linux From Scratch can be thought of as a book on how to create a Linux system from scratch (as opposed to one of the 350 distributions out there.) But you can download the 40 MB bootable CDROM image, which has all the source code on it, as a starting point.