IPMI
IPMI is a tool used for remote management of servers. It lets us do many tasks without physically going to the server to mess with it.
Contents |
Basics
IPMI is a standard interface for remote management. It works by having a small computer (the Baseboard Management Controller or "BMC") listen on a network port, independent of the operating system.
We connect our IPMI network ports to a separate switch, and use green ethernet cables so we know which cables belong to IPMI. Our IPMI IP address are of the form 10.0.1.#, where # is the number in 131.215.176.# that the server normally is.
Linux has OpenIPMI, which supports both doing local ipmi operations as well as ones over the lan. To do stuff over the lan, log in to enlil. You will find the password for IPMI in enlil:/etc/ipmi_pwd
Specifics
Poweredge 860's
The BMC in the poweredge listens on NIC 1 ONLY. Make sure that the green IPMI network cable is in NIC 1 (they are labeled), and that you re-configure "/etc/network/interfaces" as necessary.
Serial console
In BIOS, you need to set a few options to get serial console working right:
- Integrated Devices -> Serial Port = BMC NIC
- Console Redirection
- Enable after boot -> False (we use stuff from grub, kernel, inittab to do it after the initial boot. Redirecting after bios works only up until the kernel starts booting)
- Speed -> 19200
Athena
Athena has a separate IPMI card that fits in a management card slot. It is the extra ethernet port. You cannot use ipmitool to configure it- you need to use Supermirco's own thing- see /usr/local/sbin/ipmi[cli,cfg]
Serial console
We use serial console so we have access to machines even if their network is dead. See http://tldp.org/HOWTO/Remote-Serial-Console-HOWTO for a quick how-to. Basics:
- In menu.lst, add
serial --unit=0 --speed=19200 --parity=no --stop=1 terminal --timeout=5 serial console
- Also, add the following to the kopt variable (you then have to re-run update-grub)
console=tty0 console=ttyS0,19200n8
- And finally, uncomment the following line in /etc/inittab (you can run "telinit q" to make the file be re-read immediately). You will have to set the speed to 19200
TO:2345:respawn:/sbin/getty -L ttyS0 19200 vt100
SOLProxy
Since our dells don't do IPMI 2.0, we use a proprietary Dell extension to IPMI to do serial console. Dell implements this via a "serial-over-lan proxy" daemon, dsm_bmu_solproxy32d (installed on enlil). If it's not running, you can start it with
dsm_bmu_solproxy32d -daemon -b
You can then connect to it by running
telnet localhost 623
This brings up its main screen. The rest should be pretty self-explanatory. Keep in mind that you need to use escape sequences to send F# keys and other keys. See http://support.dell.com/support/edocs/software/smbmcmu/2.0A01/en/ug/bmcugacc.htm#wp1053626 F# keys can be accessed by <ESC> + # (use ! for 11, @ for 12, etc). So F2 would be <ESC>+2
The poweredge 860's support a maximum speed of 19200 baud. I know it's slow, sorry.
