Latest revision as of 17:26, 30 July 2011

IPMI is a tool used for remote management of servers. It lets us do many tasks without physically going to the server to mess with it.

Basics

IPMI is a standard interface for remote management. It works by having a small computer (the Baseboard Management Controller or "BMC") listen on a network port, independent of the operating system.

We connect our IPMI network ports to a separate switch, and use green ethernet cables so we know which cables belong to IPMI. Our IPMI IP address are of the form 10.0.1.#, where # is the number in 131.215.176.# that the server normally is.

Linux has OpenIPMI, which supports both doing local ipmi operations as well as ones over the lan. To do stuff over the lan, log in to enlil. You will find the password for IPMI in enlil:/etc/ipmi_pwd

Specifics

Poweredge 860's

The BMC in the poweredge listens on NIC 1 ONLY. Make sure that the green IPMI network cable is in NIC 1 (they are labeled), and that you re-configure "/etc/network/interfaces" as necessary.

Serial console

In BIOS, you need to set a few options to get serial console working right:

• Integrated Devices -> Serial Port = BMC NIC
• Console Redirection
  • Enable after boot -> False (we use stuff from grub, kernel, inittab to do it after the initial boot. Redirecting after bios works only up until the kernel starts booting)
  • Speed -> 19200

Athena

Athena has a separate IPMI card that fits in a management card slot. It is the extra ethernet port. You cannot use ipmitool to configure it- you need to use Supermirco's own thing- see /usr/local/sbin/ipmi[cli,cfg]

Serial console

We use serial console so we have access to machines even if their network is dead. See http://tldp.org/HOWTO/Remote-Serial-Console-HOWTO for a quick how-to. Basics:

• In menu.lst, add

serial --unit=0 --speed=19200 --parity=no --stop=1
terminal --timeout=5 serial console

• Also, add the following to the kopt variable (you then have to re-run update-grub)

console=tty0 console=ttyS0,19200n8

• And finally, uncomment the following line in /etc/inittab (you can run "telinit q" to make the file be re-read immediately). You will have to set the speed to 19200

TO:2345:respawn:/sbin/getty -L ttyS0 19200 vt100

SOLProxy

Since our dells don't do IPMI 2.0, we use a proprietary Dell extension to IPMI to do serial console. Dell implements this via a "serial-over-lan proxy" daemon, dsm_bmu_solproxy32d (installed on enlil). If it's not running, you can start it with

dsm_bmu_solproxy32d -daemon -b

You can then connect to it by running

telnet localhost 623

This brings up its main screen. The rest should be pretty self-explanatory. Keep in mind that you need to use escape sequences to send F# keys and other keys. See http://support.dell.com/support/edocs/software/smbmcmu/2.0A01/en/ug/bmcugacc.htm#wp1053626 F# keys can be accessed by <ESC> + # (use ! for 11, @ for 12, etc). So F2 would be <ESC>+2

The poweredge 860's support a maximum speed of 19200 baud. I know it's slow, sorry.