How to do remote server administration over IPMI?

Knowledge Base Supported
, , , , , ,
#1

Many modern servers provide a “baseband management controller” (BMC), which is a small embedded device which allows to manage the server even when it is turned off. It also provides sensor data (temparture, fan speed, etc) and inventory data (serial numbers) of “field replaceable units” (FRU).
Most have an LAN interface, which allows to administer the server from remote. This is often shared with the main network interface, so care must be taken to disable “Intelligent Platform Management Interface” (IPMI) when that interface is connected to a public network, as any user can read vital data. Best is to use a dedicated private network for administration only.How to do remote server administration over IPMI
You need to collect the following data:

  • IP address to assign to the IPMI interface on the host ($LANIP)
  • IP network mask ($LANNET)
  • IP address of the gateway ($LANGW)
  • MAC address of the gateway ($GWMAC)

Concepts

  • There are multiple paths to the BMC: Via some internal interface on the motherborad (KCS,SMIC,BT,SSIF), serial interface, LAN interface
  • Multiple users can be configured, which are allowed to receive events only (callback), query data (user), perform operations (operator), or administer all setting (admin).
  • channels connect users via paths the the BMC. This allows to configure different permissions based on the path used.

Setup

1. Login as root.

2. Install “ipmitools”:

ucr set repository/online/unmaintained=yes
univention-install ipmitool

3. Load the Linux kernel modules to access IPMI locally:

modprobe ipmi_devintf
modprobe ipmi_si
ucr set kernel/modules="$(ucr get kernel/modules);ipmi_devintf;ipmi_si"

4. Configure the LAN interface.

Normally this is channel 1 or 2, but other channels(0-15) may be also available. Channel 15 is the system internal channel, while channel 14 represents the channel itself which is currently used when a command runs. Run

ipmitool channel info $CHAN

until you find the right LAN channel

CHAN=1
LANIP=192.168.111.197 LANNET=255.255.255.0 LANGW=192.168.111.240 GWMAC=00:25:90:02:40:10
ipmitool lan set $CHAN ipsrc         static
ipmitool lan set $CHAN ipaddr        $LANIP
ipmitool lan set $CHAN netmask       $LANNET
ipmitool lan set $CHAN defgw ipaddr  $LANGW
ipmitool lan set $CHAN defgw macaddr $GWMAC
ipmitool lan set $CHAN arp respond   on
ipmitool lan set $CHAN auth          CALLBACK MD5 # NONE MD2 OEM
ipmitool lan set $CHAN auth          USER     MD5 # NONE MD2 OEM
ipmitool lan set $CHAN auth          OPERATOR MD5 # NONE MD2 OEM
ipmitool lan set $CHAN auth          ADMIN    MD5 # NONE MD2 OEM
ipmitool lan set $CHAN access        on

ipmitool lan print $CHAN

Warning: Make sure to disable NONE, as it allows the use of IPMI without any authentication. MD2 is insecure. MD5 should also be considered week since 2015, so using a separate network is strongly advised.

5. Create a user:

For this example the user is grated only “read” permissions. Run

ipmitool user list $CHAN

to get a list of already defined users.

USERID=4 USERNAME=phahn
export IPMI_PASSWORD=´makepasswd´
ipmitool user set name     $USERID $USERNAME
ipmitool user set password $USERID $IPMI_PASSWORD
ipmitool user enable $USERID
ipmitool channel setaccess $CHAN $USERID link=on ipmi=on callin=on privilege=2

6. [Optional] Configure serial-over-lan (SOL):

This allows you (for example) to use the serial console for BIOS configuration, GRUB selection, message redirection and login even:

ipmitool sol set set-in-progress       set-complete $CHAN
ipmitool sol set set-in-progress       set-in-progress $CHAN # this sometimes fails; then use 'set-complete' instead
ipmitool sol set privilege-level       user            $CHAN
ipmitool sol set non-volatile-bit-rate 115.2           $CHAN
ipmitool sol set volatile-bit-rate     serial          $CHAN
ipmitool sol set force-encryption      false           $CHAN
ipmitool sol set set-in-progress       set-complete    $CHAN
ipmitool sol set enabled               true            $CHAN
ipmitool sol payload                   enable          $CHAN $USERID

ipmitool sol info                                      $CHAN

7. You then need to change /boot/grub.cfg and /etc/inittab to make use of the serial console; see How to configure grub serial access (serial console) and Analyze boot problems

Example for usage

From now on you can remote administer the server from any other host, where ipmitools are also installed and which can connect to the IP address used above:

export IPMI_PASSWORD='...'
univention-install ipmitool

# Status of the chassis (power supply):
ipmitool -I lanplus -H $LANIP -U $USERNAME -L USER -E chassis status

# List of events (like power failure):
ipmitool -I lanplus -H $LANIP -U $USERNAME -L USER -E sel list

# Sensor data:
ipmitool -I lanplus -H $LANIP -U $USERNAME -L USER -E sensor list

# Serial-over-LAN:
ipmitool -I lanplus -H $LANIP -U $USERNAME -L USER -E sol activate
# To leave the console hit "ENTER" followed by '~' and '.'
Mastodon