Heartbeat is a daemon that handles all the failover in the cluster. For starters, see http://linux-ha.org/ . We are running Heartbeat V2.

Contents 1 Basics 2 Quick reference to resource types 2.1 drbddisk 2.2 Filesystem 2.3 IPaddr 3 Commands 3.1 crm_resource 3.2 hb_gui 3.3 crm_mon 3.4 crm_standby 4 Notes 5 AFS 5.1 Moving AFS server1-> server2 5.2 Moving back

Basics

Heartbeat works by managing resources on nodes. A node is a computer that runs stuff. A resource is any type of service that gets moved around. Examples of services include failover IP's, drbd disks (who is primary/secondary), filesystems (you use these to mount drbd stuff), and services. Resources are usually put into "Resource Groups". All the services in a resource group will be run on the same host, and they will be started/stopped sequentially.

There are also rules that help specify where services can be run. The most common is a "location" rule.

Quick reference to resource types

drbddisk

Parameter: 1, value: <name of DRBD resource>

Filesystem

• Parameter: device, value /dev/whatever
• Parameter: directory, value: /mountpoint
• Parameter: fstype, value xfs|reiserfs|ext3|etc

IPaddr

Parameter: 1, value: <ip address>

Commands

crm_resource

crm_resource is a command that lets you manage resources in the cluster. To use it, you must be a member of the haclient group.

crm_resource -W -r <resource>

Tells you where the specified resource is running

crm_resource -M -r <resource> [-h host]

Migrates the specified resource off of its current host. If -h is specified, it moves it to that host. This adds a location constraint with a score of -INFINITY for the resource and its current host (translation: the resource will never be run on its current host again), so you probably want to run

crm_resource -U -r <resource>

to remove this rule.

crm_resource -r <resource> -H <host> -C 

"Cleans up" a resource. You must use a real resource name, not a resource group.

crm_resource -r <resource> -p target_role -v (started|stopped)

Sets a resource's target role to either started or stopped.

hb_gui

hb_gui is a graphical interface to the heartbeat cluster. It's quite nice, and is also very useful for configuring services.

crm_mon

crm_mon is a command-line program that pretty-prints the current cluster status. You may also want to try crm_mon -n to show resources by nodes, or crm_mon -1 to just give one-shot info (not try to update it every 15sec or so)

crm_standby

crm_standyb allows you to set/clear the standby status of a machine. To put a machine into standby,

crm_standby -U <host> -v on

To take it out of standby, use either of the following commands

crm_standby -U <host> -D
crm_standby -U <host> -v off

Notes

• The raw configuration file is in /var/lib/heartbeat/crm/cib.xml . Never edit this file by hand- use cibadmin to add stuff to it. Better yet, use the gui.
• When configuring drbd stuff, use "drbddisk" instead of "drbd". The "drbd" resource is a V2 one that uses some complex master-slave stuff. Supposedly it can do cool stuff if it's set up correctly, but otherwise it's just confusing. "drbddisk" is much simpler and Just Works.

AFS

Doing stuff with AFS on heartbeat is kinda tricky. The problem is that the VLDB expects the volumes to be on the first server (server1, the primary). If you suddenly move them to the backup (server2), the VLDB doesn't know about it, and clients will try to talk to server1. The simple way to solve this is to run "vol syncvldb" on both, and then run "vos syncserv" on both. The problem is that you can't do this if one of the servers is down, and just running it on one server doesn't fix it.

Moving AFS server1-> server2

Well, I thought it worked... more testing still needs to be done.

Moving back

See above.