Example--Two Shared Disk Sets

Figure 19-1 shows an example configuration that uses two disk sets.

In this configuration, Host A and Host B share disk sets A and B. They each have their own local disk set, which is not shared. If Host A fails, Host B can take over control of Host A's shared disk set (Disk set A). Likewise, if Host B fails, Host A can take control of Host B's shared disk set (Disk set B).

Figure 19-1 Disk Sets Example

Background Information for Disk Sets

When working with disk sets, consider the following "Background Information for Disk Sets" and "Administering Disk Sets".

Requirements for Disk Sets

Solaris Volume Manager must be configured on each host that will be connected to the disk set.
Each host must have its local state database set up before you can create disk sets.
To create and work with a disk set in a clustering environment, root must be a member of Group 14, or the /.rhosts file must contain an entry for the other host name (on each host).
To perform maintenance on a disk set, a host must be the owner of the disk set or have reserved the disk set. (A host takes implicit ownership of the disk set by putting the first drives into the set.)
You cannot add a drive that is in use to a disk set. Before you add a drive, make sure it is not currently being used for a file system, database, or any other application.
Do not add a drive with existing data that you want to preserve to a disk set. The process of adding the disk to the disk set repartitions the disk and destroys existing data.
All disks that you plan to share between hosts in the disk set must be connected to each host and must have the exact same path, driver, and name on each host. Specifically, a shared disk drive must be seen on both hosts at the same device number (c#t#d#). If the numbers are not the same on both hosts, you will see the message "drive c#t#d# is not common with host xxx" when attempting to add drives to the disk set. The shared disks must use the same driver name (ssd). See "How to Add Drives to a Disk Set" for more information on setting up shared disk drives in a disk set.

Guidelines for Disk Sets

The default total number of disk sets on a system is 4. You can increase this value up to 32 by editing the /kernel/drv/md.conf file, as described in "How to Increase the Number of Default Disk Sets". The number of shared disk sets is always one less than the md_nsets value, because the local set is included in md_nsets.
Unlike local volume administration, it is not necessary to create or delete state database replicas manually on the disk set. Solaris Volume Manager tries to balance a reasonable number of replicas across all drives in a disk set.
When drives are added to a disk set, Solaris Volume Manager re-balances the state database replicas across the remaining drives. Later, if necessary, you can change the replica layout with the metadb command.

Administering Disk Sets

Disk sets can be created and configured by using the Solaris Volume Manager command-line interface (the metaset command) or the Enhanced Storage tool within the Solaris Management Console.

After drives are added to a disk set, the disk set can be reserved (or taken) and released by hosts in the disk set. When a disk set is reserved by a host, the other host in the disk set cannot access the data on the drives in the disk set. To perform maintenance on a disk set, a host must be the owner of the disk set or have reserved the disk set. A host takes implicit ownership of the disk set by putting the first drives into the set.

Reserving a Disk Set

Before a host can use drives in a disk set, the host must reserve the disk set. There are two methods of reserving a disk set:

Safely - When you safely reserve a disk set, Solaris Volume Manager attempts to take the disk set, and the other host attempts to release the disk set. The release (and therefore the reservation) might fail.
Forcibly - When you forcibly reserve a disk set, Solaris Volume Manager reserves the disk set whether or not another host currently has the set reserved. This method is generally used when a host in the disk set is down or not communicating. All disks within the disk set are taken over. The state database is read in on the host performing the reservation and the shared volumes configured in the disk set become accessible. If the other host had the disk set reserved at this point, it would panic due to reservation loss.

Normally, two hosts in a disk set cooperate with each other to ensure that drives in a disk set are reserved by only one host at a time. A normal situation is defined as both hosts being up and communicating with each other.

Note - If a drive has been determined unexpectedly not to be reserved (perhaps because another host using the disk set forcibly took the drive), the host will panic. This behavior helps to minimize data loss which would occur if two hosts were to simultaneously access the same drive.

For more information about taking or reserving a disk set, see"How to Take a Disk Set".

Releasing a Disk Set

Releasing a disk set can be useful when you perform maintenance on the physical drives in the disk set. When a disk set is released, it cannot be accessed by the host. If both hosts in a disk set release the set, neither host in the disk set can access the drives in the disk set.

For more information about releasing a disk set, see "How to Release a Disk Set".

Scenario--Disk Sets

The following example, drawing on the sample system shown in Chapter 4, Configuring and Using Solaris Volume Manager (Scenario), describes how disk sets should be used to manage storage that resides on a SAN (Storage Area Network) fabric.

Assume that the sample system has an additional controller that connects to a fiber switch and SAN storage. Storage on the SAN fabric is not available to the system as early in the boot process as other devices, such as SCSI and IDE disks, and Solaris Volume Manager would report logical volumes on the fabric as unavailable at boot. However, by adding the storage to a disk set, and then using the disk set tools to manage the storage, this problem with boot time availability is avoided (and the fabric-attached storage can be easily managed within a separate, disk set controlled, namespace from the local storage).


19. Disk Sets (Overview) How Does Solaris Volume Manager Manage Disk Sets? Disk Set Name Requirements


	Example--Two Shared Disk Sets Figure 19-1 shows an example configuration that uses two disk sets. In this configuration, Host A and Host B share disk sets A and B. They each have their own local disk set, which is not shared. If Host A fails, Host B can take over control of Host A's shared disk set (Disk set A). Likewise, if Host B fails, Host A can take control of Host B's shared disk set (Disk set B). Figure 19-1 Disk Sets Example Background Information for Disk Sets When working with disk sets, consider the following "Background Information for Disk Sets" and "Administering Disk Sets". Requirements for Disk Sets Solaris Volume Manager must be configured on each host that will be connected to the disk set. Each host must have its local state database set up before you can create disk sets. To create and work with a disk set in a clustering environment, `root` must be a member of Group 14, or the `/.rhosts` file must contain an entry for the other host name (on each host). To perform maintenance on a disk set, a host must be the owner of the disk set or have reserved the disk set. (A host takes implicit ownership of the disk set by putting the first drives into the set.) You cannot add a drive that is in use to a disk set. Before you add a drive, make sure it is not currently being used for a file system, database, or any other application. Do not add a drive with existing data that you want to preserve to a disk set. The process of adding the disk to the disk set repartitions the disk and destroys existing data. All disks that you plan to share between hosts in the disk set must be connected to each host and must have the exact same path, driver, and name on each host. Specifically, a shared disk drive must be seen on both hosts at the same device number (`c#t#d#`). If the numbers are not the same on both hosts, you will see the message "drive `c#t#d#` is not common with host xxx" when attempting to add drives to the disk set. The shared disks must use the same driver name (`ssd`). See "How to Add Drives to a Disk Set" for more information on setting up shared disk drives in a disk set. Guidelines for Disk Sets The default total number of disk sets on a system is 4. You can increase this value up to 32 by editing the `/kernel/drv/md.conf` file, as described in "How to Increase the Number of Default Disk Sets". The number of shared disk sets is always one less than the `md_nsets` value, because the local set is included in `md_nsets`. Unlike local volume administration, it is not necessary to create or delete state database replicas manually on the disk set. Solaris Volume Manager tries to balance a reasonable number of replicas across all drives in a disk set. When drives are added to a disk set, Solaris Volume Manager re-balances the state database replicas across the remaining drives. Later, if necessary, you can change the replica layout with the `metadb` command. Administering Disk Sets Disk sets can be created and configured by using the Solaris Volume Manager command-line interface (the `metaset` command) or the Enhanced Storage tool within the Solaris Management Console. After drives are added to a disk set, the disk set can be reserved (or taken) and released by hosts in the disk set. When a disk set is reserved by a host, the other host in the disk set cannot access the data on the drives in the disk set. To perform maintenance on a disk set, a host must be the owner of the disk set or have reserved the disk set. A host takes implicit ownership of the disk set by putting the first drives into the set. Reserving a Disk Set Before a host can use drives in a disk set, the host must reserve the disk set. There are two methods of reserving a disk set: Safely - When you safely reserve a disk set, Solaris Volume Manager attempts to take the disk set, and the other host attempts to release the disk set. The release (and therefore the reservation) might fail. Forcibly - When you forcibly reserve a disk set, Solaris Volume Manager reserves the disk set whether or not another host currently has the set reserved. This method is generally used when a host in the disk set is down or not communicating. All disks within the disk set are taken over. The state database is read in on the host performing the reservation and the shared volumes configured in the disk set become accessible. If the other host had the disk set reserved at this point, it would panic due to reservation loss. Normally, two hosts in a disk set cooperate with each other to ensure that drives in a disk set are reserved by only one host at a time. A normal situation is defined as both hosts being up and communicating with each other. Note - If a drive has been determined unexpectedly not to be reserved (perhaps because another host using the disk set forcibly took the drive), the host will panic. This behavior helps to minimize data loss which would occur if two hosts were to simultaneously access the same drive. For more information about taking or reserving a disk set, see"How to Take a Disk Set". Releasing a Disk Set Releasing a disk set can be useful when you perform maintenance on the physical drives in the disk set. When a disk set is released, it cannot be accessed by the host. If both hosts in a disk set release the set, neither host in the disk set can access the drives in the disk set. For more information about releasing a disk set, see "How to Release a Disk Set". Scenario--Disk Sets The following example, drawing on the sample system shown in Chapter 4, Configuring and Using Solaris Volume Manager (Scenario), describes how disk sets should be used to manage storage that resides on a SAN (Storage Area Network) fabric. Assume that the sample system has an additional controller that connects to a fiber switch and SAN storage. Storage on the SAN fabric is not available to the system as early in the boot process as other devices, such as SCSI and IDE disks, and Solaris Volume Manager would report logical volumes on the fabric as unavailable at boot. However, by adding the storage to a disk set, and then using the disk set tools to manage the storage, this problem with boot time availability is avoided (and the fabric-attached storage can be easily managed within a separate, disk set controlled, namespace from the local storage).