| System Management Services (SMS) 1.2 Administrator Guide |
|
| System Management Services (SMS) 1.2 Administrator Guide |
|
| C H A P T E R 6 |
Domain Services |
Sun Fire 15K system hardware incorporates internal, private point-to-point Ethernet connections between the SC and each domain. This network, called the Management Network (MAN), is used to provide support services for each domain. This chapter describes those services.
The Management Network (MAN) function maintains the private point-to-point network connections between the SC and each domain. No packets addressed to one domain can be routed along the network connection between the SC and another domain ( FIGURE 6-1 ).
The hardware built into the Sun Fire 15K chassis to support MAN is complex. It includes 18 network interfaces ( NICs) on each SC that are connected in a point-to-point fashion to NICs located on each of the 18 expander IO slots. Using this design, the number of point-to-point Ethernet links between an SC and a given DSD vary based on the number of IO boards configured in that DSD. Each NIC from the SC connects to a hub and NIC on the I/O board. The NIC is an internal part of the I/O board and not a separate adapter card. Likewise, the Ethernet hub is on the I/O board. The hub is intelligent and can collect statistics. All of these point-to-point links are collectively called the I1 network. Since there can be multiple I/O boards in a given domain, multiple redundant network connections from the SC to a domain are possible.
|
Note - The I1 MAN network is a private network, not a general purpose network. No external IP traffic should be routed across it. Access to MAN is restricted to the system controller and the domains. |
On the SC, MAN software creates a meta-interface for the I1 network presenting to the Solaris operating environment a single network interface, scman0 . For more information refer to the Solaris scman (1M) man page.
MAN software detects communication errors and automatically initiates a path switch, provided an alternate path is available. MAN software also enforces domain isolation of network traffic on the I1 network. Similar software operates on the domain side.
There is also an internal network between the two system controllers consisting of two NICs per system controller. This network is called the I2 network. It is a private SC-to-SC network and is entirely separate from the I1 network.
MAN software creates a meta-interface for the I2 network as well. This interface is presented to the Solaris software as scman1 . As with the I1 network, I2 has a mechanism for detecting path failure and switching paths, providing an alternative is available.
The virtual network adapter on the SC presents itself as a standard network adapter. It can be managed and administered just like any other network adapter (for example, qfe, hme). The usual system administration tools such as ndd (1M), netstat (1M), and ifconfig (1M), can be used to manage the virtual network adapter. Certain operations of these tools (for example, changing the Ethernet address) should not be allowed for security reasons.
MAN operates and is managed as an IP network with special characteristics (for example, IP forwarding is disallowed by the MAN software). As such, the MAN operation is the same as any other IP network, with the noted exception documented above. Domains can be connected to your network depending on your site configuration and security requirements. Connecting domains is not within the scope of this document, refer to the System Administration Guide:Resource Management and Network Services.
External Network Monitoring for the Sun Fire 15K system provides highly available network connections from the SCs to customer networks called communities. This feature is built on top of the IP Network Multipathing (IPMP) framework provided in Solaris 9. For more information on IPMP refer to the System Adinistration Guide: IP Services.
External networks are made up of two communities. During installation, user communities are connected by physical cable to the RJ45 jacks on the SC connecting a node to the network.
For more information on connecting external networks, refer to the Sun Fire 15K System Site Planning Guide .
The term community refers to an IP network at your site. For example, you may have an engineering community and an accounting community. A community name is used as the interface group name . An interface group is a group of network interfaces that attach to the same community.
Configuring External Network Monitoring requires allocating several additional IP addresses for each system controller.
The addresses can be categorized as follows:
Test Addressees - These are IP addresses that are assigned to the external network interfaces on each system controller ( hme0 and eri1 ). Each IP test address is used to test the health of the particular network interface to which it is assigned. There is one IP test address assigned to each network interface. They are permanently associated with a particular network interface. If the network interface fails, the IP test address associated with the network interface becomes unreachable.
|
Note - In the case of IPv6, test addresses are not used. The link local address is used. |
Failover Addresses - There are two types of failover addresses:
SC Path Group Specific Addresses - These are IP addresses that are assigned to a particular interface group on each system controller. These IP addresses are used to provide highly available IP connectivity to a particular system controller for a given community. The SC path group specific address is reachable as long as one or more network interfaces in the interface group is functioning.
Community Failover Addresses - These are IP addresses that are assigned to a particular community on the MAIN SC (e.g. Community C1). They are used to provide IP connectivity to the MAIN SC, regardless of whether that is SC 0 or SC1.
All external software should reference the community failover address when communicating with the SC. This address will always connect to the MAIN SC. That way, if a failover occurs, external clients do not need to alter their configuration to reach the SC. For more information on SC failover, see SC Failover .
For more information on the MAN daemon and device drivers, refer to the mand (1M), and Solaris scman (1M) and dman (1M) man pages. See also Management Network Daemon .
The primary network services that MAN provides between the SC and the domains are:
Domain consoles
Message Logging
Dynamic Reconfiguration (DR)
Network boot/Solaris installation
System Controller (SC) heartbeats
The software running in a domain (OpenBoot PROM, kadb , and Solaris software) uses the system console for critical communications.
The domain console supports a login session and is secure, since the default configuration of the Solaris environment allows only the console to accept superuser logins. Domain console access is provided securely to remote administrators over a possibly public network.
The behavior of the console reflects the health of the software running in the domain. Character echo for user entries are nearly equivalent to that of a 9600 baud serial terminal attached to the domain. Output characters that are not echoes of user input are typically either the output from an executed command, a command interpreter or unsolicited log messages from the Solaris software. Activity on other domains or SMS support activity for the domain do not noticeably alter user entry echo response latency.
You can run kadb on the domain's Solaris software from the domain console. Interactions with OpenBoot PROM running on a domain use the domain console. The console can serve as the destination for log messages from the Solaris software; refer to syslog.conf (4). The console is available when software (Solaris, OpenBoot PROM, kadb ) is running on the domain.
You can open multiple connections to view the domain console output. However, the default is an exclusive locked connection.
For more information see SMS Console Window .
A domain administrator can forcibly break the domain console connection held by another.
You can forcibly break into OpenBoot PROM or
kadb
from the domain console, however it is not recommended. (This is a replacement for the physical
L1-A
or
STOP-A
key sequence available on a Sun SPARC
system with a physical console.) SMS captures console output history for subsequent analysis of domain crashes. A snapshot of the last console output for every domain is available in
/var/opt/SUNWSMS/adm/
domain_id
/console
.
The Sun Fire 15K system provides the hardware to either implement a shared-memory console or implement an alternate network data path for console. The hardware utilized for shared-memory console imposes less direct latency upon console data transfers, but is also used for other monitoring and control purposes for all domains, so there is a risk of latency introduced by contention for the hardware resources.
MAN provides private network paths to securely transfer domain console traffic to the SC, see Management Network Services . The console has a dual-pathed nature so that at least one path provides acceptable console response latency when the Solaris software is running. The dual-pathed console is robust in the face of errors. It detects failures on one domain console path and fails over to the other domain console path automatically. It supports user-directed selection of the domain console path to use.
smsconfig (1M) is the SC configuration utility that initially configures or later modifies the hostname, IP address, and netmask settings used by management network daemon, mand (1M). See Management Network Daemon .
mand initializes and updates these respective fields in the platform configuration database (pcd).
mand is automatically started by ssd . The management network daemon runs on the main SC in main mode and on the spare SC in spare mode.
For more information, refer to the SMS console (1M), mand (1M), and smsconfig (1M) man pages and the Solaris dman and (1M), scman (1M) man pages.
When configured to do so, MAN transports copies of important syslog messages from the domains to disk storage on the SC in order to facilitate failure analysis for crashed or unbootable domains. For more information, see Log File Maintenance .
Support for automatic network reconfiguration for a DR operation that removes the I/O board network connection endpoint from the domain, is initiated from the SC.
The MAN software layer is used to simplify the interface to the MAN hardware. MAN software handles the aspects of dynamic reconfiguration (DR) used by a DSD without requiring network configuration work by the domain or platform administrator.
Software in the domains using MAN need not be aware of which SC is currently the main SC. For more information on dynamic reconfiguration, refer to the System Management Services (SMS) 1.2 Dynamic Reconfiguration User Guide .
The SC provides network Solaris boot services to each domain.
When Solaris software is first installed on a domain, the network interface connecting it to the MAN is automatically created for subsequent system reboots. There are no additional tasks required by the domain administrator to configure or use MAN.
MAN is configured as a private network. A default address assignment for the management network is provided using the IP address space reserved for private networks. You can override the default address assignment for MAN to handle the case where the Sun Fire 15K is connected to a private customer network that already uses the selected MAN default IP address range.
The SC supports simultaneous network boots of domains running at least two different versions of Solaris software.
The SC provides software installation services to, at most, one domain at a time.
The I2 network is the intersystem controller communication. This is also called the heartbeat network. SMS failover mechanisms on the main use this network as one means of determining the health of the spare SC. For more information see Chapter 8 .
| System Management Services (SMS) 1.2 Administrator Guide | 816-3267-10 |
|
Copyright © 2002, Sun Microsystems, Inc. All rights reserved.