Solaris sar %sys cpu high

How to find out which pid is consuming %usr kernel cpu resources
#prstat -m
eg.
$ prstat -m
PID USERNAME USR SYS TRP TFL DFL LCK SLP LAT VCX ICX SCL SIG PROCESS/NLWP
3828 hpfimhas 24 75 1.1 0.0 0.0 0.0 0.0 0.0 0 0 87K 0 prstat/1
3812 root 0.1 0.1 0.0 0.0 0.0 0.0 100 0.0 9 0 224 0 sshd/1
3820 hpfimhas 0.0 0.0 0.0 0.0 0.0 0.0 100 0.0 38 0 128 0 sshd/1
3822 hpfimhas 0.0 0.0 0.0 0.0 0.0 0.0 100 0.0 3 0 110 0 sh/1
6105 instbas 0.0 0.0 0.0 0.0 0.0 73 27 0.0 10 0 14 0 java/48
6006 lmcbejo 0.0 0.0 0.0 0.0 0.0 87 13 0.0 23 1 16 0 java/30
13769 instbas 0.0 0.0 0.0 0.0 0.0 91 9.4 0.0 4 0 9 0 java/128
5974 instbas 0.0 0.0 0.0 0.0 0.0 84 16 0.0 16 0 10 0 java/57

For solaris 10 can use dtrace, see example below

On with the show: let's say you're looking at mpstat(1) output on your
multiuser server. You might see something like this:

CPU minf mjf xcal intr ithr csw icsw migr smtx srw syscl usr sys wt idl
12 1 27 3504 338 206 765 27 114 65 1 337 9 19 41 31
13 1 19 5725 98 68 723 22 108 120 0 692 3 17 20 61
14 0 57 3873 224 192 670 22 75 86 1 805 7 10 35 48
15 36 7 1551 42 28 689 6 68 59 0 132 2 9 35 54
16 14 7 7209 504 457 1031 37 125 244 0 459 6 30 4 60
17 5 5 4960 150 108 817 37 98 154 0 375 6 26 6 62
18 5 6 6085 1687 1661 741 60 76 248 0 434 3 33 0 64
19 0 15 10037 72 41 876 23 100 291 1 454 2 19 9 71
20 12 5 5890 746 711 992 32 122 216 2 960 10 33 4 53
21 60 5 1567 729 713 467 15 80 59 0 376 2 35 10 53
22 0 6 4378 315 291 751 17 84 142 1 312 3 16 1 80
23 0 6 12119 33 3 874 20 82 384 1 513 4 24 11 62

And well you may wonder (as perhaps you often have) -- what the hell is
causing all of those cross calls, anyway? (Cross calls appear in the "xcal"
column; see mpstat(1).)

Using DTrace, investigating this is a snap:

# dtrace -n xcalls'{@[execname] = count()}'
dtrace: description 'xcalls' matched 4 probes
[ letting this run for a few seconds ]
^C

mozilla-bin 1
lockd 1
in.mpathd 2
nsrmmd 5
grep 6
chmod 6
cat 6
nwadmin 13
ls 24
in.tftpd 28
nsrindexd 34
fsflush 38
cut 42
find 42
mkdir 66
rm 76
ipop3d 78
scp 79
inetd 96
dtrace 111
nawk 118
imapd-simmonmt 126
rmdir 132
sshd 138
rpc.rstatd 159
mv 398
save 1292
gzip 1315
get_all2 1678
sched 1712
nfsd 3709
tar 27054 <----- high usage

Solaris 10: mount a disk slice in a sub-zone

Non-global zones in Solaris 10 do not have the ability to see drives or disk slices (to prove this, list the contents of /dev/dsk from within a zone). What filesystems are mounted in which zones is controlled exclusively from the global zone. This recipe describes persistently mounting a disk slice in a non-global zone.
To mount the device c0t1d0s3 in the existing zone testzone under the mount point /mnt, login to the global zone and become root or a priviledged user and complete the following steps:



zonecfg -z testzone
zonecfg:testzone> add fs
zonecfg:testzone:fs> set dir=/mnt
zonecfg:testzone:fs> set special=/dev/dsk/c0t1d0s3
zonecfg:testzone:fs> set raw=/dev/rdsk/c0t1d0s3
zonecfg:testzone:fs> set type=ufs
zonecfg:testzone:fs> end
zonecfg:testzone> verify
zonecfg:testzone> commit
zonecfg:testzone> exit

Following the commit command, the filesystem will be immediately available in the zone. Substitute the desired mount point and device name for your system. If the zone has not yet been created, you can use the 'add fs' commands when creating the zone.

Solaris SVM grow concat//stripe

HOw to grow filesystem under disksuite without soft partiotion for concat stripe
find unused disk by #metastat | grep -i 2f5
then, use format cmd to repatition the disk
[05:58:15] root@server[31]# format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
0. c0t0d0
/pci@780/pci@0/pci@9/scsi@0/sd@0,0
1. c0t1d0
/pci@780/pci@0/pci@9/scsi@0/sd@1,0
2. c4t60060E800543BE00000043BE000000BBd0
/scsi_vhci/ssd@g60060e800543be00000043be000000bb
3. c4t60060E800543BE00000043BE000000BCd0
/scsi_vhci/ssd@g60060e800543be00000043be000000bc
4. c4t60060E800543BE00000043BE000000BDd0
/scsi_vhci/ssd@g60060e800543be00000043be000000bd
5. c4t60060E800543BE00000043BE000000BEd0
/scsi_vhci/ssd@g60060e800543be00000043be000000be
6. c4t60060E800543BE00000043BE000002F4d0
/scsi_vhci/ssd@g60060e800543be00000043be000002f4
7. c4t60060E800543BE00000043BE000002F5d0
/scsi_vhci/ssd@g60060e800543be00000043be000002f5
Specify disk (enter its number):
[05:58:15] root@server[32]#metattach d17 /dev/dsk/c4t60060E800543BE00000043BE000002F5d0s0
d107: component is attached
[05:58:15] root@server[33]# df -k /dbo
Filesystem kbytes used avail capacity Mounted on
/dev/md/dsk/d107 183417068 173603462 8489617 96% /dbo
[05:58:15] root@server[34]# grep dbo /etc/mnttab
/dev/md/dsk/d107 /dbo ufs rw,intr,largefiles,logging,xattr,onerror=panic,dev=154006b 1202293409
[05:58:15] root@server[35]# growfs -M /dbo /dev/md/rdsk/d107
/dev/md/rdsk/d107: 476067840 sectors in 77485 cylinders of 48 tracks, 128 sectors
232455.0MB in 4843 cyl groups (16 c/g, 48.00MB/g, 5824 i/g)
super-block backups (for fsck -F ufs -o b=#) at:
32, 98464, 196896, 295328, 393760, 492192, 590624, 689056, 787488, 885920,
Initializing cylinder groups:
...............................................................................
.................
super-block backups for last 10 cylinder groups at:
475107488, 475205920, 475304352, 475402784, 475501216, 475599648, 475698080,
475796512, 475894944, 475993376
[05:58:15] root@server[36]# df -k /dbo
Filesystem kbytes used avail capacity Mounted on
/dev/md/dsk/d107 234430636 173603462 59503185 75% /dbo
[05:58:15] root@server[37]#

Solaris Netapps add new filesystem

The following volume options should be configured:
Nosnap No automatic scheduled Snapshot™ duplications
Minra Minimal read ahead
Nvfail NVRAM check and behavior
sapfiler1> vol options sapdata nosnap on
sapfiler1> vol options sapdata minra on
sapfiler1> vol options sapdata nvfail on
sapfiler1> vol options saplog nosnap on
sapfiler1> vol options saplog minra on
sapfiler1> vol options saplog nvfail on
sapfiler1> qtree create /vol/sapdata/sapdata_lun
sapfiler1> qtree create /vol/saplog/saplog_lun
sapfiler1> qtree create /vol/saplog/sapmnt_lun
sapfiler1> qtree create /vol/saplog/sapusr_lun
sapfiler1> qtree create /vol/saplog/trans_lun
Create the LUNs
sapfiler1> lun create -s 30g -t solaris /vol/sapdata/sapdata_lun/sapdata
sapfiler1> lun create -s 5g -t solaris /vol/saplog/saplog_lun/saplog
sapfiler1> lun create -s 500m -t solaris /vol/saplog/sapmnt_lun/sapmnt
sapfiler1> lun create -s 500m -t solaris /vol/saplog/sapusr_lun/sapusr
sapfiler1> lun create -s 200m -t solaris /vol/saplog/trans_lun/trans
Define the initiator group
To create the initiator group you need to know the WWPN of the host, which will use these LUNs. The
WWPN can be obtained with the sanlun command on the host.
bash-2.03# sanlun fcp show adapter
lpfc0 WWPN:10000000c92d55f3
sapfiler1> igroup create -f -t solaris pp400 10:00:00:00:c9:2d:55:f3
Map the LUNs to the initiator group
sapfiler1> lun map /vol/sapdata/sapdata_lun/sapdata pp400 0
sapfiler1> lun map /vol/saplog/saplog_lun/saplog pp400 6
sapfiler1> lun map /vol/saplog/sapusr_lun/sapusr pp400 7
sapfiler1> lun map /vol/saplog/sapmnt_lun/sapmnt pp400 8
sapfiler1> lun map /vol/saplog/trans_lun/trans pp400 9

Configure persistent binding
Persistent binding is configured with the tool /usr/sbin/lpfc/lputil . To configure persistent
binding you need to know the WWNN of the filer. The WWNN can be obtained with the sysconfig
command at the filer console.
sapfiler1> sysconfig –v
…………
…………
slot 3: Fibre Channel Target Host Adapter 3a
(Dual-channel, QLogic 2312 (2342) rev. 2, 64-bit, )
Firmware rev: 3.1.15
Host Port Addr: 011000
Cacheline size: 8
SRAM parity: Yes
FC Nodename: 50:a9:80:00:02:00:88:f7 (50a98000020088f7)
FC Portname: 50:a9:80:03:02:00:88:f7 (50a98003020088f7)
Connection: PTP, Fabric
After using lputil command the lpfc.conf file has the following entry.
File /kernel/drv/lpfc.conf
…………
…………
# BEGIN: LPUTIL-managed Persistent Bindings
fcp-bind-WWNN="50a98000020088f7:lpfc0t1";
Network Appliance Inc. Proprietary
7
TECHNICAL REPORT
File /kernel/drv/lpfc.conf
…………
…………
# BEGIN: LPUTIL-managed Persistent Bindings
fcp-bind-WWNN="50a98000020088f7:lpfc0t1";
Edit entries in /kernel/drv/sd.conf
The LUN IDs that were used when mapping the LUNs to the initiator groups and the SCSI ID that was
used with the lputil command have to be used in the entries in /kernel/drv/sd.conf.
…………
…………
name="sd" parent="lpfc" target=1 lun=0;
name="sd" parent="lpfc" target=1 lun=1;
name="sd" parent="lpfc" target=1 lun=2;
name="sd" parent="lpfc" target=1 lun=3;
name="sd" parent="lpfc" target=1 lun=4;
name="sd" parent="lpfc" target=1 lun=5;
name="sd" parent="lpfc" target=1 lun=6;
name="sd" parent="lpfc" target=1 lun=7;
name="sd" parent="lpfc" target=1 lun=8;
name="sd" parent="lpfc" target=1 lun=9;

Reboot with reconfigure
After the new entries in /kernel/drv/sd.conf are edited, the host needs to be rebooted with the
reconfigure option. reboot -- -r

Configure new disks with format command
The new disks can now be configured with the format command.

VCS Adding node to the existing cluster

Adding the node to the existing cluster
Perform the tasks on one of the existing nodes in the cluster.
To add the new node to the existing cluster
1 Enter the command:
# haconf -makerw
2 Add the new system to the cluster:
# hasys -add east
3 Enter the following command:
# haconf -dump
4 Copy the main.cf file from an existing node to your new node:
# rcp /etc/VRTSvcs/conf/config/main.cf east:/etc/VRTSvcs/conf/
config/
5 Start VCS on the new node:
# hastart
6 If necessary, modify any new system attributes.
7 Enter the command:
# haconf -dump -makero
Start VCS after adding the new node to the cluster and verify the cluster.
To start VCS and verify the cluster
1 From the new system, start VCS with the new system added to the cluster:
# hastart
2 Run the GAB configuration command on each node to verify that Port a and
Port h include the new node in the membership:
# /sbin/gabconfig -a
GAB Port Memberships
===================================
Port a gen a3640003 membership 012
Port h gen fd570002 membership 012

Solaris emc powerpath link problem

If you boot a Solaris host with all socal host adapters to storage
system volumes disconnected or dysfunctional, PowerPath will not
configure any socal host adapter paths. After physically restoring the
socal connections, run the following commands to restore the paths
in PowerPath:

On hosts running this OS Run these commands
Solaris 7 and 8
devfsadm
powercf -q
powermt config
Solaris 2.6
drvconfig; disks; devlinks
powercf -q
powermt config

AIX configuring emc powerpath on bootdisk

This section describes the process for converting a system with AIX
installed on internal disks to boot from storage system logical
devices. The process first transfers a copy of the complete operating
system from an internal disk to logical devices on a storage system. It
then configures PowerPath so the root volume group takes advantage
of multipathing and failover capabilities. This is the recommended
process, as it allows you to revert to the internal disks in the event of
a problem.
Before you start:

Ensure that the AIX alt_disk_install LPP is installed on the
system. The LPP is on the AIX installation CD.

Apply the rte and boot_images filesets.
Then follow these steps:
1. Ensure that all device connections to the storage system are
established.
2. Ensure that all hdisks are configured properly
3. Run powermt config.
4. Use the rmdev command with the -d option to remove all
PowerPath devices, including the powerpath0 device. PowerPath
should remain installed, but all PowerPath devices must be
deleted.
5. Run lsdev -Ct power. No devices should be listed in the output.
6. Determine which hdisks on the storage system will receive the
copy of the operating system.
7. Run alt_install_disk -C hdisk_list to create the copy on the
storage system hdisk(s).
8. Reboot the system.
The system should boot using the hdisks specified in the previous
step.
9. Run powermt config.
10. Run bootlist -m normal -o to determine which hdisk is in the
bootlist.
11. Use powermt to determine which hdiskpower contains the hdisk
in the boot list.
12. Use the bootlist command to include all the path hdisks for the
hdiskpower found in the previous step.
13. Run pprootdev on.
14. Reboot the system.
When the system comes up, rootvg should be using hdiskpower
devices