Preamble
Before entering in Veritas Volume Manager (VxVM) details we must go in fine detail of our hardware configuration. Let’s first confirm at Linux level that I see my 2 fiber channel (FC) cards with each 2 FC ports:
[root@server1 ~]# lspci | grep -i fibre
09:00.0 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03)
09:00.1 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03)
0c:00.0 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03)
0c:00.1 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03) |
[root@server1 ~]# lspci | grep -i fibre
09:00.0 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03)
09:00.1 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03)
0c:00.0 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03)
0c:00.1 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to PCI Express HBA (rev 03)
[root@server1 ~]# ll /sys/class/fc_host/host*/device
lrwxrwxrwx 1 root root 0 Jun 5 17:49 /sys/class/fc_host/host0/device -> ../../../devices/pci0000:00/0000:00:03.0/0000:09:00.0/host0
lrwxrwxrwx 1 root root 0 Jun 5 17:51 /sys/class/fc_host/host1/device -> ../../../devices/pci0000:00/0000:00:03.0/0000:09:00.1/host1
lrwxrwxrwx 1 root root 0 Jun 5 17:51 /sys/class/fc_host/host2/device -> ../../../devices/pci0000:00/0000:00:05.0/0000:0c:00.0/host2
lrwxrwxrwx 1 root root 0 Jun 5 17:51 /sys/class/fc_host/host3/device -> ../../../devices/pci0000:00/0000:00:05.0/0000:0c:00.1/host3 |
[root@server1 ~]# ll /sys/class/fc_host/host*/device
lrwxrwxrwx 1 root root 0 Jun 5 17:49 /sys/class/fc_host/host0/device -> ../../../devices/pci0000:00/0000:00:03.0/0000:09:00.0/host0
lrwxrwxrwx 1 root root 0 Jun 5 17:51 /sys/class/fc_host/host1/device -> ../../../devices/pci0000:00/0000:00:03.0/0000:09:00.1/host1
lrwxrwxrwx 1 root root 0 Jun 5 17:51 /sys/class/fc_host/host2/device -> ../../../devices/pci0000:00/0000:00:05.0/0000:0c:00.0/host2
lrwxrwxrwx 1 root root 0 Jun 5 17:51 /sys/class/fc_host/host3/device -> ../../../devices/pci0000:00/0000:00:05.0/0000:0c:00.1/host3
We will see this later but we need correspondence between SCSI chain number and its World Wide Name (WWN), here it is:
[root@server1 ~]# for host in `ls /sys/class/fc_host/`; do echo "$host WWN: `cat /sys/class/fc_host/$host/port_name`"; done
host0 WWN: 0x50060b0000c35e0c
host1 WWN: 0x50060b0000c35e0e
host2 WWN: 0x50060b0000c35e08
host3 WWN: 0x50060b0000c35e0a |
[root@server1 ~]# for host in `ls /sys/class/fc_host/`; do echo "$host WWN: `cat /sys/class/fc_host/$host/port_name`"; done
host0 WWN: 0x50060b0000c35e0c
host1 WWN: 0x50060b0000c35e0e
host2 WWN: 0x50060b0000c35e08
host3 WWN: 0x50060b0000c35e0a
So if everything is fiber wired (FC cards/ports to FC switches) I should see any disk array (enclosure) device/LUN as 4 equivalent name. Please note that this is absolutely not mandatory, what’s really important in my configuration is to have each disk visible through my 2 FC cards (so by one port minimum so each disk/LUN will have 2 names).
Please note that this post is more for DBAs to see and understand what have been done by you system teammates i.e. I’m not entering in the creation/modification of VxVM disk/disk group/controller and so on…
All commands have been executed on a Red Hat Enterprise Linux Server release 5.5 (Tikanga) server configured with Veritas Volume Manager release VRTSvxvm-5.1.132.000-SP1RP2_RHEL5.
To avoid putting real server name I’m using the below tips for my prompt customization:
export PS1="[\u@server1 \W]# " |
export PS1="[\u@server1 \W]# "
Dynamic Multipathing (DMP)
Veritas Dynamic Multi-Pathing (DMP or VxDMP) is the VxVM functionality implemented to support disk availability from multiple paths. Even if it looks complex to start with this functionality it is unfortunately mandatory before doing the more simple things as it is based configuration on which all other components will be created (subdisks, plexes, volumes).
Configured disk arrays or enclosures accessible from this server:
[root@server1 ~]# vxdmpadm listenclosure
ENCLR_NAME ENCLR_TYPE ENCLR_SNO STATUS ARRAY_TYPE LUN_COUNT
=======================================================================================
emc0 EMC 000290102696 CONNECTED A/A 54
emc1 EMC 000290105264 CONNECTED A/A 1
other_disks OTHER_DISKS OTHER_DISKS CONNECTED OTHER_DISKS 1 |
[root@server1 ~]# vxdmpadm listenclosure
ENCLR_NAME ENCLR_TYPE ENCLR_SNO STATUS ARRAY_TYPE LUN_COUNT
=======================================================================================
emc0 EMC 000290102696 CONNECTED A/A 54
emc1 EMC 000290105264 CONNECTED A/A 1
other_disks OTHER_DISKS OTHER_DISKS CONNECTED OTHER_DISKS 1
I/O policy default configuration of this enclosure:
[root@server1 ~]# vxdmpadm getattr enclosure emc0
ENCLR_NAME ATTR_NAME DEFAULT CURRENT
============================================================================
emc0 iopolicy MinimumQ MinimumQ
emc0 partitionsize 512 512
emc0 use_all_paths - -
emc0 failover_policy Global Global
emc0 recoveryoption[throttle] Nothrottle[0] Nothrottle[0]
emc0 recoveryoption[errorretry] Timebound[300] Timebound[300]
emc0 redundancy 0 0
emc0 dmp_lun_retry_timeout 0 0
emc0 failovermode - - |
[root@server1 ~]# vxdmpadm getattr enclosure emc0
ENCLR_NAME ATTR_NAME DEFAULT CURRENT
============================================================================
emc0 iopolicy MinimumQ MinimumQ
emc0 partitionsize 512 512
emc0 use_all_paths - -
emc0 failover_policy Global Global
emc0 recoveryoption[throttle] Nothrottle[0] Nothrottle[0]
emc0 recoveryoption[errorretry] Timebound[300] Timebound[300]
emc0 redundancy 0 0
emc0 dmp_lun_retry_timeout 0 0
emc0 failovermode - -
Disk default naming convention of VxVM:
[root@server1 ~]# vxddladm get namingscheme
NAMING_SCHEME PERSISTENCE LOWERCASE USE_AVID
============================================================
Enclosure Based Yes Yes Yes |
[root@server1 ~]# vxddladm get namingscheme
NAMING_SCHEME PERSISTENCE LOWERCASE USE_AVID
============================================================
Enclosure Based Yes Yes Yes
List of configured disk on your system:
[root@server1 ~]# vxdisk -e list
DEVICE TYPE DISK GROUP STATUS OS_NATIVE_NAME ATTR
cciss/c0d0 auto:none - - online invalid cciss/c0d0 -
emc0_00f3 auto:cdsdisk - - online sdbg std
emc0_3d6b auto:cdsdisk vgp1148_01 vgp1148 online clone_disk sdbn srdf-r1
emc0_3d7b auto:cdsdisk vgp1148_03 vgp1148 online clone_disk sdbp srdf-r1
emc0_3d73 auto:cdsdisk vgp1148_02 vgp1148 online clone_disk sdbo srdf-r1
emc0_3ebb auto:cdsdisk vgp1118_02 vgp1118 online clone_disk sdca srdf-r1
emc0_3eb3 auto:cdsdisk vgp1118_01 vgp1118 online clone_disk sdbx srdf-r1
emc0_3ecb auto:cdsdisk vgp1318_01 vgp1318 online clone_disk sdcc srdf-r1
emc0_3ec3 auto:cdsdisk vgp1118_03 vgp1118 online clone_disk sdcb srdf-r1
.
. |
[root@server1 ~]# vxdisk -e list
DEVICE TYPE DISK GROUP STATUS OS_NATIVE_NAME ATTR
cciss/c0d0 auto:none - - online invalid cciss/c0d0 -
emc0_00f3 auto:cdsdisk - - online sdbg std
emc0_3d6b auto:cdsdisk vgp1148_01 vgp1148 online clone_disk sdbn srdf-r1
emc0_3d7b auto:cdsdisk vgp1148_03 vgp1148 online clone_disk sdbp srdf-r1
emc0_3d73 auto:cdsdisk vgp1148_02 vgp1148 online clone_disk sdbo srdf-r1
emc0_3ebb auto:cdsdisk vgp1118_02 vgp1118 online clone_disk sdca srdf-r1
emc0_3eb3 auto:cdsdisk vgp1118_01 vgp1118 online clone_disk sdbx srdf-r1
emc0_3ecb auto:cdsdisk vgp1318_01 vgp1318 online clone_disk sdcc srdf-r1
emc0_3ec3 auto:cdsdisk vgp1118_03 vgp1118 online clone_disk sdcb srdf-r1
.
.
Disks of a disk group (please note it makes relationship between VxVM disks and the ones you see at system level):
[root@server1 ~]# vxdisk -g vgp1417 -e list
DEVICE TYPE DISK GROUP STATUS OS_NATIVE_NAME ATTR
emc0_084b auto:cdsdisk vgp1417_04 vgp1417 online clone_disk sdax srdf-r1
emc0_084f auto:cdsdisk vgp1417_05 vgp1417 online clone_disk sdaz srdf-r1
emc0_085b auto:cdsdisk vgp1417_08 vgp1417 online clone_disk sdbd srdf-r1
emc0_526f auto:cdsdisk vgp1417_09 vgp1417 online clone_disk sdbh srdf-r1
emc0_527b auto:cdsdisk vgp1417_10 vgp1417 online clone_disk sdbi srdf-r1
emc0_527f auto:cdsdisk vgp1417_11 vgp1417 online clone_disk sdbj srdf-r1
emc0_0853 auto:cdsdisk vgp1417_06 vgp1417 online clone_disk sdbb srdf-r1
emc0_0857 auto:cdsdisk vgp1417_07 vgp1417 online clone_disk sdbc srdf-r1 |
[root@server1 ~]# vxdisk -g vgp1417 -e list
DEVICE TYPE DISK GROUP STATUS OS_NATIVE_NAME ATTR
emc0_084b auto:cdsdisk vgp1417_04 vgp1417 online clone_disk sdax srdf-r1
emc0_084f auto:cdsdisk vgp1417_05 vgp1417 online clone_disk sdaz srdf-r1
emc0_085b auto:cdsdisk vgp1417_08 vgp1417 online clone_disk sdbd srdf-r1
emc0_526f auto:cdsdisk vgp1417_09 vgp1417 online clone_disk sdbh srdf-r1
emc0_527b auto:cdsdisk vgp1417_10 vgp1417 online clone_disk sdbi srdf-r1
emc0_527f auto:cdsdisk vgp1417_11 vgp1417 online clone_disk sdbj srdf-r1
emc0_0853 auto:cdsdisk vgp1417_06 vgp1417 online clone_disk sdbb srdf-r1
emc0_0857 auto:cdsdisk vgp1417_07 vgp1417 online clone_disk sdbc srdf-r1
Main problem with above command is that you don’t see multipathing details of VxVM disk, to go deeper in detail use:
[root@server1 ~]# vxdisk list emc0_084b
Device: emc0_084b
devicetag: emc0_084b
type: auto
hostid: server1
disk: name=vgp1417_04 id=1325653859.61.server1
group: name=vgp1417 id=1318435685.29.server1
info: format=cdsdisk,privoffset=256,pubslice=3,privslice=3
flags: online ready private autoconfig noautoimport imported clone_disk
pubpaths: block=/dev/vx/dmp/emc0_084bs3 char=/dev/vx/rdmp/emc0_084bs3
guid: -
udid: EMC%5FSYMMETRIX%5F000290102696%5F960084B000
site: -
version: 3.1
iosize: min=512 (bytes) max=1024 (blocks)
public: slice=3 offset=65792 len=70539872 disk_offset=0
private: slice=3 offset=256 len=65536 disk_offset=0
update: time=1335608893 seqno=0.17
ssb: actual_seqno=0.0
headers: 0 240
configs: count=1 len=51360
logs: count=1 len=4096
Defined regions:
config priv 000048-000239[000192]: copy=01 offset=000000 enabled
config priv 000256-051423[051168]: copy=01 offset=000192 enabled
log priv 051424-055519[004096]: copy=01 offset=000000 enabled
lockrgn priv 055520-055663[000144]: part=00 offset=000000
Multipathing information:
numpaths: 2
sdax state=enabled
sdh state=enabled |
[root@server1 ~]# vxdisk list emc0_084b
Device: emc0_084b
devicetag: emc0_084b
type: auto
hostid: server1
disk: name=vgp1417_04 id=1325653859.61.server1
group: name=vgp1417 id=1318435685.29.server1
info: format=cdsdisk,privoffset=256,pubslice=3,privslice=3
flags: online ready private autoconfig noautoimport imported clone_disk
pubpaths: block=/dev/vx/dmp/emc0_084bs3 char=/dev/vx/rdmp/emc0_084bs3
guid: -
udid: EMC%5FSYMMETRIX%5F000290102696%5F960084B000
site: -
version: 3.1
iosize: min=512 (bytes) max=1024 (blocks)
public: slice=3 offset=65792 len=70539872 disk_offset=0
private: slice=3 offset=256 len=65536 disk_offset=0
update: time=1335608893 seqno=0.17
ssb: actual_seqno=0.0
headers: 0 240
configs: count=1 len=51360
logs: count=1 len=4096
Defined regions:
config priv 000048-000239[000192]: copy=01 offset=000000 enabled
config priv 000256-051423[051168]: copy=01 offset=000192 enabled
log priv 051424-055519[004096]: copy=01 offset=000000 enabled
lockrgn priv 055520-055663[000144]: part=00 offset=000000
Multipathing information:
numpaths: 2
sdax state=enabled
sdh state=enabled
Remark:
You may also use the opposite command:
[root@server1 ~]# vxdisk list sdax
Device: emc0_084b
devicetag: emc0_084b
type: auto
hostid: server1
disk: name=vgp1417_04 id=1325653859.61.server1
group: name=vgp1417 id=1318435685.29.server1
info: format=cdsdisk,privoffset=256,pubslice=3,privslice=3
flags: online ready private autoconfig noautoimport imported clone_disk
pubpaths: block=/dev/vx/dmp/emc0_084bs3 char=/dev/vx/rdmp/emc0_084bs3
guid: -
udid: EMC%5FSYMMETRIX%5F000290102696%5F960084B000
site: -
version: 3.1
iosize: min=512 (bytes) max=1024 (blocks)
public: slice=3 offset=65792 len=70539872 disk_offset=0
private: slice=3 offset=256 len=65536 disk_offset=0
update: time=1335608893 seqno=0.17
ssb: actual_seqno=0.0
headers: 0 240
configs: count=1 len=51360
logs: count=1 len=4096
Defined regions:
config priv 000048-000239[000192]: copy=01 offset=000000 enabled
config priv 000256-051423[051168]: copy=01 offset=000192 enabled
log priv 051424-055519[004096]: copy=01 offset=000000 enabled
lockrgn priv 055520-055663[000144]: part=00 offset=000000
Multipathing information:
numpaths: 2
sdax state=enabled
sdh state=enabled |
[root@server1 ~]# vxdisk list sdax
Device: emc0_084b
devicetag: emc0_084b
type: auto
hostid: server1
disk: name=vgp1417_04 id=1325653859.61.server1
group: name=vgp1417 id=1318435685.29.server1
info: format=cdsdisk,privoffset=256,pubslice=3,privslice=3
flags: online ready private autoconfig noautoimport imported clone_disk
pubpaths: block=/dev/vx/dmp/emc0_084bs3 char=/dev/vx/rdmp/emc0_084bs3
guid: -
udid: EMC%5FSYMMETRIX%5F000290102696%5F960084B000
site: -
version: 3.1
iosize: min=512 (bytes) max=1024 (blocks)
public: slice=3 offset=65792 len=70539872 disk_offset=0
private: slice=3 offset=256 len=65536 disk_offset=0
update: time=1335608893 seqno=0.17
ssb: actual_seqno=0.0
headers: 0 240
configs: count=1 len=51360
logs: count=1 len=4096
Defined regions:
config priv 000048-000239[000192]: copy=01 offset=000000 enabled
config priv 000256-051423[051168]: copy=01 offset=000192 enabled
log priv 051424-055519[004096]: copy=01 offset=000000 enabled
lockrgn priv 055520-055663[000144]: part=00 offset=000000
Multipathing information:
numpaths: 2
sdax state=enabled
sdh state=enabled
Using vxdmpadm command you can also display complete multipathing information and obviously also confirm that each VxVM disk is accessible through 2 paths:
[root@server1 ~]# vxdmpadm getsubpaths dmpnodename=emc0_084b
NAME STATE[A] PATH-TYPE[M] CTLR-NAME ENCLR-TYPE ENCLR-NAME ATTRS
================================================================================
sdax ENABLED(A) - c2 EMC emc0 -
sdh ENABLED(A) - c0 EMC emc0 - |
[root@server1 ~]# vxdmpadm getsubpaths dmpnodename=emc0_084b
NAME STATE[A] PATH-TYPE[M] CTLR-NAME ENCLR-TYPE ENCLR-NAME ATTRS
================================================================================
sdax ENABLED(A) - c2 EMC emc0 -
sdh ENABLED(A) - c0 EMC emc0 -
[root@server1 ~]# vxdmpadm -v getdmpnode nodename=sdax
NAME STATE ENCLR-TYPE PATHS ENBL DSBL ENCLR-NAME SERIAL-NO ARRAY_VOL_ID
========================================================================================================
emc0_084b ENABLED EMC 2 2 0 emc0 960084B000 084B |
[root@server1 ~]# vxdmpadm -v getdmpnode nodename=sdax
NAME STATE ENCLR-TYPE PATHS ENBL DSBL ENCLR-NAME SERIAL-NO ARRAY_VOL_ID
========================================================================================================
emc0_084b ENABLED EMC 2 2 0 emc0 960084B000 084B
[root@server1 ~]# vxdmpadm getdmpnode enclosure=emc0
NAME STATE ENCLR-TYPE PATHS ENBL DSBL ENCLR-NAME
==============================================================================
emc0_00f3 ENABLED EMC 2 2 0 emc0
emc0_3d6b ENABLED EMC 2 2 0 emc0
emc0_3d7b ENABLED EMC 2 2 0 emc0
emc0_3d73 ENABLED EMC 2 2 0 emc0
emc0_3ebb ENABLED EMC 2 2 0 emc0
emc0_3eb3 ENABLED EMC 2 2 0 emc0
emc0_3ecb ENABLED EMC 2 2 0 emc0
emc0_3ec3 ENABLED EMC 2 2 0 emc0
.
. |
[root@server1 ~]# vxdmpadm getdmpnode enclosure=emc0
NAME STATE ENCLR-TYPE PATHS ENBL DSBL ENCLR-NAME
==============================================================================
emc0_00f3 ENABLED EMC 2 2 0 emc0
emc0_3d6b ENABLED EMC 2 2 0 emc0
emc0_3d7b ENABLED EMC 2 2 0 emc0
emc0_3d73 ENABLED EMC 2 2 0 emc0
emc0_3ebb ENABLED EMC 2 2 0 emc0
emc0_3eb3 ENABLED EMC 2 2 0 emc0
emc0_3ecb ENABLED EMC 2 2 0 emc0
emc0_3ec3 ENABLED EMC 2 2 0 emc0
.
.
[root@server1 ~]# vxdmpadm list dmpnode dmpnodename=emc0_084b
dmpdev = emc0_084b
state = enabled
enclosure = emc0
cab-sno = 000290102696
asl = libvxemc.so
vid = EMC
pid = SYMMETRIX
array-name = EMC
array-type = A/A
iopolicy = MinimumQ
avid = 084B
lun-sno = 960084B000
udid = EMC%5FSYMMETRIX%5F000290102696%5F960084B000
dev-attr = srdf-r1
###path = name state type transport ctlr hwpath aportID aportWWN attr
path = sdax enabled(a) - FC c2 c2 7b-b 50:06:04:84:52:a7:6a:36 -
path = sdh enabled(a) - FC c0 c0 10b-b 50:06:04:84:52:a7:6a:39 - |
[root@server1 ~]# vxdmpadm list dmpnode dmpnodename=emc0_084b
dmpdev = emc0_084b
state = enabled
enclosure = emc0
cab-sno = 000290102696
asl = libvxemc.so
vid = EMC
pid = SYMMETRIX
array-name = EMC
array-type = A/A
iopolicy = MinimumQ
avid = 084B
lun-sno = 960084B000
udid = EMC%5FSYMMETRIX%5F000290102696%5F960084B000
dev-attr = srdf-r1
###path = name state type transport ctlr hwpath aportID aportWWN attr
path = sdax enabled(a) - FC c2 c2 7b-b 50:06:04:84:52:a7:6a:36 -
path = sdh enabled(a) - FC c0 c0 10b-b 50:06:04:84:52:a7:6a:39 -
[root@server1 ~]# vxdmpadm getlungroup dmpnodename=sdax
NAME STATE ENCLR-TYPE PATHS ENBL DSBL ENCLR-NAME
=========================================================================
emc0_084b ENABLED EMC 2 2 0 emc0 |
[root@server1 ~]# vxdmpadm getlungroup dmpnodename=sdax
NAME STATE ENCLR-TYPE PATHS ENBL DSBL ENCLR-NAME
=========================================================================
emc0_084b ENABLED EMC 2 2 0 emc0
[root@server1 ~]# vxdmpadm getsubpaths dmpnodename=sdax
NAME STATE[A] PATH-TYPE[M] CTLR-NAME ENCLR-TYPE ENCLR-NAME ATTRS
================================================================================
sdax ENABLED(A) - c2 EMC emc0 -
sdh ENABLED(A) - c0 EMC emc0 - |
[root@server1 ~]# vxdmpadm getsubpaths dmpnodename=sdax
NAME STATE[A] PATH-TYPE[M] CTLR-NAME ENCLR-TYPE ENCLR-NAME ATTRS
================================================================================
sdax ENABLED(A) - c2 EMC emc0 -
sdh ENABLED(A) - c0 EMC emc0 -
Remark:
Please note the two controllers that are linked to the FC card we have identified in previous chapter:
[root@server1 ~]# vxdmpadm getsubpaths enclosure=emc0 ctlr=c0
NAME STATE[A] PATH-TYPE[M] DMPNODENAME ENCLR-NAME CTLR ATTRS
================================================================================
sdo ENABLED(A) - emc0_00f3 emc0 c0 -
sdz ENABLED(A) - emc0_0837 emc0 c0 -
sdh ENABLED(A) - emc0_084b emc0 c0 -
sdi ENABLED(A) - emc0_084f emc0 c0 -
sdj ENABLED(A) - emc0_0853 emc0 c0 -
sdk ENABLED(A) - emc0_0857 emc0 c0 -
sdl ENABLED(A) - emc0_085b emc0 c0 -
.
. |
[root@server1 ~]# vxdmpadm getsubpaths enclosure=emc0 ctlr=c0
NAME STATE[A] PATH-TYPE[M] DMPNODENAME ENCLR-NAME CTLR ATTRS
================================================================================
sdo ENABLED(A) - emc0_00f3 emc0 c0 -
sdz ENABLED(A) - emc0_0837 emc0 c0 -
sdh ENABLED(A) - emc0_084b emc0 c0 -
sdi ENABLED(A) - emc0_084f emc0 c0 -
sdj ENABLED(A) - emc0_0853 emc0 c0 -
sdk ENABLED(A) - emc0_0857 emc0 c0 -
sdl ENABLED(A) - emc0_085b emc0 c0 -
.
.
The vxdisk command can also display the multipathing information:
[root@server1 ~]# vxdisk path -g vgp1417
SUBPATH DANAME DMNAME GROUP STATE
sdax emc0_084b vgp1417_04 vgp1417 ENABLED
sdh emc0_084b vgp1417_04 vgp1417 ENABLED
sdaz emc0_084f vgp1417_05 vgp1417 ENABLED
sdi emc0_084f vgp1417_05 vgp1417 ENABLED
sdbd emc0_085b vgp1417_08 vgp1417 ENABLED
sdl emc0_085b vgp1417_08 vgp1417 ENABLED
sdbh emc0_526f vgp1417_09 vgp1417 ENABLED
sdp emc0_526f vgp1417_09 vgp1417 ENABLED
sdbi emc0_527b vgp1417_10 vgp1417 ENABLED
sdq emc0_527b vgp1417_10 vgp1417 ENABLED
sdbj emc0_527f vgp1417_11 vgp1417 ENABLED
sdr emc0_527f vgp1417_11 vgp1417 ENABLED
sdbb emc0_0853 vgp1417_06 vgp1417 ENABLED
sdj emc0_0853 vgp1417_06 vgp1417 ENABLED
sdbc emc0_0857 vgp1417_07 vgp1417 ENABLED
sdk emc0_0857 vgp1417_07 vgp1417 ENABLED |
[root@server1 ~]# vxdisk path -g vgp1417
SUBPATH DANAME DMNAME GROUP STATE
sdax emc0_084b vgp1417_04 vgp1417 ENABLED
sdh emc0_084b vgp1417_04 vgp1417 ENABLED
sdaz emc0_084f vgp1417_05 vgp1417 ENABLED
sdi emc0_084f vgp1417_05 vgp1417 ENABLED
sdbd emc0_085b vgp1417_08 vgp1417 ENABLED
sdl emc0_085b vgp1417_08 vgp1417 ENABLED
sdbh emc0_526f vgp1417_09 vgp1417 ENABLED
sdp emc0_526f vgp1417_09 vgp1417 ENABLED
sdbi emc0_527b vgp1417_10 vgp1417 ENABLED
sdq emc0_527b vgp1417_10 vgp1417 ENABLED
sdbj emc0_527f vgp1417_11 vgp1417 ENABLED
sdr emc0_527f vgp1417_11 vgp1417 ENABLED
sdbb emc0_0853 vgp1417_06 vgp1417 ENABLED
sdj emc0_0853 vgp1417_06 vgp1417 ENABLED
sdbc emc0_0857 vgp1417_07 vgp1417 ENABLED
sdk emc0_0857 vgp1417_07 vgp1417 ENABLED
Let’s come back to the controller and display the ones configured on my system (each enclosure is configured with two controllers):
[root@server1 ~]# vxdmpadm listctlr all
CTLR-NAME ENCLR-TYPE STATE ENCLR-NAME
=====================================================
c0 EMC ENABLED emc0
c2 EMC ENABLED emc0
c0 EMC ENABLED emc1
c2 EMC ENABLED emc1
c360 OTHER_DISKS ENABLED other_disks |
[root@server1 ~]# vxdmpadm listctlr all
CTLR-NAME ENCLR-TYPE STATE ENCLR-NAME
=====================================================
c0 EMC ENABLED emc0
c2 EMC ENABLED emc0
c0 EMC ENABLED emc1
c2 EMC ENABLED emc1
c360 OTHER_DISKS ENABLED other_disks
We can re-conciliate WWN identified in previous chapter with VxDMP commands:
[root@server1 ~]# vxdmpadm getctlr c0
LNAME PNAME VENDOR CTLR-ID
=============================================================================================
c0 c0 Qlogic-Corp. 50:06:0b:00:00:c3:5e:0c
[root@server1 ~]# vxdmpadm getctlr c2
LNAME PNAME VENDOR CTLR-ID
=============================================================================================
c2 c2 Qlogic-Corp. 50:06:0b:00:00:c3:5e:08 |
[root@server1 ~]# vxdmpadm getctlr c0
LNAME PNAME VENDOR CTLR-ID
=============================================================================================
c0 c0 Qlogic-Corp. 50:06:0b:00:00:c3:5e:0c
[root@server1 ~]# vxdmpadm getctlr c2
LNAME PNAME VENDOR CTLR-ID
=============================================================================================
c2 c2 Qlogic-Corp. 50:06:0b:00:00:c3:5e:08
And confirm, as anticipated, that each of my FC card is wired with only one FC cable. Each of my FC or Host Bus Adapter (HBA) cards has two ports and only one is used:
[root@server1 ~]# vxddladm list hbas
HBA-ID DRIVER FIRMWARE DISCOVERY STATE ADDRESS
======================================================================================
c1 8.03.07.03.05.07-k 5.03.16 (496) fabric Offline 50:06:0b:00:00:c3:5e:0e
c3 8.03.07.03.05.07-k 5.03.16 (496) fabric Offline 50:06:0b:00:00:c3:5e:0a
c360 - - - Online -
c0 8.03.07.03.05.07-k 5.03.16 (496) fabric Online 50:06:0b:00:00:c3:5e:0c
c2 8.03.07.03.05.07-k 5.03.16 (496) fabric Online 50:06:0b:00:00:c3:5e:08 |
[root@server1 ~]# vxddladm list hbas
HBA-ID DRIVER FIRMWARE DISCOVERY STATE ADDRESS
======================================================================================
c1 8.03.07.03.05.07-k 5.03.16 (496) fabric Offline 50:06:0b:00:00:c3:5e:0e
c3 8.03.07.03.05.07-k 5.03.16 (496) fabric Offline 50:06:0b:00:00:c3:5e:0a
c360 - - - Online -
c0 8.03.07.03.05.07-k 5.03.16 (496) fabric Online 50:06:0b:00:00:c3:5e:0c
c2 8.03.07.03.05.07-k 5.03.16 (496) fabric Online 50:06:0b:00:00:c3:5e:08
To obtain the configured ports:
[root@server1 ~]# vxddladm list ports
PORT-ID HBA-ID STATE ADDRESS
==========================================
c0_p3 c0 Online 50:06:04:84:52:a7:6a:39
c0_p2 c0 Online 50:06:04:84:52:a7:6a:08
c0_p1 c0 Online 50:06:04:84:52:a9:ec:09
c2_p3 c2 Online 50:06:04:84:52:a7:6a:36
c2_p2 c2 Online 50:06:04:84:52:a7:6a:07
c2_p1 c2 Online 50:06:04:84:52:a9:ec:06
[root@server1 ~]# vxddladm list targets
TARGET-ID ALIAS HBA-ID STATE ADDRESS
=====================================================
c0_p3_t0 - c0 Online 50:06:04:84:52:a7:6a:39
c0_p2_t0 - c0 Online 50:06:04:84:52:a7:6a:08
c0_p1_t0 - c0 Online 50:06:04:84:52:a9:ec:09
c2_p3_t0 - c2 Online 50:06:04:84:52:a7:6a:36
c2_p2_t0 - c2 Online 50:06:04:84:52:a7:6a:07
c2_p1_t0 - c2 Online 50:06:04:84:52:a9:ec:06
[root@server1 ~]# vxddladm list targets hba=c0
TARGET-ID ALIAS HBA-ID STATE ADDRESS
=====================================================
c0_p3_t0 - c0 Online 50:06:04:84:52:a7:6a:39
c0_p2_t0 - c0 Online 50:06:04:84:52:a7:6a:08
c0_p1_t0 - c0 Online 50:06:04:84:52:a9:ec:09 |
[root@server1 ~]# vxddladm list ports
PORT-ID HBA-ID STATE ADDRESS
==========================================
c0_p3 c0 Online 50:06:04:84:52:a7:6a:39
c0_p2 c0 Online 50:06:04:84:52:a7:6a:08
c0_p1 c0 Online 50:06:04:84:52:a9:ec:09
c2_p3 c2 Online 50:06:04:84:52:a7:6a:36
c2_p2 c2 Online 50:06:04:84:52:a7:6a:07
c2_p1 c2 Online 50:06:04:84:52:a9:ec:06
[root@server1 ~]# vxddladm list targets
TARGET-ID ALIAS HBA-ID STATE ADDRESS
=====================================================
c0_p3_t0 - c0 Online 50:06:04:84:52:a7:6a:39
c0_p2_t0 - c0 Online 50:06:04:84:52:a7:6a:08
c0_p1_t0 - c0 Online 50:06:04:84:52:a9:ec:09
c2_p3_t0 - c2 Online 50:06:04:84:52:a7:6a:36
c2_p2_t0 - c2 Online 50:06:04:84:52:a7:6a:07
c2_p1_t0 - c2 Online 50:06:04:84:52:a9:ec:06
[root@server1 ~]# vxddladm list targets hba=c0
TARGET-ID ALIAS HBA-ID STATE ADDRESS
=====================================================
c0_p3_t0 - c0 Online 50:06:04:84:52:a7:6a:39
c0_p2_t0 - c0 Online 50:06:04:84:52:a7:6a:08
c0_p1_t0 - c0 Online 50:06:04:84:52:a9:ec:09
To understand which devices are using which ports and the name of the VxVM driver:
[root@server1 ~]# vxddladm list devices
DEVICE TARGET-ID STATE DDL-STATUS (ASL)
===============================================================
cciss/c0d0 - Online -
sdg c0_p3_t0 Online SKIPPED
sdi c0_p3_t0 Online CLAIMED (libvxemc.so)
sdf c0_p3_t0 Online SKIPPED
sdh c0_p3_t0 Online CLAIMED (libvxemc.so)
sde c0_p3_t0 Online SKIPPED
sdd c0_p3_t0 Online SKIPPED
sdr c0_p3_t0 Online CLAIMED (libvxemc.so)
.
.
[root@server1 ~]# vxddladm list devices target=c0_p1_t0
DEVICE TARGET-ID STATE DDL-STATUS (ASL)
===============================================================
sdbl c0_p1_t0 Online CLAIMED (libvxemc.so)
sdbk c0_p1_t0 Online SKIPPED |
[root@server1 ~]# vxddladm list devices
DEVICE TARGET-ID STATE DDL-STATUS (ASL)
===============================================================
cciss/c0d0 - Online -
sdg c0_p3_t0 Online SKIPPED
sdi c0_p3_t0 Online CLAIMED (libvxemc.so)
sdf c0_p3_t0 Online SKIPPED
sdh c0_p3_t0 Online CLAIMED (libvxemc.so)
sde c0_p3_t0 Online SKIPPED
sdd c0_p3_t0 Online SKIPPED
sdr c0_p3_t0 Online CLAIMED (libvxemc.so)
.
.
[root@server1 ~]# vxddladm list devices target=c0_p1_t0
DEVICE TARGET-ID STATE DDL-STATUS (ASL)
===============================================================
sdbl c0_p1_t0 Online CLAIMED (libvxemc.so)
sdbk c0_p1_t0 Online SKIPPED
To get the list of supported enclosures:
[root@server1 ~]# vxddladm listsupport all
LIBNAME VID PID
=================================================================================================
libvxpurple.so SUN T300
libvxveritas.so VERITAS All
libvxhillsannet2.so DotHill SANnet II FC, SANnet II SCSI, SANnet II SATA, SANnet II U320
libvxhds.so HITACHI All
libvxpp.so EMC, DGC All
libvxFJTSYe8k.so FUJITSU All
libvxfje3k4ka.so FUJITSU E3000, E400A
libvxibmds6k.so IBM 1750
libvxsun7x10.so SUN Sun Storage 7410, Sun Storage 7310, Sun Storage 7210, Sun Storage 7110, ZFS Storage 7120, ZFS Storage 7320, ZFS Storage 7420, ZFS Storage 7720
libvxfsc.so FSC FibreCAT_SX1
libvxxp1281024.so HP All
libvxlsiall.so IBM, SUN, STK, SGI All
libvxnipnyis.so NEC iStorage 1000, iStorage 2000, iStorage 4000
libvxtsbaf.so TOSHIBA AF_AF3500, AF_AF1500, AF2_AF7000, AF2_AF2000, AF3_AF7500, AF3_AF2500
libvxfusionio.so FIO ioDrive
libvxhpalua.so HP, COMPAQ HSV101, HSV111 (C)COMPAQ, HSV111, HSV200, HSV210, HSV300, HSV400, HSV450
libvxxp12k.so HP All
libvxhitachi.so HITACHI DF350, DF400, DF400F, DF500, DF500F
libvxpillaraxiom.so Pillar Axiom 300, Axiom 500, Axiom 600
libvxFJTSYe6k.so FUJITSU E6000
libvxxiv.so XIV, IBM NEXTRA, 2810XIV
libvxxp256.so HP All
libvxramsan.so TMS RamSan 400
libvxhdsalua.so HITACHI DF600, DF600-V, DF600F, DF600F-V
libvxhpmsa.so HP MSA VOLUME
libvxhds9980.so HITACHI All
libvxhdsusp.so HITACHI All
libvxCLARiiON.so DGC All
libvxnetapp.so NETAPP All
libvxeqlogic.so EQLOGIC 100E-00
libvxemc.so EMC SYMMETRIX
libvxddns2a.so DDN S2A 9550, S2A 9900, S2A 9700
libvxfjtsye2k.so FUJITSU E2000, ETERNUS_DXL
libvxmsa2kfc_sa.so HP MSA2312fc, MSA2324fc, MSA2012sa, MSA2312sa, MSA2324sa, MSA2312i, MSA2324i, P2000 G3 FC, P2000G3 FC/iSCSI, P2000 G3 SAS, P2000 G3 iSCSI
libvxdothill.so DotHill R/Evo 2730-2R, R/Evo 2530-2R, R/Evo 2330-2R, R/Evo 2130-2RX, R/Evo 2130-2J, R/Evo 5730-2R
libvxcompellent.so COMPELNT Compellent Vol
libvxibmds8k.so IBM 2107
libvxmsa2k.so HP MSA2012fc, MSA2212fc, MSA2012i
libvxxiotechE5k.so XIOTECH ISE1400
libvxcopan.so COPANSYS 8814, 8818
libvxibmsvc.so IBM 2145, 2062
libvxhuawei.so HUAWEI, HS S5300, S5500, S5600, S2300, S2100, V1800, V1500, VIS6000, S2300E, S2600, S6800E, S8000-I, S8000
libvx3par.so 3PARdata VV
libvxhpsvsp.so HP HSVX740
libvxshark.so IBM 2105 |
[root@server1 ~]# vxddladm listsupport all
LIBNAME VID PID
=================================================================================================
libvxpurple.so SUN T300
libvxveritas.so VERITAS All
libvxhillsannet2.so DotHill SANnet II FC, SANnet II SCSI, SANnet II SATA, SANnet II U320
libvxhds.so HITACHI All
libvxpp.so EMC, DGC All
libvxFJTSYe8k.so FUJITSU All
libvxfje3k4ka.so FUJITSU E3000, E400A
libvxibmds6k.so IBM 1750
libvxsun7x10.so SUN Sun Storage 7410, Sun Storage 7310, Sun Storage 7210, Sun Storage 7110, ZFS Storage 7120, ZFS Storage 7320, ZFS Storage 7420, ZFS Storage 7720
libvxfsc.so FSC FibreCAT_SX1
libvxxp1281024.so HP All
libvxlsiall.so IBM, SUN, STK, SGI All
libvxnipnyis.so NEC iStorage 1000, iStorage 2000, iStorage 4000
libvxtsbaf.so TOSHIBA AF_AF3500, AF_AF1500, AF2_AF7000, AF2_AF2000, AF3_AF7500, AF3_AF2500
libvxfusionio.so FIO ioDrive
libvxhpalua.so HP, COMPAQ HSV101, HSV111 (C)COMPAQ, HSV111, HSV200, HSV210, HSV300, HSV400, HSV450
libvxxp12k.so HP All
libvxhitachi.so HITACHI DF350, DF400, DF400F, DF500, DF500F
libvxpillaraxiom.so Pillar Axiom 300, Axiom 500, Axiom 600
libvxFJTSYe6k.so FUJITSU E6000
libvxxiv.so XIV, IBM NEXTRA, 2810XIV
libvxxp256.so HP All
libvxramsan.so TMS RamSan 400
libvxhdsalua.so HITACHI DF600, DF600-V, DF600F, DF600F-V
libvxhpmsa.so HP MSA VOLUME
libvxhds9980.so HITACHI All
libvxhdsusp.so HITACHI All
libvxCLARiiON.so DGC All
libvxnetapp.so NETAPP All
libvxeqlogic.so EQLOGIC 100E-00
libvxemc.so EMC SYMMETRIX
libvxddns2a.so DDN S2A 9550, S2A 9900, S2A 9700
libvxfjtsye2k.so FUJITSU E2000, ETERNUS_DXL
libvxmsa2kfc_sa.so HP MSA2312fc, MSA2324fc, MSA2012sa, MSA2312sa, MSA2324sa, MSA2312i, MSA2324i, P2000 G3 FC, P2000G3 FC/iSCSI, P2000 G3 SAS, P2000 G3 iSCSI
libvxdothill.so DotHill R/Evo 2730-2R, R/Evo 2530-2R, R/Evo 2330-2R, R/Evo 2130-2RX, R/Evo 2130-2J, R/Evo 5730-2R
libvxcompellent.so COMPELNT Compellent Vol
libvxibmds8k.so IBM 2107
libvxmsa2k.so HP MSA2012fc, MSA2212fc, MSA2012i
libvxxiotechE5k.so XIOTECH ISE1400
libvxcopan.so COPANSYS 8814, 8818
libvxibmsvc.so IBM 2145, 2062
libvxhuawei.so HUAWEI, HS S5300, S5500, S5600, S2300, S2100, V1800, V1500, VIS6000, S2300E, S2600, S6800E, S8000-I, S8000
libvx3par.so 3PARdata VV
libvxhpsvsp.so HP HSVX740
libvxshark.so IBM 2105
And more information (Active/Active mode) on the one I use:
[root@server1 ~]# vxddladm listsupport libname=libvxemc.so
ATTR_NAME ATTR_VALUE
=================================================================================================
LIBNAME libvxemc.so
VID EMC
PID SYMMETRIX
ARRAY_TYPE A/A
ARRAY_NAME EMC |
[root@server1 ~]# vxddladm listsupport libname=libvxemc.so
ATTR_NAME ATTR_VALUE
=================================================================================================
LIBNAME libvxemc.so
VID EMC
PID SYMMETRIX
ARRAY_TYPE A/A
ARRAY_NAME EMC
VxVM Basic commands
Disk groups
How to determine if a disk is associated to a disk group:
[root@server1 ~]# vxdisk -s list sdax
Disk: emc0_084b
type: auto
flags: online ready private autoconfig noautoimport imported clone_disk
guid: -
udid: EMC%5FSYMMETRIX%5F000290102696%5F960084B000
site: -
diskid: 1325653859.61.server1
dgname: vgp1417
dgid: 1318435685.29.server1
hostid: server1
info: format=cdsdisk,privoffset=256,pubslice=3,privslice=3 |
[root@server1 ~]# vxdisk -s list sdax
Disk: emc0_084b
type: auto
flags: online ready private autoconfig noautoimport imported clone_disk
guid: -
udid: EMC%5FSYMMETRIX%5F000290102696%5F960084B000
site: -
diskid: 1325653859.61.server1
dgname: vgp1417
dgid: 1318435685.29.server1
hostid: server1
info: format=cdsdisk,privoffset=256,pubslice=3,privslice=3
Disk groups list:
[root@server1 ~]# vxdg list
NAME STATE ID
vgp1417a enabled,cds 1318432411.19.server1
vgoem3 enabled,cds 1320851456.27.server1
vgptws02a enabled,cds 1318847636.93.server1
.
. |
[root@server1 ~]# vxdg list
NAME STATE ID
vgp1417a enabled,cds 1318432411.19.server1
vgoem3 enabled,cds 1320851456.27.server1
vgptws02a enabled,cds 1318847636.93.server1
.
.
More information of a disk group:
[root@server1 ~]# vxdg list vgp1417
Group: vgp1417
dgid: 1318435685.29.server1
import-id: 1024.62
flags: cds
version: 160
alignment: 8192 (bytes)
ssb: on
autotagging: on
detach-policy: global
dg-fail-policy: dgdisable
copies: nconfig=default nlog=default
config: seqno=0.1556 permlen=51360 free=51325 templen=17 loglen=4096
config disk emc0_084b copy 1 len=51360 state=clean online
config disk emc0_084f copy 1 len=51360 state=clean online
config disk emc0_085b copy 1 len=51360 state=clean online
config disk emc0_526f copy 1 len=51360 disabled
config disk emc0_527b copy 1 len=51360 disabled
config disk emc0_527f copy 1 len=51360 disabled
config disk emc0_0853 copy 1 len=51360 state=clean online
config disk emc0_0857 copy 1 len=51360 state=clean online
log disk emc0_084b copy 1 len=4096
log disk emc0_084f copy 1 len=4096
log disk emc0_085b copy 1 len=4096
log disk emc0_526f copy 1 len=4096 disabled
log disk emc0_527b copy 1 len=4096 disabled
log disk emc0_527f copy 1 len=4096 disabled
log disk emc0_0853 copy 1 len=4096
log disk emc0_0857 copy 1 len=4096 |
[root@server1 ~]# vxdg list vgp1417
Group: vgp1417
dgid: 1318435685.29.server1
import-id: 1024.62
flags: cds
version: 160
alignment: 8192 (bytes)
ssb: on
autotagging: on
detach-policy: global
dg-fail-policy: dgdisable
copies: nconfig=default nlog=default
config: seqno=0.1556 permlen=51360 free=51325 templen=17 loglen=4096
config disk emc0_084b copy 1 len=51360 state=clean online
config disk emc0_084f copy 1 len=51360 state=clean online
config disk emc0_085b copy 1 len=51360 state=clean online
config disk emc0_526f copy 1 len=51360 disabled
config disk emc0_527b copy 1 len=51360 disabled
config disk emc0_527f copy 1 len=51360 disabled
config disk emc0_0853 copy 1 len=51360 state=clean online
config disk emc0_0857 copy 1 len=51360 state=clean online
log disk emc0_084b copy 1 len=4096
log disk emc0_084f copy 1 len=4096
log disk emc0_085b copy 1 len=4096
log disk emc0_526f copy 1 len=4096 disabled
log disk emc0_527b copy 1 len=4096 disabled
log disk emc0_527f copy 1 len=4096 disabled
log disk emc0_0853 copy 1 len=4096
log disk emc0_0857 copy 1 len=4096
Free space of a disk group:
[root@server1 ~]# vxdg -g vgp1417 free
DISK DEVICE TAG OFFSET LENGTH FLAGS
vgp1417_04 emc0_084b emc0_084b 56623104 13916768 -
vgp1417_05 emc0_084f emc0_084f 64389600 6150272 -
vgp1417_09 emc0_526f emc0_526f 17592064 52947808 -
vgp1417_10 emc0_527b emc0_527b 0 70539872 -
vgp1417_11 emc0_527f emc0_527f 0 70539872 - |
[root@server1 ~]# vxdg -g vgp1417 free
DISK DEVICE TAG OFFSET LENGTH FLAGS
vgp1417_04 emc0_084b emc0_084b 56623104 13916768 -
vgp1417_05 emc0_084f emc0_084f 64389600 6150272 -
vgp1417_09 emc0_526f emc0_526f 17592064 52947808 -
vgp1417_10 emc0_527b emc0_527b 0 70539872 -
vgp1417_11 emc0_527f emc0_527f 0 70539872 -
Remark:
So how much at the end ? For vgp1417_04 (emc0_084b) it is 13916768*512 (sector size)/(1024 * 1024) MB i.e. 6795MB free (I have already seen more user friendly command output)…
Even if not written in official VxVM documentation there is an alternative command with a much better output:
[root@server1 ~]# vxassist -g vgp1417 maxsize
Maximum volume size: 109616431104 (104538Mb) |
[root@server1 ~]# vxassist -g vgp1417 maxsize
Maximum volume size: 109616431104 (104538Mb)
Complete information of a disk group (subdisks, plexes, …):
[root@server1 ~]# vxprint -g vgp1417
TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0
dg vgp1417 vgp1417 - - - - - -
dm vgp1417_04 emc0_084b - 70539872 - - - -
dm vgp1417_05 emc0_084f - 70539872 - - - -
dm vgp1417_06 emc0_0853 - 70539872 - - - -
dm vgp1417_07 emc0_0857 - 70539872 - - - -
dm vgp1417_08 emc0_085b - 70539872 - - - -
dm vgp1417_09 emc0_526f - 70539872 - - - -
dm vgp1417_10 emc0_527b - 70539872 - - - -
dm vgp1417_11 emc0_527f - 70539872 - - - -
v lvol1 fsgen ENABLED 6291456 - ACTIVE - -
pl lvol1-01 lvol1 ENABLED 6291456 - ACTIVE - -
sd vgp1417_04-01 lvol1-01 ENABLED 6291456 0 - - -
.
. |
[root@server1 ~]# vxprint -g vgp1417
TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0
dg vgp1417 vgp1417 - - - - - -dm vgp1417_04 emc0_084b - 70539872 - - - -
dm vgp1417_05 emc0_084f - 70539872 - - - -
dm vgp1417_06 emc0_0853 - 70539872 - - - -
dm vgp1417_07 emc0_0857 - 70539872 - - - -
dm vgp1417_08 emc0_085b - 70539872 - - - -
dm vgp1417_09 emc0_526f - 70539872 - - - -
dm vgp1417_10 emc0_527b - 70539872 - - - -
dm vgp1417_11 emc0_527f - 70539872 - - - -v lvol1 fsgen ENABLED 6291456 - ACTIVE - -
pl lvol1-01 lvol1 ENABLED 6291456 - ACTIVE - -
sd vgp1417_04-01 lvol1-01 ENABLED 6291456 0 - - -
.
.
Volumes
Volumes of a disk group:
[root@server1 ~]# vxprint -hvt -g vgp1417
V NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPE
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
SC NAME PLEX CACHE DISKOFFS LENGTH [COL/]OFF DEVICE MODE
DC NAME PARENTVOL LOGVOL
SP NAME SNAPVOL DCO
EX NAME ASSOC VC PERMS MODE STATE
v lvol1 - ENABLED ACTIVE 6291456 SELECT - fsgen
pl lvol1-01 lvol1 ENABLED ACTIVE 6291456 CONCAT - RW
sd vgp1417_04-01 lvol1-01 vgp1417_04 0 6291456 0 emc0_084b ENA
v lvol2 - ENABLED ACTIVE 20971520 SELECT - fsgen
.
. |
[root@server1 ~]# vxprint -hvt -g vgp1417
V NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPE
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
SC NAME PLEX CACHE DISKOFFS LENGTH [COL/]OFF DEVICE MODE
DC NAME PARENTVOL LOGVOL
SP NAME SNAPVOL DCO
EX NAME ASSOC VC PERMS MODE STATEv lvol1 - ENABLED ACTIVE 6291456 SELECT - fsgen
pl lvol1-01 lvol1 ENABLED ACTIVE 6291456 CONCAT - RW
sd vgp1417_04-01 lvol1-01 vgp1417_04 0 6291456 0 emc0_084b ENAv lvol2 - ENABLED ACTIVE 20971520 SELECT - fsgen
.
.
More information of a particular volume:
[root@server1 ~]# vxprint -g vgp1417 -t lvol1
V NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPE
v lvol1 - ENABLED ACTIVE 6291456 SELECT - fsgen |
[root@server1 ~]# vxprint -g vgp1417 -t lvol1
V NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPEv lvol1 - ENABLED ACTIVE 6291456 SELECT - fsgen
Plexes
Plexes of a disk group:
[root@server1 ~]# vxprint -pt -g vgp1417
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODE
pl lvol1-01 lvol1 ENABLED ACTIVE 6291456 CONCAT - RW
pl lvol2-01 lvol2 ENABLED ACTIVE 20971520 CONCAT - RW
pl lvol3-01 lvol3 ENABLED ACTIVE 2097152 CONCAT - RW
pl lvol4-01 lvol4 ENABLED ACTIVE 6291456 CONCAT - RW
pl lvol5-01 lvol5 ENABLED ACTIVE 4194304 CONCAT - RW
pl lvol6-01 lvol6 ENABLED ACTIVE 14680064 CONCAT - RW
pl lvol7-01 lvol7 ENABLED ACTIVE 44040192 CONCAT - RW
pl lvol8-01 lvol8 ENABLED ACTIVE 169869312 CONCAT - RW
pl lvol9-01 lvol9 ENABLED ACTIVE 12582912 CONCAT - RW
pl lvol10-01 lvol10 ENABLED ACTIVE 27262976 CONCAT - RW
pl lvol11-01 lvol11 ENABLED ACTIVE 41943040 CONCAT - RW |
[root@server1 ~]# vxprint -pt -g vgp1417
PL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODEpl lvol1-01 lvol1 ENABLED ACTIVE 6291456 CONCAT - RW
pl lvol2-01 lvol2 ENABLED ACTIVE 20971520 CONCAT - RW
pl lvol3-01 lvol3 ENABLED ACTIVE 2097152 CONCAT - RW
pl lvol4-01 lvol4 ENABLED ACTIVE 6291456 CONCAT - RW
pl lvol5-01 lvol5 ENABLED ACTIVE 4194304 CONCAT - RW
pl lvol6-01 lvol6 ENABLED ACTIVE 14680064 CONCAT - RW
pl lvol7-01 lvol7 ENABLED ACTIVE 44040192 CONCAT - RW
pl lvol8-01 lvol8 ENABLED ACTIVE 169869312 CONCAT - RW
pl lvol9-01 lvol9 ENABLED ACTIVE 12582912 CONCAT - RW
pl lvol10-01 lvol10 ENABLED ACTIVE 27262976 CONCAT - RW
pl lvol11-01 lvol11 ENABLED ACTIVE 41943040 CONCAT - RW
More information of a plex:
[root@server1 ~]# vxprint -g vgp1417 -l lvol1-01
Plex: lvol1-01
info: len=6291456
type: layout=CONCAT
state: state=ACTIVE kernel=ENABLED io=read-write
assoc: vol=lvol1 sd=vgp1417_04-01
flags: busy complete
mediatype: hdd |
[root@server1 ~]# vxprint -g vgp1417 -l lvol1-01
Plex: lvol1-01
info: len=6291456
type: layout=CONCAT
state: state=ACTIVE kernel=ENABLED io=read-write
assoc: vol=lvol1 sd=vgp1417_04-01
flags: busy complete
mediatype: hdd
Subdisks
Subdisks of a particular disk group:
[root@server1 ~]# vxprint -st -g vgp1417
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
SC NAME PLEX CACHE DISKOFFS LENGTH [COL/]OFF DEVICE MODE
sd vgp1417_04-01 lvol1-01 vgp1417_04 0 6291456 0 emc0_084b ENA
sd vgp1417_04-02 lvol9-01 vgp1417_04 6291456 12582912 0 emc0_084b ENA
sd vgp1417_04-03 lvol10-01 vgp1417_04 18874368 27262976 0 emc0_084b ENA
sd vgp1417_04-04 lvol6-01 vgp1417_04 46137344 10485760 4194304 emc0_084b ENA
sd vgp1417_05-01 lvol5-01 vgp1417_05 0 4194304 0 emc0_084f ENA
sd vgp1417_05-02 lvol7-01 vgp1417_05 4194304 31457280 0 emc0_084f ENA
sd vgp1417_05-03 lvol11-01 vgp1417_05 35651584 16155104 0 emc0_084f ENA
sd vgp1417_05-04 lvol7-01 vgp1417_05 51806688 12582912 31457280 emc0_084f ENA
sd vgp1417_06-01 lvol2-01 vgp1417_06 0 20971520 0 emc0_0853 ENA
sd vgp1417_06-02 lvol11-01 vgp1417_06 20971520 25787936 16155104 emc0_0853 ENA
sd vgp1417_06-03 lvol8-01 vgp1417_06 46759456 23780416 114269008 emc0_0853 ENA
sd vgp1417_07-01 lvol8-01 vgp1417_07 0 56312048 0 emc0_0857 ENA
sd vgp1417_07-02 lvol8-01 vgp1417_07 56312048 14227824 138049424 emc0_0857 ENA
sd vgp1417_08-01 lvol3-01 vgp1417_08 0 2097152 0 emc0_085b ENA
sd vgp1417_08-02 lvol4-01 vgp1417_08 2097152 6291456 0 emc0_085b ENA
sd vgp1417_08-03 lvol6-01 vgp1417_08 8388608 4194304 0 emc0_085b ENA
sd vgp1417_08-04 lvol8-01 vgp1417_08 12582912 57956960 56312048 emc0_085b ENA
sd vgp1417_09-01 lvol8-01 vgp1417_09 0 17592064 152277248 emc0_526f ENA |
[root@server1 ~]# vxprint -st -g vgp1417
SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE
SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE
SC NAME PLEX CACHE DISKOFFS LENGTH [COL/]OFF DEVICE MODEsd vgp1417_04-01 lvol1-01 vgp1417_04 0 6291456 0 emc0_084b ENA
sd vgp1417_04-02 lvol9-01 vgp1417_04 6291456 12582912 0 emc0_084b ENA
sd vgp1417_04-03 lvol10-01 vgp1417_04 18874368 27262976 0 emc0_084b ENA
sd vgp1417_04-04 lvol6-01 vgp1417_04 46137344 10485760 4194304 emc0_084b ENA
sd vgp1417_05-01 lvol5-01 vgp1417_05 0 4194304 0 emc0_084f ENA
sd vgp1417_05-02 lvol7-01 vgp1417_05 4194304 31457280 0 emc0_084f ENA
sd vgp1417_05-03 lvol11-01 vgp1417_05 35651584 16155104 0 emc0_084f ENA
sd vgp1417_05-04 lvol7-01 vgp1417_05 51806688 12582912 31457280 emc0_084f ENA
sd vgp1417_06-01 lvol2-01 vgp1417_06 0 20971520 0 emc0_0853 ENA
sd vgp1417_06-02 lvol11-01 vgp1417_06 20971520 25787936 16155104 emc0_0853 ENA
sd vgp1417_06-03 lvol8-01 vgp1417_06 46759456 23780416 114269008 emc0_0853 ENA
sd vgp1417_07-01 lvol8-01 vgp1417_07 0 56312048 0 emc0_0857 ENA
sd vgp1417_07-02 lvol8-01 vgp1417_07 56312048 14227824 138049424 emc0_0857 ENA
sd vgp1417_08-01 lvol3-01 vgp1417_08 0 2097152 0 emc0_085b ENA
sd vgp1417_08-02 lvol4-01 vgp1417_08 2097152 6291456 0 emc0_085b ENA
sd vgp1417_08-03 lvol6-01 vgp1417_08 8388608 4194304 0 emc0_085b ENA
sd vgp1417_08-04 lvol8-01 vgp1417_08 12582912 57956960 56312048 emc0_085b ENA
sd vgp1417_09-01 lvol8-01 vgp1417_09 0 17592064 152277248 emc0_526f ENA
More information of a subdisk:
[root@server1 ~]# vxprint -g vgp1417 -l vgp1417_04
Disk: vgp1417_04
info: diskid=1325653859.61.server1
assoc: device=emc0_084b type=auto
flags: autoconfig
device: path=/dev/vx/dmp/emc0_084bs3
devinfo: publen=70539872 privlen=65536
mediatype: hdd |
[root@server1 ~]# vxprint -g vgp1417 -l vgp1417_04
Disk: vgp1417_04
info: diskid=1325653859.61.server1
assoc: device=emc0_084b type=auto
flags: autoconfig
device: path=/dev/vx/dmp/emc0_084bs3
devinfo: publen=70539872 privlen=65536
mediatype: hdd
Performance monitoring
When having I/O issues the standard command you would use could be iostat:
[root@server1 ~]# iostat -mdx
Linux 2.6.18-274.3.1.el5 (server1) 06/07/2012
Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await svctm %util
cciss/c0d0 6.45 40.26 3.54 25.84 0.05 0.17 15.73 0.82 28.01 5.13 15.07
cciss/c0d0p1 0.00 0.00 0.00 0.00 0.00 0.00 31.97 0.00 21.14 14.42 0.00
cciss/c0d0p2 6.45 40.26 3.54 25.84 0.05 0.17 15.73 0.82 28.01 5.13 15.07
sda 0.00 0.00 0.00 0.00 0.00 0.00 3.98 0.00 1.87 1.87 0.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 9.71 0.00 4.51 4.51 0.00
sdc 0.00 0.00 0.00 0.00 0.00 0.00 9.71 0.00 1.83 1.83 0.00
sdd 0.00 0.00 0.00 0.00 0.00 0.00 9.71 0.00 1.66 1.66 0.00
sde 0.00 0.00 0.00 0.00 0.00 0.00 9.71 0.00 2.36 2.36 0.00
sdf 0.00 0.00 0.00 0.00 0.00 0.00 9.71 0.00 2.51 2.51 0.00
sdg 0.00 0.00 0.00 0.00 0.00 0.00 9.71 0.00 2.34 2.34 0.00
.
. |
[root@server1 ~]# iostat -mdx
Linux 2.6.18-274.3.1.el5 (server1) 06/07/2012Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await svctm %util
cciss/c0d0 6.45 40.26 3.54 25.84 0.05 0.17 15.73 0.82 28.01 5.13 15.07
cciss/c0d0p1 0.00 0.00 0.00 0.00 0.00 0.00 31.97 0.00 21.14 14.42 0.00
cciss/c0d0p2 6.45 40.26 3.54 25.84 0.05 0.17 15.73 0.82 28.01 5.13 15.07
sda 0.00 0.00 0.00 0.00 0.00 0.00 3.98 0.00 1.87 1.87 0.00
sdb 0.00 0.00 0.00 0.00 0.00 0.00 9.71 0.00 4.51 4.51 0.00
sdc 0.00 0.00 0.00 0.00 0.00 0.00 9.71 0.00 1.83 1.83 0.00
sdd 0.00 0.00 0.00 0.00 0.00 0.00 9.71 0.00 1.66 1.66 0.00
sde 0.00 0.00 0.00 0.00 0.00 0.00 9.71 0.00 2.36 2.36 0.00
sdf 0.00 0.00 0.00 0.00 0.00 0.00 9.71 0.00 2.51 2.51 0.00
sdg 0.00 0.00 0.00 0.00 0.00 0.00 9.71 0.00 2.34 2.34 0.00
.
.
Remark:
As stated in man pages if you do not specify interval and count parameter displayed statistics are since server boot time so not really relevant…
[root@server1 ~]# iostat -md -p sda 1 2
Linux 2.6.18-274.3.1.el5 (server1) 06/07/2012
Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 0.00 0.00 0.00 7 0
Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 0.00 0.00 0.00 0 0 |
[root@server1 ~]# iostat -md -p sda 1 2
Linux 2.6.18-274.3.1.el5 (server1) 06/07/2012Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 0.00 0.00 0.00 7 0Device: tps MB_read/s MB_wrtn/s MB_read MB_wrtn
sda 0.00 0.00 0.00 0 0
You can also use nmon.
With the command we have seen in previous chapter you are now able to identify to which disk group/volume/VXVM disks it is associated…
But even more interesting you can use VxVM command to get detailed information of disk using Operation System disk names:
[root@server1 ~]# vxdmpadm iostat show pathname=sdax
cpu usage = 374381us per cpu memory = 1114112b
OPERATIONS BLOCKS AVG TIME(ms)
PATHNAME READS WRITES READS WRITES READS WRITES
sdax 1182902 4756775 328093320 294705430 5.10 20.82 |
[root@server1 ~]# vxdmpadm iostat show pathname=sdax
cpu usage = 374381us per cpu memory = 1114112b
OPERATIONS BLOCKS AVG TIME(ms)
PATHNAME READS WRITES READS WRITES READS WRITES
sdax 1182902 4756775 328093320 294705430 5.10 20.82
[root@server1 ~]# vxdmpadm iostat show dmpnodename=sdax
cpu usage = 375891us per cpu memory = 1114112b
OPERATIONS BLOCKS AVG TIME(ms)
PATHNAME READS WRITES READS WRITES READS WRITES
sdax 1183412 4762187 328098810 295062539 5.10 20.82
sdh 1133886 4773058 324904565 294251856 5.35 20.75 |
[root@server1 ~]# vxdmpadm iostat show dmpnodename=sdax
cpu usage = 375891us per cpu memory = 1114112b
OPERATIONS BLOCKS AVG TIME(ms)
PATHNAME READS WRITES READS WRITES READS WRITES
sdax 1183412 4762187 328098810 295062539 5.10 20.82
sdh 1133886 4773058 324904565 294251856 5.35 20.75
[root@server1 ~]# vxdmpadm iostat show pathname=sdax interval=2 count=2
cpu usage = 374396us per cpu memory = 1114112b
OPERATIONS BLOCKS AVG TIME(ms)
PATHNAME READS WRITES READS WRITES READS WRITES
sdax 1182902 4756777 328093320 294705543 5.10 20.82
sdax 0 0 0 0 0.00 0.00 |
[root@server1 ~]# vxdmpadm iostat show pathname=sdax interval=2 count=2
cpu usage = 374396us per cpu memory = 1114112b
OPERATIONS BLOCKS AVG TIME(ms)
PATHNAME READS WRITES READS WRITES READS WRITES
sdax 1182902 4756777 328093320 294705543 5.10 20.82sdax 0 0 0 0 0.00 0.00
[root@server1 ~]# vxdmpadm -e iostat show pathname=sdax
cpu usage = 374509us per cpu memory = 1114112b
ERROR I/Os
PATHNAME READS WRITES
sdax 0 0 |
[root@server1 ~]# vxdmpadm -e iostat show pathname=sdax
cpu usage = 374509us per cpu memory = 1114112b
ERROR I/Os
PATHNAME READS WRITES
sdax 0 0
As well as using VxVM disk names:
[root@server1 ~]# vxdmpadm -q iostat show dmpnodename=emc0_084b
cpu usage = 374484us per cpu memory = 1114112b
QUEUED I/Os PENDING I/Os
DMPNODENAME READS WRITES
emc0_084b 0 0 0 |
[root@server1 ~]# vxdmpadm -q iostat show dmpnodename=emc0_084b
cpu usage = 374484us per cpu memory = 1114112b
QUEUED I/Os PENDING I/Os
DMPNODENAME READS WRITES
emc0_084b 0 0 0
[root@server1 ~]# vxdmpadm -z iostat show dmpnodename=emc0_084b
cpu usage = 374558us per cpu memory = 1114112b
OPERATIONS BLOCKS AVG TIME(ms)
PATHNAME READS WRITES READS WRITES READS WRITES
sdax 1182910 4756814 328093384 294707962 5.10 20.82
sdh 1133430 4767643 324899821 293884532 5.35 20.76 |
[root@server1 ~]# vxdmpadm -z iostat show dmpnodename=emc0_084b
cpu usage = 374558us per cpu memory = 1114112b
OPERATIONS BLOCKS AVG TIME(ms)
PATHNAME READS WRITES READS WRITES READS WRITES
sdax 1182910 4756814 328093384 294707962 5.10 20.82
sdh 1133430 4767643 324899821 293884532 5.35 20.76
[root@server1 ~]# vxdmpadm -u bytes -z iostat show dmpnodename=emc0_084b
cpu usage = 374888us per cpu memory = 1114112b
OPERATIONS BYTES AVG TIME(ms)
PATHNAME READS WRITES READS WRITES READS WRITES
sdax 1183012 4760612 480699392 707400192 5.10 20.82
sdh 1133494 4771329 3140286976 287247360 5.35 20.76 |
[root@server1 ~]# vxdmpadm -u bytes -z iostat show dmpnodename=emc0_084b
cpu usage = 374888us per cpu memory = 1114112b
OPERATIONS BYTES AVG TIME(ms)
PATHNAME READS WRITES READS WRITES READS WRITES
sdax 1183012 4760612 480699392 707400192 5.10 20.82
sdh 1133494 4771329 3140286976 287247360 5.35 20.76
You can display information of controllers to see if your multipathing policy is working well:
[root@server1 ~]# vxdmpadm iostat show groupby=ctlr ctlr=c0
cpu usage = 374524us per cpu memory = 1114112b
OPERATIONS BLOCKS AVG TIME(ms)
CTLRNAME READS WRITES READS WRITES READS WRITES
c0 135753949 218251362 3572140375 2841606614 7.62 7.10
[root@server1 ~]# vxdmpadm iostat show groupby=ctlr ctlr=c2
cpu usage = 374541us per cpu memory = 1114112b
OPERATIONS BLOCKS AVG TIME(ms)
CTLRNAME READS WRITES READS WRITES READS WRITES
c2 135349009 215663711 3357968941 2737547261 7.96 6.92 |
[root@server1 ~]# vxdmpadm iostat show groupby=ctlr ctlr=c0
cpu usage = 374524us per cpu memory = 1114112b
OPERATIONS BLOCKS AVG TIME(ms)
CTLRNAME READS WRITES READS WRITES READS WRITES
c0 135753949 218251362 3572140375 2841606614 7.62 7.10
[root@server1 ~]# vxdmpadm iostat show groupby=ctlr ctlr=c2
cpu usage = 374541us per cpu memory = 1114112b
OPERATIONS BLOCKS AVG TIME(ms)
CTLRNAME READS WRITES READS WRITES READS WRITES
c2 135349009 215663711 3357968941 2737547261 7.96 6.92
Those interesting information can also be display at disk group level:
[root@server1 ~]# vxstat -u m -g vgp1417
OPERATIONS BLOCKS AVG TIME(ms)
TYP NAME READ WRITE READ WRITE READ WRITE
vol lvol1 827072 413478 17059m 29032m 1.51 7.34
vol lvol2 761459 357294 3465m 2781m 4.46 10.80
vol lvol3 9518191 7008657 175801m 56452m 0.26 0.02
vol lvol4 610864 15518552 259528m 176116m 1.86 0.19
vol lvol5 170534 303104 886m 998m 4.50 8.68
vol lvol6 2646674 2379866 69607m 16601m 1.19 1.66
vol lvol7 16854588 20012885 206030m 119118m 0.10 0.16
vol lvol8 79042007 128500851 899927m 689835m 0.04 0.02
vol lvol9 301556 4788315 2090m 74546m 3.00 0.83
vol lvol10 980251 4487076 287228m 193539m 3.20 0.62
vol lvol11 82761 10693 338m 82m 2.00 11.79 |
[root@server1 ~]# vxstat -u m -g vgp1417
OPERATIONS BLOCKS AVG TIME(ms)
TYP NAME READ WRITE READ WRITE READ WRITE
vol lvol1 827072 413478 17059m 29032m 1.51 7.34
vol lvol2 761459 357294 3465m 2781m 4.46 10.80
vol lvol3 9518191 7008657 175801m 56452m 0.26 0.02
vol lvol4 610864 15518552 259528m 176116m 1.86 0.19
vol lvol5 170534 303104 886m 998m 4.50 8.68
vol lvol6 2646674 2379866 69607m 16601m 1.19 1.66
vol lvol7 16854588 20012885 206030m 119118m 0.10 0.16
vol lvol8 79042007 128500851 899927m 689835m 0.04 0.02
vol lvol9 301556 4788315 2090m 74546m 3.00 0.83
vol lvol10 980251 4487076 287228m 193539m 3.20 0.62
vol lvol11 82761 10693 338m 82m 2.00 11.79
Remark
-d, -p, -s and -u m options are interesting to display information at different granularity level. -r option can also be used to reset statistics and focus on current period only (to remove boot time statistics or reorganization operations)…
There is also vxtrace command to trace operations on volumes but I have, most probably, not really understood its added value as the output is too much verbose and does not contains any synthetic information:
[root@server1 ~]# vxtrace -g vgp1417 -o disk lvol8
57 START write disk emc0_0857 op 0 block 50782432 len 16
57 END write disk emc0_0857 op 0 block 50782432 len 16 time 3
58 START write disk emc0_0857 op 0 block 195779 len 1
58 END write disk emc0_0857 op 0 block 195779 len 1 time 1
59 START write disk emc0_0857 op 0 block 1246640 len 16
59 END write disk emc0_0857 op 0 block 1246640 len 16 time 2
.
. |
[root@server1 ~]# vxtrace -g vgp1417 -o disk lvol8
57 START write disk emc0_0857 op 0 block 50782432 len 16
57 END write disk emc0_0857 op 0 block 50782432 len 16 time 3
58 START write disk emc0_0857 op 0 block 195779 len 1
58 END write disk emc0_0857 op 0 block 195779 len 1 time 1
59 START write disk emc0_0857 op 0 block 1246640 len 16
59 END write disk emc0_0857 op 0 block 1246640 len 16 time 2
.
.
Remark:
I/O statistics are gathered by default but when you system is stable or when you have high I/O demand it may be interesting to deactivate it to maximize throughput. To check if statistics are gathered:
[root@server1 ~]# vxtune vol_stats_enable
1 |
[root@server1 ~]# vxtune vol_stats_enable
1
Then to de-activate them, as well as for DMP part (not possible to check if there are activated or not):
[root@server1 ~]# vxtune vol_stats_enable 0
[root@server1 ~]# vxdmpadm iostat stop |
[root@server1 ~]# vxtune vol_stats_enable 0
[root@server1 ~]# vxdmpadm iostat stop
If you have slow response from a device group it may be also interesting to check if you have running operation in background:
[root@server1 ~]# vxtask -h list
TASKID PTID TYPE/STATE PCT PROGRESS |
[root@server1 ~]# vxtask -h list
TASKID PTID TYPE/STATE PCT PROGRESS
Even if default VxVM parameters suit most workloads you can change them using vxdmpadm command. To display tunable VxDMP parameters use:
[root@server1 ~]# vxdmpadm gettune all
Tunable Current Value Default Value
------------------------------ ------------- -------------
dmp_cache_open on on
dmp_daemon_count 10 10
dmp_delayq_interval 15 15
dmp_enable_restore on on
dmp_fast_recovery on on
dmp_health_time 60 60
dmp_log_level 1 1
dmp_low_impact_probe on on
dmp_lun_retry_timeout 0 0
dmp_path_age 300 300
dmp_pathswitch_blks_shift 9 9
dmp_probe_idle_lun on on
dmp_probe_threshold 5 5
dmp_queue_depth 32 32
dmp_restore_cycles 10 10
dmp_restore_interval 300 300
dmp_restore_policy check_disabled check_disabled
dmp_retry_count 5 5
dmp_scsi_timeout 20 20
dmp_sfg_threshold 1 1
dmp_stat_interval 1 1
dmp_monitor_fabric on on
dmp_monitor_osevent on on
dmp_native_support off off |
[root@server1 ~]# vxdmpadm gettune all
Tunable Current Value Default Value
------------------------------ ------------- -------------
dmp_cache_open on on
dmp_daemon_count 10 10
dmp_delayq_interval 15 15
dmp_enable_restore on on
dmp_fast_recovery on on
dmp_health_time 60 60
dmp_log_level 1 1
dmp_low_impact_probe on on
dmp_lun_retry_timeout 0 0
dmp_path_age 300 300
dmp_pathswitch_blks_shift 9 9
dmp_probe_idle_lun on on
dmp_probe_threshold 5 5
dmp_queue_depth 32 32
dmp_restore_cycles 10 10
dmp_restore_interval 300 300
dmp_restore_policy check_disabled check_disabled
dmp_retry_count 5 5
dmp_scsi_timeout 20 20
dmp_sfg_threshold 1 1
dmp_stat_interval 1 1
dmp_monitor_fabric on on
dmp_monitor_osevent on on
dmp_native_support off off
Remark:
You may also want to use vxbench available in VRTSspt package…
References
About Post Author
administrator
Find more about me on social media.
les says:
great information. Thanks!
Yannick Jaquier says:
Welcome and happy to have helped !
Fred says:
Hi Yannick,
Occassionally, found this a bit old article.
I have HP MSA 2050 storage and connected my servers to them via iSCSI ports. OS is RHEL 7.2.
I have VCS installed and the problem is that VCS services loaded before ISCSI finished scanning paths. So VxDMP not aware about LUNs.
I can refresh it manually with “vxdctl enable” or “vxdisk scandisks”. But I would like to have more standard solution.
May be you have some tricks for that.
Thank you in advance.
BR,
Fred
Yannick Jaquier says:
Hi Fred,
Never worked on this before so best trick I can give is to open a call to Veritas support…
Thanks,
Yannick.
MoPatel says:
Thank You –
Super helpful for anyone
….and i’m a newbie 🙂
Yannick Jaquier says:
Thanks to you for stopping by and nice comment !
Arun Gohil says:
Very good information . However I have a queston .
when you have two sites ( Site A & Site B ) with SRDF Replication how can you tell when the disks are imported to and writed enabled at which site ?
Help appreciated – Thank You .
Arun Gohil
Yannick Jaquier says:
Thanks for stopping by. Honestly I have ever dealt with that configuration as it is exactly the part I’m expecting to be handle properly by cluster product in use. Where I work we massively use Veritas Cluster Server (VCS) with SRDF replication as you mentionned. On your slave node you should see no active volume group (they will be imported at cluster switchover/failover) so nothing should be writable by default…
Youngmo Eugene Kwon says:
Thank you for your well organized and summarized document.
This is very helpful to me. Thanks again.
pradeep says:
Very good information , Thanks Mr.Yannick
Bucky2003 says:
Great information. Very helpful when troubleshooting performance issues.
shyam says:
Very good information , Thanks Mr.Yannick
Yannick Jaquier says:
Thanks to you for stopping by ! 🙂
Yannick.