*********** WARNING ***********
If the box does not have a local veritas license, you will need to
purchase one for EMC to have Veritas functionality (especially with DMP).
The issue is the FREE Local Array license.
Do a 'vxlicense -p' on the node, if you get only licenses that have
"Feature name:" of "PHOTON" or "SSA" or "D100"
You will need to buy Volume Manager licenses for that node.
-Install newer patches and newer firmware - if needed.
-Install 2 HBAs, make them on their own Sbus channels on the I/O boards if Sbus,
otherwise on PCI watch that they only go into 5V 64Bit slots.
If Sbus - use JNI/fcaw boards
If PCI - use Emulex/lpfc boards
Lay 2 Fiber cables, labeled "$HOST 0" and "$HOST 1".
Connect one end to the host as follows:
If Sbus, then first numbered slot should be slot 0, if when booting
the first fcaw number is higher, then swap cable numbers so
the lower fcaw number has cable 0, cable 1 to higher numbered.
If PCI, then lower numbered PCI slot should be 0 EXCEPT on the
Sun 250 - they are numbered opposite (higher slot number is
really the first PCI slot and should have cable 0).
Do not cable to switches yet, only to host. The reason here is if more
than one machine/host is being added, you will see many "free" WWPN on
a switch and you may get confused as to which ones are your host's cables.
***** Do/Run everything as ROOT **********
***** Make sure you are in KSH for commmands to work *****
-Load drivers
cd /nfs/packages/emc/symm/drivers
-lpfc for PCI
cp ./emulex/lpfc-sparc.tar /tmp
cd /tmp
tar xvf ./lpfc-sparc.tar
pkgadd -d . lpfc.1
answer NO for "IP networking over Fibre Channel", yes to all else
-fcaw for SBUS
pkgadd -d ./jni/fcaw.pkg
defaults for all prompts
-Modify /etc/system
-Note: the fcaw and lpfc add lines to the /etc/system already,
but the following must be there for EMC software.
**** use double greater than signs to append *****
-if non-PowerPath
cp /etc/system /etc/system.`date '+%m%d%y'`
cat /nfs/packages/emc/symm/conf-files/system >> /etc/system
-if PowerPath
cp /etc/system /etc/system.`date '+%m%d%y'`
cat /nfs/packages/emc/symm/conf-files/system-powerpath >> /etc/system
**** NOTE: if you install VXFS after this, you will need to redo
**** /etc/system value: set lwp_default_stksize=0x6000
**** as VXFS install reduces the value (you will two entires
**** of this symbol - delete one, make sure it is the higher value
-Load EMC s/w
--ESN
/nfs/packages/emc/symm/sw/esn_mgr/setup.sh
-If ESN manager
choose #1
else
choose 3 & 4 (volume logix and fiber channel)
--SYMCLI
mkdir /tmp/cli; cd /tmp/cli
tar xvf /nfs/packages/emc/symm/sw/symcli/symcli.tar
cd /tmp/cli/clirtcm/scripts; ./emc_install
Accept default directories
Choose 'i'
Choose '1'
/usr/symcli/bin/symlmf
Choose 'y'
Choose '1'
enter: 7FBE-9150-E0BC-9CCD
Choose 'n'
/usr/symcli/bin/symcfg discover
echo "License Key: CC4C-23E3-84CA-96CE SYMAPI Feature: MAPPING" >> \
/usr/emc/API/symapi/config/symapi_licenses.dat
--ECC for unix:
cp -r /nfs/emc/symm/sw/ecc /tmp
AS ROOT:
(cd /tmp/ecc/INSTALL; ./emc_install)
install ECC
select "1" for Tier-based
accept defaults for directory
"i" for install
select "02" for Two-Tier ECC AGENT Configuration
key: ak-361-440-1945
yes to symcfg
cd /etc/rc3.d
ln -s /usr/emc/ECC/eccagents S98emc_eccagents
/etc/rc3.d/S98emc_eccagents start
-cp over (lpfc.conf || fcaw.conf) && sd.conf TO /kernel/drv
if PCI
cp -p /kernel/drv/sd.conf /kernel/drv/sd.conf.`date '+%m%d%y'`
cp -p /kernel/drv/lpfc.conf /kernel/drv/lpfc.conf.`date '+%m%d%y'`
cd /nfs/packages/emc/symm/conf-files/lpfc
cp lpfc.conf /kernel/drv
cp sd.conf /kernel/drv
*********************************************************************
************************ WARNING!!! *******************************
*********************************************************************
Do not use CAPITAL letters in the lpfc.conf file for WWPN or WWNN
*********************************************************************
*********************************************************************
*************** NOTE !!! NOTE !!! NOTE !!! ************************
*********************************************************************
The Emulex/lp8000 have 10000xxxxxx as WWPN
The JNI/fcaw have 20000xxxxxx as WWPN
The Symm has 50000xxxxxx as WWPN
They are reverse of each other on this, WATCH out.
*********************************************************************
if PCI
two things to do:
-put in the correct WWPN of the Symm - switchshow on the switches, view port 15
-put in the FAs as a comment above the fcp-bind-WWPN
vi /kernel/drv/lpfc.conf (search for fcp-bind-WWPN)
these are probably fine and don't need action
check the controller numbers (grep lpfc /var/adm/messages)
look for lpfc0 and lpfc1
vi /kernel/drv/sd.conf (check the hba="" fields)
if SBUS
if non-PowerPath
cd /nfs/packages/emc/symm/conf-files/fcaw
cp fcaw.conf /kernel/drv
cp sd.conf /kernel/drv
if PowerPath
cd /nfs/packages/emc/symm/conf-files/fcaw/powerpath
cp fcaw.conf /kernel/drv
cp sd.conf /kernel/drv
vi /kernel/drv/sd.conf
check the controller numbers (grep WW /var/adm/messages)
change the wwn="numbers" (These are the FA WWPN)
(check the hba="" and wwn="" fields)
You will find the FA WWPN by doing a switchshow on the proper switch
and get the number from port_15 (usually), starts with "50:06:04"
also check that the fcaw# numbers are correct - look for
fcaw0 and fcaw1
put in the FAs as a comment above the fcp-bind-WWPN
-Do a reconfigure reboot
touch /reconfigure
init 6
-Zone switches
**** WARNING ***
One FA is for PowerPath the other is for DMP. Watch which one you use
as they have the Common Serial Number bit turned on for the particular
type they support, and not turned on for the other types (so you will
see multiple disks that are the same disk).
Cable the fiber to the switches according to the EMC layout document.
The host side should already be done, check if not sure.
Look for the broacade switch configs.
telnet to brocade switches (listed on EMC switch config doc)
The prompt will be similar to: switch-#>
To get help use: zonehelp
>switchshow
EXAMPLE: >alicreate "host_port", "WWPN of hostport name"
EXAMPLE: >alicreate "myhost_0", "10:00:00:00:c9:26:ab:80"
>alicreate "xxx", "xxx"
WWPN is listed on the port number of the host you
connected to the switch, if something else is listed,
then host is not cabled or not talking to the switch
>cfgshow
Your changes will be listed at the top in the
"Defined configuration" and not yet in the "Effective configuration:";
It will get to the Effective after you enable it later.
EXAMPLE: >zonecreate "host_port_symm_fa", "host_port;symm_fa"
EXAMPLE: >zonecreate "myhost_0_2162_4a", "myhost_0;symm2162_4a"
>zonecreate "xxx", "xxx"
You will get the FA for the host/port from the EMC docs.
>cfgshow
EXAMPLE: >cfgadd "version-date", "myhost_0_2162_4a"
>cfgadd "xxx", "xxx"
Use the line "cfg:" under "Effective configuration"
as the first argument (it is the config file),
the second argument is the new zone you are adding.
EXAMPLE: >cfgenable "version-date"
>cfgenable "xxx"
>cfgshow
You will always get about 8-10 lines of messages on the next command.
You can ignore them. You cannot ignore messages about not finding aliases/zones.
>cfgsave
-Do a reconfigure reboot
touch /reconfigure
init 6
-See if you get the 2-6.5 GB Volume Logix disks
format < /dev/null | more
-Look for the two paths
cd /usr/emc/VolumeLogix
./vcmfind
############ BEGIN: first server to go on a symm #################
### /usr/emc/VolumeLogix ###
### . ./setup.profile ###
### ESNM_HOME=/usr/emc/ESN_Manager ###
### FZ_ROOT=/usr/emc/FibreZone ###
### VLOGIX_DATA=/usr/emc/VolumeLogix ###
############ END: first server to go on a symm ##################
Log into ESN manager node
-set display to your workstation
display NODE
cd /usr/emc/ESN_Manager
./esnmgr.sh
Manage tab
########### BEGIN: first server to go on a symm #######################
### Volume Access ###
### let it run... takes time ###
### Tools ###
### expert ###
### Volume log ###
### databse ###
### expand symm id, then database, initialize database ###
########### END: first server to go on a symm #####################
select LEFT frame object: volume access
NOTE: if this is the second time through (another server done, or
redoing this one) this will take a few minutes to run
Right click over the each of the fabric clouds and do an import.
This will get the new configs from the switches.
Also do a: File Menu -> rediscover
This will redo the LUN masking display.
Find the two HBAs WWPNs that are listed in the same column as the
other hostnames. Right click on them and do a "manage names".
Remove the WWPN (xx:xx:xx...) and type in the hostname.
Do this for each HBA (give the same hostname for each, it will
join them under a single object).
Note: you will get the drives to select from EMC document. Look for
the LUN assignments table.
Note: The drives numbering is in hex, under Symm Volumes column (not LUNS)
select host
select drives
NOTE: also select at least one, if not two, gatekeepers.
They are from 0-a disks (the first disks on the list), they
are small if you look at the properties. They will already
be "boxed" as other nodes will see them.
NOTE: make sure you mouse is over one of the disk objects
NOTE: disks that are underlined are taken by other nodes
disks that are boxed are taken by this node
disks that are boxed with dashed lines need
better zoning
disks that are are neither boxed or underlined
are free
RIGHT MOUSE over disk object(s) selected and select "Add Access"
select popup button: ok
select LEFT frame object: tasks
slect run
select popup object: symmetrix (pick correct Symm ID)
select ok
done - Now back on host being added to symm
Solaris 2.6: drvconfig;disks
Solaris 2.7 and >: devfsadm -C -c disks
-You should now see each LUN twice in format or "iostat -E".
If not, then there is pathing or zoning problems.
-Label the drives
for i in $(format < /dev/null | \
awk '/configured with capacity of 8.51GB/ {print $1}' | \
sed -e s/://)
do
format $i << EOF
label
y
EOF
done
-Reinitialize veritas
vxdctl enable
-Make sure DMP is working, you should only see disks on one controller.
If the disks show up twice, then DMP is NOT working, see the sunsolve
docs for turning DMP back on.
vxdisk list
vxdisk list someCTD-from-previous-list
#see that there are two paths
vxdmpadm listctlr all
vxdmpadm listctlr ctlr=c6
vxdmpadm getsubpaths ctlr=c6
vxdmpadm listenclosure all
vxdmpadm getdmpnode enclosure=emc0
to disable:
vxdmpadm disable ctlr=/sbus@3,0/SUNW,socal@0,0/sf@1,0
to enable:
vxdmpadm enable ctlr=/sbus@3,0/SUNW,socal@0,0/sf@1,0
NOTE: For Powerpath, you will see all disks twice - so ignore this step
You should make sure the bit is set on the symm by doing a "syminq"
and making sure the S/N field has 000 for the last 3 digits, if you
have any numbers, then you do not have a unique S/N for each symm disk.
vxdisk list
-Add disks by controller, take default names
vxdiskadm
NO for encapsulation
YES for initilization
NODE=`hostname`;export NODE
/nfs/emc/binvx/vx-$NODE-config
-Check over system
-Roll the messages so the format errors don't show up
/usr/lib/newsyslog
- Refresh the zoning information in ESN Manager
- Since you have made changes to the zoning information from the Brocade command line ESN Manager will display an exclamation point over the fabric icons in the topology overview. You need to refresh the zoning information to reconcile this.
- In ESN Manager (for each fabric)
- right click on the fabric icon
- select "Refresh Zoning"
- Say yes to Refresh Zoning confirmation
-LATER: After concurrence with DBAs:
-whatever is the local data disk group - Not rootdg and Not $NODE
vxdg export datadg
Add to Work Load Analyzer for data collection, mimic other existing nodes.