poolbind command gives "Device busy" - Solaris Zones

I'm getting this when trying to pin a zone to a specific CPU:
root#my_host:/root> poolbind -p my_pool -i zoneid my_zone
poolbind: binding zoneid my_id to pool 'my_pool': Device busy
I have 8 virtual CPUs (threads) assigned to this pool. Just before this command failed, I successfully assigned the tenth zone to this pool without any errors.
Is it possibly that I'm pinning too many zones to this pool and #11 was the last straw?


New to Solaris zones. How do i setup resource memory and cpu caps?

New to Solaris 10 zones. I'm a little confused surrounding the Sun Solaris 10 Zones, and container managmement documentation I've been reading over the last couple of days.
Here is my situation, I have Solaris 10, 1 global zone and I'm about to setup 3 non-global zones within there. Within the non-global zone, I want to
restrict memory capping and how much memory the entrire zone can use, cpu-shares and how many cpus this zone can use, and swap which is how much swap
this zone can utilize.
I have the following questions in regards to this situation:
1) Do i need to use the rcapd on each and every one of my non-global zones to do the memory capping, or do I just go into my zonecfg and setup
the capped-memory setting for that zone and not run the rcapd.
2) Do I just setup the capped cpu-shares at the global level, and that is all I need to do? If not, what else?
I get confused with a lot of the documentation that Sun puts out since it says I need to use the
poolcfg, modify the /etc/project, etc. Do I need to do that? 
this is my cook book -
memory capping for a zone is easy.
1) setting the zone1 to 1 GB mem and 1 GB swap:
# zonecfg -z zone1
add capped-memory
set physical=1024m
set swap=1g
commitOr as a one-liner:
# zonecfg -z zone1 "add capped-memory; set physical=1024m; set swap=1g; end"(commit is not necessary)
then reboot the zone.
2a) dedicated CPU setting:
# zonecfg -z zone1
add dedicated-cpu
set ncpus=2-4Or as a one-liner:
# zonecfg -z zone1 "add dedicated-cpu; set ncpus=2-4; end"then reboot the zone.
2b) shared CPU setting:
# zonecfg -z zone1
set cpu-shares=30
# zonecfg -z zone2
set cpu-shares=20You get the same result with the following (just in case you've seen this in the documents):
# zonecfg -z zone1
add rctl
set name=zone.cpu-shares
add value (priv=privileged,limit=20,action=none)These configurations need a zone reboot.
If you want to have it immediately activ, run
prctl -n zone.cpu-shares -P -i zone zone1but this is not boot resistant!
To make it boot resistant, run
prctl -n zone.cpu-shares -v 20 -r -i zone zone1hth
How to verify the changes were taken?
If I'm in the reconfigured zone and type psrinfo I see both processors, so it's not apparent that I'm limited to one.
Is that how it's supposed to look?
Also, swap -l shows 8GB of swap, not 1GB.
design8# psrinfo -v
Status of virtual processor 0 as of: 12/10/2008 11:41:48
on-line since 12/02/2008 14:52:08.
The sparcv9 processor operates at 1504 MHz,
and has a sparcv9 floating point processor.
Status of virtual processor 1 as of: 12/10/2008 11:41:48
on-line since 12/02/2008 14:52:06.
The sparcv9 processor operates at 1504 MHz,
and has a sparcv9 floating point processor.
design8# swap -l
swapfile dev swaplo blocks free
/dev/swap 0,0 16 16780208 16780208
I can issue "zonecfg", then "info" to see the changes are configured, but I need
to be in the global zone to see the restricted environment.
zonecfg:design8> info
zonename: design8
zonepath: /export/design8
brand: solaris8
autoboot: true
limitpriv: default,cpc_cpu,proc_clock_highres
ip-type: shared
[max-lwps: 384]
physical: bge0
defrouter not specified
[ncpus: 1.00]
physical: 512M
[swap: 1G]
name: zone.cpu-cap
value: (priv=privileged,limit=100,action=deny)
name: zone.max-swap
value: (priv=privileged,limit=1073741824,action=deny)
name: zone.max-lwps
value: (priv=privileged,limit=384,action=deny) 
well the only reason I can imagine is that psrinfo is not the right tool to check the restriction.
If I configure a zone with say 4GB (half of the available), and check this with prstat -Z then it shows completely correct only 4GB for this zone.
Is that different in your environment?
-- Nick 
Hi Nik,
I need some clarification on dedicated CPUs.
Is it CPU or Core or Thread?
Is it possible to specify any of the above?
Thanks & rgds... Hisham 
Any update on this? I myself am rather curious to know the answer. kstat -m cpu_info is giving the same info that psrinfo -v did, so I do not know if either of them are telling the truth or not.

How to dynamically set max-rss size for a local zone?

Running solaris 10 05/08.
Following situation: Three local zones in a Veritas cluster. The zones
can fail over but normally not all three are on the same physical box.
I want to be able to restrict physical memory usage for the zones in
situations where certain combinations of them end up on the same
physical box. So, I need to be able to dynamically change memory
resources. I know how to do this with, for example locked memory:
prctl -n zone.max-locked-memory -r -v <mem> `pgrep -z <zone name>
but i need to do this for physical memory and I can't see that there
is a zone resource for this? I thought this was added in 05/08? 
You can do this by using rcapd from the global zone
- start rcapd from the global zone:
svcadm enable svc:/system/rcap:default
- set a physical memory caping value for each zone
rcapadm -z your_zone -m max_rss_value
- check the memory use for all the zone capped
rcapstat -gz
You can set the rss capping value in the zone configuration:
# zonecfg -z your_zone
zonecfg:busi-app-prod> add capped-memory
zonecfg:busi-app-prod:capped-memory> set physical=900m
zonecfg:busi-app-prod:capped-memory> end
zonecfg:busi-app-prod> verify
zonecfg:busi-app-prod> commit
zonecfg:busi-app-prod> exit
This will be take in account at the next reboot of this zone.
Have a also a look to rcapadm to tune rcapd

Limit No of CPU's visible in the Non Global Zone

We have 24 CPU Domain with Solaris 10 U4
We have a requirement where we need a non-global zone which shows total no of CPU's as 4 within this domain. It's application licensing requirement.
Can somebody suggest is it possible and if somebody has tried it how can this be done.
Thanks in Advance. 
Create a resource pool, and associate that pool with the zone.
pooladm -e # enable poos
pooladm -s # save current state
poolcfg -c 'create pset pset_fourcpus (uint pset.min = 4; uint pset.max = 4)'
poolcfg -c 'create pool pool_fourcpus'
poolcfg -c 'associate pool pool_fourcpus (pset pset_fourcpus)'
pooladm -c # activate new configuration
In your zone configuration (I assume it already exists)
zonecfg -z your_zone
zonecfg:your_zone> set pool=pool_fourcpus
zonecfg:your_zone> verify
zonecfg:your_zone> commit
zonecfg:your_zone> exit
then restart the zone. 
In this example does this mean that the four processors are only available to zones using this pool? Or is the workload still balanced over the entire resource pool. (i.e. this mechanism just acts as a limiter?)
Assigning CPUs to a processor set removes them the default set. Processes (not necessarily zones) operating against the default set would not have access to them.
As a corollary, you cannot assign all CPUs in a system to processor sets.

Issue adding zpool to zone cluster..

I've tried several different methods of adding a failover zfs filesystem to a zone cluster - all from various documentation I've found on Oracle's website but none seem to work.
Note: This is Sun Cluster 3.3.0 on Solaris 10 Update 9 (with latest recommended patch cluster as of March 2011) Architecture is x86_64.
All disks are visible to each global cluster node. (verified with cldevice)
I create a zpool using 2 disks called test1-u01
At this point I create a zfs filesystem called test1-u01/u01 (Note: I also tried not even creating a zfs filesystem at this point)
#zpool status -v test1-u01
pool: test1-u01
state: ONLINE
scrub: none requested
     test1-u01 ONLINE 0 0 0
     c6t60060E8005B1F4000000B1F4000010A3d0 ONLINE 0 0 0
     c6t60060E8005B1F4000000B1F4000010A4d0 ONLINE 0 0 0
errors: No known data errors
#zfs list | grep test1
test1-u01 112K 97.9G 23K /test1-u01
test1-u01/u01 21K 97.9G 21K /test1-u01/u01
#clzonecluster configure test1
clzc:sczone> add dataset
clzc:sczone:fs> set name=test1-u01
clzc:sczone:fs> end
clzc:sczone:fs> exit
# clresourcegroup create -Z test1 rg-hasp-zfs-u01
# clresourcetype register -Z test1 SUNW.HAStoragePlus
# clresource create -Z test1 -g rg-hasp-zfs-u01 -t SUNW.HAStoragePlus -p Zpools=test1-u01 rs-hasp-zfs-u01
clresource create -Z test1 -g rg-hasp-zfs-u01 -t SUNW.HAStoragePlus -p Zpools=test1-u01 rs-hasp-zfs-u01
clresource: example01:test1 - More than one matching zpool for 'test1-u01': zpools configured to HAStoragePlus must be unique.
clresource: (C189917) VALIDATE on resource rs-hasp-zfs-u01, resource group rg-hasp-zfs-u01, exited with non-zero exit status.
clresource: (C720144) Validation of resource rs-hasp-zfs-u01 in resource group rg-hasp-zfs-u01 on node asgdbput01-test1 failed.
clresource: (C891200) Failed to create resource "test1:rs-hasp-zfs-u01".
Anyone have any ideas? 
The clzc output looks weird...
#clzonecluster configure test1
clzc:sczone> add dataset
clzc:sczone:fs> set name=test1-u01
clzc:sczone:fs> end
clzc:sczone:fs> exit
Did you copy and paste this??? I would expect it to look like this:
#clzonecluster configure test1
clzc:test1> add dataset
clzc:test1:dataset> set name=test1-u01
clzc:test1:dataset> end
clzc:test1> commit
clzc:test1> exit
Also, make sure that you don't have a zpool configured with the same name on each node. There should be just one zpool that can be imported with the name test1-u01 across the entire cluster.
There is only one zpool with this name across the cluster. The only other zpool at all are the rpool's for each of the 3 nodes. Yea I was copy/pasting that from documentation. Here is the actual output for the zone config stuff:
node1 #clzonecluster list
node1 #clzonecluster configure chuck1
clzc:chuck1> info
zonename: chuck1
zonepath: /zones/chuck1
autoboot: true
brand: cluster
ip-type: shared
enable_priv_net: true
address: example-oracle
physical: auto
name: chuck1-u01
root_password: ViL8QkebsbWKc
name_service: "DNS{domain_name=example.com name_server=}"
nfs4_domain: dynamic
security_policy: NONE
system_locale: C
terminal: dtterm
timezone: US/Pacific
physical-host: node1
hostname: node1-chuck1
address: node1-chuck1/24
physical: bnx0
defrouter not specified
physical-host: node2
hostname: node2-chuck1
address: node2-chuck1/24
physical: bnx0
defrouter not specified
clzc:chuck1> exit
The name chuck1-u01 is unique and not used for anything else - not a vip, not a hostname, nothing at all. 
Two things:
1. Were there any errors or messages relating to this in /var/adm/messages on any of the nodes or zone nodes?
2. What happens when you try running the command from within the zone cluster itself without the -Z option to clrs create
I found my problem. I had been using the LUNs for all kinds of ZFS testing and moving them around between nodes, rebuilding zodes, etc.. Turns out zpool was seeing the residual zpool configs from previous zpool allocations. (I was expecting to see this with "zpool status -v") it turns out I could only see this with "zpool import".
My next problem was that It would not let me destroy a zpool unless it was imported. I could not import zpools that now had missing devices (some of which were now being used by active zpools)
Ultimately I had to do an "fdisk -E <device" on each LUN to reset the disk. Once I did that all my problems were gone.
So I learned a valuable lesson when using zpools on shared storage. Good thing this was a proof of concept environment.
Thanks for all the help.

Issue with LDom alternate I/o Domain Setup

Hi All, T5-2 With Latest FirmwareControl Domain / Alternate I/o Domain/ Guest LDom - 11.2 With Latest SRU 3.4.1 From Storage (Sun Storagetek 2540) same LUN is mapped to Control Domain & Alternate I/o Domain and LDOM VDSDEV are created using below command,  ldm add-vdsdev mpgroup=node1_OS /dev/dsk/c0t600A0B800067075200000C720373F696d0s2 node1_OS#primary-vds ldm add-vdsdev mpgroup=node1_OS /dev/dsk/c0t600A0B800067075200000C720373F696d0s2 node1_OS#secondary-vds  VDISK mapped to guest LDom using below command,  ldm add-vdisk vdisk0 node1_OS#primary-vds node1  When i reboot the control Domain am facing below issue, OS is up but not responsive.OS ZFS Pool went to SUSPENDED state and unable to clear this fault, requires reboot to clear the I/o faults on ZFS pool.Network is accessible root#node1:~# zpool status -x pool: rpool state: SUSPENDED status: One or more devices are unavailable in response to IO failures. The pool is suspended. action: Make sure the affected devices are connected, then run 'zpool clear' or 'fmadm repaired'. Run 'zpool status -v' to see device specific details. see: http://support.oracle.com/msg/ZFS-8000-HC scan: none requested config:  NAME STATE READ WRITE CKSUM rpool SUSPENDED 0 23 0 c1d0 ONLINE 0 4 0 root#node1:~# root#node1:~# fmadm faulty root#node1:~# zpool clear rpool cannot clear errors for rpool: I/O error root#node1:~#
HiThis depends highly on proper setup and usage of dedicated PCI on the alternate I/O domain.how is this setup? Best regards,Marcel
Hi, Do you need any command output from control domain / io domain ? After a reboot am seeing below messages so can I say that the vdisk is accessible from both path ? root#node1 # dmesg  | grep vdisk | grep onlineDec  3 19:59:47 node1 vdc: vdisk#0 is online using ldc#3,0Dec  3 19:59:47 node1 vdc: vdisk#1 is online using ldc#5,0root#node1 #  Regards,Muthu
Hi, Do you need any command output from control domain / io domain ? After a reboot am seeing below messages so can I say that the vdisk is accessible from both path ? root#node1 # dmesg  | grep vdisk | grep onlineDec  3 19:59:47 node1 vdc: [ID 625787 kern.info] vdisk#0 is online using ldc#3,0Dec  3 19:59:47 node1 vdc: [ID 625787 kern.info] vdisk#1 is online using ldc#5,0root#node1 #  Regards,Muthu
ldm list-io on the control domain lists the I/O configuration.Best regards,Marcel
Hello  I suggest you to read this doc that explian how to config I/O ldoms if after doing the config explainin in this doc you still have issue, Pleaser open a new SR http://www.oracle.com/technetwork/server-storage/vm/ovmsparc-best-practices-2334546.pdf RegardsEze