July 14, 2008

Opensolaris adventure part 2: network and mirroring

Follow-up to Opensolaris adventure, part 1 from Secret Plans and Clever Tricks

OK; this bit wasn’t quite as smooth…

first off, networking. I used this blog post for guidance here. It doesn’t seem like NWAM is really aimed at static-IP server configuration, so the ‘traditional’ approach looks most appropriate

pfexec bash # cheat
svcadm disable nwam
svcadm enable network/physical:default

cat {hostname} > /etc/hostname.bge0
vi /etc/inet/resolv.conf # add in dns stuff
vi /etc/nsswitch.conf # change 'hosts' entry to 'files dns'
vi /etc/hosts # add IP address for my hostname
vi /etc/defaultrouter #add default router
vi /etc/netmasks #add netmasks

err.. ok.. what now? I tried ifconfig nge0 down;ifconfig nge0 up, but nge0 was stuck resolutely at . I probably should have tried svcadm restart network/physical:default , but that thought didn’t occur to me until too late. I needed to reboot anyway, so I figured that would probably fix it.

Wrong! On a reboot, only the lo0 interface was present. I was about to try ‘ifconfig plumb nge0’, when it occurred to me to check svcadm. Sure enough, svcs showed me that the network milestone was offline because neither the nwam nor default physical network service was running. A quick look back through my command history showed that when I meant to run svcadm enable network/physical:default I had in fact run disable instead (lazy use of bash command history!). Enabling the service bought nge0 magically into existence, and I had a network.

Phew. Now I can work over ssh rather than over the ilom console…

So, next job; mirror the root partition.

Again using Denis Clarke’s blog entry as a guide, I started off by using format to list the disks for me.

# format
Searching for disks...done

       0. c5t0d0 <DEFAULT cyl 17830 alt 2 hd 255 sec 63>
       1. c5t1d0 <Sun-STK RAID INT-V1.0-136.61GB>
      ... continue for 14 more disks...  
I used fdisk /dev/rdsk/c5t1d0p0 to delete the existing partition, and create a new one of type ‘solaris’. I then used this piece of trickery to copy the disk layout from the root disk onto the new mirror:
 prtvtoc /dev/rdsk/c5t0d0s0 | fmthard -s - /dev/rdsk/c5t1d0s0

c5t0d0s0 was the current root disk, so c5t1d0 seemed like the logical disk to use as a mirror.

I have to confess I have very little idea what that just did. Solaris disk partitioning is something I’ve never really been involved with. As we’ll see in a moment, this may turn out to have unexpected consequences…

# zpool attach  rpool c5t0d0s0 c5t1d0s0
invalid vdev specification
use '-f' to override the following errors:
/dev/dsk/c5t1d0s0 overlaps with /dev/dsk/c5t1d0s2

hmmm… does that matter? I didn’t make a c5t1d0s2; I guess it got copied over from the vtoc on c5t0. Might I end up doing nasty things to my ZFS mirror? gah. Confusing. Let’s just try it with ‘-f’ and see what happens…

...What happens is that it appears to just work. Whether it will subsequently come back to bite me remains to be seen, but after a couple of minutes the mirror is in sync, and all looks good. A reboot comes back fine, with swap working as expected (still only on the one disk) and the ZFS mirror intact.

Next jobs: try out pkg, get the grub config mirrored, find out what the hell I did with fmthard.

- 3 comments by 1 or more people Not publicly viewable

  1. Rusty Wright

    cXtXdXs2 is, if I remember correctly, the partition that spans the entire disk, so that you can, for example, image copy the disk to another disk with the same geometry (heads, tracks, sectors per track).

    I’m puzzled as to why you used t1d0 rather than t0d1. Not because I think you should have, just puzzled because I can’t remember the details. cX is the controller, tX is the target, and dX is the disk. I’m trying to remember what the difference is between the target and disk. Is target the SCSI id of the box, for example, an external box which could have multiple disks in it?

    14 Jul 2008, 21:53

  2. Chris May

    The internet knows the answer :-)


    Technically a target can relate to multiple disks. These disks would share the same target number but have different offsets. I have seen this but it would be more than 15 years ago. Modern disk subsystems do not normally use this

    my X4240 has c5t0d0 through to c5t15d0. Why it should be c5, and what happened to c0 through c4, who can say? :-)

    14 Jul 2008, 23:01

  3. Mike Bleiweiss

    Thanks for documenting this. I found a lot of information here that I couldn’t find anywhere else.

    19 Aug 2008, 17:28

Add a comment

You are not allowed to comment on this entry as it has restricted commenting permissions.

Most recent entries


Search this blog

on twitter...


    Not signed in
    Sign in

    Powered by BlogBuilder
    © MMXIX