solaris NFS performance wierdness

Spanky new Sun X4600 box. Solaris 10u4. Multipathed e1000g GB interfaces. NFS-mounted volume, totally default.

$nfsstat -m /package/orabackup
/package/orabackup from nike:/vol/orabackup/dionysus-sbr
 Flags:         vers=4,proto=tcp,sec=sys,hard,intr,link,symlink,acl,rsize=32768,wsize=32768,retrans=5,timeo=600
 Attr cache:    acregmin=3,acregmax=60,acdirmin=30,acdirmax=60

$ /opt/filebench/bin/filebench
filebench> load webserver
filebench> run 60
IO Summary:      255700 ops 4238.6 ops/s, (1366/138 r/w)  23.1mb/s,   4894us cpu/op,  66.0ms latency

mutter, grumble,... remount the NFS vol with -v3

$ nfsstat -m /package/orabackup
/package/orabackup from nike:/vol/orabackup/dionysus-sbr
 Flags:         vers=3,proto=tcp,sec=sys,hard,intr,link,symlink,acl,rsize=32768,wsize=32768,retrans=5,timeo=600
 Attr cache:    acregmin=3,acregmax=60,acdirmin=30,acdirmax=60

$ /opt/filebench/bin/filebench
filebench> load webserver
filebench> run 60

IO Summary:      4397877 ops 72839.3 ops/s, (23495/2351 r/w) 396.4mb/s,    221us cpu/op,   3.1ms latency

What the … ? The default configuration for an NFS 4 mount on this box appears to be 20 times slower than the equivalent V3 mount. How can this be right? Either there’s something very wierd going on with our network topology, or there’s something badly broken about the way the mount is configured. Either way, it’s beyond me to work out what it is. NFS 3 ain’t broken (well, not very) so, unless Sun support can offer some illumination we’ll be sticking with that.

Chris May : 24 Sep 2007 11:12 |  Tags: Solaris Tech |  Comments (1) |  Report a problem