November 25, 2010

IPMP and IP Filter

So here is the problem.  You need to configure IP Filter on Solaris to meet the needs of your security policy but you also need to configure IPMP to provide resiliency against network port failure.  The default configuration for IP Filter on Solaris refers to the ingress or egress interface by device name in the ipf.conf file.   You may have, for example an entry like this:

pass in log first level quick on bge0 proto tcp from to any flags S keep state

The problem is, of course that when using IPMP the system will, when required, fail to a different interface. Say, e1000g0.  Of course the ipmp group may *contain* bge0, but we need to be sure we are going to cater successfully for the case where ipmp fails over to the partner interface.  One option would be to duplicate all your IP Filter rules for every interface in the IPMP group.  Clearly not sensible, nor necessary; the solution (post Solaris 10u2) is to use the ipmp_hook_emulation ndd setting for the /dev/ip driver.  For versions less than or equal to S10u2 go and research the /dev/pfil setting qif_ipmp_set.

First, we need to set ipmp_hook_emulation for the /dev/ip driver to 1.

bash-3.00# /usr/sbin/ndd -get /dev/ip ipmp_hook_emulation
bash-3.00# /usr/sbin/ndd -set /dev/ip ipmp_hook_emulation 1
bash-3.00# /usr/sbin/ndd -get /dev/ip ipmp_hook_emulation

Once that is one, we change the interface name in ipf.conf to the ipmp groupname;

Check what it is with ifconfig:

groupname app
e1000g0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3
groupname app
e1000g0:1: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 3

Set appropriately in ipf.conf:

pass in log first level quick on app proto tcp from to any flags S keep state

Now, restart ipfilter and monitor the logfile.  We see the firewall acting on entries for the IPMP group, rather than by specific interface.  Now, if mpathd fails over the interface, the firewall continues to operate as expected.

Nov 24 14:49:00 test-serva ipmon[1160]: [ID 702911] 14:48:59.955694 app0 @0:9 p,123 ->,123 PR udp len 20 76 K-S IN

The second part to this solution is to make the changes persistent.

We can make sure ipfilter will be restarted persistently on boot by enabling the SMF service 'network/ipfilter'. The ndd setting however, will not persist.  How do we set the ndd setting on boot?  One option is a legacy RC script, but this does not leverage the dependency framework of SMF to ensure that the setting is applied before the ipfilter service is started. 

Creating an SMF service to do this that sets itself as a dependent to ipfilter is quite straight forward.  The nice thing about SMF is that you can make ipfilter depend on the new tuning service *without* changing the setup of ipfilter.  To do this, create a new service that configures the ndd parameter(s) you are interested in and set the 'network/ipfilter' service and a *dependent* in the XML manifest;

First, a manifest is required, so create /var/svc/manifest/site/ndd-nettune.xml; notice the dependent ipfilter entry.

<?xml version="1.0"?>

<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">


ident    "@(#)ndd-nettune.xml    1.0    04/09/21 SMI"


<service_bundle type='manifest' name='SUNWcsr:ndd'>





                       <create_default_instance enabled='true' />

                       <single_instance />






                   <service_fmri value='svc:/system/filesystem/minimal' />







                   <service_fmri value='svc:/network/loopback' />







                   <service_fmri value='svc:/network/physical' />







                   <service_fmri value='svc:/network/ipfilter' />






   timeout_seconds='3' >






          timeout_seconds='3' >


               <property_group name='startd' type='framework'>

                   <propval name='duration' type='astring'  value='transient' />


               <stability value='Unstable' />



                           <loctext xml:lang='C'>

                                   ndd network tuning




                           <manpage title='ndd' section='1M'

           manpath='/usr/share/man' />





Run xmllint on the file to check it out:

xmllint /var/svc/manifest/site/ndd-nettune.xml 

Validate it with svccfg

svccfg validate /var/svc/manifest/site/ndd-nettune.xml

Create the script that it will run

bash-3.00# vi /lib/svc/method/ndd-nettune

"/lib/svc/method/ndd-nettune" 15 lines, 453 characters

. /lib/svc/share/
. /lib/svc/share/

# Make sure that the libraries essential to this stage of booting  can be found.
echo "Performing IP Tuning..." >> /tmp/smf.out
/usr/sbin/ndd -set /dev/ip ipmp_hook_emulation 1

# Reset the library path now that we are past the critical stage

Import the new service:

svccfg import /var/svc/manifest/site/ndd-nettune.xml 

Enable if necessary:

svcadm enable ndd-nettune

Check again that the ipmp_hook_emulation is set:

ndd -get /dev/ip ipmp_hook_emulation

Do some brief testing;

Check the value of ipmp_hook_emulation;

bash-3.00# ndd -get /dev/ip ipmp_hook_emulation

Check IP Filter and the ndd service are running;

bash-3.00# svcs ipfilter
online         14:45:45 svc:/network/ipfilter:default
bash-3.00# svcs ndd-nettune
online         14:45:45 svc:/network/ndd-nettune:default

Check the new service restarts ok:

bash-3.00# svcadm restart ndd-nettune
bash-3.00# svcs -a |grep nett
online         14:47:48 svc:/network/ndd-nettune:default

And that the ipfilter service can restart:

bash-3.00# svcadm restart ipfilter   

Disable ndd-nettune;

bash-3.00# svcadm disable ndd-nettune

And attempt to restart ipfilter;

bash-3.00# svcadm restart ipfilter

Excellent, it won't start because the dependent isn't met.

offline        14:48:03 svc:/network/ipfilter:default
bash-3.00# svcs -x
svc:/network/ndd-nettune:default (ndd network tuning)
State: disabled since 24 November 2010 14:48:02 GMT
Reason: Disabled by an administrator.
See: ndd(1M)
See: /var/svc/log/network-ndd-nettune:default.log
Impact: 1 dependent service is not running.  (Use -v for list.)

Finally, enable ndd-nettune and ensure ipfilter now starts.

bash-3.00# svcadm enable ndd-nettune
bash-3.00# svcadm restart ipfilter
bash-3.00# svcs -x

So, we have a method to run both IPMP for network resiliency and IP filter for network security.  The method is clean, easy and persistent across reboots.  This functionality was introduced in Solaris 10 update 3, but I should say that it is changing again.  The next release(s) of Solaris will see the introduction of the Clearview project and the IPMP implementation will create virtual interfaces, such as ipmp0, ipmp1 which can be treated just like any other interfaces for the purposes of technologies such as ipfilter.


- One comment Not publicly viewable

  1. MFV

    Hey Egg, check out ipadm in Sol11 express, no more pratting around with ndd, /etc/hostname. etc.

    26 Nov 2010, 21:48

Add a comment

You are not allowed to comment on this entry as it has restricted commenting permissions.

November 2010

Mo Tu We Th Fr Sa Su
Oct |  Today  | Dec
1 2 3 4 5 6 7
8 9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30               

Search this blog



Most recent comments

  • Started sorting out new vers for sparc:… by Maria MacCallum on this entry
  • Solaris 11.1 is slightly different, I only had to do this before starting ipfilter: svccfg –s setpro… by Maria MacCallum on this entry
  • Really useful information, thanks a lot! I do a NAT using IPFILTER and all was working good, until I… by Nilton on this entry
  • Paul, Thanks for your information. It got me started quickly. I have discovered , thought I've not s… by Tom C on this entry
  • Are you familiar with the Monty Python sketch? by Ian Eiloart on this entry

Blog archive

Not signed in
Sign in

Powered by BlogBuilder