Checkpoint FW-1 VPN Connectivity

with Nortel Contivity

Steve Luke
Atos Origin
14th February 2003
Version Description
0.1 Draft – Steve Luke
0.2 Initial Proof Release – Steve Luke
1.0 Official Release (Added Appendix / TOC) – Steve Luke

CP FW-1 to Nortel Contivity VPN Configuration -2-

Table of Contents

Table of Contents..............................................................................................3
Introduction......................................................................................................4
Test Environment..............................................................................................4
Checkpoint FW-1 Configuration ..........................................................................5
Using NAT with the VPN ..................................................................................11
Nortel Contivity Configuration ..........................................................................12
Appendix A: Troubleshooting / Issues Experienced...........................................18

CP FW-1 to Nortel Contivity VPN Configuration -3-

Introduction

This document defines the configurations needed for site-to-site VPN connectivity between a
Checkpoint FW-1 firewall and a Nortel Contivity Extranet VPN switch.

Test Environment

Checkpoint

Sun Netra T1 UltraSparc IIi

CPU 440Mhz
512Mb RAM
Solaris 2.6 (Latest Patches per DIBBS)
Checkpoint FW-1 ver 4.1 SP6
FM License with 3DES Encryption

Nortel Contivity

Nortel Contivity 600 Switch

Contivity OS ver V03_60.45
BIOS P03
Celeron 300 Mhz
128Mb RAM
2Gb HDD

Schematic

Checkpoint FW-1 Nortel Contivity

VPN Tunnel

LAN LAN

Test PC 1 Test PC 2

CP FW-1 to Nortel Contivity VPN Configuration -4-

Checkpoint FW-1 Configuration

The following are screenshots and descriptions of the configuration necessary to create a VPN
tunnel between a Checkpoint FW-1 firewall and a Nortel Contivity Extranet Switch.

Objects needed to be created on the CP firewall.

• CP FW Object (Naturally!!!)
• Nortel Contivity Object
• Network Object for Local Network (Encryption Domain)
• Network Object for Remote Network (Encryption Domain)

Other Settings

• Encryption tab of the Policy Properties

• Key Exchange Rules
• Encryption Rules

CP FW Object

The checkpoint object should have the external IP address as the main Interface address on
the ‘General’ tab of the object properties. It should also be licensed to this address.
Although this is not essential for some operation, it has been see in test and production
environments to cause connectivity issues for VPN’s and Client VPN’s.

The important settings for this object are the VPN tab settings:

CP FW-1 to Nortel Contivity VPN Configuration -5-

Click Edit under the ‘Encryption Schemes Defined’

It does not matter for this screen, if you have more than one option selected (ie. DES + CAST
+ 3DES, or MD5 + SHA1) as long as DES is selected, MD5 and Support keys…..

Nortel Contivity Object

This object should be set as an External / Gateway on the General tab as follows:

CP FW-1 to Nortel Contivity VPN Configuration -6-

Go to the VPN tab:

CP FW-1 to Nortel Contivity VPN Configuration -7-

Click Edit:

Network Objects

You should create a network object for each of the 2 encryption domains. The size of the
local encryption domain subnet is critical, this is the network object you are going to specify
in the CP FW Object in the VPN tab. If you already use a group object with many networks in
it as your encryption domain, then the network object in that group in which the connections
will be initiated from must be correct. If this is not the same as what is specified in the
Branch Connection Static Routing setting on the Contivity, this will not work.

The remote network object should be specified as the domain in the Contivity FW Object
(VPN tab)

CP FW-1 to Nortel Contivity VPN Configuration -8-

Encryption tab of the Policy Properties

This is where you specify the IKE / Rekey timeouts for the connection. As noted before this
is not essential, but preferable for smooth operation.

CP FW-1 to Nortel Contivity VPN Configuration -9-

Key Exchange Rules

These are the rules that specify how the Initial VPN key exchange and setup is performed
between the 2 firewalls.

Encryption Rules

These rules specify the traffic that should be encrypted and sent down the tunnel.

Right Click on ‘Encrypt’ and select ‘Edit Properties’. Here you will specify how the traffic is
encrypted.

CP FW-1 to Nortel Contivity VPN Configuration - 10 -

Using NAT with the VPN

If you need to use NAT on the Checkpoint side, so that the Nortel side does not see your
private IP addresses, the following should be done:

1. A NAT rule should be created in the NAT Rulebase.

2. Both the real network and NATted subnet range should be included in the encryption
domain group.
3. The ‘Branch Connection Static Routing’ on the Nortel Contivity should be configured
with the NATted subnet range.

Conclusion

That concludes the configuration of the Checkpoint FW-1. You should now push the policy
and bounce the FW (fwstop ; fwstart)

CP FW-1 to Nortel Contivity VPN Configuration - 11 -

Nortel Contivity Configuration

The configuration of the Contivity is broken down into these parts:

• Creation of the local network (Nortel side)

• Branch Office Connection setup

Creation of the local network is as follows (if this is not already done)

1. Go to Profiles – Networks
2. Enter a name and click ‘Create’
3. Edit the properties as below:

4. Close

CP FW-1 to Nortel Contivity VPN Configuration - 12 -

Next go to Profiles – Branch Office

1. Click Add Group

2. Enter a group name

3. OK the name
4. Click ‘Edit’ next to the new Group you have created. Then configure the options as in
the picture below.

CP FW-1 to Nortel Contivity VPN Configuration - 13 -

 5. Use the configure button next to each group to configure it as displayed above.
6. OK these changes.
7. Back on the groups page, click ‘Define Branch Office Connection’
8. Select the group you just made and enter a name for the connection

CP FW-1 to Nortel Contivity VPN Configuration - 14 -

 9. OK the new connection
10. Click Edit next to the new connection

CP FW-1 to Nortel Contivity VPN Configuration - 15 -

 11. Fill in the details, including Local and Remote firewall external IP addresses
12. Click ‘IP’ under ‘Configure Routing’
13. Add the local network (Nortel side) that you previously created
14. Add the remote network (CP side) *** This is the encryption domain on the CP side
and must be exactly the same size as the specified encryption domain on the CP FW
Object.

CP FW-1 to Nortel Contivity VPN Configuration - 16 -

 15. OK this and go back to the Branch Office Connection page.
16. Enter the Pre-Shared Secret in the box provided and OK this.
17. You should now be able to use the ‘Test’ button on the Branch Connection page and
see a successful tunnel establishment.

CP FW-1 to Nortel Contivity VPN Configuration - 17 -

Appendix A: Troubleshooting / Issues Experienced

Replication

On initial build of the test systems, using a standard VPN type configuration on both firewalls,
I received the same error messages as seen in the production environment. These errors
were:

Invalid ID Information

This is caused by a subnet size mismatch in the encryption domain of the CP Firewall, and the
Branch Office Static Routing section of the Contivity. If the subnet in the domain settings of
the Checkpoint FW object is of a different size to the network specified in the Static Routing
section of the Contivity, Phase 1 of the negotiation will complete, but at Phase 2 Stage 1, the
negotiation breaks down and the Contivity returns and error to the Checkpoint FW.

This is represented in the firewall log as:

11:24:50 keyinst x.x.x.x >daemon src x.x.x.x dst x.x.x.x IKE Log:
Phase 1 completion. DES/MD5/Pre shared secrets Negotiation Id:
471f77ee89fc09ef-7a469014265b7bc8
11:24:50 keyinst x.x.x.x >daemon src x.x.x.x dst x.x.x.x IKE Log:
Received Notification from Peer: invalid id information Negotiation
Id: fbded2df

This shows in the Contivity System log as:

11:21:29 tEvtLgMgr 0 : Security [11] Session: IPSEC[x.x.x.x]

attempting login
11:21:29 tEvtLgMgr 0 : Security [11] Session: IPSEC[x.x.x.x]:26
authenticated using LOCAL
11:21:29 tEvtLgMgr 0 : Security [11] Session: IPSEC[x.x.x.x]:26
bound to group /Base/VPN_Test/CP-VPN
11:21:29 tEvtLgMgr 0 : Security [11] Session: IPSEC[x.x.x.x]:26
authorized
11:21:29 tEvtLgMgr 0 : Security [11] Session: network IPSEC[x.x.x.x-
x.x.x.x] attempting login
*11:21:29 tEvtLgMgr 0 : ISAKMP [13] Invalid ID information in message
from x.x.x.x

The caveat here is that this problem does not show if the connection is initiated from the
Contivity (via the Test VPN option). However, even though both Phase 1 and 2 complete
successfully and the tunnel appears to be up, traffic will still not flow.

NB: This can also occur if ‘Support Key Exchange for Subnets’ is disabled on the CP FW-1.

No Proposal Chosen

CP FW-1 to Nortel Contivity VPN Configuration - 18 -

This error is because of a mismatch in the encryption and authentication settings between
the 2 firewalls. During the tests all options were tried and the error only occurred with
different settings

This appears in the CP FW log as:

12:15:30 keyinst x.x.x.x >daemon src x.x.x.x dst x.x.x.x IKE Log:
Received Notification from Peer: no proposal chosen

It appears in the Contivity System log as:

12:12:38 tEvtLgMgr 0 : ISAKMP [13] No proposal chosen in message from

x.x.x.x

This mismatch can occur because the settings are different for:

• Encryption Scheme (IKE, FWZ etc)

• Key Exchange Encryption (DES, CAST, 3DES)
• Traffic Encryption (DES, DES-40, 3DES)
• Data Integrity (MD5, SHA1)
• Transform (ESP, AH)
• Perfect Forward Secrecy setting*

*The PFS setting also produces some irregular results:

PFS Setting CP Nortel Result

1 On On PFS used, tunnel works
2 Off On Does not work (No Proposal Chosen)
3 On Off PFS used, tunnel works (Should not work)
4 Off Off Tunnel works without PFS

For the exact settings needed to synchronise both firewalls please read the next section on
configuration.

Invalid cookie

This is a very sporadic error, that only occurred twice in production and twice in test. There
are 2 possibilities as to why this happens, a possible error with PFS, or with Rekey timeouts,
both however, are issues relating to the re-establishment of a previously established tunnel,
having changed configuration. This issue does not occur on FW-FW VPN’ s, only with FW-
Other VPN’ s. Checkpoint believe this to be a possible error with encryption domains
(https://support.checkpoint.com/advanced/idsearch.jsp?id=sk6468&QueryText=%28%22inva
lid%22+AND+%22cookie%22%29&) , however this was not the case here. It was an error
that appeared after a change to the Nortel. The two solutions I have found are to bounce
the firewall (fwstop ; fwstart) or to delete the SA’ s from the state tables. The second
solution however, only applies if the policy was pushed before the last tunnel establishment,
as a policy push clears the state table anyway, and backs it up to an ‘old state table’ which
the kernel references for previous connections.

This is not an error relating to our tests, just an error that appears with certain configuration
changes that need a firewall bounce.

Successful Operation

A successful VPN tunnel establishment should appear in the FW log as:

CP FW-1 to Nortel Contivity VPN Configuration - 19 -

15:01:06 keyinst x.x.x.x >daemon src x.x.x.x dst x.x.x.x IKE Log:
Phase 1 completion. DES/MD5/Pre shared secrets Negotiation Id:
7da52c0da9411d43-fc46193060c62f51

15:01:06 keyinst x.x.x.x >daemon proto ip src x.x.x.x dst x.x.x.x

srckeyid 0xc1a32fe2 dstkeyid 0x000f4d8e rule 0 scheme: IKE methods:
Combined ESP: DES + MD5 + PFS (phase 2 completion) for subnet:
x.x.x.x (mask= 255.255.255.0) and for subnet: x.x.x.x (mask=
255.255.255.0)

In the Contivity System Log, this appears as:

16:24:28 tEvtLgMgr 0 : Security [11] Session: IPSEC[x.x.x.x]

attempting login
16:24:28 tEvtLgMgr 0 : Security [11] Session: IPSEC[x.x.x.x]:12
authenticated using LOCAL

16:24:28 tEvtLgMgr 0 : Security [11] Session: IPSEC[x.x.x.x]:12

bound to group /Base/VPN_Test/CP-VPN
16:24:28 tEvtLgMgr 0 : Security [11] Session: IPSEC[x.x.x.x]:12
authorized
16:24:28 tEvtLgMgr 0 : Security [11] Session: network IPSEC[x.x.x.x-
255.255.0.0] attempting login
16:24:28 tEvtLgMgr 0 : Security [11] Session: network IPSEC[x.x.x.x-
255.255.0.0] logged in from gateway [x.x.x.x]
16:24:28 tEvtLgMgr 0 : Security [12] Session: IPSEC[x.x.x.x]:12
physical addresses: remote x.x.x.x local x.x.x.x
16:24:28 tEvtLgMgr 0 : Security [12] Session: IPSEC[-]:13 physical
addresses: remote x.x.x.x local x.x.x.x

The highlighted text above is the name of the Branch Office Connection that was configured
on the Contivity for the CP network. This name must match the one configured.

NB: These Contivity messages do not necessarily indicate a successful connection, if the
failure is on the CP side, you can still see these messages in the log. A successful connection
is indicated by 1) both logs reporting successful, and 2) traffic being passed.

Notes

1. Many documents suggest that the IKE and Rekey timeouts should be synchronised
for this to work, however, in testing the connection worked even with different
timeouts, although this may cause intermittent issues during the connection.
2. Many documents also say that ‘Supports Aggressive Mode’ is not supported, whereas
in the test environment, it made no difference being on or off.

CP FW-1 to Nortel Contivity VPN Configuration - 20 -