You are on page 1of 291

R77 VSX Course

2014 Check Point Software Technologies Ltd.

Course Timetables

9:00

Day 1

Day 2

Day 3

Course Introduction

VSX Clustering

VSX Conversion

vsx_utill

Gaia VS CTX & New Features


(SNMP, Jumbo Frames)

Mgmt. Implementation

Gaia VSX Intro

10:00

11:00

R77VSX Introduction

12:00
13:00

Lunch Break

14:00
VSX Networking

GW Implementation

15:00
Open Questions

16:00

VSX CoreXL Affinity


& Memory RC

Debug & Troubleshooting

17:00

2014 Check Point Software Technologies Ltd.

2
2

R77 Introduction

2014 Check Point Software Technologies Ltd.

Agenda
VSX introduction

VSX Virtual Devices


How to integrate a VSX
infrastructure into my
enterprise network?

What is VSX and why


should I consider it?

VSX Clustering

VSX Management

Is my VSX
infrastructure robust,
scalable and fast?

Is management of a
VSX infrastructure
complex?

Gizmo vs R67

VSX Gizmo

Whats new in Gizmo

What is a
User Space VSX?

2014 Check Point Software Technologies Ltd.

4
4

Why Virtualization?

Hardware Cost Savings

Simplified Security Management

Better availability and scalability

Simplified Security Provisioning

2014 Check Point Software Technologies Ltd.

5
5

VSX Virtual System Extension

What is VSX

A VSX is a Gateway running


several separate firewalls
each protecting a different
network (customer).

A VSX is a Gateway with the


ability to virtualize physical
network components into one
physical gateway.

2014 Check Point Software Technologies Ltd.

6
6

What do we Virtualize?

Networking (IP, Routing table, IP stack)


INSPECT filter (and tables)
Kernel tables
Configuration (global) parameters
Policy (rules, anti-spoofing, etc.)
SIC entities
File handling
CP Registry
And more

2014 Check Point Software Technologies Ltd.

7
7

Virtual Routing and Firewalling

VSX establishes a Virtual Network Environment


consists of multiple virtual devices
Virtual System (VS)

Firewall Module

Virtual System In
Bridge Mode

Firewall Module In
Bridge Mode

Virtual Router (VR)

IP Router

Virtual Switch (V-SW)

Switch

Virtual Cable (warp link)

Network Cable

2014 Check Point Software Technologies Ltd.

8
8

Virtual Devices
Virtual System (VS)

Virtualizing Check Points Firewall

Each Virtual System is a unique routing and security domain

Each Virtual System has its own separate FW properties.

2014 Check Point Software Technologies Ltd.

9
9

VSX virtual devices:


Firewall objects

Virtual System (VS)


Virtual System

Each VS functions as a stand-alone, independent


FW gateway
Interfaces list
IP Addresses
Routing table
ARP table
Dynamic Routing
Configuration
Etc.

Interfaces list
IP Addresses
Routing table
ARP table
Dynamic Routing
Configuration
Etc.

State Table

FW

Layer
3
Security
& VPN

VPN

Policies
Configuration
Parameters
Layer
2
Logging
Configuration

Cluster XL

Dynamic
Routing

SSL VPN

State Table
Security & VPN
Policies
Configuration
Parameters
Logging
Configuration

AUTH
(Client & Session)

2014 Check Point Software Technologies Ltd.

Secure XL

10
10

Layer 2 Virtual Devices


Virtual System in Bridge Mode (VSB)

Firewall capabilities of a Virtual System, Except NAT &VPN

Easier configuration of Virtual Systems.

Does not segment an existing network.

Needs anti-spoofing to be manually defined.

2014 Check Point Software Technologies Ltd.

11
11

Layer 2 Virtual Devices


Virtual Switch (VSW)

L-2 connectivity between Virtual Systems, and to a shared interface.

Maintains a forwarding table with a list of MAC addresses and their


associated ports.

Simplifies configuration of connected Virtual Systems.

2014 Check Point Software Technologies Ltd.

12
12

Virtual Devices
Virtual Router (VR)

independent routing domains within a VSX Gateway

Designed to route traffic between interfaces connected to it.

Protects itself from traffic directed to or originating from it.

All other packets are forwarded according to the route table entries.

2014 Check Point Software Technologies Ltd.

13
13

warp Interfaces

Regular Interfaces
Physical interfaces
Virtual interfaces - VLANS
VSX Gateway introduces a new type of interfaces
warp links interface between component of the VSX gateway

Eth1 (physical interface)


Wrp Interface

Eth0.101

Eth0.100

Eth0 (VLAN Trunk interface)


2014 Check Point Software Technologies Ltd.

14
14

Example: Physical Network Layout

Internet

2014 Check Point Software Technologies Ltd.

17
17

Example: VSX Deployment

VSX

Internet

VS
X

Switch

2014 Check Point Software Technologies Ltd.

18
18

VSX management
SMART
Consoles

3-tier management architecture with


either SmartCenter or Provider-1

Provider-1

SmartCenter

CLI Management: vsx_util


#
#
#
#

vsx_util
vsx_util
vsx_util
vsx_util

vsls
redistribute_vsls
reconfigure
add_member

VSX Gateways

2014 Check Point Software Technologies Ltd.

19
19

VSX management
Provider-1 focus

Main DMS manages the VSX infrastructure


Target DMSs manage one or more Virtual Devices
Multiple concurrent administrators
Granular permissions
Separate object databases

2014 Check Point Software Technologies Ltd.

20
20

VSX - Whats New in R76-R77


1.
2.
3.

4.
5.
6.
7.

8.
9.
10.
11.
12.
13.
14.

VSX Merged to Maintain


Supports most software blades
Runs on Gaia
VSs Infrastructure Segregation
User Mode FW (FWK)
High performance and capacity (64bit & CoreXL)
Support Jumbo Frames
Dynamic routing (routed)
Source based routing
SNMP per VS
Improved CPU and memory monitoring (per VS)
Conversion between GW and VS
OSU zero downtime upgrade
IPv6 support was added (New in R76 vs R75.40VS)

2014 Check Point Software Technologies Ltd.

21
21

VSX - Whats New in R77.10


Gaia OS improved with resolved issues.
Mobile Access support for IPv6, VSX
Integrated Appliance Hardware Diagnostic Tool.
QoS support for SecureXL and CoreXL is included and disabled
by default.
Routing stability fixes and enhancements.

Acceleration enhancements.
VPN enhancements.

2014 Check Point Software Technologies Ltd.

22
22

Software Blades
R77 supports Software Blade Architecture on every
Virtual System

Supporting Software Blades including Firewall,


VPN, Intrusion Prevention (IPS), Identity
Awareness, Application Control, URL Filtering,
*Anti-virus and Anti-bot.
Administrators have the flexibility to configure any
Software Blades with any security policy to any
Virtual System.

* Anti-virus and Anti-bot will be added in the near future.

2014 Check Point Software Technologies Ltd.

23
23

Virtualization and segregation


R67

R77

Resilience

Kernel panic effects all VSs,


and takes minutes to recover

An FWK dying effects one VS, and


takes seconds to recover.

Segregation

All memory shared between


VSs and instances. A bug on
one VS can cause a memory
corruption on another VS.

Separate address spaces for each


FWK. Excellent segregation.

CPU monitoring per


VS.

Resource Control. Not


completely accurate (due to
wasted lock time), and not
standard.

Standard OS tools (top).

RAM monitoring per


VS.

Currently no method. Will


require a lot of code changes.

Standard OS tools (ps)

RAM limiting per VS

Not possible. Will require


exact accounting of
consumption per VS.

Can be easily done.

Changing of CP
global kernel
parameters

Not possible today on a per


VS basis. Global parameters
shared for all VSs.

Can be easily done, per VS.

2014 Check Point Software Technologies Ltd.

24
24

VSX (R67) architecture

fwd
UM

cpd

cplogd

vpnd
vpnd
vpnd

1.

Ioctls ex. policy install


From cpd to fw kernel

All kernel code


had inside
virtualization

KM

Trap example logs


From fw kernel to cplogd

2.

Fw kernel virtualized

VPN kernel virtualized

Ppack virtualized
3.

NIC

Tables per VS
Parameters per
VS or global

Most of the UM
processes were
virtualized
(fwd/cpd/cplogd)
Some were per
VS (vpnd)

NIC

2014 Check Point Software Technologies Ltd.

25
25

R77 VSX architecture


cpd

cpd

cpd

fwd

fwd

fwd

vpnd

vpnd

vpnd

fwk

fwk

fwk

VS

VS

VS

Trap example logs


From fwk to fwd

UM
KM

Ioctls ex. policy install


From cpd to fwk

1.

2.

Firewall dispatcher

3.

Ppack virtualized
NIC

NIC

2014 Check Point Software Technologies Ltd.

Fwk is the fws


kernel code
compiled to a dll
PPK remains
virtualized
I/S to simulate
traps and ioctls,
over TCP
between fwd/cpd
and fwk fwasync_rpc
26
26

CoreXL per VS - 1

You can use CoreXL to increase the performance of the


VSX Gateway. You can also assign each instance to a
specific CPU core using fw ctl affinity command.

You can configure multiple instances for each of


the Virtual Systems

Each firewall instance that you create uses


additional system memory.

Downside, a Virtual System with five instances would


use approximately the same amount of memory as five
separate Virtual Systems.

2014 Check Point Software Technologies Ltd.

27
27

CoreXL per VS - 2

Firewall instances are


configured differently on
VSX Gateway (VS0), and
on Virtual Systems.
VSX Gateway - Use
the CLI to configure
the number of
instances.
Other Virtual Systems
- Use SmartDashboard
to configure the
number of instances.

2014 Check Point Software Technologies Ltd.

28
28

Jumbo Frames Support

VSX in R77supports Jumbo Frames, up to


9,000 MTU on virtual devices:
1.
2.
3.
4.

Virtual System
Virtual Switch
Virtual Router
Virtual System in Bridge Mode

Configuring the MTU on Bond interfaces

Configuring the MTU on Warp interfaces

Configuring the MTU on VLANs interfaces

2014 Check Point Software Technologies Ltd.

29
29

SNMP per VS

There are two modes of SNMP monitoring that you can


use with VSX :

Default mode
- only monitors VS0
VS mode
- supports SNMP monitoring per VS

The per-VS monitoring such as :


- Interface state and statistics
- Policy name
- Policy date

2014 Check Point Software Technologies Ltd.

30
30

Memory Resource control overview

Memory Resource control (fw vsx mstat) gives the


user overview information about:

Memory consumption of the system

Memory consumption per virtual device

2014 Check Point Software Technologies Ltd.

31
31

VSX Memory Resource Control Examples

fw vsx mstat unit B -vs 2-7 sort 3

VSX Memory Status


=================
Memory Total: 1045659648 Bytes
Memory Free: 242528256 Bytes
Swap Total: 2146787328 Bytes
Swap Free: 2146607104 Bytes
Swap-in rate: 0 Bytes
VSID | Memory Consumption
======+====================
3 |
45741252 Bytes
2 |
44537028 Bytes
6 |
44360900 Bytes

fw vsx mstat debug

VSX Memory Status


=================
Memory Total: 1021152.00 KB
Memory Free: 235680.00 KB
Swap Total: 2096472.00 KB
Swap Free: 2096296.00 KB
Swap-in rate: 0.47 KB
VSID |
Private_Clean |
Private_Dirty |
DispatcherGConn |
DispatcherHTab |
SecureXL
======+====================+====================+====================+====================+====================
0 |
13336.00 KB |
121856.00 KB |
0.00 KB |
0.00 KB |
2850.00 KB
1 |
968.00 KB |
39724.00 KB |
0.00 KB |
0.00 KB |
2833.54 KB
2 |
968.00 KB |
39692.00 KB |
0.00 KB |
0.00 KB |
2833.19 KB
3 |
777.00 KB |
41060.00 KB |
0.00 KB |
0.00 KB |
2833.19 KB
4 |
968.00 KB |
39512.00 KB |
0.00 KB |
0.00 KB |
2833.19 KB
5 |
777.00 KB |
39600.00 KB |
0.00 KB |
0.00 KB |
2833.19 KB
6 |
977.00 KB |
39512.00 KB |
0.00 KB |
0.00 KB |
2833.19 KB
7 |
784.00 KB |
39516.00 KB |
0.00 KB |
0.00 KB |
2833.19 KB
8 |
3008.00 KB |
88592.00 KB |
0.00 KB |
0.00 KB |
2833.19 KB

2014 Check Point Software Technologies Ltd.

32
32

VSX CLISH commands

Several new commands were introduced in R75.40VS


such as switching context, assign resources to
specific VSs, and more :

>set virtual-system <vsid>


>add rba role adminRole virtual-system-access 1

All commands related to interfaces or routes


configuration are disabled in CLISH along with
everything else controlled from Smart Dashboard

2014 Check Point Software Technologies Ltd.

33
33

Thank you !
Please proceed to lab 1

2014 Check Point Software Technologies Ltd.

VSX R77 Networking

2014 Check Point Software Technologies Ltd.

Course Timetables

9:00

Day 1

Day 2

Day 3

Course Introduction

VSX Clustering

VSX Conversion

vsx_utill

Gaia VS CTX & New Features


(Conversion, SNMP, JF)

Mgmt. Implementation

Gaia VSX Intro

10:00
11:00

R77 VSX Introduction

12:00
13:00

Lunch Break

14:00
VSX Networking

GW Implementation

15:00
Open Questions
16:00

VSX CoreXL Affinity


&Memory RC

Debug & Troubleshooting

17:00

2014 Check Point Software Technologies Ltd.

36
36

VSX features

Overlapping IP space support


Inter-VS Routing
Unnumbered Interfaces
Routes Propagation
NAT in VSX
Source-Based Routing

2014 Check Point Software Technologies Ltd.

37
37

Overlapping IP space support


Internet

Each Virtual Device Provides end


to end separation of Network and
Security Infrastructure.

VSX supports protected networks


with overlapping IP spaces.

MPLS Core

VSX facilitates connectivity of


overlapping IP spaces.
Customer D
10.10.10.0/24
Customer C

Customer A
10.10.10.0/24

Customer B

10.10.10.0/24

10.10.10.0/24
2014 Check Point Software Technologies Ltd.

38
38

Inter-VS Routing

802.1q

Application Servers
Virtual Switch

802.1q

Database Servers

Virtual Router

Both Web and Application Servers require services from


the Database servers.

Web Servers

Each service requires different security handling.

Each VS handles the specific security requirements of the segment.


Virtual Switches and Routers facilitate inter VS connectivity.
2014 Check Point Software Technologies Ltd.

39
39

Unnumbered interfaces
Unnumbered interfaces In order to reduce the number of IPs used
in a VSX configuration, a Virtual System, when connected to a Virtual
Router, can use the same IP for multiple interfaces.

The external VS interfaces IP


acts as a next hop for the VR

Warp Links
P-T-P
connections

192.168.1.1

172.169.1.1

192.150.2.1

200.128.4.1

Reducing the systems


overall IP addresses
192.168.1.1

172.169.1.1

192.150.2.1

200.128.4.1

Internal Interface
Unnumbered interfaces borrow an IP
address from one of the VSs interfaces

2014 Check Point Software Technologies Ltd.

40
40

Unnumbered Interface Limitations

Limitations
The following limitations apply to Unnumbered
Interfaces:
Unnumbered interfaces must connect to a Virtual
Router.
You can only "borrow" an individual interface IP
address once.
In order to use VPN or Hide NAT, the borrowed
address must be routable.

2014 Check Point Software Technologies Ltd.

41
41

Routes Propagation

NOT Dynamic Routing

Routes can be propagated to


adjacent Virtual Devices.

update Virtual Devices routing


tables with minimal effort.

Virtual Router

Requires the VS to be
connected to VR or VSW

Virtual Switch

2014 Check Point Software Technologies Ltd.

42
42

Propagating routes to Virtual Router

If a Virtual System is connected to a Virtual Router, the


routes are propagated from the VS to the VR in the following
way:

Route on the VS:

Propagated route on the VR:

Destination: SUBNET

Destination: SUBNET

Next Hop:

Next Hop:

GW

2014 Check Point Software Technologies Ltd.

wrpj interface
connecting VR to the VS

43
43

Propagating routes through Virtual Switch

If several Virtual Systems are connected to a Virtual Switch, the routes


are propagated from one VS to the other VSs in the following way:

Original route on the VS:

Propagated route on other VSs:

Destination: SUBNET

Destination: SUBNET

Next Hop:

Next Hop: IP of wrp interface


connecting the propagator
VS to the VSW.

GW

2014 Check Point Software Technologies Ltd.

44
44

Routes Propagation
Simple & Easy configuration through the Interface properties of the VS

Propagating manual route

Propagating automatic route


2014 Check Point Software Technologies Ltd.

45
45

Network Address Translation in VSX

Virtual Systems support


NAT
Hide
Static
Virtual Router

Some configuration
required when
connected to VR

2014 Check Point Software Technologies Ltd.

46
46

Network Address Translation in VSX

NATed addresses ranges should be defined on a Virtual System in


the Topology page > NAT Addresses... dialog.

The ranges are converted to routes and automatically propagated.

2014 Check Point Software Technologies Ltd.

47
47

Network Address Translation in VSX

Virtual System
connected to a
Virtual Switch.
4.0.0.1
192.168.8.1

Same behavior as
regular interfaces

192.168.8.9

Virtual Switch

4.0.0.2
192.168.8.1

4.0.0.9

192.168.8.9

4.0.0.9

192.168.8.9
2014 Check Point Software Technologies Ltd.

48
48

Source-Based Routing

Source-Based Routing:

VSX includes advanced routing capabilities


(policy based routing), which enable the
definition of source-based routing rules on
Virtual Routers.

Advanced routing enables routing according to


source IP address or a combination of source
and destination IP addresses.

Advanced routing rules take precedence over


ordinary routing decisions (both static and
dynamic).
2014 Check Point Software Technologies Ltd.

49
49

Source-Based Routing

Useful in cases where no


VLAN tagging is used.

Internet

192.168.35.1
EVR

192.168.35.4

Each VS is connected to
Internal Virtual Router.

192.168.1.1

VS2

VS1

192.168.1.3

VS3

VSX
Gateway
192.168.1.1

192.168.1.2

192.168.1.3

IVR

192.168.50.4

VR forwarding routing
based on source IP
address.

192.168.1.2

192.168.50.1

Source-Based
Routing

10.50.50.2/24
10.1.1.2/24
10.100.100.2/24

2014 Check Point Software Technologies Ltd.

50
50

Deployment scenarios

Inter-VS connectivity, without an external connection.

Source-based routing with Virtual Switches.

Allowing Customer to manage its security.

Non DMI Replacement.

2014 Check Point Software Technologies Ltd.

52
52

Inter-VS connectivity, without an external


connection

Interconnect Virtual
Systems

No shared interface

Only allowed with VSW

Virtual Switch

2014 Check Point Software Technologies Ltd.

53
53

Source-based routing with Virtual Switches

Another way of using a single


physical interface without
VLAN tagging to connect to
several protected networks is
by connecting Virtual Systems
using a Virtual Switch.

Internet

192.168.35.4

192.168.35.1

192.168.35.2

VS2

VS1

192.168.35.3

VS3

VSX

Source-based routing should


be performed by external
Router.

Gateway
192.168.50.1

192.168.50.2

192.168.50.3

192.168.50.4

Source-Based
Routing

The Router uses source-based


routing to forward traffic to the
relevant Virtual System based
on source IP address.

10.50.50.2/24
10.1.1.2/24
10.100.100.2/24

2014 Check Point Software Technologies Ltd.

54
54

Allowing Customer to manage its security #1

Customers want to
manage their own
security policy.

Configure routing on the


VS, VSX and on the
management.

Set Policy to allow CPMI


and FW connections.

Internet

VSX

Management
interface

Management P-1

VS
Virtual Switch

SmartDashboard

2014 Check Point Software Technologies Ltd.

55
55

Allowing Customer to manage its security #2

Another solution to the


same problem

The VSW is directly


connected to the mgmt
network.

Configure routing on the


VS and the management
server.

Internet

VSX

Management
interface

Management P-1
VS
Virtual Switch

Policy changes are


required only on the VS.

SmartDashboard

2014 Check Point Software Technologies Ltd.

56
56

Non DMI Replacement


Internet

Internet

Management + External
interface

External Interface

VSX

VSX

Non Dedicated Management Interface

Management
interface

Dedicated Management Interface

Check Point recommends no to use Non-DMI


deployments
Above is a more elegant solution for this need.
2014 Check Point Software Technologies Ltd.

57
57

Thank you !
Please proceed to lab 2,3

2014 Check Point Software Technologies Ltd.

VSX
CoreXL and CPU affinity

2014 Check Point Software Technologies Ltd.

Course Timetables

9:00

Day 1

Day 2

Day 3

Course Introduction

VSX Clustering

VSX Conversion

vsx_utill

Gaia VS CTX & New Features


(Conversion, SNMP, JF)

Mgmt. Implementation

Gaia VSX Intro

10:00
11:00

R77 VSX Introduction

12:00
13:00

Lunch Break

14:00
VSX Networking

GW Implementation

15:00
Open Questions
16:00

VSX CoreXL Affinity


&Memory RC

Debug & Troubleshooting

17:00

2014 Check Point Software Technologies Ltd.

60
60

CoreXL

CoreXL architecture
Parallelise security gateway kernel
Leverage modern processor architectures
Suited to medium path

2014 Check Point Software Technologies Ltd.

61
61

Security Gateway CoreXL

Firewall kernel Replication


Firewall kernel is replicated multiple times. Each runs on one
processing core.
Each instance is independent FW-1 kernel.
Instances can run concurrently dont share a global lock.
Dispatcher
New component introduced in CoreXL.
Receives packets and forwards them to the kernel instances.
Acts as a load balancer. The dispatching is based on a hash of the
source IP, Destination IP, Destination port and IP protocol (4-tuple)
The dispatcher must maintain core stickiness per connection

2014 Check Point Software Technologies Ltd.

62
62

CoreXL - First Packet Flow

Record
Conn

fw0
WT

conn
table

fw1
WT

fw2

conn
table

WT

conn
table

2
Queue

Queue

Queue

Arbitrary
Decision
Lookup.
Not found

Dispatcher

global conn table

PKT
2014 Check Point Software Technologies Ltd.

63
63

CoreXL - Second Packet Flow

fw0
WT
Queue

conn
table

fw1
WT

fw2

conn
table

WT

Queue

Lookup.
Found

conn
table

Queue

Dispatcher

global conn table

2
PKT

2014 Check Point Software Technologies Ltd.

64
64

CoreXL - Parallel Processing

fw0
WT
Queue

conn
table

fw1
WT

fw2

conn
table

WT

Queue

conn
table

Queue

Dispatcher

global conn table

0 1 2

PKT
2014 Check Point Software Technologies Ltd.

65
65

CoreXL
Core #0

Dispatcher
PPAK

SND

eth1

PPAK

SND

eth0

Dispatcher

Core #1

Core #2

Core #3

fw5

fw4

Medium Path
Queue

Medium Path
Queue

Core #4

Core #5

Core #6

Core #7

fw3

fw2

fw1

fw0

Medium Path
Queue

Medium Path
Queue

Medium Path
Queue

Medium Path
Queue

Accelerated Path Cores are allocated via Interface IRQ


Affinity
Secure Network Dispatcher queues packets to firewall
instances running Firewall and Medium Paths
2014 Check Point Software Technologies Ltd.

66
66

Accelerated Path No Template


Core #...
FW
Path

Core #4
Medium
Path

FW
Path

Queue

Core #...
Medium
Path

FW
Path

Queue

Core #0

C
Medium
Path

Queue

Core #1

Dispatcher

Dispatcher

Performance Pack

Performance Pack

eth0

eth1
Syn
SynAck + subsequent S2C packets
Subsequent C2S packets
2014 Check Point Software Technologies Ltd.

67
67

Accelerated Path With Template


Core #...
FW
Path

Core #4
Medium
Path

FW
Path

Queue

Core #...
Medium
Path

FW
Path

Queue

Core #0

C
Medium
Path

Queue

Core #1

Dispatcher

Dispatcher

Performance Pack

Performance Pack

eth0

eth1
Syn + subsequent C2S packets
SynAck + subsequent S2C packets
2014 Check Point Software Technologies Ltd.

68
68

Medium Path IPS Traffic


Core #...
FW
Path

Core #4
Medium
Path

FW
Path

Queue

Core #...
Medium
Path

FW
Path

Queue

C
Medium
Path

Queue

Core #0

Core #1

Secure Dispatcher

Secure Dispatcher

Performance Pack

Performance Pack

eth0

eth1
Syn + subsequent C2S packets
SynAck + subsequent S2C packets
2014 Check Point Software Technologies Ltd.

69
69

VSX CoreXL

VSX CoreXL
Same idea as applied for SG is applied to VSX
CoreXL.
Main difference, instance in FWK (fw kernel
equivalent) are executed by UM threads.
VSX CoreXL can be applied for any existing
VS simultaneously with different number of
instances.

2014 Check Point Software Technologies Ltd.

70
70

VSX CoreXL configuration

CoreXL configuration for VS0 is done using cpconfig

This program will let you re-configure


your Check Point products configuration.
Configuration Options:
---------------------(1) Licenses and contracts
(2) SNMP Extension
(3) PKCS#11 Token
(4) Random Pool
(5) Secure Internal Communication
(6) Enable cluster membership for this gateway
(7) Disable Check Point SecureXL
(8) Configure Check Point CoreXL
(9) Automatic start of Check Point Products
(10) Exit
Enter your choice (1-10) :

CoreXL for VS which is not VS0 is done using SmartDashboard

Note: changing CoreXL configuration (num of instances) will require downtime of the VS (VS0 or other).

2014 Check Point Software Technologies Ltd.

72
72

VSX Affinity Usage


Setting Affinity

Interface Affinity: fw ctl affinity -s -i <interface> <cpuids | all>


VS affinity (VS,VR,VSW): fw ctl affinity -s -d [-vsid <ranges>] -cpu <ranges>

Process affinity - fw ctl affinity -s -d -pname <process name> [-vsid <ranges>] -cpu <ranges>

pid Affinity - fw ctl affinity -s -p <pid> <cpuids | all>


FWK instance affinity - fw ctl affinity -s -d -inst <ranges> -cpu <ranges>
All FWKs affinity - fw ctl affinity -s -d -fwkall <num of CPUs>

Note: If vsid flag is omitted, the current context will be used.

Listing Affinity

Configured affinity - fw ctl affinity -l

Extended Affinity - fw ctl affinity -l -x [-vsid <ranges>] [-cpu <ranges>] [-flags e|k|t|n]

Flags:
e don't print exception processes

k don't print kernel threads


t print also all process threads
n print process name instead of /proc/<pid>/cmdline
h print CPU mask in hex format

2014 Check Point Software Technologies Ltd.

73
73

Usage Examples
Setting affinity examples

fw
fw
fw
fw
fw

ctl
ctl
ctl
ctl
ctl

affinity
affinity
affinity
affinity
affinity

-s
-i
-s
-s
-s

-d -fwkall 3
eth0 0 3 7
-d -inst 0 2 4 -cpu 5
-d -pname cpd -vsid 0-12 -cpu 7
-d -vsid 0-2 4 6-8 -cpu 0-2 4

Listing Affinity example


fw ctl affinity -l
Output:
eth0: CPU 1
VS_0 FWK_INSTANCE_0:
VS_0 fwk: CPU 2 3
VS_1 FWK_INSTANCE_0:
VS_1 FWK_INSTANCE_1:
VS_1 FWK_INSTANCE_2:
VS_1 fwk: CPU 2 3
VS_2 cpd: CPU 1 2 3
VS_2 fwk: CPU 2 3
VS_3 fwd: CPU 1 3
VS_3 fwk: CPU 0 3

CPU 0 1 2
CPU 0
CPU 1
CPU 2

2014 Check Point Software Technologies Ltd.

74
74

Usage Examples (cont)


Extended Affinity List example
fw ctl affinity l x vsid 1 flags tnek
Output:
------------------------------------------------------|PID
|VSID |
CPU
|SRC|V|KT |EXC|
------------------------------------------------------|
4835 |
1 |
all |
| |
|
|
| 21094 |
1 |
all |
| |
|
|
| 21096 |
1 |
all |
| |
|
|
| 21241 |
1 |
all |
| |
|
|
| 21244 |
1 |
all |
| |
|
|
| 21245 |
1 |
all |
| |
|
|
| 21107 |
1 |
all |
| |
|
|
| 21115 |
1 |
2 3 | P | |
|
|
| 21116 |
1 |
0 | I | |
|
|
| 21117 |
1 |
1 | I | |
|
|
| 21118 |
1 |
2 | I | |
|
|
| 21119 |
1 |
2 3 | P | |
|
|
| 21401 |
1 |
all |
| |
|
|
| 21411 |
1 |
all |
| |
|
|
| 21412 |
1 |
all |
| |
|
|
| 21413 |
1 |
all |
| |
|
|
-------------------------------------------------------

2014 Check Point Software Technologies Ltd.

NAME
routed
fwk_wd
cpd
|---cpd
|---cpd
|---cpd
mpdaemon
fwk1_dev
|---fwk1_0
|---fwk1_1
|---fwk1_2
|---fwk1_3
fw
|---fw
|---fw
|---fw

75
75

CoreXL with Affinity Example


Command used for viewing fwk setup in the following example
fw ctl affinity l x vsid 1 flags tn | grep fwk | grep v fwk_

Before Setting CoreXL


|
|

21115 |
21116 |

1 |
1 |

2 3 | P | |
0 | I | |

|
|

| fwk1_dev
| |---fwk1_0

|
|
|
|
|

|
|
|
|
|

fwk1_dev
|---fwk1_0
|---fwk1_1
|---fwk1_2
|---fwk1_3

|
|
|
|
|

fwk1_dev
|---fwk1_0
|---fwk1_1
|---fwk1_2
|---fwk1_3

After Setting CoreXL (Used SDB to configure 4 instances)


|
|
|
|
|

21115
21116
21117
21118
21119

|
|
|
|
|

1
1
1
1
1

|
|
|
|
|

2
2
2
2
2

3
3
3
3
3

|
|
|
|
|

P
P
P
P
P

|
|
|
|
|

|
|
|
|
|

Set affinity for instance 0 (fw ctl affinity s d inst 0 cpu 0)


|
|
|
|
|

21115
21116
21117
21118
21119

|
|
|
|
|

1
1
1
1
1

|
|
|
|
|

2 3
0
2 3
2 3
2 3

|
|
|
|
|

P
I
P
P
P

|
|
|
|
|

|
|
|
|
|

|
|
|
|
|

Set affinity for instance 2 and 3 (fw ctl affinity s d inst 2 3 cpu 1 2)
|
|
|
|
|

21115
21116
21117
21118
21119

|
|
|
|
|

1
1
1
1
1

|
|
|
|
|

2 3
0
2 3
1 2
1 2

|
|
|
|
|

P
I
P
I
I

2014 Check Point Software Technologies Ltd.

|
|
|
|
|

|
|
|
|
|

|
|
|
|
|

|
|
|
|
|

fwk1_dev
|---fwk1_0
|---fwk1_1
|---fwk1_2
|---fwk1_3

76
76

VSX R77
Virtual System Load
Sharing

2014 Check Point Software Technologies Ltd.

Agenda

Genesis
Motivation
Architecture
State Synchronization
Virtual System Load Sharing Fail over
Performance
Summary

2014 Check Point Software Technologies Ltd.

78
78

Genesis
First

there was HA

Member 1

Member 2

Member 3

Member 4

VS1

VS2

VS3

2014 Check Point Software Technologies Ltd.

79
79

Course Timetables
Day 1

Day 2

Day 3
RC & QoS

10:00
11:00

vsx_utill

Gaia VS CTX & New Features


(Conversion, SNMP, JF)

Mgmt. Implementation

Gaia VSX Intro

R77 VSX Introduction

12:00
13:00

Lunch Break

14:00
VSX Networking

GW Implementation

15:00
Open Questions
16:00

17:00

VSX CoreXL Affinity &


Debug & Troubleshooting
Memory RC

2014 Check Point Software Technologies Ltd.

80
80

Motivation

Improve VSX cluster performance (synchronization


traffic limits scalability).
Provide cost-effective Load Sharing in VSX, using the
VS as the basis for Load Sharing.

Standard Cluster

Performance

VSLS Cluster

Number of Cluster Members


2014 Check Point Software Technologies Ltd.

81
81

Architecture

Each VS behaves as a FW-1 cluster.


ClusterXL HA is used (gratuitous ARP).
VSX machines are connected to the outside world
using standard L2 switches.
L2 switch

member
1

member
2

member
3

L2 switch
2014 Check Point Software Technologies Ltd.

82
82

Architecture

Virtual Systems are distributed between Cluster


members.
VS1

VS2

VS3

member 1

member 2

2014 Check Point Software Technologies Ltd.

member 3

83
83

Architecture

Virtual Systems are distributed between Cluster


members.
VS1

A
A

Active VS

VS2

VS3

member 1

member 2

2014 Check Point Software Technologies Ltd.

member 3

84
84

Architecture

Virtual Systems are distributed between Cluster


members.
VS1

VS2

VS3

member 1

member 2

member 3

2014 Check Point Software Technologies Ltd.

Active VS

Standby VS

85
85

Architecture

Virtual Systems are distributed between Cluster


members.
VS1

VS2

Active VS

Standby VS

Backup VS

VS3

member 1

member 2

member 3

2014 Check Point Software Technologies Ltd.

86
86

Architecture

Each cluster member has some of the VSs, but


not all of them.
VS1

VS2

Active VS

Standby VS

Backup VS

VS3

member 1

member 2

member 3

2014 Check Point Software Technologies Ltd.

87
87

Architecture

Each VS is installed on some of the members,


but not all of them.
VS1

VS2

Active VS

Standby VS

Backup VS

VS3

member 1

member 2

member 3

2014 Check Point Software Technologies Ltd.

88
88

Architecture

Backup Virtual Systems do not consume system


resources.
VS1

VS2

Active VS

Standby VS

Backup VS

VS3

member 1

member 2

member 3

2014 Check Point Software Technologies Ltd.

89
89

State Synchronization

2014 Check Point Software Technologies Ltd.

90
90

Broadcast Sync in VSX

Member 1

Member 2

Member 3

Member 4

VS1

VS2

VS3

Broadcast is more efficient when the VSs are installed on all the
members
2014 Check Point Software Technologies Ltd.

91
91

The Mode of Cluster Control Protocol


(CCP) in VSX cluster
In VSX cluster:
VSX NGX / VSX NGX R65 / VSX NGX R67 / VSX NGX R68:
The only possible mode of CCP is Broadcast.
R75.40VS / R76 and above:
CCP mode over Sync Network is Broadcast for all Virtual Systems.
CCP mode over non-Sync Networks is Multicast.
In VSLS configuration, when instances of Virtual Systems are not running
on all cluster members (e.g., only 2 VSs were configured on a VSX cluster that
has 4 cluster members), the Delta Sync packets generated by a Virtual System,
are sent in Unicast only to those members that run the instance of the same
Virtual System.

On 61000 appliance (starting in R75.40VS for 61000), the 'asg_sync_manager' option '8) Enable / Disable unicast sync' - enables the user to define its required
synchronization level - the user can enable / disableUnicast sync (correction
layer will be enabled / disabled accordingly) and return to legacy
synchronization scheme (synchronize connections to all SGMs).
2014 Check Point Software Technologies Ltd.

92
92

Broadcast in VSX for Sync Network

Connection is created on VS1

Member 1

Member 2

Member 3

Member 4

VS1

VS2

VS3

2014 Check Point Software Technologies Ltd.

93
93

Broadcast Sync in VSX

Sync is sent to all members via broadcast

Member 1

Member 2

Member 3

Member 4

VS1

VS2

VS3

2014 Check Point Software Technologies Ltd.

94
94

Broadcast Sync in VSX

Sync is sent to all members via broadcast

Member 1

Member 2

Member 3

Member 4

VS1

VS2

VS3

2014 Check Point Software Technologies Ltd.

95
95

Broadcast Sync in VSLS

Connection is created on VS1

Member 1

VS1
VS2

Member 2

Member 3

Member 4

S
A

S
S

VS3

2014 Check Point Software Technologies Ltd.

96
96

Broadcast Sync in VSLS

Sync is sent to all members via broadcast

Member 1

VS1
VS2

Member 2

Member 3

Member 4

S
A

S
S

VS3

2014 Check Point Software Technologies Ltd.

97
97

Broadcast Sync in VSLS

Sync is sent to all members via broadcast

Member 1

VS1
VS2

Member 2

Member 3

Member 4

S
A

VS3

2014 Check Point Software Technologies Ltd.

S
A

98
98

Broadcast Sync in VSLS

Members 3 and 4 discard the sync packet

Member 1

VS1
VS2

Member 2

Member 3

Member 4

S
A

VS3

2014 Check Point Software Technologies Ltd.

S
A

99
99

Virtual System Load Sharing Fail over

2014 Check Point Software Technologies Ltd.

100
100

Step 1 distributed VSs

VS1

VS2

VS3

Member 1

Member 2

Member 3

2014 Check Point Software Technologies Ltd.

101
101

Step 2 Machine Failure

VS1

S
A

VS2

VS3

Member 2

Member 3

Member 1

Member 1 fails, causing an immediate failover of VS1 to member 3.


2014 Check Point Software Technologies Ltd.

102
102

Step 3 Machine recovered

VS1

VS2

Member 2

Member 3

VS3

Member 1

Member 1 goes up again and the system performs full sync for VS1 and VS2.
2014 Check Point Software Technologies Ltd.

103
103

Step 4 Load sharing restored

VS1

A
S

VS2

S
A

VS3

Member 1

Member 2

Member 3

The system can now force VS1 to become active on member 1, so the load is shared
equally between the machines.
2014 Check Point Software Technologies Ltd.

104
104

Performance

2014 Check Point Software Technologies Ltd.

105
105

Limitations
When working with ClusterXL Virtual System Load
Sharing (VSLS):

Virtual Routers are not supported.

Each Virtual Switch must have a physical or VLAN interface that


provides connectivity between cluster members.

2014 Check Point Software Technologies Ltd.

106
106

Summary

VSLS currently provides the best performance and scalability.

Easy to administer and monitor.

Administrator needs not to be involved in failover process.

System intelligently provides the best performance and redundancy


possible.

2014 Check Point Software Technologies Ltd.

107
107

Thank you !
Please proceed to Lab 4

2014 Check Point Software Technologies Ltd.

R77 VSX
Memory Resource Control

2014 Check Point Software Technologies Ltd.

Course Timetables
Day 1

Day 2

Day 3
RC & QoS

10:00
11:00

vsx_utill

Gaia VS CTX & New Features


(Conversion, SNMP, JF)

Mgmt. Implementation

Gaia VSX Intro

R77 VSX Introduction

12:00
13:00

Lunch Break

14:00
VSX Networking

GW Implementation

15:00
Open Questions
16:00

17:00

VSX CoreXL Affinity &


Debug & Troubleshooting
Memory RC

2014 Check Point Software Technologies Ltd.

110
110

Memory Resource control overview

Memory Resource control (fw vsx mstat) gives the


user overview information about:
Memory consumption of the system
Memory consumption per virtual device

2014 Check Point Software Technologies Ltd.

111
111

Overview cont.
The memory consumption per VS is a sum of the following:
Kernel Space
SIM (SecureXL) memory per VS allocations.

Dispatcher memory per VS allocations.


User Space

Accumulations of clean pages (private) and dirty pages (private) per process for all processes per VS

2014 Check Point Software Technologies Ltd.

112
112

Overview cont.
Global info given about system:

Total amount of physical memory


Free memory
Total amount of swap memory
Free swap memory
Swap rate (total number of kilobytes the system paged in
from disk per second)

2014 Check Point Software Technologies Ltd.

113
113

VSX Memory Resource Control Usage

fw vsx mstat - Displays total memory consumption per virtual system

Flags

-vs - displays memory status for specific VSIDs


sort - sorts the virtual systems by their memory size
unit - change the displayed memory unit
swap - updates the swap-in sample rate in minutes

fw vsx mstat debug - Displays memory consumption debug info per VS.
fw vsx mstat enable - Enables this feature, requires a reboot.
fw vsx mstat disable - Disables this feature, affected immediately.
fw vsx mstat status - Displays feature status, (enabled/disabled).
fw vsx mstat help - Displays help.

2014 Check Point Software Technologies Ltd.

114
114

Monitoring Memory Resources


[Expert@GIZA42G204:0]# fw vsx mstat sort all
VSX Memory Status
=================
Memory Total: 997.22 MB
Memory Free: 232.52 MB
Swap Total: 2047.34 MB
Swap Free: 2047.16 MB
Swap-in rate: 0.00 MB
VSID | Memory Consumption
======+====================
0|
133.49 MB
8|
92.41 MB
3|
43.81 MB
2|
42.47 MB
1|
42.47 MB
2014 Check Point Software Technologies Ltd.

115
115

The vsx_util

VSX Management maintenance tool

2014 Check Point Software Technologies Ltd.

Course Timetables

9:00

Day 1

Day 2

Day 3

Course Introduction

VSX Clustering

VSX Conversion

vsx_utill

Gaia VS CTX & New Features


(Conversion, SNMP, JF)

Mgmt. Implementation

Gaia VSX Intro

10:00
11:00

RR77 VSX Introduction

12:00
13:00

Lunch Break

14:00
VSX Networking

GW Implementation

15:00
Open Questions
16:00
17:00

VSX CoreXL Affinity &


Debug & Troubleshooting
Memory RC

2014 Check Point Software Technologies Ltd.

117
117

vsx_util

vsx_util is a tool for performing VSX maintenance


activities.
vsx_util is a CPMI client that connects to the
Management server (Main DMS in case of
Provider-1), just like SmartDashboard.
The Management machine from which vsx_util is run
must be defined as a GUI client (just like SDB)
Not supported for the (very old) version of VSX NG AI
and below.

2014 Check Point Software Technologies Ltd.

118
118

vsx_util reconfigure / add_member_reconf

Used to deploy an existing configuration on a freshly


installed Gateway / Cluster Member

Useful after a machine hardware failure

Used also after upgrading module to a new version.

2014 Check Point Software Technologies Ltd.

119
119

vsx_util reconfigure limitations and common


errors

Limitations

The new module must have the same configuration as the reconfigured
member (same number of interfaces, management IP etc.)
The module must be newly installed without any previous VSX
configuration

Common Errors:

vsx_util reconfigure asks to re-install policy on the VSX and run the
command again

The message occurs when the reconfigure process cannot retrieve the name
of the policy installed on the VSX Gateway/Cluster
This might happen after the Management was upgraded using upgrade_import
from a version older than R61
Workaround: copy $FWDIR/state/links.C from the original Management

2014 Check Point Software Technologies Ltd.

120
120

vsx_util add_member / remove_member

vsx_util add_member

Used to add a member to an existing VSX Cluster


configuration
After running add_member, vsx_util
add_member_reconf needs to be run to configure the
new member with the VSX configuration

vsx_util remove_member

Used to remove a member from an existing VSX


Cluster configuration
Can be performed only if the VSX Cluster has at least 3
members
2014 Check Point Software Technologies Ltd.

121
121

vsx_util change_private_net

Used to change the VSX Cluster Private


Network on an existing Cluster configuration

Limitations:

The new Cluster Private Network must not be used


anywhere behind the VSX Cluster/Gateway or its
Virtual Systems
The new Cluster Private Network has to match the
private network mask 255.255.252.0

2014 Check Point Software Technologies Ltd.

122
122

vsx_util change_mgmt_ip

Used to change the management IP address


of VSX member/ gateway.

Limitations:

The IP address should stay in the same subnet.


Not supported in Non Dedicated Management
Interface mode.

2014 Check Point Software Technologies Ltd.

123
123

vsx_util change_mgmt_subnet

Used to change the management IP of a VSX


cluster/gateway to a new subnet.
Allows to change the VSX cluster management
IP,
VSX members management IP and the
management subnet mask.
Limitations:

Not supported in Non Dedicated Management


Interface mode.
2014 Check Point Software Technologies Ltd.

124
124

vsx_util vsls

Displays the current Virtual System Load Sharing (VSLS)


configuration as it appears in the management database,
and allows exporting to CSV.

The displayed configuration is the desired state. It does


not necessarily reflect the actual configuration on the
Cluster Members.
In case the Cluster Members encounter a certain problem, they
might not enforce the desired VSLS configuration

2014 Check Point Software Technologies Ltd.

125
125

vsx_util vsls (was redistribute_vsls)

Distributes the Virtual Systems among the VSX Cluster Members.


vsx_util suggests the following options:
Automatically distribute the Virtual Systems over all Cluster
Members in a way that all Cluster Members are equally loaded.
Have all Virtual Systems active on the same Cluster Member
similar to ClusterXL mode High Availability, except for the
fact that for each Virtual System there is only one Standby
Cluster Member while the rest of the Cluster Members are
backup
Manually configure the weight and members priority list for a
specific Virtual System
Import priority and weight from a CSV.

2014 Check Point Software Technologies Ltd.

126
126

vsx_util vsls (was redistribute_vsls) cont.

Notes:
A Virtual System is created with a default weight (10)
Changing the weight of a Virtual System will only have
influence when the Priority List for a new Virtual System
is calculated
For the new configured weight to take effect
immediately, the Virtual Systems need to be
redistributed automatically among all Cluster Members
The redistribution is a rather heavy operation because it
might trigger Virtual System Failovers from one Cluster
Member to another

2014 Check Point Software Technologies Ltd.

127
127

vsx_util convert_cluster

Converts the ClusterXL mode to one of the


following:

High Availability All Virtual Systems are active on


the same Cluster Member, the other members are
in standby state
Virtual System Load Sharing The active Virtual
Systems are spread among all Cluster Members to
equally balance the load over all Cluster Members
Each

Virtual System is active on one Cluster Member,


Standby on another Cluster Member and in backup mode
on the rest of the Cluster Members (not resources
consuming)
2014 Check Point Software Technologies Ltd.

128
128

vsx_util view_vs_conf

Displays Virtual Device configuration on


Management versus VSX gateways
Displays the interface configuration table and
the routing table of the Virtual Device
Used to see if there is a configuration
mismatch between what is defined on the
management and the VSX gateways

2014 Check Point Software Technologies Ltd.

129
129

vsx_util change_interfaces

Used to replace between interfaces in an existing configuration


Management Only Mode:
Changes the management database only.
Very useful to convert from open server to CheckPoint appliance
Should only be used with freshly installed Gateway/Cluster
members
New Configuration will be applied to the VSX members using
vsx_util reconfigure.
Push configuration to VSX Gateway/Cluster members immediately
mode:
Changes are applied to the VSX Gateway/Cluster members
immediately
Very useful when adding new bond interfaces to an existing
configuration
Displays a summary report with individual status per virtual device
2014 Check Point Software Technologies Ltd.

130
130

vsx_util show_interfaces

Displays information about interfaces configuration on the


Management

Displays the type of interface, the virtual device it is connected to in


the VSX, IP address and netmask

Output is displayed on screen and saved to interfacesconfig.csv file

2014 Check Point Software Technologies Ltd.

131
131

Resume capabilities

The following actions support resume:


reconfigure / add_member_reconf
upgrade
add_member
remove_member
change_private_net
change_mgmt_ip
change_mgmt_subnet
change_interfaces
vsx_util remembers the point of failure and when it is
executed again, it will continue from there.
2014 Check Point Software Technologies Ltd.

132
132

Thank you !
Please proceed to lab 5, 6

2014 Check Point Software Technologies Ltd.

The vsx_util

VSX Management maintenance tool

2014 Check Point Software Technologies Ltd.

Course Timetables

9:00

Day 1

Day 2

Day 3

Course Introduction

VSX Clustering

VSX Conversion

vsx_utill

Gaia VS CTX & New Features


(Conversion, SNMP, JF)

Mgmt. Implementation

Gaia VSX Intro

10:00
11:00

RR77 VSX Introduction

12:00
13:00

Lunch Break

14:00
VSX Networking

GW Implementation

15:00
Open Questions
16:00
17:00

VSX CoreXL Affinity &


Debug & Troubleshooting
Memory RC

2014 Check Point Software Technologies Ltd.

135
135

vsx_util

vsx_util is a tool for performing VSX maintenance


activities.
vsx_util is a CPMI client that connects to the
Management server (Main DMS in case of
Provider-1), just like SmartDashboard.
The Management machine from which vsx_util is run
must be defined as a GUI client (just like SDB)
Not supported for the (very old) version of VSX NG AI
and below.

2014 Check Point Software Technologies Ltd.

136
136

vsx_util reconfigure / add_member_reconf

Used to deploy an existing configuration on a


freshly installed Gateway / Cluster Member
Useful after a machine hardware failure
Used also after upgrading module to a new
version.

2014 Check Point Software Technologies Ltd.

137
137

vsx_util reconfigure limitations and common


errors

Limitations

The new module must have the same configuration as the reconfigured
member (same number of interfaces, management IP etc.)
The module must be newly installed without any previous VSX
configuration

Common Errors:

vsx_util reconfigure asks to re-install policy on the VSX and run the
command again

The message occurs when the reconfigure process cannot retrieve the name
of the policy installed on the VSX Gateway/Cluster
This might happen after the Management was upgraded using upgrade_import
from a version older than R61
Workaround: copy $FWDIR/state/links.C from the original Management

2014 Check Point Software Technologies Ltd.

138
138

vsx_util add_member / remove_member

vsx_util add_member

Used to add a member to an existing VSX Cluster


configuration
After running add_member, vsx_util
add_member_reconf needs to be run to configure the
new member with the VSX configuration

vsx_util remove_member

Used to remove a member from an existing VSX


Cluster configuration
Can be performed only if the VSX Cluster has at least 3
members
2014 Check Point Software Technologies Ltd.

139
139

vsx_util change_private_net

Used to change the VSX Cluster Private


Network on an existing Cluster configuration

Limitations:

The new Cluster Private Network must not be used


anywhere behind the VSX Cluster/Gateway or its
Virtual Systems
The new Cluster Private Network has to match the
private network mask 255.255.252.0

2014 Check Point Software Technologies Ltd.

140
140

vsx_util change_mgmt_ip

Used to change the management IP address


of VSX member/ gateway.

Limitations:

The IP address should stay in the same subnet.


Not supported in Non Dedicated Management
Interface mode.

2014 Check Point Software Technologies Ltd.

141
141

vsx_util change_mgmt_subnet

Used to change the management IP of a VSX


cluster/gateway to a new subnet.
Allows to change the VSX cluster management
IP,
VSX members management IP and the
management subnet mask.
Limitations:

Not supported in Non Dedicated Management


Interface mode.
2014 Check Point Software Technologies Ltd.

142
142

vsx_util vsls

Displays the current Virtual System Load


Sharing (VSLS) configuration as it appears in
the management database, and allows
exporting to CSV.

The displayed configuration is the desired


state. It does not necessarily reflect the actual
configuration on the Cluster Members.
In case the Cluster Members encounter a certain
problem, they might not enforce the desired VSLS
configuration
2014 Check Point Software Technologies Ltd.

143
143

vsx_util vsls (was redistribute_vsls)

Distributes the Virtual Systems among the VSX Cluster Members.


vsx_util suggests the following options:
Automatically distribute the Virtual Systems over all Cluster
Members in a way that all Cluster Members are equally loaded.
Have all Virtual Systems active on the same Cluster Member
similar to ClusterXL mode High Availability, except for the
fact that for each Virtual System there is only one Standby
Cluster Member while the rest of the Cluster Members are
backup
Manually configure the weight and members priority list for a
specific Virtual System
Import priority and weight from a CSV.

2014 Check Point Software Technologies Ltd.

144
144

vsx_util vsls (was redistribute_vsls) cont.

Notes:

A Virtual System is created with a default weight (10)


Changing the weight of a Virtual System will only have
influence when the Priority List for a new Virtual System
is calculated
For the new configured weight to take effect
immediately, the Virtual Systems need to be
redistributed automatically among all Cluster Members
The redistribution is a rather heavy operation because it
might trigger Virtual System Failovers from one Cluster
Member to another

2014 Check Point Software Technologies Ltd.

145
145

vsx_util convert_cluster

Converts the ClusterXL mode to one of the


following:

High Availability All Virtual Systems are active on the


same Cluster Member, the other members are in
standby state
Virtual System Load Sharing The active Virtual
Systems are spread among all Cluster Members to
equally balance the load over all Cluster Members
Each Virtual System is active on one Cluster
Member, Standby on another Cluster Member and in
backup mode on the rest of the Cluster Members
(not resources consuming)

2014 Check Point Software Technologies Ltd.

146
146

vsx_util view_vs_conf

Displays Virtual Device configuration on


Management versus VSX gateways
Displays the interface configuration table and
the routing table of the Virtual Device
Used to see if there is a configuration
mismatch between what is defined on the
management and the VSX gateways

2014 Check Point Software Technologies Ltd.

147
147

vsx_util change_interfaces

Used to replace between interfaces in an existing configuration


Management Only Mode:
Changes the management database only.
Very useful to convert from open server to CheckPoint appliance
Should only be used with freshly installed Gateway/Cluster
members
New Configuration will be applied to the VSX members using
vsx_util reconfigure.
Push configuration to VSX Gateway/Cluster members immediately
mode:
Changes are applied to the VSX Gateway/Cluster members
immediately
Very useful when adding new bond interfaces to an existing
configuration
Displays a summary report with individual status per virtual device
2014 Check Point Software Technologies Ltd.

148
148

vsx_util show_interfaces

Displays information about interfaces configuration on the


Management

Displays the type of interface, the virtual device it is connected to in


the VSX, IP address and netmask

Output is displayed on screen and saved to interfacesconfig.csv file

2014 Check Point Software Technologies Ltd.

149
149

Resume capabilities

The following actions support resume:


reconfigure / add_member_reconf
upgrade
add_member
remove_member
change_private_net
change_mgmt_ip
change_mgmt_subnet
change_interfaces
vsx_util remembers the point of failure and when it is
executed again, it will continue from there.

2014 Check Point Software Technologies Ltd.

150
150

Thank you !
Please proceed to lab 5, 6

2014 Check Point Software Technologies Ltd.

VSX Management
Implementation

2014 Check Point Software Technologies Ltd.

Course Timetables

9:00

Day 1

Day 2

Day 3

Course Introduction

VSX Clustering

VSX Conversion

vsx_utill

Gaia VS CTX & New Features


(Conversion, SNMP, JF)

Mgmt. Implementation

Gaia VSX Intro

10:00
11:00

R77 VSX Introduction

12:00
13:00

Lunch Break

14:00
VSX Networking

GW Implementation

15:00
Open Questions
16:00
17:00

VSX CoreXL Affinity &


Debug & Troubleshooting
Memory RC

2014 Check Point Software Technologies Ltd.

153
153

Management side implementation

General communication flow


Management database
Network Configuration Script
NCS files and their location
Provider-1 Forwarding Concept
SIC implementation
Cluster Private Network

2014 Check Point Software Technologies Ltd.

154
154

General Communication Flow

3-tier management architecture:

GUI - SmartDashboard
Management Server - Provider-1 or SmartCenter
VSX Gateway

Communications flow between the participants in a VSX setup:

SmartDashboard

Management Server

2014 Check Point Software Technologies Ltd.

VSX Gateway

155
155

Management Models
VSX can be managed in two ways:

SmartCenter management

Provider-1 management
Each domain management
server (DMS) may manage one
or more Virtual Devices

2014 Check Point Software Technologies Ltd.

156
156

Managing VSX from Provider-1

Virtual Devices can be managed


by different DMSs.
The DMS where the VSX is
defined is called the Main DMS.
DMSs where Virtual Devices are
defined are called Target
DMSs.
DMS is being Target or Main
relatively to a specific VSX.

DMS Mgr (Main)

VS A

DMS A (Target)

DMS B (Target)
VS B

2014 Check Point Software Technologies Ltd.

157
157

Management database

Each Virtual Device has 2 objects representing it:


network_object
vs_slot

Network object

Resides on the Target DMS


The object the user sees in SmartDashboard.

vs_slot objects

Reside on the Main DMS

2014 Check Point Software Technologies Ltd.

158
158

Management database

vs_slot objects

Contain the list of interfaces belonging to a Virtual Device.


Contain the list of routes belonging to a Virtual Device.
Contain other VSX specific attributes. For instance, a reference to the
VSX object to which the Virtual Device belongs.
The vs_slot objects are used in order to create the Virtual Devices and
their networking properties on a VSX Gateway (Later will be explained
how).
vs_slot objects are stored in the Check Point database in a table
called vs_slot_objects.

2014 Check Point Software Technologies Ltd.

159
159

VSX Management

VSX related information is stored in the management

Initial configuration is fetched from the VSX gateway

After the VSX creation, any configuration change is


done through the management.

Its possible to push the VSX configuration, stored in


the database, to a blank module.

2014 Check Point Software Technologies Ltd.

160
160

Management to Module communication

NCS (Network Configuration Script) files pass the


VSX configuration from the management to the GW.
The files are generated on the management machine
The GW receives the configuration files, parses and
executes them.

Management Server

2014 Check Point Software Technologies Ltd.

VSX Gateway

161
161

The NCS
Example of NCS file

Creates a VS with 2
interfaces
One VLAN interface
The other leading to
Virtual Switch
Adds routes

#NCS Build 620001002


#local.vs for Virtual System VS5 on VSX GW EcuCluster created at Tue Mar,13 18:
58:12 2007
begin
[mem1:]vs create vs 8 vr 8 name mem1_VS5 uid {0727F24C-D184-11DB-8CF0-0000
00004F4F} is_junction 0 is_bridge 0 cpu_usage 0.000000 main_ip sic_name CN=mem1
_VS5,O=peter.marie..pngba3 bw_limit 0 bw_guarantee 0 conn_limit 15000 masters
_addresses 192.168.100.100,194.29.37.83 otp "5e54c2cf66dc45b61f712c12badc90dfcf6
ed1fb" cluster_name VS5
[mem2:]vs create vs 8 vr 8 name mem2_VS5 uid {0728215E-D184-11DB-8CF0-000000
004F4F} is_junction 0 is_bridge 0 cpu_usage 0.000000 main_ip sic_name CN=mem2
_VS5,O=peter.marie..pngba3 bw_limit 0 bw_guarantee 0 conn_limit 15000 masters_ad
dresses 192.168.100.100,194.29.37.83 otp "b52e55829f28a63001f65cc94899de97f9fa77
9a" cluster_name VS5
[mem1:]vlan create tag 5 dev eth3
[mem1:]interface set dev eth3.5 address 192.168.196.1 netmask 255.255.255.240
mtu 1500 vr 8 cluster_ip 1.1.1.5 cluster_mask 255.255.255.0
[mem1:]warp create name_a wrp512 name_b wrpj512 mac_id_a 6 mac_id_b 7
[mem1:]interface set dev wrp512 address 192.168.196.17 netmask 255.255.255.24
0 mtu 1500 vr 8 cluster_ip 10.10.10.5 cluster_mask 255.255.255.0
[mem1:]interface set dev wrpj512 address 0.0.0.0 netmask 0.0.0.0 mtu 1500 vr 3
[mem1:]bridge attach name br3 dev wrpj512
[mem2:]vlan create tag 5 dev eth3
[mem2:]interface set dev eth3.5 address 192.168.196.2 netmask 255.255.255.240
mtu 1500 vr 8 cluster_ip 1.1.1.5 cluster_mask 255.255.255.0
[mem2:]warp create name_a wrp512 name_b wrpj512 mac_id_a 6 mac_id_b 7
[mem2:]interface set dev wrp512 address 192.168.196.18 netmask 255.255.255.240
mtu 1500 vr 8 cluster_ip 10.10.10.5 cluster_mask 255.255.255.0
[mem2:]interface set dev wrpj512 address 0.0.0.0 netmask 0.0.0.0 mtu 1500 vr 3
[mem2:]bridge attach name br3 dev wrpj512
[mem1:]route set dest 10.10.10.0 netmask 255.255.255.0 metric 0 dev wrp512 vr 8
[mem1:]route set dest 1.1.1.0 netmask 255.255.255.0 metric 0 dev eth3.5 vr 8
[mem2:]route set dest 10.10.10.0 netmask 255.255.255.0 metric 0 dev wrp512 vr 8
[mem2:]route set dest 1.1.1.0 netmask 255.255.255.0 metric 0 dev eth3.5 vr 8
end

2014 Check Point Software Technologies Ltd.

162
162

The NCS file structure

3 files are passed to the VSX gateway after each


configuration change done in SmartDashboard:

local.vs - NCS file, the last configuration change.

local.vsall - NCS file, contains the full configuration.


This file is executed at system startup.
local.vskeep - contains the list of existing VSIDs.

These files are created based on vs_slot_objects table.


2014 Check Point Software Technologies Ltd.

163
163

Composition of local.vs

Each vs_slot object has 2 attributes containing


interfaces lists:

Each vs_slot object has 2 attributes containing routes


lists:

interfaces
interfaces_installed

routes
routes_installed

local.vs file is the product of comparing the two


couples
2014 Check Point Software Technologies Ltd.

164
164

Composition of local.vsall and local.vskeep

a Virtual Device has 2 NCS files on the management:

VD_name.vsnew - NCS file containing interfaces


VD_name.vsrt - NCS file containing routes

Files are updated each time configuration is changed

local.vsall is a product of combining the files of all the


Virtual Devices.

local.vskeep is created by going over vs_slot_objects


table and writing all the Virtual Devices VSIDs to it.
2014 Check Point Software Technologies Ltd.

165
165

Files Location

On the management

* .vsnew and *.vsrt files are located in Main DMS.


Under $FWDIR/conf/vs_repository/VSX_NAME directory
composed local.vs, local.vsall and local.vskeep files are put
in a state directory, then sent to the VSX gateway.
Under $FWDIR/state/VSX_NAME/VSX/ directory

On the VSX gateway

Received local.vs, local.vsall and local.vskeep files are first


located under $FWDIR/state/__tmp/VSX/ directory.
If files processing succeeds, they are copied to
$FWDIR/state/local/VSX/ directory.

2014 Check Point Software Technologies Ltd.

166
166

Provider-1 Forwarding Concept

Configuring Virtual Device in Provider-1

SmartDashboard is connected to the Target DMS.


Both Target and main DMSs needs updating

Command to Target DMS, is forwarded to the Main


DMS (requires lock)
SmartDashboard

Management Server

Target
DMS

VSX Gateway

Main
DMS

2014 Check Point Software Technologies Ltd.

167
167

SIC - Secure Internal Communication

SIC is a Check Point proprietary protocol developed to secure


all communication amongst all Check Point's distributed
components belonging to a single management domain.
Examples of "Internal Communications" are Management to
Module communication (e.g. download a policy, send logs) and
GUI client to Management server communications.
"Securing" the communications generally includes:

Authenticating that the peer is indeed who it claims to be


Clients: Ensuring that this is the same peer the client wished to
communicate with
Servers: Ensuring that the peer is allowed to perform the actions it
requests
Privacy and Integrity - Making sure that the data received is the data
sent, and that no party other than the intended peer could read it

2014 Check Point Software Technologies Ltd.

168
168

SIC implementation

Single IP for SIC - a single IP Address is used for


managing all the Virtual Devices on the VSX gateway.

Trust with the VSX gateway is established in the


same way it is done for regular Firewall-1 object.

Trust with Virtual Devices is established in a special


way, since they dont have an IP Address to which the
SIC certificate can be pushed.

2014 Check Point Software Technologies Ltd.

169
169

Trust Establishment with Virtual Device

SIC certificate for the Virtual Device is created by


CA of the Target DMS.
After the device is created, SmartDashboard issues
a command for pulling the new certificate.
Main DMS transfers the command to the VSX GW
VSX GW pulls the SIC certificate from the Target
DMS.
SmartDashboard

Management Server

VSX Gateway

VS
Target
DMS

Main
DMS

2014 Check Point Software Technologies Ltd.

170
170

Trust Establishment with Virtual Device

SIC certificate for the Virtual Device is created by Certificate


Authority of the Target DMS.
After the Virtual Device is created on the VSX gateway (as a result
of local.vs file processing), SmartDashboard issues a command for
pulling the newly created certificate. This command is forwarded to
the Main DMS.
Main DMS transfers the command to the VSX gateway over the SIC
channel established between them.
VSX gateway pulls the SIC certificate for the Virtual Device from the
Target DMS.
SmartDashboard

Management Server

Target
DMS

VSX Gateway

VS

Main
DMS

2014 Check Point Software Technologies Ltd.

171
171

Cluster Private Network

unique IP address are required for each interface on


each member.
These IP addresses are automatically allocated.
Cluster Private Network is defined on the VSX
Cluster Object.

Default network: 192.168.196.0/22

Cluster Private Network can be changed:

In SmartDashboard before VS creation


At any stage using vsx_util change_private_net.

2014 Check Point Software Technologies Ltd.

172
172

Cluster Private Network

New interface on a Virtual Device, is assigned an IP


Address from a cluster private network pool on all
members.

IP Addresses must allow inter-VS routing and


uniqueness on a given Virtual Device:

On a specific member - each IP is from a different subnet


Between members - all IPs are from the same subnet

2014 Check Point Software Technologies Ltd.

173
173

VSX - R77
Gateway
Implementation

2014 Check Point Software Technologies Ltd.

Course Timetables

9:00

Day 1

Day 2

Day 3

Course Introduction

VSX Clustering

VSX Conversion

vsx_utill

Gaia VS CTX & New Features


(Conversion, SNMP, JF)

Mgmt. Implementation

Gaia VSX Intro

10:00
11:00

R77 VSX Introduction

12:00
13:00

Lunch Break

14:00
VSX Networking

GW Implementation

15:00
Open Questions
16:00
17:00

VSX CoreXL Affinity &


Debug & Troubleshooting
Memory RC

2014 Check Point Software Technologies Ltd.

175
175

VSX Gateway side implementation

R77 New Architecture


Linux virtualization VRF solution
VSX Packet Flow
FW-1 kernel virtualization
Acceleration
Files Structure
Registry Structure
Context Database
Virtual Device creation stages
Dynamic Routing
VSX Upgrade procedure

2014 Check Point Software Technologies Ltd.

176
176

VSX (R67) architecture

fwd
UM

cpd

cplogd

vpnd
vpnd
vpnd

1.

Ioctls ex. policy install


From cpd to fw kernel

All kernel code


had inside
virtualization

KM

Trap example logs


From fw kernel to cplogd

2.

Fw kernel virtualized

VPN kernel virtualized

Ppack virtualized
3.

NIC

Tables per VS
Parameters per
VS or global

Most of the UM
processes were
virtualized
(fwd/cpd/cplogd)
Some were per
VS (vpnd)

NIC

2014 Check Point Software Technologies Ltd.

177
177

R77 VSX architecture


cpd

cpd

cpd

fwd

fwd

fwd

vpnd

vpnd

vpnd

fwk

fwk

fwk

VS

VS

VS

Trap example logs


From fwk to fwd

UM
KM

Ioctls ex. policy install


From cpd to fwk

1.

2.

Firewall dispatcher

3.

Ppack virtualized
NIC

NIC

2014 Check Point Software Technologies Ltd.

Fwk is the fws


kernel code
compiled to a dll
PPK remains
virtualized
I/S to simulate
traps and ioctls,
over TCP
between fwd/cpd
and fwk fwasync_rpc
178
178

FWK User mode Firewall

FW-1 dispatcher is a driver, which sees the packets, and puts them in the
processing queue of the right FWK.
FWK process does kernel processing. Has libfwk.so, libvpnk.so, librtmk.so
loaded into it, multiple times if multiple instances are configured.

FWK is per VS, so all parameters, policy, tables, inspect code, etc. are
obviously separate, without the need to change almost anything in the code.

Reads packets from queue, performs processing, and tells dispatcher if to


pass/drop them.

Forked from FWK_FORKER, after a fork request arrives from FWK_WD


(per VS)

2014 Check Point Software Technologies Ltd.

179
179

VRF Linux CLI enhancements

vsenv [vsid] changes context of expert


shell almost all commands (CP and OS)
operate on this context.
In Gaias clish, set virtual-system [vsid],
which effects many commands, like routing

2014 Check Point Software Technologies Ltd.

180
180

VRFs and VSs


The illustration above shows the connection between Linux VRFs and Check Point Virtual devices.
A Virtual Devices is always tied to a VRF on the OS level, interfaces interconnect those VRFs

CP
drivers
Linux
kernel

VS

VRF 0

VR

VS

VRF 1

wrp1 wrpj1
eth2.5 eth2.10

VSW

VRF 2

VRF 3

VS

VRF 4

wrp2 wrpj2 wrpj3 wrp3


eth2.20

eth2.30
eth5

eth3
2014 Check Point Software Technologies Ltd.

181
181

VSX Packet Flow

When a packet arrives, the VSX Gateway determines


which Virtual Device should handle it. This process is
called Context Determination.
Each interface has a VRF ID.
VRF ID on packet translated to VS ID when FW-1
processing begins.
Currently VRF ID = VS ID.

2014 Check Point Software Technologies Ltd.

182
182

VSX Packet Flow Virtual Switch


Packet arrives at a shared i/f connected to a VSW

The VSW determines which Virtual System should handle the


packet

Based on the forwarding decision, the packet is sent to the


relevant Virtual System

Broadcasts when no matching forwarding entry exists

The Virtual System WRP interface will only handle packets


destined to its MAC address.

2014 Check Point Software Technologies Ltd.

183
183

VSX Packet Flow Virtual Router


Packet arrives at a shared i/f connected to a VR

If the targeted at the Virtual Router, the packet is


matched against VR security policy and sent to its IP
stack (if its policy permits).

Otherwise, the VR determines which VS should handle


the packet by doing a route lookup its routing table.

The packet is then forwarded to the relevant Virtual


System through a warp link.

2014 Check Point Software Technologies Ltd.

184
184

Performance Pack Warp Jump

Internet

Routing

Is the VR a performance
bottleneck? NO!

Only 1st packet is


inspected by the VS and
the VR.

Inspection

2014 Check Point Software Technologies Ltd.

190
190

Performance Pack Warp Jump


Internet

Inspection

following packets go
directly from the Virtual
System to the Physical
interface.

The Virtual Router is


skipped

We call it Warp Jump

2014 Check Point Software Technologies Ltd.

191
191

Files Structure

For the VSX (VS 0) the configuration files are located in


the regular folders under $FWDIR/$CPDIR
Each Virtual System has its own $FWDIR and $CPDIR
file structure (and some other DIRs), pointing to their
CTX directories to VSX (VS 0) $FWDIR
($FWDIR/CTX/CTX[vsid])
vsenv sets $FWDIR/$CPDIR to the correct values for
example: VS 0 $FWDIR = /opt/CPsuite-R77/fw1
VS 1 $FWDIR = /opt/CPsuite-R77/fw1/CTX/CTX00001
Binaries and configuration files which are global and
doesnt change between different Virtual Systems will be
symbolic links to the same file in VS 0

2014 Check Point Software Technologies Ltd.

192
192

Context Database and Registry


Context Database

The ctxdb.C file (under $CPDIR/conf of VS0) contains an


entry for each Virtual Device that holds Virtual Devices
specific configuration such as VRF, SIC name and more.

ctxdb.C is a global configuration file who is accessed by


all Virtual Systems via symbolic link
Registry

Each Virtual Device has its own registry, in the usual


place of $CPDIR/registry/HKLM_registry.data

2014 Check Point Software Technologies Ltd.

193
193

Main Processes in VSX

CONFD, RAD, GEOD single process, handle all VSs by


internal virtualization
CPWD single process, adapted to know about contexts,
and to pass environment variables to spawned processes.
FWK, FWD, CPD, VPND, ROUTED, CPHAMCSET and
many many others process per VS, with little/no changes
internally
FWK_FORKER, CPSICDEMUX, ROUTED manager
new processes, explained in next slide

2014 Check Point Software Technologies Ltd.

194
194

FWK_forker

FWK_forker process is an instance of fwk process

Responsible to spawn fwk processes upon a request

Coming from fwk_wd process invoked in a given context

2014 Check Point Software Technologies Ltd.

195
195

Technology

CPSICDemux

New daemon called


CPSICDemux was
added to dispatch
incoming SIC traffic
between the various
processes (ex. during
install policy) of different
VSs.

2014 Check Point Software Technologies Ltd.

196
196

CPSICDEMUX

Check Point SIC Demultiplexer runs in VSX context (VS 0), listens
to many SIC ports (18191, 18192, 257, 256, etc.).
New SIC connections from other member or from management
arrives to cpsicdemux, starts SIC handshake, and then connection is
passed to the correct CPD/FWD/etc. according to SIC name in the
handshake.

2014 Check Point Software Technologies Ltd.

197
197

CPSICDEMUX

2014 Check Point Software Technologies Ltd.

199
199

ROUTED manager

Routed manager

Runs in VSX context (VS 0)


Spawns routed instances per VS.
Handles connections from other members for route synchronization
Starts routed sync protocol negotiation, and then passes connection to
correct routed based on vsid passed in negotiation.

2014 Check Point Software Technologies Ltd.

200
200

Dynamic Routing

Full layer-3 dynamic routing are supported in VSX, in Virtual Systems


and Virtual Routers.
Supported protocols:

Unicast OSPF,RIP-v2, BGPv4


Multicast - IGMP,PIM-SM,PIM-DM

Each Virtual Device on each VSX cluster member has to be


configured separately since each Virtual Device has its own routing
daemon. Read /etc/routed[vsid].conf
Done via Gaias standard clish commands, after running set virtualsystem [vsid]

2014 Check Point Software Technologies Ltd.

201
201

Virtual Device creation stages


The following stages are executed as part of Virtual Device creation:

License validation (Additive license).


Update context database with Virtual Device information
$CPDIR/conf/ctxdb.C)
Create Virtual Device directories and soft-links:
$CPDIR/CTX/CTX00xxx/conf
$FWDIR/CTX/CTX00xxx/log, database,
Create Virtual Device registry
Create initial policy for the VS
Create the OS VRF instance
Start VS processes FWK, CPD, FWD, etc.
Pulls certificate for VS

2014 Check Point Software Technologies Ltd.

202
202

VSX Upgrade Procedure


Upgrade steps VSX Gateway to R77:
1. Install R77 on the VSX Gateway
2. Reboot the VSX Gateway.
3. Close SmartDashboard.
4. Upgrade the VSX Gateways in the Security Management server.
a) From the Security Management server CLI, run vsx_util upgrade.
b) Do the on-screen instructions.
5. Push the configuration to the VSX Gateways. Do these steps for each VSX Gateway or
cluster member.
a) Run vsx_util reconfigure.
b) Do the on-screen instructions.
The existing security policy is installed and configured on the upgraded VSX Gateway
and this message is shown:
Reconfigure module operation completed successfully
c) Reboot the VSX Gateway.
6. Install the necessary licenses.

2014 Check Point Software Technologies Ltd.

203
203

Optimal Service Upgrade

OSU provides a solution


for upgrading a VSX
cluster and Security
Gateway cluster to R77
without losing connectivity
Two cluster members are
used to maintain
connectivity, while you
upgrade all the other VSX
cluster members

2014 Check Point Software Technologies Ltd.

204
204

Optimal Service Upgrade procedure


1. For R67.10 VSX Gateways - Install the Optimal Service Upgrade hotfix on a cluster member.
2. Disconnect all old cluster members from the network, except for one cluster member.
3. Install R77 on all the cluster members that are not connected to the network.
4. On the old cluster member, run 'cphaosu start'
5. Reconnect the SYNC interface of one new cluster member to the network.
6. Move traffic to the new cluster member that is connected to the network. Do these steps:
a) Make sure the new cluster member is in ready state.
b) Connect the other new cluster member interfaces to the network.
c) On the new cluster member, run 'cphaosu start'
d) On the old cluster member, run 'cphaosu stat'
The network traffic statistics are shown.
e) When the old cluster member does not have many connections, run 'cphaosu finish'
7. On the new cluster member, run 'cphaosu finish'
8. Disconnect the old cluster member from the network.
9. Reconnect the other new cluster members to the network one at a time. Do these steps on each
cluster member:
a) Run cphastop
b) Connect the new cluster member to the network.
c) Run cphastart
10. Upgrade the old cluster member and reconnect it to the network.

2014 Check Point Software Technologies Ltd.

205
205

Thank you !
Please proceed to lab 7

2014 Check Point Software Technologies Ltd.

VSX R77
Debug &
Troubleshooting

2014 Check Point Software Technologies Ltd.

Course Timetables

9:00

Day 1

Day 2

Day 3

Course Introduction

VSX Clustering

VSX Conversion

vsx_utill

Gaia VS CTX & New Features


(Conversion, SNMP, JF)

Mgmt. Implementation

Gaia VSX Intro

10:00

11:00

R77 VSX Introduction

12:00
13:00

Lunch Break

14:00
VSX Networking

GW Implementation

15:00
Open Questions

16:00
17:00

VSX CoreXL Affinity &


Debug & Troubleshooting
Memory RC

2014 Check Point Software Technologies Ltd.

208
208

Management
Debugging
2014 Check Point Software Technologies Ltd.

209
209

Debugging VSX - fwm

Process fwm is the Management server main


process
All fwm debug messages are written to
$FWDIR/log/fwm.elg
VSX provisioning module vsxm is
implemented as a COM object within the fwm
process

2014 Check Point Software Technologies Ltd.

210
210

fwm debug debug flags

TDERROR is an error logging infrastructure used for


reporting debug messages
Messages are printed to console or to an error log file.
Messages have topics and severity levels assigned to
them.
There is a way to enable different debugging "filters" at
runtime
VSX Debugging useful debug flags:
VSX provisioning and vsx_util:
TDERROR_ALL_VSXM
Policy installation: TDERROR_ALL_INSTMGR

2014 Check Point Software Technologies Ltd.

211
211

fwm debugs how to

How to turn debugs on:


Signal the fwm process to set its debugs on
fw debug fwm on TDERROR_ALL_ALL=INFO
Set the debug flag and restart the process

Export TDERROR_ALL_ALL=INFO
Restart fwm process

Turning debugs off:


This should work:
fw

debug fwm TDERROR_ALL_ALL=0


fw debug fwm off

In case it doesnt work - restarting the process will do the job

2014 Check Point Software Technologies Ltd.

212
212

Fwm debugs - cont

The debugs output will be written to the file:


$FWDIR/log/fwm.elg

For Provider-1, the debugging has to be set in the


DMS context. The debug output file will also exist
per DMS.

VSX provisioning debugs will appear in main DMS


Policy installation debugs will appear in target DMS

2014 Check Point Software Technologies Ltd.

213
213

Policy installation CLI

Install policy from CLI:

fwm load <policy_name> target

2014 Check Point Software Technologies Ltd.

214
214

Debugging SIC

cpca certificate authority process

fw debug cpca on TDERROR_ALL_ALL=5

Useful command line:

Revoking a certificate in the management

cpca_client revoke_cert -n <sic_name>

An example for the SIC name for Virtual System VS1e on


Member ec1:
"cn=ec1_vs1e,o=DMS1..9jdypf"
To find the SIC name for a certain object:
cpca_dbutil

print $FWDIR/conf/InternalCA.db | grep


<object_name>
2014 Check Point Software Technologies Ltd.

216
216

SmartView Monitor

SmartView Monitor
Various counters (Dropped, accepted, rejected) available per VS
Real time monitoring (Top Connections , Users) available per VS
SNMP
Chkpnt.mib available in the VSX module under $CPDIR/lib/snmp
OID for VSX queries 1.3.6.1.4.1.2620.1.16
All VS are queried via the only management IP (of the VSX GW)

2014 Check Point Software Technologies Ltd.

217
217

Module
Debugging
2014 Check Point Software Technologies Ltd.

218
218

vsx stat

See the status of the VSX and Virtual System:

2014 Check Point Software Technologies Ltd.

219
219

OS sniffer

tcpdump i <if name> expression

Example: tcpdump I eth2.11 arp or icmp

2014 Check Point Software Technologies Ltd.

220
220

Firewall monitor
fw monitor [v <vsid>] [-e expression]
Example: fw monitor v 4 e port(520) and
ip_p=17,accept;

2014 Check Point Software Technologies Ltd.

221
221

Firewall tables
Per context command.
fw [-i k] tab -t table_name [-s]
Example - obtain vs4 connections table:
fw tab t connections -s

2014 Check Point Software Technologies Ltd.

222
222

Kernel debugs

Setting Kernel debugs:

fw [-i k] ctl zdebug [-v "<vs ids>"|all] [-x] [-m


<module>] [+|- <debug_flags> ]
Usage example:
ctl zdebug vs 1 + conn ld
fw ctl zdebug m cluster + forward
fw i k ctl zdebug (dispatcher debugging only)
fw

See all possible debugs:


fw

ctl debug --help

2014 Check Point Software Technologies Ltd.

223
223

Kernel debugs debug drop

Kernel debug drop shows the packets that are


dropped with the reason:

2014 Check Point Software Technologies Ltd.

224
224

ClusterXL debugs

Get Cluster status

cphaprob [-vs vsid] stat

(VS0 shows global state)

See the member interfaces status

Advanced debugging:

cphaprob [-vs vsid] stat a if


cphaprob [-vs vsid] list

Down a member useful to test failovers

clusterXL_admin up/down
2014 Check Point Software Technologies Ltd.

225
225

ClusterXL Cont.

cphaprob stat output:

2014 Check Point Software Technologies Ltd.

226
226

SecureXL
fwaccel vs <vsid>
{conns|templates|stat|on|off}

2014 Check Point Software Technologies Ltd.

227
227

SecureXL debugs

Setting debugs:

Performance Pack debugs:

fwaccel dbg

sim dbg [-m <...>] [resetall | reset | list | all |


mask | +/- <flags>]

The output file is /var/log/messages unless a


buffer was allocated using
fw ctl debug buf <buffer size>

2014 Check Point Software Technologies Ltd.

228
228

CPD debugs

Useful in debugging Push configuration,


SIC issues, Policy installation

Turning on CPD debugs:

cpd_admin debug on[TDERROR_ALL_ALL=5]

The output is written to:

$CPDIR/log/cpd.elg
2014 Check Point Software Technologies Ltd.

229
229

Fetching policy

Fetching the last installed policy:


fw [-vs <vsid>] fetch local

Fetching the last policy that failed to be installed


fw fetchlocal -d $FWDIR/state/__tmp/FW1/

Unloading the policy:


fw [-vs <vsid>] unloadlocal

Unload policy from all Virtual Systems:


fw vsx unloadall

2014 Check Point Software Technologies Ltd.

230
230

Fetching Configuration

Fetching configuration:

fw vsx fetch

2014 Check Point Software Technologies Ltd.

231
231

Fetching configuration cont.

Fetching configuration for a specific Virtual


System:

Fetching the last configuration that failed to be


installed

fw vsx fetchvs <vsid>

fw vsx fetch -v lastbad

Verify that configuration is updated

fw vsx fetch n

2014 Check Point Software Technologies Ltd.

232
232

NCS

See the NCS script for a specific Virtual


Device:

fw vsx showncs <vsid>

Shows the
part of
local.vsall
that is
relevant for
VS with
vsid 1

2014 Check Point Software Technologies Ltd.

233
233

SIC

Resetting the SIC on the VSX Gateway/Cluster:

cp_conf sic init <new OTP>

Resetting SIC on a specific Virtual System:

fw vsx sicreset (per context command)

Manually pulling the certificate for a specific Virtual


System(per context command):
cp_pull_cert -d -h <mgmt_ip> -n <vs_name
For example:
- cp_pull_cert -d -h 172.16.16.145 -n Jack_vs2

2014 Check Point Software Technologies Ltd.

234
234

Watchdog

Viewing all monitored processes (fwk, cpd,


fwd):

cpwd_admin list

Viewing monitored process of a specific VS:

cpwd_admin list ctx <vsid>

2014 Check Point Software Technologies Ltd.

235
235

FWK Debugging

FWK is the FW-1 driver.


There are many ways to debug FWK:

$FWDIR/log/fwk.elg
fw ctl zdebug v <fwk VSID>
gdb/valgrind (or any other U/M debugging tool).

2014 Check Point Software Technologies Ltd.

236
236

Common Error
Messages

2014 Check Point Software Technologies Ltd.

237
237

Push configuration common errors

Failed to configure ecu with the


following errors:
ecu2 error :Failed to get VSX
gateway's name from database.

Cause: This error occurs when there is no policy


installed on the VSX Cluster/Gateway (VSID 0)
Solution: Simply install policy on the VSX
Cluster/Gateway

2014 Check Point Software Technologies Ltd.

238
238

Error Scenario
Push
configuration
to VS1 fails

View the error in the Report Dialog. Examine


the error. Is it coming from the module or the
management?
Note: errors coming from the module have
<module_name: > in the error message
Management error

Module error
Try to manually load and debug the
new configuration on the module:
fw d vsx fetch -v lastbad

Turn the fwm debug on,


on the relevant DMS

Press the OK
button again

Look for the reason of the error


inside the debug prints
2014 Check Point Software Technologies Ltd.

239
239

Thank you !
Please proceed to lab 8

2014 Check Point Software Technologies Ltd.

VSX R77
Conversion to VSX

2014 Check Point Software Technologies Ltd.

Course Timetables

9:00

Day 1

Day 2

Day 3

Course Introduction

VSX Clustering

VSX Conversion

vsx_utill

Gaia VS CTX & New Features


(Conversion, SNMP, JF)

Mgmt. Implementation

Gaia VSX Intro

10:00
11:00

R77 VSX Introduction

12:00
13:00

Lunch Break

14:00
VSX Networking

GW Implementation

15:00
Open Questions
16:00
17:00

VSX CoreXL Affinity &


Debug & Troubleshooting
Memory RC

2014 Check Point Software Technologies Ltd.

242
242

Agenda

Conversion
Implicit conversion

2014 Check Point Software Technologies Ltd.

243
243

Conversion to VSX

What is conversion to VSX?


Conversion is the action of transforming a regular Security
gateway/cluster into a VSX gateway/cluster.
When conversion is used?
The conversion mechanism is used when a user wishes to change its
security gateway/cluster to a VSX gateway/cluster.
or when the user creates a VSX gateway/cluster from a clean installed
gateway/s.
Where the conversion is taking place?
The Conversion is done in the management database and for every
gateway/s.

2014 Check Point Software Technologies Ltd.

244
244

Conversion to VSX

How to convert Security gateway to VSX ?


Open SmartDashboard.
From the Network Objects tree, right-click the Security Gateway or
cluster and select Convert to VSX.
The Welcome window opens.
Click Next The Compatibility Check window opens.
The wizard makes sure that the Security Gateway or cluster is
compatible with VSX.
Click Convert The Conversion Process window opens.
Click Finish The Converting window is shown as the management
database is updated.

2014 Check Point Software Technologies Ltd.

245
245

Conversion to VSX

How conversion is done?


The conversion command is initiated by the management.
In order to communicate with the gateways, the management is using
an infrastructure called cprid.
The management initiates a conversion script for every member called
set_fw_opt_mode.bash the script does not run simultaneously in
every member, the management waits until one member is done, and
then initiates the script at the next member, the script is located at the
gateway in $FWDIR/bin directory.
The script stops some firewall processes and removes kernel
modules.
Two flags in the registry indicating whether the machine is a VSX
machine, are changed.
The modules and the processes related to the VSX are turned on.
The whole conversion is done without reboot.
2014 Check Point Software Technologies Ltd.

246
246

Conversion to VSX stages

In a regular conversion to VSX the mechanism goes through the


following stages.
Compatibility Check
- GUI Compatibility Check

Firewall must be on
blades that are not supported in VSX must be off( Mobile access, Anti-spam, DLP)
Legacy blades must be off ( URL filtering, Traditional Anti-Virus .)

- Management Compatibility Check

check that there is only one sync interface.


Check that the interface names are the same for the members and the cluster.
Check that the members IPs are from the same subnet as the cluster IP.

2014 Check Point Software Technologies Ltd.

247
247

Conversion to VSX stages

Check connectivity to the gateways.


If it is a cluster - Set the conversion order of the members ( the active
member is last in order to avoid connectivity loss).
Run the conversion script for every member.
Script stages:

Gateway Compatibility Check

Check that the FW module in installed


Check that there are no virtual devices.
Check that the interfaces type is compatible ( ethernet, vlan, bond, loopback).
Check that there are no aliases.
Check that IPV6 is not used.
Check that source base routing is disabled.

Create a timeout auto rollback script


Stop networking
Stop firewall processes
Remove firewall modules from the kernel ( fw, vpn, sim).
Change the registry flags VSX and USERMODE to on ( most important ).
2014 Check Point Software Technologies Ltd.

248
248

Conversion to VSX stages

Script stages (continue):

Load firewall drivers.


Start VSX processes.
Start networking.
Call post install hooks ( configuring VSX mode in the clish and enabling the
vsenv command )

The management connects to the gateway and kill the


self rollback script called converter_terminator.

Update management database ( Create vs_slot objects


).

2014 Check Point Software Technologies Ltd.

249
249

Conversion to VSX terminator

The terminator is a script spawned from the conversion script


that sleeps for 9 minutes.
It was designed to reboot the gateway after the sleep if the
connectivity to the management is lost and the reconnection
attempts failed.
Two script at the gateway startup sequence are called in order
to rollback the gateway to its former configuration ( before the
conversion ).
If the connection to the gateway was restored The management
sends a remote kill command to the terminator script.

2014 Check Point Software Technologies Ltd.

250
250

Conversion to VSX log


A log is created at the gateway at
/var/log/conversion_%day_%time.
Example: cat /var/log/conversion_25_18_12_18.log
log:[0] checking prerequisites
log:[0] obtaining current firewall status and operation mode status
log:[0] checking if the requested mode vsx/um is already set
log:[0] passed prerequisites
log:[1] Successfully created rollback script
log:[2] succeeded to create terminatior
log:[3] succeeded to start terminatior
log:[4] stopping networking
log:[4] networking stopped
log:[5] stopping routed
log:[5] adding ctx column to watchdog
log:[5] executing cpstop
log:[5] cpstop operation done
2014 Check Point Software Technologies Ltd.

251
251

Conversion to VSX log cont.


log:[6] setting appropriate variables in registry for operation mode
log:[6] setting appropriate variables in registry for firewall mode
log:[6] mode changed to vsx/um
log:[7] executing cpstart
log:[7] starting routed
log:[7] cpstart operation done
log:[8] starting networking
log:[8] networking started
log:[9] running post script hooks
log:[9] executing post script /opt/CPsuite-R77/fw1/scripts/post_conversion_hooks//dbset_hook.bash
log:[9] executing post script /opt/CPsuite-R77/fw1/scripts/post_conversion_hooks//vsenv_hook.bash
log:[9] total conversion time: 55 seconds

2014 Check Point Software Technologies Ltd.

252
252

Implicit Conversion to VSX


Implicit conversion
Implicit conversion is done when creating a new VSX cluster/gateway
object at the management.
Although the gateways are not VSX, they are converted to a VSX
gateways.
In implicit conversion there are no compatibility checks and no networking
restarts.

All the scripts run simultaneously.


At the end a vsx_slot object is created at the management.
The script for implicit conversion is called gw_to_vsx and it located at
$FWDIR/bin.

2014 Check Point Software Technologies Ltd.

253
253

Implicit Conversion to VSX log


A log is created at the gateway at /var/log/gw_to_vsx.log
Example: cat /var/log/gw_to_vsx.log.
Wed Jul 25 19:14:53 IDT 2012: converting module from gw to cluster
Wed Jul 25 19:14:53 IDT 2012: Recieved the command convert.
Wed Jul 25 19:14:53 IDT 2012: Stopping all processes.
Wed Jul 25 19:15:16 IDT 2012: Removing drivers.
Wed Jul 25 19:15:21 IDT 2012: Starting all processes and drivers.
Wed Jul 25 19:16:10 IDT 2012: Verifying...
Wed Jul 25 19:16:11 IDT 2012: Verification: Ok.
Wed Jul 25 19:16:11 IDT 2012: running post script hooks
Wed Jul 25 19:16:11 IDT 2012: executing post script /opt/CPsuiteR77/fw1/scripts/post_conversion_hooks//dbset_hook.bash
Wed Jul 25 19:16:11 IDT 2012: executing post script /opt/CPsuiteR77/fw1/scripts/post_conversion_hooks//vsenv_hook.bash
Wed Jul 25 19:16:11 IDT 2012: total conversion time: 78 seconds
Wed Jul 25 19:16:11 IDT 2012: conversion done successfully!
Wed Jul 25 19:16:15 IDT 2012: Received the command terminator_stop.

2014 Check Point Software Technologies Ltd.

254
254

Summary

Conversion is done when changing a gateway/cluster to a VSX


cluster.
Implicit conversion is initiated when creating a new vsx
gateway/cluster object at the management.
Non implicit conversion is initiated when choosing the convert
to VSX at the Smarthdashboard.

2014 Check Point Software Technologies Ltd.

255
255

Thank you !
Please proceed to lab 9

2014 Check Point Software Technologies Ltd.

Annex VSX Layer 2 HA

2014 Check Point Software Technologies Ltd.

Course Timetables

9:00

Day 1

Day 2

Day 3

Course Introduction

VSX Clustering

VSX Conversion

vsx_utill

Gaia VS CTX & New Features


(Conversion, SNMP, JF)

Mgmt. Implementation

Gaia VSX Intro

10:00
11:00

R77 VSX Introduction

12:00
13:00

Lunch Break

14:00
VSX Networking

GW Implementation

15:00
Open Questions
16:00
17:00

VSX CoreXL Affinity &


Debug & Troubleshooting
Memory RC

2014 Check Point Software Technologies Ltd.

258
258

Agenda

Spanning Tree Protocol


STP Bridge Mode
Limitations of STP Bridge Mode
Active/Standby Bridge Mode (A.K.A. Donald)
Active/Active Bridge mode with LACP

2014 Check Point Software Technologies Ltd.

259
259

Spanning Tree Protocol

2014 Check Point Software Technologies Ltd.

260
260

Spanning Tree Protocol

Why use STP?

What is STP?

To build redundant layer-2 networks


Allows switches to communicate with each other to discover Layer-2 loops
and activate an algorithm to create a loop-free topology.

Two main stages of operation

Electing a Root switch

Each switch has a MAC address and a configurable priority number

Determining and verifying the topology of the network

Sending Bridge Protocol Data Units (BPDU) packets


2 second interval

2014 Check Point Software Technologies Ltd.

261
261

Phase 1: Root Election

My ID is:

My ID is:

AA:AA:AA:AA:AA:AA

BB:BB:BB:BB:BB:BB

Election!
My ID is:

My ID is:
CC:CC:CC:CC:CC:CC

DD:DD:DD:DD:DD:DD

Each switch can initiate an election


2014 Check Point Software Technologies Ltd.

262
262

Phase 1: Root Election

Im the root!

Ok

Ok

Ok

The switch with the lowest ID is chosen as root


2014 Check Point Software Technologies Ltd.

263
263

Phase 2: Sending BPDUs


Root
A

19

B
38

19

I will block
this port
4

Sender ID is:
BB:BB:BB:BB:BB:BB

38

I found a
loop!

Sender ID is:
CC:CC:CC:CC:CC:CC
2014 Check Point Software Technologies Ltd.

264
264

Cluster in STP Bridge Mode

2014 Check Point Software Technologies Ltd.

265
265

STP Bridge Mode

Similar to 3rd party mode


Load Sharing decisions done by STP
Both members are Active
No CCP on interfaces
Link State only
Provides High Availability
Does not provide Load Sharing in STP
Can provide Load Sharing with PVST not easy
to configure
2014 Check Point Software Technologies Ltd.

266
266

STP Bridge Mode

Root

Processing
All traffic

Traffic arrives
to switch
B

2014 Check Point Software Technologies Ltd.

Processing
some of the traffic
Traffic blocked
Trafficby
is switch
sent
but dropped
because the
port is blocked

267
267

STP Bridge Mode Limitations

2014 Check Point Software Technologies Ltd.

268
268

Problems in STP Bridge Mode

Works only with STP


1 minute failover timeout (STP)
5 seconds failover timeout (RSTP)
Active/Standby decision determined by STP

Both members handle some of the packets


Unsupported features, like in regular FW-1:

Active streaming
VPN
Authentication
Security Servers
NAT

Does not support VSLS

2014 Check Point Software Technologies Ltd.

269
269

Three-Layered Hierarchical Model

Access
Its main function is to connect users.
LAN Switches

Distribution
The distribution or policy layer performs the policy-based
operations: routing, firewalling.
Routers

Core
The backbone of the network. It should be high-speed and
concerned mainly with switching traffic as quickly as possible.
Backbone Switches
2014 Check Point Software Technologies Ltd.

270
270

Deployment Scenario

VLAN 20

VLAN 20

VLAN 10

VLAN 10

Customers requirement is to connect the firewalls to the routers


2014 Check Point Software Technologies Ltd.

271
271

Active/Standby Bridge Mode

2014 Check Point Software Technologies Ltd.

272
272

Active/Standby Bridge Mode

Cluster state dictated by ClusterXL


Standby member drops all traffic
All members do not pass STP
Active member learns MAC addresses
MAC addresses synchronized to standby
member
During Failover, switches are updated to forward
traffic to newly active member

2014 Check Point Software Technologies Ltd.

273
273

Active/Standby Bridge Mode

br_shadow

br_shadow
00:12:00:ab:00:01 eth0
00:12:00:ab:00:02 eth0
00:12:00:ab:00:03 eth1
00:12:00:ab:00:04 eth1
00:12:00:ab:00:05 eth1
00:12:00:ab:00:06 eth1

2014 Check Point Software Technologies Ltd.

A
S

00:12:00:ab:00:01 eth0
00:12:00:ab:00:02 eth0
00:12:00:ab:00:03 eth1
00:12:00:ab:00:04 eth1
00:12:00:ab:00:05 eth1
00:12:00:ab:00:06 eth1

274
274

Analysis

Advantages

Full control of the bridge failover


Instantaneous failover
VSLS can now work in bridge mode
Distribution layer can now be protected
Supports Mixed Mode (bridges connected to a router)
Bridge Interface monitoring

Limitations

STP tree is broken

2014 Check Point Software Technologies Ltd.

275
275

Active/Active Bridge Mode with


LACP

2014 Check Point Software Technologies Ltd.

276
276

Active/Active Bridge Mode with LACP

Still experimental. Not officially supported by


CP.
All members are active.
Switches behave as external load balancers.
Similar to STP Mode in terms of FW-1
configuration and behavior.

2014 Check Point Software Technologies Ltd.

277
277

LACP

Link Aggregation Control Protocol


Industry standard (802.3ad/802.1AX)
Used to dynamically detect multiple links
between switches
Dynamically bond links together to create a
single logical link, with more bandwidth and
reliability.
Switches send/receive LACP packets to learn
topology.
Special procedure required to allow LACP
packets to pass through FW-1
2014 Check Point Software Technologies Ltd.

278
278

LACP example
LACP negotiation

LACP packets

LACP packets

2014 Check Point Software Technologies Ltd.

279
279

LACP - Load Balance method

Once a logical link is established, switches


send packets on logical link
Choice of physical link done using some
algorithm
Non-standard algorithms. Depend on vendor
and model.
Usually considers source/destination MAC/IP,
or some combination of them.

2014 Check Point Software Technologies Ltd.

280
280

LACP load-balance example

2014 Check Point Software Technologies Ltd.

281
281

LACP with VSX - diagram

2014 Check Point Software Technologies Ltd.

282
282

LACP with VSX

Each member inspects one of physical wires,


while working in Bridge Mode.
A VS can inspect untagged traffic, or a specific
VLAN.
ClusterXL is almost inactive. Switches decide
where to send packets to.
LACP packets pass via all members. (on
VLAN #1)

2014 Check Point Software Technologies Ltd.

283
283

LACP with VSX - stickiness

For correct inspection of traffic, the same


connection must pass on the same member.
(symmetric routing)
It is the responsibility of the administrator to
configure the switches in such a way that
connections are sticky
Algorithm varies between vendors and models.
Even the order of the cables in the switches
may be important.
2014 Check Point Software Technologies Ltd.

284
284

LACP with VSX - failover

When a member goes down, LACP packet dont pass


through it. Switches renegotiate and pass around failed
member.

LACP packets

2014 Check Point Software Technologies Ltd.

285
285

Analysis

Advantages

Provides Load Sharing


Scalable to multiple members

Limitations

Slow failover. Depends on switch, but usually around


30 seconds.
Requires same algorithm on both switches for
stickiness.
No active monitoring by ClusterXL. Only by switches.
Not as scalable as VSLS no Backup mode to reduce
Sync

2014 Check Point Software Technologies Ltd.

286
286

Summary

STP bridge mode is ideal for internal network


deployment.
Active-Standby bridge mode is for network
perimeter deployment.
Active-Standby bridge mode has wider
functionality and feature set.
Active-Active with LACP is generally easy to
deploy and provides true LS, but may have
slow failovers.
2014 Check Point Software Technologies Ltd.

287
287

Thank you !

2014 Check Point Software Technologies Ltd.

R77 SNMP Per VS

2014 Check Point Software Technologies Ltd.

Motivation

In versions prior to R77, VSX GW is capable of


reply SNMP requests regarding VS0 statuses
and a very limited set of non-VS0 statuses.
The capability of polling data regarding nonVS0 Virtual Devices is important for several
reasons:
1.

2.

Extended Monitoring Capabilities: Each VS


should have similar monitoring capabilities as FW1.
Error Detection: By using SNMP request in some
polling interval, administrators are capable to
detect and fix errors.
2014 Check Point Software Technologies Ltd.

290
290

Design Overview

There are two modes of SNMP monitoring that you can use with VSX:
1.
Default mode

lets you monitor only VS0.

snmpd0 listens on all interfaces.


2.
VS mode

lets you monitor all of the Virtual Systems in the VSX


Gateway.

snmpd0 listens on all interfaces, non-snmpd0 listens on


loopback and on UDS.
When SNMP agent is enabled, PM starts snmpd0 and a process
named snmp_launcher.

2014 Check Point Software Technologies Ltd.

291
291

Supported SNMP Versions

VS mode uses SNMP version 3 to query the Virtual


Systems. You can run remote SNMP queries on Virtual
Systems in the VSX Gateway without changing the Virtual
System environment.
For systems that only support SNMP versions 1 and 2:
You cannot run remote SNMP queries for each Virtual
System.
You can only run a remote SNMP query on VS0.
You can use the CLI to change the Virtual System
context and then run a local SNMP query on a Virtual
System.

2014 Check Point Software Technologies Ltd.

292
292

Enabling VS Mode using CLISH

To enable VS mode on the VSX Gateway:


1.

Configure an SNMP v3 user


VSX-Box> add snmp usm user admin security-level authNoPriv auth-passphrase zubur123

2.

Enable VS mode
VSX-Box> set snmp mode vs

3.

Start the SNMP agent


VSX-Box> set snmp agent on

2014 Check Point Software Technologies Ltd.

293
293

SNMP commands Examples

OS OIDs request
[Expert@VSX-Box:2]
Query from VS
context:
snmpwalk
-v 2c -c public localhost ifDescr

snmpwalk
-n ctxname_vsid2
-v 3 -l authNoPriv
[Expert@VSX-Box:0]
Query non-VS0
from
VS0 context:
-u admin -A zubur123 localhost ifDescr

[Expert@Mgmt] snmpwalk -n ctxname_vsid2 -v 3 -l authNoPriv -u


admin -A zubur123 172.16.16.77 ifDescr

Query non-VS0 via Remote host:

2014 Check Point Software Technologies Ltd.

294
294

SNMP commands Examples

CP OIDs request:
[Expert@VSX-Box:2]
Query from VS
context:
snmpwalk
-m $CPDIR/lib/snmp/chkpnt.mib -v 2c -c public
localhost fwFilterDate

[Expert@VSX-Box:0] snmpwalk -m $CPDIR/lib/snmp/chkpnt.mib -n ctxname_vsid2

-vQuery
non-VS0
from
VS0 context:
3 -l authNoPriv
-u admin
-A zubur123
localhost fwFilterDate

[Expert@Mgmt] snmpwalk -m $CPDIR/lib/snmp/chkpnt.mib -n ctxname_vsid2 -v 3


-l authNoPriv -u admin -A zubur123 172.16.16.77 fwFilterDate

Query non-VS0 via Remote host:

2014 Check Point Software Technologies Ltd.

295
295

SNMP commands Examples

Important OS full trees:


snmpwalk
ALL statuses
query:
-v 2c -c public
localhost

snmpwalk -v 2c -c public localhost iftable

Standard interface table query:

snmpwalk -v 2c -c public localhost ifxtable

Extended interface table query:

2014 Check Point Software Technologies Ltd.

296
296

Troubleshooting

1.

Steady state of SNMP per VS should be


composed of:

Snmp_launcher process running


snmpd process running under each VRF context.
i.
ii.

VS0 will have a process named 'snmpd'.


Non-VS0 will have a process named snmpd_<vsid>.

[Expert@VSX-Box:0]# ps fx | grep snmp


5252 ?
Ss
0:00 \_ /usr/sbin/snmp_launcher
5265 ?
S
0:00 |
\_ snmpd_1 -f -C -c /etc/snmp/vsx-proxy/CTX/1/snmpd.user.conf,/etc/snmp/vsxproxy/CTX/1/snmpd.local.conf /tmp/snmpd1_uds localhost
5274 ?
S
0:00 |
\_ snmpd_2 -f -C -c /etc/snmp/vsx-proxy/CTX/2/snmpd.user.conf,/etc/snmp/vsxproxy/CTX/2/snmpd.local.conf /tmp/snmpd2_uds localhost
5323 ?
Ss
0:00 \_ /usr/sbin/snmpd -f -c /etc/snmp/vsx-proxy/snmpd.vsx.proxy.conf -p
/etc/snmp/snmpd.pid

2014 Check Point Software Technologies Ltd.

297
297

Troubleshooting

2.

If query to some non-VS0 fails, the following


should be verified:

3.

If the request is OS OID verify that snmpd0


and snmpd_<vsid> exist.
If the request is CP OID verify that snmpd0
and snmpd_<vsid> exist and CPD of
corresponding VS is running.

If snmpd0 is query-able via SNMP V1/2 but is


not query-able via SNMP V3, check that a
USM is configured properly.
2014 Check Point Software Technologies Ltd.

298
298

R77
Gaia and VSX specific
commands

2014 Check Point Software Technologies Ltd.

Gaia and Clish- Overview

Gaia is the next generation operating system, unifying IPSO and


SPLAT to support all appliance product lines
Default shell is an IPSO like shell called clish

2014 Check Point Software Technologies Ltd.

300
300

Gaia and Clish- Overview

Gaia uses a WebUI which is available only during the First Time
Wizard

All configuration,
whether done from
WebUI or clish,
is saved under
/config/active

2014 Check Point Software Technologies Ltd.

301
301

Clish - VSX specific commands

To enable/disable virtualization, use:


set vsx on
set vsx off

To show virtual system/s


show virtual-system all

Default context is VSX-Box. All clish commands are similar to those


of regular Gaia OS, but context-dependent. In order to switch the
context use:
set virtual-system <vsid>

Interface configuration is disabled in VSX mode - set commands


are blocked.

2014 Check Point Software Technologies Ltd.

302
302