IsarFlow for Cisco Carrier Grade NAT (CGN) – Q & A
Introduction
Internet providers have to implement some NAT solution due to the IPv4 address depletion to enable IPv6 transition. Due to the huge number of concurrent IP connections there is a hardware based solution available by e.g. Cisco, the CGSE and ASR1000/ASR9000. Other options to implement CGN are ASA firewalls or other vendors. There are different technologies available like NAT 44, NAT 64, DS-lite, resulting in different reporting types. There are strong requirements of storing the NAT events (e.g. IP addresses used by end customers) for reasons like data retention, law enforcement or carrier operation. This is addressed by IsarFlow using a special module which provides CGN analyses. IsarFlow receives the logging data via Netflow v9 and provides multiple analyses. It supports NAT 44, NAT 64, DS-lite and NSEL. The data collector performance is optimized to support even highest event rates and to store the data in a compressed format (this compress the data up to 10:1).
The IsarFlow system allows to implement multiple analyzers each running a netflow collector with one portal to run analyses (distributed setup). In such an environment it is recommended to place the analyzers close to the data sources (e.g. one analyzer per CGN device per region). IsarFlow offers also a high availability option, which reduces the risk of loss of data to a minimum. This means, every analyzer is implemented as a pair of two servers, sharing a virtual IP address. The data are stored in a compressed format to reduce the size of the system, especially in the case of high volume implementations. If accounting information is available (e.g. radius accounting records, MSISDN, username, …), it can be included and correlated by the analysis.
Benefits of IsarFlow CGN Q: Why do I need IsarFlow CGN? A: There are strong requirements to store NAT bindings in logs (e.g. by law). IsarFlow CGN solves this issue even in high performance environment.
Q: What is the maximum retention time of the data?
A: The retention time is limited only by storage size and can be configured.
Q: What to do in case of a server outage?
A: There is the option of high availability for the server, the data are sent to a virtual server (virtual ip). The two servers have to be in the same L2 subnet. All other requirements are fulfilled with the IsarFlow CGN installation. HA is not available for the portal function yet.
Q: What happens, if there is any data loss between CGN device and IsarFlow (database)? A: The (CGN Netflow as well as Radius) data have to be complete and accurate to rely on the analysis result. If there is any data loss, the analysis result cannot be used in any case, since it does not satisfy a legally correct use.
Project Rollout Q: Are there any recommendation regarding rollout? A: The minimum time depends mostly on purchase process All: Project kickoff meeting = week 0 Customer: to put HW/Storage/IsarNet purchase order = week 1 Customer: receive HW/Storage = week 2 to week 9 Customer: HW/Storage installation @ commissioning = week 9 to week 10 Partner supported by IsarNet: IsarFlow CGN installation = week 11 All: system integration if needed = + weeks Customer: acceptance test = week 12+
Q: What does “system integration” mean?
A: technical aspects like setting up IsarFlow CGN users or integrate Radius data A: organizational aspects like integrating in customer’s workflow
Technical Solution IsarFlow CGN Q: What CGN platforms do you support? A: Cisco CGSE, ASR9000, ASR1000, ASA, Huawei E8000, E9000
Q: What CGN technologies do you support?
A: NAT44, NAT64, DS-lite, DBL (destination based logging), BPA (bulk port allocation), NSEL (netflow secure event logging)
Q: Who can help us configuring CGN?
A: Usually this is part of the overall project and provided by the vendor (Cisco, Huawei) or a local partner. IsarFlow can submit a bid for consulting if needed. Some examples of the netflow export configuration are included in the user guide.
Q: Does IsarFlow CGN support usual Netflow v9 or Netflow v5?
A: IsarFlow CGN can only handle CGN events that are sent via Netflow v9. Usual flow records, flexible Netflow, or other flow capturing protocols are not supported by IsarFlow CGN. Please ask for “classical IsarFlow” if this information is needed.
Q: Can I have multiple exports to multiple IsarFlow servers for redundancy?
A: Cisco doesn’t support multiple exports for CGN events yet.
Q: Do I need any kind of special operating system?
A: No, the necessary SuSE Linux and third party licenses are included in the offer of IsarNet. Other Linux OS are not supported. We don't want to spend all the effort (we had to test a lot more, the complexity increased and so on).
Q: Is it possible to use my standard database server?
A: No. IsarFlow CGN uses a special database engine for performance reason.
Q: How many flows per second can be sent to IsarFlow collector?
A: It depends on NAT technology and vendor. Up to 1.8 million flow/sec in case of Cisco CGSE NAT44 events can be processed per IsarFlow server (during CGN device failover).
Q: Where from do I get the numbers of CGN events for dimensioning?
A: There is a Cisco tool available to calculate such numbers. Please ask Cisco to provide this tool. Of course it’s just a rough estimation, as traffic patterns are quite different.
Q: Do you support time zones?
A: Yes, the time zone can be changed for every analysis. The server is running UTC time zone.
Q: Which timing information does IsarFlow CGN use? A: The IsarFlow CGN system is using IsarFlow server's timestamps for both netflow (CGN events) and subscriber (Radius) data. Proper clock synchronization of the IsarFlow servers using NTP is mandatory.
Q: What can be searched in IsarFlow CGN?
A: 1. for the public IP address to find out the associated private IP address(es)/port 2. for the private IP address to find out the associated public IP address/port. Account information can be displayed additionally if the according tables are available.
Q: What means public/private address?
A: This is based on Cisco’s wording of CGSE devices. Other vendors/devices use other wording. There will be a special whitepaper available explaining this wording.
Q: How can I include subscriber information?
A: This type of information can be included into analyses. Prerequisite is to fill a database table in IsarFlow e.g. via ODBC. There is also the option to run a Radius accounting server on IsarFlow server to get this information.
Q: Is there a 100%-correlation of CGN events and subscriber data possible?
A: No, this is technically impossible. It depends on a large number of system parameter, like timing of CGN device, subscriber server etc., or the reliability of event transport (UDP based!).
Q: There is only one time to choose in the analysis form, why?
A: The main purpose of such analysis is to find a special NAT entry at a specific time (this time is provided e.g. by the lawyer). Therefore a search is performed “around” this time. For a more specific explanation please see the user guide.
Q: Sometimes the analysis needs a long time; it tells me something like “loading data”. Why? A: To get an unlimited retention time (configurable, depending on storage) the data will be offloaded and loaded into the database during the analysis. Therefore the analysis needs some more time. Please keep in mind you are searching in terabytes of data usually.
Q: What does “data offloading” mean?
A: This means historical data are moved from the database into binary data files. If the data are needed during analysis of past events they will be loaded into the database during analysis (on demand).
Q: Is there any type of encryption of the data
A: No, IsarFlow CGN is designed for a maximum performance.
Q: What data are stored on the portal servers?
A: The portal is responsible for configuration data and meta data (means information, on which server are what offline data files stored). These information can be restored any time from scratch, and configuration can be backup-ed.
Hardware/Storage Q: Do I need a distributed IsarFlow setup? A: Depends on your CGN setup. Check the necessary traffic crossing the WAN for your setup!
Q: How many servers do I need?
A: Depends on CGN setup. In case of fully loaded CGSE modules a 1:1 relation CGSE:server is recommended due to failover peak load.
Q: Must the server be placed close to the CGN netflow source?
A: This is recommended at least in high volume installations. Long term storage may be centralized.
Q: Can I extend the IsarFlow CGN installation to include more servers, if more CGN sources go active / if more CGN traffic is observed A: This is possible anytime from a technical perspective.
Q: What type of hardware do I need for server?
A: Generally spoken, the hardware must be certified for the SuSE Linux SLES 11 operating system.
Q: What is recommended for the server?
A: The hardware recommendation is: 2x Intel Xeon X5675, 3.06GHz 32 GB Memory 8 hard disks RAID Controller, Raid 10 16X DVD-ROM Drive (or other options to mount an ISO image for installation) Redundant Power Supply Remote Management Card Portal: 300GB HDD Analyzer: 2TB HDD, but depends on sizing
Q: Do I need a DVD for installation?
A: No. You can use the virtual drive functionality of your server’s management card to mount the ISO image.
Q: Do I really need so many hard disks?
A: Hard disk space mostly depends on online data storage time and used NAT technology. For long-term retention time a storage system is used usually.
Q: How to connect storage to the server?
A: The recommended way is Fibre Channel or iSCSI, but no NFS. There is the option to use multipath connections to the storage. Please see the user guide how to integrate it.
Q: There are too much data to store, what can be done? A: The data are already stored in a compressed format, so think about your requirements. Maybe it’s an option to start with a smaller storage and increase when needed, as all storage numbers given are based on assumptions.
Q: How much rack space do I need?
A: Depends on your server/storage setup, verify this during project initialization.
Q: Do I have to install the operating system before the installation of IsarFlow CGN? A: No, the SuSE SLES 11 Linux operating system is included in the IsarFlow CGN installation routine.
Q: What information is needed during installation?
A: hostnames/IP addresses of IsarFlow server(s) NTP server for time synchronization, (server time zone is UTC) E-mail relay server (for critical status information) IsarFlow license file CGN timeout configuration of CGSE User(s)/group(s) to run IsarFlow
Q: Which communication must be allowed for an IsarFlow server (ACL setup)
A: Single server: HTTP/HTTPS, Netflow, SSH, DNS, NTP, SMTP, ODBC/Radius (for accounting data), LDAP/Radius/Tacacs+ (for external user authentication) Server Management card: usually HTTP, HTTPS, SSH Additionally for distributed servers: MySQL, SNMP
Q: Are there any special security recommendations?
A: We recommend to setup IsarFlow in a trusted network zone, secured by firewalls. Please verify with your companies security guidelines. Of course changing the default passwords is recommended.
During Operation Q: Why is there a sudden increase of memory consumption at a certain time? A: The database is configured to take a large amount of memory for efficient operation. It does not take the memory right after the start, but after a certain time of operation.
Q: Who is responsible for server operation?
A: The customer is responsible to monitor critical server parameter like server availability and storage utilization. It is recommended to include the server into customers monitoring solution and to monitor long-term growth of hard disk utilization (to verify assumptions of the sizing). IsarNet cannot be made responsible for data loss in cases like increased session rate, changed CGN configuration/parameter, full hard disk.
Q: What data should I backup?
A: We recommend to backup the configuration, the radius data (if valid in your system) and the measuring data, assuming these data are all important to be protected against e.g. HDD or administrator failures. A local IsarFlow configuration backup can be found in /var/IsarNet/IsarFlow/data/archive/configurations_*.tar.gz The radius and measurement data are stored as binary data file by the “data offline mechanism” into /var/IsarNet/IsarFlow/data/archive/offlineData/ (this is located on a storage system (connected to the portal) usually, but a storage system is not a data backup). So we recommend to backup /var/IsarNet/IsarFlow/data/archive/ regularly, even if they are very large.
