A User Key Management White Paper

The Challenge of Identity and Access Management in Secure Shell Environments

The Challenge of Identity and Access Management in Secure Shell Environments TABLE OF CONTENTS:
Introduction.......................................................................................................................... 1 Secure Shell in the Enterprise.............................................................................................. 1 Security Risks Related to Public Key Authentication............................................................ 2 Administrators who have left years ago that still have access to critical systems.......... 2 Other Unauthorized Copies of Private Keys..................................................................... 3 Unused User Keys Still Granting Access to Critical Hosts.................................................. 3 Lack of Key Rotation........................................................................................................ 3 Unintended Escalation of Access..................................................................................... 3 Lack of visibility of trust relationships crossing production and test boundaries....... 3 Lack of visibility of trust relationships crossing organizational boundaries................. 3 Inability to enforce and audit policy compliance............................................................. 4 The quantity of individuals who can create permanent trust relationships................... 4 Human errors in manual key setup and removal process................................................. 4 Best Practices for Reducing Risk........................................................................................... 5 Implementing Best Practices with Universal SSH Key Manager........................................... 6 Discovery.......................................................................................................................... 6 Remediation..................................................................................................................... 6 Management.................................................................................................................... 6 Summary............................................................................................................................... 9

About SSH Communications Security:

Founded in 1995, SSH Communications Security is the company that invented the SSH protocol - the gold standard protocol for data-in-transit security solutions. Today, over 3,000 customers across the globe, including 7 of the Fortune 10, trust our Information Assurance Platform to secure the path to their information assets. Our platform enables businesses of all types and sizes to protect their information assets by providing the gold standard data-in-transit security solutions that prevents data loss in both internal and external environments, hardened perimeter security through our multi-channel two-factor authentication and internal security control management solutions that enables organizations to more easily manage user keys and monitor administrator traffic across your networks.

2013 SSH Communications Security

The Challenge of Identity and Access Management in Secure Shell Environments

Introduction
SSH (Secure Shell) is a protocol and software suite for securely transmitting files and data between computers, point-to-point tunneling of sensitive data, and securely administering remote computers. Developed in 1995, by Tatu Ylnen, it is now an IETF (Internet Engineering Task Force) standard and is available in commercial and open source versions. Implementations of the SSH protocol are available for practically all platforms including Unix, Linux, Windows, IBM Mainframes as well as routers, embedded systems and smart phones. It is deployed on millions of servers and used ubiquitously across IT infrastructure. Almost every systems administrator working in a large IT environment relies on SSH to perform their day to day job functions. SSH typically is implemented with public key authentication because it is very well suited to supporting automated processes and efficient system administration. The public key system is initialized with the creation of a private and public key pair. The private key resides on the system requesting access to another system, and the corresponding public key is installed on all the systems granting access. The public keys that grant access to services are called SSH User Keys. As organizations deploy SSH across the IT infrastructure the process of key creation and distribution to target servers is repeated many, many times. The usual result is an extended, poorly managed access infrastructure built upon SSH user keys. Enterprises face numerous challenges in managing this infrastructure. These include manual, time consuming and error prone processes for creating new keys and trust-relationships; a lack of process, visibility and tools for key removal; the risk that private keys can be copied and misused; a lack of visibility as to which user accounts have access to what servers and services. In short, the accepted security standards for identity and access management (IAM) are generally lacking with respect to systems and application administration. This is exposing large organizations to heightened risk, compliance failures and cost. This white paper explains these IAM issues in more detail and describes the practical challenges enterprises face in addressing them. The paper concludes with a description of Universal SSH Key Manager, a comprehensive solution to this emerging problem.

Secure Shell in the Enterprise

The Secure Shell (SSH) protocol is the industry standard for administering large heterogeneous IT environments and for securing file transfers. SSH is used to transport the most crucial business data including credit card data, banking transactions, pension data, patient records, manufacturing specifications, and so on. Mission critical backups and business continuity processes are often secured by SSH. It is also used to authenticate administrators and application developers so they can securely maintain, manage and update the IT infrastructure: servers, applications and networks.

The Challenge of Identity and Access Management in Secure Shell Environments

A large global enterprise likely has up to a hundred thousand servers running the SSH protocol. These in turn run thousands of individual applications and transact thousands if not hundreds of thousands of automated data transfers very day. A typical server has multiple SSH User Keys installed. For example, there may be several different SSH User Keys granting root access to different server administrators each with authorization to update and maintain the operating system. Other User Keys may grant access to specific applications and yet other keys grant access to specific processes such as automated backups. In working with numerous enterprises, SSH Communications Security has found typical servers have anywhere from 8 to over 100 SSH User Keys installed. Multiplied by the thousands of servers running in a large enterprise the extent of the SSH user key infrastructure becomes staggering. At one global enterprise an audit conducted by SSH Communications Security found over 1.5 million SSH user keys, including over 150K keys that not only granted root access but also had no identified ownership. This is analogous to finding 150K unique username password accounts granting the highest level of privileged access, without knowing the identity of the individuals associated with those accounts. This scenario is typical of organizations that have a large IT infrastructure, whether they be commercial enterprises or in the public sector. The proliferation of SSH User Keys and lack of governance over the SSH infrastructure has developed over time. Few enterprises have a structured and automated approach to managing SSH User Keys. This has led to ballooning administrative costs as manual processes fail to keep pace with the infrastructure. There is increased risk as the lack of control over access to critical systems and processes exposes the organization to both insider and external threats. Finally, compliance mandates such as PCI-DSS, SOX and HIPAA are beginning to address the requirements for secure SSH User Key management. In short, the need to bring governance to SSH infrastructure is bubbling up for in the priority list of security initiatives in both the public and private sectors for reasons of cost, risk and compliance.

1. Security Risks Related to Public Key Authentication

1.1 Administrators who have left years ago that still have access to critical systems
When an administrator creates a key pair for an automated file transfer or other password-less login, they can easily take a copy of the private key file. When that systems or application administrator leaves the organization, there is no way of knowing whether the administrator has retained their own copy of the private key(s). This of course is a security risk. An unauthorized copy of a private key grants the same access to those user accounts and servers for which the original was configured. Even in the event that the administrators personal account closed and the original private key file destroyed, the unauthorized copy of the private key will continue to enable access as long as the corresponding public keys remain authorized on the servers. Most large enterprises never systematically remove public SSH User Keys. This is primarily due to the difficulty in doing so - most organizations do not have documentation or a database of all trust relationships and the purposes for which they were created. Lacking knowledge of whether the need for each authorization still exists or should be removed, the authorizations (i.e. public keys) are left in place for fear of breaking business critical processes.

The Challenge of Identity and Access Management in Secure Shell Environments

1.2 Other Unauthorized Copies of Private Keys

Any administrator who has access to a user account is technically able to make a copy of any private key stored in that account. Access is not restricted to the user/administrator that actually set up the public key authentication relationship. Any user who can log into the client account, or can access backups containing its files or can monitor the network through which such backups were ever made may in principle have made a copy of the private key. Such copies are just as effective for public key authentication as the original keys, as long as the public keys remain authorized on the servers.

1.3 Unused User Keys Still Granting Access to Critical Hosts

Having unused trust relations that remain enabled in the environment increases the risk of rogue administrators having copies of keys to those unused or no longer needed accounts. It also adds to the likelihood that access to one account will be propagated to other accounts. Trust relationships provided by public key authentication are transitive. Furthermore, they clutter the overall environment and make management and analysis of active relationships and more difficult, costly and error-prone.

1.4 Lack of Key Rotation

Renewing the key pairs used for public key authentication (i.e. changing the keys) is currently difficult and costly in practice. As a result, many of the private keys in the environment are years old and hundreds of administrators, contractors and consultants may have had the opportunity to make a copy.

1.5 Unintended Escalation of Access

Currently, most organizations do not have visibility into what machines their administrators can access, particularly when transitive authorizations are taken into account. Administrators are typically given access to a subset of the organizations production servers. Those same servers be authorized to initiate automated file transfers and other application-to-application connections to other production servers outside of the subset granted to the administrator. The administrator can leverage any account (to which he has access) to gain access to other accounts on other servers. Administrators with root privileges can escalate access from any account on the server. There are means available to prevent unintended transitive access but these are rarely used.

1.6 Lack of visibility of trust relationships crossing production and test boundaries
Many organizations have policies prohibiting most data transfers and application-to-application connections between production and development/test environments. These policies may be enforced by firewalls. However, firewalls do not have visibility to the user accounts used within encrypted sessions. This creates a potential hole in the enforcement mechanism. Once a connection between two systems across a firewall has been enabled, it is possible to add other public key authorizations between those systems enabling non-authorized activities. The firewall cannot enforce what operations are performed in such connections because it cannot see inside the encrypted connection.

1.7 Lack of visibility of trust relationships crossing organizational boundaries

Many enterprises outsource some or all of their IT to external service providers and consultants. Often the outsourcing provider administers the organizations servers and thus has access to its production network from its premises. Such administrative access is usually implemented using the SSH protocol. Many of these enterprises lack visibility as to whether trust relationships for passwordless authentication exist between the organizations internal network and the service provider.

The Challenge of Identity and Access Management in Secure Shell Environments

Unauthorized trust relationships to a service provider can expose the organization to rogue service providers personnel and even highly systematic data leaks. Such trust relationships might have been established years ago (possibly using a penetration attack). Unfortunately, in many cases the people who set up the initial trust relationships have since left the organization. Nonetheless, in many cases the unauthorized automated transfers are still operating.

1.8 Inability to enforce and audit policy compliance

Lack of visibility of SSH User Key infrastructure creates an enormous gap in policy control and verification (audit). For most organizations, it is currently not possible to enforce and audit the following: Private keys are regularly renewed No private key is ever reused (e.g. with a different passphrase instead of actually changing the key) No private key exists longer than a maximum permitted period Who has access to a particular data, account, host, or application when transitive public key authentication trust relationships are taken into account Who can actually create new application accounts and change their privileges (when access to administrative accounts is based on User Keys) Development engineers prohibited from accessing the production environment People no longer with the company are not able to log in to any of the computers Access privileges (authorization) is properly adjusted when a person changes roles within the organization No unauthorized data transfers can take place No unauthorized password-less trust relationships exist between an enterprise and its outsourcing partners

1.9 The quantity of individuals who can create permanent trust relationships
In a typical large organization trust relationships are set up manually by individual administrators. There is often no control over what trust relationships are set up and whether they are properly documented and approved. This makes it difficult to audit and remove unneeded authorizations because of lack of information as to what the authorizations are for. Large enterprises may have hundreds of administrators with an even greater number cycling through those positions over the years. With the increase of administrators churning through the organization and a lack of auditing and monitoring, the risks increase that User Key policies will be violated - by accident and by design. The more individuals authorized to set up trust relationships the greater the risk.

1.10 Human errors in manual key setup and removal process

A typical administrator use case involves the set up of a single private key that enables password free access to multiple systems. This enables the administrator to efficiently manage multiple systems without requiring separate manual login to each one. Other uses of password free authentication include automated processes such as file transfers, application runs and back-ups. Setting up a password-less trust relationship involves creating a key pair and then copying the generated public key to all server machines to which connection can be made. This can extend to thousands of systems. It also entails editing the SSH server configuration file on each server to which the key is

The Challenge of Identity and Access Management in Secure Shell Environments

authorized (adding the public key as an authorized key), editing the client configuration file to add the private key as an identity key, ensuring that privileges of public and private keys are properly set, and testing that the setup works This process can easily take from fifteen minutes to many hours, and becomes substantially more complicated if disaster recovery sites, replicated/fault-tolerant systems, or load sharing clusters are involved. Possible human errors during the key setup process include: Accidentally deleting other identity keys or authorized keys from the configuration file (possible impact: system outage for some unrelated application) Copying wrong public key (possible impact: someone gets unauthorized access) Copying to wrong host or account (possible impact: unintended access authorization, possibly to root account on a server if user name forgotten from copy destination) Forgetting to copy (and test) to some server (e.g., disaster recovery server) (possible impact: outage when the backup systems should go online)

2. Best Practices for Reducing Risk

Secure Shell can be deployed in a manner that reduces the risks highlighted above. Best practices include the following: 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. Standardize the key configuration across the environment Authorized key file should not allow end user write access Centralized key provisioning (no more self service provisioning). Key provisioning should be centralized and limited to a much smaller number of root level administrators Cipher configuration allow only strong ciphers and specified key lengths Require password protection for private keys Require logging of SSH activity Enable StrictModes ensure SSH server will not execute if authorized keys file and home directory is insecure Prevent privilege escalation by process spawning Segregate system accounts from person accounts Use controls to limit SSH access to specific commands and source addresses Rotate keys Remove unneeded User Keys Document key usage Regular audits.

Many of these best practices can be implemented by utilizing the configuration options and tools available within SSH itself. For example, it is technically possible to restrict the IP address from which a login using a particular public key is permitted as well as restrict the commands that may be executing using such pubic key login. However these are rarely used due to the complexity of setting them up manually and the difficulty in maintaining them in a dynamic IT environment.

The Challenge of Identity and Access Management in Secure Shell Environments

3. Implementing Best Practices with Universal SSH Key Manager

Perhaps the most significant barrier to implementing best practices is the challenge of making the needed changes without disrupting production processes. With thousands (sometimes millions) of outstanding keys and a lack of information as to what purpose each key fulfills, it is difficult to take even the fundamental step of removing unneeded keys. Organizations need tools to rationalize the current environment, implement a centralized process for key provisioning and ensure ongoing, auditable enforcement of policy. Universal SSH Key Manager is a centralized management solution that enables organizations to establish and maintain governance over SSH infrastructure. Universal SSH Key Manager (UKM) addresses the challenges through three distinct phases: Discovery, Remediation, Management.

3.1 Discovery
The first phase is to discover what private and public keys exist in the environment in their current state and to which users, service accounts or applications they relate to. The purpose of this phase is to gain visibility to the current environment and find who is able to access where and what is the overall status of the SSH user key environment. This phase establishes an inventory of private and public keys, trust relationships and other information such as key types, sizes, ciphers, SSH versions, etc. The next component of the discovery phase involves monitoring the environment. It is important to identify the keys that are actively used and those that are unused. UKM follows key activity to determine how often keys are used and where the keys are used from. This baseline information feeds into the next phase: Remediation

3.2 Remediation
Remediation comprises several components including: removing unused keys from the environment, relocation of keys from user home directories to root owned directories and policy enforcement. Policy in turn comprises several elements cipher, version and configuration control, authorization and separation of duties. For example, it may be desirable that a group such a SAP users only be able to access the SAP servers in the organization, or that a group of Unix administrators only be able to access those Unix servers they are assigned to manage.

3.3 Management
Once the organization of the environment is achieved then UKM manages the user key infrastructure in terms of automating private and public key deployments, renewals, and ensuring key removals when individuals, service accounts or application IDs are taken out of the AD or LDAP. The combined benefits include cost reduction from the elimination of manual processes related to key setups and removals, the mitigation of risk through accountability of what private and public keys may access which hosts, and compliance in terms of sound key management practices with full key rotation and removals. The UKM solution provides enforcement and auditing through monitoring of key usage and centralized control over SSH client and server configurations.

The Challenge of Identity and Access Management in Secure Shell Environments

Table 1 summarizes the challenges and corresponding solution provided by Universal SSH Key Manager.

Driver
1.1 Administrators who have left years ago that still have access to critical SSH Servers

Solution
Scan the managed environment, users and authentication keys, and discover and identify which user accounts are able to access which of the servers Integrate to existing directory sources and use up-to-date information to revoke trust-relationships that are no longer valid Identify multiple instances of the keys within the environment and enforce restrictions and access policies to restrict and lock down the private key use Enable automated private and public key renewal processes per defi ned policies Discover and report the user accounts trust relationships, who is able to access where using which of the user accounts

1.2 Unused User Keys Still Granting Access to Critical Hosts

1.3 Unauthorized Copies of Private Keys

1.4 Lack of Key Rotation 1.5 Lack of visibility who has access to what and breakdown of segregation 1.6 Lack of visibility of trust relationships crossing production boundaries 1.7 Lack of visibility of trust relationships crossing organizational boundaries 1.8 Inability to audit existing trust relationships 1.9 The quantity of individuals who can create permanent trust relationships

Enforce the creation of key setups and trustrelationships through Key Manager. All the manually created keys can be automatically noticed, revoked and informed Key Manager can automate the whole entire key creation and management process minimizing the manual work and change for mistakes

1.10 Human errors in manual key setup and removal process

Table 1: Universal Key Manager Risk Mitigation Summary

The Challenge of Identity and Access Management in Secure Shell Environments

The UKM Architecture permits enterprises to solve these risks and challenges in a minimally invasive manner by leveraging existing infrastructure and processes. Figure 1 illustrates a UKM deployment scenario. The key architectural components include: Within the discovery of the key environment, UKM has the ability to function in an agentless and thin agent mode causing minimal burden on network resources and decreasing the challenges related to deploying additional agents in the environment. UKM can also leverage the agent of Tectia Manager if this is already in use in the environment. Through a centralized database UKM works via multiple backend and front end configuration points enabling management of fragmented network environments in a controlled manner. Direct integration into Active Directory and LDAP allows SSH User Key Manager to make use of existing account structures to simplify the key deployment process and ensure the timely creation and remove of keys. APIs permit key manager to easily integrate to possible already existing approval processes within the enterprise and can function as the engine for key setup and removal with no change to that existing process.

Figure 1: Universal Key Manager Architecture

The Challenge of Identity and Access Management in Secure Shell Environments

3. Summary
Secure Shell is widely used within enterprise and public sector IT infrastructure. As an infrastructure level security protocol used primarily by automated processes and systems administrators SSH has been deployed without many of the management safeguards that govern general end user access. The result is elevated risk, excessive overhead costs and lack of compliance to security mandates. With the regular cycle of organizational changes large enterprises face, merger and acquisition activity, employee turnover and increasing virtualization, the lack of SSH key management is fast becoming a critical security risk. Alternatives such as Kerberos and X.509 certificates can address many of the governance challenges related to public key authentication but each comes with complexities and limitations. In particular, both approaches require significant, time consuming and often expensive changes to existing infrastructure and processes. Moreover, a transition to these technologies is disruptive to ongoing processes and such transitions entail operational risk and expense that few enterprises are willing to accept. Universal SSH Key Manager from SSH Communications Security provides a comprehensive, nondisruptive solution. The UKM approach of utilizing the existing SSH authentication infrastructure delivers a cost effective, methodical and less risky transition from an unmanaged to a fully managed SSH infrastructure. The benefits are risk mitigation, lower ongoing operational costs and compliance. However, the transition from unmanaged to managed SSH also requires personnel with the expertise to design, plan and implement a solution. Many organizations have a skills gap in this area. As inventors of the SSH protocol, SSH Communications Security are the industry experts. SSH Communications Security understands the challenges and brings the knowledge and experience to help organizations take control over their SSH infrastructure.

www.ssh.com