Reports

SEMINAR REPORT
ON
Sandnet: Network Trac Analysis of Malicious Software

Submitted for the Course of Third Year in Computer Engineering by Ms.Khemnar Suchitra Deoram[T80104233] Under the Guidance of Prof. M. S. Tamboli
Department of Computer Engineering

AMRUTVAHINI COLLEGE OF ENGINEERING
Sangamner-422608
UNIVERSITY OF PUNE
2012-2013
Sangamner-422608 Department of Computer Engineering
CERTIFICATE This is to certify that the SEMINAR report entitled

is submitted as partial fullment of the Seminar and Technical Presentation Third Year in Computer Engineering BY Ms.Khemnar Suchitra Deoram[T80104233]
Guide Prof. M. S. Tamboli
Seminar Coordinator Prof. S. A. Thanekar
Head of the Department Prof. R. L. Paikrao
Principal Dr. G. J. Vikhe Patil
UNIVERSITY OF PUNE
CERTIFICATE
This is to certify that, Ms.Khemnar Suchitra Deoram[T80104233] of Third Year in Computer Engineering was examined in the Seminar and Technical Presentation entitled

on 15 / 03 /2013 At
Department of Computer Engineering

Sangamner-422608
Internal Examinar
External Examiner
ACKNOWLEDGEMENTS
The satisfaction that accompanies the successful completion of any task would be incomplete without mentioning the people who made it possible. I am grateful to a number of individuals whose professional guidance along with encouragement have made it very pleasant endeavor to under take this seminar. I have a great pleasure in presenting the seminar Sandnet:Network Trac Analysis of Malicious Software under the guidance of Prof.M.S.Tamboli I am truly indebted and grateful to Head of Department Prof. R. L. Paikrao and Prof. S. A. Thanekar for their valuable guidance and encouragement. I take an opportunity to thank all the sta members of our department. Finally I express my sincere thanks to all those who helped me directly or indirectly in many ways in completion of this seminar work.
Place: Sangamner Date: March 9, 2013 Ms.Khemnar Suchitra
iii
ABSTRACT
Dynamic analysis of malware is widely used to obtain a better understanding of unknown software. While existing systems mainly focus on host-level activities of malware and limit the analysis period to a few minutes, we concentrate on the network behaviour of malware over longer periods. We provide a comprehensive overview of typical malware network behaviour by discussing the results that we obtained during the analysis of more than 100,000 malware samples.The resulting network behaviour was dissected in our new analysis environment called Sandnet that complements existing systems by focusing on network trac analysis. Our in-depth analysis of the two protocols that are most popular among malware authors, DNS and HTTP, helps to understand and characterize the usage of these prevalent protocols.
Place: Sangamner Date: March 9, 2013 Ms.Khemnar Suchitra
ABBREVIATIONS
Abbreviations
Description
AI API CDF CPU DLL DNS DoS FR HTTP IP IPv4 IRC ISO IT NAT OS OSI MX
Articial Intelligence Application Programming Interface Cumulative Distribution Function Central Processing Unit Dynamic Link Library Domain Name System Daniel of Service Foreign Resolver Hyper Text Transfer Protocol Internet Protocol Internet Protocol Version 4 Internet Relay Chat International Standard Organization Information Technology Network Address Transfer Operating System Open System Interconnection Mail Exchanger
ii
iii Abbreviations Description
PCR PE POST SDNS SMB SMTP SSL TCP TLD TLS TTL TXT VM UDP
Precongured Resolver Portable Executable Power On Self Test Secure Data Network System Server Message Block Simple Mail Transfer Protocol Secure Socket Layer Protocol Transmission Control Protocol Top Level Domain Transport Layer Security Transister Transister Logic Text File Virtual Machine User Datagram Protocol
NOTATIONS
Symbols
Description
SN et SHT T P
Subset of 45,651 samples exhibited some kind of network activity. The subset of samples using HTTP
iv
LIST OF FIGURES
4.1 4.2 5.1 6.1 6.2 6.3 6.4 6.5 6.6 6.7 6.8
Network protocols (by samples) . . . . . . . . . . . . . . . . . . . . . . . . . . Network protocols (by families/clusters). . . . . . . . . . . . . . . . . . . . . . Histogram of VirusTotal Labels per Sample. . . . . . . . . . . . . . . . . . . . Violin plot of DNS activity end distribution . . . . . . . . . . . . . . . . . . . CDF of DNS TTL per domainsn . . . . . . . . . . . . . . . . . . . . . . . . . CDF of DNS message error rate . . . . . . . . . . . . . . . . . . . . . . . . . . Resource Record distribution among samples . . . . . . . . . . . . . . . . . . Histogram of HTTP Request Distribution . . . . . . . . . . . . . . . . . . . . Distribution of HTTP error rates among samples . . . . . . . . . . . . . . . . Ratio of samples using given Content-Type . . . . . . . . . . . . . . . . . . . Distribution of # of PE binaries loaded . . . . . . . . . . . . . . . . . . . . .
16 17 22 26 26 27 28 30 33 33 35
LIST OF TABLES
4.1 4.2 5.1 5.2
Overview of observed behaviour . . . . . . . . . . . . . . . . . . . . . . . . . . Overview of network activities. . . . . . . . . . . . . . . . . . . . . . . . . . . Sandnet: Layer-7 protocol distribution compared with Anubis (% of SNet) . . Top 3 protocols over non-standard ports . . . . . . . . . . . . . . . . . . . . .
15 16 23 23
vi
CONTENTS
ABSTRACT ABBREVIATIONS NOTATIONS LIST OF FIGURES LIST OF TABLES CONTENTS 1 Introduction 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 Malware Analysis 2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 How Does Malware Analysis Help? . . . . . . . . . . . . . . . . . 2.2.1 The Need For Analysis . . . . . . . . . . . . . . . . . . . . 2.2.2 Time Has Changed (It s a Business, Not a Kiddie) . . . . 2.2.3 Signature Arms Race . . . . . . . . . . . . . . . . . . . . . 2.3 Where Does Malware Analysis Fit In? . . . . . . . . . . . . . . . 2.3.1 Infection is an Incident . . . . . . . . . . . . . . . . . . . 2.4 How Does Malware Today Work? . . . . . . . . . . . . . . . . . . 2.4.1 Droppers and Downloaders and Rootkits Oh My! . . . . . 2.4.2 How can you say you are clean if you can t trust the OS? 2.5 Playing With Fire (How To Analyse Malware) . . . . . . . . . . 2.5.1 Static Analysis . . . . . . . . . . . . . . . . . . . . . . . . 2.5.2 Runtime Analysis . . . . . . . . . . . . . . . . . . . . . . .
i ii iv v vi vii 1 1 3 3 3 3 4 5 6 6 7 7 7 8 8 8 10 10 11
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
. . . . . . . . . . . . .
3 Sandnet and System Overview 3.1 What is a Sandnet? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Network Activity Of Malware . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
viii 3.3 System Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 13 13 14 14 17 18 20 20 21 22 24 24 24 24 25 27 27 28 28 29 30 31 31 32 32 32 32 34 34 34 35 36 36 37 37 39
4 Current Malware Behaviours 4.1 Introduction . . . . . . . . . . . 4.2 Observed Malicious Behaviour . 4.3 File System Activity . . . . . . 4.4 Network Activity . . . . . . . . 4.5 Sandbox Detection . . . . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
5 System,Dataset and Networks Statistics 5.1 System Overview . . . . . . . . . . . . . 5.2 Dataset . . . . . . . . . . . . . . . . . . 5.3 Network Statistics . . . . . . . . . . . .
Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
6 DNS and HTTP 6.1 DNS . . . . . . . . . . . . . . . . . . . . . . 6.1.1 Introduction . . . . . . . . . . . . . 6.1.2 DNS Resolution . . . . . . . . . . . 6.1.3 DNS TTL Analysis . . . . . . . . . . 6.1.4 DNS Message Error Rate . . . . . . 6.1.5 Resource Record Type Distribution 6.1.6 Resolution for Other Protocols . . . 6.2 HTTP . . . . . . . . . . . . . . . . . . . . . 6.2.1 HTTP Requests . . . . . . . . . . . 6.2.2 HTTP Request Headers . . . . . . . 6.2.2.1 User-Agent . . . . . . . . . 6.2.2.2 Localization Headers . . . 6.2.3 HTTP Responses . . . . . . . . . . . 6.2.4 HTTP Response Headers . . . . . . 6.2.4.1 Content-Type . . . . . . . 6.2.4.2 Server . . . . . . . . . . . . 6.2.5 HTTP Responses with PE Binaries . 6.2.6 HTTP Servers . . . . . . . . . . . . 6.2.6.1 Ad Services . . . . . . . . . 6.2.6.2 Public Web APIs . . . . . 6.2.6.3 PE File Hosters . . . . . . 6.2.6.4 HTTP CC Servers . . . . . 7 Conclusion 7.1 Conclusion Bibliography
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
CHAPTER 1 Introduction
1.1 Introduction
Dynamic analysis, i.e. runtime monitoring, has proven to be a well-established and eective tool to understand the working of a software. Understanding the behaviour of malicious software may not only provide insights about actions of malicious intents, upcoming techniques, and underground economy trends, but it also gives the opportunity to develop novel countermeasures specically built on top of that understanding. Current analysis systems have specialized in monitoring system-level activities, e.g.accesses to the le system, but little eort has generally been devoted to understanding the network behaviour exposed by malware. In fact, similarly to system-level activities, network-level activities also show very distinct behaviours that can back up the insights provided by system-level analyses. Second, the very same network behaviours can uniquely provide further specic understanding necessary to develop novel approaches to collect, classify and eventually mitigate malicious software. Driven by this observation we focus our research on dissecting, analysing, and understanding the behaviour of malicious software as observed at the network level.[1] The observed malware behaviour highly depends on the duration of the dynamic analysis.Current systems try to analyse as many malware samples as possible in a given period of time. This results in very short analysis periods, usually lasting only a few minutes, which makes it dicult to observe malicious network behaviour that goes beyond the bootstrapping process. From a network behaviour point of view, however, the post-bootstrap behaviour is often more interesting than what happens in the rst few minutes. A thorough analysis is key to understanding the highly dynamic workings of malware, which is frequently observed to be modular and often undergoes behaviour updates in a pay-for-service model.Here we 1
CHAPTER 1. Introduction
present an in-depth analysis of malware network behaviour that we gathered with a new system called Sandnet during the last 12 months. Sandnet is an analysis environment for malware that complements existing systems by a highly detailed analysis of malicious network trac.[1] With Sandnet, we try to address two major limitations we see in publicly available dynamic analysis systems: a short analysis period and the lack of detailed network-behaviour analysis. While existing systems have usually spent only a couple of minutes to run a malware sample, we ran each sample for at least one hour. In addition, using the data collected through Sandnet, we provide a comprehensive overview of network activities of current malware. We rst present a general overview of network protocols used by malware, showing that DNS and notably HTTP are prevalent protocols used by the majority of malware samples. We will then provide an in-depth analysis of DNS and HTTP usage in malware. The results of our analysis can be used to spawn new research such as clustering malware based on network-level features or network-level malware detection. The main contributions of this work are: 1.We have in operation a new data collection and analysis environment called Sandnet that will be up for the long run and that we will continuously use to gather information on malware network behaviour. 2. We give an overview of the network activities of more than 100,000 malware samples and compare the results with data from previous eorts. 3. An in-depth analysis of DNS and HTTP trac provides details on typical protocol-specic usage behaviours of malware.[1]
CHAPTER 2 Malware Analysis

2.1 Introduction
In a typical organization, an attack from malicious software (known as malware) is not likely to go completely unnoticed. Detection of an attack may come through one or more technologies such as anti-virus software, intrusion detection systems, or it may come from systems compliance monitoring.Unfortunately, detection of the attack is no longer sucient to identify the full risk posed by malware. Often, detection occurs after the host has already been compromised. As malware evolves and grows increasingly complex, it is utilizing selfdefence mechanisms such as root kit technologies to hide processes from the kernel, disable anti-virus software, and block access to security vendor websites and operating system update information.Faced with these threats, once a host s integrity becomes compromised a crucial part of the incident response process is to determine what activity the malicious code is engaged in, and specically whether any data may have been compromised and to where it may have been sent.[3]
2.2
2.2.1
How Does Malware Analysis Help?

The Need For Analysis
The only way to really determine what a piece of malicious software is doing is to analyse it. The anti-virus industry has researchers who do this as a key part of their business. In 3
CHAPTER 2. Malware Analysis
the past this was sucient, because the motivating factor behind viruses was largely fame. Because of this, viruses were generally written by single individuals, and were designed to infect as many machines as possible. As a result, once a researcher was made aware of a threat, they could analyse it and create signatures which could be pushed out by the antivirus vendor, to protect everyone in the same fashion as they had been infected.Additionally, at this time malware was generally not very complex, in part because the authors didn t have the resources needed to create very complicated programs. This relative simplicity meant that an infected host could usually be cleaned with a high chance of success. While there were some truly devastating viruses, they were not very common.
2.2.2
Time Has Changed (It s a Business, Not a Kiddie)
Where malware was once being designed for fun, research, fame, or even to promote socio-economic activist ideals, today creating and managing malicious software has become a solid business model for criminals, and it is part of a robust underground economy.Because it s a business product, malware today has completely dierent goals than in the past. To meet these new goals, the complexity of the code, as well as the infection process, has increased substantially. Additionally, we are seeing the delivery pattern of malware change to meet the needs of the clients being served by the malware industry. For example, rather than infecting as many computers as possible, a piece of malware may be limited to perhaps a few selected computers within a specic organization. At the same time, another piece of malware may be served via a legitimate web site which has been compromised, to as many people as visit the site. These dierences in methods exist to meet the needs of clients with dierent goals.Because of the shift in the way malware is being developed and deployed, the methods used to mitigate the threat of malicious software in the past are no longer eective. Malcode which aects an organization may exist solely within that organization. It is not eective in most, nor codebase, for the anti-virus industry to manage a vast number of one o signatures, yet this is what is being done. Further, as the industry expands and resources are added which are needed to compete with the business of malware creation and distribution, problems arise as a result of the fact that there are no standards for malware management. For example, when an organization becomes aware of the fact that a host on their network has been compromised by a piece of malware, it is often necessary for them to learn more about what the malware does, and how to remove it from the infected host.This process is made very dicult and confusing when each vendor has diering information about the malware. Adding to the challenge is the fact that often the same piece of malicious software will have
CHAPTER 2. Malware Analysis multiple names, as each vendor picks their own way to uniquely identify it.[3]
2.2.3
Signature Arms Race
Anti-virus products work by creating a binary signature of a piece of malicious software. If a le on the system matches the signature, it is determined to contain that malicious software, and is dealt with according to the policies that have been set up either by the vendor, or the organization deploying the product. This works reasonably well when the quantity of unique malicious software is relatively small; however, this method does not scale well. As the samples grows, managing the signatures required to identify them becomes problematic. Further, if the number of unique samples increases at a signicant rate, the amount of time a particular piece of malcode is able to remain undetected increases as well, as the resources required to develop new signatures often do not increase to keep up with the inux. This leads to what is essentially a signature arms race, where the authors of malware take advantage of the time between their software being deployed, and the time it takes the anti-virus industry to analyse it and develop a signature pattern. In an eort to come out on top of this race, the industry has developed heuristic detection, which is used to categorize and group classes of malicious activity.While this does help, it is not sucient to catch all malicious activity. Further, to prevent heuristics from working eectively, malware is deploying in multiple stages. Since heuristics watches for specic types of activity being performed by an executable, the compromise has been broken down into several steps, making it possible for a machine to be at least partially compromised without the anti-virus product detecting it. Apart from heuristics, many anti-virus products have taken to identifying any software which is packed as being malicious. Since many legitimate software packages also use packers to decrease the size of their programs, this fosters complacency in the end user as the number of false positives increases.Further complicating this situation is the fact that since malware has become a business product, it now comes with a support model. Often included in this support is a guarantee that a given piece of malware will remain undetectable. Should an anti-virus product create a signature to detect the malware successfully, the author will alter the binary such that it no longer matches the signature for the life of the support plan. This is accomplished in a number of ways: creating routines which encrypt the code using strong cryptographic ciphers, and randomized keys completely altering the codebase itself in an automated fashion by using polymorphic routines packing and compressing the executables Each of the above methods alters the resulting binary in ways that make it dicult to analyse, let alone create a single signature pattern for it. As a result researchers are ooded with samples which may all be the same piece of
malcode, but because each one has dierent properties, they require dierent signatures. Online resources such as Virus Total assist malware authors in this process by allowing them to easily determine the detection rate of their malicious binary.For all of these reasons, there is a need for malware analysis to become part of an organization s standard security practice, and just not something that is relegated to highly skilled technicians employed by the anti-virus industry and researchers alone.
2.3
Where Does Malware Analysis Fit In?

If analysing malware is to be an essential component of an organization s security pos-
ture, it s important to understand how it relates to the process and policies already in place at that organization.[3]
2.3.1
Infection is an Incident
Because malware has been part of the computer security threat landscape for so long, and due to the media attention given to high prole attacks, viruses have become common. As a result,malware is often not seen for the serious risk it poses. The quirky names often given to viruses,(such as Slammer, Melissa, or of course I Love You), exacerbate this tendency to trivialize an infected host as a nuisance rather than a true security threat. Thus, despite the fact that the infection process and purpose of malware have signicantly changed, the response to infection and compromise has essentially remained the same: identify, create a signature, and clean. As a result, infection handling is generally left out of the incident response policies. This is a mistake. Malware is typically deployed in a multi-stage process, the end result of which is frequently complete control of the victim host by an attacker. This means that each alert from an anti-virus product could in fact be notifying you of the fact that youve now got a hostile host on your network. Worse, the attacker using the host is using whatever credentials are available, which means the theoretical malicious insider problem has just become real, only the insider isn t Bob from Accounting, it s a hostile foreign entity that now owns Bobs computer and is sending data from it to some other compromised host they control over an encrypted tunnel. Accordingly, how an organization deals with infected hosts has a number of implications. For example, if the infected host (or the end user the host belongs to) accesses sensitive information, there could be a number of legal and compliance problems that arise, including notication to customers that their data may have been compromised. In light of this, it makes sense that malware which is discovered on the organization s network needs to be
analyzed to minimally determine the following: Was the host successfully compromised? If it was, how was it ? What occurred after the compromise? Was any data taken? If data was taken, where was it sent? Were any other hosts compromised as well? These are all questions that an organization needs to be able to answer so they can determine how to form a proper response to the incident. Many of the answers can be obtained by analysing the malware. As such, malware analysis belongs in an organization s incident response policies and procedures.[3]
2.4
2.4.1
How Does Malware Today Work?

Droppers and Downloaders and Rootkits Oh My!
One of the more common methods being used to spread malicious software is compromised web sites. A link to the malware is placed on websites which are either owned by the attackers, or are compromised legitimate sites. These executables are generally not detectable by anti-virus software, and have no other purpose than to get loaded onto the victim and begin the second wave of attack, which generally consists of using HTTP to retrieve additional malicious software. The second stage malware handles things like disabling anti-virus software, manipulating host based rewall rule sets, installing root kits to hide malicious activity from the OS, and in some cases inserting code into the boot sector of the hard drive to allow it to remain in place even if the OS is cleaned. Its increasingly the case that more than one type of virus is utilized at this step in an eort to ensure successful compromise. If the anti-virus software triggers, it is usually at this point, however, by this time it is already too late, as the host has already been compromised successfully. While the anti-virus may have caught one of the new malware installation attempts,it is quite likely that there were others.[3]
2.4.2
How can you say you are clean if you can t trust the OS?
If the malware was successful in root kit installation, any investigative work being done at this stage is useless, as any information reported by the operating system kernel is suspect. Often security professionals will respond to an anti-virus alert that indicates a host was compromised by doing the following: Login to the host to investigate Viewing the processes running on the system Check open network connections If nothing unusual is found, often the decision is made that the host is clean, and the anti-virus software did its job.
The problem with this is that once a rootkit is installed, nothing the kernel tells you can be trusted. The author of the malware can use the rootkit technology to hide processes, registry entries, network connections, directories, etc. For this reason it is becoming the recommended practice that if a host becomes infected, it should be wiped and reinstalled from scratch. However, even this is not enough, as the use of boot sector rootkits is growing. To ensure the system is no longer infected, it is necessary to format the boot sector of the hard drive as well.[3]
2.5
Playing With Fire (How To Analyse Malware)

The rst method, known as static analysis, requires special skill sets, including an
extensive understanding of assembler (usually for the x86 chipset), software debugging techniques, and increasingly a solid understanding of encryption methods. Typically this level of skill means hiring a specialist, so reversing malware has generally been left to the anti-virus companies and various security research labs. As malware becomes increasingly advanced however, industries with high need for security (such as the aerospace or pharmaceutical industries for example) are beginning to employ in-house researchers with these skills.[3]
2.5.1
Static Analysis
The rst method, known as static analysis, requires special skill sets, including an extensive understanding of assembler (usually for the x86 chipset), software debugging techniques, and increasingly a solid understanding of encryption methods. Typically this level of skill means hiring a specialist, so reversing malware has generally been left to the anti-virus companies and various security research labs. As malware becomes increasingly advanced however, industries with high need for security (such as the aerospace or pharmaceutical industries for example) are beginning to employ in-house researchers with these skills.[3]
2.5.2
Runtime Analysis
The second technique, referred to as run time analysis, has a signicantly lower cost associated with it since it does not specically require a need for a specialist. This process involves gaining a solid understanding of what a piece of software does from simply observing
the system prior to,during, and after it has been successfully run. The experience needed to perform these tasks may already be available within the IT sta of an organization. It is for this type of analysis that a sandnet is used.[3]
CHAPTER 3 Sandnet and System Overview

3.1 What is a Sandnet?
Many organizations are familiar with the concept of a sandbox, or testing, host. Such systems are often used to isolate new code, server software, or even new operating systems, from the production environment or network. As its name implies, a sandnet expands this concept beyond a single host, to an entire network dedicated to testing and analysis. Specifically, a sandnet used to analyse malicious software provides a virtual Internet, within which all trac generated and any actions taken by the malware sample that is undergoing analysis can be logged and examined.The Institute for Internet Security developed Sandnet as a system to analyze the network behaviour of malware. Technically speaking, Sandnet is a conguration that we use to infect virtualised Windows hosts with recent samples of malicious software (malware). Sandnet is a controlled environment, meaning that we limit the potential damage of a malware, while guaranteeing that the infected system appears as normal as possible to the malware. Sandnet is used to generate statistics and shows trends on the network behaviour of malware. This website gives an overview of some important observations that we made analysing more than 100 malware samples in an analysis period of 12 months.[3]
10
CHAPTER 3. Sandnet and System Overview
11
3.2
Network Activity Of Malware

Our analysis has shown that most malware authors make use of DNS for resolving
domain names into IP addresses. While in the early days of botnets IP addresses tended to be hard-coded into malware binaries, nowadays DNS is used by more than 9 out of 10 malware samples as a reliable and exible way to contact servers.In addition, we have identied HTTP as a prevalent protocol among 59 % of all malware samples. While HTTP is far more prevalent, we also found out that 12 per of malware samples use the encrypted variant via SSL/TLS for communication. [4]
3.3
System Overview
In Sandnet, malware is analysed in execution environments known as sandpuppets con-
sisting of (virtualized) hardware and a software stack. Currently, we use VMs with Windows XP SP3 based on VirtualBox as sandpuppets. The machines are infected immediately after booting and gracefully shut down after a congurable time interval, which is typically one hour. Each sandpuppet is congured to have a local IPv4 address and a NATed Internet connection. A local DNS resolver is precongured. The sandherder is a Linux system hosting the sandpuppet virtual machines. Besides virtualization, the sandherder also records, controls and transparently proxies network trac to the Internet. We limit the potential damage of running malware samples by transparently redirecting certain trac (e.g. spam, infections) to local sinkholes or honeypots. In addition, we limit the number of concurrent connections as well as the network bandwidth and packet rate per sandpuppet. Internet connectivity parameters such as bandwidth and packet rate must be shared fairly among all sandpuppets in order to avoid inter-execution artifacts. The current Sandnet setup comprises ve bot sandherders with four sandpuppets each, resulting in twenty sandpuppets dedicated to malware analysis. Herders and sandpuppets can easily be added due to a exible and distributed design.After executing a malware binary, we dissect the recorded network trac for further analysis. A ow-extractor converts raw .pcap-les into UDP/TCP ows. A ow is a network stream identied by the usual 5-tuple (layer 4 protocol, source IP address, destination IP address, source port ,destination port). For TCP, a ow corresponds to a reassembled TCP connection. For UDP, a ow is considered to be a stream of packets terminated by an inactivity period of 5 minutes. Our experience shows that this time-out length is a reasonable mechanism to compensate the lack of UDP ow termination frames. Additionally, we use payload-based protocol detection in order to determine the application-level protocol of a ow. We dene a ow to be empty, if no UDP/TCP payload is transmitted in this ow.
CHAPTER 3. Sandnet and System Overview
12
Automated execution of malicious software raises some ethical concerns. Given unrestricted network connectivity,malware could potentially harm others on the Internet. Possible attack scenarios are, but not limited to, Denial-of-Service attacks, spam or infection of other hosts. We tried to nd the right balance between ethical concerns when designing Sandnet and restrict the Internet connectivity. Technically, we integrated certain honeywall techniques. The harm of DoS attacks is limited by network level rate-limiting, spam is transparently redirected to local mail servers and protocols known to be used for infection are redirected to local honeypots. Sandnet is closely monitored during execution.Admittedly, it is technically impossible to completely prevent all possible attacks. However, we are convinced that within the bounds of possibility we implemented a huge part of mitigation techniques and that the value of Sandnet strongly outweighs the reasonably limited attack potential. [1]
CHAPTER 4 Current Malware Behaviours

4.1 Introduction
Malicious software (or malware) is one of the most pressing and major security threats facing the Internet today. Anti-virus companies typically have to deal with tens of thousands of new malware samples every day. Because of the limitations of static analysis, dynamic analysis tools are typically used to analyze these samples, with the aim of understanding how they behave and how they launch attacks. This understanding is important to be able to develop eective malware countermeasures and mitigation techniques. sser This analysis and experiences are based on the malicious code samples that were collected by Anubis , the dynamic malware analysis platform. When it receives a sample, Anubis executes the binary and monitors the invocation of important system and Windows API calls, records the network trac, and tracks data ows. This provides a comprehensive view of malicious activity that is typically not possible when monitoring network trac alone. Anubis receives malware samples through a public web interface and a number of feeds from security organizations and anti-malware companies. These samples are collected by honeypots, web crawlers, spam traps, and by security analysts from infected machines. Thus, they represent a comprehensive and diverse mix of malware found in the wild. Our system has been live for a period of about two years. During this time, Anubis has analysed almost one million unique binaries. Given that processing each malware program is a time consuming task that can take up to several minutes, this amounts to more than twelve CPU years worth of analysis.When compiling statistics about the behaviours of malicious code, one has to consider that certain malware families make use of polymorphism. 13
CHAPTER 4. Current Malware Behaviours
14
Since samples are identied based on their MD5 le hashes, this means that any malware collection typically contains more samples of polymorphic malware programs than of non-polymorphic families. Unfortunately, this might skew the results so that the behaviour (or certain actions) of a single, polymorphic family can completely dominate the statistics. To compensate for this, we analyse behaviours not only based on individual samples in our database but also based on malware families (clusters).[4]
4.2
Observed Malicious Behaviour

In this section, we present detailed discussions on the le, registry, network, and botnet
activity .The goal is to provide insights into malicious behaviours that are common among a diverse range of malware programs. An overview of interesting activity is shown in Table 4.1. In this table, we show the fraction of samples that perform certain high-level activity. We also provide the behavioural information with respect to the number of malware families, approximated as clusters of samples that exhibit similar behaviours. It is interesting to observe that the dierences are often not very pronounced. One reason is that the clustering process was using a tight threshold. That is, samples are only grouped when they exhibit very similar activity, resulting in a large number of clusters. Another reason is that the activity in Table 4 is quite generic, and there is not much dierence at this level between individual samples and families. The situation changes when looking at activity at a level where individual resources (such as les, registry keys) are considered. For example, 4.49 % of all samples create the le C \ W IN DOW S \ system32 \ urdvxc.exe , but this is true for only 0.54 % of all clusters. This le is created by the wellknown, polymorphic allaple worm, and many of its instances are grouped in a few clusters.[4]
4.3
File System Activity

Looking at Table 4.1, we can see that, unsurprisingly,the execution of a large number of
malware samples (70.8% of all binaries) lead to changes on the le system.That is, new les are created and existing les are modied.When analysing the created les in more detail, we observe that they mostly belong to two main groups:One group contains executable les, typically because the malware copies or moves its binary to a known location (such as the Windows system folder). Often,this binary is a new polymorphic variant. In total,37.2% of all samples create at least one executable le. Interestingly, however, only 23.2% of all samples (or 62% of those that drop an executable) choose the Windows directory or one of
15
Table 4.1: Overview of observed behaviour
Observed Behaviour Installationof aW indowskerneldriver : Installationof aW indowsservice : M odif yingthehostsf ile : Creatingaf ile : Deletingaf ile : M odif yingaf ile : Installationof anIEBHO :: Installationof anIET oolbar : DisplayaGU Iwindow : N etworkT raf f ic : W ritingtostderr : W ritingtostdout : M odif yingaregistryvalue : Creatingaregistrykey : Creatingaprocess :
Percentage of Samples 3.34 12.12 1.97 70.78 42.57 79.87 1.72 0.07 33.26 55.18 0.76 1.09 1.04 64./71 52.19
Percentage of Clusters 4.24 7.96 2.47 69.90 43.43 75.62 1.75 0.18 42.54 45.12 0.78 1.04 69.92 52.25 50.64
its sub-folders as the target.A large fraction 15.1% create the executable in the user s folder (under Document and Settings). This is interesting, and might indicate that, increasingly,malware is developed to run successfully with the permissions of a normal user (and hence, cannot modify the system folder).The second group of les contains non-executables,and 63.8% of all samples are responsible for creating at least one. This group contains a diverse mix of temporary data les, necessary libraries (DLLs), and batch scripts. Most of the les are either in the Windows directory (53% of all samples) or in the user folder (61.3%1). One aspect that stands out is the signicant amount of temporary Internet les created by Internet Explorer (in fact, the execution of 21.3% of the samples resulted in at least one of such les).These les are created when Internet Explorer (or, more precisely, functions exported by iertutil.dll)are used to download content from the Internet. This is frequently used by malware to load additional components. Most of the DLLs are dropped into the Windows system folder.The modications to existing les are less interesting.An overwhelming majority of this activity is due to Windows recording events in the system audit le system32 \ conf ig \ SysEvent.Evt.In a small number of cases, the malware programs infect utilities in the system folder or wellknown programs (such as Internet Explorer or the Windows media player).In the next step, we examined the deleted les in more detail. We found that most delete operations target (temporary) les that the malware code has created previously. Hence, we explicitly checked for delete operations that target log les and Windows event audit les.[4] Interestingly, Windows malware does not typically attempt to clear any records of its
16
activity from log data (maybe assuming that users will not check these logs). More precisely, we nd that 0.26% of the samples delete a le, and only 0.0018% target *evt les.We also checked for specic les or le types that malware programs might look for on an infected machine.To this end, we analysed the le parameter to the NtQueryDirectoryFile system call, which allows a user (or program) to specify le masks. We found a number of interesting patterns. For example,a few hundred samples queried for les with the ending .pbk. These les store the dial-up phone books and are typically accessed by dialers. Another group of samples checked for les ending with .pbx, which are Outlook Express message folder.
Table 4.2: Overview of network activities.
Observed Behaviour Listenonaport : T CP traf f ic : U DP traf f ic : DN Srequests : ICM P traf f ic : HT T P traf f ic : IRC traf f ic : SM T P traf f ic : SSL : Addressscan : P ortscan :
Percentage of Samples 1.88 45.74 27.34 24.53 7.58 20.75 1.72 0.82 0.23 19.08 0.01
Percentage of Clusters 4.39 41.84 25.40 28.42 8.19 16.28 4.37 1.57 0.18 16.32 0.15
Figure 4.1: Network protocols (by samples)
17
Figure 4.2: Network protocols (by families/clusters).
4.4
Network Activity
Table 4.2 provides an overview of the network activities that we observed during analy-
sis. Figure 4.1 depicts the use of the HTTP, IRC, and SMTP protocols by individual samples over a one and a half year period. In contrast, Figure 5 shows the usage of the HTTP, IRC, and SMTP protocols once families of malware samples are clustered together . These two graphs clearly demonstrate the usefulness of clustering in certain cases. That is, when the rst graph is observed, one would tend to think that there is an increase in the number of samples that use the HTTP protocol. However, after the samples are clustered, one realizes that the use of the HTTP protocol remains more or less constant. Hence, the belief that there is an increase in HTTP usage is not justied,and is probably caused by an increase in the number of polymorphic samples. However, the graph in Figure 4.2 supports the assumption that IRC is becoming less popular.[4] Moreover, we observed that 0.23% of the samples used SSL to protect the communication. Almost all use of SSL was associated to HTTPS connections. However, 8 samples adopted SSL to encrypt trac targeting the non-standard SSL port (443). Interestingly, most of the time the client attempted to initiate an SSL connection, it could not nish the handshake.In the samples that we analysed, only half of the samples (47.3%) that show some network activity also query the DNS server to resolve a domain name.These queries were successful most of the time. However,in 9.2% of the cases, no result was returned.Also, 19% of the samples that we observed engaged in scanning activity. These scans were mostly initiated by worms that scan specic Windows ports (e.g.,139, 445) or ports related to back doors (e.g., 9988 Trojan Dropper Agent). Finally, 8.9% of the samples connected to a remote site to download another executable. The le sizes of these second stage malware programs, compared with the size of the executable samples submitted to Anubis is more than second stage malware. As one may
18
expect, the second stage executables are in average smaller than the rst stage malware.Note that over 70% of the samples that downloaded an executable actually downloaded more than one.In fact, we observed one sample that downloaded the same le 178 times during the analysis time of four minutes (i.e., the download was corrupted with each download, so the sample immediately attempted another download).
4.5
Sandbox Detection
Another interesting aspect of malware behaviour is its capability to detect the presence
of an analysis environment such as Anubis. Dynamic analysis systems are a popular means to gather data about malicious code, and it is not surprising that malware is using techniques to thwart such analysis. When a malware program detects a sandbox, it typically alters its behaviour -most of the time, it just quits. In this section, we attempt to estimate the number of samples that use (general and Anubis specic) anti-sandbox techniques. Sandbox detection techniques fall into two main classes: One class is comprised of instruction-level detection techniques, which are techniques that determine the dierence between a real CPU and an emulated one by only making use of CPU instructions.The second class are API-level detection methods,which query the environment by calling one or several (Windows) API functions. Unfortunately, we have currently no good way to detect instruction-level detection attempts. While it is theoretically possible to monitor single instructions in a full system emulator, for performance reasons,Anubis essentially works at the API level. Since Anubis leverages Qemu for its full system emulation, it is susceptible to the same detection methods as Qemu is. In particular, several of the detection strategies (all of them are instruction level detections) outlined in and also work for Anubis.Since we do monitor Windows API calls, we are in a better position to check for possible API-level detection methods. At the time of writing this paper,several Anubis-specic detections have been published on the Internet.All of them work by comparing the return value of a Windows API function such as GetComputerName to a a hard-coded value that is known to identify Anubis. Our system includes a data tainting component that allows us to detect when the output of an interesting function is subsequently used as an operand in a comparison operation. Thus, we were able to analyse our sample set for specic comparisons that represent known Anubis checks published in forum postings or blogs. A couple of checks deserve more discussion. Our system names each analyzed sample sample.exe. Thus, a wellknown Anubis detection routine compares the return value of the Windows API function GetCommandLine with that string. Similarly, in the past, every sample was executed as a Windows user with the name user. Consequently, it was possible to compare the result of NtQuerySystemInformation
19
with the string user to detect Anubis (as a side note, we have since changed this username to ).In total, we have found that only 0.03% of the samples (99 distinct malware programs) contain a known Anubis check. Most of these samples perform the check in the beginning of their execution, and immediately exit if they detect the presence of our sandbox.Of course, it is possible that malware samples perform a wide range of other checks. Moreover, as mentioned previously, we have no good way to identify instruction-level detections or timing-based detection attempts. Thus, the reported numbers are a clear lower bound. Under the assumption that a sample that detects Anubis (or a sandbox) does not perform much activity,we can also provide an upper bound for the samples that do sandbox detection. Based on our experience with Anubis, we consider a behavioural report to contain notmuchactivity when it contains less than 150 features. For comparison, the average prole has 1,465 features. Using this denition,we found that 12.45 % of the executable samples (13.57 % of the clusters) show not much activity suciently. Anubis only has a very limited user input simulation, which simply closes opened windows. Moreover, some programs require non-existing components at runtime (note, though, that programs that fail because of unsatised, static DLL dependencies are not included in the 12.45 %). In addition, at least 0.51% of the reports with not much activity can be attributed to samples that are protected with a packer that is known to be not correctly emulated in Qemu (such as Telock and specic packer versions of Armadillo and PE Compact). Last but not least, bugs in Anubis and Qemu are also a possible cause.[4]
CHAPTER 5 System,Dataset and Networks Statistics Overview

5.1 System Overview
In Sandnet, malware is analysed in execution environments known as sandpuppets consisting of (virtualized) hardware and a software stack. Currently, we use VMs with Windows XP SP3 based on VirtualBox as sandpuppets. The machines are infected immediately after booting and gracefully shut down after a congurable time interval, which is typically one hour. Each sandpuppet is congured to have a local IPv4 address and a NATed Internet connection. A local DNS resolver is precongured.The sandherder is a Linux system hosting the sandpuppet virtual machines. Besides virtualization, the sandherder also records, controls and transparently proxies network trac to the Internet. We limit the potential damage of running malware samples by transparently redirecting certain trac (e.g. spam, infections) to local sinkholes or honeypots. In addition, we limit the number of concurrent connections as well as the network bandwidth and packet rate per sandpuppet to mitigate DoS activities. Internet connectivity parameters such as bandwidth and packet rate must be shared fairly among all sandpuppets in order to avoid inter-execution artifacts. The current Sandnet setup comprises ve bot sandherders with four sandpuppets each, resulting in twenty sandpuppets dedicated to malware analysis. Herders and sandpuppets can easily be added due to a exible and distributed design.After executing a malware binary, we dissect the recorded network trac for further analysis. A ow-extractor converts raw .pcap-les into UDP/TCP ows. A ow is a network 20
CHAPTER 5. SYSTEM,DATASET AND NETWORK STATISTICS OVERVIEW
21
stream identied by the usual 5-tuple (layer 4 protocol, source IP addr., destination IP address,source port,destination port). For TCP, a ow corresponds to a reassembled TCP connection. For UDP, a ow is considered to be a stream of packets terminated by an inactivity period of 5 minutes. Our experience shows that this timeout length is a reasonable mechanism to compensate the lack of UDP ow termination frames. Additionally, we use payload-based protocol detection in order to determine the application-level protocol of a ow. We dene a ow to be empty, if no UDP/TCP payload is transmitted in this ow.Automated execution of malicious software raises some ethical concerns. Given unrestricted network connectivity,malware could potentially harm others on the Internet. Possible attack scenarios are, but not limited to, Denial-of-Service attacks, spam or infection of other hosts. We tried to nd the right balance between ethical concerns when designing Sandnet and restrict the Internet connectivity. Technically, we integrated certain honeywall techniques. The harm of DoS attacks is limited by network level rate-limiting, spam is transparently redirected to local mail servers and protocols known to be used for infection are redirected to local honeypots. Sandnet is closely monitored during execution.Admittedly, it is technically impossible to completely prevent all possible attacks. However, we are convinced that within the bounds of possibility we implemented a huge part of mitigation techniques and that the value of Sandnet strongly outweighs the reasonably limited attack potential.[1]
5.2
Dataset
In order to study malicious network trac, we analysed malware samples that were
provided to a great degree by partner research institutions. For each sample we acquire A/V scan results from VirusTotal. 85% of the samples that we executed had at least one scan result indicating malware (see Figure 5.1). In order to avoid accidental benign samples we collated our set of samples with a list of known software applications using Shadowserver s bintest .We randomly chose the samples from a broad distribution of all malware families. We tried to mitigate side-eects of polymorphism by extracting the family name of a given malware samples A/V labels and limit the number of analyses per malware family. For our analysis we dened the following set of samples. We analysed a total of 104,345 distinct samples over a timespan of one year. Samples were executed with regard to their age. On average, the samples were executed about 7.8 days after submission. We gradually increased the time between sample acquisition and execution from 6 hours to 150 days in order to evaluate whether the execution age signicantly in uences malware activity.The total analysis time of all samples in this data set sums up to
CHAPTER 5. SYSTEM,DATASET AND NETWORK STATISTICS OVERVIEW an analysis period of 12 years.[1]
22
Figure 5.1: Histogram of VirusTotal Labels per Sample.
5.3
Network Statistics
Of the 104,345 samples, the subset SNet of 45,651 (43.8%) samples exhibited some kind
of network activity. The network trac caused by these samples sums up to more than 70 million ows and a volume of 207 GB. It remains an open issue to understand why a majority of the samples did not show any network activity. We suspect that most of the inactive samples a) are invalid PE les b) operate on a local system only (e.g. disk encryption) c) are active only if there is user activity or d) detected that they are being analysed and stopped working. Protocol inspection reveals that a typical sample in SNet uses DNS (92.3%) and HTTP (58.6%). IRC is still quite popular: 8% of the samples exposed IRC. Interestingly, SMTP only occurred in 3.8% of the samples in SNet. As DNS and HTTP are by far the most widely used protocols in Sandnet trac, we will inspect these in more detail in Table 1. Table 1 also compares our protocol statistics with data based on Anubis provided by Bayer et al. in 2009. Interestingly, when comparing the results, the samples we analysed showed increased usage of all protocols. However, the ranking and the proportion of the protocols remain similar.We suspect this increase is a)due to a relatively long execution of malware samples and b)caused by a growing usage of dierent application-level protocols by malware. 30.1% of the ows were empty (no payload was transmitted). All these ows are presumable scanning attempts.Already 90% of the empty ows targeted NetBIOS/SMB services. The remaining empty ows are normally distributed over lots of dierent ports.Of the remaining ows with payload (69.9%), for 22.8% no well-known protocol could be determined. Over 60% of these ows account for NetBIOS or SMB-related communication (mostly scanning) according to the destination port. Again, the remaining ows with failed protocol detection are normally distributed across
CHAPTER 5. SYSTEM,DATASET AND NETWORK STATISTICS OVERVIEW
23
many destination ports.Payload-based protocol detection is a big advantage if protocols are used over other than their well-known ports. We found that 12.8% of SNet use protocols over other than the well-known ports. We speculate that in these cases malware tries to communicate via ports opened in the rewall,independent from the actual communication protocol. For instance, we regularly found IRC bots connecting to IRC servers listening on TCP port 80. Thus, non-standard port usage might serve as a malware classication or detection feature. The top 3 aected protocols are listed in Table 5.2.As additional analysis, we found out that a longer analysis period is indeed helpful for a better understanding of malware behavior. To judge on this, we performed three measurements each after an analysis period of 5 minutes and after 1 hour. First, we found out that only 23.6% of the communication endpoints that we have seen samples connecting to were contacted in the rst 5 minutes of analysis.We then calculated that only a minor fraction (6.1%) of all ows started within the rst 5 minutes. Lastly, we found that 4.8% of the samples started using a new protocol after 5 minutes that they have not used in the rst minutes.[1]
Table 5.1: Sandnet: Layer-7 protocol distribution compared with Anubis (% of SNet)
Protocol DN S HT T P IRC SM T P
Reference 44.5 37.6 2.3 1.6
Sandnet 92.3 58.6 8.0 3.8
Table 5.2: Top 3 protocols over non-standard ports
Protocol HT T P IRC F lash
SNet Samples (%) 8.17 7.13 0.19
Distinct Ports 303 174 9
CHAPTER 6 DNS and HTTP

6.1
6.1.1
DNS
Introduction
DNS is by far the most prevalent layer-7 protocol in Sandnet network trac and gives an interesting insight into malware activity. The subset of samples using DNS is denoted by SDNS.
6.1.2
DNS Resolution
Although all sandpuppets have their Windows stub resolver point to a working DNS resolver, we observed malware that used a dierent resolver or even carried its own iterative resolver. We developed the following heuristic in order to detect executions that carry an iterative resolver.An execution is considered as carrying an iterative resolver if there is an incoming DNS response from a server other than the precongured DNS resolver with a referral concerning a TLD and the Recursion Available ag set to 0. We cross checked the resulting executions whether at least one of the DNS root servers had been contacted via DNS. We can only speculate on the reasons why the precongured local DNS resolver is avoided. Using ones own resolver clearly has advantages. Resolution of certain domains might be blocked at the precongured resolvers in some environments (e.g. corporate ones).Additionally, using ones own resolver avoids leaving traces in logs or caches of the 24
CHAPTER 6. DNS and HTTP
25
precongured resolver. If the Windows stub resolver is congured to use ones own resolver, local queries can be modied at will. This could be used for phishing attacks (redirect to a proxy) or to prevent A/V software from updating. Furthrmore, precongured resolvers might be rate-limited.We found that 99% of the samples in SDNS use the precongured resolver. Given this high ratio, a DNS resolver indeed turns out to be an interesting source for network based malware detection - much more suitable than we had expected beforehand. We leave it up to future work to look into malware detection methods based on DNS resolver logs.3% of SDNS perform recursive DNS resolution with other resolvers than the precongured one (termed foreign resolvers in the following). Only 2% of SDNS expose iterative DNS resolution. Note that the sets are not disjunct, as an execution may exhibit multiple resolution methods or resolvers.We speculate that this is due to the fact that malware occasionally downloads and executes multiple binaries, each of which might have dierent resolution methods. The foreign resolvers used include Googles Public DNS (used by 0.38%) as well as OpenDNS (0.25%). However, there is a large number of foreign resolvers that are used less frequently. One resolver that was located in China got our attention because queries for well-known popular domains such as facebook.com and youtube.com resolved into arbitrary IP addresses with no recognizable relation to the domain. We consider this to be an artifact of the so-called Great Firewall of China. In total 932 of 5092 (18.3%)distinct DNS servers were used recursively at least once and thus can be regarded as publicly available recursive DNS resolvers.Furthermore, we looked into the activity distribution of the dierent resolution methods (see Figure 6.2). The precongured resolver (PCR) was typically used throughout the whole analysis period. The end of the usage of foreign resolvers (FR) is wide-spread over time, leaning toward the end of the analysis. Interestingly, iterative resolution appears to end much sooner compared to the other resolution methods.[1]
Figure 6.1: Violin plot of DNS activity end distribution
26
6.1.3
DNS TTL Analysis
The Time To Live parameter was of special interest to us, as it could be an indicator of fast ux usage. Fast ux is used as a means to provide exibility among the CC infrastructure of bots.Figure 6.2 shows that 10% of all domains have a maximum TTL of 5 minutes or below. As spotted elsewhere, we expected domains with a small TTL and a large set of distinct answer records to be fast ux candidates. However,when inspected manually, we found many domains of content distribution networks and large web sites. Using small TTLs seems to have become common among web hosters.As a result, the distinction between malicious fast ux networks and legitimate hosting services becomes much more dicult. Interestingly, we also found a couple of responses with a TTL of zero that looked themselves like CC communication. These responses were characterized by very long domain names as hex-strings. The TTL of zero prevents caching of these responses, eectively causing the resolver to always fetch the newest response from the authoritative DNS server. All in all, DNS suits well as a low-prole, low-bandwidth CC channel in heavily rewalled environments,e.g. for targeted attacks.[1]
Figure 6.2: CDF of DNS TTL per domainsn
6.1.4
DNS Message Error Rate
In order to measure DNS transaction failure, we dened the DNS request error rate as the number of DNS requests that were not successfully resolved over the total number of DNS requests. When aggregating the DNS message error rate per sample, we realized that for 10.1% of the samples in SNet all of their DNS resolution attempts fail. However,the majority of the samples in SNet (60.3%) have all DNS queries successfully resolved. The complete CDF is provided in Figure 6.3.
27
Figure 6.3: CDF of DNS message error rate
6.1.5
Resource Record Type Distribution
Figure 6.4 shows the distribution of the Resource Record types of the query section.Obviously, A records dominate DNS queries in Sandnet trac, followed by queries for MX records. All samples in SDNS have queried for an A record at least once. The high prevalence of A records is expected as A records are used to translate domain names into IP addresses. Furthermore, 2.3% of the samples in SDNS queried blacklists. MX records have been queried by far less samples (8%). Interestingly, when comparing the MX query rate with SMTP activity, we have seen both: samples that performed MX lookups but had no SMTP activity and samples that successfully used SMTP but showed no MX queries at all. We assume that is the latter case, the required information on the MX destinations is provided via other means,e.g. CC.[1]
Figure 6.4: Resource Record distribution among samples
6.1.6
Resolution for Other Protocols
DNS, though itself a layer-7 protocol, plays a special role as it provides resolution service to all other layer-7 protocols.We analysed how malware uses DNS before connecting to certain destinations. 23% of the samples in SDNS show at least one without prior DNS resolution
28
of the destination (DNS ows and scans excluded). In such a case either the destinations IP address is known (e.g. hard-coded in the binary) or resolution takes place via some other mechanism than DNS. A table providing ow destination DNS resolution by protocol can be found in annex I. Furthermore, 2.3% of the samples in SDNS queried blacklists (26% of these also sent spam).
6.2
HTTP
HTTP trac sums up to 88 GB inbound and 21 GB outbound, which makes HTTP by
far the most prevalent protocol in Sandnet measured by trac. The subset of samples using HTTP is denoted by SHTTP. Given the high detail of the Open DPI protocol classication, additional protocols that are carried in HTTP trac are treated separately and thus contribute additional trac: The Macromedia Flash protocol sums up to an additional 32 GB, video streams like MPEG and Apple Quicktime sum up to an additional 9 GB.We observed that the protocols carried in HTTP are usually caused by embedded objects included in websites that are visited by samples. The immense potential abuse of HTTP-driven services motivated us to perform an in-depth analysis of typical malware HTTP trac. Not only botnets started using HTTP as CC structures. To name but a few, click fraud (i.e. the abuse of advertising services), mail address harvesting, drive by downloads and DoS attacks on web servers are malicious activities of a wide range of malware authors. Of all samples with network activity (SNet), the majority of 58.6% exposed HTTP activity. This section provides details to which extent, how, and why malware typically utilizes the HTTP protocol.
6.2.1
HTTP Requests
The analysed samples typically act as HTTP clients and contact HTTP servers, mainly because the Sandnet communication is behind a NAT rewall. As a consequence, we can assume that virtually all recorded HTTP requests were made by malware. Figure 6.5 gives a general overview of how many HTTP requests malware typically made during the analysis period. The number of requests gives us a lead for which role malware has. Whereas one would expect a tremendous amount of requests during click fraud campaigns or DoS activities, malware update functionality and CC channels potentially need little HTTP activity only. Interestingly, only 65% of the samples in SHTTP made more than 5 HTTP requests. 16.3% of the samples in SHTTP made only one HTTP request and then stopped their HTTP activity, although 70% of these samples continued with other network activity. We manually
29
checked a fraction of these cases and found that many samples use HTTP to load secondstage binaries and continue with non-HTTP based damage functionality. The samples that completely ceased communicating after their initial HTTP ow presumably either failed to update themselves or waited for user-input triggers the GET request method was used by 89.5% of the samples in SHTTP. We observed that 72% of the samples in SHTTP additionally included GET parameters. Analysing just the fraction of GET requests with parameters, GET requests have on average 4.3 GET parameters. The average size of GET parameter were 12 characters for the key and 33.3 characters for the value. Although other means (such as steganography) allow to pass data to the sever, GET parameters seem to remain a popular method. On average,we have observed 1966 GET requests per sample with at least one request parameter. Interestingly, the number of unique GET parameter keys used by a sample is signicantly lower than the total number of GET parameters per sample. This trend is particularly strong for samples with many parametrized GET requests and indicates that parameter keys are reused for follow-up requests. On average, the ratio between the number of distinct GET parameter keys and the total number of GET parameters is merely 1:16. We plan to further analyse the vast use of GET parameters, as started in , in the future. The POST request method was used by 56.3% of the samples in SHTTP. The average body size of POST requests is 739 bytes. We manually inspected a randomly chosen fraction of POST bodies to nd out for what purpose malware uses POST requests. A large fraction of the inspected POST requests was used within CC communication with a botnet server. We commonly observed that data passed to the server was base64-encoded and usually additionally encrypted. In addition, we frequently saw POST requests directed to search engines.42% of the samples in SHTTP used both POST and GET requests. Only 0.9% of the samples in SHTTP showed HEAD requests at all. All other HTTP methods were used by less than 0.1% of the samples in SHTTP and seem insignicant.[1]
Figure 6.5: Histogram of HTTP Request Distribution
30
6.2.2
HTTP Request Headers
The HTTP headers include common headers usually used by benign web browsers. In total, we have observed 144 unique HTTP request headers. At a closer look at these, we identied a signicant amount of misspelled or non-standard headers (excluding all extension headers, i.e. those starting with X- ). Manual inspection shows that the fewer a specic header is used (in terms of samples), the more suspicious it is. Merely 5.7% of all samples in SHTTP sent an HTTP request without any header at all. As a consequence, we see a need to further analyse specic request headers that we consider interesting.
6.2.2.1
User-Agent
In an ideal world, the HTTP User-Agent header species which exact web browser (including its version number)is requesting web content. However, the client and thus also malware samples can potentially forge the User-Agent header to be less suspicious. Annex B gives a detailed list of the 30 most popular raw User-Agent strings observed in Sandnet. Most samples (98.6% of SHTTP) specied a User Agent header at least once.In an approach to get an overview of actual user agents we developed heuristics to lter the User-Agent list. First,we observed that 29.9% of the samples in SHTTP specied wrong operating systems or Windows versions in their forged HTTP User-Agent headers. Next, we identied that at least 13.4% of the samples in SHTTP claim to use nonexisting browser versions (e.g. wget 3.0, Mozilla 6.0 ). In addition,we saw that 37.8% of the samples in SHTTP specied malformed or very short and to us unknown User-Agent values.In total, 67.5% of the samples in SHTTP transmitted at least once a suspicious UserAgent string. Over the whole analysis period, only 31% of the samples in SHTTP specied apparently correct User-Agent strings.This result suggests that most samples have their own HTTP components that are bad in forging real web browsers. Interestingly, about half (50.6%) of the samples in SHTTP change or alternate the User-Agent header during their analysis period. We hypothesize that this is due to the modular architecture of malware, where the modules have inconsistent User-Agent strings. Furthermore, based on this observation, we suspect that malware adapts the User-Agent header (and possibly other headers) depending on the target website.
6.2.2.2
Localization Headers
HTTP requests typically include header that tell the server which languages and character sets the client understands (Accept-Language and Accept-Charset). We inspected these
31
two localization headers and compared it with the locale setting (German) of the sandpuppets. While the Accept-Charset header was used by 0.35% of the samples in SHTTP, the Accept-Language values are more interesting to analyse: In total, 44.3% of the samples in SHTTP included Accept-Language as an HTTP request header. Of these samples,24.1% did not respect the locale setting and specied a nonGerman language. Chinese (zh) and English (en) are the foreign languages specied most frequently, followed by Russian (ru). We speculate that in these cases malware authors forge HTTP headers either as observed at their own local systems or with respect to the target website. This would depict yet another indicator that malware carries its own (possibly self-made) HTTP implementation. Another reason could be that malware authors explicitly specify foreign languages to hoax web servers.
6.2.3
HTTP Responses
In Sandnet, all HTTP responses observed originated from HTTP servers on the Internet that were contacted by a sample. Therefore, the following analysis is not an analysis of the samples themselves, but may give indications to which type of servers malware communicates.We observed that 97.8% of the HTTP requests were answered with an HTTP response. We dene the HTTP error rate as the ratio between failed responses and all responses. Figure 6.6 shows a distribution of the sample-wise HTTP error rate. Only a small fraction (less than 10%) of samples virtually always get non-successful status-codes and apparently completely fail to retrieve the requested web content. Most samples have a relatively small error-ratio, indicating the web sites requested by the samples are still in place. We will give an overview of the requested servers in Section 6.2.6.
6.2.4
HTTP Response Headers
As opposed to HTTP request headers, response headers are set by servers and are not chosen by the malware samples. Analysing the headers helps us to understand which servers are contacted by malware samples and gives information about the type of the retrieved content.
6.2.4.1
Content-Type
The Content-Type header shows which type of web content was retrieved by the samples.Figure 6.7 shows that most samples at least retrieve web sites with Content-Type text. By far the most popular content-type of textual responses is text/html. However, only about
32
half of all samples retrieved rich documents with Content-Type set to images/* (48%) or application/* (59.4%). 23.9% of the HTTP active samples with more than a single request got textual responses only. We see two reasons for such presumably light HTTP clients: First, spidering web sites without loading images is much more ecient. Second, we hypothesize that a considerable number of samples lacks a full-blown HTTP implementation that can recursively fetch objects embedded in web sites.
Figure 6.6: Distribution of HTTP error rates among samples
Figure 6.7: Ratio of samples using given Content-Type
6.2.4.2
Server
The Server HTTP response header indicates which type of web server is responding to the malwares HTTP request.Note that the content of this header can again be forged.Moreover, the majority of contacted web servers is presumably benign. However, when manually inspecting the HTTP Server response header, we spotted servers that presented suspicious banner strings.
33
6.2.5
HTTP Responses with PE Binaries
After compromising a system with minimized exploits, attackers usually load so-called second-stage binaries. These binaries carry the actual malware functionality rather than just the exploit with minimized shell-code. In Sandnet, we usually analyse second-stage binaries instead of shell-code binaries. Yet, malware authors - as we will show - frequently load new portable executable (PE) binaries that expand or update the functionality of a malware sample. We assume this is due to a modular structure of the typical malware.We extracted all binaries downloaded via HTTP by searching for the typical PE bytes in the body of HTTP responses. This straight-forward extraction of PE binaries already discovered that 16.7% of the samples in SNet loaded additional PE les. To our surprise, we observed that 194% of these samples load binaries for multiple times - occasionally even more than 100 times. We veried that the ve binaries downloaded most often were not corrupt and lack reasonable explanations why the binaries were downloaded that often. In total, we detected 42,295 PE headers, resulting in 17,676 unique PE les. The maximum size of a downloaded binary was 978 kB, the average size is 144 kB.Figure 6.8 shows that most of the samples load more than a single PE binary. For readability of the graph we precluded 21 samples that loaded more than 100 and up to 1080 unique PE binaries. Most samples retrieve reasonable Content-Type values from the server. However, a signicant number of servers tries to camou age PE binary downloads as text, HTML, JavaScript or image les.[1]
Figure 6.8: Distribution of # of PE binaries loaded
6.2.6
HTTP Servers
When recalling that HTTP is a protocol used by malware authors excessively, we see a need in analysing which particular HTTP servers are visited by malware. Obviously,many HTTP requests were put to presumably benign web sites. The next sections should briey discuss why malware contacts these services.
CHAPTER 6. DNS and HTTP 6.2.6.1 Ad Services
34
We identied a signicant number of ad service networks in the list of popular domains. Of the Top 50 domains, we manually identied 40 domains that are related to ads. Thousands of dierent malware samples use these services. A possible reason for this is that ads are included in virtually every web site and crawlers also follow the ads.However, after manually exploring the HTTP trac of particular samples we assume that the reason for the popularity of ad services is vicious: click fraud. We leave it up to future work to analyse and mitigate the abuse of ad services by malware samples in greater detail.
6.2.6.2
Public Web APIs
Similarly to its popularity among benign users, Yahoos and particularly Googles public Web APIs are present in Sandnet trac, too. We suspect there are two reasons behind the popularity of these or similar services. First, some of these services are ubiquitous on the Internet. For example, a wide variety of web sites for example include Google Analytics to record statistics on the visitor behavior. Each time a sample visits such a web site and follows the embedded links, it will contact Google. As most of such services are open to anyone, we also suspect malicious usage of Googles and Yahoos services by malware samples to be a reason for their popularity. A typical scenario that we observed was the abuse of search engines as a kind of CC engine. In this case the malware searched for specic keywords and fetched the web sites suggested from the search results. Moreover, we have observed malware using the search engines to harvest new e-mail addresses for spamming campaigns. In general,benign human interaction with these services is particularly hard to be distinguished from abuse, especially from the client-perspective. We assume this is one of the main reasons malware authors use these HTTP-based services.
6.2.6.3
PE File Hosters
Based on the set of PE les that malware samples downloaded, we analysed the le hosting servers. 42.3% of the samples that downloaded PE les contacted the PE host directly without prior DNS resolution.This proves that still a signicant number of malware samples include hard-coded IP addresses to download binaries.We further observed that a signicant fraction of the URIs requested from le servers are non-static, although frequently only the parameters change. This observation may be important for blacklists trying to block entire URIs instead of IP addresses or domains.
CHAPTER 6. DNS and HTTP 6.2.6.4 HTTP CC Servers
35
HTTP based botnets such as e.g. Torpig switched from the classical IRC protocol to using HTTP. While manually inspecting Sandnet HTTP trac, we occasionally encounter CC trac. What we see most is that samples include infection status information in their GET request parameters. Whereas some samples include clear-text status information, we and others have observed many samples started encoding and encrypting the data exchanged with the server. However, we found it dicult to automatically spot CC servers without knowing the command syntax of specic botnets. The big dierence to IRC is that HTTP is a prevalent protocol on clean, non-infected systems and is thus harder to spot in the volume of HTTP data.[1]
CHAPTER 7 Conclusion
7.1 Conclusion
In this work, we presented a comprehensive overview of network trac as observed by typical malware samples. The data was derived by analysing more than 100k malware samples in Sandnet. Our in-depth analysis of DNS and HTTP trac has shown novel malware trends and led to numerous inspirations to combat malware. The provided data is not only of great value because it is that detailed. It also perfectly complements related work that is either outdated, analyses particular malware families only, or focuses mainly on the host behavior of malware. To share these insights with the research community, Sandnet is accessible viahttp : \ \ www if is net sandnet . We presented an in-depth analysis of malware network behavior that we gathered with a new system called Sandnet . Sandnet is an analysis environment for malware that complements existing systems by a highly detailed analysis of malicious network trac. With Sandnet, we try to address two major limitations we see in publicly available dynamic analysis systems: a short analysis period and the lack of detailed network behaviour analysis. While existing systems have usually spent only a couple of minutes to run a malware sample, we ran each sample for at least one hour. In addition, using the data collected through Sandnet, we provide a comprehensive overview of network activities of current malware. We rst present a general overview of network protocols used by malware, showing that DNS and notably HTTP are prevalent protocols used by the majority of malware samples. We provided an in-depth analysis of DNS and HTTP usage in malware. The main contributions of this work are:We have in operation a new data collection and
36
37
analysis environment called Sandnet that will be up for the long run and that we will continuously use to gather information on malware network behaviour.We give an overview of the network activities of more than 100,000 malware samples and compare the results with data from previous eorts.An in-depth analysis of DNS and HTTP trac provides details on typical protocol-specic usage behaviors of malware.
BIBLIOGRAPHY
[1] Christian Rossow,Christian J. Dietrich,Herbert Bos,Lorenzo Cavallaro,Maarten van Steen, Felix C. Freiling,Norbert Pohlmann. Sandnet: Network Trac Analysis of Malicious Software,2011 [2] Sandnet.http://www.if-is.net/sandnet/. [3] jason ross,Malware Analysis for the Enterprise ; page 4 to 9 [4] Ulrich Bayer, Imam Habibi, Davide Balzarotti, Engin Kirda, and Christopher Kruegel A View on Current Malware Behaviors [5] Ulrich Bayer Christopher Kruegel Engin Kirda Ikarus Software Technical University of Vienna TTAnalyze: A Tool for Analyzing Malware
38

Reports

Hochgeladen von

Dokumentinformationen

Copyright

Verfügbare Formate

Dieses Dokument teilen

Dokument teilen oder einbetten

Freigabeoptionen

Stufen Sie dieses Dokument als nützlich ein?

Sind diese Inhalte unangemessen?

Copyright:

Verfügbare Formate

Reports

Hochgeladen von

Copyright:

Verfügbare Formate

SEMINAR REPORT

Sandnet: Network Trac Analysis of Malicious Software

Department of Computer Engineering

AMRUTVAHINI COLLEGE OF ENGINEERING

Sangamner-422608 Department of Computer Engineering

CERTIFICATE This is to certify that the SEMINAR report entitled

Guide Prof. M. S. Tamboli

Seminar Coordinator Prof. S. A. Thanekar

Head of the Department Prof. R. L. Paikrao

Principal Dr. G. J. Vikhe Patil

Sandnet: Network Trac Analysis of Malicious Software

Department of Computer Engineering

Place: Sangamner Date: March 9, 2013 Ms.Khemnar Suchitra

Place: Sangamner Date: March 9, 2013 Ms.Khemnar Suchitra

iii Abbreviations Description

4.1 4.2 5.1 5.2

viii 3.3 System Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 13 13 14 14 17 18 20 20 21 22 24 24 24 24 25 27 27 28 28 29 30 31 31 32 32 32 32 34 34 34 35 36 36 37 37 39

CHAPTER 2 Malware Analysis

How Does Malware Analysis Help?

CHAPTER 2. Malware Analysis

Time Has Changed (It s a Business, Not a Kiddie)

Signature Arms Race

CHAPTER 2. Malware Analysis

Where Does Malware Analysis Fit In?

CHAPTER 2. Malware Analysis

How Does Malware Today Work?

CHAPTER 2. Malware Analysis

Playing With Fire (How To Analyse Malware)

CHAPTER 2. Malware Analysis

CHAPTER 3 Sandnet and System Overview

CHAPTER 3. Sandnet and System Overview

Network Activity Of Malware

CHAPTER 3. Sandnet and System Overview

CHAPTER 4 Current Malware Behaviours

CHAPTER 4. Current Malware Behaviours

Observed Malicious Behaviour

File System Activity

CHAPTER 4. Current Malware Behaviours

Table 4.1: Overview of observed behaviour

CHAPTER 4. Current Malware Behaviours

Table 4.2: Overview of network activities.

Figure 4.1: Network protocols (by samples)

CHAPTER 4. Current Malware Behaviours

Figure 4.2: Network protocols (by families/clusters).

CHAPTER 4. Current Malware Behaviours

CHAPTER 4. Current Malware Behaviours

CHAPTER 5 System,Dataset and Networks Statistics Overview

CHAPTER 5. SYSTEM,DATASET AND NETWORK STATISTICS OVERVIEW

CHAPTER 5. SYSTEM,DATASET AND NETWORK STATISTICS OVERVIEW an analysis period of 12 years.[1]

Figure 5.1: Histogram of VirusTotal Labels per Sample.

CHAPTER 5. SYSTEM,DATASET AND NETWORK STATISTICS OVERVIEW

Reference 44.5 37.6 2.3 1.6

Sandnet 92.3 58.6 8.0 3.8

Table 5.2: Top 3 protocols over non-standard ports

Protocol HT T P IRC F lash

SNet Samples (%) 8.17 7.13 0.19

Distinct Ports 303 174 9

CHAPTER 6 DNS and HTTP

CHAPTER 6. DNS and HTTP

Figure 6.1: Violin plot of DNS activity end distribution

CHAPTER 6. DNS and HTTP

DNS TTL Analysis

Figure 6.2: CDF of DNS TTL per domainsn

DNS Message Error Rate