Utilising YARA for Malware

Detection
JULY 2020

TLP GREEN

Introduction
The Australian Cyber Security Centre and other cyber security organisations may produce YARA rules in order to help
customers detect malicious software present on a system. This document provides an overview of YARA rules and
guidance on how to effectively utilise YARA rules within your organisation’s IT environment.

What is YARA?
YARA is an open source project aimed at providing the means to describe and detect malicious software. The YARA
project consists of:
 The means to describe how malicious software may be detected through the creation of YARA rules, and
 A software tool, also called YARA, used for scanning files, memory or other data for malware described in YARA
rules.
Further detailed information on YARA as well as for copies of the YARA software please see the YARA website at
https://virustotal.github.io/yara/.

Why does the ACSC release YARA rules?

The ACSC may release YARA rules developed in response to malicious software it encounters in the ACSC’s incident
response and investigation activities. By releasing YARA rules the ACSC enables customers to search for and investigate
potentially malicious activity affecting their IT environment. The use of YARA is widely adopted by the cyber security
industry and can provide customer’s flexibility in detecting this malicious activity using a variety of existing cyber
security software and appliances which support YARA.

1
Considerations when utilising YARA
The following are some general considerations when utilising YARA within your organisation’s IT environment,
regardless of the source of the YARA rules.

Understand the intent and context of the YARA rules

Understanding the intent and context of YARA rules enables organisations to make more effective and efficient use of
YARA rules and triage of results.
This can be achieved by reviewing any accompanying reporting products which may describe the malicious software or
characteristics which the YARA rule aims to identify. Additionally, the YARA rules themselves should be reviewed before
running to develop an understanding of the rules intent and context. Additional information about a YARA rule may be
included in comments, tags or the meta section.

rule pefile_example : pefile windows

{
meta:
description = "Example rule used
to identify Windows PE files."
condition:
uint16(0) == 0x5a4d
}

Examples of the importance of understanding YARA rule context:

 Operating Systems: For example, YARA rules intended to identify malicious Windows Portable Executable files
(such as .exe and .dll files) may not need to be run on hosts with non-Microsoft Windows operating systems.
 Types of hosts: Certain YARA rules may be more applicable to servers as opposed to workstations.
 Scanning of specific file types or folders: For example, YARA rules intended to detect malicious web shells may
only need to be run against certain file types (such as .php or .aspx) or certain directories (directories which hold
content served by web servers).
 Scanning of files, memory or both: Certain YARA rules by design may only be effective at detecting malicious
activity in memory. This may be because the malicious software or data is encrypted or otherwise obfuscated
while it resides on disk.
Reducing the coverage of a YARA rule through limiting the number of hosts or files scanned can increase the risk of
malicious activity remaining undetected. However organisations may accept this risk in order to reduce the resource
impact scanning has within an environment or to produce high priority results faster.

Running YARA across multiple hosts

To provide comprehensive coverage YARA rules should be run everywhere that the YARA rules are applicable and
practical to do so. This could entail running the YARA rules across all web servers or Linux hosts within an IT
environment. When utilising YARA rules organisations should consider how to easily run YARA rules across hundreds or
thousands of hosts in a manageable fashion, while also ensuring that any YARA detections are captured appropriately
and the originating host is identifiable.

2
An additional consideration for use of YARA within a network is considering the future need to run YARA rules across a
network in a repeatable, sustainable manner. Organisations could consider incorporating a YARA tool into standard
operating environments. This would remove the need to deploy YARA to hosts every time a widespread YARA scan is
performed.

Potential performance impacts

Depending on the number and complexity of the YARA rules utilised, the number and sizes of objects to scan and the
resources of the scanning host there may be potential performance impacts. For example, running a YARA scan across
an entire hard drive can be resource intensive depending on the variables identified above.
Mitigating controls to limit potential performance impact include:
 Scheduling YARA scans outside of core hours.
 Reducing the number of YARA rules utilised. For example, removing rules which are not relevant to specific hosts.
 Reducing the number of objects scanned, through targeting specific folders or file types.
 Utilising YARA tool configuration options to limit resource utilisation.

Scanning process memory with YARA

Some YARA rules may be more effective scanning live memory, rather than files on disk. YARA rules may only detect
malicious activity in memory, which may not be possible if the file is encrypted or encoded while it resides on disk.
When memory-based YARA rules are released additional context may be provided with the rule identifying which
process or processes are the best targets for scanning.
However for some malicious activity it may be impractical to specify all possible processes to be scanned. If these
memory-based rules still wish to be used then the YARA tool must scan all current running processes on a host.
If organisations wish to use YARA rules to scan process memory the following should be considered:
 How to determine the correct process or processes to scan, particularly when scanning hosts at scale where it is
impractical to specify the process ID of the target process or processes.
 How to scan the memory of all processes, if required.

3
Example YARA usage
When utilising the standard YARA tool on the command-line an overview of the configuration options are available at:
https://yara.readthedocs.io/en/latest/commandline.html. The ACSC recommends that organisations review this
documentation to understand the below examples and to allow for modifications to best suit their environments.
At the time of writing the latest YARA version was 4.0.2 and the information included below is compatible with that
version.

Recursive directory or drive scanning command-line example

The below is an example of how the standard YARA tool could be used to recursively scan a directory or entire disk
drive:
yara -r -s -f -p 8 <path_to_rules_file> <target_directory>
 -r: Recursively scans directories.

 -s: Prints the matching strings within the file. Useful for triaging and investigating hits after scan completion.

 -f: Fast scan mode, stops searching for additional occurrences of a pattern string after the first match.

 -p: Specify the number of threads used when scanning a directory. The ‘8’ value is included as an example and
should be tailored based on the environment where the YARA tool is being run and potential performance impact
considerations. A lower value should reduce the potential performance impact of a YARA scan.
It is also recommended that the output of the YARA tool be captured through an output redirection mechanism
appropriate for the target system.
A privileged account may be required to scan certain files or folders, based on applied permissions.

Scanning process memory

The below is an example of how the standard YARA tool could be used to scan process memory:
yara –s <path_to_rules_file> <process_id>
It should be noted that, in most environments, scanning memory will require the use of a privileged account.

Scanning specific file extensions examples

The below are some example Windows (PowerShell) and Linux commands which can be used to recursively walk folders
and scan files with specific extensions. In the below example the YARA rules are written to target web shells, so only
files with the .php and .aspx file extension in the main web content folder are going to be scanned.
If a list of appropriate files has already been generated the --scan-list argument for the standard YARA tool can be
used to pass the YARA tool a list of files to scan.
These commands are provided for example purposes. Any scripts or commands run should be tailored to the
environment, scanning purpose and tested before widespread use.

Scanning specific file extensions on Windows

An example of utilising PowerShell within a Windows environment:

4
 Get-ChildItem –File –Recurse –Path ‘C:\Inetpub’ | Where-Object –FilterScript {$_.Name –
match “\.aspx|\.php” } | % { & ‘C:\yara\yara.exe’ -f -s C:\Users\ACSC\rules.yar
$_.FullName } >> C:\Users\ACSC\yara_scan_results.txt
 Enumerate all files ending in .aspx or .php within C:\Inetpub and any sub-folders.
 Scan these files with the YARA tool using the rules within rules.yar.
 Capture the results within yara_scan_results.txt.

Scanning specific file extensions on Linux

An example utilising the common find command within a Linux environment:

find –type f /var/www –iname “*.aspx” –or –iname “*.php” –exec /usr/bin/local/yara –f –s
$HOME/rules.yar ‘{}’ \; &> $HOME/yara_scan_results.txt
 Enumerate all files ending in .aspx or .php within /var/www and any sub-folders.
 Scan these files with the YARA tool using the rules within rules.yar
 Capture the results, and any error messages, within yara_scan_results.txt.

5
Scanning specific process memory by process name on Windows

An example of utilising PowerShell to scan all instances of the process or processes identified by name (using
notepad.exe and calc.exe as examples). Note: scanning process memory will likely require an elevated PowerShell or
command prompt.
Get-Process notepad,calc –ErrorAction SilentlyContinue | % { C:\yara\yara.exe –s
C:\Users\ACSC\rules.yar $_.Id } –ErrorAction SilentlyContinue >>
C:\Users\ACSC\yara_scan_results.txt
 Enumerate all processes named notepad.exe or calc.exe and retrieve their process IDs.
 Scan these process IDs with the YARA tool using the rules within rules.yar
 Capture the results within yara_scan_results.txt
Scanning all process memory on Windows

Performs the same steps as above, but on all running processes. Note: scanning process memory will likely require an
elevated PowerShell or command prompt.
Get-Process | % { C:\yara\yara.exe –s C:\Users\ACSC\rules.yar $_.Id } –ErrorAction
SilentlyContinue >> C:\Users\ACSC\yara_scan_results.txt
Scanning specific process memory by process name on Linux

An example Bash script to scan all instances of the process or processes identified by name (using gedit and bash as
example processes). The use of sudo has been included in the example as it is a likely requirement to scan process
memory within most environments.
pids=$(ps –C gedit,bash –o pid=); for i in $pids; do sudo /usr/bin/local/yara –s
$HOME/rules.yar $i &> $HOME/yara_scan_results.txt; done
 Enumerate all processes named gedit or bash and retrieve their process IDs.
 Scan these process IDs with the YARA tool (utilising sudo for privileged access) using the rules within rules.yar
 Capture the results, and any error messages, within yara_scan_results.txt

Scanning all process memory on Linux

Performs the same steps as above, but on all running processes. The use of sudo has been included in the example as it
is a likely requirement to scan process memory within most environments.
pids=$(ps –e –o pid=); for i in $pids; do sudo /usr/bin/local/yara –s $HOME/rules.yar $i
>> $HOME/yara_scan_results.txt; done

6
Triaging and investigating YARA results
YARA rules can vary greatly in the conditions which must be met to match a rule. YARA rules can be written to be very
specific causing little to no false positives, or they may be written to match a broader set of criteria to increase the
chance of detecting malicious activity at the cost of increased false positives.
After performing a YARA scan any hits should be investigated to confirm any malicious hits and to rule out any false
positives. The triage of hits should involve the following:
 Reviewing the matching YARA rule to better understand its matching criteria, its intent and any comments or
meta information contained within the rule.
 Review the specific data in a file or memory which matched the YARA rule, as well as the surrounding data. This
can help identify potential false positives.
 Identify whether the matched data can be verified as legitimate.
The process of investigating YARA hits is similar to the processes organisations should follow when investigating an alert
raised by anti-virus software or other detection capabilities.
If you are unable to confirm whether a file or process is legitimate after it was identified using an ACSC-provided YARA
rule and your organisation believes it may be related to malicious activity, contact the ACSC by
emailing asd.assist@defence.gov.au or calling 1300 CYBER1 (1300 292 371).

7
Frequently Asked Questions
Do I have to use the standard YARA tool distributed through https://virustotal.github.io/yara/?
No. Any tool which supports the use of YARA rules which can be used. Certain tools may not support the same
configuration options available in the standard YARA tool.

Does the ACSC have any recommendations for a commercial tool which supports YARA rules?
No, organisations should select a tool which meets the organisations identified requirements.

What version of the standard YARA tool should I use?

The ACSC recommends that wherever possible the latest version of software be utilised. Some YARA rules may list a
version of the standard YARA tool it was tested against however these rules will likely still work with later versions of
the standard YARA tool. Where YARA rules are incompatible with the latest version of YARA organisations should
consider whether to utilise a suitable older version of the YARA tool to run these rules.

Will the ACSC provide a tool to help utilises YARA rules?

The ACSC has produced a tool in the past to run certain YARA rules. The ACSC is now providing plaintext YARA rules to
customers for the following reasons:
 Greater flexibility for the ACSC in providing new and updated rules.
 Greater flexibility for customers in how to utilise the rules.
 Providing raw rules allows for organisations to review the rules for suitability within their environment.

I’m experiencing issues running a YARA tool. Can the ACSC help?
Errors experienced while running a YARA tool are likely one of three types:
 A problem with the YARA tool or the environment, or
 A syntax error in the YARA rules.
 Warning messages produced by the YARA tool.
Errors with the YARA Tool or the Environment
If organisations experience errors similar to the following (but not limited to these specific errors):
 yara: error while loading shared libraries: libyara.so.4
 error scanning /path/to/file: could not open file
 The program can’t start because VCRUNTIME140.dll is missing from your computer.
These errors are indicative of a problem with the YARA tool being used or the environment in which it is running. The
ACSC cannot assist in this troubleshooting and organisations are recommended to consult product documentation.
Some errors may prevent the YARA tool from running at all. Other errors, such as permission or access-related errors,
may only affect the successful scanning of those specific files.

8
YARA Rule Syntax Errors
If organisations receive a syntax error, such as:
rules.yar (10): error: syntax error, unexpected ‘}’
This is indicative of a syntax error within the YARA rule itself. To confirm that it is a syntax error, as opposed to an error
with the YARA tool or the environment, the following test rule produced by the YARA project can be used:

rule dummy
{
condition:
false
}

If there is no problem with the YARA tool or the environment scanning a file with the above rule should result in the
YARA tool running and exiting successfully with no output.
YARA Warning Messages
The standard YARA tool may produce warning messages about certain rules or components of rules. For example:
rules.yar(5): warning in rule “warning_example” : $regex contains .* or .+, consider using .{,N}
or .{1,N} with a reasonable value for N
rules.yar(5): warning in rule “warning_example”: $regex is slowing down scanning
These warning will not prevent the standard YARA tool from scanning correctly and can be ignored.
If your organisation experience’s problems when utilising ACSC-provided YARA rules please report any rules related
errors or issues to asd.assist@defence.gov.au. Please note that the ACSC cannot help troubleshoot issues with installing
or running YARA software in general, or issues with rules not provided by the ACSC.

Does running YARA require use of a privileged account?

The privileges required to run the standard YARA tool depends largely on the objects being scanned and the
permissions required to access them. For example, an unprivileged user will not be able to scan objects for which they
do not have read access which may include sensitive files and process memory. If you wish to recursively scan a system
hard drive, or scan process memory then use of a privileged account will likely be required.

Are there common sources of false positives when performing a YARA scan?
The potential for false positives are highly dependent on the rules themselves, however there are some more common
causes of false positives amongst all rules particularly if the rules utilise plaintext strings for matches. These can include:
 Detection of the YARA rules themselves, either as they exist as a file on disk or in process memory, such as stored
in Terminal Services clipboard data (e.g. rdpclip.exe).
 Detection of anti-virus definition files, or of those same anti-virus definitions in memory.

Anti-virus software has identified the YARA rules file as malicious. What should I do?
Due to the way that detection conditions are written YARA rules may contain some segments of the same data which
malicious files contain, particularly strings. Anti-virus definitions may be written to detect this same content leading to
YARA rules files being flagged as malicious. If a YARA rule file is being flagged as malicious by anti-virus software,
organisations can explicitly allow this file within the organisation’s anti-virus software.

9
Contact details
Organisations or individuals with questions regarding this advice can email asd.assist@defence.gov.au.

Incident Reporting
If you have indications that your environment has been compromised, contact the ACSC by
emailing asd.assist@defence.gov.au or calling 1300 CYBER1 (1300 292 371).

10