Start with machine data and Splunk software. End with an unfair advantage.

2014 Splunk Inc. All rights reserved.

www.admin
-
magazine.com
ADMIN
Network & Security
D
ig
ita
l
S
p
e
c
ia
l
Another great
collection of simple
tools for managing,
monitoring, and
configuring your
Linux network
Bonus articles:
ngrep: Easy and efficient network monitoring
hdparm: Tune up your hard disk or DVD drive
10
MORE Terrific
Admin Tools
US$ 7.95
fOR ThE Busy AdMin
Start with machine data and Splunk software. End with an unfair advantage.
Splunk software lets you collect, analyze and transform machine-generated
big data into real-time insight. Proactively monitor and troubleshoot your
infrastructure end-to-end to avoid service degradation and prevent outages
so you can go home early.
Discover the worlds leading real-time platform for machine data.
Download Splunk for free today. www.splunk.com/download
2014 Splunk Inc. All rights reserved.
AD-Splunk-SysAdmin-Apprec-195x275-103.indd 1 7/21/14 4:35 PM
ADMIN
Network & Security
ADMIN Special
Editor in Chief Joe Casad
Managing Editor Rita L Sooby
Proofing and Polishing Amber Ankerholz
Layout / Graphic Design Dena Friesen, Lori White
Advertising www.admin-magazine.com/Advertise
Ann Jesse, ajesse@admin-magazine.com
Phone: +1-785-841-8834
Publisher Brian Osborn
Customer Service / Subscription
For USA and Canada:
Email: cs@admin-magazine.com
Phone: 1-866-247-2802
(toll-free from the US and Canada)
www.admin-magazine.com
While every care has been taken in the content of
the magazine, the publishers cannot be held
responsible for the accuracy of the information
contained within it or any consequences arising
from the use of it.
Copyright & Trademarks 2014 Linux New Media Ltd.
Cover Illustration Vladislav Kochelaevs, fotolia.com
No material may be reproduced in any form
whatsoever in whole or in part without the written
permission of the publishers. It is assumed that all
correspondence sent, for example, letters, email,
faxes, photographs, articles, drawings, are supplied
for publication or license to third parties on a non-
exclusive worldwide basis by Linux New Media
unless otherwise stated in writing.
All brand or product names are trademarks of their
respective owners. Contact us if we havent
credited your copyright; we will always correct any
oversight.
Printed in Germany
ADMIN ISSN 2045-0702
ADMIN is published by Linux New Media USA,
LLC, 616 Kentucky St, Lawrence, KS 66044, USA.
Table of Contents
pkstat 4
Network monitoring works best
when the tool is functional but not
too complicated.
di 5
A handy tool for displaying and
monitoring disk information.
Trickle 6
View traffic stats and shoot down
programs that are taking too much
bandwidth.
GoAccess 7
Study your logfiles in real time.
Mosh 8
Remote access over slow network
connections.
SSLScan 9
The easy way to manage your SSL
servers.
PortSentry 10
Identify and log port scans.
GeoIP Lookup 11
Obtain geographical information on
domain names.
Whowatch 12
Look for intruders with this process
watcher.
Snoopy 13
Log terminal commands for future
reference.
Dear Readers:
Happy SysAdmin Day! The success of last years 10 Terrific Tools list got us
excited about another round. Read on for 10 more simple but useful tools
from the toolkit of Linux Pro Magazine columnist Charly Khnast.
As a special bonus, were also including two more articles
describing other great tools for the busy admins toolkit:
ngrep 14
Ngrep is a pattern-matching tool that separates the wheat from the chaff and doubles as a
lightweight packet sniffer.
hdparm 18
Hdparm is the tool to use when it comes to tuning your hard disk or DVD drive, but it can also
measure read speed, deliver valuable information about the device, change important drive
settings, and even erase SSDs securely.
10
MORE Terrific
Admin Tools
1 0 More Terri fi c AdMi n Tools! Welcome
3
More GreAT Tools for The Busy AdMi n WWW. AdMi n- MAGAzi ne. coM
Today, Im talk-
ing about a task
that isnt exactly
a big thrill for most administrators:
providing human-readable statistics
for traffic on a network interface.
For this task, I recently discovered
pktstat [1] in the course of search-
ing for a compromise between the
monosyllabic IPTraf and the ver-
bose Wireshark. Pkstat is included
by most distributions, and the
source code is available online. To
see the current connections on an
interface, you simply type:
pktstat -i eth0
In a view that is remotely reminis-
cent of top, pktstat shows you the
network activity sorted by class
(ICMP, TCP, UDP, and so on). If
name resolution takes too long for
your liking, you can disable it by
setting the -n parameter. In the case
of protocols such as HTTP, FTP, and
X11, pktstat outputs more informa-
tion about the data transferred, such
as the path and the request method
for HTTP (i.e., GET or POST).
Figure 1 shows the download sta-
tus for the ISO image of the future
Ubuntu LTS version 12.04. You
might notice that pktstat doesnt
show the full names of the source
and target machines only the bit
up to the first dot to ensure read-
ability. If you really want the whole
name, you need to enable the -F
parameter in pktstat.
You tend to lose visibility when
things start to liven up on a net-
work interface. To keep pace, you
can resort to two tweaks. For one
thing, after 10 seconds, pktstat de-
letes from its overview those con-
nections for which no data has
been transferred. You can reduce
this value to one second using the
-k (keeptime) parameter.
Additionally, pktstat updates its
overview every five seconds. Speci-
fying -w 1 speeds it up and refreshes
the view every second. The -w pa-
rameter can be used in another
way: pktstat offers a single-shot
mode, which you enable like this:
pktstat -i eth0 -1 -w 10
The -1 parameter initiates single-
shot mode. Pktstat will run without
screen output for the number of
seconds specified in -w 10. It then
quits and leaves you a tidy over-
view of the connections it identified
as its legacy.
Re-Sorting
The tool offers some other parame-
ters for influencing the output; the
one I use most frequently is -l (last
seen). This tells pktstat to sort the
overview to
show me the
connections
that were last
active. The lon-
ger a communi-
cation is idle,
the farther
down the list it
slides. The -t
parameter (top
mode) will push
data streams
that shovel the
largest volume of data through the
interface to the top of the list. Most
command-line parameters also work
interactively at pktstat run time; you
can press the L key to enable last-
seen mode in this way.
After working with pktstat for a
while, I think you will agree that it
provides administrators an uncom-
plicated approach to discovering
the traffic situation on their net-
works. For the classic question
Which process is currently grab-
bing all of the available band-
width? well, if you want to do
some detective work, you still
need Wireshark. n
Info
[1] Pktstat:
http:// www. adaptive-enterprises.
com. au/ ~d/ software/ pktstat/
The Author
Charly Khnast is a Unix operating system
administrator at the Data Center in Moers,
Germany. His tasks include firewall and DMZ
security and availability. He divides his lei-
sure time into hot, wet, and eastern sectors,
where he enjoys cooking, freshwater aquari-
ums, and learning Japanese, respectively.
When it comes to daily tasks such as monitoring network traffic, administrators should
choose a tool that is sufficiently functional and not too complicated. By Charly Khnast
Traffic Spotting
Figure 1: pktstat was bound to notice me downloading a whole ISO image.
The source and target host names are deliberately curtailed.

H
a
n
n
u

V
i
i
t
a
n
e
n
,

1
2
3
R
F
.
c
o
m
pkstat 1 0 More Terri fi C AdMi n Tools!
4
More GreAT Tools for The Busy AdMi n WWW. AdMi n- MAGAzi ne. CoM
To be fair, I have to admit that
many two-letter commands com-
pensate for their compact size
with a breathtaking number of pa-
rameters. The tool I look at today,
Di [1], is no exception. The name
stands for disk information its
a kind df on steroids. Like its role
model, Di delivers information
about filesystems, but with much
more detail, and the output filters
are much better.
Figure 1 shows the output from
di -a, a list of all mounted filesys-
tems, including filesystems that do
not exist physically but that the
kernel hallucinates into the direc-
tory tree. The parameter -x lets
you specify filesystems you want
Di to hide (e.g., di -a -x proc
keeps the /proc entry from being
listed). You can also specify multi-
ple filesystems in a comma-sepa-
rated list:
di -a -x proc,tmpfs,fuse
Di is clever enough to interpret
fuse as fuse*; thus, my fusectl
type filesystem mounted in /sys/
fs/fuse/co is hidden in Figure 2.
However, you can also turn this
around: The -I ext4 parameter lets
you tell Di to list only ext4 filesys-
tems. Using a comma-separated
list, such as -I ext3,vfat,proc,
will work, too.
Machines as Readers
The example in Listing 1 shows
the basic information for my
(only) ext4 partition; however, of
all this information, I am only in-
terested in the
filesystem usage
stats as a percentage 19 percent
in this case. The -f switch is a par-
ticularly useful option if you want
to process the output in a script. If
I just change the command line
slightly,
di -dH -I ext4 -n -f p
it returns a neat and compact 19%.
The -n parameter suppresses the
line with the headings; -f p re-
stricts the output to the percentage
value. If I had typed an uppercase
P, incidentally, it would have given
me the percentage of free inodes.
A comma-separated list is also
useful for easy ongoing processing
of values. Di knows this and
switches to CSV mode if you ap-
pend -c:
# di -dh -I ext4 -n -c
/dev/sda6,/,"141.9G","19.9G","114.8G",U
19%,ext4
Admittedly, these more complex
Di command lines look pretty
much as though my cat has
walked across the keyboard, but
you can say that of other two-let-
ter tools, too. n
Info
[1] Di: http:// freecode. com/ projects/
diskinfo
The more frequently a command is used, the fewer letters it should have, so
the use of two-key commands like ls, mv, and df is second nature. We look at
di, a previously little-known representative of this club. By Charly Khnast
Di Is All In
Listing 1: di -dH -I ext4
Filesystem Mount Size Used Avail%Used fs Type
/dev/sda6 / 141.9G 19.9G 114.8G 19% ext4
Figure 1: The di -a command displays all filesystems, including the kernel pseudo-filesystems.
Figure 2: The -x parameter excludes specific filesystem types.
1 0 More TerrI fI C AdMI n Tools! di
5
More GreAT Tools for THe Busy AdMI n WWW. AdMI n- MAGAzI ne. CoM
I am over 40 years old and am
starting to mellow in my old age.
No, Im only joking; certain phe-
nomena still drive me up the wall.
For example, when I am using
SSH on a server to edit a configu-
ration file and the bandwidth is
so pathetic that the landing zone
is a matter of luck when you try
to position the cursor that really
makes me mad.
I know, I know, today even a
line to a Black Forest village has
enough bandwidth for an SSH
connection, if you have exclusive
access. Be-
cause hell, as
Sartre already
knew, is other
people: In my
case, its the
HTTP connec-
tions that are
pushing my
poor little SSH
to the edge. I
could turn to
Mosh [1], but
that helps with shaky connections
rather than crowded lines. My
remedy for traffic jams goes by the
name of trickle [2] [3].
This traffic-shaping tool uses
LD_PRELOAD to redirect some standard
library calls, such as socket() and
therefore only works with dynami-
cally linked binaries. However, that
practically includes all programs
that the typical user deploys to eat
up bandwidth. In the simplest case,
I might even be one of these users
myself; then, I can practice self-re-
straint when calling traffic-produc-
ing programs. To this end, I can start
Firefox, for example, with:
trickle -u 32 -d 256 firefox
This command limits the upload
speed (-u) to 100KBps and the
download speed (-d) to 300KBps.
Beware: These are actually kilo-
bytes, not bits. I can also reduce
speed in one direction only, if I am
not worried about the other direc-
tion. Figure 1 shows the successful
application of a
download limit
to 2Mbps.
Late
Throttle
Trickles boons
naturally only
occur to me
when the
download is al-
ready running
and the SSH
session hangs. Luckily, trickle has
a daemon mode. Therefore, I
launch trickle when I boot the ma-
chine with
trickled -u 32 -d 256
The values must be adapted to
match the available bandwidth.
When launched, the trickle dae-
mon searches for /etc/trickled.
conf, which can look like Listing 1.
It assigns certain protocols a prior-
ity and does some tweaking.
The values that follow
Time-Smoothing and Length-Smooth-
ing determine how great the fluctu-
ations can be over a certain inter-
val. The smaller the value, the
greater the benefits are for interac-
tive protocols such as SSH. With
larger values, sometimes a major
outlier is permissible in both up-
ward and downward directions.
Trickle has some disadvantages
compared with real traffic shaping,
but its fine for home use maybe I
really am getting soft. n
Info
[1] Charlys Column: Mosh by Charly
Khnast. Linux Magazine, November
2013, pg. 46: http://www.linux-
promagazine.com/Issues/2013/156/
Charly-s-Column-Mosh
[2] Trickle: http:// monkey. org/ ~marius/
pages/ ? page=trickle
[3] Traffic shaping with Trickle by Oliver
Frommel. Linux Magazine, January
2006, pg. 70:
http:// www. linux-magazine.
com/ Issues/ 2006/ 62/
Traffic-shaping-with-Trickle
If your data traffic suffers from congestion at times, dont worry. Now you can shoot
down programs that are heavy on traffic to free up the inflow and outflow. By Charly Khnast
Blown Away
Figure 1: Speedometer shows how a download succeeds at a
speed of around 2Mbits per second.
[ssh]
Priority = 1
Time-Smoothing = 0.1
Length-Smoothing = 1
[ftp]
Priority = 5
Time-Smoothing=3
Length-Smoothing=5
[www]
Priority = 10
Time-Smoothing = 8
Length-Smoothing = 10
Listing 1: trickled.conf
Trickle 1 0 More TerrI fI C AdMI N Tools!
6
More GreAT Tools for The Busy AdMI N www. AdMI N- MAGAzI Ne. CoM
A system admins choice of weap-
ons for dueling with the daily grind
is likely to be pretty conservative.
For example, Web alizer has been
my tool of choice for analyzing web
server logs for something approach-
ing eternity. However, there is no
shortage of alternatives: AWStats,
AWFFull, and others are available
for adventurous admins. But, why
experiment? These tools just do the
same thing that is, create intuitive
evaluations from web server logs.
I am tempted to be unfaithful,
however, if I need a real-time sum-
mary, with precision down to a sec-
ond. Apachetop gives you a line on
this, and I wrote about it some time
back [1]. Since then, a better tool
has hit the market: Go Access [2].
This tool parses the web server log-
file, evaluates it up to the present
point in time, and displays the re-
sults at the command line. Go Ac-
cess reads typical logfiles in Com-
mon Logfile format, but also in
Combined Logfile format. If you
have something more exotic, you
have the option of teaching Go Ac-
cess how to handle it.
Go Access writes its output contin-
uously, so I can watch the web server
work in real time. In the simplest
case, you just pass in one parameter,
the path to the logfile, at run time:
goaccess -f /var/log/apache2/access.log
Another practical aspect is the
ability to add an IP address and, at
the same time, tell Go Access to ig-
nore access to it:
goaccess -f /var/log/apache2/U
access.log-e 10.50.1.25
This approach avoids evaluating ac-
cess by the monitoring systems
(Nagios, Icinga, or load balancers),
all of which cyclically check whether
the server is still alive.
Forward Roll
The command-line display is di-
vided into sections Go Access calls
modules. The modules are listed
from the top down, so you need to
scroll to see them all. Figure 1 only
shows the first four modules and the
first line of the fifth. However, Go
Access displays 11 modules, includ-
ing overviews of the most com-
monly used browsers and client op-
erating sys-
tems, the most
frequently re-
ferring sites,
and search
keys that
prompted
search engines
to point users
to my website.
Another
practical thing
is that Go Ac-
cess painstak-
ingly differenti-
ates between
crawlers and
genuine brows-
ers in its evalu-
ations. A mod-
ule only shows
the Top 6 list for its category. More
details are quickly accessed, how-
ever: Each module is represented by
a number in the header; for exam-
ple, 6 - Operating Systems. If you
press 6 on the keyboard and then O
(for open Detail View), you are
treated to a full list view. Pressing F1
displays more interactive shortcuts.
Authoritative reports on closed
logfiles are naturally part of Go Ac-
cesss feature set. It looks like Ill
be sending Webalizer and Apa-
chetop off to the happy hunting
grounds soon. n
Info
[1] The Sysadmins Daily Grind: Webalizer
Xtended by Charly Khnast, Linux Maga-
zine, February 2006, pg. 65
[2] Go Access: http:// goaccess.
prosoftcorp. com
Just as a craftsman is unlikely to purchase a new angle grinder every
month, sys admins are unlikely to change their tried and trusted tools
very often. Columnist Charly Khnast ditches this conservative philosophy
this month, lured by the charms of a new logfile tool. By Charly Khnast
Sweet Logger
Figure 1: Four of 11 real-time-capable Go Access modules.
1 0 More Terri fi C AdMi n Tools! Go Access
7
More GreAT Tools for The Busy AdMi n www. AdMi n- MAGAzi ne. CoM
Mosh 1 0 More Terri fi c AdMi n Tools!
8
More GreAT Tools for The Busy AdMi n www. AdMi n- MAGAzi ne. coM
I am writing this column on the
Costa Brava and currently dang-
ling my feet in the Med. This
stretch of coast is aptly named;
brava can be translated as
wild, inhospitable. Unfortuna-
tely, this description also applies
to Internet coverage beyond the
tourist beaches although WiFi
is ubiquitous in hotels, camp-
sites, and bars. At the moment,
Im using a network operated
by the Xiringuito beach bar
near the picturesque ruins of the
ancient Greek trading exclave of
Empries, and the connection is
pretty brava.
This situation is not going to
spoil the sunny afternoon for me,
however, because I still have
Mosh [1] stashed away as an ace
in my beach bag. The SSH re-
placement consists of a client
component and a server compo-
nent along with a wrapper script.
Initially, Mosh connects the client
and server via SSH on port 22 in
the normal way. Then, the server
hands the client a key, with
which it identifies itself hence-
forth, and Mosh drops the TCP
connection.
At this point, the client and ser-
ver talk only on UDP, using a port
in the range between 60000 and
61000 by default. I can use the
--port=<Portnumber> parameter to
force Mosh to prefer a specific
port. UDP connections are very
robust; they even survive client
suspend phases.
Whats even better is that, be-
cause the client uses the key initi-
ally received from the server to
identify itself, it can even switch
IP addresses. So, if the beach cafe
network collapses and I swap to
smartphone tethering, my Mosh
session continues unfazed, and
my seaside reverie is undisturbed.
Token of Appreciation
If the only available connection is
unstable, this can lead to the
known issue that SSH does not
show you what you typed at the
terminal until the TCP connection
recovers. Although Mosh cant
work miracles in this case, it is cle-
ver enough to guess what the ter-
minal should be displaying, and it
sends the characters for output
just in case. Synchronization via
UDP continues to run in the back-
ground.
Thanks to Moshs predictive
mechanism, working at the com-
mand line is a much smoother ex-
perience for me than using SSH.
Mosh also doesnt leave you in
the dark about what has actually
been transferred and what bytes
are just predicted: The characters
that the Mobile Shell predicts are
underlined (Figure 1). So, if I want
to see the whole truth, I can disable
the prediction function with
--predict=never. Equally, I can
force prediction using
--predict=always. The default beha-
vior is a compromise: Mosh mea-
sures the latency of UDP connec-
tion in the background and swit-
ches on the predictive function if
the connection quality deteriorates.
Mosh has become indispensable
for me on the road. It cannot com-
pletely replace SSH, because it cur-
rently does not support X11 or port
forwarding and only speaks IPv4.
However, the developers are wor-
king on IPv6 as well as on an app
for Android mobile phones, which
is due for release on some other
sunny day. n
Info
[1] Mosh: http:// mosh. mit. edu
dangling your legs in the sea while enjoying the Mediterranean sunshine can
affect the prospect of a good internet connection; fortunately, charly knows
what to do. By charly Khnast
Shell on the Beach
Figure 1: Mosh underlines characters that have not been transmitted because of a poor connection.
SSL-secured services are the
rule today, rather than the excep-
tion. But, how can I quickly and
easily check a large number of
servers to see whether the en-
cryption methods in use are still
up to date? With the SSLScan
tool [1].
In the simplest case, I can just
call SSLScan with the URL of the
website that I want to test: sslscan
example.com. Listing 1 shows that
SSLScan simply tried a long list of
ciphers and returned a status of
Accepted, Rejected, or Failed for
each one.
However, I am
primarily inter-
ested in what ci-
phers the server
accepts, not what
it rejects. The fol-
lowing command:
sslscan --no-failed
U
www.example.com
helps me signifi-
cantly thin out the
output, reducing it
to a third of the
original length.
Things become
even clearer if I
add more restric-
tions. For example,
if I want to know
whether the server
still supports
SSLv2, I can check
the target like this:
sslscan --no-failed
U

--ssl2
www.example.com
The --ssl3 and
--tls1 parameters
work in the same
way; however,
SSLScan also lets you test mail serv-
ers not just web servers. You need
the --starttls parameter to do
this. Figure 1 shows the output from
sslscan --no-failed --starttls
--tlsv1kuehnast.com:25
The last column of the figure shows
which ciphers the server prefers.
Redirection
I can use --xml=<file name> to re-
direct the output to an XML file.
This method is useful for a script
with which I periodically check
and/ or document the encryption
capabilities of the server. A combi-
nation with --targets=<file name>
is useful here. I can use this to
write a list of host names to the
file along with the port numbers,
if there happen to be any ports
other than 443. SSLScan then au-
tomatically checks the machines
one after another.
Another addition to my toolbox!
The SSLScan security checker is
fast, lean, and easy to automate. n
Info
[1] SSLScan: http:// sourceforge. net/
projects/ sslscan/
If, like Charly, you manage SSL-secured servers, read on to discover a tool that
you will definitely appreciate. It checks whether the complete security setup is
up to date. By Charly Khnast
Keychain for Life
Figure 1: Charly uses SSLScan to check his mail server.
Listing 1: sslscan example.com
01 Supported Server Cipher(s):
02 <...>
03 Failed SSLv3 256 bits ECDHE-ECDSA-AES256-SHA384
04 Accepted SSLv3 256 bits ECDHE-RSA-AES256-SHA
05 Rejected SSLv3 256 bits ECDHE-ECDSA-AES256-SHA
06 <...>
1 0 More TerrI fI C AdMI n TooLS! SSLScan
9
More GreAT TooLS for The BuSy AdMI n www. AdMI n- MAGAzI ne. CoM
Scanning the ports on a machine
belonging to someone else is not
generally regarded as an attack.
Of course, any serious attack will
be preceded by a port scan. Ad-
mins who take security seriously
always take a proactive approach
to port scans, such as blocking the
IP address that initiated the scan
for an extended period of time.
Port Sentry [1] lets you do this and
is included in most distributions.
The daemon identifies and logs
port scans and runs commands af-
ter doing so. The detection mode
is set in /etc/default/portsentry:
TCP_MODE="tcp"
UDP_MODE="udp"
If you dont want PortSentry to mon-
itor UDP ports, just delete the second
line. If you replace tcp and udp with
stcp and sudp, the tool is more sensi-
tive to stealth scans. If you enter atcp
and audp, it binds all unused ports
below 1024 and reports them to the
attacker as open; this means that the
attacker knows just as much about
your system after the scan as before.
The /etc/portsentry/portsentry.
conf file gives you more scope for
setting up the system. Here, you can
define trigger ports that act as port
scan detectors. The default selection
is very useful; I would only change
it if I were running a daemon on one
of these ports.
It is more important to set the
sensitivity with the SCAN_TRIGGER
variable. The default of 0 means
that PortSentry reacts immediately
if a trigger port is addressed. Val-
ues of 1 or 2 reduce the sensitiv-
ity and thus avoid false positives.
ADVANCED_EXCLUDE_TCP= does the
same thing: Ports that are often
addressed by external hosts, such
as Ident (port 113) and NetBIOS
(port 139), are excluded in atcp
mode; similarly ADVANCED_EXCLUDE_
UDP- excludes the UDP ports 67,
137, 138, and 520 (DHCP, Net-
BIOS, RIP) (Figure 1).
By default, PortSentry doesnt
respond to scans but simply logs
their existence. You can modify
this behavior with:
BLOCK_UDP="0"
BLOCK_TCP="0"
A 1 here prevents IP addresses that
have issued port scans in the past
from opening connections by telling
PortSentry to issue
/sbin/route add -host $TARGET$ reject
which drops the connections and re-
turns a refused message (Figure 1).
The IP address that issued the port
scan is logged in /var/lib/portsen-
try/portsentry.blocked and stays
there until you restart the daemon.
Securing Your Weapons
To prevent your own systems from
falling foul of PortSentrys traps, you
have the /etc/portsentry/portsen-
try.ignore.static file, which is
where you define individual hosts
or whole networks that will not be
counterattacked. Incidentally, if you
set BLOCK_TCP and UDP to 2, Port-
Sentry will run the command that
you define as KILL_RUN_CMD this
could be something like issuing a
text alert, but it could just as easily
run the large-bore Metasploit
weapon for vicious counterattacks.
A word of caution: Pointing a dou-
ble-barreled shotgun at somebody
who knocks at your front door is
generally regarded as unfriendly. n
Info
[1] PortSentry:http:// sourceforge. net/
projects/ sentrytools/
To celebrate 10 years of his column, Charly sets up a sensitive detector that
measures the cosmic background radiation of the Internet. By Charly Khnast
Ten Years After
Figure 1: PortSentry initializing and detecting port scans in line with its configuration.
Xxx PortSentry 1 0 More TerrI fI C AdMI n ToolS!
10
More GreAT ToolS for The BuSy AdMI n www. AdMI n- MAGAzI ne. CoM
All popular distributions include
one or more packages that identify
the country of origin of an IP ad-
dress. On my Ubuntu lab machine,
I use the geoip-bin and geoip-data-
base packages. Now, you can also
use the geoiplookup command and
geoiplookup6 for IPv6 addresses,
with an IP address or a name as a
command-line parameter:
$ geoiplookup linuxfoundation.org
GeoIP Country Edition: US, United States
For most purposes, I just need to
map the IP address to a country.
My spam filters use this technique
to determine the top five spammer
domiciles on a daily basis. Figure
1 shows that this is Germany, but
this is likely because I grabbed the
screenshot on a Sunday. Germany
is very rarely in the top five during
the week.
If you need more granular reso-
lution that is, you dont just
want the country, but the city, re-
gion, or organization you can
use GeoIP data by commercial pro-
viders. Typing geoiplookup linux-
foundation.org would then reveal
the following:
GeoIP Country Edition:US, United States
GeoIP City Edition, Rev 1: US, OR, U
Medford,N/A, 42.326500, -122.875603, U
813, 541
GeoIP ASNum Edition: AS3701 Oregon U
JointGraduate Schools of Engineering
A libapache2_mod_geoip module is
available for web servers. This
helps me direct users to the area of
the site localized for them based
on their origin.
Sorting by Country
To sort by country, I added the fol-
lowing to my httpd.conf:
GeoIPEnable On
GeoIPDBFile /usr/share/geoip/geoip.dat
You might also need to modify the
path. I then added the lines from
Listing
1 to my
.htaccess file.
The accuracy of the geodetic
data is almost always good
enough, at least at the country
level, but exceptions just go to
prove the rule.
Cellular radio providers route
their HTTP traffic through man-
datory proxies. Depending on
the network load, the proxy
might be in a neighboring coun-
try, giving rise to suspicions of
mass emigrations. n
The global village is big enough to want to find out where your
friend and enemies have set up camp. Charly offers a quick
IP-based introduction to geography. By Charly Khnast
Land Ahoy!
Figure 1: Germany is the world champion! At least on this strange Sunday and for Charlys antispam system with its integrated GeoIP lookup.
Listing 1: .htaccess Additions
01 #IP Address of .de
02 RewriteEngine on
03 RewriteCond %{ENV:GEOIP_COUNTRY_CODE} ^DE$
04 RewriteRule ^(.*)$ http://www.example.com/de
05
06 #Everyone else sees the English page:
07 RewriteEngine on
08 RewriteRule ^(.*)$ http://www.example.com/en/
1 0 More TerrI fI C AdMI n Tools! GeoIP lookup
11
More GreAT Tools for The Busy AdMI n www. AdMI n- MAGAzI ne. CoM
Every server
with an IP ad-
dress on the
Internet receives
uninvited visits
at some point. The
usual scans and scripted
carpet bombing simply bounce off
my machines thanks to clever fire-
walling, port knocking [1], and tools
like Fail2ban [2]. To keep attackers
from working around my defenses,
I use two rootkit hunters: Rootkit
Hunter [3] and Chkrootkit [4]. The
latter, unfortunately, accuses my
DHCP server of packet sniffing:
eth0: PACKET SNIFFER(/usr/sbin/dhcpd[28382])
This result is a known false positive,
which I ignore. As an interim report,
I can say that my
varmint hunters
have not seen any
prey thus far.
Nevertheless, I
occasionally go on
patrol to see
whether a server is
behaving strangely.
I use whowatch [5]
for this purpose,
which launches in
the terminal with a
process list; the sec-
ond column shows
the owner. In the
third column, Who-
watch tells me
whether the user is
local or logged on
via SSH, Telnet, or
in some other way.
For remote users,
this information is followed by the IP
address, and for local users, just :0.
Hotkey Control
I have two ways of navigating this
information: I can use the arrow
keys to select a line, press Enter,
and see a tree view of the associ-
ated processes, as shown in Figure
1. Pressing O (owner) hides or dis-
plays the process owner; pressing D
(details) creates a window with de-
tailed information for the process.
My second option is to type T
(tree view) to show all running pro-
cesses. In this tree view, too, press-
ing D will display more information.
Pressing L (list of signals) shows me
the control signals that I can send to
the process, such as HUP, INT, TERM,
and in an emergency KILL. I can dis-
play the overall system status, partic-
ularly in terms of memory manage-
ment, by pressing S (sysinfo), which
tells Whowatch to display the total
load on the screen, in a style very
much reminiscent of top (Figure 2).
I have never found anything dan-
gerous on my server patrols to date,
but I do like that warm, safe, and
cozy feeling. n
Info
[1] Fwknop: http:// www. cipherdyne. org/
fwknop/
[2] Fail2ban: http:// www. fail2ban. org
[3] Rootkit Hunter: http:// rkhunter.
sourceforge. net
[4] Chkrootkit: http:// www. chkrootkit.
org (in Portuguese)
[5] Whowatch: http:// whowatch.
sourceforge. net
For no particular reason, Charly occasionally patrols his server farm and hunts
down attackers. He has put together a neat toolbox for this job. ByCharly Khnast
On Patrol
Figure 1: In the tree view, Whowatch shows admins all the processes on the system.
Figure 2: Is this top? No, its Whowatch showing the total load after the S key has been pressed.
Xxx Whowatch 1 0 More Terri Fi C AdMi n Tools!
12
More GreAT Tools For THe Busy AdMi n WWW. AdMi n- MAGAzi ne. CoM
At work, Im sometimes plagued
by annoying gaps in my memory:
What exactly was the name of
that neat tool that I used to flash
the LEDs on a specific network
adapter to help me find the NIC
in the rack? Or: How exactly did I
delete all files that were more than
a week old in a directory? The
answer to all of these questions is
in the Bash history, but Murphys
Law dictates that the history is
always a little bit too short. And,
in my case, theres another degree
of uncertainty: Which server did I
do this on?
Snoopy potentially offers a solu-
tion. The small library with the
dogs name, wraps around ex-
ecve() and always wakes up when
the computer runs a command.
Many distributions have Snoopy in
the pen, but if not, GitHub [1] will
help you out.
To enable Snoopy at boot time,
you need an entry in /etc/ld.so.
preload. I added the following line:
/<path>/snoopy.so. The path is typ-
ically lib. If you are building
Snoopy yourself, the library is
likely to be found in /usr/local/
lib/ or something similar.
Building Snoopy yourself offers
some benefits. For example, you
can edit the snoopy.h header file in
the source up front. If you enter
#define ROOT_ONLY 1
Snoopy only logs commands that
run with root privileges, but if you
install the tool from the distribu-
tion repositories, this option is not
set, and it logs any old command
no matter who ran it.
Unless configured to do other-
wise, Snoopy writes to /var/log/
auth.log. Figure 1 shows the log for
some simple commands. The struc-
ture always stays the same; each
entry starts with the user ID, fol-
lowed by the session ID and the
TTY you use. This is then followed
by the working directory, which is
important because Snoopy does not
log commands like cd /etc. Navi-
gating the system is not the same
for this dog as executing a file.
This information is followed by
the full path to the executed file
and, finally, the expanded com-
mand (e.g., aliases can cause an
expansion). Many distributions
run ls --color=auto, so, in this
case, if you only type ls, Snoopy
reveals all.
Collection Point
Now you just need to consolidate
the logs centrally. I configured one
server to accept the log messages
from other machines. If the server
runs rsyslog, you can just pass in
the -r parameter at boot time to
switch rsyslog to receive mode.
Next, you can tell your other serv-
ers also to send entries in /var/
log/auth.log to the newly config-
ured syslog server. To do this, you
just need to add one line to the
syslog configuration:
auth,authpriv.* @<192.168.2.80>
The auth log tends not to grow
drastically, which means you can
rotate on a weekly or even monthly
basis. Snoopy fills a substantial log
of my heroic deeds of administra-
tion day after day including typos
and similar peanuts. n
Info
[1] Snoopy: https:// github. com/ a2o/
snoopy
Sometimes sys admin Charly needs to know when exactly he did something
ingenious on one of his servers. Finding an infallible memory aid is difficult,
you might think. Peanuts! says Charly. By Charly Khnast
Guide Dog
Figure 1: A neatly maintained history thanks to Snoopy.
1 0 More Terri Fi C AdMi n ToolS! Snoopy
13
More GreAT ToolS For The BuSy AdMi n www. AdMi n- MAGAzi ne. CoM
You might want to inspect your
network at a very detailed level
for a number of legitimate rea-
sons. Much of the time, its to
debug an application thats misbe-
having and connecting to a server
on the wrong port, or maybe a
colleague has noticed a slowdown
on a particular network link, and
you need to diagnose where the
sudden flood of multidirectional
traffic is coming from.
On the other hand, you might
need to check the exact nature of
an attack and perform some real-
time forensic diagnostics to cir-
cumvent it. Leaving the network-
ing aspects aside for a moment,
even an admin solely responsible
for systems and not networks
(an exceptionally rare remit these
days, admittedly) needs a highly
functional packet sniffer avail-
able at all times. Because systems
rely so heavily on connectivity
for multifaceted Internet
usage, its
imperative for admins to be able
to inspect the contents of the
network deeply and interpret the
results proficiently.
The all-pervasive networking
tool tcpdump [1] is undoubtedly
still the champion of packet sniff-
ers but, for certain scenarios,
I much prefer an equally light-
weight package called ngrep [2],
sometimes called simply network
grep. As its name suggests, ngrep
does for networks what grep does
for files; its a highly functional
network pattern-matching tool
that helps the user sort the wheat
from the chaff, and on a busy net-
work, you will need a great deal
of assistance to determine what
the seemingly endless flood of
characters quickly running up the
screen actually means.
Whats the Difference?
When I first started looking at
networks in any great detail, I
was initially attracted to ngrep
because its command structure
seemed to be in plain English. It
uses words, unlike this tcpdump
example, which doesnt exactly
make sense at first glance:
# tcpdump -vv -i eth1 'tcp[13] & 2 = 2'
Matching either SYN only or SYN-ACK
U
datagrams
The preceding example looks
more like a demonstration of
why I should have listened to
my mathematics teacher prop-
erly when I was still in school.
If youre familiar with regular
expressions, then youll know one
of the aspects that made tcpdump
so popular was its flexibility. On
the other hand, ngrep follows
the same path but appears to of-
fer more of a grep-style filtering,
which, having used grep fre-
quently, to my mind at least feels
more intuitive to use. However,
you dont need to be strictly pur-
ist, and using both tcpdump and L
e
a
d

i
m
a
g
e

J
e
a
n
-
L
u
c

G
i
r
o
l
e
t
,

1
2
3
R
F
.
c
o
m
Ngrep is a pattern-matching tool that sorts the wheat from the
chaff and doubles as a lightweight packet sniffer. By Chris Binnie
Network grep
Thresher
ngrep: Network Grep 1 0 HaNdy adMI N tools
14
More Great tools for tHe Busy adMI N www. adMI N- MaGazI Ne. CoM
ngrep can provide a great deal of
invaluable functionality.
Words and Numbers
To begin, Ill look at some simplis-
tic filtering rules that make ngrep
so attractive. To access a network
interface fully, you will need el-
evated privileges (e.g., su - or sudo
-s) to fully achieve that status be-
fore running the examples below.
For those of you less concerned
with repetitive strain injury, simply
prefix sudo to your command lines.
If youre concerned about email
traffic and need to watch all TCP
traffic closely using the SMTP
port, then you could construct a
command line such as:
# ngrep -d any port 25
Here, the SMTP example shows
that (in more recent libpcap library
versions, at least) you can ask
ngrep to listen on all the available
interfaces at once; otherwise, you
might just specify -d DEV or, for ex-
ample, -d eth1 instead to specify a
particular network interface.
Now, Ill expand on that first
command a little and add more
switches to the example. By omit-
ting the -d any parameter, the
trusty ngrep will assume a default
interface, usually eth0. Just ap-
pend it as above if the examples
that follow arent what you need.
You can drill down into any
HTTP traffic on your network
link by mentioning port 80. Ad-
ditionally, you can isolate one
sender IP address that is sending
the port 80 traffic. Notice the src
host syntax:
# ngrep port 80 and src host 12.34.56.78
Moving on from a single IP ad-
dress, imagine that you have so
much data from that single IP
address that you want to refine it
even further and specify a desti-
nation address, too. In this case,
your example would look like this:
# ngrep port 80 and
U
src host 12.34.56.78 and
U
dst host 98.76.54.32
The dst host appendage followed
by the destination IP address is, I
hope, self-explanatory. If you see
fit, you can then easily interchange
the host element with net; if you
use the CIDR format [3], your com-
mand line might then look like
# ngrep port 80 and
U
src net 12.34.56.0/24 and
U
dst net 98.76.54.32/27
instead.
Master Class
By now, I hope you can see how
its possible to wade through even
the heaviest floods of network
traffic and still discern whats go-
ing on and from where. One of the
more granular functions of ngrep
is its ability to pick out certain
pieces of information quickly from
the deluge of data thats streaming
up the screen. For unencrypted
logins, this works a treat. I sin-
cerely hope its only in a LAN
environment that you are still us-
ing Telnet, but if you need to hunt
down the login prompt to a Telnet
server, you can use this:
# ngrep -t -wi "login" port 23
Running this command spawns
ngrep under the default network in-
terface and offers the following in-
formation in addition underneath:
filter: (ip or ip6) and ( port 23 )
match: ((^login\W)|(\Wlogin$)|
U
(\Wlogin\W))
Here, ngrep is saying it will listen
for both IPv4 and IPv6 traffic on
port 23 for Telnet. The match is the
pattern for which ngrep is search-
ing. The -w switch tells ngrep
to match the regular expression
(login, in this case), and the -i
switch means ignore case sensi-
tivity on that regular expression.
If youre stopping a steady
flow of traffic shooting up your
screen with Ctrl+C, then its use-
ful to have a time reference when
youre scrolling back through the
data, and thats exactly what the
-t parameter should do, with
timestamps for each match in the
form: YYYY/ MM/ DD HH:MM:SS.
UUUUUU.
Flick a Switch
Before I look at more examples,
Ill take a breather and look a little
at the some of the other available
switches that ngrep supports.
If youre keeping a keen eye
on all network traffic, you might
even have the need to look at
empty packets, which are usually
discarded because they have no
actual payload through which to
search. By adding -e to the com-
mand, then despite the added
regular expression, you can still
catch empty packets on the net-
work, which could be of a mali-
cious nature.
Conveniently, in the same vein
as the stalwart grep, you could
simply add -v to reverse the filter
to see packets that dont match
the prescribed pattern.
I mentioned using tcpdump in
hand with ngrep, and the -l
option works nicely for this. If
youve captured and saved a large
dump of network data to a file
with tcpdump, then you can run
ngrep over the top of that data file
and use its simple, yet powerful,
searching functionality to do so.
1 0 HaNdy adMI N tools ngrep: Network Grep
15
More Great tools for tHe Busy adMI N www. adMI N- MaGazI Ne. CoM
Using the example from above,
you can search for Telnet logins
from within a pre-saved tcpdump
dump file:
# ngrep -wi "login" port 23
U
-I <filename>
By enabling -X, you can inform
ngrep that youre looking for a
hexadecimal pattern, and not
plaintext, which is useful for more
advanced searching.
Finally, how about dumping di-
rectly from ngrep onto a text file of
your choice? Its a simple maneu-
ver and involves the -O parameter.
The nice thing about this fea-
ture is that it allows you to see
all of the required data on your
screen and still store it in a pcap-
compatible data file for later (the
highly portable libpcap library
format).
Lead by Example
Next, you can gather this newly
found knowledge and apply some
of these switches to what will
hopefully prove to be useful exam-
ples. Some of these are available in
more detail on the ngrep website
if you get stuck or are curious, but
Ill cover a few others, too.
Returning to the Telnet login
example above, think about an un-
encrypted and clear-text FTP login
sequence and how you might go
about pattern matching such a ses-
sion taking place on your network:
# ngrep -wi -t
U
-d eth0 'user|pass' port 21
The FTP login session capture
is frighteningly simple, as is the
Telnet login capture, and high-
lights precisely why everything
for which you can justify a little
extra complexity is encrypted on
networks these days.
Apparently ngrep can also delve
into the payloads of packets us-
ing regular expressions. This next
regular expression looks at a pre-
recorded network dump file. One
such (untested by me) expression
to examine US social security
numbers could be:
# ngrep -t -O <filename>
U
'~.*(\*|\[[^]]*)'
To spot an HTTP attack that in-
volves endless HTTP POST com-
mands, you add a caret in front of
the regular expression,
# ngrep -t '^(POST) ' 'dst port 80'
which instructs ngrep to look only
for POSTs at the beginning of the
payload associated with the packet.
Well Refined
One of the most important fea-
tures of ngrep is its ability to sort
the wheat from the chaff. If you
looked at raw port 80 traffic, you
would see lots of useful informa-
tion, as well as lots of potentially
useless information that doesnt
help you decipher whats travel-
ling across your network link. The
following HTTP sniffing example
is going to be noisy in terms of
output,
# ngrep port 80
whereas the next example, which
uses the clever byline functional-
ity, helps boil down the screeds of
information efficiently:
# ngrep -W byline port 80
The byline function is the epitome
of simplicity and wraps text when
a new line is spotted, making
those raw HTTP packets sig-
nificantly easier to read with the
human eye. It differentiates the
packet headers and their associ-
ated payload nicely, too.
Reaction Time
On my travels, I once came across
a useful tool called tcpkill [4].
In the past, I have used it on a
Linux router to drop specific con-
nections between hosts that are
unnecessary or malicious. It might
surprise you to know that ngrep
offers exactly that functionality
too; that is, it lets you capture and
disconnect certain network traffic,
disrupting the TCP flow between
hosts by sending a set number of
RSTs.
In this case, the ngrep manual
offers the following entry for the -K
parameter and mentions the tcpkill
tool as well: -K num Kill matching
TCP connections (like tcpkill). The
numeric argument controls how
many RST segments are sent.
The Beginning of the End
This bit of insight might tempt
you to turn to ngrep the next time
youre looking for something on
your networks. The clarity of its
output and its minuscule instal-
lation footprint make it an indis-
pensable tool.
I havent gone into any detail re-
lating to ngreps formatting of bi-
nary (hexadecimal) traffic, but its
certainly impressive and, again,
uses a familiar grep structure.
Combined with its other function-
ality, ngrep is undoubtedly a force
to be reckoned with. n
Info
[1] tcpdump: http:// www. tcpdump. org/
[2] ngrep: http:// ngrep. sourceforge. net/
[3] CIDR format: http:// en. wikipedia. org/
wiki/ Classless_Inter-Domain_Routing
[4] tcpkill:
http:// en. wikipedia. org/ wiki/ Tcpkill
ngrep: Network Grep 1 0 HaNdy adMI N tools
16
More Great tools for tHe Busy adMI N www. adMI N- MaGazI Ne. CoM
Shop the Shop shop.linuxnewmedia.com
FIND IT ON NEWSSTANDS NOW OR ORDER ONLINE:
shop. l i nuxnewmedi a. com/rpi
In case you missed
it last time...
You ordered your Raspberry Pi...
You got it to boot...what now?
The Raspberry Pi Handbook takes
you through an inspiring collection of
projects. Put your Pi to work as a:
media center
photo server
game server
hardware controller
and much more!
Discover Raspberry Pis special tools
for teaching kids about programming
and electronics, and explore advanced
techniques for controlling Arduino
systems and coding GPIO interrupts.
WATCH YOUR NEWSSTANDS FOR
THE ONLY RASPBERRY PI REFERENCE
YOULL EVER NEED!
LNM_Special_RaspPi_1-1.indd 1 7/17/14 2:32:54 PM
In 2005, Canadian Mark Lord
developed the small hdparm util-
ity [1] to test Linux drivers for IDE
hard drives. Since then, the pro-
gram has developed into a valu-
able tool for diagnosis and tuning
of hard drives.
For example, it tests the speed of
hard drives and solid state disks,
puts devices to sleep, and turns
the energy-saving mode on or off.
With modern devices, it can acti-
vate the acoustic mode and clean
up SSDs.
Before your first experiments
with hdparm, you should read the
safety concerns in the Warning!
box.
Need for Communication
All reasonably new distributions
already include hdparm in the
basic installation. You only need to
open a terminal and call up
hdparm -I /dev/sda | more
as administrator (Figure 1).
The tool will deliver all available
data about the chosen drive in
this case, the first hard disk sda.
The | more option makes sure the
large amount of information does
not simply rush unread through
the terminal.
Hdparm accepts any device as
mass storage that is connected to
an (E)IDE, SATA, or SAS interface,
including, therefore, DVD drives
and SSDs. USB-to-IDE adapters
often cause problems because they
do not transmit the (complete)
ATA or ATAPI commands to the
drive.
The information that hdparm
delivers is dependent on the de-
vice. The designation and firm-
ware version number are always
listed at the top under Model
Number and Firmware Revision.
Owners of an SSD especially can
find out quickly whether they are
running the current firmware
version.
On newer hard disks, you
should check whether Native Com-
mand Queuing (NCQ) is to be
found under Commands/ features.
This technology makes it possible
for the hard disk to sort queries
from the system in such a way that
Hdparm is the tool to use when it comes to tuning your hard disk or DVD drive, but
it can also measure read speed, deliver valuable information about the device,
change important drive settings, and even erase SSDs securely. By Tim Schrmann
L
e
a
d

I
m
a
g
e

A
m
y

W
a
l
t
e
r
s
,

1
2
3
R
F
.
c
o
m
Retrieving and setting hard drive parameters with hdparm
Disk Inspector
Warning
Hdparm manipulates a drive directly,
which is why using it can easily lead to
loss of data and, in the worst case, to a
defect on the device. Beyond that, the
programs documentation points out
that many of its functions are experi-
mental or dangerous. Therefore, before
you work with the program, you should
always make a backup of the complete
drive. Furthermore, you should only use
functions whose actions you fully under-
stand. The publisher and author of this
article accept no liability for damages
or loss of data.
hdparm Drive Utility 1 0 More Terri fi c ADMi n ToolS!
18
More GreAT ToolS for THe BUSy ADMi n WWW. ADMi n- MAGAzi ne. coM
the heads take the shortest possi-
ble path. SSDs, on the other hand,
distribute write accesses more effi-
ciently across storage blocks. Ide-
ally, this leads to an increase in
speed.
If NCQ is deactivated, check the
BIOS to find out whether the drive
is running in AHCI mode, which is
also necessary for other functions
such as energy management.
Speedometer
To determine how fast a drive de-
livers data, you can use the
hdparm -t /dev/sda
command. After a few seconds,
the data transfer rate appears (in
megabytes per second, MBps).
The small program reads directly
from the drive for a while regard-
less of the filesystem. The speed
measured is therefore somewhat
faster than in actual practice. To
receive an untainted result, no
other programs should be running
during the measurement, and
enough main memory should be
free.
Repeat the measurement at least
three times and then calculate the
average value. For a current
model, the result should reach at
least 80 MBps (Figure 2).
The Linux kernel deposits the
data retrieved from the hard drive
into a buffer. To determine the
speed of the unadorned drive, you
can use the
hdparm -t --direct /dev/sda
command. Hdparm then reads the
data directly from the disk. The
values thus measured will be
somewhat slower than without
--direct, but at least you can see
the pure transmission rate of the
disk (Figure 3).
Hdparm always reads the data
from the beginning of the storage
device. Hard disks, however, tend
to deliver data somewhat more
slowly from the outer areas of
magnetic disks; therefore, hdparm
lets you set an offset (from soft-
ware version 9.29 on):
Figure 3: Without the buffer, transmission rate drops dramatically. At the middle of the 320GB hard
drive, more speed losses are seen.
Figure 1: Hdparm lists the hardware properties of a six-year-old hard disk with a 320GB capacity.
Figure 2: This SATA hard drive achieved an average read speed of 80.48 MBps.
1 0 More Terri fi c ADMi n ToolS! hdparm Drive Utility
19
More GreAT ToolS for THe BUSy ADMi n WWW. ADMi n- MAGAzi ne. coM
hdparm -t --direct --offset 500 /dev/sda
The 500 stands for the number of
gigabytes to skip. On a 1TB hard
disk, the command above would
therefore deliver data from the
middle of the disk. As Figure 3
shows, reading speed drops quite
markedly in the outer areas of a
hard disk.
All the speed tests introduced
here only give a first impression of
possible problems and bottle-
necks. For a complete benchmark,
therefore, you would also need to
determine the write speed, for ex-
ample.
Faster, Faster
Some drive properties can be
changed while the device is in op-
eration; for example, most drives
allow you to turn power manage-
ment on and off. Which functions
hdparm can change and activate
on a hard drive can be called with
hdparm -I /dev/sda
and are found under Commands/
features (Figure 1). All functions
found there and marked with an
asterisk are currently active, and
hdparm can use the rest or at least
activate them.
To speed up data transmission, a
hard disk usually reads several
sectors at the same time. How
many it can deliver at the same
time is revealed by
hdparm -I /dev/sda
and is listed after R/ W multiple
sector transfer: Max =. This value
should also be found in the same
line after Current =. If that is not
the case, you can increase the
value with:
hdparm -m16 /dev/sda
This instructs the hard drive al-
ways to deliver 16 sectors at once.
Curiously, some hard drives run
slower with higher values: The hd-
parm man page mentions primarily
older Caviar drives from Western
Digital. In such cases, you should re-
duce the number of sectors again or
even turn off the function with:
hdparm -m0 /dev/sda
Beyond this, modern drives can even
retrieve a few sectors in advance
(read ahead). To define how
many, use the -a switch (Figure 4,
top), for example:
hdparm -a256 /dev/sda
Here, the drive will read in advance
the 256 sectors that are most likely
to be requested next. Higher values
speed up the reading of large files
at the cost, however, that reading
smaller ones takes longer. The cur-
rent setting is shown with
hdparm -a /dev/sda
Beyond that, many drives also pos-
sess a built-in, additional read-ahead
function. As a rule, therefore, you
can leave the setting at the default
value. How fast queries from the op-
erating system reach the hard drive
controller can be called with
hdparm -c /dev/sda
The value should be 32-bit; you can
force this value with the -c3 switch.
Full Speed
Ahead
Many modern hard
drives allow you to
slow down the head
movement. Although
doing so will in-
crease access times,
it will also reduce the noise level.
To see if your own hard drive of-
fers this acoustic mode, use:
hdparm -M /dev/sda
If a number follows the equal sign,
as shown in Figure 4 (bottom), the
drive can be put into a quiet mode
with:
hdparm -M 128 /dev/sda
To reach the highest speed, use the
maximum value:
hdparm -M 254 /dev/sda
Values between 128 and 254 are al-
lowed, resulting in a trade-off be-
tween noise level and speed. Inci-
dentally, your Linux kernel must
also support acoustic manage-
ment, which should be the case
for all current major distributions.
Some CD and DVD drives turn
out to be more like turbines: Their
high-speed rotation can hinder
audio/ video enjoyment. The
hdparm -E 4 /dev/sr0
command will provide relief. The
parameter 4 determines speed, and
/dev/sr0 specifies the DVD drive.
This example slows drive reading
speed ninefold.
Write-Back Caching
With write-back caching, the hard
drive first stores the data to be
Figure 4: Here, the read-ahead is set to 256, and acoustic
management is currently deactivated.
hdparm Drive Utility 1 0 More Terri fi c ADMi n ToolS!
20
More GreAT ToolS for THe BUSy ADMi n WWW. ADMi n- MAGAzi ne. coM
written in a buffer. In this way, it
can accept data much faster,
which in the end leads to a faster
write speed. The
hdparm -W /dev/sda
command shows whether write-
back caching is active with a 1
after the equals sign; otherwise,
you can activate the function with
the -W1 switch.
If hdparm will not allow this
change, you need to make sure
that write-back caching has been
activated in the BIOS. However,
this function is not recommended
for all situations: In the case of a
power outage, the data in the buf-
fer would be lost permanently.
If a program sensitive to data
loss such as a database is run-
ning on the system, you should
turn off the write-back cache with
the -W0 switch. Documentation for
the PostgreSQL database even ex-
plicitly recommends that this be
done.
Live Wire
If a hard disk or SSD doesnt have
anything to do for a certain period
of time, it automatically enters
sleep mode. This power-saving
feature can be influenced with the
-B parameter. Thus, using:
hdparm -B255 /dev/sda
would deactivate energy manage-
ment; however, not all drives
allow this.
Instead of 255, values between 1
and 254 are allowed. A higher
value means more power is used
but also promises higher perfor-
mance or speed. Values between 1
and 128 allow the drive to shut
down, whereas values from 129 to
254 forbid that from happening.
The most power can be saved
with a value of 1; the highest rate
of data transmission (I/ O perfor-
mance) is achieved with 254. You
can call up the current value with:
hdparm -B /dev/sda
The specific effect the different
values will have depends on the
drive itself. However, you should
keep in mind that too many shut-
downs are not good for desktop
hard drives: Each time it shuts off,
the drive must park the heads,
which increases wear and tear.
Consequently, you shouldnt wake
up your hard drive every two sec-
onds which always takes more
than two seconds to do.
You can set how many seconds
of idleness the hard drive should
wait before it goes to sleep with
the
hdparm -S 128 /dev/sda
switch; however, this value here is
not in seconds but a number be-
tween 1 and 253.
The hard drive multiplies this
value by another. The value cho-
sen in the example, 128, lies be-
tween 1 and 240, for which the
drive uses a factor of five. Conse-
quently, it would shut down after
640 seconds of idleness.
From 241 and up, the multiplica-
tion factor increases steadily. At
251, the waiting period has in-
creased to 5.5 hours. At 253, the
value is preset by the manufac-
turer, usually between eight and
12 hours. The value 254 is left out;
at 255, the drive will wait 21 min-
utes and 15 seconds. A value of 0
will deactivate sleep mode com-
pletely. To send the hard drive to
sleep immediately, enter:
hdparm -y /dev/sda
With a capital Y, the drive will go
into an even deeper state of sleep.
Depending on the drive, the drive
might only wake up from a deep
sleep after a reset of the whole
system.
Cleanup
SSDs track the location of the data
deposited on them independently
of the operating system. This can
lead to the curious situation that a
file has been deleted but the SSD
still has its former location marked
as occupied.
To remedy such conflicts, newer
versions of hdparm include the
wiper.sh script. Entering
wiper.sh /dev/sda
determines which blocks are being
used and which are not and re-
ports this to the SSD. However,
this script must be used with cau-
tion: The documentation warns
explicitly that data could be lost
and advises against its use with
the Btrfs filesystem.
Drives with ext2/ 3/ 4, Reiser3,
and XFS should be mounted as
read-only before using the wiper
command. It would be best to un-
mount the drive completely or
start wiper.sh from a Live system.
In any case, you should definitely
make a backup of the SSD before-
hand and use the script only in an
emergency. Incidentally, because
wiper is so dangerous, some distri-
butions do not even include it.
Secure Deletion
To achieve higher transfer rates
and spread use equally over the
storage chips, SSDs also reserve
some storage areas (wear level-
ing), so that simply formatting an
SSD will seldom delete the whole
drive. Most SSDs therefore offer a
function called secure erase, which
causes the drive to empty all its
1 0 More Terri fi c ADMi n ToolS! hdparm Drive Utility
21
More GreAT ToolS for THe BUSy ADMi n WWW. ADMi n- MAGAzi ne. coM
storage cells. This is especially
useful should you decide to give
up your used SSD.
Secure erase has two pitfalls: hd-
parm can only initiate a secure
erase when the BIOS also allows it.
Beyond that, the method is consid-
ered to be experimental. The docu-
mentation warns explicitly about
using the procedure because, in the
worst case, secure erase could
make the whole SSD unusable. If
you want to use this delete function
anyway, first call up the identifica-
tion information with:
hdparm -I /dev/sdb
Under Security, the line supported:
enhanced erase should show up
somewhere; otherwise, the SSD
wont support secure erase. Next,
turn on the security function of the
drive by (temporarily) setting a
password like 123456:
hdparm --user-master u
U
--security-set-pass 123456
U
/dev/sdb
When you call up the identifica-
tion information again, you will
now find enabled under Security.
To erase the SSD now, enter:
hdparm --user-master u
U
--security-erase 123456 /dev/sdb
In the process, hdparm also re-
moves the password. The whole
process takes a few minutes, de-
pending on the size of the SSD,
during which no feedback is given.
Afterward, when you call up the
identification information, the area
under Security should look like it
did before setting the password.
Relics
In the case of older hard drives
with an IDE connector (also called
PATA), you should take a look at
the using_dma line in the identifi-
cation output.
With the help of DMA (Direct
Memory Access) technology, the
hard drive itself deposits data di-
rectly into main memory. If the re-
spective flag is 0 (off), it will slow
down the data transfer. Over the
years, ever faster DMA standards
have been introduced; the fastest
possible can be activated with the
command:
hdparm -d1 /dev/hda
On some very old systems, how-
ever, the DMA mode can cause
problems. After activating it, you
should therefore copy a few larger
test files to the drive.
If problems arise or the drive
crashes, you can deactivate the
DMA mode again with:
hdparm -d0 /dev/hda
Incidentally, modern SATA drives
always use DMA.
While the hard drive is transfer-
ring the requested data, the rest of
the system can go about complet-
ing other tasks but only if an on
appears after unmaskirq in the
identification info output. You can
force this mode with the -u1
switch.
Lasting Values
After restarting the system, all
changes made with hdparm are
lost. To activate them perma-
nently, the respective hdparm
commands must be entered in the
start scripts.
How this is done depends on the
distribution you are running, but
usually the entry must be made in
/etc/rc.local.
Debian-based systems, on the
other hand, read the /etc/hdparm.
conf configuration file on system
startup. In it is a section for each
hard drive with the following for-
mat:
/dev/sda {
...
}
Modern Linux systems randomly
allocate device names (sda, sdb).
To assign the hdparm settings to a
specific drive permanently, use its
specific UUID:
/dev/ disk/ by-id/ ata-
U

SAMSUNG_HD103SJ_S246J1RZB00034 { }
The settings belong between the
curly braces. Each parameter has
its own name. Acoustic manage-
ment is set, for example, to the
value of 128 with the following
command:
acoustic_management = 128
Which name belongs to which hd-
parm parameter is revealed by the
comments at the top of the file.
Conclusions
Hdparm also includes many other
parameters that can be quite dan-
gerous to use. For example, many
SSDs can be protected with a
password, which can lead to data
loss in some situations. Its not a
coincidence that the man page
(man hdparm) warns about these
dangers.
Incidentally, hdparm is only one
useful tool among many; for exam-
ple, the smartmontools can deter-
mine the health status of a hard
drive [2]. n
Info
[1] hdparm: http:// hdparm. sourceforge. net
[2] smartmontools: http:// sourceforge.
net/ apps/ trac/ smartmontools/ wiki
hdparm Drive Utility 1 0 More Terri fi c ADMi n ToolS!
22
More GreAT ToolS for THe BUSy ADMi n WWW. ADMi n- MAGAzi ne. coM
shop.linuxpromagazine.com/trial
MOBILE USERS
search for us today at your digital newsstand!
shop.linuxpromagazine.com/trial
Only a swipe away!
Download our convenient
digital editions for your iPad,
iPhone, or Android device.
Visit our apps page for more information: www.medialinx-shop.com/apps
GOOGLE PLAY MAGAZINES
ADMIN Magazine Linux Pro Magazine Ubuntu User Raspberry Pi Geek
APPLE NEWSSTAND
ADMIN Magazine Linux Pro Magazine Ubuntu User Raspberry Pi Geek
LNM_Apple_and_Google_1-1_New.indd 1 7/11/14 4:28:44 PM
Learn the latest
techniques for better:
network security
system management
troubleshooting
performance tuning
virtualization
cloud computing
on Windows, Linux,
Solaris, and popular
varieties of Unix.
Each issue delivers
technical solutions
to the real-world
problems you face
every day.
Real SolutionS
foR Real netwoRkS
FREE
CD or DVD
in Every Issue!
O
R
D
E
R
O
N
L
IN
E
A
T
: s
h
o
p
.lin
u
x
n
e
w
m
e
d
ia
.c
o
m
6
is
s
u
e
s
p
e
r
y
e
a
r
!
LNM_Admin_sub_1-1.indd 1 6/5/14 2:31:38 PM