Sie sind auf Seite 1von 10

System Administration Toolkit: Monitoring disk space and usage

Page 1 of 10

System Administration Toolkit: Monitoring disk space and usage


Level: Intermediate Martin Brown (mc@mcslp.com), Freelance Writer, Consultant 13 Jun 2006 Look at methods for determining disk usage across multiple UNIX systems and how to create a simple warning system to alert you of potential problems. Keeping an eye on your file systems and ensuring they don't fill up is a trivial, but vital process in the day-to-day management of your UNIX systems. In this article, you'll look at methods for keeping an eye on disk space, discovering which files, users, or applications are using up the most space, and how to make use of quotas and other solutions to find the information you need.

About this series


The typical UNIX administrator has a key range of utilities, tricks, and systems he or she uses regularly to aid in the process of administration. There are key utilities, command-line chains, and scripts that are used to simplify different processes. Some of these tools come with the operating system, but a majority of the tricks come through years of experience and a desire to ease the system administrator's life. The focus of this series is on getting the most from the available tools across a range of different UNIX environments, including methods of simplifying administration in a heterogeneous environment.

Getting started: using df


Monitoring disk space is a vital part of your job as a UNIX administrator. This article gives you the tools you need to be successful, including the use of df, du, find, and even the use of quotas. Let's get started by taking a look at how useful df can be. Use of the df tool can be a bit like a nervous twitch for many administrators, as it gives you a snapshot view of the storage space used and available across all of your file systems through a single command. Depending on your UNIX environment, the default output from df might contain a range of different information. Most modern df variants show disk space, usage and availability, and usually the mountpoint (and sometimes the device). For example, Mac OS X, based on BSD, shows the information in Listing 1. Listing 1. Default disk space information for Mac OS X
Filesystem /dev/disk0s2 devfs fdesc <volfs> /dev/disk0s3 512-blocks Used Avail Capacity 268435456 119741344 148182112 45% 195 195 0 100% 2 2 0 100% 1024 1024 0 100% 43474520 7251448 36223072 17% Mounted on / /dev /dev /.vol /Volumes/Untitled

Note that the size is displayed in 512 byte blocks, not kilobytes. Nearly all traditional UNIX operating systems report the size in 512 byte blocks rather than kilobytes by default. To get the information in 1K blocks, use the -k command-line option, as shown in Listing 2.

http://www.ibm.com/developerworks/aix/library/au-satdiskmon.html

7/19/2007

System Administration Toolkit: Monitoring disk space and usage


Listing 2. Using the -k option with the df command
$ df -k Filesystem /dev/disk0s2 devfs fdesc <volfs> /dev/disk0s3 1K-blocks 134217728 97 1 512 21737260 Used 59870704 97 1 512 3625724 Avail Capacity 74091024 45% 0 100% 0 100% 0 100% 18111536 17% Mounted on / /dev /dev /.vol /Volumes/Untitled

Page 2 of 10

Some variants might also support alternative block sizes, such as megabytes (use -m) and gigabytes (use -g), as demonstrated in Listing 3. Listing 3. Using the -g option with the df command
$ df -g Filesystem /dev/disk0s2 devfs fdesc <volfs> /dev/disk0s3 1G-blocks Used Avail Capacity 128 57 70 45% 0 0 0 100% 0 0 0 100% 0 0 0 100% 20 3 17 17% Mounted on / /dev /dev /.vol /Volumes/Untitled

Obviously, as you increase the block size shown, the level of detail in the information starts to reduce, but using alternative sizes can be a useful way to quickly monitor disks, particularly very large ones. Other versions of the df tool (notably Solaris) report the bare block and file availability information by default. Most file systems have a limited number of files that can be stored (the number is so large that you are unlikely to reach the limit), so it is possible to fill up a system with files, but still have disk capacity and be unable to store any more files. You can see a sample of the default output of Solaris in Listing 4. Listing 4. Default output of Solaris
$ df / /devices /system/contract /proc /etc/mnttab /etc/svc/volatile /system/object /usr /dev/fd /var /tmp /var/run /export/home (/dev/dsk/c0t0d0s0 (/devices (ctfs (proc (mnttab (swap (objfs (/dev/dsk/c0t0d0s3 (fd (/dev/dsk/c0t0d0s4 (swap (swap (/dev/dsk/c0t0d0s7 ):14877208 ): 0 ): 0 ): 0 ): 0 ): 5737216 ): 0 ): 9076010 ): 0 ): 8110796 ): 5737216 ): 5737216 ):69362510 blocks 914042 files blocks 0 files blocks 2147483618 files blocks 16109 files blocks 0 files blocks 147177 files blocks 2147483532 files blocks 863695 files blocks 0 files blocks 483714 files blocks 147177 files blocks 147177 files blocks 4272812 files

Using the -k command-line option switches the display to one similar to that shown with earlier examples, where the data is summarized into a more easily readable format (see Listing 5). Listing 5. Using the -k option
$ df -k Filesystem /dev/dsk/c0t0d0s0 /devices ctfs proc mnttab swap objfs kbytes 7644629 0 0 0 0 2868600 0 used avail capacity 206026 7362157 3% 0 0 0% 0 0 0% 0 0 0% 0 0 0% 1016 2867584 1% 0 0 0% Mounted on / /devices /system/contract /proc /etc/mnttab /etc/svc/volatile /system/object

http://www.ibm.com/developerworks/aix/library/au-satdiskmon.html

7/19/2007

System Administration Toolkit: Monitoring disk space and usage


/dev/dsk/c0t0d0s3 fd /dev/dsk/c0t0d0s4 swap swap /dev/dsk/c0t0d0s7 8261237 3723232 4455393 0 0 0 4130238 74849 4014087 2867584 0 2867584 2867624 40 2867584 35611388 930133 34325142 46% 0% 2% 0% 1% 3% /usr /dev/fd /var /tmp /var/run /export/home

Page 3 of 10

All variants of df accept a directory or path, and then the information about that disk space for the file system that contains that path is displayed. For example, the following shows you how to get the space on the root file system:
$ df -k / Filesystem /dev/dsk/c0t0d0s0 kbytes 7644629 used avail capacity 206026 7362157 3% Mounted on /

Or, the following uses df to show the space on a user's home directory:
$ df -k ~mc Filesystem /dev/dsk/c0t0d0s7 kbytes 35611388 used avail capacity 930133 34325142 3% Mounted on /export/home

Using the df output


The df tool is actually short for disk free, and that is precisely what the tool shows. If you look at some sample output again, you can extract the important individual elements:
Filesystem /dev/dsk/c0t0d0s7 kbytes 35611388 used avail capacity 930133 34325142 3% Mounted on /export/home

The first column, usually with a header according to the block size data, shows the total size of the disk. The used column shows the number of blocks that have been used on the file system or device. The avail column shows the number of unused (available) blocks on the file system. The capacity column gives a percentage calculation of how much disk space out of the total size has actually been used. On its own, this information can give a quick indication of the space available. By default, UNIX creates file systems that can be filled up to 90 percent capacity with data by users. The remaining 10 percent is reserved by the system for use by root to perform any emergency maintenance. If a user tries to add or create a file that goes beyond that level, an error is returned (out of file system space). That emergency maintenance space is there so it can be used to recover from an overfull disk. For example, as an administrator, you could use the space to create a quick backup, or to compress existing data without having to use a secondary file system for the purpose. You can adjust the amount of free space that is reserved (using the tunefs tool), or when creating the file system. This is vital for large disks, where a 10 percent buffer could equal many gigabytes of potential space. It is generally a good idea to reserve at least one percent to give you a minimal buffer before you completely run out of space. Before you get that far, especially if a lot of your disk space is being used, you should determine who is using all of that space.

Using du
http://www.ibm.com/developerworks/aix/library/au-satdiskmon.html 7/19/2007

System Administration Toolkit: Monitoring disk space and usage

Page 4 of 10

The du command, instead of showing disk free space, shows disk usage information. The du tool is used to determine the disk usage for files and directories. To use it, change to a directory and run the tool (see Listing 6). Listing 6. du command
$ cd /var $ du 16 ./lost+found 4 ./sadm/install/admin 22 ./sadm/install/logs 28448 ./sadm/install 4 ./sadm/pkg/SUNWocfd/install 4 ./sadm/pkg/SUNWocfd/save/pspool/SUNWocfd/install 16 ./sadm/pkg/SUNWocfd/save/pspool/SUNWocfd 18 ./sadm/pkg/SUNWocfd/save/pspool 20 ./sadm/pkg/SUNWocfd/save 28 ./sadm/pkg/SUNWocfd 4 ./sadm/pkg/SUNWcsu/inst ...

The output, shown in Listing 6 above, has been trimmed. By default, du shows the file usage for every file and directory beneath the current, or specified, directory. The value given is the size of the file in the default block size for that system, just as that used with df. This might, or might not, be 1K; you can force the display to 1K blocks using the -k command-line option. You will probably want to summarize the information by the top-level file, or directory, that you are examining. Use the -s option to switch on the summary view. Here's a summarized version, shown in the /var directory, of a Solaris installation:
$ du -sk 70818 .

Note that it shows the summary information for the current directory (.). To get summary information for all of the files and directories, use the * wildcard (see Listing 7). Listing 7. Using the * wildcard to get summary information
$ du -sk * 382 adm 950 apache 683 apache2 6837 appserver 1 audit 162 cache 3 cc-ccr 2 crash 4 cron 31 dmi 22 dt 6 fm 2 imq 1 inet 3 krb5 4 ld 1 ldap 937 lib 6 log 8 lost+found 2 lp 2 mail 1 mysql 1 news 3 nfs

http://www.ibm.com/developerworks/aix/library/au-satdiskmon.html

7/19/2007

System Administration Toolkit: Monitoring disk space and usage


38 2 10034 1 96 49687 15 3 2 131 39 4 663 14 10 24 nis ntp opt preserve run sadm saf samba sma_snmp snmp spool statmon svc tmp uucp yp

Page 5 of 10

Be careful when using that tool, particularly in user directories, as the information shown will not include hidden files -- in other words, those files and directories with a single period prefix. You might want to use the following line in a user directory to get all of the summary information:
$ du -sk * .[a-zA-Z0-9]*

One final very useful option for du is -d, which prevents du from following file system boundaries. For example, to determine the disk usage on the root file system but not any other file systems, you would use the -d option:
$ du -dsk /

Some systems do not have this option but do have the -x command-line option, which only includes the file usage for files on the same device or file system as the path you specify.

Finding disk usage for a specific user


To find the disk space being used by a single user, you need to combine du with the find command to only report disk usage for a specific user:
$ find . -user mc -type f -exec du -k {} \;

The -user option allows you to specify that find will only report files that are owned by the specified user. The -type option forces find to only return the path of items of a specific type (in this case, files); this prevents du from including directories, which might be owned by one user, but contain files for many users. Then, for each found path, the du command is executed to report the disk usage. To get summary information, in other words, the total space used by a specific user, you can use awk to total the information and print out the final value:
$ find . -user mc -type f -exec du -k {} \;|awk '{ s = s+$1 } END { print "Total used: ",s }' Total used: 123721

You use the same principle with groups using the -group option to find:
$ find . -group mcslp -type f -exec du -k {} \;|awk '{ s = s+$1 } END { print

http://www.ibm.com/developerworks/aix/library/au-satdiskmon.html

7/19/2007

System Administration Toolkit: Monitoring disk space and usage


"Total used: ",s }' Total used: 542485

Page 6 of 10

There is, however, an easier way, if you enable disk quotas.

Using quotas
The quota system automatically monitors the disk usage for users on a file system by a file system basis. The quota environment not only enables you to monitor disk usage, but you can also set limits for usage that provide a warning and, more explicitly, prevent users from using disk space over their assigned quota value. The lower limit (warning) is called the soft limit and the higher limit (which prevents creation of files over that level) is called the hard limit. Some systems might also allow you to control the number of files owned by each user. The precise method required to enable quotas on your machine is dependent on the operating system you are using. Most UNIX systems include quota support by default. Linux systems might need to build a new kernel that includes quota support. Most, however, use a single file on each file system, usually called quotas, into which you place the limits for each user. To enable quotas, you should first create this file and ensure that the quotas file is only editable by root:
$ touch /export/home/quotas $ chmod 600 /export/home/quotas

Then switch on quotas using the quotaon command:


$ quotaon /export/home

Finally, you must edit the appropriate quota for each user. You can do this by using the edquota command, specifying the user:
$ edquota mc

Your default editor (or vi, if you haven't set an alternative) will be opened with a simple form for setting quota values. You can see below that quotas have been enabled on the users home directory file system with a soft limit of 200,000KB and a hard limit of 400,000KB. The settings of zero for file limits indicate that there is no set limit and the user can set and create as many files as they like.
fs /export/home blocks (soft = 200000, hard = 400000) inodes (soft = 0, hard = 0)

If you have more file systems with quotas, then you will need to configure more rows for the file systems. If you want to configure the quota for multiple users, first set the quota for a single user, then use the -p command-line option to edquota. This uses the specified user as the basis for the new users. For example, to set the quota using the settings for mc, for usernames slp, tw, and sh, do the following:
$ edquota -p mc slp tw sh

http://www.ibm.com/developerworks/aix/library/au-satdiskmon.html

7/19/2007

System Administration Toolkit: Monitoring disk space and usage

Page 7 of 10

File limit warnings with quotas


When the user creates a file that goes over their soft limit, they will get the following warning:
quota_ufs: Warning: over disk limit (pid 1738, uid 101, inum 94, fs /export/home)

Note that the user is given seven days to correct the problem; you can alter the duration by using edquota t. If the user tries to create a file that pushes them over the hard limit, the system terminates the write process and truncates the file to the limit:
$ mkfile 210000k overlimit quota_ufs: over hard disk limit (pid 1843, uid 101, inum 130, fs /export/home) overlimit: initialized 191873024 of 215040000 bytes: Disc quota exceeded

Any user can check their own quota limits and disk usage by running the quota command:
$ quota Over disk quota on /export/home, remove 199993K within 7.0 days

Quota administration
The systems administrator can check the quota of any user using the quota command; you should use the -v command-line option to provide a full report on the file system, usage, and limit information (see Listing 8). Listing 8. Using the -v option
$ quota -v mc Disk quotas for mc (uid 101): Filesystem usage quota limit timeleft /export/home 399993 200000 400000 timeleft 6.9 days files 151 quota 0 limit 0

To get a report about disk and quota usage use on a file system basis detailing all users, use the repquota command , specifying the file system to report (see Listing 9). Listing 9. Using the repquote command
$ repquota -v /export/home /dev/dsk/c0t0d0s7 (/export/home): Block limits User used soft hard timeleft mc +- 399993 200000 400000 6.9 days File limits soft hard timeleft 0 0

used 151

To ensure that the quota information is kept up to date, you should use the quotacheck command. This validates the file storage data with the quota information used to report quota information. You should run this automatically using cron -- once a day should be fine (the process is comparatively time consuming).

http://www.ibm.com/developerworks/aix/library/au-satdiskmon.html

7/19/2007

System Administration Toolkit: Monitoring disk space and usage

Page 8 of 10

Quota tricks and traps


The quota system provides the best combination of monitoring and automatic administration, but you should be careful about the file systems on which you enable quotas and the limits that you set, as they might actually end up hindering the work of your users rather than controlling their disk usage. For example, programmers might need significantly more space to actually build an application than you might have configured for their use. You can get around this issue by providing an unrestricted area for compilation (by setting an alternative temporary directory) while retaining quotas on their home directories, or by setting appropriate soft limits while setting the hard limits very high -- maybe even the maximum size of the file system. The result should be a warning when the soft limit is reached, but no enforcement of the hard limit. Users should still be able to build and create temporary files but, as they are naturally deleted during the build process, they should never be prevented from creating the files they need. Quotas can also be used to help monitor and alert you to usage of special user accounts. I have, in the past, used quotas to monitor Web user accounts, so called nobody accounts and others, to ensure that they do not have the ability to write files to file systems they should not otherwise have access to. To do this, set the hard limit to 1KB for the file system and user you want to protect.

Automatic monitoring
Monitoring disk space usage by hand is fine, but you don't want to be constantly running df (or even du) to determine the disk space being used, or that is available. You can automate the process and automatically send the administrator (or an administration group) an e-mail when the available space gets below a certain level. The script in Listing 10 monitors disk space, and you can set the warning (warninglimit) and emergency (lowlimit) limits, as well as the list of file systems that are checked. Listing 10. Monitoring disk space
#!/bin/sh warninglimit=500000 lowlimit=250000 filesystems="/export/data /export/home /" for fs in $filesystems do size=`df -k $fs|grep $fs|awk '{ print $4; }'` if [ $size -le $lowlimit ] then mailx -s "URGENT: Low disk space for $fs ($size)" break fi if [ $size -le $warninglimit ] then mailx -s "WARNING: Low disk space for $fs ($size)" fi done

The key is the line that extracts the amount of free disk space for each file system:
size=`df -k $fs|grep $fs|awk '{ print $4; }'`

http://www.ibm.com/developerworks/aix/library/au-satdiskmon.html

7/19/2007

System Administration Toolkit: Monitoring disk space and usage

Page 9 of 10

The script uses df, extracts only the line you want with grep, and then uses awk to extract the fourth column of data, which is the amount of free space. You can then check the free space against the warning/low limits and generate a suitable error. To prevent the system from raising both a low and a warning error, the script checks for a low warning first and uses break to skip to the next file system in the loop before the warning test is tried.

Summary
Monitoring disk space is a vital part of your job as an administrator; run out of disk space and you could seriously impair the work of your users and, in severe cases, lose data, or crash systems as they run out of the vital disk space required to operate. You can determine the free space and usage for entire file systems using df, but this only gives you part of the picture. To determine where that disk space is being used, you need to use the du tool to study individual directories. Through the use of find, you can even use du to find out how much space specific users are using. For more automated user-level disk usage and control, quotas are a much better option.

Resources
Learn
l l l

System Administration Toolkit: Check out other parts in this series. developerWorks Podcasts: Listen to interesting interviews and discussions for software developers. IBM Redbooks: IBM has a Redbook on AIX for Sun Solaris Administrators that contains useful information on the differences between AIX from the perspective of Solaris system administrators. HP-UX 11i System Administration Handbook and Toolkit: Marty Poniatowski's (Prentice Hall PTR, ISBN: 0-13-060081-3) book contains information not only about HP-UX, but also compares the HPUX commands and generated output to other UNIX operating systems. AIX and UNIX articles: Check out other articles written by Martin Brown. Search the AIX and UNIX library by topic: System administration Application development Performance Porting Security Tips Tools and utilities Java technology Linux Open source AIX and UNIX: The AIX and UNIX developerWorks zone provides a wealth of information relating to all aspects of AIX systems administration and expanding your UNIX skills.

l l

http://www.ibm.com/developerworks/aix/library/au-satdiskmon.html

7/19/2007

System Administration Toolkit: Monitoring disk space and usage


l l l l

Page 10 of 10

New to AIX and UNIX: Visit the New to AIX and UNIX page to learn more about AIX and UNIX. AIX 5L Wiki: A collaborative environment for technical information related to AIX. Safari bookstore: Visit this e-reference library to find specific technical resources. developerWorks technical events and webcasts: Stay current with developerWorks technical events and webcasts. Podcasts: Tune in and catch up with IBM technical experts.

Get products and technologies l IBM trial software: Build your next development project with software for download directly from developerWorks.

Discuss l Participate in the discussion forum.


l l

Participate in the developerWorks blogs and get involved in the developerWorks community. Participate in the AIX and UNIX forums: AIX 5L -- technical forum AIX for Developers Forum Cluster Systems Management IBM Support Assistant Performance Tools -- technical Virtualization -- technical More AIX and UNIX forums

About the author


Martin Brown has been a professional writer for more than seven years. He is the author of numerous books and articles across a range of topics. His expertise spans myriad development languages and platforms -- Perl, Python, Java, JavaScript, Basic, Pascal, Modula-2, C, C++, Rebol, Gawk, Shellscript, Windows, Solaris, Linux, BeOS, Mac OS X and more -- as well as Web programming, systems management, and integration. He is a Subject Matter Expert (SME) for Microsoft and regular contributor to ServerWatch.com, LinuxToday.com, and IBM developerWorks. He is also a regular blogger at Computerworld, The Apple Blog, and other sites. You can contact him through his Web site.

IBM, AIX, and AIX 5L are registered trademarks of International Business Machines Corporation in the United States, other countries, or both. Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both. Linux is a trademark of Linus Torvalds in the United States, other countries, or both. Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both. UNIX is a registered trademark of The Open Group in the United States and other countries. Other company, product, or service names may be trademarks or service marks of others.

http://www.ibm.com/developerworks/aix/library/au-satdiskmon.html

7/19/2007