Beruflich Dokumente
Kultur Dokumente
2012 Consumer Security Products Performance Benchmarks (Edition 1) M. Baquiran, D. Wren PassMark Software 17 August 2011 1 antivirus_12-performance-testing-ed1.docx
PassMark Software
Table of Contents
TABLE OF CONTENTS......................................................................................................................................... 2 REVISION HISTORY............................................................................................................................................ 4 REFERENCES...................................................................................................................................................... 4 EXECUTIVE SUMMARY ...................................................................................................................................... 5 OVERALL SCORE ................................................................................................................................................ 6 PRODUCTS AND VERSIONS ............................................................................................................................... 7 INTERNET SECURITY SOFTWARE .................................................................................................................................. 7 ANTIVIRUS SOFTWARE .............................................................................................................................................. 7 PERFORMANCE METRICS SUMMARY ................................................................................................................ 8 INTERNET SECURITY SOFTWARE TEST RESULTS ............................................................................................ 11 BENCHMARK 1 BOOT TIME ................................................................................................................................... 11 BENCHMARK 2 SCAN TIME ................................................................................................................................... 11 BENCHMARK 3 USER INTERFACE LAUNCH TIME ........................................................................................................ 12 BENCHMARK 4 MEMORY USAGE DURING SYSTEM IDLE .............................................................................................. 12 BENCHMARK 5 BROWSE TIME ............................................................................................................................... 13 BENCHMARK 6 INTERNET EXPLORER LAUNCH TIME ................................................................................................... 13 BENCHMARK 7 INSTALLATION TIME ........................................................................................................................ 14 BENCHMARK 8 INSTALLATION SIZE ......................................................................................................................... 14 BENCHMARK 9 REGISTRY KEYS ADDED .................................................................................................................... 15 BENCHMARK 10 FILE COPY, MOVE AND DELETE ....................................................................................................... 15 BENCHMARK 11 INSTALLATION OF THIRD PARTY APPLICATIONS ................................................................................... 16 BENCHMARK 12 NETWORK THROUGHPUT ............................................................................................................... 16 BENCHMARK 13 FILE FORMAT CONVERSION ............................................................................................................ 17 BENCHMARK 14 FILE COMPRESSION AND DECOMPRESSION ........................................................................................ 17 BENCHMARK 15 FILE WRITE, OPEN AND CLOSE ........................................................................................................ 18 BENCHMARK 16 PE SCAN TIME ............................................................................................................................. 18 BENCHMARK 17 FILE COPY DISK TO DISK ................................................................................................................ 19 ANTIVIRUS SOFTWARE TEST RESULTS .......................................................................................................... 20 BENCHMARK 1 BOOT TIME ................................................................................................................................... 20 BENCHMARK 2 SCAN TIME ................................................................................................................................... 20 BENCHMARK 3 USER INTERFACE LAUNCH TIME ........................................................................................................ 21 BENCHMARK 4 MEMORY USAGE DURING SYSTEM IDLE .............................................................................................. 21 BENCHMARK 5 BROWSE TIME ............................................................................................................................... 22 BENCHMARK 6 INTERNET EXPLORER LAUNCH TIME ................................................................................................... 22 BENCHMARK 7 INSTALLATION TIME ........................................................................................................................ 23 BENCHMARK 8 INSTALLATION SIZE ......................................................................................................................... 23 BENCHMARK 9 REGISTRY KEYS ADDED .................................................................................................................... 24 BENCHMARK 10 FILE COPY, MOVE AND DELETE ....................................................................................................... 24 BENCHMARK 11 INSTALLATION OF THIRD PARTY APPLICATIONS ................................................................................... 25 BENCHMARK 12 NETWORK THROUGHPUT ............................................................................................................... 25 Performance Benchmark Report 1 Page 2 of 40 8 September 2011
PassMark Software
BENCHMARK 13 FILE FORMAT CONVERSION ............................................................................................................ 26 BENCHMARK 14 FILE COMPRESSION AND DECOMPRESSION ........................................................................................ 26 BENCHMARK 15 FILE WRITE, OPEN AND CLOSE ........................................................................................................ 27 BENCHMARK 16 PE SCAN TIME ............................................................................................................................. 27 BENCHMARK 17 FILE COPY DISK TO DISK ................................................................................................................ 28 DISCLAIMER AND DISCLOSURE ....................................................................................................................... 29 CONTACT DETAILS .......................................................................................................................................... 29 APPENDIX 1 TEST ENVIRONMENT ................................................................................................................ 30 APPENDIX 2 METHODOLOGY DESCRIPTION ................................................................................................. 31
PassMark Software
Revision History
Rev Revision History Initial version of this report, includes new results for Norton Internet Security 2012, G Data Internet Security 2012, Kaspersky Internet Security 2011, Panda Internet Security 2012, Avast! Internet Security 6, Norton AntiVirus 2012, G Data AntiVirus 2012, Kaspersky Anti-Virus 2012, Panda Antivirus 2012 and Avast! Pro Antivirus 6. Date Report 1 11 August 2011
References
Ref # Document Author O. Warner, The PC Spy Date
2001-2009
PassMark Software
Executive Summary
PassMark Software conducted objective performance testing on five (5) Internet Security software products, five (5) AntiVirus software products on Windows 7 Ultimate Edition (64-bit) SP1 between April 2011 and August 2011. This report presents our results and findings as a result of performance benchmark testing conducted for these consumer security products. Subsequent editions of this report will include new products released for 2012 as they are made available. For more details on which versions were tested, please see the section Products and Versions. Testing was performed on all products using seventeen (17) performance metrics. These performance metrics are as follows: Boot Time; Scan Time; User Interface Launch Time; Memory Usage during System Idle; Browse Time; Internet Explorer Launch Time; Installation Time; Installation Size; Registry Keys Added; File Copy, Move and Delete; Installation of Third Party Applications; Network Throughput (previously named Binary Download Test) File Format Conversion; File Compression and Decompression; File Write, Open and Close; PE Scan Time and; File Copy Disk to Disk .
PassMark Software
Overall Score
PassMark Software assigned every product a score depending on its ranking in each metric compared to other products in the same category.
Product Name Norton Internet Security 2012 Avast! Internet Security 2012 Kaspersky Internet Security 2012 G Data Internet Security 2012 Panda Internet Security 2012
Overall Score 66 53 50 48 38
Antivirus Software
In the following table the highest possible score attainable is 85; in a hypothetical situation where a product has attained first place in all 17 metrics. Antivirus products have been ranked by their overall scores:
Product Name Norton AntiVirus 2012 Avast! Pro Antivirus 6 Kaspersky Anti-Virus 2012 G Data AntiVirus 2012 Panda AntiVirus 2012
Overall Score 71 54 53 40 37
PassMark Software
Antivirus Software
Manufacturer Symantec Corp G Data Software AG Kaspersky Lab Panda Security AVAST Software a.s. Product Name Norton Antivirus 2012 G Data Antivirus 2012 Kaspersky Antivirus 2012 Panda Antivirus Pro 2012 Avast! Pro Antivirus 6 Release Year 2011 2011 2011 2011 2011 Product Version 19.1.0.21 22.0.2.25 12.0.0.374 11.00.00 6.0.1203 Date Tested Aug 2011 Jun 2011 Jun 2011 Jul 2011 Aug 2011
PassMark Software
PassMark Software
PassMark Software
PassMark Software
28.9
31.7
34.1
2012 Average
34.5
37.0
38.2
0s
5s
10 s
15 s
20 s
25 s
30 s
35 s
40 s
45 s
G Data Internet Security 2012 Norton Internet Security 2012 Avast! Internet Security 6 2012 Average Kaspersky Internet Security 2012 Panda Internet Security 2012 0s 10 s
22.5
23.3
36.8
40.8
45.6
75.8
20 s
30 s
40 s
50 s
60 s
70 s
80 s
PassMark Software
241
305
395
2012 Average
714
992
1637
0s
16.93
24.04
40.30
2012 Average
44.58
55.03
86.60
0 MB 10 MB 20 MB 30 MB 40 MB 50 MB 60 MB 70 MB 80 MB 90 MB 100 MB
PassMark Software
43.09
44.47
47.65
60.04
2012 Average
68.46
147.03
0s
20 s
40 s
60 s
80 s
100 s
120 s
140 s
160 s
854
890
1051
2012 Average
1058
1082
1415
0 ms
200 ms
400 ms
600 ms
800 ms
PassMark Software
53
83
2012 Average
169
194
249
265
0s
50 s
100 s
150 s
200 s
250 s
300 s
249.54
302.28
339.90
2012 Average
442.47
520.30
800.32
0 MB
PassMark Software
4738
5471
2012 Average
6479
7226
7270
7690
24.27
24.57
24.66
2012 Average
25.95
26.22
30.01
0s
5s
10 s
15 s
20 s
25 s
30 s
35 s
PassMark Software
97.98
100.05
2012 Average
103.72
106.03
107.03
107.53
92 s
94 s
96 s
98 s
10.70
13.34
2012 Average
16.86
26.56
32.17
0s
5s
10 s
15 s
20 s
25 s
30 s
35 s
*Results for Kaspersky Internet Security 2012 could not be obtained as the test was blocked by the software.
PassMark Software
75.69
75.74
75.99
76.07
2012 Average
76.12
77.13
75 s
75 s
76 s
76 s
77 s
77 s
78 s
81.66
82.37
82.40
84.44
2012 Average
86.09
99.59
0s
20 s
40 s
60 s
80 s
100 s
120 s
PassMark Software
Kaspersky Internet Security 2012 Panda Internet Security 2012 Norton Internet Security 2012 Avast! Internet Security 6 2012 Average G Data Internet Security 2012 0s
26.46
26.64
26.83
29.97
312.15
1450.85
200 s
400 s
600 s
800 s
1,000 s
1,200 s
1,400 s
1,600 s
30.95
58.00
69.88
2012 Average
111.29
153.88
168.74
0s
20 s
40 s
60 s
80 s
PassMark Software
106.02
116.27
2012 Average
121.04
121.42
123.42
138.07
0s
20 s
40 s
60 s
80 s
100 s
120 s
140 s
160 s
PassMark Software
29.2
29.8
33.2
2012 Average
33.5
34.7
35.3
0s
5s
10 s
15 s
20 s
25 s
30 s
35 s
40 s
G Data AntiVirus 2012 Norton AntiVirus 2012 Avast! Pro Antivirus 6 2012 Average Kaspersky Anti-Virus 2012 Panda Antivirus 2012 0s 10 s
21.9
24.9
37.0
43.6
45.4
71.5
20 s
30 s
40 s
50 s
60 s
70 s
80 s
PassMark Software
237
240
345
2012 Average
813
907
2335
0 ms
500 ms
1,000 ms
1,500 ms
2,000 ms
2,500 ms
8.75
31.36
2012 Average
41.54
55.03
55.77
56.80
0 MB
10 MB
20 MB
30 MB
40 MB
50 MB
60 MB
PassMark Software
40.27
44.07
44.80
57.87
2012 Average
66.22
144.07
0s
20 s
40 s
60 s
80 s
100 s
120 s
140 s
160 s
889
1004
1093
2012 Average
1104
1234
1301
0 ms
200 ms
400 ms
600 ms
PassMark Software
53
74
76
2012 Average
104
113
202
0s
50 s
100 s
150 s
200 s
250 s
231.27
308.67
384.90
474.75
2012 Average
485.80
1029.40
0 MB
200 MB
400 MB
600 MB
800 MB
1,000 MB 1,200 MB
PassMark Software
4222
4297
4471
2012 Average
5176
6288
6600
1000
2000
3000
4000
5000
6000
7000
24.13
24.35
25.88
2012 Average
26.55
26.86
31.52
0s
5s
10 s
15 s
20 s
25 s
30 s
35 s
PassMark Software
98.24
98.43
2012 Average
103.74
103.83
104.61
113.59
90 s
95 s
100 s
105 s
110 s
115 s
10.76
12.94
2012 Average
19.28
26.06
27.35
0s
5s
10 s
15 s
20 s
25 s
30 s
*Results for Kaspersky Anti-Virus 2012 could not be obtained as the test was blocked by the software.
PassMark Software
74.92
75.93
2012 Average
76.21
76.24
76.96
77.02
74 s
74 s
75 s
75 s
76 s
76 s
77 s
77 s
78 s
81.12
81.25
84.20
84.88
2012 Average
86.08
98.94
0s
20 s
40 s
60 s
80 s
100 s
120 s
PassMark Software
23.60
26.78
27.13
31.09
2012 Average
308.02
1431.52
1,400 s
1,600 s
22.88
33.63
70.13
2012 Average
86.85
152.50
155.13
0s
20 s
40 s
60 s
80 s
100 s
120 s
140 s
160 s
180 s
PassMark Software
116.90
120.42
121.35
2012 Average
124.34
126.80
136.22
135 s
140 s
PassMark Software
Disclaimer of Liability
While every effort has been made to ensure that the information presented in this report is accurate, PassMark Software Pty Ltd assumes no responsibility for errors, omissions, or out-of-date information and shall not be liable in any manner whatsoever for direct, indirect, incidental, consequential, or punitive damages resulting from the availability of, use of, access of, or inability to use this information.
Disclosure
Symantec Corporation funded the production of this report and supplied some of the test scripts used for the tests.
Trademarks
All trademarks are the property of their respective owners.
Contact Details
PassMark Software Pty Ltd Suite 202, Level 2 35 Buckingham St. Surry Hills, 2010 Sydney, Australia Phone + 61 (2) 9690 0444 Fax Web + 61 (2) 9690 0445 www.passmark.com
Download Location
An electronic copy of this report can be found at the following location: http://www.passmark.com/tpsreport11
PassMark Software
PassMark Software
PassMark Software
was 982 MB. Most of these files come from the Windows system folders. As the file types can influence scanning speed, the breakdown of the main file types, file numbers and total sizes of the files in the sample set is given here:
File Extension .dll .exe .sys .gif .doc .wmf .png .html .nls .jpg .ini .ico .mof .ax .xls .ime .drv .txt .chm .cpl .mfl .inf .hlp .imd .py .msc .vbs .xml .rtf .ocx .tsp .com .xsl .h .vsd .scr .aw .js .zip .lex .ppt .acm .wav Total
Number of Files 2589 695 332 302 281 185 149 126 80 70 59 58 43 39 38 35 31 31 30 29 29 26 22 20 20 18 18 18 16 16 14 14 14 13 12 12 12 12 11 9 9 9 7 6159
File Size 490MB 102MB 23MB 1MB 64MB 2MB 2MB 1MB 6MB 1MB 2MB <1MB 6MB 4MB 3MB 5MB 1MB 1MB 6MB 4MB 3MB 2MB 3MB 18MB <1MB 1MB 1MB 1MB 62MB 4MB 1MB <1MB <1MB <1MB 2MB 2MB 2MB 1MB 25MB 10MB 4MB 1MB 5MB 982
This scan was run without launching the products user interface, by right-clicking the test folder and choosing the Scan Now option. To record the scan time, we have used products built-in scan timer or reporting system. Where this was not possible, scan times were taken manually with a stopwatch.
PassMark Software
For each product, five samples were taken with the machine rebooted before each sample to clear any caching effects by the operating systems. In 2009, we noticed many more products showing a substantial difference between the initial scan time (first scan) and subsequent scan times (scans 2 to 5). We believe this behavior is due to products themselves caching recently scanned files. As a result of this mechanism, we have averaged the four subsequent scan times to obtain an average subsequent scan time. Our final result for this test is an average of the subsequent scan average and the initial scan time.
PassMark Software
The set of websites used in this test include front pages of high traffic pages. This includes shopping, social, news, finance and reference websites. The Browse Time test is executed five times and our final result is an average of these five samples. The local server is restarted between different products and one initial test run is conducted prior to testing to install Adobe Flash Player, an add-on which is used by many popular websites.
PassMark Software
We have excluded product activation times due to network variability in contacting vendor servers or time taken in account creation.
PassMark Software
PassMark Software
UCE EXE DLL AX CPL CPX DRV ICO MSC NT ROM SCR SYS TLB TSK UCE Total
1 19 104 1 2 2 10 1 1 1 2 2 1 3 1 1 812
22,984 2,952,914 29,261,568 18,432 2,109,440 4,384 154,864 107,620 41,587 1,688 36,611 2,250,240 37,528,093 135,580 1,152 22,984 760,867,636
This test was conducted five times to obtain the average time to copy, move and delete the sample files, with the test machine rebooted between each sample to remove potential caching effects.
This test was conducted five times to obtain the average time to install/uninstall the above third party programs, with the test machine rebooted between each sample to remove potential caching effects.
PassMark Software
File format JPEG GIF PNG MOV RM AVI WMV MP3 PDF ZIP 7Z Total
Size (bytes) 30,668,312 360,349 494,780 57,360,371 5,658,646 78,703,408 46,126,167 191,580,387 136,298,049 6,295,987 92,238 553,638,694
This test was conducted five times to obtain the average time to download this sample of files, with the test machine rebooted between each sample to remove potential caching effects.
PassMark Software
.doc .docx .gif .jpg .png .mov .rm .avi .wma .avi .wma Total
35.9 MB 24.5 MB 1.10 MB 66.2 MB 48.9 MB 54.7 MB 5.39 MB 459 MB 48.6 MB 459 MB 48.6 MB 783 MB
This test was conducted five times to obtain the average file compression and decompression speed, with the test machine rebooted between each sample to remove potential caching effects.
PassMark Software
A total of five runs of this test were performed, with a machine restart between each run. The time taken to copy files was measured and recorded by CommandTimer.exe. All the files were copied between a folder on the local drive and a 2
nd
nd