Summary Report 2020
Awards, Winners, Comments
| Release date | 2021-01-18 |
| Revision date | 2021-01-15 |
| Test Period | January - December 2020 |
Introduction
We are an independent test lab, providing rigorous testing of security software products. We were founded in 2004 and are based in Innsbruck, Austria.
 |
AV-Comparatives is an ISO 9001:2015 certified organisation. We received the TÜV Austria certificate for our management system for the scope: “Independent Tests of Anti-Virus Software”. |
 |
AV-Comparatives is the first certified EICAR Trusted IT-Security Lab |
At the end of every year, AV-Comparatives releases a Summary Report to comment on the various consumer anti-virus products tested over the course of the year, and to highlight the high-scoring products of the different tests that took place over the twelve months. Please bear in mind that this report considers all the Consumer Main-Series Tests of 2020, i.e. not just the latest ones. Comments and conclusions are based on the results shown in the various comparative test reports, as well as from observations made during the tests (https://www.av-comparatives.org/consumer/test-methods/).
Tested Products
The following vendors’ products were included in AV-Comparatives’ Public Consumer Main Test-Series of 2020 and had the effectiveness of their products independently evaluated. We are happy that this year’s tests helped several vendors to find critical and other bugs in their software, and that this has contributed to improving the products.
Approved Security Product Award
The tested products of all the 17 vendors above are AV-Comparatives 2020 Approved Windows Security Products.
Management Summary
Tests
In 2020, AV-Comparatives subjected 17 consumer security products for Windows to rigorous investigation. All the programs were tested for their ability to protect against real-world Internet threats, identify thousands of recent malicious programs, defend against advanced targeted attacks, and provide protection without slowing down the PC.
Results and Awards
Whilst all of the programs in our test reached an acceptable level overall, some programs outperformed others. For details, please see “Overview of levels reached during 2020”. In order to recognise those products that achieve outstanding scores in our tests, we have given a number of end-of-year awards that highlight the best results in each test, and overall. The Product of the Year and Top Rated awards are based on overall performance in the Public Consumer Main Test Series; there are also Gold, Silver and Bronze awards for each individual test type. Please see the Award Winners section for more details of the awards. The 2020 Product of the Year Award goes to Kaspersky; Bitdefender and ESET receive Outstanding Product Awards; the Top Rated Product Awards go to Avast and AVG.
Overview of tested products
Here we provide a summary for each of the programs tested, with a note of each one’s successes during the year. Although the user interface does not affect any awards, we have noted some of the best UI features as well.
Avast is a Top-Rated Product in 2020. It received an Advanced+ Award in four of this year’s tests, and Advanced in the other three. It also takes a joint Silver Award for Real-World Protection and a joint Bronze Award for Advanced Threat Protection. It has a very clean, modern interface, and the setup wizard offers ideal options for both expert and non-expert users.
AVG receives a Top-Rated Product award this year. In four of this year’s tests it took an Advanced+ Award, along with Advanced in the other three. It also received a joint Silver Award for Real-World Protection, and a joint Bronze Award for Advanced Threat Protection. It has a touch-friendly interface and good setup options.
Avira took three Advanced+ and three Advanced Awards in this year’s tests. It features a modern, touch-friendly interface.
Bitdefender wins an Outstanding Product Award, having reached Advanced+ in all seven tests in 2020. It additionally receives joint Silver Awards for Malware Protection and Advanced Threat Protection, and the Bronze Award for False Positives. Its well-designed user interface includes a customisable home page, and real-time protection is highly sensitive.
ESET gets an Outstanding Product Award for 2020, as it reached Advanced+ level in all 7 tests this year. It also receives the Gold Award for False Positives, and joint Silver Awards for Malware Protection and Advanced Threat Protection. Reviewers were impressed with the clear and simple layout of the GUI, and ease of use.
F-Secure took three Advanced+ and three Advanced Awards this year. It features an easy-to-use, simply laid-out interface.
G Data takes this year’s Gold Award for Malware Protection. It also received 4 Advanced+ and 1 Advanced Award in the 2020 tests. Reviewers noted its especially detailed status display and excellent access control.
K7 takes this year’s Gold Award for Performance. It also got two Advanced+ and three Advanced Awards in the 2020 tests. Reviewers liked its simple design and impressive scanning speed.
Kaspersky is AV-Comparatives’ Product of the Year for 2020, having got Advanced+ Awards in all 7 of the year’s tests. In addition, it receives Gold Awards for Real-World Protection and Advanced Threat Protection, the Silver Award for False Positives, and Bronze Awards for Malware Protection and Performance. It has an easy-to-use tiled interface, and a wide range of configuration options.
McAfee received three Advanced+ and two Advanced Awards in the year’s tests, along with a Silver Award for Performance. Its user interface is clean, modern and touch friendly. The program’s status alerts are exemplary.
Microsoft took the Bronze Award for Real-World Protection this year. It also received two Advanced+ Awards and one Advanced Award in the year’s tests. The product is integrated into Windows 10, and has a simple, unobtrusive interface.
NortonLifeLock’s product took six Advanced Awards in this year’s tests. It has a well-designed overall user experience, with detailed malware information accessible from alerts.
Panda received two Advanced+ and three Advanced Awards in this year’s tests. Reviewers noted its security-blog feature, which lets you read articles on various IT-security related topics.
Total AV got one Advanced+ and four Advanced Awards in the 2020 tests. It features a very simple, easy-to-navigate program window.
Total Defense took one Advanced+ and four Advanced Awards this year. Its user interface stands out for its simplicity.
Trend Micro received one Advanced+ and two Advanced Awards in this year’s tests. The user interface presents a simple overview, but allows easy access to advanced options. Its persistent malware and status alerts stand out.
Vipre took four Advanced+ and two Advanced Awards in this year’s tests. It has a very clean design and good online help feature, which you can search directly from the program.
Advice on Choosing Computer Security Software
There is no such thing as the perfect security program, or the best one for all needs and every user. Being recognized as “Product of the Year” does not mean that a program is the “best” in all cases and for everyone: it only means that its overall performance in our tests throughout the year was consistent and unbeaten. Before selecting a security product, please visit the vendor’s website and evaluate their software by downloading a trial version. Our awards are based on test results only and do not consider other important factors (such as available interface languages, price, and support options), which you should evaluate for yourself.
Overview of levels reached during 2020
AV-Comparatives provides a wide range of tests and reviews in comprehensive reports (https://www.av-comparatives.org/consumer/test-methods/). Annual awards for 2020 are based on the Public Consumer Main Test-Series: Real-World Protection Test, Performance Test, Malware Protection Test, False-Alarm Test and the Advanced Threat Protection Test.
All the programs tested are from reputable and reliable manufacturers. Please note that even the STANDARD level/award requires a program to reach a good standard, although it indicates areas which need further improvement compared to other products. ADVANCED indicates that a product has areas which may need some improvement, but is already very competent. Below is an overview of awards reached by the various anti-virus products in AV-Comparatives’ consumer main test-series of 2020.
Key: * = Standard, ** = Advanced, *** = Advanced+
Award Winners
Awards for individual tests
For each of the test types* in the Public Consumer Main Test Series (Real-World Protection, Malware Protection, Advanced Threat Protection, Performance and False Positives), we give Gold, Silver and Bronze awards, for the first, second and third highest-scoring products, respectively.
Awards for all combined scores of all tests
As in previous years, in 2020 we are giving our Product of the Year Award to the product with the highest overall scores across all the tests in the Public Consumer Main Test Series. This depends on the number of Advanced+ awards received in all the tests. As the overall scores are considered, a product can receive the Product of the Year award without necessarily reaching the highest score in any individual test. A product cannot win the Product of the Year Award in 2 consecutive years if in the second year there is another product (or other products) with the same highest award levels.
We sometimes have a situation where two products reach exactly the same highest award levels. We think it is fair to highlight the fact that more than one product has reached an excellent level, and so in such cases we give the Product of the Year Award to the product that didn’t get it most recently. The other product with the same highest award levels will receive the Outstanding Product Award. It even happens that three or more products reach the same highest award levels (as is the case this year). In this situation, the product with the highest individual scores wins Product of the Year, while the others receive the Outstanding Product Award.
As in previous years, we will also be giving Top Rated Awards to a select group of tested products which reached a very high standard in the Public Consumer Main Series tests. We have used the results over the year to designate products as “Top Rated”. Results from all the tests are assigned points as follows: Tested = 0, Standard = 5, Advanced = 10, Advanced+ = 15. Products with 90 points or more are given the Top Rated award.
To get the Approved Windows Security Product Award (see page 5), at least 35 points must be reached.
*For some test types, there may be two actual tests conducted in a year; the awards are based on the combined score of both tests.
Product of the Year 2020
AV-Comparatives’ 2020 Product of the Year Award goes to:
Kaspersky
Outstanding Products 2020
AV-Comparatives’ 2020 Outstanding Product Awards go to (in alphabetical order):
Bitdefender, ESET
Top-Rated Products 2020
AV-Comparatives’ Top-Rated Awards for 2020 go to (in alphabetical order):
Avast, AVG
Please see our summary and awards pages – links below:
https://www.av-comparatives.org/test-results/
https://www.av-comparatives.org/awards/
Real-World Protection Test winners
Security products include various different features to protect systems against malware. Such protection features are taken into account in the Real-World Protection Test, which tests products under realistic Internet usage conditions. Products must provide a high level of protection without producing too many false alarms, and without requiring the user to make a decision as to whether something is harmful or not.
The programs with the best overall results over the course of the year were from: Kaspersky, Avast, AVG and Microsoft.
 |
Kaspersky |
 |
Avast, AVG |
 |
Microsoft |
For details and full results of the 2020 Real-World Protection tests, please click the link below:
https://www.av-comparatives.org/consumer/testmethod/real-world-protection-tests/
Malware Protection winners
The Malware Protection Test evaluates an AV product’s ability to protect against malware coming from removable devices or network shares. Products must provide a high level of protection without producing too many false alarms. In the Malware Protection Test, all samples not detected on-demand or on-access are executed.
G Data, Bitdefender, ESET and Kaspersky scored well in both tests.
 |
G Data |
 |
Bitdefender, ESET |
 |
Kaspersky |
For details and full results of the 2020 Malware Protection tests, please click the link below:
https://www.av-comparatives.org/consumer/testmethod/malware-protection-tests/
False Positives winners
False positives can cause as much trouble as a real infection. Due to this, it is important that anti-virus products undergo stringent quality assurance testing before release to the public, in order to avoid false positives. AV-Comparatives carry out extensive false-positive testing as part of the Malware Protection Tests. Additionally, also false alarms from the Real-World Protection Test are counted for this category.
The products with the lowest rates of false positives during 2020 were ESET (5), Kaspersky (15) and Bitdefender (19). These figures represent the SUM of the false positives from all FP Tests.
 |
ESET |
 |
Kaspersky |
 |
Bitdefender |
False Alarm Testing is included in each Protection Test. For additional details about False Positives in the Malware Protection Test, please click the link below:
https://www.av-comparatives.org/consumer/testmethod/false-alarm-tests/
Overall Performance (Low System-Impact) winners
Security products must remain turned on under all circumstances, while users are performing their usual computing tasks. Some products may have a higher impact than others on system performance while performing some tasks.
K7, McAfee and Kaspersky demonstrated a lower impact on system performance than other products.
 |
K7 |
 |
McAfee |
 |
Kaspersky |
For details and full results of the 2020 Performance tests, please click the link below:
https://www.av-comparatives.org/testmethod/performance-tests/
Advanced Threat Protection (Enhanced Real-World Test) winners
This tests a program’s ability to protect against advanced targeted and fileless attacks.
Kaspersky blocked 14 targeted attacks (out of 15), Bitdefender and ESET blocked 13 attacks, Avast and AVG blocked 11 attacks.
 |
Kaspersky |
 |
Bitdefender,ESET |
 |
Avast, AVG |
For details and full results of the 2020 Enhanced Real-World Test, please click the link below:
https://www.av-comparatives.org/testmethod/enhanced-real-world-tests/
Pricing
AV-Comparatives’ awards and rankings are based entirely on products’ technical capabilities, not on any other factors such as costs. However, the price of a security product is obviously a factor that users consider. We have listed here some considerations that readers may like to take into account when choosing their security software.
We would not recommend choosing a security product based on price alone. We suggest that you look at protection, performance and ease of use first, and consider the price last.
It is clear that some free programs protection and performance on a par with paid-for programs, and are easy to use. One of the main disadvantages to free programs can be limited technical support, however. Additional features may also be lacking or limited. Finally, some free programs make extensive advertising for their paid-for counterparts, which many users may find irritating.
It is possible to buy security programs from third-party vendors (e.g. online or in electronics stores) more cheaply than the vendor’s list price. We would advise users to check that they are buying the latest version of the product, or that the product purchased can be upgraded to the latest version without additional cost.
When purchasing a product from the vendor’s own website, there are two factors that users might like to consider. The first concerns multi-platform licences. Many vendors now offer a licence for e.g. 5 devices, which you can use for Windows, macOS or Android devices, or a mix. In some cases, the price may vary depending on which section of the website you buy from. For example, a multi-platform licence bought from the “Products for Mac” page may be a different price from an (effectively identical) product bought from the “Products for Windows” page.
The second point to consider is auto-renewal. Some vendors offer or automatically apply auto-renewal of the subscription when you buy from their website. Unless you cancel this, you will be charged again at the end of the initial licence period, and the subscription will be extended accordingly. Clearly this is to the advantage of the vendor, as it makes it easy for them to keep you as a customer. If you buy an AV product from the vendor’s own website, we suggest that you check the auto-renewal situation first. Some vendors do not have auto-renewal at all. Others let you opt in by putting a tick in a checkbox, while others have auto-renewal activated by default, but let you opt out easily by removing the tick from the checkbox. In some cases, auto-renewal is automatically applied, and cannot be deactivated at the time of purchase; you have to message the vendor afterwards to cancel it. This gives the vendor the opportunity to try to keep you as a customer, by offering various incentives.
Before agreeing to purchase a product with auto-renewal, we suggest that you find out what the renewal price will be when your subscription expires. In some cases, this may be very much higher than the initial purchase price. However, it might also be cheaper. It is also possible that if you opt out of auto-renewal at the time of purchase, the price shown in the basket will increase. Our 2020 Security Survey indicates that about seven out of eight users are not happy with mandatory auto-renewal.
In the table below we have listed the (rounded) current discount price, full list price and auto-renewal prices (where applicable) for the paid products in the 2020 Main Test Series.
It is possible that some vendors may offer additional discounts at specific times or under specific circumstances.
Key: Ratio of rounded full list price/autorenewal price to rounded discounted first-year price is (green) no more than twice; (yellow) more than twice but no more than three times; (red) more than three times.
*We presume that the auto-renewal price will be the same as the full list price. However, either the information on the vendor’s website was unclear, or it stated that the vendor had the right to change the price on auto-renewal.
Where “Auto-renewal on by default” is shown as “optional”, it means that auto-renewal is activated by default, but can be deactivated at the time of purchase, e.g. by removing a tick/checkmark in the relevant box. Where it is shown as “mandatory”, you cannot deactivate it at the time of purchase, but have to cancel it afterwards. Each vendor has its own procedure for deactivating auto-renewal, so we suggest that readers find out about this in good time before the renewal date. It might be that e.g. uninstalling the product from the computer makes cancelling auto-renew more difficult.
The aim of this table is to compare each product’s full list price with both its discounted price for the first year and its renewal price for the second year of the subscription. We advise readers NOT to use the data here to compare prices between products. Some products provide just malware protection, whilst others include e.g. parental controls as well, so it would not be a fair comparison. Our 2020 Consumer Main Test Series tested free products by Avast, AVG, Microsoft and Panda. These products are not shown in the table, as pricing does not apply to them. For two of the products shown in the table, the lowest-price subscription allows you to install the product on three devices. If you only want to protect one device with these products, you will still have to pay the price shown here.
The terms “full list price” and “discounted price” could potentially be used by vendors in a misleading way. Some countries, such as the UK and Germany, have laws stating that a vendor can only use terms like “special offer” if the lower price is offered for a shorter period than the full list price. Readers might like to check the applicable law in the country from which they are purchasing the product. We have given the prices shown on the respective vendor’s website at the time of writing (December 2020), applicable to users in Austria. We have not investigated if or for how long the full list prices stated on the vendors’ websites have been offered. Thus we cannot say if they are in accordance with the applicable law in the country of purchase.
We should point out some good practices by some vendors. G Data is actually cheaper in the second year, and along with Bitdefender, ESET, F-Secure, K7 and Kaspersky does not impose auto-renewal on users.
Product Reviews
Review Format
For each of the tested products, we have looked at the following points (where applicable).
About the program
To start off with, we state whether the program is free or has to be paid for. We don’t list individual protection components (e.g. signatures, heuristics, behavioural protection), for the following reasons. Our protection tests verify how well each program protects the system, whereby it is not important which component(s) are involved. It is not the number of features that is important, but how effectively they work. Also, different vendors may have different names for individual functions, or combine multiple types of functionality under one name. This could make it misleading to compare products using the vendors’ component names. For readers’ convenience, we do note any non-malware-related features, such as parental controls or spam filtering. With the exception of a replacement firewall (see below), we do not check the functionality of these additional features.
Setup
We note any options available, whether you have to make any decisions, and any other points of interest, such as introductory wizards that explain the program’s features. We suggest that there should be a simple installation option for non-expert users. If at any stage the user has to make a decision in order to proceed, the options should be explained simply and clearly.
System Tray icon
Here we state what functionality is available from the program’s System Tray icon. This can be a convenient way of accessing commonly-used functions, such as scans and updates. A System Tray icon is a standard feature for modern security programs for consumers. We regard it as a very useful means of showing that the program is running.
Security alerts
First, we disable the program’s real-time protection, and check to see what alerts are shown. We also look for a quick and easy means of reactivating the protection. An effective status display, which shows a clear warning if protection is disabled, is a very standard feature, as is a “Fix-All” button/link with which the user can easily re-enable protection. We regard both of these as important for non-expert users. Additional pop-up alerts, which the user would see even if the program window were not open, are a desirable bonus. Next, we check how each program reacts when malware is encountered. We start off by downloading the EICAR test file (a harmless text file that antivirus products are programmed to detect for test purposes). We look to see what sort of alert is shown, if there are any options provided for dealing with the malware, if any information about the malware is given, and how long the alert is displayed. For non-expert users at least, we feel it is appropriate for a security program to show an alert when malware is detected, so that the user understands why the file in question cannot be downloaded/accessed. Also for non-expert users, we regard it as ideal if the malware is deleted or quarantined automatically, without the user having to make a decision on what to do with it. We would definitely recommend that any alert box should not include an option to instantly whitelist the file (i.e. allow it to be executed there and then). A much safer option is to quarantine the file, after which power users could go into the program’s settings to whitelist and restore it if they wanted.
The next step is to connect a USB flash drive to the review system, containing a few very prevalent and well-known malware samples. We note if any action is taken automatically by the security program, or if it prompts to scan the drive. If a scan is offered, we decline it, and open the drive in Windows Explorer. If the malware is not detected at this point, we attempt to copy the files on the drive to the Windows Desktop. If this is successful, we then execute them. We note at which stage the malware is detected, what sort of alert is shown, and if any action needs to be taken by the user.
All the programs in our Consumer Main-Test Series detect malware on execution, which is enough to keep the system free of infection. However, many users may expect their security program to detect malware on access, i.e. when the drive or folder containing it is opened in Windows Explorer, or at the latest when it is moved or copied. This would prevent them inadvertently passing on malicious files to somebody else on an external drive. We note that programs without on-access scanning may have a performance advantage as a result.
Scan options
Here we look at the different types of on-demand scan provided by each program, how to access and configure them, set scan exclusions, schedule scans, and what options are provided for PUA detection. We also look at how the results are displayed at the end of an on-demand scan, and whether the user needs to make any decisions. If multiple malicious files are found in a scan, it should be easy to carry out a safe action on all of them at once, rather than having to select an action for each one individually.
Quarantine
In the program’s quarantine function, we look to see what information it provides about the detection location/time and the malware itself, and what options are available for processing it, e.g. delete, restore or submit to vendor for analysis.
Access control
For users who do not share their computer with anyone, this section is not relevant. However, if you share a computer, e.g. with your family at home, or colleagues in a small business, you might want to read it. We look to see if it possible to prevent other users of the computer from disabling the security program’s protection features, or uninstalling it altogether. There are two ways of doing this. Firstly, access can be limited using Windows User Accounts: users with Administrator Accounts can change settings and thus disable protection, whereas those with Standard User Accounts can’t. Alternatively, a program can provide password protection, so that any user – regardless of account type – can only change settings by entering a password. Some programs provide both methods, which we regard as ideal. When testing access control, we try to find all possible means of disabling protection, to ensure that any restrictions apply to all of them.
Help
In this section, we take a quick look at whatever help features can be directly accessed from the program itself. Some vendors will have additional online resources, such as manuals and FAQ pages, that can be found by visiting their respective websites.
Logs
Here we note what information is provided in the program’s log function.
Firewall
Some of the products in this year’s tests have a replacement firewall. That is to say, they include their own firewall, which is used in place of Windows Firewall. For these products, we perform a very simple functionality test, to check that basic functions of their replacement firewalls work as expected. In essence, this just verifies that network discovery and file sharing are allowed on private networks, but blocked on public ones.
For this test, we use a laptop PC, running Windows 10 Professional, with a wireless network adapter. We share the Documents folder, with read and write permissions for “Everyone”, and enable Remote Desktop access. In Windows Firewall/Advanced Sharing Settings, we turn on network discovery, file sharing, and Remote Desktop access for Private networks, but turn them all off for Public networks. We then verify that network access is working as expected in both Private and Public networks. It is initially connected to a wireless network that is defined as Private in Windows network status settings.
We then install the security product with default settings, and reboot the computer. If during installation the third-party firewall in the security product were to prompt us to define the current network as public or private, we would designate it as private at that point. After the reboot, we check to see if we can still ping the PC, open and edit a document in its shared folder, and gain Remote Desktop access. We would expect the third-party firewall to allow all these types of access.
We then connect the laptop to a new, unknown wireless network, which we define as Public in Windows’ network status prompt. If the third-party firewall were to display its own network-status prompt, we would also choose the public/untrusted option here. Next, we attempt to ping the test laptop (IPv4 and IPv6) from another computer on the same network, access its file share, and log in with Remote Desktop. We would expect the third-party firewall to block all these forms of access, as Windows Firewall would do.
We also check what happens if the network status is changed from Private to Public in Windows network settings, i.e. if the third-party firewall in the tested product picks up the new status automatically, or displays its own prompt at that point.
In our opinion, a third-party firewall in a security program should either adopt Windows’ network status settings automatically, or achieve the same result by means of displaying its own prompts. This allows laptop users to share files when at home, but keep intruders out when using public networks.
We recognise that some users may like to use Windows Firewall – which is a known standard – rather than the third-party firewall in their security product. For such users, it is ideal if the security product’s own firewall can be cleanly disabled (i.e. permanently disabled, without security alerts being constantly shown), and Windows Firewall can be activated instead. We check to see if this is possible.
Other points of interest
Here we note anything we observe or find out about a product that we think is relevant. This may include privacy-related items, descriptions of the product on the vendor’s website, unusual places to find features, customisation options, prompts to install additional features, upselling, bugs, explanations of functions, and out-of-the-ordinary features and notifications.
Copyright and Disclaimer
This publication is Copyright © 2021 by AV-Comparatives ®. Any use of the results, etc. in whole or in part, is ONLY permitted after the explicit written agreement of the management board of AV-Comparatives prior to any publication. AV-Comparatives and its testers cannot be held liable for any damage or loss, which might occur as result of, or in connection with, the use of the information provided in this paper. We take every possible care to ensure the correctness of the basic data, but a liability for the correctness of the test results cannot be taken by any representative of AV-Comparatives. We do not give any guarantee of the correctness, completeness, or suitability for a specific purpose of any of the information/content provided at any given time. No one else involved in creating, producing or delivering test results shall be liable for any indirect, special or consequential damage, or loss of profits, arising out of, or related to, the use or inability to use, the services provided by the website, test documents or any related data.
For more information about AV-Comparatives and the testing methodologies, please visit our website.
AV-Comparatives
(January 2021)