Scan Speed Benchmark – Software Testing Weekly #5

Benchmark Tests / / 4 minutes of reading

Scan speed benchmark of the latest versions

This scan speed benchmark test case came to mind after the latest version of jv16 PowerTools 2017 has been released. We then decided to test the actual scan speed versus number of errors found for the latest three public versions of the Clean and SpeedUp My Computer tool.

Why this test now?

On the occasion of releasing jv16 PowerTools 2017 version 4.1.0.1728 it seemed like the perfect opportunity to fill a Windows 10 virtual machine with residual registry and file system data, then compare the performance of our latest three publicly released scanner versions.

Using this scan speed benchmark, we hope to show the differences in terms of scanner speed (the latest version of jv16 PowerTools 2017 is very fast) and how we managed to obtain it (faster speed most of the times means lower accuracy).

The “crap.reg” registry file imported to the test system contains over 10000 unique registry and file system orphaned entries and other errors of various severity levels.

We use such test data to ensure the fact that the latest version of our scanner does not have any false positives. This will be the subject of an upcoming quality testing report as part of the weekly test routines.

Tested jv16 PowerTools versions

The following jv16 PowerTools 2017 versions have been tested:

  • jv16 PowerTools 2017 version 4.10.1689
  • jv16 PowerTools 2017 version 4.10.1703
  • jv16 PowerTools 2017 version 4.10.1728

To download the latest version of jv16 PowerTools, always go to: https://jv16powertools.com/download/

Test Cases

The objective of this one-off test case was set to:

  • Test Case #1: Compare the volume of Registry Errors, Temp Files and History Data found versus the overall Clean and SpeedUp My Computer scanner speed between latest three publicly released versions.

The information on which this test case relies is provided by the ML_CSMC_Scan log file, which can be found in the ‘Backups’ folder of your jv16 PowerTools 2017 installation path.

Test Environment

The current benchmark test was performed using an Oracle VirtualBox (version: 5.1.22 r115126) virtual environment running Windows 10 with 4 CPU cores and 8 GB of RAM assigned.

Said virtual machine was run on a host computer running Ubuntu Linux 17.04 LTS (64 bit) with an Intel-i5 processor, 20 GB of DDR3 virtual memory and a SSD system disk.

Such an ordinary test environment was created and used in order to simulate the easiest conditions that can be reproduced by any software consumer to verify the validity of these tests.

Verifiability

All of the past and current test results are accompanied by screenshots and/or video recordings which have been saved during various testing phases in order to address some of the most common test result credibility concerns for this type of studies.

Test Case #1: Summary

→ scan speed for the latest versions of jv16 PowerTools

Below you can find the actual scan speed benchmark scores recorded by each of the tested jv16 PowerTools versions:

  • jv16 PowerTools 2017 build 1689 completed scanning using Clean and SpeedUp My Computer in 160 seconds
  • jv16 PowerTools 2017 build 1703 completed scanning using Clean and SpeedUp My Computer in 244 seconds
  • jv16 PowerTools 2017 build 1728 completed scanning using Clean and SpeedUp My Computer in 95 seconds

The latest version of jv16 PowerTools 2017 (version 4.1.0.1728) completed scan of the entire system much faster than any of the other two latest public versions.

→ number of errors found using the latest versions of jv16 PowerTools

Below you can find the total number of errors found by each of the tested jv16 PowerTools versions:

  • jv16 PowerTools 2017 build 1689 found a total of 12235 system issues
  • jv16 PowerTools 2017 build 1703 found a total of 12281 system issues
  • jv16 PowerTools 2017 build 1728 found a total of 12174 system issues

Scan Speed Benchmark: conclusions

The latest version of jv16 PowerTools 2017 – build 1728 found only 0.87% less system issues than the previous public version while it is more than twice as fast.

Related weekly benchmark test reports:

Wider benchmark test reports:

Since February of 2009 (when the first test of this kind was done in our lab), Macecraft Software has published a total of seven registry cleaner product comparison studies: