Time goes on, there appear new notebook models, new applications, new benchmarks. Capacities of our test lab grow as well. Perhaps it's time to lay down our notebook test procedure. This article contains a structure of a typical review with our comments.
So, that's what we start with -
Frankly speaking, there exist just a few platforms. So if you have a look at specifications and don't notice (for example) a new graphics controller, you may already predict the performance results of a given notebook.
A standard software bundle for watching DVD or recoding optical discs (if a notebook is equipped with a proper drive) is a must. Various trifles, like a wireless mouse, may be a pro as well as a con for different users (they don't come shipped for nothing).
Evaluating the exterior and convenience of the port layout. Real weight. Photos of the front panel, rear panel (with the lid open), flanks, and bottom of a notebook.
Ergonomics and usability
Evaluating usability of a given notebook. Keyboard, touchpad, application buttons, media players, etc. As for me, I find it very difficult to lay down some objective rating criteria. That's why this part contains mostly descriptive material, conclusions coming to author's convenient/not convenient. If the key size differs from the standard 19 mm, new dimensions are to be mentioned. This part also describes interesting proprietary software that comes shipped with a notebook.
That's the end of the subjective part. There follow objective results of instrumental tests.
Objective test results
We test the line-out signal in RightMark Audio Analyzer 5.5 using the Terratec DMX 6fire sound card. Total results are published in the main review. Program-generated report is published in a separate document.
The tests will be carried out according to the LCD test procedure. You can read the details in this article. The only difference has to do with the limited number of settings in notebook matrices (in fact there is only one hardware setting - brightness), we carry out the tests with maximum brightness. The main review shows a response time, the average value and deviations of black and white spot luminance and contrast, as well as angles of view, measured by contrast for CR 10:1 and 5:1. Full results and graphs are published in a separate document.
Performance and battery life tests
We use BAPCO tests (the most adequate tests in our opinion) to determine performance and battery life of a notebook. Their only drawback is the horrible (I mean it) technical support. Thus, you are not guaranteed that the test will run on a given notebook correctly.
Testing a notebook, operating on a battery. We use BAPCO MobileMark 2005 Patch 1 here. Portable/Laptop power scheme. All additional power management utilities are disabled. If necessary, proprietary power management utilities are reviewed separately. Display is never turned off. According to the original test procedure, when a notebook works on a battery, its display brightness is set to minimum, exceeding 60 cd/m2. Brightness is measured with the SpyderPRO (PANTONE) colorimeter using OptiCAL software. The package consists of four components.
Besides, we may publish test-generated reports in a separate page.
You can read the details about this test, scoring procedures, and other aspects of MobileMark 2005 in this document (pdf format, English, ~1MB).
Testing a notebook, operating on mains power supply. The main test is BAPCO SysMark 2004SE. Portable/Laptop power scheme. Test tasks are divided into two subtests - Internet Content Creation and Office Productivity.
The result is Performance Rating - notebook performance versus a reference platform. We calculate the rating for each group of applications, the rating of subtests and the total system rating. Each potential user may consult the results he/she is interested in. This version of the test package is approved by AMD, who previously made a complaint about the performance rating procedure in BAPCO tests. You can read the details about this test, scoring procedures, and other aspects of SysMark 2004SE in this document (pdf format, English, ~1.5MB). Unfortunately, the current version of the test script for Adobe Premiere 6.5 is unstable with HD Audio cards in notebooks based on Intel 915M chipset. That's why the Internet Content Creation test is not always correct. We hope that this problem will be fixed.
We are going to use BAPCO SysMark 2004SE together with stable SysMark 2002 for some time. Unfortunately, this test is not subdivided into groups. Besides, program versions are already old, though the general ideology is the same.
3D performance tests
The main test is FutureMark 3DMark03. We use Patch 3.30. Test settings are by default.
Portable/Laptop power scheme. The results - 3DMarks for all resolutions supported. In some cases a review of a new controller may also include fps values for games from the test package, but with their parameters set not by default (mainly with different settings). You can read about the test in this document (pdf format, English, ~1.1MB). Reviews of new graphics controllers may also use notebook results in real games. In our opinion, it makes sense to test integrated video controllers only once - when a new representative appears on the market.
Memory bandwidth tests
This test confirms (when the results are close to the expected) that the notebook assemblage is of high quality. If the results obtained are much lower than the expected, we can establish a fact that the notebook has problems. We plan on using RightMark Memory Analyzer. Always On power scheme. We plan on publishing Quick Test results - average and peak memory throughput for reading and writing.
In our opinion, the current test procedure offers an adequate evaluation of notebook capacities. The set of tests may be expanded in future.
Nikolai Dorofeyev (firstname.lastname@example.org)
January 9, 2006
Write a comment below. No registration needed!