Group 2: 3D rendering

This group includes 3ds max, Maya, and Lightwave with their rendering results. Why did we choose to analyze rendering speed in a separate part of the test method? This task belongs to a narrow specific class. For one, it's almost a perfect process for running in several threads: performance gain from adding another processor core often reaches 100%. Thus, it will be logical to compare processors with different number of cores using rendering alone: on one hand, it reveals advantages of systems with more cores; on the other hand, rendering results do not interfere with visualization performance (where the number of cores still does not affect the speed much). For two, not all users care about rendering speed (even if they work with 3D modeling). Big design companies often do final renders on special dedicated computers (sometimes even render-farms with several computers). Individual designers rarely do final renders (not previews). And even if they do, they do not work on their computers at that time (for example, they render scenes at night, when they sleep). It seemed right to bring render speed results into a separate chapter, so that if you are really interested in this task, you can get full information, which is not diluted with other results. And on the contrary, if you are indifferent to render speed results, you won't have to sift through the test results for necessary information.

Technical details

1. Software: 3ds max 2009 x64 + SP1, V-Ray for 3ds max x64 1.50 SP2, SPECapc for 3ds max 9. Testing: Install SPECapc for 3ds max 2007, then download UIarrayAutomation.ms and copy (or overwrite, if this file is already present) it to \Program Files\Autodesk\3ds Max 2009\MAXTest\ScriptFiles. Go to \Program Files\Autodesk\3ds Max 2009\MAXTest\Scenes, replace throne_shadowmap, cballs2, and space_flyby scenes with Scenes 01, 02, and 07 from Evermotion Archinteriors Vol 1 (that is the scenes from Archinteriors must be placed in the ...MAXTest\Scenes folder and renamed to the original benchmark scenes). Copy graphics files that come with Evermotion Archinteriors scenes to \Program Files\Autodesk\3ds Max 2009\maps. Start the test: \Program Files\Autodesk\3ds Max 2009\MaxTestAuto.bat. The Rendering score is used in this part of the test method. The test is run five times, results obtained are averaged out.
2. Software: Maya 2009 x64 SP1, scene for rendering. The result of this test is the time is takes to render a scene.
3. Software: Lightwave 3D x64 9.6. SPECapc for Lightwave 3D 9.6. Instructions on how to start the test come with SPECapc for Lightwave 3D 9.6, no changes are necessary. We use the Rendering score here: the lower the score, the better. The test is run five times, results obtained are averaged out.
4. Everything is by default after the installation.

The total score in this group is calculated as a geomean of results demonstrated by all tests. In this chapter all test results take part as 1/x, except for the 3ds max test, which results shouldn't be modified.

Group 3: Scientific and engineering analysis

This part of the test method accumulates computation results. It's designed after the scientific part from the previous version of the test method. The new version of the test method accumulates all such results in one chapter. Indeed, MAPLE, Mathematica, MATLAB are all tools for scientists and engineers. However, CAD/CAM packages (SolidWorks, Pro/ENGINEER, UGS NX) also deal with engineering computations, and tests with these programs also provide separate computation results. Besides, despite the external differences, the main point is the same: Pro/ENGINEER allows to rotate objects on the screen, and it looks great, but these objects must be computed prior to that. So this part of the test method includes scientific packages as well as processing scores from SolidWorks, Pro/ENGINEER, and UGS NX. As the latter have been already described, we'll dwell on mathematical packages.

Mathematica test (to be more exact, two tests) hasn't changed since the previous version of the test method, only Mathematica version is different. MAPLE test actually remains the same, but some of its parameters have been increased (array and matrix dimensions, etc.) So, memory requirements of this test have grown: it had used about 4-6 MB of memory for data (which means that some processors most likely performed all operations in L2-Cache). But now the test uses about 40-45 MB of memory, which apparently exceeds cache sizes of any x64 processor (it will probably stay that way for a long time). However, the most interesting changes were made to the MATLAB test.

MATLAB has a built-in benchmark. Who would know better how to test performance than its developers? Unfortunately, this is not the case (or they had added this benchmark long ago and never improved it). In fact, MATLAB 2008 benchmark is very unstable: even if we average results of 100 (!) iterations, the spread of results may reach 20%! In fact, we even wanted to remove MATLAB from the list of test packages because of its inadequate test results. Fortunately, we were not alone who faced inadequacy of the integrated MATLAB benchmark, so we managed to find an interesting third-party project to use in our tests. By the way, this benchmark is more stable, predictable, and it provides much more details.

Technical details

1. Software: Mathematica x64 7.01. Tests: Internal.nb, MMA.nb. Run the tests (command prompt): math.exe <path and filename of the test>. Each test is run five times, results obtained are averaged out. Total score -- the geomean of two averaged results.
2. Software: MAPLE 12. Test: SciGmark.txt. Run the test (command prompt): cmaple.exe scigmark.txt. The test is executed five times, results obtained are averaged out.
3. Software: MATLAB x64 r2008b (7.7). Test: benchmark from SciViews.org. The test is run ten times, results are averaged out (the number of iterations can be specified in the benchmark by using the runs variable).
4. Software: Pro/ENGINEER Wildfire x64 4.0 (M070). Test: OCUS Benchmark v5.1 (64-bit version). We use the CPU Related Tasks score here: the lower the score, the better. The test is executed five times, results obtained are averaged out.
5. Software: SolidWorks x64 2009 SP0.0. Test: SPECapc for SolidWorks 2007. We use the CPU score here: the lower the score, the better. The test is executed five times, results obtained are averaged out.
6. Software: UGS NX x64 6.0. Test: SPECapc for UGS NX 4. The Total CPU score is used in this part of the test method. The test is executed five times, results obtained are averaged out.
7. All packages and tests are configured by default.

The total score in a group is calculated as a geomean of all test results (if the lowest value means the best result, the geomean result works as 1/x).

Write a comment below. No registration needed!