Help Index
WinTach: Windows Application Benchmarks

Overview

Disclaimer

Revision History

WinTach Benchmarking Objectives

Executing WinTach Application Benchmarks

The WinTach RPM Index

Calculation of the WinTach RPM Index

WinTach RPM Results Reporting

Command Menus

Reports Menu

Start Benchmark Menu

Resolution Menu

Tests Menu

Help Menu

Application Descriptions

Word Processing Application

Cad / Draw Application

Spreadsheet Application

Paint Application


Benchmark Objectives
WinTach Benchmark Objectives

The objective of the WinTach benchmark program is to provide a tool for estimating how Windows 3.1 applications will run on your system.

WinTach tests the most important Windows 3.1 API calls as determined by extensive profiling of commercial application sessions.  It encapsulates these API calls with realistic program logic to accurately simulate the type of graphics intensive application sessions encountered in today's work place.  The results are evaluated and displayed as the RPM, or Relative Performance Measurement, which estimates the performance of your Windows workstation running real applications.

The WinTach RPM index consists of 2 parts; a performance index followed by an operating environment identifier. Together these can be used to easily compare the performance of your system to others.  All that is required to compare RPMs is to make certain the environment identifiers are the same, and then compare the indexes. For example, if your Word Processing RPM index is double that of another system, you could expect Word Processor screen performance to be twice as fast on your system.

WinTach's application oriented approach to benchmarking provides all the benefits of "application level" benchmarking without requiring commercial applications.  This is significantly more useful than benchmarks that measure graphics subsystem performance in terms of maximum pixels per second.  These types of benchmarks can be misleading if one attempts to use them to predict actual applications performance.  Often the operations performed and the weighting of those operations have little correlation to application profiles, thus showing one system as "faster" when it is actually slower running real applications.  Consequently, the results of these benchmarks have little or no meaning in terms of predicting user productivity.


Executing WinTach Benchmarks
Executing WinTach Application Benchmarks

The main WinTach screen is divided into three areas:

1.	System Configuration information

2.	Individual WinTach RPM index results

3.	Overall WinTach RPM index

The System Configuration area provides details about the system currently in use.

The individual and overall WinTach RPM index areas report the RPM results of each WinTach application benchmark program.  To run one or all of the benchmark programs:

1.	Check the Windows environment.  WinTach runs best when there are no other tasks executing in the Windows environment (just like a real Windows application).  Dos sessions, other Windows applications, and network connections can all effect WinTach results.  To obtain the best performance results for comparison to other systems, verify that you have closed down all other Windows applications, and ideally, are not connected to a network.

2.	Select the resolution at which the tests will be run using the Resolution menu command.

3.	Run each WinTach application benchmark program.  To start a particular benchmark, click on it's application icon.  Alternatively, you can run all four benchmark programs in succession by clicking on the WinTach icon in the lower left-hand corner of the WinTach screen.

The WinTach RPM index for a particular benchmark program is displayed after completion of that benchmark program.  The overall WinTach RPM index (a simple average of the four individual benchmark RPM index values) is displayed only after all four benchmark programs have completed.

If you run all four benchmarks in succession by clicking on the WinTach icon, you can abort the tests by pressing the ESC key or by pressing the left mouse button.

If you are not using a mouse, the benchmark tests can be started using the Start Benchmark menu command.

Bar graphs illustrating relative WinTach RPM performance of the current system can be displayed by clicking on the WinTach RPM index value for the specific benchmark test.  The bar graph is displayed automatically at the completion of a singularly run benchmark test.

Note: Bar graphs are not displayed in the European version of Wintach. To determine what version of Wintach you have check the about dialog. The word Europe will be displayed after the Wintach revision if this is the case.

After running all of the benchmark tests at a particular resolution, you can display and print WinTach RPM Result Reports.  These options are available under the Reports menu command.


System Configuration information

Information describing the current system configuration is displayed on the right-hand side of the main WinTach screen, and on WinTach RPM reports.  Since WinTach RPM results are highly dependant on the host PC configuration, direct comparison of RPM results should be made only when generated on the same PC configuration.  Details included are:

System Processor

Type of host-PC processor.  For example, 386 refers to the Intel 80386 processor.

Display Driver

A description of the current display driver in use.

Display Type

The type of video hardware currently in use.  Possible types are:

Frame Buffer

This type of video board is controlled entirely by the host processor.  No video acceleration functions are provided.  Standard VGA video boards are examples of a frame buffer display type.

Controller

This type of video hardware provides a fixed set of video acceleration functions.  Video boards based on the 8514 chipset are an example of a controller display type.

32 Bit Coprocessor

This type of video hardware provides a programmable set of video acceleration functions, operating in parallel with the host processor.  Video boards based on Texas Instrument's TMS340 graphics processors are examples of Coprocessor display types.

Display Resolution

The current resolution the display driver is running at.  This value determines the possible resolutions at which the WinTach application benchmarks may be run.

Display Colors

The number of display colors provided by the display driver.  This value is used in the WinTach RPM index to indicate the number of colors under which the RPM value was obtained.


WinTach RPM Index
The WinTach RPM Index

This WinTach RPM Index is the means by which WinTach reports performance data relating to a specific system.  There is an Index calculated for each individual application, as well as an overall performance measurement.  The application Index is an indication of system speed executing that class of application, while the overall Index is designed to give a general indication of system performance.  The Index is defined as having the following format:

12.04 / B8

The Index consists of a performance index prefix, and an operating environment suffix.  The prefix represents the relative performance of the operating environment described by the suffix. The Index suffix describes the resolution and pixel size at which the data is taken.  The RPM Index can be considered a direct representation of system performance relative to any other Index having the same suffix code.

The index suffix consists of a single letter, followed by a one or two digit number.  The letter portion of the suffix indicates the screen resolution at which the WinTach RPM index was obtained.  The following table lists the possible resolutions:

Letter suffix	Resolution
A	640x480
B	800x600
C	1024x768
D	1280x1024

The numeric portion of the suffix indicates the system color depth (pixel size), which determines the number of colors available to Windows applications.  The following table lists the possible color depths:

Numeric suffix	Color depth
1	1bpp (2 Colors)
4	4bpp (16 Colors)
8	8bpp (256 Colors)
16	16bpp (Direct Color)
32	32bpp (True Color)


WinTach RPM Index Calculation

The WinTach RPM Index is reported as a performance index prefix, and an operating environment suffix.

The RPM suffix is defined to be a single letter, followed by a one or two digit number, and is solely determined by the display resolution and display pixel depth at which the tests were executed.  The suffix is intended to distinguish RPM indices taken at different resolution and pixel sizes, to prevent "apples to oranges" comparisons.  Any two indices with an identical suffix can be compared as a direct representation of their respective system performance.

The RPM prefix is a scaled ratio of the time required for the standard display system to complete the benchmark, over the time required for the target display system.  For example, if the standard system required 30 seconds to complete the Word Processing benchmark, and the target system required only 24 seconds, the resulting un-scaled performance index would be:

Raw Performance Ratio = 30 sec / 24 sec = 1.25

Next, the operating environment is taken into consideration by scaling the raw performance ratio by the ratio of target system display bits to that of the standard display system.  For example, the standard display system has a resolution of 640 by 480 pixels, and a pixel depth of 4 bits (16 colors).  If our target system had a resolution of 800 by 600, and a pixel depth of 8 bits (256 colors), the index scalar would be:

Index Scalar = (800 x 600 x 8) / (640 x 480 x 4) = 3.125

The performance index prefix for our Word Processing example would be calculated as:

Performance Index 	= Raw Performance Ratio * Index Scalar

= 1.25 * 3.125

= 3.91

The final WinTach RPM Index for our example would be reported with its suffix as:

WinTach Word Processing RPM Index = 3.91/B8

The overall WinTach RPM Index is an equal waited average of the four benchmark components.


WinTach RPM Results Reporting

To better present all the information gathered by the WinTach benchmark components, WinTach is able to create and print a comprehensive report.  The WinTach RPM report contains the following information (example results are provided for completeness):

System Information:

System Processor	: 386

Display Driver	: TIGA

Display Type	: 32 Bit Coprocessor

Display Resolution	: 1280 x 1024

Display Colors	: 256

Test Resolution	: A8 (640x480)

Cursor Stability	: Good

Benchmark results:

Word Processing RPM	: 3.04 / A8

CAD / Draw RPM	: 6.96 / A8

Spreadsheet RPM	: 7.62 / A8

Paint RPM	: 7.28 / A8

Overall WinTach RPM	: 6.22 / A8


This is the RPM index, a relative Windows application performance indicator scaled to the current operating environment.


The ability of the video device to maintain a stable, continuously visible cursor shape regardless of the amount of video processing occurring.


The letter portion of the suffix indicates the system screen resolution:

A:	640x480

B:	800x600

C:	1024x768

D:	1280x1024


The numeric portion of the suffix indicates the number of system display colors:

 1:	1bpp (2 Colors)

 4:	4bpp (16 Colors)

 8:	8bpp (256 Colors)

16:	16bpp (Direct Color)

32:	32bpp (True Color)


In the above example, the prefix is "12.04", indicating a relative performance 12.04 times greater than a common reference.


In example above, the suffix is "B8", describing the operating environment as an 800x600 display board, capable of displaying 256 colors.


Standard System: Compaq Deskpro 386/20, Compaq 16bit VGA, using the standard Microsoft Windows Version 3.1 VGA display driver.


Target System: The system on which the benchmarks are being executed.


Display Bits: The number of bits of display memory required to hold the display image.


Report Menu
Reports Menu

This menu contains commands that let you view, print, and copy into the clipboard WinTach RPM Reports.  Report options are available only after all four of the WinTach application benchmark tests have been run, and the overall WinTach RPM index has been calculated.

Print Report

Print the current WinTach RPM report to your default printer.

Copy Report

Copy the current WinTach RPM report into the Windows clipboard.  The contents of the clipboard can then be pasted into other applications.


Benchmark Menu
Start Benchmark Menu

This menu provides an alternate method of starting a benchmark test in the case where no mouse is being used with Windows.  The normal method of starting a benchmark test is to move the mouse cursor over one of the four application icons on the main WinTach screen and press the left mouse button.

Word Processor

Start the Word Processor application benchmark test.

Cad/Draw

Start the Cad/Draw application benchmark test.

Spreadsheet

Start the Spreadsheet application benchmark test.

Paint Program

Start the Paint Program application benchmark test.

Overall RPM

Run each of the application benchmarks listed above in succession.

Run Continuous

Repeatedly run each of the application benchmarks in succession.  This option is useful for demonstration purposes.


Resolution Menu

This menu allows you to select the resolution at which the WinTach application benchmarks tests run.  The resolution selected is reported as part of the WinTach RPM index.  Possible resolutions are:

(A) 	640x480

(B) 	800x600

(C) 	1024x768

(D) 	1280x1024

Of the four possible resolutions, valid resolutions are those which are less than or equal to the resolution of your video driver (your display driver resolution is displayed in the System Configuration information on the main WinTach screen).  Valid resolutions are displayed as solid text.  Invalid resolutions are displayed as grayed text.

For example, if your video driver is running at 640x480, only the (A) 640x480 resolution is available.  If your video driver is running at 1024x768, then all resolutions except (D) 1280x1024 are available.


Tests Menu

This menu allows you to run other tests, which demonstrate non-performance aesthetics of your video board.

Cursor stability

This tests the ability of the video device to maintain a stable, continuously visible cursor shape regardless of the amount of video processing occurring. Many video displays require that the cursor be removed from the screen when it is in an area being updated by an application. This constant removal and replacement of the cursor during the screen update process causes a noticable and in many cases, annoying flicker if the video driver or hardware has no provision for synchronizing the updates with the display monitor refresh.

The amount of cursor flicker you see when running this test is indicative of what can be expected when running applications which perform a large amount of video output on your video board.  It is advantageous to have as little cursor flicker as possible.


Help Menu

This menu provides WinTach help information.

About WinTach

Display WinTach version number and copyright information.

WinTach Overview

Accesses WinTach's help system.


RPM: Relative Performance Measurement


API: Application Program Interface


Word Processing Application
Word Processing

Application Description:

Word processing applications are centered around the creation of formatted text documents, using an extensive variety of fonts, styles, and formatting options.

Although the largest percentage of time in a word processor is spent on text entry, video performance is most needed when large amounts of text are repeatedly redrawn.  An example of this would be page scrolling.

The primary graphics operation in word processing is the text output call.  When page scrolling, a page of text is drawn for each screen blit performed.  Even when line scrolling, a line of text is drawn for every one screen blit performed.

Operation Modeling:

The primary objective of the word processing application benchmark, is to measure the raw text draw speed of the Windows system.  All text blocks were rendered in one of the four common formats (left aligned, right aligned, centered, and justified).  All four formats were used equally during the benchmark.

The text blocks rendered included both single font / single style blocks, as well as multiple pages using a mixture of fonts, styles and alignments.  The goal of the benchmark was to balance raw text rendering capability with the ability to switch font styles and paragraph alignments on the fly.

Windows Operational Profile:

To better comprehend the operation of the Word Processing application benchmark, the operational profile is provided.  Application profiles vary according to application use, but a profile of an average word processing session should yield similar results.

Below is a breakdown of the Windows GDI calls made by the application benchmark during execution.  The API entry points are ranked by call frequency.

Windows API Name	Percentage of Calls
ExtTextOut	19.36%
SelectObject	16.42%
LineTo	16.41%
SetTextJustification	9.75%
MoveTo	8.29%
DeleteObject	6.02%
CreateFontIndirect	5.96%
GetCharWidth	5.89%
Rectangle	2.49%
BitBlt	2.19%


Cad / Draw Application
Cad / Draw

Application Description:

Cad applications are centered around the creation of technical drawings, using the computer's processing power to store and manipulate a large number of drawing objects.

Although the majority of time in a CAD application is spent entering drawing data, video performance is most needed when the drawing must be re-sized or scrolled.

The primary graphics operation used during a CAD session is the rendering of vector style graphics.  The actual graphics breakdown is dependent on the type of drawing used, but is usually a collection of lines and a variety of curves.  Filled shapes are not used extensively in typical CAD type drawings.

Operation Modeling:

CAD applications are typically display list driven.  This means the drawing is stored as a display list database, which is manipulated by the user until the drawing is complete.  The primary objective of the benchmark is to measure the speed at which the Windows system can visually manipulate a drawing database.  This includes both scaling and display list rendering.  Both zooms and pans are modeled in the benchmark.

Windows Operational Profile:

To better comprehend the operation of the CAD / Draw application benchmark, the operational profile is provided.  Application profiles vary according to application use, but a profile of an average CAD session should yield similar results.

Below is a breakdown of the Windows GDI calls made by the application benchmark during execution.  The API entry points are ranked by call frequency.

Windows API Name	Percentage of Calls
SelectObject	60.28%
Ellipse	17.92%
LineTo	9.99%
MoveTo	9.54%
Arc	0.86%
Rectangle	0.61%
SetTextColor	0.17%
TextOut	0.16%
BitBlt	0.09%
CreateCompatibleDC	0.09%


Spreadsheet Application
Spreadsheet

Application Description:

Spreadsheet applications are centered around the management of table driven data.  This includes both storage, manipulation, and both text and graphical reporting.

A large amount of time is spent entering data on the spreadsheet.  During this time, and at most other times, the primary operation performed is scrolling the sheet on the screen. high performance video is best appreciated during these scrolling operations.

Along with scrolling, spreadsheet applications are used for graphing the numerical data contained in the spreadsheet.  A variety of graphs may be used, but the standard bar graph is the most predominant.

Scrolling the spreadsheet is usually dominated by the screen blit function in the time domain, but as can be seen in the profile below, is made up mostly by calls to draw cells scrolled into the current view.

Operation Modeling:

The required functionality of the Spreadsheet application benchmark is obvious.  The application loads a small sample spreadsheet, does some quick numerical operations, and the proceeds to graph the results.  The session is ended by extensive scrolling of the sheet within the window.

Windows Operational Profile:

To better comprehend the operation of the Spreadsheet application benchmark, the operational profile is provided.  Application profiles vary according to application use, but a profile of an average spreadsheet session should yield similar results.

Below is a breakdown of the Windows GDI calls made by the application benchmark during execution.  The API entry points are ranked by call frequency.

Windows API Name	Percentage of Calls
SelectObject	28.40%
LineTo	21.28%
MoveTo	14.70%
SetBkMode	8.83%
SetTextColor	8.62%
Polygon	4.63%
Rectangle	3.32%
TextOut	2.58%
BitBlt	1.71%
DeleteObject	1.64%


Paint Application
Paint

Application Description:

Paint applications are primarily intended for creating and manipulating bitmapped images. These images are created using a variety of drawing tools. These tools allow various graphic primitives to be drawn directly into the bitmapped image. Common drawing operations include lines, rectangles, floodfills, pixels, and freehand drawing with rectangular and elliptical brushes.

While gathering data for the paint model, we found that less common operations, such as pie charts, chords, arcs, round rectangles, and bitmap insertion had increased variance across sample drawing sessions. We have therefore incorporated the testing of a wide variety of graphic operations using the average percentages determined across many sample sessions.

Although a large percentage of time is spent selecting and reselecting drawing tools, colors, and other drawing operands, video performance is significant when drawing single objects, and critical when performing an operation which requires a large number of repetitive graphics operations, such as drawing with a brush. For example, a single brush stroke can generate hundreds of graphic operations used to form the image.

Operation Modeling:

The Paint benchmark application measures the speed at which graphic operations can be rendered into a bitmapped image. It does this using 2 methods. Method 1 simulates a user drawing environment, in which the tool, color, and other parameters are selected between drawing operations. The result of this series of operations is a picture on the screen. Method 2 renders a variety of graphic operations and images into the bitmap real time. This simulates the wide variety of graphic operations found across paint sessions, and also models the repetitive graphic operations required for certain drawing operations, such as brush strokes discussed earlier.

Windows Operational Profile:

Below is a breakdown of the Windows GDI calls made by the application benchmark during execution.  The API entry points are ranked by call frequency.

Windows API Name	Percentage of Calls
SelectObject	63.17%
Rectangle	16.75%
Ellipse	2.96%
LineTo	1.86%
Polygon	1.42%
BitBlt	0.82%
Chord	0.72%
Pie	0.72%
SetTextColor	0.70%
TextOut	0.59%
*CreateCompatibleDC, DeleteDC, GetObject, GetMapMode, and SetMapMode, were all 1.00%.


Disclaimer
WinTach: Disclaimer

DISCLAIMER OF WARRANTY AND DAMAGES

Texas Instruments Incorporated ("TI") disclaims all warranties, whether express or implied, including but not limited to implied warranties of merchantability or fitness for a particular purpose.  In no event will TI be liable for special, collateral, incidental, or consequential damages in connection with or arising out of the use of this software.  These excluded damages include but are not limited to cost of removal or reinstallation, outside computer time, labor costs, loss of goodwill, loss of profits, loss of savings, or loss of use or interruption of business.

COPYRIGHT

Programs and Documentation Copyright 1992 Texas Instruments Inc.  This program may be freely copied and distributed in it's original form.


History
WinTach: Revision History

1.0	First Release, Comdex, March 1992

1.01	Informal Release

- Modified to show graphs in 'continuous run' mode

- Problem fixed in paint benchmark module when TrueType fonts disabled

1.2	Released, Comdex, October 1992

- Modification to accomodate larger RPMs on screen

- Minor internal and cosmetic bug fixes