Go to the next, previous, or main section.

Usage

Once you have compiled the benchmark, using it is fairly straightforward. You simply run the program, passing command-line parameters that specify what will be benchmarked (see below). (Alternatively, if you don't pass any command-line parameters, you will be prompted interactively for all of the options.)

Benchmark Output

The benchmark output is directed to two files: bench.options.dat and bench.options.log. Here, options is an abbreviation of the options describing the benchmark run (essentially, it is a concatenation of the relevant command-line parameters). (Alternatively, you can enforce the DOS 8.3 filename restrictions by using the "-dos" command-line switch.)

bench.options.dat is a comma-delimited text file containing the results in tabular form, suitable for import into a spreadsheet. (We use commas rather than tabs because tabs have an annoying tendency to get converted into spaces, especially by email programs.) The first column is the transform size, and subsequent columns are the speed or accuracy results for each transform. (Speed is measured in "mflops" and accuracy is the mean fractional error. See the methodology section for an explanation of these quantities.)

At the top of each column is the name of the FFT corresponding to the data in the column. These names are, in most cases, the last name of the author of the code. A complete listing of the FFTs in the benchmark, along with links to where they may be downloaded, can be found in the FFT Software section of this manual.

Average Performance

When transform speed is being measured, the last line of bench.options.dat contains "normalized average" speeds for all the FFTs. This is a kind of summary of the results, and is separated by a blank line from the rest of the data. (We caution users, however, that average results are probably not as useful as the results for a particular problem size of interest.) The average is computed as follows:

  1. For each transform size, the speed results are normalized with respect to the fastest FFT for that size. (i.e. the fastest becomes 1.0, an FFT half as fast becomes 0.5, etcetera.)
  2. These normalized results are then averaged over all transform sizes for each FFT. This is what is reported in the last line of the data file.

bench.options.log contains verbose descriptions of the benchmark computations. (It is mainly intended for us, the developers, as a debugging tool and as an aid in understanding the results that other people might send us.)

Command-Line Options

The following are the command-line options that may be used to control the behavior of the benchmark:

-h
Prints a help message describing the command-line parameters.

-submit
This option is for submitting benchmark results to the benchFFT repository. See the section "Submitting benchFFT Results" for more information.

-name s
Make s the root name for the output files (the default is "bench"). The output files are then s.options.dat and s.options.log. We usually like to choose a name that describes the machine we are running on (e.g. "trs80", "eniac", or "ppc686").

-dos
Follow the DOS 8.3 file naming restrictions when creating output files. This makes the output file names considerably more cryptic, so we don't recommend it unless you are using DOS (which we don't recommend either).

-1d
Benchmark one-dimensional complex-complex transforms.

-3d
Benchmark three-dimensional complex-complex transforms.

You must specify either -1d or -3d; otherwise, no benchmark will be performed. You may specify both.


-s
Measure the speed of the transforms (the default).

-a
Measure the accuracy of the transforms, rather than their speed.

-p2
Only benchmark transform sizes that are powers of two.

-np2
Only benchmark transform sizes that are not powers of two.

-pa
Benchmark all transform sizes (the default).

-size N
Only benchmark transforms of size N (for 3D transforms, NxNxN). (In this case, output will only be to stdout.) The default is to benchmark a hard-coded range of representative sizes. If this option is used, the -p2, etcetera, options are ignored.

-maxmem MB
Use approximately MB megabytes of memory, at most (actually, a bit more memory than this will be used). The default is 50 megabytes. This determines the maximum transform size that is benchmarked by the program. (We usually set it to 10-20% below the actual amount of memory available.)

Go to the next, previous, or main section.