Surveytools

This wiki page documents the script suite inside the surveytools folder.

• survey is the main entrance script, able to invoke the functionality of the various sub-modules, as decided by issuing command line switches, the most notable being:

  from --usage:

Collects and plots all survey data

Invoke with one or more of the following flags:
  --clean         Clean up the various build files
  --setup         Setup necessary GHC installations and build benchmarks
  --benchmark     Runs benchmarks and generates graphs
  --plot          Plot the lastest benchmark execution
  --revdeps       Count reverse dependencies
  --thebigbutton  Alias for '--setup --benchmark --plot'
  --nightly       Alias for '-- clean --setup --benchmark --plot'
  --usage

• The setup folder hosts the setup script install-hsenv, that prepares execution environments, hsenvs, to compile the benchmarks inside. To do this it uses a small haskell utility called shaketool (residing in setup/shaketool), which is capable of cloning/pulling git repositories and installing cabal packages.

• the benchmarkrunner folder hosts the benchmark execution code, consisting of two parts: an executable benchmarkrunner, and the script run.sh. The benchmarkrunner program is a thin wrapper around Criterion, that runs our benchmarks. Any program that adheres to our simple out:OK - (in:<input> - out:RESULT <output>)* - in:EXIT - out:OK protocol on stdin and stdout are benchmarkable by the benchmarkrunner.

  The run.sh script is invoked as run.sh <benchmark> <platform> <logtag>, eg. run.sh binomial-gpu Accelerate manualtest to run the Accelerate implementation of the binomial-gpu benchmark. It works by sourcing the files benchmarks/<benchmark>/<platform>/{setup,run}.sh (with obvious purposes). This file is provided a shell environment with functions defined that facilitate building cabal files inside hsenvs, and running the resulting executables with consistent benchmark parameters.

  Thus, every benchmark must have local run.sh and setup.sh scripts to be executable by the surveytools infrastructure.