Extend guidelines for benchmarking
# Benchmarking Guidelines
Please check the following guidelines for advice
-on how to get reliable benchmark results.
+on how to get reliable benchmark results when using BenchExec.
+For more general guidelines about benchmarking,
+e.g., what to do when implementing a benchmarking tool,
+please read our paper [Benchmarking and Resource Measurement](https://www.sosy-lab.org/%7Edbeyer/Publications/2015-SPIN.Benchmarking_and_Resource_Measurement.pdf).
### Use the latest version of BenchExec
New versions of BenchExec might contain features or bug fixes
@@ -11,6 +14,10 @@ that make benchmarking more precise.
The [container mode](container.md) available with `--container` since BenchExec 1.9
isolates individual runs and makes benchmarking more reliable.
+### Specify memory limit
+Without a fixed memory limit, the amount of memory available for benchmarking
+is non-deterministic.
+
### Check warnings of BenchExec
In certain situations, BenchExec will issue a warning during benchmarking,
e.g., if Turbo Boost is enabled, the system overheated etc.
