Add the Metrics API to the documentation
@@ -144,6 +144,58 @@ captured output. To interpret control characters like a console this would do:
ex.captured_out_filter = apply_backspaces_and_linefeeds
+Metrics API
+-----------------
+You might want to measure various values during your experiments, such as
+the progress of prediction accuracy over the training steps.
+
+Sacred supports tracking of numerical series (e.g. int, float) using the Metrics API.
+To access the API in experiments, the experiment must be running and the variable referencing the current run
+must be available in the scope. The ``_run.metrics.log_scalar_metric(metric_name, step, value)`` method takes
+a metric name (e.g. "training.loss"), the step in which the value was measured and the value itself.
+Step should be an integer describing the position of the value in the series. Steps can be numbered either sequentially
+0, 1, 2, 3, ... or they may be given a different meaning, for instance the current iteration round. This is useful
+when the logging occurs only every e.g. 10th iteration: The step can be first 10, then 20, etc.
+In any case, the numbers should form an increasing sequence.
+
+.. code-block:: python
+
+ @ex.automain
+ def example_metrics(_run):
+ counter = 0
+ while counter < 20:
+ counter+=1
+ value = counter
+ ms_to_wait = random.randint(5, 5000)
+ time.sleep(ms_to_wait/1000)
+ _run.metrics.log_scalar_metric("training.loss", counter, value * 1.5)
+ _run.metrics.log_scalar_metric("training.accuracy", counter, value * 2)
+
+
+Currently, the information is collected only by the :ref:`mongo_observer`. Metrics are stored in the metrics collection
+of MongoDB and are identified by their name (e.g. "training.loss") and the experiment run id they belong to.
+
+Metrics Records
+...............
+
+A metric record is composed of the metric name, the id of the corresponding experiment run,
+and of the measured values, arranged in an array in the order they were captured using the ``log_scalar_metric(...)``
+function.
+For the value located in the i-th index (``metric["y"][i]``),
+the step number can be found in ``metric["x"][i]`` and the time of the measurement in ``metric["timestamps"][i]``.
+
+ ================== =======================================================
+ Key Description
+ ================== =======================================================
+ ``_id`` Unique identifier
+ ``name`` The name of the metric (e.g. training.loss)
+ ``run_id`` The identifier of the run (``_id`` in the runs collection)
+ ``x`` Array of steps (e.g. ``[0, 1, 2, 3, 4]``)
+ ``y`` Array of measured values
+ ``timestamps`` Array of times of the individual
+ ================== =======================================================
+
+
Resources and Artifacts
=======================
It is possible to add files to an experiment, that will then be added to the database
