Add documentation about turning off output capture
@@ -64,7 +64,7 @@ But it is also available from ``run.experiment_info['repositories']``, as a
list of dictionaries of the form
``{'url': URL, 'commit': COMMIT_HASH, 'dirty': True}``.
-To disable this, pass ``save_git_info=False`` to the ``Experiement``
+To disable this, pass ``save_git_info=False`` to the ``Experiment``
or ``Ingredient`` constructor.
@@ -160,7 +160,7 @@ Note that, the captured output behaves differently from a console in that
it doesn't by default interpret control characters like backspace
(``'\b'``) or carriage return (``'\r'``).
As an effect, some updating progressbars or the like might me more verbose
-than intended. This behaviour can be changed by adding a custom filter to the
+than intended. This behavior can be changed by adding a custom filter to the
captured output. To interpret control characters like a console this would do:
.. code-block:: python
@@ -169,6 +169,16 @@ captured output. To interpret control characters like a console this would do:
ex.captured_out_filter = apply_backspaces_and_linefeeds
+Long running and verbose experiments can overload the observer's
+storage backend. For example, the MongoObserver is limited to
+16 MB per run, which can result in experiments being unexpectedly
+terminated. To avoid this you can turn of output capturing by
+applying a custom filter like so
+
+.. code-block:: python
+
+ ex.captured_out_filter = lambda captured_output: "Output capturing turned off."
+
Metrics API
-----------
