Updated random noise generation tutorial to remove logarithm from recommentations.
Logarithm of timestamp will return 35 for next many years thus not suitable for random seed generation :|
"id": "TnyhlV0fIxYR"
},
"source": [
- "A general recommendation in TFF is to use the functional `tf.random.stateless_*` functions for random noise generation. These functions take `seed` as an explicit input argument to generate random noise. We first define a helper class to maintain the seed as pseudo state. The helper `RandomSeedGenerator` has functional operators in a state-in-state-out fashion. It is reasonable to use a counter as pseudo state for `tf.random.stateless_*` as these functions scramble the seed before using it to make noises generated by correlated seeds statistically uncorrelated."
+ "A general recommendation in TFF is to use the functional `tf.random.stateless_*` functions for random noise generation. These functions take `seed` (a Tensor with shape `[2]` or a `tuple` of two scalar tensors) as an explicit input argument to generate random noise. We first define a helper class to maintain the seed as pseudo state. The helper `RandomSeedGenerator` has functional operators in a state-in-state-out fashion. It is reasonable to use a counter as pseudo state for `tf.random.stateless_*` as these functions [scramble](https://github.com/tensorflow/tensorflow/blob/919f693420e35d00c8d0a42100837ae3718f7927/tensorflow/core/kernels/stateless_random_ops.cc#L50-L64) the seed before using it to make noises generated by correlated seeds statistically uncorrelated."
]
},
{
},
"outputs": [],
"source": [
+ "def timestamp_seed():\n",
+ " # tf.timestamp returns microseconds as decimal places, thus scaling by 1e6.\n",
+ " return tf.math.cast(tf.timestamp() * 1e6, tf.int64)\n",
+ "\n",
"class RandomSeedGenerator():\n",
"\n",
" def initialize(self, seed=None):\n",
" if seed is None:\n",
- " return tf.cast(tf.stack(\n",
- " [tf.math.floor(tf.timestamp()*1e6),\n",
- " tf.math.floor(tf.math.log(tf.timestamp()*1e6))]), dtype=tf.int64)\n",
+ " return tf.stack([timestamp_seed(), 0])\n",
" else:\n",
" return tf.constant(self.seed, dtype=tf.int64, shape=(2,))\n",
"\n",
" def next(self, state):\n",
- " return state + 1\n",
+ " return state + tf.constant([0, 1], tf.int64)\n",
"\n",
" def structure_next(self, state, nest_structure):\n",
- " \"Returns seed in nexted structure and the next state seed.\"\n",
+ " \"Returns seed in nested structure and the next state seed.\"\n",
" flat_structure = tf.nest.flatten(nest_structure)\n",
- " flat_seeds = [state+i for i in range(len(flat_structure))]\n",
+ " flat_seeds = [state + tf.constant([0, i], tf.int64) for\n",
+ " i in range(len(flat_structure))]\n",
" nest_seeds = tf.nest.pack_sequence_as(nest_structure, flat_seeds)\n",
- " return nest_seeds, flat_seeds[-1] + 1"
+ " return nest_seeds, flat_seeds[-1] + tf.constant([0, 1], tf.int64)"
]
},
{
