Update Tokenizer documentation to reflect token_match and url_match signatures
@@ -45,10 +45,12 @@ cdef class Tokenizer:
`re.compile(string).search` to match suffixes.
`infix_finditer` (callable): A function matching the signature of
`re.compile(string).finditer` to find infixes.
- token_match (callable): A boolean function matching strings to be
+ token_match (callable): A function matching the signature of
+ `re.compile(string).match`, for matching strings to be
recognized as tokens.
- url_match (callable): A boolean function matching strings to be
- recognized as tokens after considering prefixes and suffixes.
+ url_match (callable): A function matching the signature of
+ `re.compile(string).match`, for matching strings to be
+ recognized as urls.
EXAMPLE:
>>> tokenizer = Tokenizer(nlp.vocab)
