DOC improve strip_accents relevance to non-Roman scripts
@@ -371,12 +371,16 @@ class HashingVectorizer(BaseEstimator, VectorizerMixin, TransformerMixin):
values are 'ignore' and 'replace'.
strip_accents : {'ascii', 'unicode', None}
- Remove accents during the preprocessing step.
+ Remove accents and perform other character normalization
+ during the preprocessing step.
'ascii' is a fast method that only works on characters that have
an direct ASCII mapping.
'unicode' is a slightly slower method that works on any characters.
None (default) does nothing.
+ Both 'ascii' and 'unicode' use NFKD normalization from
+ :func:`unicodedata.normalize`.
+
analyzer : string, {'word', 'char', 'char_wb'} or callable
Whether the feature should be made of word or character n-grams.
Option 'char_wb' creates character n-grams only from text inside
@@ -575,12 +579,16 @@ class CountVectorizer(BaseEstimator, VectorizerMixin):
values are 'ignore' and 'replace'.
strip_accents : {'ascii', 'unicode', None}
- Remove accents during the preprocessing step.
+ Remove accents and perform other character normalization
+ during the preprocessing step.
'ascii' is a fast method that only works on characters that have
an direct ASCII mapping.
'unicode' is a slightly slower method that works on any characters.
None (default) does nothing.
+ Both 'ascii' and 'unicode' use NFKD normalization from
+ :func:`unicodedata.normalize`.
+
analyzer : string, {'word', 'char', 'char_wb'} or callable
Whether the feature should be made of word or character n-grams.
Option 'char_wb' creates character n-grams only from text inside
@@ -1167,12 +1175,16 @@ class TfidfVectorizer(CountVectorizer):
values are 'ignore' and 'replace'.
strip_accents : {'ascii', 'unicode', None}
- Remove accents during the preprocessing step.
+ Remove accents and perform other character normalization
+ during the preprocessing step.
'ascii' is a fast method that only works on characters that have
an direct ASCII mapping.
'unicode' is a slightly slower method that works on any characters.
None (default) does nothing.
+ Both 'ascii' and 'unicode' use NFKD normalization from
+ :func:`unicodedata.normalize`.
+
analyzer : string, {'word', 'char'} or callable
Whether the feature should be made of word or character n-grams.
