Change JA inflection separator to semicolon
Hyphen is unsuitable because of interactions with the JA data fields,
but pipe is also unsuitable because it has a different meaning in UD
data, so it's better to use something that has no significance in either
case. So this uses semicolon.
@@ -94,7 +94,7 @@ class JapaneseTokenizer(DummyTokenizer):
DetailedToken(
token.surface(), # orth
"-".join([xx for xx in token.part_of_speech()[:4] if xx != "*"]), # tag
- "|".join([xx for xx in token.part_of_speech()[4:] if xx != "*"]), # inf
+ ";".join([xx for xx in token.part_of_speech()[4:] if xx != "*"]), # inf
token.dictionary_form(), # lemma
token.normalized_form(),
token.reading_form(),
