Skip to content

Commit fece12a

Browse files
committed
OPENNLP-1928: Document compatibility mode configuration and scope
1 parent ed2d55c commit fece12a

2 files changed

Lines changed: 29 additions & 9 deletions

File tree

opennlp-docs/src/docbkx/introduction.xml

Lines changed: 22 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -398,12 +398,28 @@ Arguments description:
398398
</screen>
399399
</para>
400400
<para>
401-
Accepted values are <code>LEGACY</code> and <code>CURRENT</code> (the default),
402-
case-insensitive; any other value throws an
403-
<code>IllegalArgumentException</code>. The setting is independent of
404-
<code>opennlp.whitespace.mode</code>, applies process-wide, so training and
405-
decoding of a given model must use the same value, and is scheduled for
406-
removal in 4.0.
401+
Accepted values are <code>LEGACY</code> and <code>CURRENT</code>,
402+
case-insensitive. An unset or blank property selects <code>CURRENT</code>;
403+
invalid values fail mode initialization. Set the property before OpenNLP
404+
initializes the mode, and use the same setting for training and inference.
405+
Legacy support is scheduled for removal in 4.0; selecting it logs a warning
406+
once per process.
407+
</para>
408+
<para>
409+
This setting affects only classes that explicitly consult
410+
<code>CompatibilityMode</code>. It does not restore every behavior of an
411+
earlier release: parser bracket spacing, corpus validation, and stream
412+
resource fixes are not switched back. It is independent of
413+
<code>opennlp.whitespace.mode</code>, which selects the whitespace definition.
414+
Selecting one does not select the other.
415+
</para>
416+
<para>
417+
Embedders can override the process-wide setting with
418+
<code>CompatibilityMode.setActive</code>; <code>reset()</code> reads the
419+
system property again. These are not per-model settings. Configure them
420+
before constructing or using model components, and do not switch them while
421+
requests are running. Record both modes with the training configuration;
422+
loading a model does not select them automatically.
407423
</para>
408424
</section>
409425
</section>

opennlp-docs/src/docbkx/langdetect.xml

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -52,9 +52,13 @@ under the License.
5252
Plane (U+10000 and above) with a single space, which removes most emoji
5353
but also other supplementary characters such as CJK Extension B
5454
ideographs. Since 3.0.0 hyphens and Basic Multilingual Plane characters
55-
are kept; models trained with an earlier release were built with them
56-
blanked and get that output again under
57-
<code>-Dopennlp.compat.mode=LEGACY</code> until they are retrained.
55+
are kept, as are unpaired surrogates. Under
56+
<code>-Dopennlp.compat.mode=LEGACY</code>, each run of ASCII hyphens
57+
and code points U+D83C through U+10FC00 becomes one space instead;
58+
code points above U+10FC00 are kept. This reproduces the earlier
59+
input transformation for models that depend on it. Neither mode
60+
is an emoji-only filter. Keep the mode consistent between training
61+
and inference; see <xref linkend="intro.compatibility.mode"/>.
5862
Deprecated; use EmojiToEmoticonCharSequenceNormalizer to keep
5963
emoji as signal.</entry>
6064
</row>

0 commit comments

Comments
 (0)