org.lionsoul.jcseg.tokenizer

Class ComplexSeg

java.lang.Object

org.lionsoul.jcseg.tokenizer.ASegment

org.lionsoul.jcseg.tokenizer.ComplexSeg

All Implemented Interfaces:

ISegment

public class ComplexSeg
extends ASegment

Jcseg complex segmentation implements extended from the ASegment class this will need the filter works of the four MMSeg rules:

• 1.maximum match chunk.
• 2.largest average word length.
• 3.smallest variance of words length.
• 4.largest sum of degree of morphemic freedom of one-character words.

Author:

chenxin

Field Summary

Fields inherited from class org.lionsoul.jcseg.tokenizer.ASegment

config, ctrlMask, dic, ialist, idx, isb, reader, wordPool

Fields inherited from interface org.lionsoul.jcseg.tokenizer.core.ISegment

CHECK_CE_MASk, CHECK_CF_MASK, START_SS_MASK

Constructor Summary

Constructors

Constructor and Description
ComplexSeg(JcsegTaskConfig config, ADictionary dic)
ComplexSeg(Reader input, JcsegTaskConfig config, ADictionary dic)

Method Summary

Methods

Modifier and Type	Method and Description
IChunk	getBestCJKChunk(char[] chars, int index) an abstract method to gain a CJK word from the current position.

Methods inherited from class org.lionsoul.jcseg.tokenizer.ASegment

appendLatinSyn, appendWordFeatures, enSecondSeg, findCHName, findCHName, getConfig, getDict, getNextCJKWord, getNextLatinWord, getNextMatch, getNextPunctuationPairWord, getPairPunctuationText, getStreamPosition, next, nextBasicLatin, nextCJKSentence, nextCNNumeric, nextLetterNumber, nextOtherNumber, pushBack, readNext, reset, setConfig, setDict

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ComplexSeg

public ComplexSeg(JcsegTaskConfig config,
          ADictionary dic)
           throws IOException

Throws:

IOException

ComplexSeg

public ComplexSeg(Reader input,
          JcsegTaskConfig config,
          ADictionary dic)
           throws IOException

Throws:

IOException

Method Detail

getBestCJKChunk

public IChunk getBestCJKChunk(char[] chars,
                     int index)
                       throws IOException

Description copied from class: ASegment

an abstract method to gain a CJK word from the current position. simpleSeg and ComplexSeg is different to deal this, so make it a abstract method here

Specified by:

getBestCJKChunk in class ASegment

Returns:

IChunk

Throws:

IOException

See Also:

ASegment.getBestCJKChunk(char[], int)