Uses of Interface
org.antlr.runtime.Token

Packages that use Token

Package	Description
org.antlr.runtime
org.antlr.runtime.debug
org.antlr.runtime.tree

Uses of Token in org.antlr.runtime

Classes in org.antlr.runtime that implement Token

Modifier and Type	Class and Description
class	ClassicToken A Token object like we'd use in ANTLR 2.x; has an actual string created and associated with this object.
class	CommonToken

Fields in org.antlr.runtime declared as Token

Modifier and Type	Field and Description
static Token	Token.INVALID_TOKEN
static Token	Token.SKIP_TOKEN In an action, a lexer rule can set token to this SKIP_TOKEN and ANTLR will avoid creating a token for this symbol and try to fetch another.
Token	ParserRuleReturnScope.start
Token	ParserRuleReturnScope.stop
Token	RecognizerSharedState.token The goal of all lexer rules/methods is to create a token object.
Token	RecognitionException.token The current Token when an error occurred.

Fields in org.antlr.runtime with type parameters of type Token

Modifier and Type	Field and Description
protected List<Token>	LegacyCommonTokenStream.tokens Record every single token pulled from the source so we can reproduce chunks of it later.
protected List<Token>	BufferedTokenStream.tokens Record every single token pulled from the source so we can reproduce chunks of it later.

Methods in org.antlr.runtime that return Token

Modifier and Type	Method and Description
Token	Lexer.emit() The standard method called to automatically emit a token at the outermost lexical rule.
Token	UnbufferedTokenStream.get(int i)
Token	TokenStream.get(int i) Get a token at an absolute index i; 0..n-1.
Token	LegacyCommonTokenStream.get(int i) Return absolute token i; ignore which channel the tokens are on; that is, count all tokens not just on-channel tokens.
Token	BufferedTokenStream.get(int i)
Token	Lexer.getEOFToken() Returns the EOF token (default), if you need to return a custom token instead override this method.
Token	UnwantedTokenException.getUnexpectedToken()
protected Token	LegacyCommonTokenStream.LB(int k) Look backwards k tokens on-channel tokens
protected Token	CommonTokenStream.LB(int k)
protected Token	BufferedTokenStream.LB(int k)
Token	TokenStream.LT(int k) Get Token at current input pointer + i ahead where i=1 is next Token.
Token	LegacyCommonTokenStream.LT(int k) Get the ith token from the current position 1..n where k=1 is the first symbol of lookahead.
Token	CommonTokenStream.LT(int k)
Token	BufferedTokenStream.LT(int k)
Token	UnbufferedTokenStream.nextElement()
Token	TokenSource.nextToken() Return a Token object from your input stream (usually a CharStream).
Token	Lexer.nextToken() Return a token from this source; i.e., match a token on the char stream.

Methods in org.antlr.runtime that return types with arguments of type Token

Modifier and Type	Method and Description
List<? extends Token>	LegacyCommonTokenStream.get(int start, int stop) Get all tokens from start..stop inclusively
List<? extends Token>	BufferedTokenStream.get(int start, int stop) Get all tokens from start..stop inclusively
List<? extends Token>	LegacyCommonTokenStream.getTokens()
List<? extends Token>	BufferedTokenStream.getTokens()
List<? extends Token>	LegacyCommonTokenStream.getTokens(int start, int stop)
List<? extends Token>	BufferedTokenStream.getTokens(int start, int stop)
List<? extends Token>	LegacyCommonTokenStream.getTokens(int start, int stop, BitSet types) Given a start and stop index, return a List of all tokens in the token type BitSet.
List<? extends Token>	BufferedTokenStream.getTokens(int start, int stop, BitSet types) Given a start and stop index, return a List of all tokens in the token type BitSet.
List<? extends Token>	LegacyCommonTokenStream.getTokens(int start, int stop, int ttype)
List<? extends Token>	BufferedTokenStream.getTokens(int start, int stop, int ttype)
List<? extends Token>	LegacyCommonTokenStream.getTokens(int start, int stop, List<Integer> types)
List<? extends Token>	BufferedTokenStream.getTokens(int start, int stop, List<Integer> types)

Methods in org.antlr.runtime with parameters of type Token

Modifier and Type	Method and Description
void	TokenRewriteStream.delete(String programName, Token from, Token to)
void	TokenRewriteStream.delete(Token indexT)
void	TokenRewriteStream.delete(Token from, Token to)
void	Lexer.emit(Token token) Currently does not support multiple emits per nextToken invocation for efficiency reasons.
String	BaseRecognizer.getTokenErrorDisplay(Token t) How should a token be displayed in an error message? The default is to display just the text, but during development you might want to have a lot of information spit out.
void	TokenRewriteStream.insertAfter(String programName, Token t, Object text)
void	TokenRewriteStream.insertAfter(Token t, Object text)
void	TokenRewriteStream.insertBefore(String programName, Token t, Object text)
void	TokenRewriteStream.insertBefore(Token t, Object text)
boolean	UnbufferedTokenStream.isEOF(Token o)
void	TokenRewriteStream.replace(String programName, Token from, Token to, Object text)
void	TokenRewriteStream.replace(Token indexT, Object text)
void	TokenRewriteStream.replace(Token from, Token to, Object text)
String	UnbufferedTokenStream.toString(Token start, Token stop)
String	TokenStream.toString(Token start, Token stop) Because the user is not required to use a token with an index stored in it, we must provide a means for two token objects themselves to indicate the start/end location.
String	LegacyCommonTokenStream.toString(Token start, Token stop)
String	BufferedTokenStream.toString(Token start, Token stop)

Method parameters in org.antlr.runtime with type arguments of type Token

Modifier and Type	Method and Description
List<String>	BaseRecognizer.toStrings(List<? extends Token> tokens) A convenience method for use most often with template rewrites.

Constructors in org.antlr.runtime with parameters of type Token

Constructor and Description
ClassicToken(Token oldToken)
CommonToken(Token oldToken)

Uses of Token in org.antlr.runtime.debug

Classes in org.antlr.runtime.debug that implement Token

Modifier and Type	Class and Description
static class	RemoteDebugEventSocketListener.ProxyToken

Fields in org.antlr.runtime.debug declared as Token

Modifier and Type	Field and Description
protected Token	Profiler.lastRealTokenTouchedInDecision

Methods in org.antlr.runtime.debug that return Token

Modifier and Type	Method and Description
Token	DebugTokenStream.get(int i)
Token	DebugTreeAdaptor.getToken(Object t)
Token	DebugTokenStream.LT(int i)

Methods in org.antlr.runtime.debug with parameters of type Token

Modifier and Type	Method and Description
void	DebugTreeAdaptor.addChild(Object t, Token child)
Object	DebugTreeAdaptor.becomeRoot(Token newRoot, Object oldRoot)
void	Profiler.consumeHiddenToken(Token token)
void	ParseTreeBuilder.consumeHiddenToken(Token token)
void	DebugEventSocketProxy.consumeHiddenToken(Token t)
void	DebugEventRepeater.consumeHiddenToken(Token token)
void	DebugEventListener.consumeHiddenToken(Token t) An off-channel input token was consumed.
void	DebugEventHub.consumeHiddenToken(Token token)
void	BlankDebugEventListener.consumeHiddenToken(Token token)
void	Profiler.consumeToken(Token token)
void	ParseTreeBuilder.consumeToken(Token token)
void	DebugEventSocketProxy.consumeToken(Token t)
void	DebugEventRepeater.consumeToken(Token token)
void	DebugEventListener.consumeToken(Token t) An input token was consumed; matched by any kind of element.
void	DebugEventHub.consumeToken(Token token)
void	BlankDebugEventListener.consumeToken(Token token)
Object	DebugTreeAdaptor.create(int tokenType, Token fromToken)
Object	DebugTreeAdaptor.create(int tokenType, Token fromToken, String text)
Object	DebugTreeAdaptor.create(Token payload)
void	TraceDebugEventListener.createNode(Object node, Token token)
void	DebugEventSocketProxy.createNode(Object node, Token token)
void	DebugEventRepeater.createNode(Object node, Token token)
void	DebugEventListener.createNode(Object node, Token token) Announce a new node built from an existing token.
void	DebugEventHub.createNode(Object node, Token token)
void	BlankDebugEventListener.createNode(Object node, Token token)
Object	DebugTreeAdaptor.errorNode(TokenStream input, Token start, Token stop, RecognitionException e)
void	Profiler.LT(int i, Token t) Track refs to lookahead if in a fixed/nonfixed decision.
void	DebugEventSocketProxy.LT(int i, Token t)
void	DebugEventRepeater.LT(int i, Token t)
void	DebugEventListener.LT(int i, Token t) Somebody (anybody) looked ahead.
void	DebugEventHub.LT(int index, Token t)
void	BlankDebugEventListener.LT(int i, Token t)
protected String	DebugEventSocketProxy.serializeToken(Token t)
void	DebugTreeAdaptor.setTokenBoundaries(Object t, Token startToken, Token stopToken)
String	DebugTokenStream.toString(Token start, Token stop)

Uses of Token in org.antlr.runtime.tree

Fields in org.antlr.runtime.tree declared as Token

Modifier and Type	Field and Description
Token	CommonErrorNode.start
Token	CommonErrorNode.stop
Token	CommonTree.token A single token is the payload

Fields in org.antlr.runtime.tree with type parameters of type Token

Modifier and Type	Field and Description
List<Token>	ParseTree.hiddenTokens

Methods in org.antlr.runtime.tree that return Token

Modifier and Type	Method and Description
Token	CommonTreeAdaptor.createToken(int tokenType, String text) Tell me how to create a token for use with imaginary token nodes.
abstract Token	BaseTreeAdaptor.createToken(int tokenType, String text) Tell me how to create a token for use with imaginary token nodes.
Token	CommonTreeAdaptor.createToken(Token fromToken) Tell me how to create a token for use with imaginary token nodes.
abstract Token	BaseTreeAdaptor.createToken(Token fromToken) Tell me how to create a token for use with imaginary token nodes.
Token	CommonTree.getToken()
Token	TreeAdaptor.getToken(Object t) Return the token object from which this node was created.
Token	CommonTreeAdaptor.getToken(Object t) What is the Token associated with this node? If you are not using CommonTree, then you must override this in your own adaptor.
Token	RewriteRuleTokenStream.nextToken()

Methods in org.antlr.runtime.tree with parameters of type Token

Modifier and Type	Method and Description
Object	TreeAdaptor.becomeRoot(Token newRoot, Object oldRoot) Create a node for newRoot make it the root of oldRoot.
Object	BaseTreeAdaptor.becomeRoot(Token newRoot, Object oldRoot)
Object	TreeAdaptor.create(int tokenType, Token fromToken) Create a new node derived from a token, with a new token type.
Object	BaseTreeAdaptor.create(int tokenType, Token fromToken)
Object	TreeAdaptor.create(int tokenType, Token fromToken, String text) Same as create(tokenType,fromToken) except set the text too.
Object	BaseTreeAdaptor.create(int tokenType, Token fromToken, String text)
Object	TreeWizard.TreePatternTreeAdaptor.create(Token payload)
Object	TreeAdaptor.create(Token payload) Create a tree node from Token object; for CommonTree type trees, then the token just becomes the payload.
Object	CommonTreeAdaptor.create(Token payload)
Token	CommonTreeAdaptor.createToken(Token fromToken) Tell me how to create a token for use with imaginary token nodes.
abstract Token	BaseTreeAdaptor.createToken(Token fromToken) Tell me how to create a token for use with imaginary token nodes.
Object	TreeAdaptor.errorNode(TokenStream input, Token start, Token stop, RecognitionException e) Return a tree node representing an error.
Object	BaseTreeAdaptor.errorNode(TokenStream input, Token start, Token stop, RecognitionException e) create tree node that holds the start and stop tokens associated with an error.
void	TreeAdaptor.setTokenBoundaries(Object t, Token startToken, Token stopToken) Where are the bounds in the input token stream for this node and all children? Each rule that creates AST nodes will call this method right before returning.
void	CommonTreeAdaptor.setTokenBoundaries(Object t, Token startToken, Token stopToken) Track start/stop token for subtree root created for a rule.

Constructors in org.antlr.runtime.tree with parameters of type Token

Constructor and Description
CommonErrorNode(TokenStream input, Token start, Token stop, RecognitionException e)
CommonTree(Token t)
TreeWizard.TreePattern(Token payload)
TreeWizard.WildcardTreePattern(Token payload)