org.antlr.runtime.debug
Class Profiler

java.lang.Object
  org.antlr.runtime.debug.BlankDebugEventListener
      org.antlr.runtime.debug.Profiler

All Implemented Interfaces:

DebugEventListener

public class Profiler
extends BlankDebugEventListener

Using the debug event interface, track what is happening in the parser and record statistics about the runtime.

Nested Class Summary

static class	Profiler.DecisionDescriptor
static class	Profiler.DecisionEvent
static class	Profiler.ProfileStats

Field Summary

protected int	backtrackDepth
protected Stack<String>	currentGrammarFileName
protected Stack<Integer>	currentLine
protected Stack<Integer>	currentPos
protected Stack<String>	currentRuleName
static String	DATA_SEP
protected List<Profiler.DecisionEvent>	decisionEvents
protected DoubleKeyMap<String,Integer,Profiler.DecisionDescriptor>	decisions
protected Stack<Profiler.DecisionEvent>	decisionStack
protected Token	lastRealTokenTouchedInDecision
static String	newline
DebugParser	parser Ack, should not store parser; can't do remote stuff.
protected int	ruleLevel
static String	RUNTIME_STATS_FILENAME
protected Set<String>	uniqueRules
static String	Version Because I may change the stats, I need to track that for later computations to be consistent.

Fields inherited from interface org.antlr.runtime.debug.DebugEventListener

FALSE, PROTOCOL_VERSION, TRUE

Constructor Summary

Profiler()

Profiler(DebugParser parser)

Method Summary

void	beginBacktrack(int level) Track backtracking decisions.
void	consumeHiddenToken(Token token) An off-channel input token was consumed.
void	consumeToken(Token token) An input token was consumed; matched by any kind of element.
protected Profiler.DecisionEvent	currentDecision()
void	endBacktrack(int level, boolean successful) Successful or not, track how much lookahead synpreds use
void	enterDecision(int decisionNumber, boolean couldBacktrack) Every decision, fixed k or arbitrary, has an enter/exit event so that a GUI can easily track what LT/consume events are associated with prediction.
void	enterRule(String grammarFileName, String ruleName) The parser has just entered a rule.
void	examineRuleMemoization(IntStream input, int ruleIndex, int stopIndex, String ruleName) Track memoization; this is not part of standard debug interface but is triggered by profiling.
void	exitDecision(int decisionNumber)
void	exitRule(String grammarFileName, String ruleName) This is the last thing executed before leaving a rule.
List	getDecisionEvents()
DoubleKeyMap	getDecisionStats()
String	getDecisionStatsDump()
int	getNumberOfHiddenTokens(int i, int j) Get num hidden tokens between i..j inclusive
Profiler.ProfileStats	getReport()
boolean	inDecision() The parser is in a decision if the decision depth > 0.
void	location(int line, int pos) To watch a parser move through the grammar, the parser needs to inform the debugger what line/charPos it is passing in the grammar.
protected String	locationDescription()
protected String	locationDescription(String file, String rule, int line, int pos)
void	LT(int i, Token t) Track refs to lookahead if in a fixed/nonfixed decision.
void	mark(int i) The parser is going to look arbitrarily ahead; mark this location, the token stream's marker is sent in case you need it.
void	memoize(IntStream input, int ruleIndex, int ruleStartIndex, String ruleName) Warning: doesn't track success/failure, just unique recording event
void	recognitionException(RecognitionException e) A recognition exception occurred such as NoViableAltException.
void	rewind() Rewind to the input position of the last marker.
void	rewind(int i) After an arbitrairly long lookahead as with a cyclic DFA (or with any backtrack), this informs the debugger that stream should be rewound to the position associated with marker.
void	semanticPredicate(boolean result, String predicate) A semantic predicate was evaluate with this result and action text
void	setParser(DebugParser parser)
void	terminate() Parsing is over; successfully or not.
protected int[]	toArray(List a)
String	toNotifyString()
String	toString()
static String	toString(Profiler.ProfileStats stats)
protected int[]	trim(int[] X, int n)

Methods inherited from class org.antlr.runtime.debug.BlankDebugEventListener

addChild, becomeRoot, beginResync, commence, consumeNode, createNode, createNode, endResync, enterAlt, enterSubRule, errorNode, exitSubRule, LT, nilNode, setTokenBoundaries

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DATA_SEP

public static final String DATA_SEP

See Also:

Constant Field Values

newline

public static final String newline

Version

public static final String Version

Because I may change the stats, I need to track that for later computations to be consistent.

See Also:

Constant Field Values

RUNTIME_STATS_FILENAME

public static final String RUNTIME_STATS_FILENAME

See Also:

Constant Field Values

parser

public DebugParser parser

Ack, should not store parser; can't do remote stuff. Well, we pass input stream around too so I guess it's ok.

ruleLevel

protected int ruleLevel

lastRealTokenTouchedInDecision

protected Token lastRealTokenTouchedInDecision

uniqueRules

protected Set<String> uniqueRules

currentGrammarFileName

protected Stack<String> currentGrammarFileName

currentRuleName

protected Stack<String> currentRuleName

currentLine

protected Stack<Integer> currentLine

currentPos

protected Stack<Integer> currentPos

decisions

protected DoubleKeyMap<String,Integer,Profiler.DecisionDescriptor> decisions

decisionEvents

protected List<Profiler.DecisionEvent> decisionEvents

decisionStack

protected Stack<Profiler.DecisionEvent> decisionStack

backtrackDepth

protected int backtrackDepth

Constructor Detail

Profiler

public Profiler()

Profiler

public Profiler(DebugParser parser)

Method Detail

enterRule

public void enterRule(String grammarFileName,
                      String ruleName)

Description copied from interface: DebugEventListener

The parser has just entered a rule. No decision has been made about which alt is predicted. This is fired AFTER init actions have been executed. Attributes are defined and available etc... The grammarFileName allows composite grammars to jump around among multiple grammar files.

Specified by:

enterRule in interface DebugEventListener

Overrides:

enterRule in class BlankDebugEventListener

exitRule

public void exitRule(String grammarFileName,
                     String ruleName)

Description copied from interface: DebugEventListener

This is the last thing executed before leaving a rule. It is executed even if an exception is thrown. This is triggered after error reporting and recovery have occurred (unless the exception is not caught in this rule). This implies an "exitAlt" event. The grammarFileName allows composite grammars to jump around among multiple grammar files.

Specified by:

exitRule in interface DebugEventListener

Overrides:

exitRule in class BlankDebugEventListener

examineRuleMemoization

public void examineRuleMemoization(IntStream input,
                                   int ruleIndex,
                                   int stopIndex,
                                   String ruleName)

Track memoization; this is not part of standard debug interface but is triggered by profiling. Code gen inserts an override for this method in the recognizer, which triggers this method. Called from alreadyParsedRule().

memoize

public void memoize(IntStream input,
                    int ruleIndex,
                    int ruleStartIndex,
                    String ruleName)

Warning: doesn't track success/failure, just unique recording event

location

public void location(int line,
                     int pos)

Description copied from interface: DebugEventListener

To watch a parser move through the grammar, the parser needs to inform the debugger what line/charPos it is passing in the grammar. For now, this does not know how to switch from one grammar to the other and back for island grammars etc... This should also allow breakpoints because the debugger can stop the parser whenever it hits this line/pos.

Specified by:

location in interface DebugEventListener

Overrides:

location in class BlankDebugEventListener

enterDecision

public void enterDecision(int decisionNumber,
                          boolean couldBacktrack)

Description copied from interface: DebugEventListener

Every decision, fixed k or arbitrary, has an enter/exit event so that a GUI can easily track what LT/consume events are associated with prediction. You will see a single enter/exit subrule but multiple enter/exit decision events, one for each loop iteration.

Specified by:

enterDecision in interface DebugEventListener

Overrides:

enterDecision in class BlankDebugEventListener

exitDecision

public void exitDecision(int decisionNumber)

Specified by:

exitDecision in interface DebugEventListener

Overrides:

exitDecision in class BlankDebugEventListener

consumeToken

public void consumeToken(Token token)

Description copied from interface: DebugEventListener

An input token was consumed; matched by any kind of element. Trigger after the token was matched by things like match(), matchAny().

Specified by:

consumeToken in interface DebugEventListener

Overrides:

consumeToken in class BlankDebugEventListener

inDecision

public boolean inDecision()

The parser is in a decision if the decision depth > 0. This works for backtracking also, which can have nested decisions.

consumeHiddenToken

public void consumeHiddenToken(Token token)

Description copied from interface: DebugEventListener

An off-channel input token was consumed. Trigger after the token was matched by things like match(), matchAny(). (unless of course the hidden token is first stuff in the input stream).

Specified by:

consumeHiddenToken in interface DebugEventListener

Overrides:

consumeHiddenToken in class BlankDebugEventListener

LT

public void LT(int i,
               Token t)

Track refs to lookahead if in a fixed/nonfixed decision.

Specified by:

LT in interface DebugEventListener

Overrides:

LT in class BlankDebugEventListener

beginBacktrack

public void beginBacktrack(int level)

Track backtracking decisions. You'll see a fixed or cyclic decision and then a backtrack. enter rule ... enter decision LA and possibly consumes (for cyclic DFAs) begin backtrack level mark m rewind m end backtrack level, success exit decision ... exit rule

Specified by:

beginBacktrack in interface DebugEventListener

Overrides:

beginBacktrack in class BlankDebugEventListener

endBacktrack

public void endBacktrack(int level,
                         boolean successful)

Successful or not, track how much lookahead synpreds use

Specified by:

endBacktrack in interface DebugEventListener

Overrides:

endBacktrack in class BlankDebugEventListener

mark

public void mark(int i)

Description copied from interface: DebugEventListener

The parser is going to look arbitrarily ahead; mark this location, the token stream's marker is sent in case you need it.

Specified by:

mark in interface DebugEventListener

Overrides:

mark in class BlankDebugEventListener

rewind

public void rewind(int i)

Description copied from interface: DebugEventListener

After an arbitrairly long lookahead as with a cyclic DFA (or with any backtrack), this informs the debugger that stream should be rewound to the position associated with marker.

Specified by:

rewind in interface DebugEventListener

Overrides:

rewind in class BlankDebugEventListener

rewind

public void rewind()

Description copied from interface: DebugEventListener

Rewind to the input position of the last marker. Used currently only after a cyclic DFA and just before starting a sem/syn predicate to get the input position back to the start of the decision. Do not "pop" the marker off the state. mark(i) and rewind(i) should balance still.

Specified by:

rewind in interface DebugEventListener

Overrides:

rewind in class BlankDebugEventListener

currentDecision

protected Profiler.DecisionEvent currentDecision()

recognitionException

public void recognitionException(RecognitionException e)

Description copied from interface: DebugEventListener

A recognition exception occurred such as NoViableAltException. I made this a generic event so that I can alter the exception hierachy later without having to alter all the debug objects. Upon error, the stack of enter rule/subrule must be properly unwound. If no viable alt occurs it is within an enter/exit decision, which also must be rewound. Even the rewind for each mark must be unwount. In the Java target this is pretty easy using try/finally, if a bit ugly in the generated code. The rewind is generated in DFA.predict() actually so no code needs to be generated for that. For languages w/o this "finally" feature (C++?), the target implementor will have to build an event stack or something. Across a socket for remote debugging, only the RecognitionException data fields are transmitted. The token object or whatever that caused the problem was the last object referenced by LT. The immediately preceding LT event should hold the unexpected Token or char. Here is a sample event trace for grammar: b : C ({;}A|B) // {;} is there to prevent A|B becoming a set | D ; The sequence for this rule (with no viable alt in the subrule) for input 'c c' (there are 3 tokens) is: commence LT(1) enterRule b location 7 1 enter decision 3 LT(1) exit decision 3 enterAlt1 location 7 5 LT(1) consumeToken [c/<4>,1:0] location 7 7 enterSubRule 2 enter decision 2 LT(1) LT(1) recognitionException NoViableAltException 2 1 2 exit decision 2 exitSubRule 2 beginResync LT(1) consumeToken [c/<4>,1:1] LT(1) endResync LT(-1) exitRule b terminate

Specified by:

recognitionException in interface DebugEventListener

Overrides:

recognitionException in class BlankDebugEventListener

semanticPredicate

public void semanticPredicate(boolean result,
                              String predicate)

Description copied from interface: DebugEventListener

A semantic predicate was evaluate with this result and action text

Specified by:

semanticPredicate in interface DebugEventListener

Overrides:

semanticPredicate in class BlankDebugEventListener

terminate

public void terminate()

Description copied from interface: DebugEventListener

Parsing is over; successfully or not. Mostly useful for telling remote debugging listeners that it's time to quit. When the rule invocation level goes to zero at the end of a rule, we are done parsing.

Specified by:

terminate in interface DebugEventListener

Overrides:

terminate in class BlankDebugEventListener

setParser

public void setParser(DebugParser parser)

toNotifyString

public String toNotifyString()

toString

public String toString()

Overrides:

toString in class Object

getReport

public Profiler.ProfileStats getReport()

getDecisionStats

public DoubleKeyMap getDecisionStats()

getDecisionEvents

public List getDecisionEvents()

toString

public static String toString(Profiler.ProfileStats stats)

getDecisionStatsDump

public String getDecisionStatsDump()

trim

protected int[] trim(int[] X,
                     int n)

toArray

protected int[] toArray(List a)

getNumberOfHiddenTokens

public int getNumberOfHiddenTokens(int i,
                                   int j)

Get num hidden tokens between i..j inclusive

locationDescription

protected String locationDescription()

locationDescription

protected String locationDescription(String file,
                                     String rule,
                                     int line,
                                     int pos)