Overview   Project   Class   Tree   Deprecated   Index 
Grammatica 1.4 Documentation
FRAMES    NO FRAMES
SUMMARY:  INNER | FIELD | CONSTR | METHOD

PerCederberg.Grammatica.Parser
Class Token

System.Object
   |
   +--Node
         |
         +--Token

   in Token.cs

class Token
extends Node

A token node. This class represents a token (i.e. a set of adjacent characters) in a parse tree. The tokens are created by a tokenizer, that groups characters together into tokens according to a set of token patterns.


Constructor Summary
Token( TokenPattern pattern, string image, int line, int col )
          Creates a new token.
 
Method Summary
 override int GetEndColumn()
          The column number of the last character in the token image.
 override int GetEndLine()
          The line number of the last character in the token image.
 override int GetId()
          Returns the token (pattern) id.
 string GetImage()
          Returns the token image (i.e. the characters).
 override string GetName()
          Returns the token node name.
 Token GetNextToken()
          Returns the next token.
 internal TokenPattern GetPattern()
          Returns the token pattern.
 Token GetPreviousToken()
          Returns the previuos token.
 override int GetStartColumn()
          The column number of the first character in the token image.
 override int GetStartLine()
          The line number of the first character in the token image.
 internal void SetNextToken( Token next )
          Sets the next token in the token list.
 internal void SetPreviousToken( Token previous )
          Sets the previous token in the token list.
 string ToShortString()
          Returns a short string representation of this token.
 override string ToString()
          Returns a string representation of this token.
   
Methods inherited from class Node
IsHidden, GetId, GetName, GetStartLine, GetStartColumn, GetEndLine, GetEndColumn, GetParent, SetParent, GetChildCount, GetChildAt, GetDescendantCount, GetValueCount, GetValue, GetAllValues, AddValue, AddValues, RemoveAllValues, PrintTo
 

Constructor Detail

Token

public Token( TokenPattern pattern, string image, int line, int col );
Creates a new token.
Parameters:
pattern - the token pattern
image - the token image (i.e. characters)
line - the line number of the first character
col - the column number of the first character


Method Detail

GetEndColumn

public override int GetEndColumn();
The column number of the last character in the token image.
Returns:
the column number of the last token character

GetEndLine

public override int GetEndLine();
The line number of the last character in the token image.
Returns:
the line number of the last token character

GetId

public override int GetId();
Returns the token (pattern) id. This value is set as a unique identifier when creating the token pattern to simplify later identification.
Returns:
the token id

GetImage

public string GetImage();
Returns the token image (i.e. the characters).
Returns:
the token characters

GetName

public override string GetName();
Returns the token node name.
Returns:
the token node name

GetNextToken

public Token GetNextToken();
Returns the next token. The next token may be a token that has been ignored in the parsing. Note that if the token list feature hasn't been used in the tokenizer, this method will always return null. By default the token list feature is not used.
Returns:
the next token, or null if no such token is available
Since:
1.4
See Also:
getPreviousToken, Tokenizer.getUseTokenList, Tokenizer.setUseTokenList

GetPattern

 internal TokenPattern GetPattern();
Returns the token pattern.
Returns:
the token pattern

GetPreviousToken

public Token GetPreviousToken();
Returns the previuos token. The previous token may be a token that has been ignored in the parsing. Note that if the token list feature hasn't been used in the tokenizer, this method will always return null. By default the token list feature is not used.
Returns:
the previous token, or null if no such token is available
Since:
1.4
See Also:
getNextToken, Tokenizer.getUseTokenList, Tokenizer.setUseTokenList

GetStartColumn

public override int GetStartColumn();
The column number of the first character in the token image.
Returns:
the column number of the first token character

GetStartLine

public override int GetStartLine();
The line number of the first character in the token image.
Returns:
the line number of the first token character

SetNextToken

 internal void SetNextToken( Token next );
Sets the next token in the token list. This method will also modify the token specified to have this token as it's previous token.
Parameters:
next - the next token, or null for none
Since:
1.4

SetPreviousToken

 internal void SetPreviousToken( Token previous );
Sets the previous token in the token list. This method will also modify the token specified to have this token as it's next token.
Parameters:
previous - the previous token, or null for none
Since:
1.4

ToShortString

public string ToShortString();
Returns a short string representation of this token. The string will only contain the token image and possibly the token pattern name.
Returns:
a short string representation of this token

ToString

public override string ToString();
Returns a string representation of this token.
Returns:
a string representation of this token

 Overview   Project   Class   Tree   Deprecated   Index 
Grammatica 1.4 Documentation
FRAMES    NO FRAMES
SUMMARY:  INNER | FIELD | CONSTR | METHOD