    All Packages    This Package    Previous    Next

# Class java.io.StreamTokenizer

    java.lang.Object
       |
       +----java.io.StreamTokenizer

public class StreamTokenizer
extends Object

A class to turn an input stream into a stream of tokens. There are a number of methods that define the lexical syntax of tokens.

Author:
  James Gosling

##

TT_EOF
  End of file token
TT_EOL
  End of line token
TT_NUMBER
  Number token.
TT_WORD
  Word token.
eolIsSignificant
  Set to true if end-of-lines are significant (TT_EOL will be returned by nexttoken).
hashComments
  If true, '#' starts a single line comment.
lineno
  The line number of the last token read
nval
slashSlashComments
  Set to true to recognize C++ style // comments
slashStarComments
  Set to true to recognize C style /* comments
sval
ttype
  The type of the last token returned.

##

StreamTokenizer(InputStream)
  Create a stream tokenizer that parses the given input stream.

##

commentChar(int)
  Specify that this character starts a single line comment.
nextToken()
  Parse a token from the input stream.
ordinaryChar(int)
  Specify that this character is 'ordinary': it removes any significance as a word, comment, string, whitespace or number character.
ordinaryChars(int, int)
  Specify that characters in this range are 'ordinary': it removes any significance as a word, comment, string, whitespace or number character.
parseNumbers()
  Specify that numbers should be parsed.
quoteChar(int)
  Specify that matching pairs of this character delimit string constants.
resetSyntax()
  Reset the syntax table so that all characters are special
toString()
whitespaceChars(int, int)
  Specify that characters in this range are whitespace characters
wordChars(int, int)
  Specify that characters in this range are word characters

##

lineno

      public int lineno

  The line number of the last token read

eolIsSignificant

      public boolean eolIsSignificant

  Set to true if end-of-lines are significant (TT_EOL will be returned by nexttoken). If false, they will be treated as whitespace

slashSlashComments

      public boolean slashSlashComments

  Set to true to recognize C++ style // comments

slashStarComments

      public boolean slashStarComments

  Set to true to recognize C style /* comments

hashComments

      public boolean hashComments

  If true, '#' starts a single line comment. This is here solely for compatibility with 1.0a1. Use commentChar('#') instead.

ttype

      public int ttype

  The type of the last token returned. It's value will either be one of the following TT_* constants, or a single character. For example, if '+' is encountered and is not a valid word character, ttype will be '+'

TT_EOF

      public final static int TT_EOF

  End of file token

TT_EOL

      public final static int TT_EOL

  End of line token

TT_NUMBER

      public final static int TT_NUMBER

  Number token. The value is in nval

TT_WORD

      public final static int TT_WORD

  Word token. The value is in sval

sval

      public String sval

nval

      public double nval

##

StreamTokenizer

      public StreamTokenizer(InputStream I)

  Create a stream tokenizer that parses the given input stream. By default, it recognizes numbers, all the alphabetics are valid word characters, and strings quoted with single and double quote.

##

resetSyntax

      public void resetSyntax()

  Reset the syntax table so that all characters are special

wordChars

      public void wordChars(int low,
                            int hi)

  Specify that characters in this range are word characters

whitespaceChars

      public void whitespaceChars(int low,
                                  int hi)

  Specify that characters in this range are whitespace characters

ordinaryChars

      public void ordinaryChars(int low,
                                int hi)

  Specify that characters in this range are 'ordinary': it removes any significance as a word, comment, string, whitespace or number character. When encountered by the parser, they return a ttype equal to the character.

ordinaryChar

      public void ordinaryChar(int ch)

  Specify that this character is 'ordinary': it removes any significance as a word, comment, string, whitespace or number character. When encountered by the parser, it returns a ttype equal to the character.

commentChar

      public void commentChar(int ch)

  Specify that this character starts a single line comment.

quoteChar

      public void quoteChar(int ch)

  Specify that matching pairs of this character delimit string constants. When a string constant is recognized ttype will be the character that delimits the string, and sval will have the body of the string.

parseNumbers

      public void parseNumbers()

  Specify that numbers should be parsed. It accepts double precision floating point numbers and returns a ttype of TT_NUMBER with the value in nval.

nextToken

      public int nextToken()

  Parse a token from the input stream. The return value is the same as the value of ttype. Typical clients of this class first set up the syntax tables and then sit in a loop calling nextToken to parse successive tokens until TT_EOF is returned.

toString

      public String toString()

    All Packages    This Package    Previous    Next