The CharStream Classes MiniTutorial This document describes in some detail the methods of the CharStream classes. Note that some of the details may not be relevant for the CharStream interface (to be used with USER_CHAR_STREAM).
There are 4 different kinds of char stream classes that are generated based on combinations of various options.
This class treats the input as a stream of 1-byte (ISO-LATIN1) characters. Note that this class can also be used to parse binary files. It just reads a byte and returns it as a 16 bit quantity to the lexical analyzer. So any character returned by this class will be in the range '\u0000'-'\u00ff'.
This class treats the input as a stream of 1-byte characters. However, the special escape sequence
This class treats the input as a stream of 2-byte characters. So it reads 2 bytes b1 and b2 and returns them as a single character using the expression b1 << 8 | b2 assuming bigendian order. So in particular all the characters in the range 0x00-0xff are assumed to be stored as 2 bytes with the first (higher-order) byte being 0.
This class input is a stream of 2-byte characters (just like the UCode_CharStream class) and the special escape sequence
Takes an input stream, starting line and column numbers and constructs a CharStream object. It also creates buffers of initial size 4K for buffering the characters and also for line and column numbers for each of those characters.
Takes an input stream, starting line and column numbers and constructs a CharStream object. It also creates buffers of initial size buffsize for buffering the characters and also for line and column numbers for each of those characters.
So when you have an estimate on the maximum size of any token that can occur, you can use that size to optimize the buffer sizes. Note however that these sizes are only initial sizes and they will be expanded as and when needed (in 2K steps).
This method returns the next "character" in the input according to the rules of the CharStream class as described above. It will throw java.io.IOException if it reaches EOF during the process of "constructing" the character. It also updates the line and column number and buffers the character for any possible backtracking that may be required later. It also stores the line and column numbers for the same purpose.
This method returns the line number for the begining of the current match.
This method returns the column number for the begining of the current match.
This method returns the line number for the ending of the current match.
This method returns the column number for the ending of the current match.
This method puts back amount number of characters into the stream. Note that the amount indicates the number of characters as constructed by readChar. Since the buffers used are circular buffers, it cannot check for illegal amount values, it just wraps around. So it is the user's responsibility to make sure that those many characters are really produced before a call to this method.
Returns the image of the current match. As far as the XXXCharStream is concerned, all characters after the last call to the private method BeginToken are considered a part of the current match.
This method reintializes the XXXCharStream classes with a (possibly new) input stream and starting line and column numbers.
This method reintializes the XXXCharStream classes with a (possibly new) input stream and starting line and column numbers and adjusts the size of the buffers to buffersize, by extending them. Note that if the value of buffersize is less than the current buffer sizes, they remain unchanged.
This method adjusts the line and column number of the beginning of the current match to newLine and newCol and also adjusts the line and column numbers for all the characters in the look ahead buffer.