Java.io.StreamTokenizer Class in Java | Set 1
Last Updated :
09 Jan, 2017
Java.io.StreamTokenizer class parses input stream into "tokens".It allows to read one token at a time. Stream Tokenizer can recognize numbers, quoted strings, and various comment styles.
Declaration :
public class StreamTokenizer
extends Object
Constructor :
StreamTokenizer(Reader arg) : Creates a tokenizer that parses the given character stream.
Methods :
- commentChar :
java.io.StreamTokenizer.commentChar(int arg) ignores all characters from the single-line comment character to the end of the line by this StreamTokenizer.
Syntax :
public void commentChar(int arg)
Parameters :
arg : the character after which all characters are ignored in the line.
Return :
No value is returned.
Implementation :
Java
// Java Program illustrating use of commentChar() method
import java.io.*;
public class NewClass
{
public static void main(String[] args) throws InterruptedException,
FileNotFoundException, IOException
{
FileReader reader = new FileReader("ABC.txt");
BufferedReader bufferread = new BufferedReader(reader);
StreamTokenizer token = new StreamTokenizer(bufferread);
// Use of commentChar() method
token.commentChar('a');
int t;
while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)
{
switch (t)
{
case StreamTokenizer.TT_NUMBER:
System.out.println("Number : " + token.nval);
break;
case StreamTokenizer.TT_WORD:
System.out.println("Word : " + token.sval);
break;
}
}
}
}
This program won't run here as no 'ABC' file exists. You can check this code on Java compiler on your system.
To check this code, create a file 'ABC' on your system.
'ABC' file contains :
Programmers
1
2
3
Geeks
Hello
a Program is explained here my friends.
Output:
Word : Progr
Number : 1.0
Number : 2.0
Number : 3.0
Word : Geeks
Word : Hello
lineno() : java.io.StreamTokenizer.lineno() returns the current line number of this StreamTokenizer.
Syntax :
public int lineno()
Parameters :
arg : the character after which all characters are ignored in the line.
Return :
returns the current line number of this StreamTokenizer.
Implementation :
Java
// Java Program illustrating use of lineno() method
import java.io.*;
public class NewClass
{
public static void main(String[] args) throws InterruptedException,
FileNotFoundException, IOException
{
FileReader reader = new FileReader("ABC.txt");
BufferedReader bufferread = new BufferedReader(reader);
StreamTokenizer token = new StreamTokenizer(bufferread);
token.eolIsSignificant(true);
// Use of lineno() method
// to get current line no.
System.out.println("Line Number:" + token.lineno());
token.commentChar('a');
int t;
while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)
{
switch (t)
{
case StreamTokenizer.TT_EOL:
System.out.println("");
System.out.println("Line No. : " + token.lineno());
break;
case StreamTokenizer.TT_NUMBER:
System.out.println("Number : " + token.nval);
break;
case StreamTokenizer.TT_WORD:
System.out.println("Word : " + token.sval);
break;
}
}
}
}
Line Number:1
Word : Progr
Line No. : 2
Number : 1.0
Line No. : 3
Number : 2.0
Line No. : 4
Number : 3.0
Line No. : 5
Word : Geeks
Line No. : 6
Word : Hello
Line No. : 7
Word : This
Word : is
toString() : java.io.StreamTokenizer.toString() represents current Stream token as a string along with it's line no.
Syntax :
public String toString()
Return :
represents current Stream token as a string along with it's line no.
Implementation :
Java
// Java Program illustrating use of toString() method
import java.io.*;
public class NewClass
{
public static void main(String[] args) throws InterruptedException,
FileNotFoundException, IOException
{
FileReader reader = new FileReader("ABC.txt");
BufferedReader bufferread = new BufferedReader(reader);
StreamTokenizer token = new StreamTokenizer(bufferread);
int t;
while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)
{
switch (t)
{
case StreamTokenizer.TT_NUMBER:
// Value of ttype field returned by nextToken()
System.out.println("Number : " + token.nval);
break;
// Value of ttype field returned by nextToken()
case StreamTokenizer.TT_WORD:
// Use of toStringn() method
System.out.println("Word : " + token.toString());
break;
}
}
}
}
Word : Token[Programmers], line 1
Number : 1.0
Number : 2.0
Number : 3.0
Word : Token[Geeks], line 5
Word : Token[Hello], line 6
Word : Token[a], line 7
Word : Token[Program], line 7
Word : Token[is], line 7
Word : Token[explained], line 7
Word : Token[here], line 7
Word : Token[my], line 7
Word : Token[friends.], line 7
eolIsSignificant() : java.io.StreamTokenizer.eolIsSignificant(boolean arg) tells whether to treat end of line as a token or not. If 'arg' is true, then it End Of Line is treated as a token. If true, then the method returns TT_EOL and set the ttype field when End Of Line is reached.
If 'arg' is false then the End Of Line is treated simply as a white space.
Syntax :
public void eolIsSignificant(boolean arg)
Parameters :
arg : boolean which tells whether to take EOL as a token or white space
Return :
No value is returned.
Implementation :
Java
// Java Program illustrating use of eolIsSignificant() method
import java.io.*;
public class NewClass
{
public static void main(String[] args) throws InterruptedException,
FileNotFoundException, IOException
{
FileReader reader = new FileReader("ABC.txt");
BufferedReader bufferread = new BufferedReader(reader);
StreamTokenizer token = new StreamTokenizer(bufferread);
boolean arg = true;
// Use of eolIsSignificant() method
token.eolIsSignificant(arg);
// Here the 'arg' is set true, so EOL is treated as a token
int t;
while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)
{
switch (t)
{
case StreamTokenizer.TT_EOL:
System.out.println("End of Line encountered.");
break;
case StreamTokenizer.TT_NUMBER:
System.out.println("Number : " + token.nval);
break;
case StreamTokenizer.TT_WORD:
System.out.println("Word : " + token.sval);
break;
}
}
}
}
This program won't run here as no 'ABC' file exists. You can check this code on Java compiler on your system.
To check this code, create a file 'ABC' on your system.
'ABC' file contains :
1
Geeks
2
For
3
Geeks
Output :
Number : 1.0
End of Line encountered.
Word : Geeks
End of Line encountered.
Number : 2.0
End of Line encountered.
Word : For
End of Line encountered.
Number : 3.0
End of Line encountered.
Word : Geeks
nextToken() : java.io.StreamTokenizer.nextToken() parses the next token from the Input Stream and returns it's value to the ttype field along with the additional fields like 'sval', 'nval'.
Syntax :
public int nextToken()
Parameters :
------
Return :
value to the ttype field
Implementation :
Java
// Java Program illustrating use of nextToken() method
import java.io.*;
public class NewClass
{
public static void main(String[] args) throws InterruptedException,
FileNotFoundException, IOException
{
FileReader reader = new FileReader("ABC.txt");
BufferedReader bufferread = new BufferedReader(reader);
StreamTokenizer token = new StreamTokenizer(bufferread);
// Use of nextToken() method to parse Next Token from the Input Stream
int t = token.nextToken();
while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)
{
switch (t)
{
case StreamTokenizer.TT_NUMBER:
System.out.println("Number : " + token.nval);
break;
case StreamTokenizer.TT_WORD:
System.out.println("Word : " + token.sval);
break;
}
}
}
}
This program won't run here as no 'ABC' file exists. You can check this code on Java compiler on your system.
To check this code, create a file 'ABC' on your system.
'ABC' file contains :
1
This program tells
2
about use of
3
next token() method
Output :
Word : This
Word : program
Word : tells
Number : 2.0
Word : about
Word : use
Word : of
Number : 3.0
Word : next
Word : token
Word : method
lowerCaseMode() : java.io.StreamTokenizer.lowerCaseMode(boolean arg) tells whether to lowercase the word tokens automatically or not. If the flag - 'arg' is set true, then the value of 'sval' field is lowered.
Syntax :
public void lowerCaseMode(boolean arg)
Parameters :
arg : indicates whether to lowercase the word tokens automatically or not
Return :
void
Implementation :
Java
// Java Program illustrating use of lowerCaseMode() method
import java.io.*;
public class NewClass
{
public static void main(String[] args) throws InterruptedException,
FileNotFoundException, IOException
{
FileReader reader = new FileReader("ABC.txt");
BufferedReader bufferread = new BufferedReader(reader);
StreamTokenizer token = new StreamTokenizer(bufferread);
/* Use of lowerCaseMode() method to
Here, the we have set the Lower Case Mode ON
*/
boolean arg = true;
token.lowerCaseMode(arg);
int t;
while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)
{
switch (t)
{
case StreamTokenizer.TT_NUMBER:
System.out.println("Number : " + token.nval);
break;
case StreamTokenizer.TT_WORD:
System.out.println("Word : " + token.sval);
break;
}
}
}
}
This program won't run here as no 'ABC' file exists. You can check this code on Java compiler on your system.
To check this code, create a file 'ABC' on your system.
'ABC' file contains :
Hello Geeks
This Is About
LowerCaseMode()
Output :
Word : hello
Word : geeks
Word : this
Word : is
Word : about
Word : lowercasemode
ordinaryChar() : java.io.StreamTokenizer.ordinaryChar(int arg) sets 'arg' character as an ordinary character. It will remove the arg character, if it has any significance as word, number, white space or comment Character.
Syntax :
public void ordinaryChar(int arg)
Parameters :
arg : the character which is to be set as an Ordinary Character
Return :
void
Implementation :
Java
// Java Program illustrating use of ordinaryChar() method
import java.io.*;
public class NewClass
{
public static void main(String[] args) throws InterruptedException,
FileNotFoundException, IOException
{
FileReader reader = new FileReader("ABC.txt");
BufferedReader bufferread = new BufferedReader(reader);
StreamTokenizer token = new StreamTokenizer(bufferread);
// Use of ordinaryChar() method
// Here we have taken 's' as an ordinary character
token.ordinaryChar('s');
int t;
while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)
{
switch (t)
{
case StreamTokenizer.TT_NUMBER:
System.out.println("Number : " + token.nval);
break;
case StreamTokenizer.TT_WORD:
System.out.println("Word : " + token.sval);
break;
}
}
}
}
This program won't run here as no 'ABC' file exists. You can check this code on Java compiler on your system.
To check this code, create a file 'ABC' on your system.
'ABC' file contains :
Hello Geeks
Thissss Issszz About
ordinaryChar()
This method has remove 's' from the entire Stream
Output :
Word : Hello
Word : Geek
Word : Thi
Word : I
Word : zz
Word : About
Word : ordinaryChar
ordinaryChars() : java.io.StreamTokenizer.ordinaryChars(int low, int high) sets character in the range - 'a to b' to Ordinary character in the StreamTokenizer
Syntax :
public void ordinaryChars(int low, int high)
Parameters :
low : lower limit of range
high : higher limit of range
Return :
void
Implementation :
Java
// Java Program illustrating use of ordinaryChars() method
import java.io.*;
public class NewClass
{
public static void main(String[] args) throws InterruptedException,
FileNotFoundException, IOException
{
FileReader reader = new FileReader("ABC.txt");
BufferedReader bufferread = new BufferedReader(reader);
StreamTokenizer token = new StreamTokenizer(bufferread);
// Use of ordinaryChars() method
// Here we have taken low = 'a' and high = 'c'
token.ordinaryChars('a','c');
int t;
while ((t = token.nextToken()) != StreamTokenizer.TT_EOF)
{
switch (t)
{
case StreamTokenizer.TT_NUMBER:
System.out.println("Number : " + token.nval);
break;
case StreamTokenizer.TT_WORD:
System.out.println("Word : " + token.sval);
break;
}
}
}
}
This program won't run here as no 'ABC' file exists. You can check this code on Java compiler on your system.
To check this code, create a file 'ABC' on your system.
'ABC' file contains :
Hello Geeks
This
is
about
ordinaryChars()
Output :
Word : Hello
Word : Geeks
Word : This
Word : is
Word : out
Word : ordin
Word : ryCh
Word : rs
Next Article -
Java.io.StreamTokenizer Class in Java | Set 2
Similar Reads
Java.io.StreamTokenizer Class in Java | Set 2
StringTokenizer Class in Java | Set 1 Methods: parseNumbers() : java.io.StreamTokenizer.parseNumbers() specifies that the number in StreamTokenizer is parsed, so that each character - " 0, 1, 2, 3, 4, 5, 6, 7, 8, 9 " has a numeric attribute. When the parser encounters a word token that has the forma
8 min read
Java.io.Printstream Class in Java | Set 1
A PrintStream adds functionality to another output stream, namely the ability to print representations of various data values conveniently. Unlike other output streams, a PrintStream never throws an IOException; instead, exceptional situations merely set an internal flag that can be tested via the c
5 min read
Java.io.Printstream Class in Java | Set 2
Java.io.Printstream Class in Java | Set 1More Methods: PrintStream printf(Locale l, String format, Object... args) : A convenience method to write a formatted string to this output stream using the specified format string and arguments. Syntax :public PrintStream printf(Locale l, String format, Obje
6 min read
Java.io.ObjectInputStream Class in Java | Set 1
ObjectInputStream Class deserializes the primitive data and objects previously written by ObjectOutputStream. Both ObjectOutputStream and ObjectInputStream are used as it provides storage for graphs of object.It ensures that the object it is working for, matches the classes of JVM i.e Java Virtual M
9 min read
Java.io.OutputStream class in Java
This abstract class is the superclass of all classes representing an output stream of bytes. An output stream accepts output bytes and sends them to some sink. Applications that need to define a subclass of OutputStream must always provide at least a method that writes one byte of output. Constructo
2 min read
Java.io.StringReader class in Java
StringReader class in Java is a character stream class whose source is a string. It inherits Reader Class. Closing the StringReader is not necessary, it is because system resources like network sockets and files are not used. Let us check more points about StringReader Class in Java. Declare StringR
4 min read
Java.io.StringWriter class in Java
Java StringWriter class creates a string from the characters of the String Buffer stream. Methods of the StringWriter class in Java can also be called after closing the Stream as this will raise no IO Exception. Declaration in Java StringWriter Classpublic class StringWriter extends WriterConstructo
6 min read
Java.io.ObjectOutputStream Class in Java | Set 1
An ObjectOutputStream writes primitive data types and graphs of Java objects to an OutputStream. The objects can be read (reconstituted) using an ObjectInputStream. Persistent storage of objects can be accomplished by using a file for the stream. Only objects that support the java.io.Serializable in
9 min read
Java.io.PipedInputStream class in Java
Pipes in IO provides a link between two threads running in JVM at the same time. So, Pipes are used both as source or destination. PipedInputStream is also piped with PipedOutputStream. So, data can be written using PipedOutputStream and can be written using PipedInputStream.But, using both threads
5 min read
Java.io.ObjectInputStream Class in Java | Set 2
Java.io.ObjectInputStream Class in Java | Set 1 Note : Java codes mentioned in this article won't run on Online IDE as the file used in the code doesn't exists online. So, to verify the working of the codes, you can copy them to your System and can run it over there. More Methods of ObjectInputStrea
6 min read