It seems to me that you don't really understand what the token contains. Let's take a step back and talk about that.
1 2 3 4 5
|
enum Token {
//integer and string constant
ICONST, SCONST,
BEGIN, DONE
};
|
Okay, so you have 4
types of tokens - integer constant, string constant, BEGIN, and DONE. BEGIN and DONE probably represent the start and end of the input file.
Now suppose you read an integer constant. You might want to assign that to a variable, or print it out, or add it to some other integer constant. To do that, you need more than the type of the token, you need the actual
value. Is the integer 17? 833? -123543? You need to store the value.
So now we have two properties of the token - it's
type and its
value.
A third property that's very handy for error messages is the
line number where a token appears in the input file. This is because sometimes you don't detect an error in the syntax until several lines later.
So lets document the fields in the declaration:
1 2 3 4 5 6
|
class Tok {
Token token; // type of token
string str; // actual value of the token
int lnum; // input line number where the token occurred.
...
};
|
Given this, it should be clear that yes, the second argument to the Tok() constructor should be lexeme