error reading integers at the end of a file

[tara072]

if at the end of a file there is d = 10. the lexer returns up to the = as ['identifier', 'd', 10] ['operator', '=', 10] and also doesn't read the period.

• need to determine if this is only with integers or with doubles too
• fix this so it's also reading the period at the end

[tara072]

note: this is because when handling any hanging/unfinished lexemes in the lexical analysis section, the state of an integer followed by a dot is not accounted for -- it's "in-between" the integer and double states waiting for the next character to determine which it is. but in this case, the code ends, this goes unhandled and the returned <lexeme, token> pairs are missing the integer and final period.