#include <lexer.h>
Lexer class. This class provides functionality to scan text files. the stream is tokenized for easy reading.
Static Public Member Functions | |
static MAXON_METHOD LexerInterface * | Alloc (MAXON_SOURCE_LOCATION_DECLARATION) |
Private Member Functions | |
MAXON_INTERFACE_NONVIRTUAL (LexerInterface, MAXON_REFERENCE_NORMAL, "net.maxon.interface.lexer") | |
|
private |
|
static |
[in] | allocLocation | Source location. |
MAXON_METHOD Result<void> Init | ( | UrlOrInputStream && | fileName, |
LEXERINITFLAGS | flags, | ||
const String & | stringEscapeCharacters, | ||
const StringDecodingRef & | stringDecoding, | ||
LEXERSYMBOLFLAGS | firstParseFlag = LEXERSYMBOLFLAGS::NONE |
||
) |
Initializes the lexer class. This functions opens the stream.
[in] | fileName | Url or Input Stream. |
[in] | flags | Control flags for the lexer. See LEXERINITFLAGS for description. |
[in] | stringEscapeCharacters | Defines characters that allow to escape " ' and the escape character itself within strings. By Default this is '\'. |
[in] | stringDecoding | String encoding for this file. If a nullptr is passed (recommended) then StreamConversions::UtfTextDecoder will be used to auto-detect the file encoding. |
[in] | firstParseFlag | The lexer will parse the first symbol from the file with ReadNextSymbol. This flag will be passed to the call. |
MAXON_METHOD Result<void> AddOperator | ( | const String & | chars | ) |
Adds an operator symbol to the lexer. This is a symbol which consists of several non-letter characters such as && or +=. The lexer will then return such a character sequence as a single symbol of type LEXERSYMBOL::OPERATOR.
[in] | chars | The symbol to add. |
MAXON_METHOD Result<void> Close | ( | ) |
Closes the stream.
MAXON_METHOD Result<Utf32Char> ReadNextChar | ( | ) |
Reads the next character out of the stream.
MAXON_METHOD Result<Bool> SkipSpaces | ( | ) |
Skips the spaces from the current position.
MAXON_METHOD Result<Bool> SkipLineToEnd | ( | Bool | collectSkipped = false | ) |
Skips line to the end. If the lexer is in NEED_EOL mode the method stops at line end. Otherwise it will advance to the next line and stops when non-whitespace is found.
[in] | collectSkipped | If true then the skipped bytes are returned with the next GetIdent() call. |
MAXON_METHOD Result<LEXERSYMBOL> ReadNextSymbol | ( | LEXERSYMBOLFLAGS | flags = LEXERSYMBOLFLAGS::NONE | ) |
Reads the next symbol in the stream. GetIdent() returns the name of the symbol/ident/number/operator.
[in] | flags | See LEXERSYMBOLFLAGS for details. |
MAXON_METHOD Result<void> ReadNextSymbolExpected | ( | LEXERSYMBOL | expectedSymbols, |
LEXERSYMBOLFLAGS | flags = LEXERSYMBOLFLAGS::NONE |
||
) |
Reads the next symbol in the stream and checks if it's identical to requiredSymbols. GetIdent() returns the name of the symbol/ident/number/operator.
[in] | expectedSymbols | Symbol that is required. |
[in] | flags | See LEXERSYMBOLFLAGS for details. |
MAXON_METHOD void RewindSymbol | ( | ) |
Rewinds the current symbol so that the next ReadNextSymbol returns exactly the same as before.
MAXON_METHOD LEXERSYMBOL GetSymbol | ( | ) | const |
Returns the current symbol.
MAXON_METHOD Int GetLeadingSpace | ( | ) | const |
Returns the number of leading spaces of the symbol. Tabs count as up to 8 spaces.
MAXON_METHOD String GetIdent | ( | ) | const |
Returns the current String ident.
MAXON_METHOD Result<void> GetFloat | ( | Float32 & | val | ) | const |
In case of LEXERSYMBOL::NUMBER this function returns the floating point value of this number.
[out] | val | Reference to the value. |
MAXON_METHOD Result<void> GetFloat | ( | Float64 & | val | ) | const |
In case of LEXERSYMBOL::NUMBER this function returns the floating point value of this number.
[in] | val | Reference to the value. |
MAXON_METHOD Result<void> GetInt | ( | Int32 & | val | ) | const |
In case of LEXERSYMBOL::NUMBER this function returns the integer value of this number.
[in] | val | Reference to the value. |
MAXON_METHOD Result<void> GetInt | ( | Int64 & | val | ) | const |
In case of LEXERSYMBOL::NUMBER this function returns the integer value of this number.
[in] | val | Reference to the value. |
MAXON_METHOD Result<void> GetUInt | ( | UInt32 & | val | ) | const |
In case of LEXERSYMBOL::NUMBER this function returns the integer value of this number.
[in] | val | Reference to the value. |
MAXON_METHOD Result<void> GetUInt | ( | UInt64 & | val | ) | const |
In case of LEXERSYMBOL::NUMBER this function returns the integer value of this number.
[in] | val | Reference to the value. |
MAXON_METHOD Result<Int64> GetCurrentReadPosition | ( | ) | const |
Returns the current reading position.
MAXON_METHOD Result<Int64> GetSymbolPosition | ( | ) | const |
Returns the position of the current symbol.
MAXON_METHOD Bool CheckIdent | ( | const Char * | cmp | ) | const |
Checks against the current identifier.
[in] | cmp | C-style string to compare with. |
MAXON_METHOD Bool CheckIdent | ( | const String & | cmp | ) | const |
Checks against the current identifier.
[in] | cmp | String to compare with. |
MAXON_METHOD Int GetLine | ( | ) | const |
Returns the current line number within the file.
MAXON_METHOD void SetStringQuoteCharacters | ( | const String & | quoteChar | ) |
Sets the quote characters that are used to detect Strings when LEXERSYMBOLFLAGS::ALLOWSTRINGS is used.
[in] | quoteChar | Defines characters that is used to detect quoted strings. By Default this is "\"'". |