lexer.h File Reference

Classes

class  LexerInterface
 

Namespaces

 maxon
 

Enumerations

enum  LEXERSYMBOL : UChar {
  NONE,
  ILLEGAL,
  ENDOFFILE,
  IDENTIFIER,
  STRING,
  STRINGCHR,
  NUMBER,
  NEWLINE,
  CURLYBRACKET_OPEN,
  CURLYBRACKET_CLOSE,
  PARENTHESE_OPEN,
  PARENTHESE_CLOSE,
  SQUAREBRACKET_OPEN,
  SQUAREBRACKET_CLOSE,
  SLASH,
  EQUAL,
  COMMA,
  SEMICOLON,
  COLON,
  PLUS,
  TILDE,
  PREPROCESSOR,
  MULTIPLY,
  PIPE,
  AMPERSAND,
  OPERATOR
}
 
enum  LEXERSYMBOLFLAGS {
  NONE,
  NO_NUMBER,
  ALLOWSTRINGS,
  MINUS_OPERATOR
}
 
enum  LEXERINITFLAGS {
  NONE,
  NEED_EOL,
  REWIND_WHEN_DONE,
  ALLOW_DOLLAR_IDENT,
  ALLOW_AT_IDENT,
  NON_ASCII_IS_IDENT,
  HASH_COMMENTS,
  ESCAPE_IN_SINGLE_QUOTED,
  C_ESCAPE_SEQUENCES,
  DOUBLEQUOTEESCAPE,
  SPACESCOMPATIBILITY,
  C4DSTRCOMPATIBILITY
}
 

Functions

enum maxon::LEXERSYMBOL UChar MAXON_ENUM_LIST (LEXERSYMBOL)
 
enum maxon::LEXERSYMBOLFLAGS MAXON_ENUM_FLAGS (LEXERSYMBOLFLAGS)
 
enum maxon::LEXERINITFLAGS MAXON_ENUM_FLAGS (LEXERINITFLAGS)
 

Variables

 NONE
 
 ILLEGAL
 
 ENDOFFILE
 
 IDENTIFIER
 
 STRING
 
 STRINGCHR
 
 NUMBER
 
 NEWLINE
 
 CURLYBRACKET_OPEN
 
 CURLYBRACKET_CLOSE
 
 PARENTHESE_OPEN
 
 PARENTHESE_CLOSE
 
 SQUAREBRACKET_OPEN
 
 SQUAREBRACKET_CLOSE
 
 SLASH
 
 EQUAL
 
 COMMA
 
 SEMICOLON
 
 COLON
 
 PLUS
 
 TILDE
 
 PREPROCESSOR
 
 MULTIPLY
 
 PIPE
 
 AMPERSAND
 
 OPERATOR
 
 NO_NUMBER
 
 ALLOWSTRINGS
 
 MINUS_OPERATOR
 
 NEED_EOL
 
 REWIND_WHEN_DONE
 
 ALLOW_DOLLAR_IDENT
 
 ALLOW_AT_IDENT
 
 NON_ASCII_IS_IDENT
 
 HASH_COMMENTS
 
 ESCAPE_IN_SINGLE_QUOTED
 
 C_ESCAPE_SEQUENCES
 
 DOUBLEQUOTEESCAPE
 
 SPACESCOMPATIBILITY
 
 C4DSTRCOMPATIBILITY
 

Variable Documentation

◆ NONE

NONE

◆ ILLEGAL

ILLEGAL

◆ ENDOFFILE

ENDOFFILE

The end of file or stream was reached.

◆ IDENTIFIER

IDENTIFIER

An arbitrary symbol that is neither a string, nor a value. It may contain unicode characters.

◆ STRING

STRING

A double-quoted string.

◆ STRINGCHR

STRINGCHR

A single-quoted string.

◆ NUMBER

NUMBER

A numerical value.

◆ NEWLINE

NEWLINE

'\r', '\n' or a combination of both.

◆ CURLYBRACKET_OPEN

CURLYBRACKET_OPEN

'{' symbol.

◆ CURLYBRACKET_CLOSE

CURLYBRACKET_CLOSE

'}' symbol.

◆ PARENTHESE_OPEN

PARENTHESE_OPEN

'(' symbol.

◆ PARENTHESE_CLOSE

PARENTHESE_CLOSE

')' symbol.

◆ SQUAREBRACKET_OPEN

SQUAREBRACKET_OPEN

'[' symbol.

◆ SQUAREBRACKET_CLOSE

SQUAREBRACKET_CLOSE

']' symbol.

◆ SLASH

SLASH

'/' symbol.

◆ EQUAL

EQUAL

'=' symbol.

◆ COMMA

COMMA

',' symbol.

◆ SEMICOLON

SEMICOLON

';' symbol.

◆ COLON

COLON

':' symbol.

◆ PLUS

PLUS

'+' symbol.

◆ TILDE

TILDE

'~' symbol.

◆ PREPROCESSOR

PREPROCESSOR

'#' symbol.

◆ MULTIPLY

MULTIPLY

'*' symbol.

◆ PIPE

PIPE

'|' symbol.

◆ AMPERSAND

AMPERSAND

'&' symbol.

◆ OPERATOR

OPERATOR

< One of the operator symbols added by LexerInterface::AddOperator.

◆ NO_NUMBER

NO_NUMBER

If set, numbers (and characters as '.' or '-') are not recognized as numbers. Instead either an LEXERSYMBOL::IDENTIFIER or LEXERSYMBOL::ILLEGAL will be generated (and GetIdent() returns the symbol).

◆ ALLOWSTRINGS

ALLOWSTRINGS

If set, strings starting with " or ' will be detected and LEXERSYMBOL::STRING is returned. GetIdent() returns the content of the string.

◆ MINUS_OPERATOR

MINUS_OPERATOR

If set, the minus symbol in front of a number will be reported as own token.

◆ NEED_EOL

NEED_EOL

The lexer stops at each end of line and returns LEXERSYMBOL::OPERATOR_NEWLINE whenever a newline appears.

◆ REWIND_WHEN_DONE

REWIND_WHEN_DONE

The lexer will rewind the stream to the last interpreted character when it is done. Only set this if you want to continue scanning the open stream. It will only work if the stream supports seeking. Also a string decoding that has an exact counterpart (e.g. StringEncodings::Utf8) needs to be passed to the lexer.

◆ ALLOW_DOLLAR_IDENT

ALLOW_DOLLAR_IDENT

Allow $ as part of identifiers.

◆ ALLOW_AT_IDENT

ALLOW_AT_IDENT

Allow @ as part of identifiers (but not as start).

◆ NON_ASCII_IS_IDENT

NON_ASCII_IS_IDENT

If set, non-ASCII characters (code > 127) are considered to be part of identifiers.

◆ HASH_COMMENTS

HASH_COMMENTS

If set, comments are introduced by the hash character (instead of C++-like comments).

◆ ESCAPE_IN_SINGLE_QUOTED

ESCAPE_IN_SINGLE_QUOTED

If set, escape characters are also considered in single-quoted-strings.

◆ C_ESCAPE_SEQUENCES

C_ESCAPE_SEQUENCES

If set, a subset of C escape sequences is recognized in strings (\n, \t and \r).

◆ DOUBLEQUOTEESCAPE

DOUBLEQUOTEESCAPE

If set, double quotes will escape the quote char.

◆ SPACESCOMPATIBILITY

SPACESCOMPATIBILITY

If set, a newline character will be included in the result of LexerInterface::GetLeadingSpaces.

◆ C4DSTRCOMPATIBILITY

C4DSTRCOMPATIBILITY

If set, the lexer will handle the escape character in a special manner. Then the escape character works only for the following character if it matches ".