Aleph::Compiler_Token Struct Reference

Lexical token representation with its value and location. More...

#include <Compiler_Token.H>

Collaboration diagram for Aleph::Compiler_Token:

Public Member Functions
bool	is_eof () const noexcept
	Returns whether the token marks the end of the file.
bool	is_invalid () const noexcept
	Returns whether the token represents a lexical error.
bool	is (const Compiler_Token_Kind expected) const noexcept
	Fast kind check.
std::string	to_string () const
	Formats the token as a descriptive string for debugging.

Public Attributes
Compiler_Token_Kind	kind = Compiler_Token_Kind::Invalid
	Logical category (e.g., Kw_If).
std::string	lexeme
	Exact text found in the source.
Source_Span	span
	Precise location in the source code.

Detailed Description

Lexical token representation with its value and location.

It is the primary object that the Lexer delivers to the Parser. It combines the "what" (kind), the "how" (lexeme), and the "where" (span).

Note

Thread-safety: Not thread-safe for concurrent mutation. Const methods are safe for concurrent readers.

Complexity: O(1) for construction/copy (plus std::string management).

Definition at line 241 of file Compiler_Token.H.

Member Function Documentation

is()

bool Aleph::Compiler_Token::is ( const Compiler_Token_Kind  expected ) const

inlinenoexcept

Fast kind check.

Parameters

expected

The kind to verify.

Returns

true if the token matches the expected kind.

Note

Thread-safety: Safe for concurrent readers.

Complexity: O(1).

Definition at line 274 of file Compiler_Token.H.

References kind.

is_eof()

bool Aleph::Compiler_Token::is_eof ( ) const

inlinenoexcept

Returns whether the token marks the end of the file.

Note

Thread-safety: Safe for concurrent readers.

Complexity: O(1).

Definition at line 252 of file Compiler_Token.H.

References Aleph::End_Of_File, and kind.

Referenced by Aleph::Compiler_Lexer::eof().

is_invalid()

bool Aleph::Compiler_Token::is_invalid ( ) const

inlinenoexcept

Returns whether the token represents a lexical error.

Note

Thread-safety: Safe for concurrent readers.

Complexity: O(1).

Definition at line 262 of file Compiler_Token.H.

References Aleph::Invalid, and kind.

to_string()

std::string Aleph::Compiler_Token::to_string ( ) const

inline

Formats the token as a descriptive string for debugging.

Formats the token for deterministic debugging and testing.

Escapes special characters in the lexeme to ensure readable output. Useful for logging and diagnostics.

Note

Thread-safety: Safe for concurrent readers.

Complexity: O(N) where N is lexeme length.

Definition at line 689 of file Compiler_Token.H.

References Aleph::compiler_token_kind_name(), Aleph::Compiler_Token_Detail::escape_token_lexeme(), Aleph::Source_Span::is_valid(), kind, lexeme, out, span, and Aleph::Source_Span::to_string().

Referenced by main().

Member Data Documentation

kind

Compiler_Token_Kind Aleph::Compiler_Token::kind = Compiler_Token_Kind::Invalid

Logical category (e.g., Kw_If).

Definition at line 243 of file Compiler_Token.H.

Referenced by Aleph::Compiler_Parser_Stream::check(), Aleph::compiler_unquote_string_token(), is(), is_eof(), is_invalid(), Aleph::Compiler_Parser::parse_function(), Aleph::Compiler_Parser::parse_type_primary(), TEST(), and to_string().

lexeme

std::string Aleph::Compiler_Token::lexeme

Exact text found in the source.

Definition at line 244 of file Compiler_Token.H.

Referenced by Aleph::compiler_unquote_string_token(), and to_string().

span

Source_Span Aleph::Compiler_Token::span

Precise location in the source code.

Definition at line 245 of file Compiler_Token.H.

Referenced by Aleph::Compiler_Parser::parse_function(), Aleph::Compiler_Parser::parse_if_statement(), Aleph::Compiler_Parser::parse_let_statement(), Aleph::Compiler_Parser::parse_statement_terminator(), and to_string().

---

The documentation for this struct was generated from the following file:

• Compiler_Token.H