Module: parser.shared

Source: compiler/stage2_l0/src/parser/shared.l0 Language: Dea/L0

Imports / Includes

• tokens
• util.intset
• util.diag
• ast
• std.vector
• std.text
• std.unit
• std.string

Symbols

• ps_create
• ps_destroy
• ps_emit_error
• ps_has_error
• tok_is
• tok_is_ident
• tok_is_int
• tok_is_byte
• tok_is_string
• tok_is_future_extension
• ps_peek
• ps_skip_lexer_errors
• ps_logical_token_at_cursor
• ps_peek_logical_ahead
• ps_emit_remaining_lexer_errors
• ps_last
• ps_at_end
• ps_advance
• ps_check
• ps_match
• ps_match_any2
• ps_match_any3
• ps_match_any4
• ps_check_ident
• ps_match_ident
• ps_check_int
• ps_match_int
• ps_check_byte
• ps_match_byte
• ps_check_string
• ps_match_string
• ps_is_top_level_start
• ps_sync_top_level
• ps_at_stmt_start
• ps_sync_stmt
• ps_expect
• ps_expect_ident
• token_ident_text
• token_int_text
• token_int_value
• token_byte_text
• token_byte_value
• token_string_text
• token_string_value
• token_text_for_width
• token_width
• ps_span_start
• ps_extend_span
• ps_expect_variable_name
• parse_dotted_name_rest
• qn_result_free_all
• ps_try_parse_qualified_name
• ps_parse_type
• ps_check_reserved_binary_op
• ImplParseResult
• ParserState
• QualifiedNameResult

Function ps_create

func ps_create(tokens: TokenVector, filename: string) -> ParserState*

Create a parser state for a token stream and source file.

Parameters:

• tokens: Token stream to parse.
• filename: Source filename used in diagnostics.

Returns: A new parser state with fresh arenas and diagnostics.

Function ps_destroy

func ps_destroy(self: ParserState*)

Release a parser state and its owned data.

Parameters:

• self: Parser state to free.

Function ps_emit_error

func ps_emit_error(self: ParserState*, code: string, message: string, tok: Token)

Emit a parser diagnostic anchored to a token.

Parameters:

• self: Parser state collecting diagnostics.
• code: Diagnostic code.
• message: Diagnostic message.
• tok: Token whose span should be highlighted.

Function ps_has_error

func ps_has_error(self: ParserState*) -> bool

Report whether the parser has recorded any errors.

Parameters:

• self: Parser state to inspect.

Returns: true if parser diagnostics contain an error.

Function tok_is

func tok_is(tok: Token, kind_ord: int) -> bool

Check whether a token has the given kind ordinal.

Parameters:

• tok: Token to inspect.
• kind_ord: Token kind ordinal to compare against.

Returns: true if tok has the requested kind.

Function tok_is_ident

func tok_is_ident(tok: Token) -> bool

Check whether a token is an identifier.

Parameters:

• tok: Token to inspect.

Returns: true if tok is a TT_IDENT.

Function tok_is_int

func tok_is_int(tok: Token) -> bool

Check whether a token is an integer literal.

Parameters:

• tok: Token to inspect.

Returns: true if tok is a TT_INT.

Function tok_is_byte

func tok_is_byte(tok: Token) -> bool

Check whether a token is a byte literal.

Parameters:

• tok: Token to inspect.

Returns: true if tok is a TT_BYTE.

Function tok_is_string

func tok_is_string(tok: Token) -> bool

Check whether a token is a string literal.

Parameters:

• tok: Token to inspect.

Returns: true if tok is a TT_STRING.

Function tok_is_future_extension

func tok_is_future_extension(tok: Token) -> bool

Check whether a token is reserved for a future language extension.

Parameters:

• tok: Token to inspect.

Returns: true if tok is TT_FUTURE_EXTENSION.

Function ps_peek

func ps_peek(self: ParserState*) -> Token

Return the current lookahead token.

TT_LEXER_ERROR wrappers are interpreted as logical tokens. Wrappers with recovery are exposed as that token; wrappers without recovery are skipped.

Parameters:

• self: Parser state to inspect.

Returns: Token at the current parse position.

Function ps_skip_lexer_errors

func ps_skip_lexer_errors(self: ParserState*)

Emit and step over any TT_LEXER_ERROR tokens at the cursor.

Parameters:

• self: Parser state to advance.

Function ps_logical_token_at_cursor

func ps_logical_token_at_cursor(self: ParserState*) -> Token

Return the logical current token and skip unrecoverable lexer wrappers.

Parameters:

• self: Parser state to inspect.

Returns: Logical current token.

Function ps_peek_logical_ahead

func ps_peek_logical_ahead(self: ParserState*, offset: int) -> Token

Return a logical lookahead token without moving the parser cursor.

Parameters:

• self: Parser state to inspect.
• offset: Logical token offset from the current cursor.

Returns: Logical lookahead token.

Function ps_emit_remaining_lexer_errors

func ps_emit_remaining_lexer_errors(self: ParserState*)

Emit deferred lexer diagnostics that parsing never reached.

Parameters:

• self: Parser state whose physical token stream should be scanned.

Function ps_last

func ps_last(self: ParserState*) -> Token

Return the most recently consumed token.

Parameters:

• self: Parser state to inspect.

Returns: Last consumed token, or the first token if nothing has been consumed yet.

Function ps_at_end

func ps_at_end(self: ParserState*) -> bool

Check whether the parser is positioned at EOF.

Parameters:

• self: Parser state to inspect.

Returns: true if the current token is TT_EOF.

Function ps_advance

func ps_advance(self: ParserState*) -> Token

Consume and return the current token.

Parameters:

• self: Parser state to advance.

Returns: Token that was current before advancing.

Function ps_check

func ps_check(self: ParserState*, kind_ord: int) -> bool

Check whether the current token has the given kind ordinal.

Parameters:

• self: Parser state to inspect.
• kind_ord: Token kind ordinal to compare against.

Returns: true if the current token matches kind_ord.

Function ps_match

func ps_match(self: ParserState*, kind_ord: int) -> bool

Consume the current token if it has the given kind ordinal.

Parameters:

• self: Parser state to advance.
• kind_ord: Token kind ordinal to match.

Returns: true if a matching token was consumed.

Function ps_match_any2

func ps_match_any2(self: ParserState*, k1: int, k2: int) -> bool

Consume the current token if it matches either of two kinds.

Parameters:

• self: Parser state to advance.
• k1: First acceptable token kind ordinal.
• k2: Second acceptable token kind ordinal.

Returns: true if one of the requested token kinds was consumed.

Function ps_match_any3

func ps_match_any3(self: ParserState*, k1: int, k2: int, k3: int) -> bool

Consume the current token if it matches any of three kinds.

Parameters:

• self: Parser state to advance.
• k1: First acceptable token kind ordinal.
• k2: Second acceptable token kind ordinal.
• k3: Third acceptable token kind ordinal.

Returns: true if one of the requested token kinds was consumed.

Function ps_match_any4

func ps_match_any4(self: ParserState*, k1: int, k2: int, k3: int, k4: int) -> bool

Consume the current token if it matches any of four kinds.

Parameters:

• self: Parser state to advance.
• k1: First acceptable token kind ordinal.
• k2: Second acceptable token kind ordinal.
• k3: Third acceptable token kind ordinal.
• k4: Fourth acceptable token kind ordinal.

Returns: true if one of the requested token kinds was consumed.

Function ps_check_ident

func ps_check_ident(self: ParserState*) -> bool

Check whether the current token is an identifier.

Parameters:

• self: Parser state to inspect.

Returns: true if the current token is TT_IDENT.

Function ps_match_ident

func ps_match_ident(self: ParserState*) -> bool

Consume the current token if it is an identifier.

Parameters:

• self: Parser state to advance.

Returns: true if an identifier token was consumed.

Function ps_check_int

func ps_check_int(self: ParserState*) -> bool

Check whether the current token is an integer literal.

Parameters:

• self: Parser state to inspect.

Returns: true if the current token is TT_INT.

Function ps_match_int

func ps_match_int(self: ParserState*) -> bool

Consume the current token if it is an integer literal.

Parameters:

• self: Parser state to advance.

Returns: true if an integer literal token was consumed.

Function ps_check_byte

func ps_check_byte(self: ParserState*) -> bool

Check whether the current token is a byte literal.

Parameters:

• self: Parser state to inspect.

Returns: true if the current token is TT_BYTE.

Function ps_match_byte

func ps_match_byte(self: ParserState*) -> bool

Consume the current token if it is a byte literal.

Parameters:

• self: Parser state to advance.

Returns: true if a byte literal token was consumed.

Function ps_check_string

func ps_check_string(self: ParserState*) -> bool

Check whether the current token is a string literal.

Parameters:

• self: Parser state to inspect.

Returns: true if the current token is TT_STRING.

Function ps_match_string

func ps_match_string(self: ParserState*) -> bool

Consume the current token if it is a string literal.

Parameters:

• self: Parser state to advance.

Returns: true if a string literal token was consumed.

Function ps_is_top_level_start

func ps_is_top_level_start(self: ParserState*) -> bool

Check whether the current token can start a top-level declaration.

Parameters:

• self: Parser state to inspect.

Returns: true if the current token starts a declaration accepted at module scope.

Function ps_sync_top_level

func ps_sync_top_level(self: ParserState*)

Recover after a parse error by advancing to the next top-level boundary.

Parameters:

• self: Parser state to resynchronize.

Function ps_at_stmt_start

func ps_at_stmt_start(self: ParserState*) -> bool

Report whether the current token can start a statement.

Parameters:

• self: Parser state to inspect.

Returns: True if the current token is a statement-start keyword.

Function ps_sync_stmt

func ps_sync_stmt(self: ParserState*)

Recover after a failed statement by advancing to the next statement boundary.

Skips to and consumes the next ; , or stops before a } , a statement-start keyword, a top-level start, or end of input. This keeps a single in-body statement error from unwinding to the top-level loop.

Parameters:

• self: Parser state to resynchronize.

Function ps_expect

func ps_expect(self: ParserState*, kind_ord: int, code: string, msg: string) -> Token?

Require the current token to have a specific kind.

Parameters:

• self: Parser state to inspect.
• kind_ord: Required token kind ordinal.
• code: Diagnostic code to emit on mismatch.
• msg: Expectation message prefix used in the diagnostic.

Returns: Consumed token, or null after emitting an error if the token does not match.

Function ps_expect_ident

func ps_expect_ident(self: ParserState*, code: string, msg: string) -> Token?

Require the current token to be an identifier.

Parameters:

• self: Parser state to inspect.
• code: Diagnostic code to emit on mismatch.
• msg: Expectation message prefix used in the diagnostic.

Returns: Consumed identifier token, or null after emitting an error.

Function token_ident_text

func token_ident_text(tok: Token) -> string?

Extract identifier text from a token.

Parameters:

• tok: Token to inspect.

Returns: Identifier spelling, or null if tok is not TT_IDENT.

Function token_int_text

func token_int_text(tok: Token) -> string?

Extract integer literal text from a token.

Parameters:

• tok: Token to inspect.

Returns: Integer literal spelling, or null if tok is not TT_INT.

Function token_int_value

func token_int_value(tok: Token) -> int?

Extract the parsed integer value from a token.

Parameters:

• tok: Token to inspect.

Returns: Parsed integer value, or null if tok is not TT_INT.

Function token_byte_text

func token_byte_text(tok: Token) -> string?

Extract byte literal text from a token.

Parameters:

• tok: Token to inspect.

Returns: Byte literal spelling, or null if tok is not TT_BYTE.

Function token_byte_value

func token_byte_value(tok: Token) -> byte?

Extract byte literal numeric value from a token.

Parameters:

• tok: Token to inspect.

Returns: Byte literal value, or null if tok is not TT_BYTE.

Function token_string_text

func token_string_text(tok: Token) -> string?

Extract string literal text from a token.

Parameters:

• tok: Token to inspect.

Returns: String literal spelling, or null if tok is not TT_STRING.

Function token_string_value

func token_string_value(tok: Token) -> string?

Extract decoded string-literal bytes from a token.

Parameters:

• tok: Token to inspect.

Returns: Decoded string bytes, or null if tok is not TT_STRING.

Function token_text_for_width

func token_text_for_width(tok: Token) -> string

Return the source-like text used to measure a token’s display width.

Parameters:

• tok: Token to inspect.

Returns: Source spelling or token label used for width calculations.

Function token_width

func token_width(tok: Token) -> int

Compute the display width of a token for diagnostics.

Parameters:

• tok: Token to measure.

Returns: Width of the token text in bytes.

Function ps_span_start

func ps_span_start(self: ParserState*) -> Span

Capture a zero-width span at the current token position.

Parameters:

• self: Parser state to inspect.

Returns: Span starting and ending at the current token location.

Function ps_extend_span

func ps_extend_span(self: ParserState*, start: Span) -> Span

Extend a starting span to the end of the last consumed token.

Parameters:

• self: Parser state providing the current position.
• start: Starting span captured before parsing the construct.

Returns: Span from start through the last consumed token.

Function ps_expect_variable_name

func ps_expect_variable_name(self: ParserState*, code: string, msg: string) -> Token?

Require an identifier that is legal as a variable name.

Parameters:

• self: Parser state to inspect.
• code: Diagnostic code to emit on mismatch.
• msg: Expectation message prefix used in the diagnostic.

Returns: Variable-name token, or null after emitting an error.

Function parse_dotted_name_rest

func parse_dotted_name_rest(self: ParserState*, first: string) -> string?

Parse the remaining components of a dotted name.

Parameters:

• self: Parser state to advance.
• first: Already parsed first identifier component.

Returns: Full dotted name, or null on parse error.

Function qn_result_free_all

func qn_result_free_all(self: QualifiedNameResult*)

Free a qualified-name parse result and its owned vectors.

Parameters:

• self: Qualified-name result to free.

Function ps_try_parse_qualified_name

func ps_try_parse_qualified_name(self: ParserState*) -> QualifiedNameResult*?

Parse a qualified name when the lookahead matches Module.Path::Name .

Parameters:

• self: Parser state to inspect.

Returns: Parsed qualified-name components, or null if the lookahead is not a qualified name.

Function ps_parse_type

func ps_parse_type(self: ParserState*) -> TypeRef*?

Parse a type reference.

Parameters:

• self: Parser state to advance.

Returns: Parsed type reference, or null on parse error.

Function ps_check_reserved_binary_op

func ps_check_reserved_binary_op(self: ParserState*) -> Unit?

Reject reserved binary operators at the current parse position.

Parameters:

• self: Parser state to inspect.

Returns: present() if the token is allowed here, or null after emitting an error.

Struct ImplParseResult

Represents the ImplParseResult structure.

ImplParseResult Field root_module

root_module: Module*?

ImplParseResult Field expr_arena

expr_arena: ExprArena*

ImplParseResult Field stmt_arena

stmt_arena: StmtArena*

ImplParseResult Field pattern_arena

pattern_arena: PatternArena*

ImplParseResult Field diags

diags: DiagCollector*

Struct ParserState

Represents the ParserState structure.

ParserState Field tokens

tokens: TokenVector

ParserState Field index

index: int

ParserState Field last_token

last_token: Token

ParserState Field filename

filename: string

ParserState Field diags

diags: DiagCollector*

ParserState Field expr_arena

expr_arena: ExprArena*

ParserState Field stmt_arena

stmt_arena: StmtArena*

ParserState Field pattern_arena

pattern_arena: PatternArena*

ParserState Field eof_aborted

eof_aborted: bool

ParserState Field emitted_lex_errors

emitted_lex_errors: IntSet*

Struct QualifiedNameResult

Represents the QualifiedNameResult structure.

QualifiedNameResult Field module_path

module_path: StringVector*

QualifiedNameResult Field name_qualifier

name_qualifier: StringVector*?

QualifiedNameResult Field name_tok

name_tok: Token