Parsing issues are not detected for MISSING tokens #554

Closed
opened 2025-07-30 19:44:02 +02:00 by wetneb · 0 comments
Owner

When a source doesn't parse, tree-sitter emits two kinds of nodes:

  • "error" nodes, for things that aren't expected in that context
  • "missing" nodes, for nodes that are missing to make the context parse correctly

We currently treat a source as syntactically incorrect when we encounter an error node, but not when we encounter missing ones.

For instance:

int main() {
     int var
}

is parsed by mgf_dev parse (in C) as:

└translation_unit
  └function_definition
    ├type: primitive_type int
    ├declarator: function_declarator
    │ ├declarator: identifier main
    │ └parameters: parameter_list
    │   ├(
    │   └)
    └body: compound_statement
      ├{
      ├declaration
      │ ├type: primitive_type int
      │ ├declarator: identifier foo
      │ └; 
      └}

Note the ; token that actually missing from the source.

It would likely be more consistent to reject those cases as well.

When a source doesn't parse, tree-sitter emits two kinds of nodes: * "error" nodes, for things that aren't expected in that context * "missing" nodes, for nodes that are missing to make the context parse correctly We currently treat a source as syntactically incorrect when we encounter an `error` node, but not when we encounter `missing` ones. For instance: ```c int main() { int var } ``` is parsed by `mgf_dev parse` (in C) as: ``` └translation_unit └function_definition ├type: primitive_type int ├declarator: function_declarator │ ├declarator: identifier main │ └parameters: parameter_list │ ├( │ └) └body: compound_statement ├{ ├declaration │ ├type: primitive_type int │ ├declarator: identifier foo │ └; └} ``` Note the `;` token that actually missing from the source. It would likely be more consistent to reject those cases as well.
wetneb self-assigned this 2025-09-11 15:06:30 +02:00
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: mergiraf/mergiraf#554
No description provided.