feat(Python): Enable commutation of class attributes, with docstrings #590

wetneb · 2025-09-15T23:16:49+02:00

wetneb commented

2025-09-15 23:16:49 +02:00

Closes #589.

I am not very happy with the overall approach, but it feels like it's the best I can do.

Ideally, the bundling of the comments should be done in the grammar, similarly to what we do in Rust, since the attachment is done by Python itself, regardless of the amount of whitespace surrounding it. But I haven't been able to update the grammar accordingly so far.

We could also decide to define the additional comments via a tree-sitter query, but as things stand it would still not remove the need to fork the grammar because the assignment nodes are wrapped inside an expression_statement node, making it impossible to define the commutative children groups as things stand (except if we also migrate those to support TS queries…)
That being said, there are good signs that the tree-sitter-python maintainer will accept a move to hide those expression_statement nodes in the upstream grammar. When that gets published, we could then be able to switch back to upstream, assuming we change the comment_nodes to use a tree-sitter query instead. But that also has downsides, since that's going to add another query to execute at the beginning of the parsing, complicating the AstNode construction a bit further.

In the interest of meeting Stainless' needs, I think it would still be good to ship this (or some variant of it…)

Closes #589. I am not very happy with the overall approach, but it feels like it's the best I can do. Ideally, the bundling of the comments should be done in the grammar, similarly to what we do in Rust, since the attachment is done by Python itself, regardless of the amount of whitespace surrounding it. But I haven't been able to update the grammar accordingly so far. We could also decide to define the additional comments via a tree-sitter query, but as things stand it would still not remove the need to fork the grammar because the `assignment` nodes are wrapped inside an `expression_statement` node, making it impossible to define the commutative children groups as things stand (except if we also migrate those to support TS queries…) That being said, there are [good signs that the tree-sitter-python maintainer will accept a move to hide those `expression_statement` nodes](https://github.com/tree-sitter/tree-sitter-python/pull/317) in the upstream grammar. When that gets published, we could then be able to switch back to upstream, assuming we change the `comment_nodes` to use a tree-sitter query instead. But that also has downsides, since that's going to add another query to execute at the beginning of the parsing, complicating the `AstNode` construction a bit further. In the interest of meeting Stainless' needs, I think it would still be good to ship this (or some variant of it…)

wetneb added 4 commits

2025-09-15 23:16:50 +02:00

chore: Switch to tree-sitter-python-orchard 417824161f

feat(Python): Enable commutation of class attributes a3322e2333

feat: Let language profiles declare additional comment types f9f6d3b90a

feat(Python): enable commutation of class attrs with docstrings

/ test (pull_request) Successful in 1m37s

Details

94dc6eea1a

wetneb changed title from ~~589-python-docstrings-bundle~~ to feat(Python): Enable commutation of class attributes, with docstrings

2025-09-15 23:17:20 +02:00

wetneb reviewed

2025-09-15 23:22:01 +02:00

examples/python/working/class_attributes/Expected.py Outdated

					
				@ -0,0 +3,4 @@

				    y = 2

				    z = 3

wetneb commented

2025-09-15 23:22:01 +02:00

This example is crying for being spelt out without blank lines between arguments, but merging class attributes with docstrings requires us to still insert a blank line between the merged attributes, otherwise the docstrings end up being at equal distance between two consecutive attributes, which prevents them from being bundled correctly when re-parsing the merge result. This is calling for #373.