Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-52385

Remove TempResolvedColumns from InheritAnalysisRules name

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 4.1.0
    • 4.1.0
    • SQL

    Description

      In this issue I propose to remove the TempResolvedColumn nodes when computing the name for InheritAnalysisRules nodes (they are not removed during the ResolveAggregateFunctions rule). This is the right behavior as TempResolvedColumn is an internal node and shouldn't be exposed to the users.

      The following query:

      SELECT sum(col1) FROM VALUES(1) GROUP BY ALL HAVING sum(ifnull(col1, 1)) = 1

      Would have following analyzed plans:
      Before the change:
      Project sum(col1)#2L
      +- Filter (sum(ifnull(tempresolvedcolumn(col1), 1))#4L = cast(1 as bigint))
      +- Aggregate [sum(col1#0) AS sum(col1)#2L, sum(ifnull(tempresolvedcolumn(col1#0, col1, false), 1)) AS
      sum(ifnull(tempresolvedcolumn(col1), 1))#4L]
      +- LocalRelation col1#0

      After the change:
      Project sum(col1)#2L
      +- Filter (sum(ifnull(col1, 1))#4L = cast(1 as bigint))
      +- Aggregate sum(col1#0) AS sum(col1)#2L, sum(ifnull(tempresolvedcolumn(col1#0, col1, false), 1)) AS sum(ifnull(col1, 1))#4L
      +- LocalRelation col1#0

      Attachments

        Issue Links

          Activity

            People

              mihailoale-db Mihailo Aleksic
              mihailoale-db Mihailo Aleksic
              Wenchen Fan Wenchen Fan
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: