Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-52157

Use earlier computed name for OuterReference in single-pass resolver

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 4.1.0
    • 4.1.0
    • SQL

    Description

      In the following query we would have min(outer(t2.t2a)) as a name for min(t2a) expression.

      SELECT t1a 
      FROM   t1
      WHERE  t1a IN (SELECT t2a 
                     FROM   t2
                     WHERE  EXISTS (SELECT min(t2a) 
                                    FROM   t3))
      

      This is a problem in compatibility between single-pass resolver and fixed-point analyzer because names in single-pass are generated after we finish resolution of aggregate expression min(t2a) (bottom-up manner) and at that point we have OuterReference wrapped around aggregate expression (name looks like outer(min(t2a))).
      In this issue I propose that we store the name earlier (in a form of a tag) and use it when needed

      Attachments

        Issue Links

          Activity

            People

              mihailoale-db Mihailo Aleksic
              mihailoale-db Mihailo Aleksic
              Wenchen Fan Wenchen Fan
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: