Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-52055

Suppress the exception in ExecutePlanResponseReattachableIterator.__del__

    XMLWordPrintableJSON

Details

    Description

      ./bin/pyspark --remote local
      

      and exit(0) directly will show a warning as below.

      Exception ignored in: <function ExecutePlanResponseReattachableIterator.__del__ at 0x12229f7e0>
      Traceback (most recent call last):
        File "/.../python/pyspark/sql/connect/client/reattach.py", line 347, in __del__
          return self.close()
                 ^^^^^^^^^^^^
        File "/.../python/pyspark/sql/connect/client/reattach.py", line 343, in close
          self._release_all()
        File "/.../python/pyspark/sql/connect/client/reattach.py", line 245, in _release_all
          thread_pool.submit(target)
        File "/opt/miniconda3/envs/python3.11/lib/python3.11/concurrent/futures/thread.py", line 169, in submit
          raise RuntimeError('cannot schedule new futures after '
      RuntimeError: cannot schedule new futures after interpreter shutdown
      

      Attachments

        Issue Links

          Activity

            People

              gurwls223 Hyukjin Kwon
              gurwls223 Hyukjin Kwon
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: