[SPARK-51079] Support large variable types in pandas UDF, createDataFrame and toPandas with Arrow - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Resolved
Priority: Major
Resolution: Fixed
Affects Version/s: 4.0.0
Fix Version/s: 4.0.0
Component/s: PySpark
Labels:
- pull-request-available

Description

https://issues.apache.org/jira/browse/SPARK-39979 implemented large string but it does not support all cases, see https://github.com/apache/spark/pull/41569

Attachments

Issue Links

links to

GitHub Pull Request #49790

GitHub Pull Request #50513

Activity

People

Assignee:: Hyukjin Kwon

Reporter:: Hyukjin Kwon

Votes:: 0 Vote for this issue

Watchers:: 1 Start watching this issue

Dates

Created:: 04/Feb/25 09:54

Updated:: 07/Apr/25 01:16

Resolved:: 05/Feb/25 23:58