how to get full result of the sql() function #1795

Activity

1. mohamd764 commented on Oct 16, 2026

   

   mohamd764

   on Oct 16, 2026

   More actions

   print(df) only shows a preview (the first 10 rows by default), so "Data truncated." doesn't mean the result is incomplete. I checked these options with datafusion 55.0.0 (Python 3.13, Linux) using your table:

   # 1. print more rows (default num=20)
   df.show(num=100)
   
   # 2. raise the limit used by print(df) / repr
   from datafusion.dataframe_formatter import configure_formatter
   configure_formatter(max_rows=50)
   print(df)        # all 11 rows, no "Data truncated."
   
   # 3. get the full result as data instead of a printout
   df.to_pandas()   # needs pandas
   df.to_pylist()   # list of dicts (11 rows here)
   df.collect()     # list of pyarrow RecordBatches
   df.to_arrow_table()

   configure_formatter applies to every DataFrame for the rest of the session. reset_formatter() from the same module restores the defaults.

   One thing to watch for: print(df) and df.show() run the query each time. If you want to look at the result more than once, convert it once (for example pdf = df.to_pandas()), or use df.cache() to keep it in memory.

   🚀React with 🚀1Reacted by l1t1
2. l1t1 commented on Oct 16, 2026

   

   l1t1

   on Oct 16, 2026

   Author

   More actions

   Thanks @mohamd764
3. l1t1

   closed this as completedon Oct 16, 2026