> ## Documentation Index
> Fetch the complete documentation index at: https://rajanand.org/llms.txt
> Use this file to discover all available pages before exploring further.

# Spark: display() function

The `display()` function is commonly used in **Databricks notebooks** to render DataFrames, charts, and other visualizations in an interactive and user-friendly format. It is not a native Spark function but is specific to Databricks. The `display()` function provides a rich set of features for data exploration, including tabular views, charts, and custom visualizations.

***

## 1. **Syntax**

* **Databricks:**
  ```python theme={"system"}
  display(df)
  ```
* **PySpark (outside Databricks)**:
  * Use `df.show()` or `df.toPandas()` for similar functionality.

## 2. **Key Features**

* **Interactive Tables**: Displays DataFrames in an interactive table with sorting, filtering, and pagination.
* **Visualizations**: Supports built-in charts (e.g., bar charts, line charts, pie charts) for data exploration.
* **Custom Visualizations**: Allows custom visualizations using libraries like Matplotlib, Plotly, or Seaborn.
* **Rich Output**: Can display images, HTML, and other rich content.

## 3. **Examples**

### **Example 1: Displaying a DataFrame as a Table**

* **Databricks:**
  ```python theme={"system"}
  from pyspark.sql import SparkSession

  spark = SparkSession.builder.appName("DisplayExample").getOrCreate()

  # Create DataFrame
  data = [("Anand", 25, 3000), ("Bala", 30, 4000), ("Kavitha", 28, 3500), ("Raj", 35, 4500)]
  columns = ["Name", "Age", "Salary"]

  df = spark.createDataFrame(data, columns)

  # Display the DataFrame
  display(df)
  ```

**Output**:

* An interactive table with columns `Name`, `Age`, and `Salary`.

### **Example 2: Displaying a Chart**

* **Databricks:**
  ```python theme={"system"}
  # Display a bar chart of Salary by Name
  display(df)
  ```
  * After running the above code, click on the **Chart** button in the Databricks notebook to visualize the data as a bar chart.

**Output**:

* A bar chart showing `Salary` on the y-axis and `Name` on the x-axis.

### **Example 3: Displaying a Pie Chart**

* **Databricks:**
  ```python theme={"system"}
  # Display a pie chart of Age distribution
  display(df)
  ```
  * After running the above code, click on the **Chart** button and select **Pie Chart** to visualize the data.

**Output**:

* A pie chart showing the distribution of `Age`.

### **Example 4: Displaying Custom Visualizations**

* **Databricks:**
  ```python theme={"system"}
  import matplotlib.pyplot as plt
  import pandas as pd

  # Convert Spark DataFrame to Pandas DataFrame
  pandas_df = df.toPandas()

  # Create a custom bar chart
  plt.bar(pandas_df["Name"], pandas_df["Salary"])
  plt.xlabel("Name")
  plt.ylabel("Salary")
  plt.title("Salary by Name")
  plt.show()

  # Display the chart
  display()
  ```

**Output**:

* A custom bar chart created using Matplotlib.

### **Example 5: Displaying a DataFrame with Filters**

* **Databricks:**
  ```python theme={"system"}
  # Display the DataFrame with filters
  display(df)
  ```
  * After running the above code, use the filter options in the interactive table to filter rows.

**Output**:

* An interactive table with filter options.

### **Example 6: Displaying a Line Chart**

* **Databricks:**
  ```python theme={"system"}
  # Display a line chart of Salary by Age
  display(df)
  ```
  * After running the above code, click on the **Chart** button and select **Line Chart** to visualize the data.

**Output**:

* A line chart showing `Salary` on the y-axis and `Age` on the x-axis.

### **Example 7: Displaying HTML Content**

* **Databricks:**
  ```python theme={"system"}
  # Display HTML content
  html_content = "<h1>Hello, Databricks!</h1>"
  displayHTML(html_content)
  ```

**Output**:

* Rendered HTML content in the notebook.

## 4. **Common Use Cases**

* Exploring and analyzing data interactively in Databricks notebooks.
* Creating visualizations for data insights and reporting.
* Sharing results with stakeholders in a user-friendly format.

## 5. **Performance Considerations**

* `display()` is optimized for Databricks notebooks and works efficiently with large datasets.
* Use it judiciously for very wide DataFrames (many columns), as it processes all specified columns.

## 6. **Key Takeaways**

1. **Purpose**: The `display()` function is used in Databricks notebooks to render DataFrames, charts, and visualizations interactively.
2. **Interactive Tables**: Provides sorting, filtering, and pagination for DataFrames.
3. **Visualizations**: Supports built-in charts and custom visualizations.
4. **Common Use Cases**:
   * Exploring and analyzing data interactively.
   * Creating visualizations for data insights.
   * Sharing results in a user-friendly format.
5. **Performance**: `display()` is optimized for Databricks notebooks and works efficiently with large datasets.