Skip to main content

Documentation Index

Fetch the complete documentation index at: https://rajanand.org/llms.txt

Use this file to discover all available pages before exploring further.

A SparkSession is the entry point to Spark SQL, which is the Spark module for structured data processing.
It provides a single point of entry to interact with Spark data, and it allows you to create DataFrames, register DataFrames as tables, execute SQL queries, and more.
  • Any spark application has to create a SparkSession or have to use an existing one.
  • SparkSession is the entry point to Spark SQL.
  • Before Spark 2.0, there was SparkConf and then we have different contexts like SparkContext, SQLContext, HiveContext, StreamContext etc.
  • In Spark 2.0, SparkSession is introduced which is a combination of SQLContext, HiveContext, and StreamContext.