上QQ阅读APP看书，第一时间看更新

How to do it...

This section walks through the steps to initialize and configure a Spark cluster.

Import SparkSession using the following script:

from pyspark.sql import SparkSession

Configure SparkSession with a variable named spark using the following script:

spark = SparkSession.builder \
    .master("local[*]") \
    .appName("GenericAppName") \
    .config("spark.executor.memory", "6gb") \
.getOrCreate()