/
Spark UI in notebook

Spark UI in notebook


Subject

When you run a SparkSession on a notebook like Jupyter or Zeppelin you don't have access to the SparkUI. Here the solution to have access.

How to

  • From the Saagie Manager launch a Jupyter Spark Python notebook or a Zeppelin notebook
  • Get or create a SparkSession (
  • Get the APPLICATION ID ("spark" is a SparkSession) :
    Example in Jupyter Python :
    • Scala : spark.sparkContext.applicationId
    • Python : spark.sparkContext.applicationId
    • Java : spark.sparkContext().applicationId(


Related content

Choose your Java version
Choose your Java version
More like this
Pyspark - Read & Write files from Hive
Pyspark - Read & Write files from Hive
More like this
Pyspark - Read & Write files from HDFS
Pyspark - Read & Write files from HDFS
More like this
Apache Spark
More like this
Add a Python job
Add a Python job
More like this
Spark Scala - Read & Write files from MongoDB
Spark Scala - Read & Write files from MongoDB
More like this