⚙️
Setting Up Cloudera Data Platform(CDP)
  • CDP Overview
  • Why CDP?
  • CDP Services
  • Setting up Google Cloud Platform(GCP) for Cloudera
  • Creating User
  • Configuring Network Settings
  • Configuring Oracle Java
  • Installing Server
  • Configuring MySQL
  • Set Firewall rule on GCP
  • Cloudera Data Platform Installation
  • Working with Cloudera Manager
  • Set Up a Cluster
  • Testing Your Hadoop Installation
  • Installing Hive
  • Hive Validation
  • Deploying Spark 2.4
  • Running Job on Apache Spark2
  • Installing Kafka
  • Kafka Validation
  • Common Warnings and Errors
Powered by GitBook
On this page

Deploying Spark 2.4

PreviousHive ValidationNextRunning Job on Apache Spark2

Last updated 3 years ago

From the Cloudera Manager Home page, select the ‘Add a Service’ menu option from the drop-down menu to the right of Cluster Name.

The Add Service Wizard appears.

Select Spark and Continue

Customize Role Assignments

  • Spark History server: datacouch.training.io

  • Spark Gateway: datacouch.training.io

Use default settings click on continue

Click Restart Stale Services.

Before Running pyspark2

Set:

Click on search box

Set “yarn.scheduler.maximum-allocation-mb” 10 GB then click on save

Set “yarn.nodemanager.resource.memory-mb” 10 GB then click on save

Restart stale service