Apache Spark Designer Full-time Job

Responsibilities

Translate business needs to technical specifications

Architecting, Design, build and deploy data processing solutions

Hands on tools to process, stage, store data Conduct recon of data

Collaborate with teams to integrate systems Develop and execute Spark Queries, database queries and conduct analyses

Create dataset, RDD, Data Frame for requested projects Develop and update technical documentation

Sound knowledge about effort estimation methods for data processing solutions requirements Proven experience as Apache Spark Designer.

Hands-on coding experience for Apache Spark designing

Background in data processing design (e.g. large scale data processing with API based status) and data curation

In-depth understanding of database management systems, Spark clustering, Spark executor, Driver, real time data processing platform, Java 1.8, and ETL (Extract, transform, load) framework

Knowledge of SQL queries, Spark Engine and Spark Services, Experience of Spark query tuning and performance optimization