- Job Title
- Spark Developer
- Job ID
- Plano, TX
- Other Location
From our start in 2009, Conexess has established itself in 3 markets, employing nearly 200+ individuals nation-wide. Operating in over 15 states, our client base ranges from Fortune 500/1000 companies, to mid-small range companies. For the majority of the mid-small range companies, we are exclusively used due to our outstanding staffing track record
Who We Are:
Conexess is a full-service staffing firm offering contract, contract-to hire, and direct placements. We have a wide range of recruiting capabilities extending from help desk technicians to CIOs. We are also capable of offering project based work.
Conexess Group is aiding a Plano based client in their search for a Spark Developer. This is a long term opportunity with a competitive compensation package.
******We are unable to work C2C on this role******
Ideal candidate must have a strong spark programming and python experience. An exposure to handling very large data sets is strongly preferred. A solid critical thinker with strong programming skills with sound grounding in big datasets will be successful in this role.
- Minimum 6 years demonstrated experience with software development
- Good experience on designing and developing data pipelines for data ingestion and transformation using Py-Spark.
- 4 + years demonstrated experience in distributed computing experience using Python and Py-Spark.
- Good understanding of spark framework and spark architecture.
- Experience working in AWS Cloud based Big data infrastructure.
- Should have development experience using Lambda and Python.
- Excellent in trouble shooting the performance and data skew issues.
- Must have good understanding of spark run time metrics and tune applications based on metrics.
- Deep knowledge in partitioning, bucketing concepts of data ingestion.
- Good understanding of AWS services like Glue, Athena, S3, Lamda, Cloudformation.
- Preferred working knowledge on the implementation of datalake ETL using AWS glue.
- Production experience in deploying Big data applications.
- Knowledge of challenges associated with Bigdata and how to build systems that scale seamlessly.