Post free Jobs, C2C Hotlists, vendors list and more

Home>Long Term Contract :: Data Engineer :: Remote :: Ex. Walmart Needed

Long Term Contract :: Data Engineer :: Remote :: Ex. Walmart Needed

C2C
  • C2C
  • Anywhere

Hello,

Hope you are doing well.

Please find the requirement below the share matching resume to  Reena.Ch@sparinfosys.com

 

Role: Data Engineer || Ex. Walmart Needed

Location: Remote

Duration: Long Term Contract

 

Job Description

Responsibilities

  • Design, develop, and maintain robust and scalable ETL workflows and data pipelines using tools like Hive, Spark, and Airflow.
  • Implement and manage data storage and processing solutions using Apache Hudi and BigQuery.
  • Develop and optimize data pipelines for structured and unstructured data in GCP environments, leveraging GCS for data storage.
  • Write clean, maintainable, and efficient code in Scala and Python to process and transform data.
  • Ensure data quality, integrity, and consistency by implementing appropriate data validation and monitoring techniques.
  • Work with cross-functional teams to understand business requirements and deliver data solutions that drive insights and decision-making.
  • Troubleshoot and resolve performance and scalability issues in data processing and pipelines.
  • Stay updated with the latest developments in big data technologies and tools and incorporate them into the workflow as appropriate.

Required Skills and Qualifications

  • Proven experience as a Data Engineer, preferably in a big data environment.
  • Expertise in Hive, Spark, and Apache Hudi for big data processing and storage.
  • Hands-on experience with BigQuery and Google Cloud Platform (GCP) services such as GCS, Dataflow, and Pub/Sub.
  • Strong programming skills in Scala and Python, with experience in building data pipelines and ETL processes.
  • Proficiency with workflow orchestration tools like Apache Airflow.
  • Solid understanding of data warehousing concepts, data modelling, and schema design.
  • Knowledge of distributed systems and parallel processing.
  • Strong problem-solving skills and ability to work with large datasets in a fast-paced environment.

 

 

Reena Chauhan

Spar Information Systems

Desk No: 469-409-0307,Ext. 370

Email:Reena.Ch@sparinfosys.com

 

 

 

 

 

 

 

 

 

 


From:
Reena Chauhan,
Spar Information Systems
reena.ch@sparinfosys.com
Reply to:   reena.ch@sparinfosys.com