Data Engineer with Python /Pyspark

Updated: March 02, 2024 04:36 PM GMT

Full-time Job in Ukrainka, Kyiv Oblast

Unison Consulting Pte Ltd

Ukrainka, Kyiv Oblast, Ukraine

• Develop and implement data pipelines for ingesting and collecting data from various sources into a centralized data platform.
• Develop and maintain ETL jobs using AWS Glue services to process and transform data at scale.
• Optimize and troubleshoot AWS Glue jobs for performance and reliability.
• Utilize Python and PySpark to efficiently handle large volumes of data during the ingestion process.
• Design and implement scalable data processing solutions using PySpark to transform raw data into a structured and usable format.
• Apply data cleansing, enrichment, and validation techniques to ensure data quality and accuracy.
• Create and maintain ETL processes using Python and PySpark to move and transform data between different systems.
• Optimize ETL workflows for performance and efficiency.
• Collaborate with data architects to design and implement data models that support business requirements.
• Ensure data structures are optimized for analytics and reporting.
• Work with... More Detail