We are looking for a talented and experienced Data Engineer to join our data engineering team. In this role, you will be responsible for designing, implementing, and maintaining our data pipelines and infrastructure.

You will collaborate closely with cross-functional teams, including data scientists, analysts, and software engineers, to ensure the availability, accessibility, and quality of our data assets. Your expertise in data processing, database technologies, and ETL processes will be essential in enabling effective data utilization and analysis.

Responsibilities:

• Design, develop, and maintain efficient and scalable data pipelines that extract, transform, and load data from various sources into target systems
• Monitor and optimize data pipelines for performance, reliability, and data quality
• Implement and manage data storage solutions, including relational databases, data warehouses, and NoSQL databases
• Implement data partitioning, indexing, and compression strategies for improved performance and scalability
• Identify and resolve data quality issues, including data cleansing, standardization, and validation
• Identify and implement performance optimization techniques to enhance data processing speed, query performance, and system scalability
• Monitor and analyze data infrastructure and pipeline performance, identifying bottlenecks and implementing optimizations
• Stay updated on emerging technologies and trends in data engineering, proactively identifying opportunities for innovation and improvement

Requirements:

• 4+ years of experience as a Data engineer
• Strong SQL abilities and hands-on experience with SQL and no-SQL DBs, performing analysis and performance optimizations
• Hands-on experience in Python or equivalent programming language
• Experience with data warehouse solutions (like Bigquery/ Redshift/ Snowflake or others)
• Experience with data modeling, data catalog concepts, data formats, data pipelines/ETL design, implementation, and maintenance
• Experience with AWS/GCP cloud services such as GCS/S3, Lambda/Cloud Function, EMR/Dataproc, Glue/Dataflow, and Athena

Advantages:

• Experience with Airflow and DBT
• Experience with MLOps
• Experience with data visualization tools and infrastructures (like Tableau/Looker)
• Experience with development practices – Agile, CI/CD, TDD
• Relevant Bachelor's degree – preferably CS, Engineering/ Information Systems, or other equivalent Software Engineering background