Senior Data Engineer

About the Role

The Senior Data Engineer is a key contributor responsible for collecting, transforming, and managing data from various sources, ensuring its quality, security, and accessibility. They design data pipelines, optimize storage solutions, and collaborate with data stakeholders to support informed decision-making, and have the opportunity to work with pre-built machine learning models.

Responsibilities

• Manage and maintain end-to-end data pipelines, including orchestration, scheduling, and automation for batch and real-time streaming.
• Set up mechanisms to collect data from various sources such as databases, external APIs, log files, and IoT devices.
• Implement and operate data extraction processes (ETL, ELT, CT, CDC) and document data jobs and flows.
• Design and maintain transformations and data models using dbt, and represent data lineage.
• Build and operate orchestration using Airflow and ensure monitoring, logging, and error handling for reliable data flows.
• Use Confluent or Kafka for streaming pipelines and ensure data schedule and processing intervals.
• Implement CI/CD pipelines for data workflows and deployments.
• Optimize data performance, including SQL script performance tuning and data storage optimization.
• Perform web scraping analysis where required.
• Ensure data integrity, validation checks, retention/archiving policies, governance and compliance with local regulations (e.g., PDPA) and security requirements.
• Document data processes, configurations, best practices, and communicate effectively with team members and stakeholders.

Qualifications

• Bachelor's degree in Computer Science, Computer Engineering, Quant/Maths or related field.
• Experience with cloud services, especially Google Cloud Platform.
• Strong programming skills in SQL and Python.
• Experience with Airflow and dbt; understanding of modern data stack trends.
• Familiarity with Confluent/Kafka and CI/CD pipelines.
• Experience in SAP and Machine Learning is a plus; understanding of MLOps is a plus.
• Good communication in Thai and English.
• Highly motivated, proactive, able to work under pressure, independent, team player, and eager to learn new services.

Skills

• Data pipeline design and orchestration
• Data governance, lineage and documentation
• Monitoring, logging and error handling for data systems