-Design and Develop ETL Processes:

Lead the design and implementation of ETL processes using all kinds of batch/streaming tools to extract, transform, and load data from various sources into GCP.

Collaborate with stakeholders to gather requirements and ensure that ETL solutions meet business needs.

-Data Pipeline Optimization:

Optimize data pipelines for performance, scalability, and reliability, ensuring efficient data processing workflows.

Monitor and troubleshoot ETL processes, proactively addressing issues and bottlenecks.

-Data Integration and Management:

Integrate data from diverse sources, including databases, APIs, and flat files, ensuring data quality and consistency.

Manage and maintain data storage solutions in GCP (e.g., BigQuery, Cloud Storage) to support analytics and reporting.

-GCP Dataflow Development:

Write Apache Beam based Dataflow Job for data extraction, transformation, and analysis, ensuring optimal performance and accuracy.

Collaborate with data analysts and data scientists to prepare data for analysis and reporting.

-Automation and Monitoring:

Implement automation for ETL workflows using tools like Apache Airflow or Cloud Composer, enhancing efficiency and reducing manual intervention.

Set up monitoring and alerting mechanisms to ensure the health of data pipelines and compliance with SLAs.

-Reliability Strategy: Develop and implement strategies to improve the reliability and performance of applications and infrastructure, focusing on service level objectives (SLOs) and service level indicators (SLIs).

-Incident Management: Oversee incident response processes, ensuring timely resolution of incidents and minimizing downtime. Conduct post-mortem analyses to identify root causes and implement preventive measures.

-Automation and Tooling: Drive the automation of operational tasks and processes, leveraging tools and technologies to enhance efficiency and reduce manual intervention.

-Monitoring and Alerting: Establish comprehensive monitoring and alerting systems to proactively identify and address performance issues, ensuring system health and availability.

-Documentation: Maintain clear and comprehensive documentation of systems, processes, and procedures to facilitate knowledge sharing and compliance.

-Security and Compliance: Ensure that all systems and processes adhere to security best practices and regulatory requirements, collaborating with security teams as needed.

-continuous Improvement: Identify opportunities for process improvements and lead initiatives to enhance the overall reliability and performance of systems and services.

-Bachelor's degree in Computer Science, Information Technology, or a related field.

-Proven experience in designing and managing ETL solutions, including data modeling, data warehousing, and SQL development.

-Proven experience in Site Reliability Engineering, DevOps, or a related field, with a strong understanding of system architecture and cloud technologies.

-Experience of cloud-based solutions, especially in GCP, cloud certified candidate is preferred.

-Experience and knowledge of Bigdata data processing in batch mode and streaming mode, proficient in Bigdata eco systems, e.g. Hadoop, HBase, Hive, MapReduce, Kafka, Flink, Spark, etc.

-Familiarity with Java & Python for data manipulation on Cloud/Bigdata platform.

-Experience with incident management and response, including post-mortem analysis and root cause identification.

-Excellent problem-solving skills and the ability to work under pressure in a fast-paced environment.

-Strong communication skills, both verbal and written, with the ability to convey technical concepts to diverse audiences.

-Proven experience in AI and machine learning, preferably in the banking or financial services industry