Design and Build Data Processing Systems:

• Collaborate with cross-functional teams to understand data requirements and design efficient data pipelines.
• Implement data ingestion, transformation, and enrichment processes using GCP services (such as BigQuery, Dataflow, and Pub/Sub).
• Ensure scalability, reliability, and performance of data processing workflows.

Data Ingestion and Processing:

• Collect and ingest data from various sources (both batch and real-time) into GCP.
• Cleanse, validate, and transform raw data to ensure its quality and consistency.
• Optimize data processing for speed and efficiency.

Data Storage and Management:

• Choose appropriate storage solutions (e.g., Bigtable, Cloud Storage) based on data characteristics and access patterns.
• Create and manage data warehouses, databases, and data lakes.
• Define data retention policies and archival strategies.

Data Preparation for Analysis:

• Prepare data for downstream analytics, reporting, and machine learning.
• Collaborate with data scientists and analysts to understand their requirements.
• Ensure data is accessible, well-organized, and properly documented.

Automation and Monitoring:

• Automate data workflows using tools like Apache Airflow or Cloud Composer.
• Monitor data pipelines, troubleshoot issues, and proactively address bottlenecks.
• Implement alerting mechanisms for data anomalies.

Security and Compliance:

• Apply security best practices to protect sensitive data.
• Ensure compliance with industry regulations (e.g., GDPR, HIPAA) and internal policies.
• Collaborate with security teams to address vulnerabilities.

Documentation and Knowledge Sharing:

• Document data pipelines, architecture, and processes.
• Share knowledge with team members through documentation, training sessions, and workshops.

Education:

• Bachelor’s degree in Computer Science, Information Systems, or a related field.

Experience:

• Minimum of 3 years of industry experience, including at least 1 year designing and managing solutions using Google Cloud.
• Familiarity with GCP services (BigQuery, Dataflow, Pub/Sub, etc.) and related technologies.
• Experience with data modeling, ETL processes, and data warehousing, SQL
• Previous work in highly regulated or complex organizations is a plus.

Skills:

• Proficiency in Python for data manipulation and scripting.
• Knowledge of Terraform for infrastructure as code (IaC).
• Familiarity with Jenkins for continuous integration and deployment.
• Understanding of event streaming platforms (e.g., Kafka, Google Cloud Pub/Sub).
• Strong problem-solving skills and attention to detail.

Benefits:

• Competitive salary and benefits package.
• Opportunity to work in a dynamic, collaborative environment.
• Chance to contribute to cutting-edge projects using Google Cloud technologies.

If you’re passionate about data engineering, thrive in a fast-paced environment, and want to make an impact, we encourage you to apply!