Senior Data Engineer

Location

Responsibilities

• Design, develop, and maintain large-scale data pipelines using Python and GCP/AWS.
• Develop and implement ETL processes to extract, transform, and load data from various sources.
• Collaborate with cross-functional teams to integrate machine learning models into cloud-native applications.
• Develop and maintain APIs using Python to support data exchange between systems.
• Design and implement data storage solutions using GCP/AWS services such as Bigtable, Cloud Storage, and Cloud SQL.
• Develop and maintain data processing workflows using GCP/AWS services such as Dataflow, Cloud Functions, and Cloud Pub/Sub.
• Collaborate with data scientists to integrate generative AI/LLM models into data pipelines and applications.
• Develop and maintain documentation for data pipelines, APIs, and cloud-native applications.

Requirements

• 3-5 years of experience in data engineering with a focus on Python, AI, GCP, and AWS.
• Strong understanding of data engineering principles, including data modeling, data warehousing, and data governance.
• Experience with ETL processes, data pipelines, and data processing workflows.
• Proficiency in Python programming language, including experience with popular libraries such as NumPy, pandas, and scikit-learn.
• Experience with GCP and/or AWS cloud platforms, including services such as Bigtable, Cloud Storage, Cloud SQL, Dataflow, Cloud Functions, and Cloud Pub/Sub.
• Experience with API development using Python, including experience with frameworks such as Flask and Django.
• Strong understanding of machine learning concepts, including supervised and unsupervised learning, regression, classification, and clustering.
• Experience with generative AI/LLM models, including experience with popular libraries such as TensorFlow and PyTorch.
• Strong communication and collaboration skills, with the ability to work effectively with cross-functional teams.

Benefits

• Medical insurance for self and family members.
• Free meals and snacks in the office.
• Access to state-of-the-art technology and tools.
• Opportunities for professional growth and development.