Data Engineer- DevOps

Summary

We are seeking a skilled Snowflake Data Engineer with expertise in DevOps to join our team. The ideal candidate will spearhead the development of an Enterprise data warehouse platform, Snowflake. They will be responsible for developing solutions to analyze raw data sets and identify trends. They play a crucial role in building and managing the data pipelines and data warehouse that enables efficient and reliable data integration, transformation, and delivery for all data users across the Enterprise.

We want to ensure our stakeholders can easily self-serve the data and build the reports they need to drive their projects and make decisions. If you're excited to work on a fast-moving team using cutting-edge technologies to collect, store, transform, analyze, and model data, we want to meet you!

Essential Duties and Responsibilities (include the following, and other duties may be assigned):

Data Modeling:

• Design, develop and maintain robust, and scalable data models and schemas to support analytics and reporting requirements.
• Knowledge in data modeling and familiarity with concepts like star schema, snowflake schema, and normalization
• Collaborate with stakeholders and cross-functional teams to understand business requirements and define data structures and relationships.

Data Integration and Data pipelines:

• Integrate data from different sources, both internal and external, to create a unified and comprehensive view of the data.
• Build and maintain the CI/CD pipelines to improve developer productivity, agility and code quality.
• Work closely with cross-functional teams to understand data requirements and ensure successful integration.

ETL Development and Data Integrity:

• Develop, optimize, and maintain ETL/ELT processes to extract, load and transform data from various sources into our Snowflake data platform.
• Implement data quality checks and validation routines within data pipelines to ensure the accuracy, consistency, and completeness of data.
• Interact and coordinate work with other technical and testing members of the team.
• Review and write code that meets set quality gates.

Performance Tuning:

• Optimize data pipelines and data processing workflows for performance, scalability, and efficiency.
• Optimize design efficiency to minimize data refresh lags, improve performance and enable data as a service through reusable assets.

Support:

• Monitor and manage application performance and service quality, including initial troubleshooting, identification of root cases and issue resolution. Implement and improve alerting.
• Perform quality checks and user acceptance testing.

Documentation:

• Create, update, and maintain technical documentation of the data processes, pipelines, and models.

Education and Experience Requirements:

• Bachelor's or Master’s Degree in Computer Science, Information Technology or a related field
• Minimum 5 years of experience in a technical role in data extracts, analysis, and reporting.
• 3+ years of advanced SQL development experience coupled with robust Python or Java programming capabilities. Extensive experience building and optimizing ETL pipelines, data modeling and schema design in cloud data warehousing technologies (Snowflake is preferred, BigQuery, AWS, Azure)
• Strong experience with AWS services such as EC2, S3, Glue, Lambda, and RDS
• Proficiency in DevOps tools and practices, including CI/CD using Git and GitHub, automation, and monitoring.
• Experience with integrating Snowflake with DevOps pipelines.
• Proficiency in one or more coding languages like Python, PySpark, Java
• Strong automation scripting skills and practical experience with AWS, especially using Lambda
• Certification in relevant areas (e.g., Snowflake, AWS Certified Data Analytics, Google Cloud Professional Data Engineer) is preferred
• Proven Experience with reporting and analytical tools like Power BI, Tableau, etc.
• Continuously stay up-to-date on industry trends and advancements in data engineering and analytics.
• Able to work with little supervision while being accountable for individual and departmental results
• Able to multi-task and meet deadlines under pressure
• Solid understanding of data security and compliance requirements.
• Effective communication skills, conveying complex technical concepts to non-technical stakeholders.

Competencies To perform the job successfully, an individual should demonstrate the following competencies:

• Technical Expertise: Maintains current technical knowledge and best practices and actively builds new skills. Provides ideas to help solve business and technical problems for the organization through technical expertise. Able to work across multiple platforms and applications and see interconnections with some guidance.
• Action Orientation: Uses time effectively to complete tasks in a thorough and timely manner. Focuses on most important items first. Maintains high levels of personal organization and works at high level of efficiency and rapid pace. Seeks out challenges and initiates action on issues even when scope is unclear. Maintains composure in times of stress.
• Collaboration: Understands the importance of relationships to ensure team success. Builds positive relationships, uses tact in sensitive situations; listens well to various points of view; relates well to others at all levels. Speaks and writes in a clear and concise manner. Takes time to get to know others outside of his/her immediate functional area. Aligns personal goals to team and organizational goals.
• Communication: Establishes rapport and is straightforward and approachable. Listens carefully, asks pertinent questions, responds effectively and adapts personal style to suit the audience. Actively
• gathers customer viewpoint and feedback to aid in decision-making. Speaks, writes and presents in a clear and concise manner. Uses effective meeting facilitation tools and techniques to attain meeting objectives in a timely manner. Establishes and maintains positive long-term relationships with a diverse network of contacts, including internal and external customers.
• Customer Engagement: Engaging external customers and internal resources to achieve mutually beneficial outcomes in a way that provides an optimal experience for the customer.