Senior Data Engineer
JBS Custom Software Solutions
JBS is looking for a Senior Data Engineer to help build out an enterprise data lake and help design and implement related ETL processes. The role requires knowledge of a variety of data warehouse and data lake architectures and experience with many of the AWS data services and tools. Technologies included are AWS EMR/Apache Spark, Python, AWS Lambda, AWS Step Functions, and AWS RDS (PostgreSQL). Required of this role is also experience with dimensional modeling, OLAP design principles and knowledge of how to handle the operational aspects of such systems, such as fault tolerance, performance, efficient data partitioning, security, and data governance. As a senior level architect, this role will be working closely with the implementation team and product owners.
Required Skills
Key Responsibilities
- Translate requirements to appropriate data models and designs
- Architect ETL pipelines using EMR for transformations and Step Functions for workflow/coordination
- Develop and enhance existing code for performance, scalability and fault tolerance
- Work with product owners and help support implementation teams in translating requirements and designs
Basic Qualifications
- Minimum of 5 years of architecting and building data warehouse ETL solutions
- Dimensional Modelling/OLAP design
- SQL
- Analytical Query Development (Aggregates, window functions, statistical functions)
- Designing for fault tolerance
- Designing for performance/scale
- Understanding of data governance issues
- Understanding of security controls and data access
Preferred Qualifications
- AWS Redshift
- Experience with typical data lake file formats – Parquet, Avro, ORC, etc.
- Knowledge/Experience with AWS data lake-centric services (Glue, Lake Formation, S3, EMR, etc.)
- Experience with setting up workflows with AWS Step Functions and AWS Lambda
- Experience with typical Python data libraries (Pandas, Numpy, etc.)
- Experience with Python building ETL and data-centric solutions
Benefits
- Competitive base salary
- Paid overtime
- Generous PTO policy, company holidays
- 401k with company match
- Health, Dental, Life, LTD
As an equal opportunity employer, JBS does not discriminate in hiring or terms and conditions of employment on the basis of any federal, state, or locally protected class. JBS only hires individuals authorized for employment in the United States on a W2 basis.
Similar Jobs
