Senior Data Engineer (North America)
CircleCI
We are looking for a Senior data engineer to make actionable data a part of our product and business strategy. In this role, you will:
Build mission-critical data pipelines and datasets, Architect a cutting-edge data platform, and Champion data quality, data security, and data governance.
About CircleCI
CircleCI is the best platform for software teams looking to rapidly build quality projects, at scale. Our intelligent continuous integration and delivery tools are simple yet powerful. Our aim is to provide the wisdom of a connected development ecosystem to every team member making technology decisions.
We run 12M+ builds a month on our platform for companies like Spotify, Kickstarter, Sony, and Coinbase. Over 25,000 organizations and 300,000 developers actively build, test, and deploy on CircleCI. We’ve raised $59.5M in venture capital from Industry Ventures, Top Tier Capital, Scale Venture Partners, DFJ, Harrison Metal Capital, and Baseline Ventures.
About Data at CircleCI
Data is at the heart of the CircleCI’s product and business strategy. The Data team is responsible for data engineering and data science at CircleCI and is critical to CircleCI’s success. The Data team collaborates closely with DevOps, Engineering, Product Management, Marketing, Finance and leadership to build a data-driven company.
What you’ll do:
Data pipelines and datasets:
Ingest data from production, system-of-record and third-party data sources.
Design, build, maintain and manage complex data pipelines.
Publish data to reporting tools, through RESTful API and microservices.
Create and document conceptual and logical data models.
Build, clean and maintain strategic datasets. (Approximately 40% of your time).
Data architecture and data platform: Evolve our data architecture.
Assess strengths and weakness of our data stack.
Roadmap capabilities. Implement raw, trusted, reporting and sandboxed zones in our data lake.
Implement prod and non-prod environments. Build an experimentation platform. (Approximately 30% of your time)
Data quality, security and governance:
Champion data quality.
Drive unit and integration testing.
Improve monitoring.
Eliminate bottlenecks and failure points.
Enable scale.
Automate, automate, automate. Improve data security.
Build data governance processes. (Approximately 30% of your time.)
What we're looking for:
Data engineering experience:
6+ years in the trenches of data engineering. In junior and senior roles. Working with batch and streaming pipelines. Working on quick-turnaround as well as strategic projects.
Programming experience:
2+ additional years as a software engineer. Fluent in an object-oriented or functional language. Ideally, Python or Java. Maybe, Clojure or C++. Comfortable with today’s SDLC and CI/CD. A strong technical education. Ideally, a Bachelor’s degree or higher in Computer Science,
Engineering or another technical field.
Hands-on with a big-data, distributed, cloud-based toolkit: Able to MacGyver big-data solutions from multiple tools, frameworks and technologies, including Spark, Docker, Kubernetes, Linux, Bash and AWK. Familiar with an AWS data stack, including some of the following: S3, RDS, Athena, ECR, EMR, Lambda, Data Pipelines, Glacier, Neptune, Redshift and CLI. Comfortable with distributed systems. Worked with billion-row datasets, petabytes of storage and 10+ hour pipelines.
Data skills: Fluent in SQL. Understand data access and storage in theory and practice. Know relational and NoSQL datastores, including PostgreSQL and MongoDB. Designed and documented enterprise data models.
Soft skills: Passionate about data. Able to learn from, and mentor, teammates. Entrepreneurial. Willing to move fast. Looking to grow. Effective communicator and listener. Ability to balance vision and execution.
If you’re interested in joining the Data team at CircleCI, please send a resumé and cover letter describing what inspires you and why you think you are a good fit for the team. If you contribute to an open source project, write a blog, or have a presence on the web (Github, Stack, Quora, etc.), we would love to hear about it.
CircleCI is proud to be an Equal Opportunity and Affirmative Action employer. We do not discriminate based upon race, religion, color, national origin, sexual orientation, gender, gender identity, gender expression, transgender status, sexual stereotypes, age, status as a protected veteran, status as an individual with a disability, or other applicable legally protected characteristics. We also consider qualified applicants with criminal histories, consistent with applicable federal, state and local law.
Similar Jobs
