As a Level 4 Data Engineer, you will be responsible for independently owning and delivering features and small projects that support our team's goals. You will own key components of our data systems, ensuring they are reliable, scalable, and well-maintained.
Responsibilities:
- Feature and Project Ownership
- Independently own and deliver features and small projects on data engineering, from design to deployment, that advance the team's goals.
- Take responsibility for the development, operation, and maintenance of specific components within our data ecosystem.
- Engineering & Operational Excellence
- Contribute to investments that improve code quality, reduce technical debt, and modernize our technology stack.
- Write reliable, performant, and scalable code.
- Ensure comprehensive testing for your features, including unit and end-to-end tests.
- Monitor the stability of your deployed code, proactively identifying and fixing bugs.
- Contribute to the team's operational responsibilities, including on-call rotations and SEV handling, and propose improvements to processes.
- Develop and optimize DAGs for reliability, performance, and maintainability using Apache Airflow 2.0 and Astronomer.
- Planning & Collaboration
- Accurately evaluate and estimate the effort for assigned tasks and features.
- Participate in roadmapping discussions and provide valuable feedback.
- Write clear technical documentation and runbooks for your work.
- Lead with support as needed in code reviews, post-mortems, and planning discussions.
- Collaborate with cross-functional partners (Product, Analytics, and Data Science) to understand data needs and deliver solutions and reliable data workflows
Experience:
- Required
- 5+ years of experience in data engineering and data platforms, particularly with cloud technologies like AWS, Databricks, and Snowflake.
- Proficiency in Python and SQL.
- Experience with big data compute and storage technologies (e.g., Spark, Trino, Hive, Cloud Storage).
- Experience in applying software development practices to data, including testing and CI/CD.
- A good understanding of analytic and data needs within corporate functions.
- Experience in creating and implementing frameworks and APIs for automated data management and governance.
- Proven ability to deliver features independently and collaborate effectively with others.
- Preferred
- Experience with Graph databases, Vector databases, and Conversational analytics.
- Experience in building Agentic applications for data engineering and operations.
