About our group:
We are a team of experienced IT MES (Manufacturing Execution System) business analysts and developers based in Woodlands. We focus on both business and IT initiatives to support and improve the operational and functional efficiency of the Global MES Tracking System, deployed at regional sites in Singapore, Malaysia, US, Thailand and China.
Our key focus is to design, develop, and manage data models using data pipelines (ETL/ELT) for structured and unstructured data from various sources. If you're a Master of the “big picture”, with demonstrated success in developing Big Data solutions where you build massive Data Lakes, then we want to talk with you. Snap up on this opportunity for your next big career challenge to create business value as you showcase your strength in technical architecture for designing systems and leading projects. You'll primarily work with our BigData and Analytics group and based in our Singapore and Pune,India office.
About the role - you will:
- Part of 10-12 Platform Engineers that are the crux for developing and maintaining Big Data (Data Lake, Data Warehouse and Data Integration) and advanced analytics platforms at SeaGate
- Apply your hands-on subject matter expertise in the Architecture of and administration of Big Data platforms - Data Warehouse Appliances and experience with ML and Data Science platforms (Spark ML , H2O , KNIME)
- Develop and manage SPARK ETL Frameworks, Data orchestration with Airflow and support building PRESTO/Trino queries for the key stakeholders
- Design, scale and deploy Machine Learning pipelines.
- Collaborate with Application Architects and Business SMEs to design and develop end-to-end data pipelines and supporting infrastructure.
- Testing and troubleshooting to ensure system reliability, performance, and usability
- Managing the project by handling multiple tasks, collaborating with mentors and stakeholders, and ensuring timelines are met
About you:
- Excellent coding skills in any language with a deep desire to learn new skills and technologies.
- A passionate professional who is up to the challenge of blending the fast-changing technology landscape of Big Data analytics with the complex and high-impact space of HiTech and Manufacturing analytics
- Exceptional data engineering skills in large, high-scale Data platforms and applications using cloud and big data technologies like Hadoop ecosystem and Spark
- Highly independent and able to work with minimal supervision
- Adaptable to changes and responsive to evolving project needs
- Strong communication and interpersonal skills for effective collaboration
- Passionate and committed to delivering quality work
- Proactive and self-motivated with a problem-solving mindset
Your experience includes:
- Pursuing a degree in Computer Science, Software Engineering, or a related field Big data processing frameworks knowledge: Spark, Hadoop, Hive, Kafka, EMR
- Big data solutions on cloud (AWS or Other)
- Data Warehouse Appliances, Hadoop (AWS EMR), Data Lake Technologies (AWS S3/GCS/Other) and experience with ML and Data Science platforms (Spark ML , H2O , KNIME )
- Python, Java, Scala
- DevOps, Continuous Delivery, and Agile development
- Creating a culture of technical excellence by leading code and design reviews, promoting mentorship, and identifying and promoting educational opportunities for engineers
