Data Engineer

Responsibilities:
• Drive the technical design of large-scale data platforms, utilizing modern and open source technologies in a cloud environment.
• Optimize data and data pipeline architecture, as well as optimizing data flow and collection.
• Be able to work on complex data engineering efforts and lead technical teams through the solution design process.
• Research, evaluate and formally recommend third party software and technology package.
• Stay abreast of emerging technologies and projects in the modern Data Engineering/Machine Learning space.
Requirements:
• Computer Science or Math background preferred.
• Hands-on experience with IT platform implementation in a highly technical and analytical role.
• Hands-on experience with Big Data platforms such as EMR, Hadoop and Spark.
• Experience with AWS, GCP or Alicloud.
• Fluency in at least 2 object-oriented language, preferably Python and java or Scala.
• Hands-on experience with image data pre-processing, image processing, image denoising, etc.
• Hands-on experience with Python libraries such as cv2, PIL, etc.
• Experience Architecting Big Data platforms using Apache Hadoop, Cloudera, Hortonworks and MR distributions.
• Experience in JavaScript, NPM, Git Command, Linux
• Proficient in SQL; experience with Hive.
• Experience with API design and deploy parallel processing frameworks.
• Knowledge and ability to build data science and machine learning applications from end-to-end.
• Experience in using business analysis and project management skills
• Experience in managing development team.
• Technical Development experience and methodologies
• Ability to document processes, writing of software design specifications and user cases.
• Ability to interact with vendor, users and third parties as appropriate.
• Experience in best-practice design, consulting and research analysis.