Who we're looking for
We are looking for a Senior Data Engineer to join our growing team in Hong Kong.
What you will be doing
Define the architecture of Measurable AI Analytics platform.
Build robust and scalable foundation of data, from logging to streaming pipelines, data filtering and email parser.
Implement our data stack using modern technologies such as Apache Spark, Kafka, HDFS and Cassandra.
Optimize our data pipelines and establish best practices for table schemas, data models and data storage.
Build efficient solutions on top of AWS, Azure using Python, Scala and Go.
Minimum of 4 years experience building scalable backend systems.
Experience working on, and deploying, large scale systems
Experience working with and managing varied forms of distributed data systems such as Kafka, HDFS, Storm or Spark.
Experience working with popular cloud solutions such as AWS, Azure or GCP.
Knowledge of Docker and Kubernetes.
Love to work with high volume heterogeneous data and distributed systems.
Proven track record of working with large scale data infrastructure.
Experience working with Hadoop, Cassandra, MongoDB, Redis or MySQL.
Expert knowledge of SQL, MapReduce.
MS / PhD in Computer Science, Applied Mathematics or a related field.
Server side shell scripting experience.