Shortlisted candidate will be assigned to data engineering team in Digital Content Department, which is also be part of Rakuten Global Data Office. Team scope is:
Design and develop Rakuten Global Data Platform for US and EU,
Work closely with Rakuten Japan Data Team for internal data project improvement,
Build data driven best practice for Rakuten Media & Sport Companies and Rakuten US business,
Fully improve the data utilization and data driven innovation in Rakuten
Responsibilities:
Involve in whole data platform development process including:
infrastructure,
data platform service development,
data ETL (batch and streaming),
data solution implementation.
POC and adopting new technologies to improve data platform management in large scale high throughput.
Qualifications:
Must-have
Decent programming skills and SQL skills.
Knowledge in distributed system e.g. mesos, Hadoop, spark, kafka.
Knwoledge in different type of DB syste e.g. Relational DB, Column DB, Document DB, KV.
knowledge in LINUX based system operation ans shell scripting.
Knowledge in RESTful API development.
Experience in java, python and scala programming.
Comfortable with git version control.
Good knowledge in System, application status monitoring.
Enthusiasm and an interest in new open source project
Non-Business hour emergency support for very critical issues.
Large Scale data pipeline management experience
Streaming pipeline development experience