Loading…
Shanghai, China
June 24–26, 2019
Click here for more information and registration

Simultaneous translation will be provided for all keynote and breakout sessions.
我们将为所有主题演讲和分组会议提供同声传译服务。

To view the Chinese version of this schedule please go here.
请点击此处查看中文版本。

Venue + Sponsor Showcase Map
场馆 + 赞助商展示区地图
Back To Schedule
Tuesday, June 25 • 11:45 - 12:20
Embracing Big Data Workload in Cloud-Native Environment with Data Locality - Sammi Chen, Tencent & Xiaoyu Yao, Cloudera

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

Tweet Share
Feedback form is now closed.
Kubernetes support schedule workloads based on CPU and memory resource with node affinity, pod affinity and anti-affinity. This works very well for stateless workloads. For stateful workloads, especially big data workloads, scheduling compute close to data source can greatly boost performance, reliability and availability. However, in many cloud based storage systems, the data locality info is either unavailable or not exposed to container orchestra.

In this talk, we will first compare the data locality support from mainstream container attached storage for Kubernetes. Then we will introduce network topology support from Apache Hadoop Ozone and how to use it as locality aware container attached storage via Ozone CSI plugin for better workloads scheduling. Last, we will use Spark on K8s to demo the benefits of data locality aware scheduling with Apache Hadoop Ozone.
Speakers
avatar for Sammi Chen

Sammi Chen

Software Engineer, Tencent
Sammi Chen is a software engineer at Tencent Cloud, working on Apache Hadoop HDFS and Ozone projects. She is a committer and PMC member of Apache Hadoop Projects.
avatar for Xiaoyu Yao

Xiaoyu Yao

Principal Software Engineer, Cloudera
Xiaoyu Yao is a principal software engineer at Cloudera Inc., working on Apache Hadoop HDFS and Ozone projects. He is a committer and PMC member of Apache Hadoop and Ratis Projects with 12 years of experience developing and supporting distributed storage and file system.

KubeCon China 2019 xyao 0619 pdf
Tuesday June 25, 2019 11:45 - 12:20 CST
609
  KC+CNC - Storage