Twitter Heron at Scale

存储架构 2016-06-08

Twitter generates billions and billions of events per day. Analyzing these events in real time presents a massive challenge. Twitter designed and deployed a new streaming system called Heron. Heron has been in production nearly 2 years and is widely used by several teams for diverse use cases. In this talk, we will share our operating experiences and challenges of running Heron at scale and the approaches that we took to solve those challenges.

Karthik Ramasamy is the engineering manager and technical lead for Real Time Analytics at Twitter. He is the co-creator of Heron and has more than two decades of experience working in parallel databases, big data infrastructure and networking. He cofounded Locomatix, a company that specializes in real time streaming processing on Hadoop and Cassandra using SQL that was acquired by Twitter.

This talk is from the DataEngConf SF
in April 2016.

责编内容by:Hakka Labs (源链)。感谢您的支持!

您可能感兴趣的

The Case for Heron For the past few months, I’ve been teaching at companies who are heavy users of Apache Storm. They’re also undertaking massive projects to move off...
Architecting and building end-to-end streaming app... In this episode of theData Show, I spoke with Karthik Ramasamy , adjunct faculty member at UC Berkeley, former engineering manager at Twitter, ...
Twitter开源了看上去完爆Apache Storm的Heron!... 去年才说 Heron 短时间内不会开源,这才一年时间就开源了。嘴上说着不要,身体倒是很诚实嘛~ 去年,Twitter 对外宣布了新的分布式流计算系统 Heron,随后消息称 Twitter 已经用 Heron 替换了 Storm。此举将吞吐量最高提升了 14 倍,单词计数拓扑时间延迟...
Twitter开源了流数据处理引擎Heron Twitter 宣布 开源Heron。Heron是 Apache Storm 的后继者,也是一种流数据处理引擎。为方便开发人员对Heron的采用,Heron向后兼容Apache Storm。Heron所给出的可扩展性、调试能力、在共享集群架构中的工作能力以及更优的性能,使得其在T...
Had it with Apache Storm? Heron swoops to the resc... Last year, Twitter dropped two bombshells. First, it would no longer use Apache Storm in production. Second, it had replaced it with a hom...