技术控

    今日:123| 主题:49331
收藏本版 (1)
最新软件应用技术尽在掌握

[其他] Cyclades: Conflict-free asynchronous machine learning

[复制链接]
我怀念的 发表于 2016-10-4 13:28:43
157 6

立即注册CoLaBug.com会员,免费获得投稿人的专业资料,享用更多功能,玩转个人品牌!

您需要 登录 才可以下载或查看,没有帐号?立即注册

x
CYCLADES: Conflict-free asynchronous machine learning Pan et al. NIPS 2016
   “Conflict-free,” the magic words that mean we can process things concurrently or in parallel at full speed, with no need for coordination. Today’s paper introduces Cyclades, a system for speeding up machine learning on a single NUMA node. In the evaluation, the authors used NUMA nodes with 18 CPUs and 1TB memory. Extending Cyclades to work across NUMA nodes, or even in a distributed setting using aparameter server is reserved for future investigation.
  The overall hierarchy of ML speed and scale probably looks something like this:
  
       
  • Clusters of GPUs [w. CPU support] – Speed and Scale   
  • Multi-GPUs on a single node [w. CPU support] for SPEED,Clusters of CPUs for SCALE   
  • Multi-core CPUs on a single node   
  • Single core (thread) systems  
   Remember not to underestimate what can be achieved on a single thread . Because of their accessibility and ease of programming, systems that work on a single box can have great utility. Cyclades efficiently utilises multiple cores through a graph partitioning step that allocates connected-component subgraphs to cores, eliminating the need for cross-core coordination. One of the compelling things that is unique about Cyclades is that unlike other systems we’ve seen ( Asynchronous Complex Analytics ,Petuum) that use asynchronous processing to speed up ML, Cyclades maintains serial equivalence . That is, it guarantees to give the same results as a serial implementation. Without serial equivalence, other systems rely on analyses that the deviation from serial is not significant (stays within some bound).
   Since it returns exactly the same output as a serial implementation, any algorithm parallelized by our framework inherits the correctness proof of the serial counterpart without modifications. Additionally, if a particular heuristic serial algorithm is popular, but does not have a rigorous analysis, such as backpropagation on neural networks , Cyclades still guarantees that its execution will return a serially equivalent output.
   Cyclades works for a large family of algorithms based on stochastic updates . This includes logistic regression, least squares, support vector machines, word embeddings, stochastic gradient descent, matrix completion and factorization, and more.
  At the core of Cyclades is a clever partitioning strategy. Consider a graph with two types of vertices (bipartite), update functions and model variables. We create an edge from an update function to a variable if that function reads or writes it.
   
Cyclades: Conflict-free asynchronous machine learning-1 (asynchronous,machine,machine,learning,machine,learning,ex6,machine,learning,pdf,machine,learning,ex4)

   From this we can create a conflict graph (Cyclades never actually fully materializes such a graph, but it’s a useful conception). The conflict graph contains only update function vertices, and links two vertices with an edge if they share at least one variable in the update graph.

Cyclades: Conflict-free asynchronous machine learning-2 (asynchronous,machine,machine,learning,machine,learning,ex6,machine,learning,pdf,machine,learning,ex4)

  The key now is to partition the conflict graph into connected components, and allocate those connected components to cores.
1234下一页
友荐云推荐




上一篇:Python科学计算与数据分析库/包大全
下一篇:Classic Emacs editor gets a new-school makeover
酷辣虫提示酷辣虫禁止发表任何与中华人民共和国法律有抵触的内容!所有内容由用户发布,并不代表酷辣虫的观点,酷辣虫无法对用户发布内容真实性提供任何的保证,请自行验证并承担风险与后果。如您有版权、违规等问题,请通过"联系我们"或"违规举报"告知我们处理。

我爱她她不爱我 发表于 2016-10-4 14:28:59
我是一个有名望的恶棍…
回复 支持 反对

使用道具 举报

坤彩煜 发表于 2016-10-6 21:05:04
煮熟的鸭子飞不了,煮熟的楼主在哪里?
回复 支持 反对

使用道具 举报

不该有的小情绪 发表于 2016-10-9 11:05:18
爱我的人请继续,恨我的人别放弃.
回复 支持 反对

使用道具 举报

贾学磊 发表于 2016-10-17 13:17:34
我也是坐沙发的
回复 支持 反对

使用道具 举报

幼珊 发表于 2016-11-3 18:54:13
楼上的能详细介绍一下么?
回复 支持 反对

使用道具 举报

蕾雁 发表于 2016-11-7 15:03:52
小时候的梦想并不是要当什么科学家,幻想自己是地主家的少爷,家有良田千顷,终日不学无术,没事领着一群狗奴才上街去调戏一下良家少女……
回复 支持 反对

使用道具 举报

*滑动验证:
您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

我要投稿

推荐阅读

扫码访问 @iTTTTT瑞翔 的微博
回页顶回复上一篇下一篇回列表手机版
手机版/CoLaBug.com ( 粤ICP备05003221号 | 文网文[2010]257号 )|网站地图 酷辣虫

© 2001-2016 Comsenz Inc. Design: Dean. DiscuzFans.

返回顶部 返回列表