RDD take 和 takeOrdered 方法

http://stackoverflow.com/questions/33563575/on-sparks-rdds-take-and-takeordered-methods

In order to explain how ordering works we create an RDD with integers from 0 to 99:
val myRdd = sc.parallelize(Seq.range(0, 100))

We can now perform:
myRdd.take(5)

Wich will extract the first 5 elements of the RDD and we will obtain an Array[Int] containig the first 5 integers of myRDD: '0 1 2 3 4 5' (with no ordering function, just the first 5 elements in the first 5 position)
The takeOrdered(5) operation works in a similar way: it will extract the first 5 elements of the RDD as an Array[Int] but we have to opportunity to specify the ordering criteria:
myRdd.takeOrdered(5)( Ordering[Int].reverse)

Will extract the first 5 elements according to specified ordering. In our case the result will be: '99 98 97 96 95'
If you have a more complex data structure in your RDD you may want to perform your own ordering function with the operation:
myRdd.takeOrdered(5)( Ordering[Int].reverse.on { x => ??? })

Which will extract the first 5 elements of your RDD as an Array[Int] according to your custom ordering function.

最后编辑于
©著作权归作者所有,转载或内容合作请联系作者
平台声明:文章内容(如有图片或视频亦包括在内)由作者上传并发布,文章内容仅代表作者本人观点,简书系信息发布平台,仅提供信息存储服务。

推荐阅读更多精彩内容

  • **2014真题Directions:Read the following text. Choose the be...
    又是夜半惊坐起阅读 13,413评论 0 23
  • 假如我永远离去, 假如新爱之网压死旧爱, 仍要记住我呀! 假如我近在咫尺, 你视而不见,我像似有似...
    黑色狼毒草O阅读 1,527评论 0 1
  • 秋夜无言愁乱,凄风凉雨孤单。 霓虹避掩雾街寒,半点炊烟难见。 丹桂坠落庭院,黄花胡乱幽怜。 一壶酩酊醉皇天,酣畅周...
    寒菊阅读 3,233评论 12 6
  • *问题:字体在屏幕上显示的时候需要将字体栅格化为一个个像素点,采用黑白像素点渲染,无法体现字体的细节之处,会造成边...
    Axiba阅读 5,147评论 0 0