Window Operations詳解

window(windowLength, slideInterval)：返回窗口長度爲windowLength，每隔slideInterval滑動一次的window DStream

countByWindow(windowLength, slideInterval)：返回窗口中元素的個數

reduceByWindow(func, windowLength, slideInterval)：對window中的元素作reduce操做

// x, y 是window中的元素
val ds1 = wordCounts.reduceByWindow((x, y) => {
    println(x)
    println(y)
    x
}, Seconds(30), Seconds(20))　

reduceByKeyAndWindow(func, windowLength, slideInterval, [numTasks]) 針對window內的數據作reduceByKey

// x y 是相同key的value 
wordCounts.reduceByKeyAndWindow((x: Int, y:Int) => x + y, Seconds(30), Seconds(20))　

reduceByKeyAndWindow(func, invFunc, windowLength, slideInterval, [numTasks]): invFunc：假設invFunc的參數爲x和y，那麼x是上個window通過func操做後的結果，y爲這次window與上次window在時間上交叉的元素通過func操做後結果

sc.setCheckpointDir("D://checkpoints/")
// m是上個window key相同的元素的reduceByKeyAndWindow第一個參數操做後的結果，n爲上個window與當前window在時間上不重複的key相同的元素的reduceByKeyAndWindow第一個參數操做後的結果
val ds1 = wordCounts.reduceByKeyAndWindow((x, y) => x + y, (m, n) => { m - n}, Seconds(10), Seconds(10))

　　

// 這個方法的做用和<strong>reduceByKeyAndWindow(func, windowLength, slideInterval, [numTasks])相同
wordCounts.reduceByKeyAndWindow((x, y) => x + y, (x, y) => x - y, Seconds(10), Seconds(10))

　　

countByValueAndWindow(windowLength, slideInterval, [numTasks])：window中key出現的次數