SparkStream監聽socket時是當客戶端使用

當SparkStream經過socket方式(TCP)獲取輸入的數據源時,其自己是做爲客戶端去鏈接服務器端的,並非本身做爲服務器端等待客戶端發送數據給本身。java

​
​
public class TestSparkStream implements Serializable{

    public static void main(String[] args) throws InterruptedException {
        SparkConf sparkConf = new SparkConf();
        sparkConf.setMaster("local[2]");
        sparkConf.setAppName("test_spark_st");
        sparkConf.set("spark.ui.enabled","false");
        Duration du = new Duration(5000);
        JavaStreamingContext sc = new JavaStreamingContext(sparkConf,du);
        //這裏做爲客戶端,鏈接服務器
        JavaReceiverInputDStream<String> stream = sc.socketTextStream("101.218.118.144", 9999, StorageLevel.MEMORY_ONLY());

        stream.print();
        sc.start();
        sc.awaitTermination();
        sc.stop();
    }
}

​

​
相關文章
相關標籤/搜索