tags: gsacjavascript
[TOC]java
系統 : CentOS release 6.8 (Final) _x64 集成工具 : xampp 1.8.1 (包含 mysql 5.5.27, proFtp) 軟件 : DataWorks-SW jdk : java version "1.7.0_111"python
在 DataWorks-SW
壓縮包中,存在安裝說明,如下的安裝步驟就是根據安裝說明來的,若是須要了解更詳細的安裝說明,請閱讀 README_Dataworks_Installation_Operation
文件。mysql
按照文件 Dataworks-SW/database/README_create_MySQL_database
的說明來建立數據庫。 使用 /opt/lampp/bin/mysql -u root -p
命令登陸數據庫。linux
mysql> source /home/data/apps/Dataworks-SW/database/Dataworks_demo.sql; # 導入數據文件,會自動建立數據庫 mysql> show tables; # 查看數據庫的表狀況 +-----------------------------------+ | Tables_in_Dataworks_GSAC_database | +-----------------------------------+ | access | | agency | | antenna | | country | | datafile | | datafile_type | | ellipsoid | | equip_config | | locale | | metpack | | monument_style | | network | | radome | | receiver_firmware | | station | | station_status | | station_style | +-----------------------------------+ 17 rows in set (0.00 sec)
不能有任何錯誤,不然須要從新建立。git
GSAC
, 而且測試GSAC
的安裝分爲兩個部分, Part 1
的說明在 Dataworks-SW/dataworks-gsac/src/org/gsac/README
中,Part 2
的說明在 Dataworks-SW/dataworks-gsac/src/org/dataworks/gsac/README
, 按照兩部分的說明來安裝。web
Part 1
ant
構建工具來構建java項目,命令 yum install ant
執行安裝。 注意: 由於我用的jdk
版本爲 1.7
,因此能夠直接構建,若是使用的jdk
版本爲1.6
,必定要參照說明。Dataworks-SW/dataworks-gsac/src/org/gsac
所在目錄ant
命令.repositorymap.js
,在dataworks-gsac/src/org/gsac/gsl/htdocs/repositorymap.js
打開repositorymap.js
文件,按照提示設置/* set zoom level for map of GSAC sites; zoom level 3 is about 10,000 km wide; 4 ~ 4000 km; 5 ~ 2000 km; 6 ~ km.設置地圖顯示的距離大小是多少千米,分爲 3, 4, 5, 6 四個等級 */ var defaultZoomLevel = 5; /* set center of GSAC site map, with (LONGITUDE, LATITUDE ) ; longitude east (west is negative), latitude north 這是中心座標點,這裏我用的房山的座標點*/ var defaultLocation = new OpenLayers.LonLat(39.00, 116.00);
dataworks-gsac/src/org/gsac/gsl/
目錄下執行ant
命令Part 2
經過命令進入 cd dataworks-gsac/src/org/dataworks/gsac/
目錄,按照README
的說明完成第二部分。sql
resources/gsac.properties
文件,參照說明,若是沒有須要,能夠不用修改dbresources/gsacdb_test.properties
文件,設置數據庫相關信息## This is the database name and account information for GSAC gsac.db.username=root gsac.db.password=chgsac gsac.db.jdbcurl=jdbc:mysql://127.0.0.1:3306/Dataworks_GSAC_database?useUnicode=yes&characterEncoding=UTF-8
org/dataworks/gsac
目錄下的 build.xml
文件<target name="localinit"> <!-- base name for build products, 此處的 `dataworksgsac` 要保證相同 --> <property name="basename" value="dataworksgsac"/>
org/dataworks/gsac
目錄下執行 ant
命令[root@localhost gsac]# ant Buildfile: build.xml test_build: localinit: init: clean: build: [echo] Creating test build [copy] Copying 1 file to /home/data/apps/Dataworks-SW/dataworks-gsac/src/org/dataworks/gsac/resources [copy] Copying 1 file to /home/data/apps/Dataworks-SW/dataworks-gsac/src/org/dataworks/gsac/resources localinit: init: clean: compile: [javac] Compiling 7 source files [javac] \u6ce8: /home/data/apps/Dataworks-SW/dataworks-gsac/src/org/dataworks/gsac/DataworksSiteManager.java\u4f7f\u7528\u4e86\u672a\u7ecf\u68c0\u67e5\u6216\u4e0d\u5b89\u5168\u7684\u64cd\u4f5c\u3002 [javac] \u6ce8: \u6709\u5173\u8be6\u7ec6\u4fe1\u606f, \u8bf7\u4f7f\u7528 -Xlint:unchecked \u91cd\u65b0\u7f16\u8bd1\u3002 gsacjar: [echo] Making jar /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsactest1.0.jar [jar] Building jar: /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsactest1.0.jar release: [echo] Making /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsactest1.0.zip [delete] Deleting directory /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsactest1.0 [mkdir] Created dir: /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsactest1.0 [copy] Copying 1 file to /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsactest1.0 [copy] Copying 1 file to /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsactest1.0 [copy] Copying 1 file to /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsactest1.0 [copy] Copying 1 file to /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsactest1.0 [copy] Copying 1 file to /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsactest1.0 [copy] Copying 1 file to /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsactest1.0 [copy] Copying 1 file to /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsactest1.0 [copy] Copying 1 file to /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsactest1.0 [copy] Copying 1 file to /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsactest1.0 [copy] Copying 1 file to /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsactest1.0 [copy] Copying 1 file to /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsactest1.0 [copy] Copying 1 file to /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsactest1.0 [copy] Copying 1 file to /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsactest1.0 [zip] Building zip: /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsactest1.0.zip localinit: init: war: [copy] Copying 1 file to /home/data/apps/Dataworks-SW/dataworks-gsac/dist [delete] Deleting: /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsactest1.0/javax.servlet-api-3.1.0.jar [war] Building war: /home/data/apps/Dataworks-SW/dataworks-gsac/dist/dataworksgsac.war [echo] ******************************** [echo] To run the server run: [echo] ant runserver [echo] ******************************** BUILD SUCCESSFUL Total time: 4 seconds
GSAC
執行命令 ant runserver
, 結果報錯[java] Error:java.lang.RuntimeException: java.lang.RuntimeException: java.lang.RuntimeException: java.security.AccessControlException: access denied ("javax.management.MBeanTrustPermission" "register") [java] java.lang.RuntimeException: java.lang.RuntimeException: java.lang.RuntimeException: java.security.AccessControlException: access denied ("javax.management.MBeanTrustPermission" "register")
解決錯誤:設置Javax Security java.policy file
編輯/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.111.x86_64/jre/lib/security/java.policy
文件shell
grant { // many other lines // JMX Java Management eXtensions # 添加的註釋 permission javax.management.MBeanTrustPermission "register"; # 添加的內容 };
再來執行 ant runserver
,啓動成功數據庫
[root@localhost gsac]# ant runserver Buildfile: build.xml localinit: init: runserver: [java] JVM args ignored when same JVM is used. [java] 2016-09-07 10:55:18.814:INFO::main: Logging initialized @467ms [java] GSAC: GsacServlet:initServlet: making repository:org.dataworks.gsac.DataworksRepository [java] GSAC: Starting this GSAC server, at 2016-09-07T02:55:19 +0000 [java] Running stand-alone GSAC server at: http://localhost:8080/dataworksgsac [java] 2016-09-07 10:55:19.758:INFO:oejs.Server:main: jetty-9.2.6.v20141205 [java] 2016-09-07 10:55:19.831:INFO:oejsh.ContextHandler:main: Started o.e.j.s.ServletContextHandler@29ab993{/,null,AVAILABLE} [java] 2016-09-07 10:55:19.904:INFO:oejs.ServerConnector:main: Started ServerConnector@4a6e1c9a{HTTP/1.1}{0.0.0.0:8080} [java] 2016-09-07 10:55:19.905:INFO:oejs.Server:main: Started @1562ms
登陸網站 http://localhost:8080/dataworksgsac
訪問,可以打開網頁看到 GSAC
內容。 至此 GSAC
安裝完成。
導入數據使用的是Dataworks-SW/mirror
目錄下的 mirrorStations.py
和 mirrorData.py
這兩個 python
腳本, mirrorStations.py
的做用是導入站點數據,mirrorData.py
的做用是根據導入的站點來導入真實的數據信息到數據庫。 注意: 兩個腳本在執行的時候必須先導入站點數據,才能再執行導入數據的腳本。不能顛倒!!!
在執行腳本以前須要安裝 python-mysqldb
,在 CentOS
系統中使用命令 yum install MySQL-python.x86_64
來安裝 命令解析: ./mirrorStations.py "STK2;MIZU;KSMV;TSK2" dbHostname dbAccount dbPasswd dbName
` 執行腳本命令 要導入的站點 ; 隔開 數據庫ip地址 用戶名 密碼 數據庫名
# 導入國外站點 [root@localhost mirror]# ./mirrorStations.py "DAEJ;SUWN;YUNS;YSSK;STK2;MIZU;KSMV;TSK2;MTKA;KGNI;USUD;SMST;AIRA;GMSD;CCJ2;MAGO;YAKT;TIXI;NRIL;IRKJ;NBADG;NOVM;NVSK;ARTU;ULAB;KRTV;PODG;SUMK;POL2;CHUM;BIKO;KAZA;TALA;TASH;KIT3;MANM;SYBC;CHLM;DNC4;LMJG;DNSG;JMSM;MSKT;BMCL;DNGD;CUSV;BAKO;BNOA;BTNG;PNGM;PIMO;PIAG" localhost root root dataworks_gsac_database Traceback (most recent call last): File "./mirrorStations.py", line 996, in <module> gsacdb = MySQLdb.connect(dbhost, dbacct, dbacctpw, dbname) File "/usr/lib64/python2.6/site-packages/MySQLdb/__init__.py", line 81, in Connect return Connection(*args, **kwargs) File "/usr/lib64/python2.6/site-packages/MySQLdb/connections.py", line 187, in __init__ super(Connection, self).__init__(*args, **kwargs2) _mysql_exceptions.OperationalError: (2002, "Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)")
報錯:Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock
解決辦法以下:
# 導入國外站點 [root@localhost mirror]# mkdir -p /var/lib/mysql [root@localhost mirror]# ln -s /opt/lampp/var/mysql/mysql.sock /var/lib/mysql/mysql.sock # 建立軟鏈接 [root@localhost mirror]# ./mirrorStations.py "DAEJ;SUWN;YUNS;YSSK;STK2;MIZU;KSMV;TSK2;MTKA;KGNI;USUD;SMST;AIRA;GMSD;CCJ2;MAGO;YAKT;TIXI;NRIL;IRKJ;NBADG;NOVM;NVSK;ARTU;ULAB;KRTV;PODG;SUMK;POL2;CHUM;BIKO;KAZA;TALA;TASH;KIT3;MANM;SYBC;CHLM;DNC4;LMJG;DNSG;JMSM;MSKT;BMCL;DNGD;CUSV;BAKO;BNOA;BTNG;PNGM;PIMO;PIAG" 127.0.0.1 root root dataworks_gsac_database Connected to the database dataworks_gsac_database with account root, on 127.0.0.1 First, get site and equipment metadata at those sites from the UNAVCO GSAC. The GSAC API Linux command is /usr/bin/curl -L "http://www.unavco.org/data/web-services/gsacws/gsacapi/site/search?output=sitefull.csv&site.interval=interval.normal&site.code=DAEJ;SUWN;YUNS;YSSK;STK2;MIZU;KSMV;TSK2;MTKA;KGNI;USUD;SMST;AIRA;GMSD;CCJ2;MAGO;YAKT;TIXI;NRIL;IRKJ;NBADG;NOVM;NVSK;ARTU;ULAB;KRTV;PODG;SUMK;POL2;CHUM;BIKO;KAZA;TALA;TASH;KIT3;MANM;SYBC;CHLM;DNC4;LMJG;DNSG;JMSM;MSKT;BMCL;DNGD;CUSV;BAKO;BNOA;BTNG;PNGM;PIMO;PIAG" > dataworks_stations.csv % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 65180 0 65180 0 0 1432 0 --:--:-- 0:00:45 --:--:-- 7129
# 導入國內站點,導入成功樣例,可能耗時會有點久,須要等待 [root@localhost mirror]# ./mirrorStations.py "kunm;lhaz;guao;urum;chan;bjfs;bjnm;wuhn;jfng;shao;twtf;tcms" localhost root root Dataworks_GSAC_database Connected to the database Dataworks_GSAC_database with account root, on localhost First, get site and equipment metadata at those sites from the UNAVCO GSAC. The GSAC API Linux command is /usr/bin/curl -L "http://www.unavco.org/data/web-services/gsacws/gsacapi/site/search?output=sitefull.csv&site.interval=interval.normal&site.code=kunm;lhaz;guao;urum;chan;bjfs;bjnm;wuhn;jfng;shao;twtf;tcms" > dataworks_stations.csv % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 9806 0 9806 0 0 55 0 --:--:-- 0:02:56 --:--:-- 897 Metadata for station ID BJFS named Beijing Fangshan / China Metadata for station ID BJNM named BJNM antenna at NIM Metadata for station ID CHAN named CHANGCHUN Metadata for station ID GUAO named GUAO Metadata for station ID JFNG named JIUFENG Metadata for station ID KUNM named Kunming Metadata for station ID LHAZ named Lhasa / Tibet / China Metadata for station ID SHAO named Shanghai Obs/Sheshan Station Metadata for station ID TCMS named Hsinchu Metadata for station ID TWTF named Taoyuan Metadata for station ID URUM named Urumqi - People Republic of China Metadata for station ID WUHN named WUHAN
添加定時任務: 使用crontab -e
命令添加定時任務,注意服務器重啓之後會失效,須要從新添加。
# 天天晚上 0 點 1 分,開始更新站點數據,而且不保存日誌 1 0 * * * /home/gsac/apps/Dataworks-SW/mirror/mirrorStations.py "kunm;lhaz;guao;urum;chan;bjfs;bjnm;wuhn;jfng;shao;twtf;tcms" localhost root root Dataworks_GSAC_database > /dev/null 2>&1
命令解析: ./mirrorData.py dbhost dbacct dbpw dbname 4daysback today "kunm;lhaz"
腳本 數據庫ip 帳號 密碼 數據庫名 4天前 今天 站點代號
# 導入國內數據文件信息到數據庫 ./mirrorData.py 127.0.0.1 root root Dataworks_GSAC_database 4daysback today "kunm;lhaz;guao;urum;chan;bjfs;bjnm;wuhn;jfng;shao;twtf;tcms" # 導入國外數據文件信息 ./mirrorData.py 127.0.0.1 root root Dataworks_GSAC_database 4daysback today "DAEJ;SUWN;YUNS;YSSK;STK2;MIZU;KSMV;TSK2;MTKA;KGNI;USUD;SMST;AIRA;GMSD;CCJ2;MAGO;YAKT;TIXI;NRIL;IRKJ;NBADG;NOVM;NVSK;ARTU;ULAB;KRTV;PODG;SUMK;POL2;CHUM;BIKO;KAZA;TALA;TASH;KIT3;MANM;SYBC;CHLM;DNC4;LMJG;DNSG;JMSM;MSKT;BMCL;DNGD;CUSV;BAKO;BNOA;BTNG;PNGM;PIMO;PIAG"
爲數據導入添加定時任務:
[root@localhost apps]# crontab -l # mei tian 24 dian kaishi genxin zhandian 1 0 * * * /home/gsac/apps/Dataworks-SW/mirror/mirrorStations.py "kunm;lhaz;guao;urum;chan;bjfs;bjnm;wuhn;jfng;shao;twtf;tcms" localhost root root Dataworks_GSAC_database > /dev/null 2>&1 # mei tian 3 dian kaishi dao ru shuju wenjian 1 3 * * * /home/gsac/apps/Dataworks-SW/mirror/mirrorData.py 127.0.0.1 root root Dataworks_GSAC_database 4daysback today "kunm;lhaz;guao;urum;chan;bjfs;bjnm;wuhn;jfng;shao;twtf;tcms" > /dev/null 2>&1
至此:站點導入和數據信息的導入完成,之後會天天定時執行導入工做。
在 linux
系統下,會使用 wget
命令來充 GNSS
獲取數據,如: wget -c -x -P ~/data/file ftp://data-out.unavco.org/pub/rinex/obs/2000/061/ugeo0610.00d.Z
經過前面的操做後數據都寫入進數據庫了,以下所示,replace.this.org
讓咱們替換成本身本地的ftp ip
mysql> select datafile_id ,url_path from datafile; +-------------+----------------------------------------------------------+ | datafile_id | url_path | +-------------+----------------------------------------------------------+ | 2049 | ftp://replace.this.org/rinex/nav/2016/249/bjnm2490.16g.Z | | 2050 | ftp://replace.this.org/rinex/nav/2016/249/bjnm2490.16n.Z | | 2051 | ftp://replace.this.org/rinex/obs/2016/249/bjnm2490.16d.Z | | 2052 | ftp://replace.this.org/rinex/nav/2016/249/chan2490.16n.Z | | 2053 | ftp://replace.this.org/rinex/obs/2016/249/chan2490.16d.Z | +-------------+----------------------------------------------------------+ 29 rows in set (0.00 sec)
以 shell 腳原本完成上述操做
#!/bin/bash ############################## # @file DataDownload.sh # @brief 用來下載 GNSS 數據的 shell 腳本 # @author liukai # @version 1.0 # @date 2016-09-08 ############################## ip="168.168.169.199" # 主機ip地址 dbUser="root" # 數據庫用戶名 dbPasswd="root" # 數據庫密碼 dbName="Dataworks_GSAC_database" # 數據庫實例名 dbUrlPath="replace.this.org" # datafile 表中 url_path 字段中要替換的鏈接名 replaceUrlPath="data-out.unavco.org/pub" # 被替換的鏈接名 ftpUrlPath="${ip}/${replaceUrlPath}" # 最後替換的本地ftp路徑 # 獲取要下載數據的 ftp path getUrlPaths() { # 查詢數據庫,將 url_path 放到變量 paths 中 command="select url_path from datafile;" declare paths=`/opt/lampp/bin/mysql -u${dbUser} -p${dbPasswd} -D${dbName} -e "${command}" --skip-column-name` # 循環數組,拼接成 GNSS 的 ftp 下載地址,而後使用 wget 命令將文件下載到ftp目錄 for url in $paths do copyUrl="${url}" if [ ${url:6:16} = ${dbUrlPath} ] ; then `wget -c -x -P /opt/lampp/htdocs/ ${copyUrl/${dbUrlPath}/${replaceUrlPath}}` fi done # 最後,更新 url_path 爲本地 ftp 路徑 command2="update datafile set url_path=replace(url_path,\"${dbUrlPath}\",\"${ftpUrlPath}\");" `/opt/lampp/bin/mysql -u${dbUser} -p${dbPasswd} -D${dbName} -e "${command2}"` } echo "--------------------開始下載文件------------------------------" getUrlPaths echo "--------------------文件下載完成------------------------------"
# 這是國內站點的任務調度內容 # mei tian 24 dian kaishi genxin zhandian 1 22 * * * cd /home/gsac/apps/Dataworks-SW/mirror && ./mirrorStations.py "kunm;lhaz;guao;urum;chan;bjfs;bjnm;wuhn;jfng;shao;twtf;tcms" 127.0.0.1 root root Dataworks_GSAC_database >> /home/gsac/logs/station`date +\%Y\%m\%d`.log 2>&1 # mei tian 3 dian kaishi dao ru shuju wenjian 1 0 * * * cd /home/gsac/apps/Dataworks-SW/mirror && ./mirrorData.py 127.0.0.1 root root Dataworks_GSAC_database 4daysback today "kunm;lhaz;guao;urum;chan;bjfs;bjnm;wuhn;jfng;shao;twtf;tcms" >> /home/gsac/logs/data`date +\%Y\%m\%d`.log 2>&1 1 3 * * * /home/gsac/apps/Dataworks-SW/mirror/DataDownload.sh >> /home/gsac/logs/download`date +\%Y\%m\%d`.log 2>&1 # 這個國外站點的調度任務 1 21 * * * cd /home/gsac/apps/Dataworks-SW/mirror && ./mirrorStations.py "DAEJ;SUWN;YUNS;YSSK;STK2;MIZU;KSMV;TSK2;MTKA;KGNI;USUD;SMST;AIRA;GMSD;CCJ2;MAGO;YAKT;TIXI;NRIL;IRKJ;NBADG;NOVM;NVSK;ARTU;ULAB;KRTV;PODG;SUMK;POL2;CHUM;BIKO;KAZA;TALA;TASH;KIT3;MANM;SYBC;CHLM;DNC4;LMJG;DNSG;JMSM;MSKT;BMCL;DNGD;CUSV;BAKO;BNOA;BTNG;PNGM;PIMO;PIAG" 127.0.0.1 root root Dataworks_GSAC_database >> /home/gsac/logs/station`date +\%Y\%m\%d`.log 2>&1 1 1 * * * cd /home/gsac/apps/Dataworks-SW/mirror && ./mirrorData.py 127.0.0.1 root root Dataworks_GSAC_database 4daysback today "DAEJ;SUWN;YUNS;YSSK;STK2;MIZU;KSMV;TSK2;MTKA;KGNI;USUD;SMST;AIRA;GMSD;CCJ2;MAGO;YAKT;TIXI;NRIL;IRKJ;NBADG;NOVM;NVSK;ARTU;ULAB;KRTV;PODG;SUMK;POL2;CHUM;BIKO;KAZA;TALA;TASH;KIT3;MANM;SYBC;CHLM;DNC4;LMJG;DNSG;JMSM;MSKT;BMCL;DNGD;CUSV;BAKO;BNOA;BTNG;PNGM;PIMO;PIAG" >> /home/gsac/logs/data`date +\%Y\%m\%d`.log 2>&1 1 4 * * * /home/gsac/apps/Dataworks-SW/mirror/DataDownload.sh >> /home/gsac/logs/download`date +\%Y\%m\%d`.log 2>&1
GSAC
鏡像映射方案採用 python
腳原本完成鏡像的映射。 ./mirrorStations_199.py "bjnm;wuhn" 127.0.0.1 root root Dataworks_GSAC_database
命令將從199
機器獲取站點編號爲kunm;lhaz
的兩個站點的station
信息。 ./mirrorData_199.py 127.0.0.1 root root Dataworks_GSAC_database 4daysback today "bjnm;wuhn"
命令會將上面更新站點信息的兩個站點數據下載到本地。
./mirrorStations_200.py "DAEJ,NOVM" 127.0.0.1 root root Dataworks_GSAC_database
./mirrorData_200.py 127.0.0.1 root root Dataworks_GSAC_database 4daysback today "DAEJ,NOVM"