1,需求:已知空氣監測數據在hbase中存儲,要求按照時間,查詢citycode爲110000(北京)一個月的數據,數據爲每日的監測數據java
ID ,CITYCODE,SO2 ,CO,NO2 ,O3, PM10,PM2_5,AQI,MEASURE, TIMEPOINT工具
13110000020141120, 110000,31,3.939,141,8,368,301,351,6,2014-11-20this
5110000020150108, 110000,53,3.305,101,12,176,143,190,4,2015-01-08編碼
key值設計規則: String yearMoth= "201411"; //年月 String time="20141120"; //年月日 String citycode="1100000"; //城市編碼 //hbase 爲20個分區,數據進入hbase的分區規則以下 int region=Math.abs((yearMoth+citycode).hashCode())%20; //結果:8 //hbase的key值爲 String key =region+citycode+time;
這樣設計key的好處是:同一個城市,每一個月的數據的分區相同,即region相同,只須要設置startRowkey與endRowKey便可設計
例如 查詢北京 2014 年 11 月的數據code
startRowkey:13110000020141100orm
endRowkey: 13110000020141200 便可對象
但:若是查詢 2014 年11 月20 號 --- 2015年01月08號的數據 則須要把每一個月的數據查詢出來,合併到一塊兒ci
如下 爲工具類的的方法:字符串
建立 時間範圍的javaBean對象
public class TimeRange { private String startPoint ; private String endPoint ; public String getStartPoint() { return startPoint; } public void setStartPoint(String startPoint) { this.startPoint = startPoint; } public String getEndPoint() { return endPoint; } public void setEndPoint(String endPoint) { this.endPoint = endPoint; } public String toString() { return startPoint + " - " + endPoint ; } }
計算時間範圍的工具類:
startStr =20141120
endStr=20150108
獲得的結果 List<TimeRange> 爲 20141120-20141201
20141201-20150101
20150101-20150108
/** * 計算查詢時間範圍 */ public static List<TimeRange > getCallLogRanges(String startStr , String endStr){ try{ SimpleDateFormat sdfYMD = new SimpleDateFormat("yyyyMMdd"); SimpleDateFormat sdfYM = new SimpleDateFormat("yyyyMM"); DecimalFormat df00 = new DecimalFormat("00"); // List<TimeRange > list = new ArrayList<>(); //字符串時間 String startPrefix = startStr.substring(0, 6); String endPrefix = endStr.substring(0, 6); int endDay = Integer.parseInt(endStr.substring(6, 8)); //結束點 String endPoint = endPrefix + df00.format(endDay + 1); //日曆對象 Calendar c = Calendar.getInstance(); //同年月 if (startPrefix.equals(endPrefix)) { TimeRange range = new TimeRange (); range.setStartPoint(startStr); //設置起始點 range.setEndPoint(endPoint); //設置結束點 list.add(range); } else { //1.起始月 TimeRange range = new TimeRange (); range.setStartPoint(startStr); //設置日曆的時間對象 c.setTime(sdfYMD.parse(startStr)); c.add(Calendar.MONTH, 1); range.setEndPoint(sdfYM.format(c.getTime())); list.add(range); //是不是最後一月 while (true) { //到告終束月份 if (endStr.startsWith(sdfYM.format(c.getTime()))) { range = new TimeRange (); range.setStartPoint(sdfYM.format(c.getTime())); range.setEndPoint(endPoint); list.add(range); break; } else { range = new TimeRange (); //起始時間 range.setStartPoint(sdfYM.format(c.getTime())); //增長月份 c.add(Calendar.MONTH, 1); range.setEndPoint(sdfYM.format(c.getTime())); list.add(range); } } } return list ; } catch(Exception e){ e.printStackTrace(); } return null ; }
Hbase 查詢的時 遍歷List<TimeRange>
public List<Map> getLogByContion(String citycode, List<TimeRange> ranges) { Configuration conf = HBaseConfiguration.create(); Connection conn = ConnectionFactory.createConnection(conf); TableName tableName = TableName.valueOf("hbase:airDay"); table=conn.getTable(tableName); List<Map> list = new ArrayList<>(); try { for (TimeRange range: ranges) { Scan scan = new Scan(); //設置掃描起始行 結束行 scan.setStartRow(Bytes.toBytes(HbaseUtils.getStartRowkey(citycode,range.getStartPoint()))); scan.setStopRow(Bytes.toBytes(HbaseUtils.getStopRowkey(citycode,range.getStartPoint(),range.getEndPoint()))); ResultScanner rs = table.getScanner(scan); Iterator<Result> it = rs.iterator(); byte[] f1 = Bytes.toBytes("f1"); byte[] caller = Bytes.toBytes("citycode"); byte[] callee = Bytes.toBytes("so2"); byte[] callTime = Bytes.toBytes("co"); byte[] callDuration = Bytes.toBytes("no2"); while (it.hasNext()) { Result r = it.next(); Map map = new HashMap(); map.put("rowkey",Bytes.toString(r.getRow())); map.put("caller",Bytes.toString(r.getValue(f1,citycode))); map.put("callee",Bytes.toString(r.getValue(f1,so2))); map.put("callTime",Bytes.toString(r.getValue(f1,co))); map.put("callDuration",Bytes.toString(r.getValue(f1,no2))); list.add(map); } } }catch (Exception e){ } return list; } public static String getStartRowkey(String citycode, String time) { int region=Math.abs((yearMoth+citycode).hashCode())%20; return region+citycode+time; } public static String getStopRowkey(String citycode,String starttime, String endtime) { int region=Math.abs((yearMoth+citycode).hashCode())%20; return region+citycode+endtime; }