Linux下解析和格式化輸出JSON的命令行工具 - jq

 

JSON 是一種輕量級且與語言無關的數據存儲格式，易於與大多數編程語言集成，也易於理解 。雖然它以 JavaScript 開頭，並且主要用於在服務器和瀏覽器之間交換數據，但如今正在用於許多領域，包括嵌入式系統。JSON是前端編程常常用到的格式，對於PHP或者Python，解析JSON都不是什麼大事，尤爲是PHP的json_encode和json_decode,乾的至關的漂亮。Linux下也有處理JSON的神器：jq。對於JSON格式而言，jq就像sed/awk/grep這些神器同樣的方便，jq沒有亂七八糟的依賴，只須要一個binary文件jq就能夠了。Linux 上使用命令行工具jq來解析並格式化打印 JSON，它對於在 shell 腳本中處理大型 JSON 數據或在 shell 腳本中處理 JSON 數據很是有用。

1）安裝jq工具

centos7能夠直接yum安裝jq
[root@k8s-master01 ~]# yum install -y jq
[root@k8s-master01 ~]# jq --version
jq-1.5

2) 使用jq
簡單的來講，jq是linux下一個解析json格式數據的實用工具，能夠將在linux下一團糟的json數據按格式輸出，還能夠直接選中關鍵字打印出它的值。

JSON 數據可能放在一行上令人難以解讀，所以爲了使其具備必定的可讀性，JSON 格式化輸出就可用於此目的的。以下示例：

[root@k8s-master01 ~]# wget -cq http://jsonip.com/ -O -
{"ip":"114.242.193.201","about":"/about","Pro!":"http://getjsonip.com","reject-fascism":"Pro abortion is pro life and pro rights"}

如今使用jq格式化輸出它們：
[root@k8s-master01 ~]# wget -cq http://jsonip.com/ -O - | jq '.'
{
  "ip": "114.242.193.201",
  "about": "/about",
  "Pro!": "http://getjsonip.com",
  "reject-fascism": "Women make their own choices. Support abortion rights."
}

jq 的基本過濾和標識符功能
jq 能夠從 STDIN 或文件中讀取 JSON 數據。你能夠根據狀況使用。
單個符號 . 是最基本的過濾器。這些過濾器也稱爲對象標識符-索引。jq 使用單個 . 過濾器基本上至關將輸入的 JSON 文件格式化輸出。
◈ 單引號：沒必要始終使用單引號。可是若是你在一行中組合幾個過濾器，那麼你必須使用它們。
◈ 雙引號：你必須用兩個雙引號括起任何特殊字符，如 @、＃、$，例如 jq .foo.」@bar」。
◈ 原始數據打印：無論出於任何緣由，若是你只須要最終解析的數據（不包含在雙引號內），請使用帶有 -r 標誌的 jq 命令，以下所示：jq -r .foo.bar。

jq能夠解析特定數據
要過濾出 JSON 的特定部分，你須要瞭解格式化輸出的 JSON 文件的數據層次結構。以下示例

[root@k8s-master01 ~]# cat test.json
[{"hostCompany":"Beijing Autelan Technology","hostModel":"CS-VIC-2000-C","hostsn":"01010730b12014A00477","mac":"00:1F:64:CE:F3:8E","cpuModel":"MIPS 74Kc V4.12","cpuSN":"000000","memoryModel":"abcdefg","memorySN":"000000","boardSN":"01010730b12014A00477","networkCardMac":"00:1F:64:CE:F3:8F","lowFreModel":"AR9344","lowFreSN":"000000","hignFreModel":"AR9582","hignFreSN":"000000","gpsModel":"abcdefg","gpsSN":"000000","MEID_3g":"A000004E123ABD2","Company_3g":"ZTEMT INCORPORATED","modelOf3g":"MC271X","snOf3g":"A000004E123ABD2","iccid":"89860314400200885980","Operators":"CTCC","hardVersion":"1.20","firmwareVersion":"1.0.6.29"}]


[root@k8s-master01 ~]# jq . test.json
[
  {
    "hostCompany": "Beijing Autelan Technology",
    "hostModel": "CS-VIC-2000-C",
    "hostsn": "01010730b12014A00477",
    "mac": "00:1F:64:CE:F3:8E",
    "cpuModel": "MIPS 74Kc V4.12",
    "cpuSN": "000000",
    "memoryModel": "abcdefg",
    "memorySN": "000000",
    "boardSN": "01010730b12014A00477",
    "networkCardMac": "00:1F:64:CE:F3:8F",
    "lowFreModel": "AR9344",
    "lowFreSN": "000000",
    "hignFreModel": "AR9582",
    "hignFreSN": "000000",
    "gpsModel": "abcdefg",
    "gpsSN": "000000",
    "MEID_3g": "A000004E123ABD2",
    "Company_3g": "ZTEMT INCORPORATED",
    "modelOf3g": "MC271X",
    "snOf3g": "A000004E123ABD2",
    "iccid": "89860314400200885980",
    "Operators": "CTCC",
    "hardVersion": "1.20",
    "firmwareVersion": "1.0.6.29"
  }
]


[root@k8s-master01 ~]# jq .[] test.json
{
  "hostCompany": "Beijing Autelan Technology",
  "hostModel": "CS-VIC-2000-C",
  "hostsn": "01010730b12014A00477",
  "mac": "00:1F:64:CE:F3:8E",
  "cpuModel": "MIPS 74Kc V4.12",
  "cpuSN": "000000",
  "memoryModel": "abcdefg",
  "memorySN": "000000",
  "boardSN": "01010730b12014A00477",
  "networkCardMac": "00:1F:64:CE:F3:8F",
  "lowFreModel": "AR9344",
  "lowFreSN": "000000",
  "hignFreModel": "AR9582",
  "hignFreSN": "000000",
  "gpsModel": "abcdefg",
  "gpsSN": "000000",
  "MEID_3g": "A000004E123ABD2",
  "Company_3g": "ZTEMT INCORPORATED",
  "modelOf3g": "MC271X",
  "snOf3g": "A000004E123ABD2",
  "iccid": "89860314400200885980",
  "Operators": "CTCC",
  "hardVersion": "1.20",
  "firmwareVersion": "1.0.6.29"
}


[root@k8s-master01 ~]# jq -r .[].mac test.json
00:1F:64:CE:F3:8E

[root@k8s-master01 ~]# jq -r '.[] | .mac' test.json
00:1F:64:CE:F3:8E

[root@k8s-master01 ~]# jq -r '.[] |.mac, .gpsSN' test.json
00:1F:64:CE:F3:8E
000000

[root@k8s-master01 ~]# jq -r '.[].mac, .[].gpsSN' test.json
00:1F:64:CE:F3:8E
000000