Scala Http 請求調用 － scalaj-http

Simplified Http

This is a fully featured http client for Scala which wraps java.net.HttpURLConnection

Features:

• Zero dependencies
• Cross compiled for Scala 2.10, 2.11, 2.12, and 2.13-M3
• OAuth v1 request signing
• Automatic support of gzip and deflate encodings from server
• Easy to add querystring or form params. URL encoding is handled for you.
• Multipart file uploads

Non-Features:

• Async execution
  • The library is thread safe. HttpRequest and HttpResponse are immutable. So it should be easy to wrap in an execution framework of your choice.

Works in Google AppEngine and Android environments.

Note: 2.x.x is a new major version which is both syntactically and behaviorally different than the 0.x.x version.

Previous version is branched here: https://github.com/scalaj/scalaj-http/tree/0.3.x

Big differences:

• Executing the request always returns a HttpResponse[T] instance that contains the response-code, headers, and body
• Exceptions are no longer thrown for 4xx and 5xx response codes. Yay!
• Http(url) is the starting point for every type of request (post, get, multi, etc)
• You can easily create your own singleton instance to set your own defaults (timeouts, proxies, etc)
• Sends "Accept-Encoding: gzip,deflate" request header and decompresses based on Content-Encoding (configurable)
• Redirects are no longer followed by default. Use .option(HttpOptions.followRedirects(true)) to change.

Installation

in your build.sbt

libraryDependencies +=  "org.scalaj" %% "scalaj-http" % "2.4.1"

　　

maven

<dependency>
  <groupId>org.scalaj</groupId>
  <artifactId>scalaj-http_${scala.version}</artifactId>
  <version>2.4.1</version>
</dependency>  

　　

If you're including this in some other public library. Do your users a favor and change the fully qualified name so they don't have version conflicts if they're using a different version of this library. The easiest way to do that is just to copy the source into your project :)

Usage

Simple Get

import scalaj.http._
  
val response: HttpResponse[String] = Http("http://foo.com/search").param("q","monkeys").asString
response.body
response.code
response.headers
response.cookies

 

Immutable Request

Http(url) is just shorthand for a Http.apply which returns an immutable instance of HttpRequest.
You can create a HttpRequest and reuse it:

val request: HttpRequest = Http("http://date.jsontest.com/")

val responseOne = request.asString
val responseTwo = request.asString

 

Additive Request

All the "modification" methods of a HttpRequest are actually returning a new instance. The param(s), option(s), header(s) methods always add to their respective sets. So calling .headers(newHeaders) will return a HttpRequest instance that has newHeaders appended to the previous req.headers

Simple form encoded POST

Http("http://foo.com/add").postForm(Seq("name" -> "jon", "age" -> "29")).asString

 

OAuth v1 Dance and Request

Note: the .oauth(...) call must be the last method called in the request construction

import scalaj.http.{Http, Token}

val consumer = Token("key", "secret")
val response = Http("https://api.twitter.com/oauth/request_token").postForm(Seq("oauth_callback" -> "oob"))
  .oauth(consumer).asToken

println("Go to https://api.twitter.com/oauth/authorize?oauth_token=" + response.body.key)

val verifier = Console.readLine("Enter verifier: ").trim

val accessToken = Http("https://api.twitter.com/oauth/access_token").postForm.
  .oauth(consumer, response.body, verifier).asToken

println(Http("https://api.twitter.com/1.1/account/settings.json").oauth(consumer, accessToken.body).asString)

 

Parsing the response

Http("http://foo.com").{asString, asBytes, asParams}

 

Those methods will return an HttpResponse[String | Array[Byte] | Seq[(String, String)]] respectively

Advanced Usage Examples

Parse the response InputStream directly

val response: HttpResponse[Map[String,String]] = Http("http://foo.com").execute(parser = {inputStream =>
  Json.parse[Map[String,String]](inputStream)
})

 

Post raw Array[Byte] or String data and get response code

Http(url).postData(data).header("content-type", "application/json").asString.code

 

Post multipart/form-data

Http(url).postMulti(MultiPart("photo", "headshot.png", "image/png", fileBytes)).asString

 

You can also stream uploads and get a callback on progress:

Http(url).postMulti(MultiPart("photo", "headshot.png", "image/png", inputStream, bytesInStream, 
  lenWritten => {
    println(s"Wrote $lenWritten bytes out of $bytesInStream total for headshot.png")
  })).asString

 

Stream a chunked transfer response (like an event stream)

Http("http://httpbin.org/stream/20").execute(is => {
  scala.io.Source.fromInputStream(is).getLines().foreach(println)
})

 

note that you may have to wrap in a while loop and set a long readTimeout to stay connected

Send https request to site with self-signed or otherwise shady certificate

Http("https://localhost/").option(HttpOptions.allowUnsafeSSL).asString

 

Do a HEAD request

Http(url).method("HEAD").asString

 

Custom connect and read timeouts

These are set to 1000 and 5000 milliseconds respectively by default

Http(url).timeout(connTimeoutMs = 1000, readTimeoutMs = 5000).asString

 

Get request via a proxy

val response = Http(url).proxy(proxyHost, proxyPort).asString

 

Other custom options

The .option() method takes a function of type HttpURLConnection => Unit so you can manipulate the connection in whatever way you want before the request executes.

Change the Charset

By default, the charset for all param encoding and string response parsing is UTF-8. You can override with charset of your choice:

Http(url).charset("ISO-8859-1").asString

 

Create your own HttpRequest builder

You don't have to use the default Http singleton. Create your own:

object MyHttp extends BaseHttp (
  proxyConfig = None,
  options = HttpConstants.defaultOptions,
  charset = HttpConstants.utf8,
  sendBufferSize = 4096,
  userAgent = "scalaj-http/1.0",
  compress = true
)

 

Full API documentation

scaladocs here

Dealing with annoying java library issues

Overriding the Access-Control, Content-Length, Content-Transfer-Encoding, Host, Keep-Alive, Origin, Trailer, Transfer-Encoding, Upgrade, Via headers

Some of the headers are locked by the java library for "security" reasons and the behavior is that the library will just silently fail to set them. You can workaround by doing one of the following:

• Start your JVM with this command line parameter: -Dsun.net.http.allowRestrictedHeaders=true
• or, do this first thing at runtime: System.setProperty("sun.net.http.allowRestrictedHeaders", "true")

 

以上官方說明，下面具體怎麼使用：

首先Http請求，返回json類型數據，就要有json解析及轉化工具。

Scala　sbt引入scalaj-http及spray-json.

"org.scalaj" % "scalaj-http_2.12" % "2.3.0",
"com.github.wangzaixiang" %% "spray-json" % "1.3.4",
"com.google.code.gson" % "gson" % "2.7"

　　 

工具類以下：

package test.scalajhttp

import com.google.gson.Gson
import scalaj.http.{Http, HttpOptions, HttpResponse}
import spray.json.DefaultJsonProtocol._
import spray.json.{JsValue, _}

/**
  * 類功能描述：Scalaj http 工具類
  *
  * @author WangXueXing create at 19-8-15 下午5:56
  * @version 1.0.0
  */
object ScalajHttpUtil {
  /**
    * 成功
    */
  val STATUS_SUCCESS: String = "success"
  /**
    * 失敗
    */
  val STATUS_FAIL: String = "fail"

  /**
    * http公共請求類
    *
    * @param url  Http路徑如：https://www.baidu.com
    * @param params 參數
    * @return
    */
  def requestUrl(url: String, params: Map[String, String]) = {
    val httpRequest = Http(url)
                      .header("Content-Type", "application/xml")//"application/json"
                      .header("Charset", "UTF-8")
                      .option(HttpOptions.readTimeout(10000))
    val response: HttpResponse[String] = (params != null && !params.isEmpty) match {
        case true =>  httpRequest.params(params).asString
        case false => httpRequest.asString
    }

    response.body
  }

  /**
    * http公共請求類
    *
    * @param url  Http路徑如：https://www.baidu.com
    * @param requestJson Json參數
    * @return
    */
  def postRequestUrl(url: String, requestJson: String) = {
    val httpRequest = Http(url)
      .header("Content-Type", "application/xml")//"application/json"
      .header("Charset", "UTF-8")
      .option(HttpOptions.readTimeout(10000))
    val response: HttpResponse[String] = (requestJson != null && !requestJson.isEmpty) match {
      case true =>  httpRequest.postData(requestJson).asString
      case false => httpRequest.asString
    }

    response.body
  }

  /**
    * 將object映射json string
    *
    * @param obj
    * @return
    */
  def objToJson(obj: NcRespose): String = {
    new Gson().toJson(obj)
  }

  /**
    * 將map數據映射到Object
    *
    * @param jsonStr
    * @return
    */
  def jsonToObject(jsonStr: String): NcRespose = {
    val jsonObj = jsonStr.parseJson.convertTo[Map[String, JsValue]]
    mapToObject(jsonObj)
  }

  /**
    * 將map數據映射到PRackType類上
    *
    * @param jsonMap
    * @return
    */
  def mapToJson(jsonMap: Map[String, JsValue]): String = {
    val obj = mapToObject(jsonMap)
    new Gson().toJson(obj)
  }

  /**
    * 將map數據映射到Object
    *
    * @param jsonObj
    * @return
    */
  def mapToObject(jsonObj: Map[String, JsValue]): NcRespose = {
    NcRespose(
      status = toStringValue(jsonObj.get("status").get),
      message = toStringValue(jsonObj.get("message").get)
    )
  }

  def toStringValue(jsValue: JsValue): String = {
    s"$jsValue".replace("\"", "")
  }

  def toIntValue(jsValue: JsValue): Int = {
    s"$jsValue".replace("\"", "").toInt
  }

  def toLongValue(jsValue: JsValue): Long = {
    s"$jsValue".replace("\"", "").toLong
  }

  def main(args: Array[String]): Unit = {
    val jsonStr1 = "{\"billhead\":{\"bill_date\":\"2019-08-13\",\"creationtime\":\"2019-08-13 17:55:00\",\"creator\":\"1000\",\"local_money\":100,\"local_rate\":1,\"objecttype\":0,\"payway\":0,\"pk_currtype\":\"CNY\",\"pk_org\":\"101\",\"primal_money\":100},\"items\":[{\"billdetail_no\":0,\"pay_local\":100,\"pay_primal\":100,\"pk_oppaccount\":\"421860158018800081218\"}]}"
    val jsonStr = postRequestUrl("http://10.100.99.98:8000/uapws/rest/SyncCMPPayBillRestful/add", jsonStr1)

    val gson = new Gson()
    val respose = gson.fromJson(jsonStr, classOf[NcRespose])

//    val respose = jsonToObject(jsonStr)
//    println(jsonStr)
    if(respose.status == STATUS_SUCCESS){
      println(s"success: ${respose.message}")
    } else {
      println(s"fail: ${respose.message}")
    }
  }
}

 

case class NcRespose(status: String, message: String)

 

scala中使用json，比java中使用json複雜一些。嘗試了json-smart；fast json； gson；json4s以及scala自帶的json。

  總結以下：

 1. 普通json string 轉化 object, 使用JSON.pareseFull 。         

JSON.parseFull(str).get.asInstanceOf[Map[String, String]]("key") 

 

 2. json 與 map 互相轉化       

val colors:Map[String,Object] = Map("red" -> "123456", "azure" -> "789789")
val json = JSONObject(colors)
println(json)
val jsonMap = JSON.parseFull(json.toString).get.asInstanceOf[Map[String,Object]]
println(jsonMap) 

3. json 與 class 互相轉化

case class Student( name:String , no: String )
 
    val gson = new Gson
    val student = Student("張三", "100")
    val str = gson.toJson(student, classOf[Student])
    println(str)
    val student2 = gson.fromJson(str, classOf[Student])
    println(student2)

尤爲注意：使用 classOf ，不能使用 getclass

4. json與Map互相轉化，需使用java的map 

val map = new util.HashMap[String, Object]()
map.put("abc", List(s1,s2).toArray)
val gson = new Gson()
println( gson.toJson(map) )
另附，json判斷代碼

def isGoodJson(json: String):Boolean = {

if(null == json) {
return false
}
val result = JSON.parseFull(json) match {
case Some(_: Map[String, Any]) => true
case None => false
case _ => false
}
result
}

 

請參考：

scalaj-http