Scala Macros － 元編程 Metaprogramming with Def Macros

    Scala Macros對scala函數庫編程人員來講是一項不可或缺的編程工具，能夠經過它來解決一些用普通編程或者類層次編程（type level programming）都沒法解決的問題，這是由於Scala Macros能夠直接對程序進行修改。Scala Macros的工做原理是在程序編譯時按照編程人員的意旨對一段程序進行修改產生出一段新的程序。具體過程是：當編譯器在對程序進行類型驗證（typecheck）時若是發現Macro標記就會將這個Macro的功能實現程序（implementation）：一個語法樹（AST, Abstract Syntax Tree）結構拉過來在Macro的位置進行替代，而後從這個AST開始繼續進行類型驗證過程。

下面咱們先用個簡單的例子來示範分析一下Def Macros的基本原理和使用方法：

1 object modules { 2    greeting("john") 3  } 4  
5  object mmacros { 6    def greeting(person: String): Unit = macro greetingMacro 7    def greetingMacro(c: Context)(person: c.Expr[String]): c.Expr[Unit] = ... 8  }

以上是Def Macros的標準實現模式。基本原理是這樣的：當編譯器在編譯modules遇到方法調用greeting("john")時會進行函數符號解析、在mmacros裏發現greeting是個macro，它的具體實如今greetingMacro函數裏，此時編譯器會運行greetingMacro函數並將運算結果-一個AST調用替表明達式greeting("john")。注意編譯器在運算greetingMacro時會以AST方式將參數person傳入。因爲在編譯modules對象時須要運算greetingMacro函數，因此greetingMacro函數乃至整個mmacros對象必須是已編譯狀態，這就意味着modules和mmacros必須分別在不一樣的源代碼文件裏，並且還要確保在編譯modules前先完成對mmacros的編譯，咱們能夠從sbt設置文件build.sbt看到它們的關係：

 1 name := "learn-macro"
 2 
 3 version := "1.0.1"
 4 
 5 val commonSettings = Seq(  6   scalaVersion := "2.11.8",  7   scalacOptions ++= Seq("-deprecation", "-feature"),  8   libraryDependencies ++= Seq(  9     "org.scala-lang" % "scala-reflect" % scalaVersion.value, 10     "org.scala-lang.modules" %% "scala-parser-combinators" % "1.0.1", 11     "org.specs2" %% "specs2" % "2.3.12" % "test", 12     "org.scalatest" % "scalatest_2.11" % "2.2.1" % "test"
13  ), 14   addCompilerPlugin("org.scalamacros" % "paradise" % "2.1.0" cross CrossVersion.full) 15 ) 16 
17 lazy val root = (project in file(".")).aggregate(macros, demos) 18 
19 lazy val macros = project.in(file("macros")).settings(commonSettings : _*) 20 
21 lazy val demos  = project.in(file("demos")).settings(commonSettings : _*).dependsOn(macros)

注意最後一行：demos dependsOn(macros)，由於咱們會把全部macros定義文件放在macros目錄下。

下面咱們來看看macro的具體實現方法：

 1 import scala.language.experimental.macros  2 import scala.reflect.macros.blackbox.Context  3 import java.util.Date  4 object LibraryMacros {  5   def greeting(person: String): Unit = macro greetingMacro  6 
 7   def greetingMacro(c: Context)(person: c.Expr[String]): c.Expr[Unit] = {  8  import c.universe._  9     println("compiling greeting ...") 10     val now = reify {new Date().toString} 11  reify { 12       println("Hello " + person.splice + ", the time is: " + new Date().toString) 13  } 14  } 15 }

以上是macro greeting的具體聲明和實現。代碼放在macros目錄下的MacrosLibrary.scala裏。首先必須import macros和Context。

macro調用在demo目錄下的HelloMacro.scala裏：

1 object HelloMacro extends App { 2  import LibraryMacros._ 3   greeting("john") 4 }

注意在編譯HelloMacro.scala時產生的輸出：

Mac-Pro:learn-macro tiger-macpro$ sbt [info] Loading global plugins from /Users/tiger-macpro/.sbt/0.13/plugins [info] Loading project definition from /Users/tiger-macpro/Scala/IntelliJ/learn-macro/project [info] Set current project to learn-macro (in build file:/Users/tiger-macpro/Scala/IntelliJ/learn-macro/) > project demos [info] Set current project to demos (in build file:/Users/tiger-macpro/Scala/IntelliJ/learn-macro/) > compile [info] Compiling 1 Scala source to /Users/tiger-macpro/Scala/IntelliJ/learn-macro/macros/target/scala-2.11/classes... [info] 'compiler-interface' not yet compiled for Scala 2.11.8. Compiling... [info] Compilation completed in 7.876 s [info] Compiling 1 Scala source to /Users/tiger-macpro/Scala/IntelliJ/learn-macro/demos/target/scala-2.11/classes... compiling greeting ... [success] Total time: 10 s, completed 2016-11-9 9:28:24
> 

從compiling greeting ...這條提示咱們能夠得出在編譯demo目錄下源代碼文件的過程當中應該運算了greetingMacro函數。測試運行後產生結果：

Hello john, the time is: Wed Nov 09 09:32:04 HKT 2016 Process finished with exit code 0

運算greeting其實是調用了greetingMacro中的macro實現代碼。上面這個例子使用了最基礎的Scala Macro編程模式。注意這個例子裏函數greetingMacro的參數c: Context和在函數內部代碼中reify，splice的調用：因爲Context是個動態函數接口，每一個實例都有所不一樣。對於大型的macro實現函數，可能會調用到其它一樣會使用到Context的輔助函數（helper function），容易出現Context實例不匹配問題。另外reify和splice能夠說是最原始的AST操做函數。咱們在下面這個例子裏使用了最新的模式和方法：

 1   def tell(person: String): Unit = macro MacrosImpls.tellMacro  2   class MacrosImpls(val c: Context) {  3  import c.universe._  4       def tellMacro(person: c.Tree): c.Tree = {  5       println("compiling tell ...")  6       val now = new Date().toString  7       q"""  8           println("Hello "+$person+", it is: "+$now)  9         """ 10  } 11   }

在這個例子裏咱們把macro實現函數放入一個以Context爲參數的class。咱們能夠把全部使用Context的函數都擺在這個class裏面你們共用統一的Context實例。quasiquotes是最新的AST操做函數集，能夠更方便靈活地控制AST的產生、表達式還原等。這個tell macro的調用仍是同樣的：

1 object HelloMacro extends App { 2  import LibraryMacros._ 3   greeting("john") 4   tell("mary") 5 }

測試運算產生下面的結果：

Hello john, the time is: Wed Nov 09 11:42:21 HKT 2016 Hello mary, it is: Wed Nov 09 11:42:20 HKT 2016 Process finished with exit code 0

Def Macros的Macro實現函數能夠是泛型函數，支持類參數。在下面的例子咱們示範如何用Def Macros來實現通用的case class與Map類型的轉換。假設咱們有個轉換器CaseClassMapConverter[C]，那麼C類型能夠是任何case class，因此這個轉換器是泛型的，那麼macro實現函數也就必須是泛型的了。大致來講，咱們但願實現如下功能：把任何case class轉成Map：

 1  def ccToMap[C: CaseClassMapConverter](c: C): Map[String,Any] =
 2  implicitly[CaseClassMapConverter[C]].toMap(c)  3 
 4   case class Person(name: String, age: Int)  5   case class Car(make: String, year: Int, manu: String)  6 
 7   val civic = Car("Civic",2016,"Honda")  8   println(ccToMap[Person](Person("john",18)))  9  println(ccToMap[Car](civic)) 10 
11 ... 12 Map(name -> john, age -> 18) 13 Map(make -> Civic, year -> 2016, manu -> Honda)

反向把Map轉成case class：

 1   def mapTocc[C: CaseClassMapConverter](m: Map[String,Any]) =
 2  implicitly[CaseClassMapConverter[C]].fromMap(m)  3 
 4   val mapJohn = ccToMap[Person](Person("john",18))  5   val mapCivic = ccToMap[Car](civic)  6  println(mapTocc[Person](mapJohn))  7  println(mapTocc[Car](mapCivic))  8 
 9 ... 10 Person(john,18) 11 Car(Civic,2016,Honda)

咱們來看看這個Macro的實現函數：macros/CaseClassConverter.scala

 1 import scala.language.experimental.macros  2 import scala.reflect.macros.whitebox.Context  3 
 4 trait CaseClassMapConverter[C] {  5  def toMap(c: C): Map[String,Any]  6  def fromMap(m: Map[String,Any]): C  7 }  8 object CaseClassMapConverter {  9   implicit def Materializer[C]: CaseClassMapConverter[C] = macro converterMacro[C] 10   def converterMacro[C: c.WeakTypeTag](c: Context): c.Tree = { 11  import c.universe._ 12 
13     val tpe = weakTypeOf[C] 14     val fields = tpe.decls.collectFirst { 15       case m: MethodSymbol if m.isPrimaryConstructor => m 16     }.get.paramLists.head 17 
18     val companion = tpe.typeSymbol.companion 19     val (toParams,fromParams) = fields.map { field =>
20     val name = field.name.toTermName 21     val decoded = name.decodedName.toString 22     val rtype = tpe.decl(name).typeSignature 23 
24       (q"$decoded -> t.$name", q"map($decoded).asInstanceOf[$rtype]") 25 
26  }.unzip 27 
28     q""" 29        new CaseClassMapConverter[$tpe] { 30         def toMap(t: $tpe): Map[String,Any] = Map(..$toParams) 31         def fromMap(map: Map[String,Any]): $tpe = $companion(..$fromParams) 32  } 33       """ 34  } 35 }

首先，trait CaseClassMapConverter[C]是個typeclass，表明了C類型數據的行爲函數toMap和fromMap。咱們同時能夠看到Macro定義implicit def Materializer[C]是隱式的，並且是泛型的，運算結果類型是CaseClassMapConverter[C]。從這個能夠推斷出這個Macro定義經過Macro實現函數能夠產生CaseClassMapConverter[C]實例，C能夠是任何case class類型。在函數ccToMap和mapTocc函數須要的隱式參數CaseClassMapConverter[C]實例就是由這個Macro實現函數提供的。注意咱們只能用WeakTypeTag來獲取類型參數C的信息。在使用quasiquotes時咱們通常是在q括號中放入原始代碼。在q括號內調用AST變量用$前綴（稱爲unquote）。對類型tpe的操做能夠參考scala.reflect api。示範調用代碼在demo目錄下的ConverterDemo.scala裏：

 1 import CaseClassMapConverter._  2 object ConvertDemo extends App {  3 
 4   def ccToMap[C: CaseClassMapConverter](c: C): Map[String,Any] =
 5  implicitly[CaseClassMapConverter[C]].toMap(c)  6 
 7   case class Person(name: String, age: Int)  8   case class Car(make: String, year: Int, manu: String)  9 
10   val civic = Car("Civic",2016,"Honda") 11   //println(ccToMap[Person](Person("john",18))) 12   //println(ccToMap[Car](civic))
13 
14   def mapTocc[C: CaseClassMapConverter](m: Map[String,Any]) =
15  implicitly[CaseClassMapConverter[C]].fromMap(m) 16 
17   val mapJohn = ccToMap[Person](Person("john",18)) 18   val mapCivic = ccToMap[Car](civic) 19  println(mapTocc[Person](mapJohn)) 20  println(mapTocc[Car](mapCivic)) 21   
22 }

在上面這個implicit Macros例子裏引用了一些quasiquote語句（q"xxx"）。quasiquote是Scala Macros的一個重要部分，主要替代了原來reflect api中的reify功能，具有更強大、方便靈活的處理AST功能。Scala Def Macros還提供了Extractor Macros，結合Scala String Interpolation和模式匹配來提供compile time的extractor object生成。Extractor Macros的具體用例以下：

1  import ExtractorMicros._ 2   val fname = "William"
3   val lname = "Wang"
4   val someuser =  usr"$fname,$lname"  //new FreeUser("William","Wang")
5 
6  someuser match { 7     case usr"$first,$last" => println(s"hello $first $last") 8   }

在上面這個例子裏usr"???"的usr是一個pattern和extractor object。與一般的string interpolation不一樣的是usr不是一個方法（method）,而是一個對象（object）。這是因爲模式匹配中的unapply必須在一個extractor object內，因此usr是個object。咱們知道一個object加上它的apply能夠看成method來調用。也就是說若是在usr object中實現了apply就能夠用usr(???)做爲method使用，以下：

1   implicit class UserInterpolate(sc: StringContext) { 2     object usr { 3       def apply(args: String*): Any = macro UserMacros.appl 4       def unapply(u: User): Any = macro UserMacros.uapl 5  } 6   }

經過Def Macros在編譯過程當中自動生成apply和unapply，它們分別對應了函數調用：

val someuser =  usr"$fname,$lname"

case usr"$first,$last" => println(s"hello $first $last")

下面是macro appl的實現：

1 def appl(c: Context)(args: c.Tree*) = { 2  import c.universe._ 3     val arglist = args.toList 4     q"new FreeUser(..$arglist)"
5   }

主要經過q"new FreeUser(arg1,arg2)"實現了個AST的構建。macro uapl的實現相對複雜些，對quasiquote的應用會更深刻些。首先要肯定類型的primary constructor的參數數量和名稱，而後經過quasiquote的模式匹配分解出相應的sub-AST，再從新組合造成最終完整的AST：

 1 def uapl(c: Context)(u: c.Tree) = {  2  import c.universe._  3     val params = u.tpe.members.collectFirst {  4       case m: MethodSymbol if m.isPrimaryConstructor => m.asMethod  5     }.get.paramLists.head.map {p => p.asTerm.name.toString}  6 
 7     val (qget,qdef) = params.length match {  8       case len if len == 0 =>
 9         (List(q""),List(q"")) 10       case len if len == 1 =>
11         val pn = TermName(params.head) 12         (List(q"def get = u.$pn"),List(q"")) 13       case  _ =>
14         val defs = List(q"def _1 = x",q"def _2 = x",q"def _3 = x",q"def _4 = x") 15         val qdefs = (params zip defs).collect { 16           case (p,d) =>
17             val q"def $mname = $mbody" = d 18             val pn = TermName(p) 19             q"def $mname = u.$pn"
20  } 21         (List(q"def get = this"),qdefs) 22  } 23 
24       q""" 25         new { 26           class Matcher(u: User) { 27             def isEmpty = false
28  ..$qget 29  ..$qdef 30  } 31           def unapply(u: User) = new Matcher(u) 32  }.unapply($u) 33       """ 34  } 35 }

前面大部分代碼就是爲了造成List[Tree] qget和qdef,最後組合一個完整的quasiquote q""" new {...}"""。

完整的Macro實現源代碼以下：

 1 trait User {  2  val fname: String  3  val lname: String  4 }  5 
 6 class FreeUser(val fname: String, val lname: String) extends User {  7   val i = 10
 8   def f = 1 + 2
 9 } 10 class PremiumUser(val name: String, val gender: Char, val vipnum: String) //extends User
11 
12 object ExtractorMicros { 13   implicit class UserInterpolate(sc: StringContext) { 14     object usr { 15       def apply(args: String*): Any = macro UserMacros.appl 16       def unapply(u: User): Any = macro UserMacros.uapl 17  } 18  } 19 } 20 object UserMacros { 21   def appl(c: Context)(args: c.Tree*) = { 22  import c.universe._ 23     val arglist = args.toList 24     q"new FreeUser(..$arglist)"
25  } 26   def uapl(c: Context)(u: c.Tree) = { 27  import c.universe._ 28     val params = u.tpe.members.collectFirst { 29       case m: MethodSymbol if m.isPrimaryConstructor => m.asMethod 30     }.get.paramLists.head.map {p => p.asTerm.name.toString} 31 
32     val (qget,qdef) = params.length match { 33       case len if len == 0 =>
34         (List(q""),List(q"")) 35       case len if len == 1 =>
36         val pn = TermName(params.head) 37         (List(q"def get = u.$pn"),List(q"")) 38       case  _ =>
39         val defs = List(q"def _1 = x",q"def _2 = x",q"def _3 = x",q"def _4 = x") 40         val qdefs = (params zip defs).collect { 41           case (p,d) =>
42             val q"def $mname = $mbody" = d 43             val pn = TermName(p) 44             q"def $mname = u.$pn"
45  } 46         (List(q"def get = this"),qdefs) 47  } 48 
49       q""" 50         new { 51           class Matcher(u: User) { 52             def isEmpty = false
53  ..$qget 54  ..$qdef 55  } 56           def unapply(u: User) = new Matcher(u) 57  }.unapply($u) 58       """ 59  } 60 }

調用示範代碼：

 1 object Test extends App {  2  import ExtractorMicros._  3   val fname = "William"
 4   val lname = "Wang"
 5   val someuser =  usr"$fname,$lname"  //new FreeUser("William","Wang")
 6 
 7  someuser match {  8     case usr"$first,$last" => println(s"hello $first $last")  9  } 10 }

Macros Annotation（註釋）是Def Macro重要的功能部分。對一個目標，包括類型、對象、方法等進行註釋意思是在源代碼編譯時對它們進行拓展修改甚至徹底替換。好比咱們下面展現的方法註釋（method annotation）：假設咱們有下面兩個方法：

1   def testMethod[T]: Double = { 2     val x = 2.0 + 2.0
3  Math.pow(x, x) 4  } 5 
6   def testMethodWithArgs(x: Double, y: Double) = { 7     val z = x + y 8  Math.pow(z,z) 9   }

若是我想測試它們運行所需時間的話能夠在這兩個方法的內部代碼前設定開始時間，而後在代碼後截取完成時間，完成時間-開始時間就是運算所須要的時間了，以下：

1 def testMethod[T]: Double = { 2     val start = System.nanoTime() 3     
4     val x = 2.0 + 2.0
5  Math.pow(x, x) 6     
7     val end = System.nanoTime() 8     println(s"elapsed time is: ${end - start}") 9   }

咱們但願經過註釋來拓展這個方法：具體作法是保留原來的代碼，同時在方法內部先後增長几行代碼。咱們看看這個註釋的目的是如何實現的：

 1 def impl(c: Context)(annottees: c.Tree*): c.Tree = {  2  import c.universe._  3 
 4  annottees.head match {  5       case q"$mods def $mname[..$tpes](...$args): $rettpe = { ..$stats }" => {  6         q"""  7             $mods def $mname[..$tpes](...$args): $rettpe = {  8                val start = System.nanoTime()  9                val result = {..$stats} 10                val end = System.nanoTime() 11                println(${mname.toString} + " elapsed time in nano second = " + (end-start).toString()) 12  result 13  } 14           """ 15  } 16       case _ => c.abort(c.enclosingPosition, "Incorrect method signature!") 17     }

能夠看到：咱們仍是用quasiquote來分拆被註釋的方法，而後再用quasiquote重現組合這個方法。在重組過程當中增長了時間截取和列印代碼。下面這行是典型的AST模式分拆（pattern desctruction）：

      case q"$mods def $mname[..$tpes](...$args): $rettpe = { ..$stats }" => {...}

用這種方式把目標分拆成重組須要的最基本部分。Macro Annotation的實現源代碼以下：

 1 import scala.annotation.StaticAnnotation  2 import scala.language.experimental.macros  3 import scala.reflect.macros.blackbox.Context  4 
 5 class Benchmark extends StaticAnnotation {  6   def macroTransform(annottees: Any*): Any = macro Benchmark.impl  7 }  8 object Benchmark {  9   def impl(c: Context)(annottees: c.Tree*): c.Tree = { 10  import c.universe._ 11 
12  annottees.head match { 13       case q"$mods def $mname[..$tpes](...$args): $rettpe = { ..$stats }" => { 14         q""" 15             $mods def $mname[..$tpes](...$args): $rettpe = { 16                val start = System.nanoTime() 17                val result = {..$stats} 18                val end = System.nanoTime() 19                println(${mname.toString} + " elapsed time in nano second = " + (end-start).toString()) 20  result 21  } 22           """ 23  } 24       case _ => c.abort(c.enclosingPosition, "Incorrect method signature!") 25  } 26 
27  } 28 }

註釋的調用示範代碼以下：

 1 object annotMethodDemo extends App {  2 
 3  @Benchmark  4   def testMethod[T]: Double = {  5     //val start = System.nanoTime()
 6 
 7     val x = 2.0 + 2.0
 8  Math.pow(x, x)  9 
10     //val end = System.nanoTime() 11     //println(s"elapsed time is: ${end - start}")
12  } 13  @Benchmark 14   def testMethodWithArgs(x: Double, y: Double) = { 15     val z = x + y 16  Math.pow(z,z) 17  } 18 
19  testMethod[String] 20   testMethodWithArgs(2.0,3.0) 21 
22 
23 }

有一點值得注意的是：Macro擴展是編譯中遇到方法調用時發生的，而註釋目標的擴展則在更早一步的方法聲明時。咱們下面再看註釋class的例子：

 1 import scala.annotation.StaticAnnotation  2 import scala.language.experimental.macros  3 import scala.reflect.macros.blackbox.Context  4 
 5 class TalkingAnimal(val voice: String) extends StaticAnnotation {  6   def macroTransform(annottees: Any*): Any = macro TalkingAnimal.implAnnot  7 }  8 
 9 object TalkingAnimal { 10   def implAnnot(c: Context)(annottees: c.Tree*): c.Tree = { 11  import c.universe._ 12 
13  annottees.head match { 14       case q"$mods class $cname[..$tparams] $ctorMods(..$params) extends Animal with ..$parents {$self => ..$stats}" =>
15         val voice = c.prefix.tree match { 16           case q"new TalkingAnimal($sound)" => c.eval[String](c.Expr(sound)) 17           case _ =>
18  c.abort(c.enclosingPosition, 19                     "TalkingAnimal must provide voice sample!") 20  } 21         val animalType = cname.toString() 22         q""" 23             $mods class $cname(..$params) extends Animal { 24  ..$stats 25               def sayHello: Unit =
26                 println("Hello, I'm a " + $animalType + " and my name is " + name + " " + $voice + "...") 27  } 28           """ 29       case _ =>
30  c.abort(c.enclosingPosition, 31                 "Annotation TalkingAnimal only apply to Animal inherited!") 32  } 33  } 34 }

咱們看到：一樣仍是經過quasiquote進行AST模式拆分：

      case q"$mods class $cname[..$tparams] $ctorMods(..$params) extends Animal with ..$parents {$self => ..$stats}" =>

而後再從新組合。具體使用示範以下：

 1 object AnnotClassDemo extends App {  2  trait Animal {  3  val name: String  4  }  5   @TalkingAnimal("wangwang")  6   case class Dog(val name: String) extends Animal  7 
 8   @TalkingAnimal("miaomiao")  9   case class Cat(val name: String) extends Animal 10 
11   //@TalkingAnimal("") 12   //case class Carrot(val name: String) 13   //Error:(12,2) Annotation TalkingAnimal only apply to Animal inherited! @TalingAnimal
14   Dog("Goldy").sayHello 15   Cat("Kitty").sayHello 16 
17 }

 運算結果以下：

Hello, I'm a Dog and my name is Goldy wangwang...
Hello, I'm a Cat and my name is Kitty miaomiao...
 Process finished with exit code 0