在上一篇文章加快apk的構建速度,如何把編譯時間從130秒降到17秒中講了優化的思路與初步的實現,通過一段時間的優化性能和穩定性都有很大的提升,這裏要感謝你們提的建議以及github上的issue,這篇文章就把主要優化的點和新功能以及填的坑介紹下。html
項目地址: github.com/typ0520/fas…
對應tag: github.com/typ0520/fas…
demo代碼: github.com/typ0520/fas…java
注: 建議把fastdex的代碼和demo代碼拉下來,本文中的絕大部分例子在demo工程中能夠直接跑
注: 本文對gradle task作的說明都創建在關閉instant run的前提下
注: 本文全部的代碼、gradle任務名、任務輸出路徑、所有使用debug這個buildType做說明
注: 本文使用./gradlew執行任務是在mac下,若是是windows換成gradlew.batandroid
###1、攔截transformClassesWithJarMergingForDebug任務git
以前補丁打包的時候,是把沒有變化的類從app/build/intermediates/transforms/jarMerging/debug/jars/1/1f/combined.jar中移除,這樣的作法有兩個問題github
如今首先須要拿到transformClassesWithJarMergingForDebug任務執行先後的生命週期,實現的方式和攔截transformClassesWithDexForDebug時用的方案差很少,完整的測試代碼地址
github.com/typ0520/fas…shell
public class MyJarMergingTransform extends Transform {
Transform base
MyJarMergingTransform(Transform base) {
this.base = base
}
@Override
void transform(TransformInvocation invocation) throws TransformException, IOException, InterruptedException {
List<JarInput> jarInputs = Lists.newArrayList();
List<DirectoryInput> dirInputs = Lists.newArrayList();
for (TransformInput input : invocation.getInputs()) {
jarInputs.addAll(input.getJarInputs());
}
for (TransformInput input : invocation.getInputs()) {
dirInputs.addAll(input.getDirectoryInputs());
}
for (JarInput jarInput : jarInputs) {
println("==jarmerge jar : ${jarInput.file}")
}
for (DirectoryInput directoryInput : dirInputs) {
println("==jarmerge directory: ${directoryInput.file}")
}
File combinedJar = invocation.outputProvider.getContentLocation("combined", base.getOutputTypes(), base.getScopes(), Format.JAR);
println("==combinedJar exists ${combinedJar.exists()} ${combinedJar}")
base.transform(invocation)
println("==combinedJar exists ${combinedJar.exists()} ${combinedJar}")
}
}
public class MyDexTransform extends Transform {
Transform base
MyDexTransform(Transform base) {
this.base = base
}
@Override
void transform(TransformInvocation transformInvocation) throws TransformException, IOException, InterruptedException {
List<JarInput> jarInputs = Lists.newArrayList();
List<DirectoryInput> dirInputs = Lists.newArrayList();
for (TransformInput input : transformInvocation.getInputs()) {
jarInputs.addAll(input.getJarInputs());
}
for (TransformInput input : transformInvocation.getInputs()) {
dirInputs.addAll(input.getDirectoryInputs());
}
for (JarInput jarInput : jarInputs) {
println("==dex jar : ${jarInput.file}")
}
for (DirectoryInput directoryInput : dirInputs) {
println("==dex directory: ${directoryInput.file}")
}
base.transform(transformInvocation)
}
}
project.afterEvaluate {
android.applicationVariants.all { variant ->
project.getGradle().getTaskGraph().addTaskExecutionGraphListener(new TaskExecutionGraphListener() {
@Override
public void graphPopulated(TaskExecutionGraph taskGraph) {
for (Task task : taskGraph.getAllTasks()) {
if (task.getProject().equals(project) && task instanceof TransformTask && task.name.toLowerCase().contains(variant.name.toLowerCase())) {
Transform transform = ((TransformTask) task).getTransform()
//若是開啓了multidex有這個任務
if ((((transform instanceof JarMergingTransform)) && !(transform instanceof MyJarMergingTransform))) {
project.logger.error("==fastdex find jarmerging transform. transform class: " + task.transform.getClass() + " . task name: " + task.name)
MyJarMergingTransform jarMergingTransform = new MyJarMergingTransform(transform)
Field field = getFieldByName(task.getClass(),'transform')
field.setAccessible(true)
field.set(task,jarMergingTransform)
}
if ((((transform instanceof DexTransform)) && !(transform instanceof MyDexTransform))) {
project.logger.error("==fastdex find dex transform. transform class: " + task.transform.getClass() + " . task name: " + task.name)
//代理DexTransform,實現自定義的轉換
MyDexTransform fastdexTransform = new MyDexTransform(transform)
Field field = getFieldByName(task.getClass(),'transform')
field.setAccessible(true)
field.set(task,fastdexTransform)
}
}
}
}
});
}
}複製代碼
把上面的代碼放進app/build.gradle執行./gradlew assembleDebugmacos
開啓multidex(multiDexEnabled true)時的日誌輸出**bootstrap
:app:mergeDebugAssets
:app:transformClassesWithJarMergingForDebug
==jarmerge jar : /Users/tong/Projects/fastdex-test-project/jarmerging-test/app/libs/exist-in-app-libs-2.1.2.jar
==jarmerge jar : /Users/tong/Projects/fastdex-test-project/jarmerging-test/app/build/intermediates/exploded-aar/com.android.support/multidex/1.0.1/jars/classes.jar
==jarmerge jar : /Users/tong/Applications/android-sdk-macosx/extras/android/m2repository/com/android/support/support-annotations/23.3.0/support-annotations-23.3.0.jar
==jarmerge jar : /Users/tong/Projects/fastdex-test-project/jarmerging-test/app/build/intermediates/exploded-aar/com.jakewharton/butterknife/8.0.1/jars/classes.jar
==jarmerge jar : /Users/tong/.gradle/caches/modules-2/files-2.1/com.jakewharton/butterknife-annotations/8.0.1/345b89f45d02d8b09400b472fab7b7e38f4ede1f/butterknife-annotations-8.0.1.jar
==jarmerge jar : /Users/tong/Projects/fastdex-test-project/jarmerging-test/javalib/build/libs/javalib.jar
==jarmerge jar : /Users/tong/Projects/fastdex-test-project/jarmerging-test/app/build/intermediates/exploded-aar/jarmerging-test/aarlib/unspecified/jars/classes.jar
==jarmerge directory: /Users/tong/Projects/fastdex-test-project/jarmerging-test/app/build/intermediates/classes/debug
==combinedJar exists false /Users/tong/Projects/fastdex-test-project/jarmerging-test/app/build/intermediates/transforms/jarMerging/debug/jars/1/1f/combined.jar
==combinedJar exists true /Users/tong/Projects/fastdex-test-project/jarmerging-test/app/build/intermediates/transforms/jarMerging/debug/jars/1/1f/combined.jar
:app:transformClassesWithMultidexlistForDebug
:app:transformClassesWithDexForDebug
===dex jar : /Users/tong/Projects/fastdex-test-project/jarmerging-test/app/build/intermediates/transforms/jarMerging/debug/jars/1/1f/combined.jar
:app:mergeDebugJniLibFolders複製代碼
關閉multidex(multiDexEnabled false)時的日誌輸出**windows
:app:mergeDebugAssets
:app:transformClassesWithDexForDebug
===dex jar : /Users/tong/Projects/fastdex-test-project/jarmerging-test/app/libs/exist-in-app-libs-2.1.2.jar
===dex jar : /Users/tong/Applications/android-sdk-macosx/extras/android/m2repository/com/android/support/support-annotations/23.3.0/support-annotations-23.3.0.jar
===dex jar : /Users/tong/Projects/fastdex-test-project/jarmerging-test/app/build/intermediates/exploded-aar/com.jakewharton/butterknife/8.0.1/jars/classes.jar
===dex jar : /Users/tong/.gradle/caches/modules-2/files-2.1/com.jakewharton/butterknife-annotations/8.0.1/345b89f45d02d8b09400b472fab7b7e38f4ede1f/butterknife-annotations-8.0.1.jar
===dex jar : /Users/tong/Projects/fastdex-test-project/jarmerging-test/javalib/build/libs/javalib.jar
===dex jar : /Users/tong/Projects/fastdex-test-project/jarmerging-test/app/build/intermediates/exploded-aar/jarmerging-test/aarlib/unspecified/jars/classes.jar
===dex directory: /Users/tong/Projects/fastdex-test-project/jarmerging-test/app/build/intermediates/classes/debug
:app:mergeDebugJniLibFolders複製代碼
從上面的日誌輸出能夠看出,只須要在下圖紅色箭頭指的地方作patch.jar的生成就能夠了api
另外以前全量打包作asm code注入的時候是遍歷combined.jar若是entry對應的是項目代碼就作注入,反之認爲是第三方庫跳過注入(第三方庫不在修復之列,爲了節省注入花費的時間因此忽略);如今攔截了jarmerge任務,直接掃描全部的DirectoryInput對應目錄下的全部class作注入就好了,效率會比以前的作法有很大提高
###2、對直接依賴的library工程作支持
如下面這個工程爲例
github.com/typ0520/fas…
這個工程包含三個子工程
app工程依賴aarlib和javalib
dependencies {
compile fileTree(dir: 'libs', include: ['*.jar'])
compile 'com.jakewharton:butterknife:8.0.1'
apt 'com.jakewharton:butterknife-compiler:8.0.1'
compile project(':javalib')
compile project(':aarlib')
compile project(':libgroup:javalib2')
}複製代碼
對於使用compile project(':xxx')這種方式依賴的工程,在apk的構建過程當中是當作jar處理的,從攔截transformClassesWithJarMergingForDebug任務時的日誌輸出能夠證實
===dex jar: /Users/tong/Projects/fastdex-test-project/jarmerging-test/javalib/build/libs/javalib.jar
===dex jar: /Users/tong/Projects/fastdex-test-project/jarmerging-test/app/build/intermediates/exploded-aar/jarmerging-test/aarlib/unspecified/jars/classes.jar複製代碼
以前修改了library工程的代碼補丁打包之因此沒有生效,就是由於補丁打包時只從DirectoryInput中抽離變化的class而沒有對library工程的輸出jar作抽離,這個時候就須要知道JarInput中那些屬於library工程那些屬於第三方庫。最直接的方式是經過文件系統路徑區分,可是這樣須要排除掉library工程中直接放在libs目錄下依賴的jar好比
==jarmerge jar: /Users/tong/Projects/fastdex-test-project/jarmerging-test/app/libs/exist-in-app-libs-2.1.2.jar複製代碼
其次若是依賴的library目錄和app工程不在同一個目錄下還要作容錯的判斷
==jarmerge jar: /Users/tong/Projects/fastdex-test-project/jarmerging-test/libgroup/javalib2/build/libs/javalib2.jar複製代碼
最終放棄了判斷路徑的方式,轉而去找android gradle的api拿到每一個library工程的輸出jar路徑,翻閱了源碼發現2.0.0、2.2.0、2.3.0對應的api都不同,經過判斷版本的方式能夠解決,代碼以下
public class LibDependency {
public final File jarFile;
public final Project dependencyProject;
public final boolean androidLibrary;
LibDependency(File jarFile, Project dependencyProject, boolean androidLibrary) {
this.jarFile = jarFile
this.dependencyProject = dependencyProject
this.androidLibrary = androidLibrary
}
boolean equals(o) {
if (this.is(o)) return true
if (getClass() != o.class) return false
LibDependency that = (LibDependency) o
if (jarFile != that.jarFile) return false
return true
}
int hashCode() {
return (jarFile != null ? jarFile.hashCode() : 0)
}
@Override
public String toString() {
return "LibDependency{" +
"jarFile=" + jarFile +
", dependencyProject=" + dependencyProject +
", androidLibrary=" + androidLibrary +
'}';
}
private static Project getProjectByPath(Collection<Project> allprojects, String path) {
return allprojects.find { it.path.equals(path) }
}
/**
* 掃描依賴(<= 2.3.0)
* @param library
* @param libraryDependencies
*/
private static final void scanDependency(com.android.builder.model.Library library,Set<com.android.builder.model.Library> libraryDependencies) {
if (library == null) {
return
}
if (library.getProject() == null) {
return
}
if (libraryDependencies.contains(library)) {
return
}
libraryDependencies.add(library)
if (library instanceof com.android.builder.model.AndroidLibrary) {
List<com.android.builder.model.Library> libraryList = library.getJavaDependencies()
if (libraryList != null) {
for (com.android.builder.model.Library item : libraryList) {
scanDependency(item,libraryDependencies)
}
}
libraryList = library.getLibraryDependencies()
if (libraryList != null) {
for (com.android.builder.model.Library item : libraryList) {
scanDependency(item,libraryDependencies)
}
}
}
else if (library instanceof com.android.builder.model.JavaLibrary) {
List<com.android.builder.model.Library> libraryList = library.getDependencies()
if (libraryList != null) {
for (com.android.builder.model.Library item : libraryList) {
scanDependency(item,libraryDependencies)
}
}
}
}
/**
* 掃描依賴(2.0.0 <= android-build-version <= 2.2.0)
* @param library
* @param libraryDependencies
*/
private static final void scanDependency_2_0_0(Object library,Set<com.android.builder.model.Library> libraryDependencies) {
if (library == null) {
return
}
if (library.getProject() == null){
return
}
if (libraryDependencies.contains(library)) {
return
}
libraryDependencies.add(library)
if (library instanceof com.android.builder.model.AndroidLibrary) {
List<com.android.builder.model.Library> libraryList = library.getLibraryDependencies()
if (libraryList != null) {
for (com.android.builder.model.Library item : libraryList) {
scanDependency_2_0_0(item,libraryDependencies)
}
}
}
}
/**
* 解析項目的工程依賴 compile project('xxx')
* @param project
* @return
*/
public static final Set<LibDependency> resolveProjectDependency(Project project, ApplicationVariant apkVariant) {
Set<LibDependency> libraryDependencySet = new HashSet<>()
VariantDependencies variantDeps = apkVariant.getVariantData().getVariantDependency();
if (Version.ANDROID_GRADLE_PLUGIN_VERSION.compareTo("2.3.0") >= 0) {
def allDependencies = new HashSet<>()
allDependencies.addAll(variantDeps.getCompileDependencies().getAllJavaDependencies())
allDependencies.addAll(variantDeps.getCompileDependencies().getAllAndroidDependencies())
for (Object dependency : allDependencies) {
if (dependency.projectPath != null) {
def dependencyProject = getProjectByPath(project.rootProject.allprojects,dependency.projectPath);
boolean androidLibrary = dependency.getClass().getName().equals("com.android.builder.dependency.level2.AndroidDependency");
File jarFile = null
if (androidLibrary) {
jarFile = dependency.getJarFile()
}
else {
jarFile = dependency.getArtifactFile()
}
LibDependency libraryDependency = new LibDependency(jarFile,dependencyProject,androidLibrary)
libraryDependencySet.add(libraryDependency)
}
}
}
else if (Version.ANDROID_GRADLE_PLUGIN_VERSION.compareTo("2.2.0") >= 0) {
Set<Library> librarySet = new HashSet<>()
for (Object jarLibrary : variantDeps.getCompileDependencies().getJarDependencies()) {
scanDependency(jarLibrary,librarySet)
}
for (Object androidLibrary : variantDeps.getCompileDependencies().getAndroidDependencies()) {
scanDependency(androidLibrary,librarySet)
}
for (com.android.builder.model.Library library : librarySet) {
boolean isAndroidLibrary = (library instanceof AndroidLibrary);
File jarFile = null
def dependencyProject = getProjectByPath(project.rootProject.allprojects,library.getProject());
if (isAndroidLibrary) {
com.android.builder.dependency.LibraryDependency androidLibrary = library;
jarFile = androidLibrary.getJarFile()
}
else {
jarFile = library.getJarFile();
}
LibDependency libraryDependency = new LibDependency(jarFile,dependencyProject,isAndroidLibrary)
libraryDependencySet.add(libraryDependency)
}
}
else {
Set librarySet = new HashSet<>()
for (Object jarLibrary : variantDeps.getJarDependencies()) {
if (jarLibrary.getProjectPath() != null) {
librarySet.add(jarLibrary)
}
//scanDependency_2_0_0(jarLibrary,librarySet)
}
for (Object androidLibrary : variantDeps.getAndroidDependencies()) {
scanDependency_2_0_0(androidLibrary,librarySet)
}
for (Object library : librarySet) {
boolean isAndroidLibrary = (library instanceof AndroidLibrary);
File jarFile = null
def projectPath = (library instanceof com.android.builder.dependency.JarDependency) ? library.getProjectPath() : library.getProject()
def dependencyProject = getProjectByPath(project.rootProject.allprojects,projectPath);
if (isAndroidLibrary) {
com.android.builder.dependency.LibraryDependency androidLibrary = library;
jarFile = androidLibrary.getJarFile()
}
else {
jarFile = library.getJarFile();
}
LibDependency libraryDependency = new LibDependency(jarFile,dependencyProject,isAndroidLibrary)
libraryDependencySet.add(libraryDependency)
}
}
return libraryDependencySet
}
}複製代碼
把上面的這段代碼,和下面的代碼都放進build.gradle中
project.afterEvaluate {
android.applicationVariants.all { variant ->
def variantName = variant.name.capitalize()
if ("Debug".equals(variantName)) {
LibDependency.resolveProjectDependency(project,variant).each {
println("==androidLibrary: " + it.androidLibrary + " ,jarFile: " + it.jarFile)
}
}
}
}
task resolveProjectDependency<< {
}複製代碼
執行./gradlew resolveProjectDependency 能夠獲得如下輸出
==androidLibrary: true ,jarFile: /Users/tong/Projects/fastdex-test-project/jarmerging-test/app/build/intermediates/exploded-aar/jarmerging-test/aarlib/unspecified/jars/classes.jar
==androidLibrary: false ,jarFile: /Users/tong/Projects/fastdex-test-project/jarmerging-test/javalib/build/libs/javalib.jar
==androidLibrary: false ,jarFile: /Users/tong/Projects/fastdex-test-project/jarmerging-test/libgroup/javalib2/build/libs/javalib2.jar複製代碼
有了這些路徑咱們就能夠在遍歷JarInput是進行匹配,只要在這個路徑列表中的都屬於library工程的輸出jar,用到這塊有兩處地方
全量打包時注入library輸出jar ClassInject.groovy
public static void injectJarInputFiles(FastdexVariant fastdexVariant, HashSet<File> jarInputFiles) {
def project = fastdexVariant.project
long start = System.currentTimeMillis()
Set<LibDependency> libraryDependencies = fastdexVariant.libraryDependencies
List<File> projectJarFiles = new ArrayList<>()
//獲取全部依賴工程的輸出jar (compile project(':xxx'))
for (LibDependency dependency : libraryDependencies) {
projectJarFiles.add(dependency.jarFile)
}
if (fastdexVariant.configuration.debug) {
project.logger.error("==fastdex projectJarFiles : ${projectJarFiles}")
}
for (File file : jarInputFiles) {
if (!projectJarFiles.contains(file)) {
continue
}
project.logger.error("==fastdex ==inject jar: ${file}")
ClassInject.injectJar(fastdexVariant,file,file)
}
long end = System.currentTimeMillis()
project.logger.error("==fastdex inject complete jar-size: ${projectJarFiles.size()} , use: ${end - start}ms")
}複製代碼
public static void generatePatchJar(FastdexVariant fastdexVariant, TransformInvocation transformInvocation, File patchJar) throws IOException {
Set<LibDependency> libraryDependencies = fastdexVariant.libraryDependencies
Map<String,String> jarAndProjectPathMap = new HashMap<>()
List<File> projectJarFiles = new ArrayList<>()
//獲取全部依賴工程的輸出jar (compile project(':xxx'))
for (LibDependency dependency : libraryDependencies) {
projectJarFiles.add(dependency.jarFile)
jarAndProjectPathMap.put(dependency.jarFile.absolutePath,dependency.dependencyProject.projectDir.absolutePath)
}
//全部的class目錄
Set<File> directoryInputFiles = new HashSet<>();
//全部輸入的jar
Set<File> jarInputFiles = new HashSet<>();
for (TransformInput input : transformInvocation.getInputs()) {
Collection<DirectoryInput> directoryInputs = input.getDirectoryInputs()
if (directoryInputs != null) {
for (DirectoryInput directoryInput : directoryInputs) {
directoryInputFiles.add(directoryInput.getFile())
}
}
if (!projectJarFiles.isEmpty()) {
Collection<JarInput> jarInputs = input.getJarInputs()
if (jarInputs != null) {
for (JarInput jarInput : jarInputs) {
if (projectJarFiles.contains(jarInput.getFile())) {
jarInputFiles.add(jarInput.getFile())
}
}
}
}
}
def project = fastdexVariant.project
File tempDir = new File(fastdexVariant.buildDir,"temp")
FileUtils.deleteDir(tempDir)
FileUtils.ensumeDir(tempDir)
Set<File> moudleDirectoryInputFiles = new HashSet<>()
DiffResultSet diffResultSet = fastdexVariant.projectSnapshoot.diffResultSet
for (File file : jarInputFiles) {
String projectPath = jarAndProjectPathMap.get(file.absolutePath)
List<String> patterns = diffResultSet.addOrModifiedClassesMap.get(projectPath)
if (patterns != null && !patterns.isEmpty()) {
File classesDir = new File(tempDir,"${file.name}-${System.currentTimeMillis()}")
project.copy {
from project.zipTree(file)
for (String pattern : patterns) {
include pattern
}
into classesDir
}
moudleDirectoryInputFiles.add(classesDir)
directoryInputFiles.add(classesDir)
}
}
JarOperation.generatePatchJar(fastdexVariant,directoryInputFiles,moudleDirectoryInputFiles,patchJar);
}複製代碼
fastdex目前須要對比的地方有三處
以第一種場景爲例,說下對比的原理,全量打包時生成一個文本文件把當前的依賴寫進去以換行符分割
/Users/tong/Projects/fastdex/sample/app/libs/fm-sdk-2.1.2.jar
/Users/tong/Projects/fastdex/sample/javalib/build/libs/javalib.jar複製代碼
補丁打包時先把這個文本文件讀取到ArrayList中,而後把當前的依賴列表頁放進ArrayList中
,經過如下操做能夠獲取新增項、刪除項,只要發現有刪除項和新增項就認爲依賴發生了變化
ArrayList<String> old = new ArrayList<>();
old.add("/Users/tong/Projects/fastdex/sample/app/libs/fm-sdk-2.1.2.jar");
old.add("/Users/tong/Projects/fastdex/sample/javalib/build/libs/javalib.jar");
ArrayList<String> now = new ArrayList<>();
now.add("/Users/tong/Projects/fastdex/sample/app/libs/fm-sdk-2.1.2.jar");
now.add("/Users/tong/Projects/fastdex/sample/javalib/build/libs/new.jar");
//獲取刪除項
Set<String> deletedNodes = new HashSet<>();
deletedNodes.addAll(old);
deletedNodes.removeAll(now);
//新增項
Set<String> increasedNodes = new HashSet<>();
increasedNodes.addAll(now);
//若是不用ArrayList套一層有時候會發生移除不掉的狀況 why?
increasedNodes.removeAll(old);
//須要檢測是否變化的列表
Set<String> needDiffNodes = new HashSet<>();
needDiffNodes.addAll(now);
needDiffNodes.addAll(old);
needDiffNodes.removeAll(deletedNodes);
needDiffNodes.removeAll(increasedNodes);複製代碼
注: 文本的對比不存在更新,可是文件對比是存在這種狀況的
全部的快照對比都是基於上面這段代碼的抽象,具體能夠參考這裏
github.com/typ0520/fas…
全量打包之後,按照正常的開發節奏發生變化的源文件會愈來愈多,相應的參與dex生成的class也會愈來愈多,這樣會致使補丁打包速度愈來愈慢。
解決這個問題比較簡單的方式是把每次生成的patch.dex放進全量打包時的dex緩存中(必須排在以前的dex前面),而且更新下源代碼快照,這樣作有兩個壞處
app/build/intermediates/transforms/dex/debug/folders/1000/1f/main複製代碼
解決第二個問題的方案是把patch.dex中的class合併到緩存的dex中,這樣就不須要保留全部的patch.dex了,一個比較棘手的問題是若是緩存的dex的方法數已經有65535個了,在往裏面加新增的class,確定會爆掉了,最終fastdex選擇的方案是第一次觸發dex merge時直接把patch.dex扔進緩存(merged-patch.dex),之後在觸發dex merge時就拿patch.dex和merged-patch.dex作合併(這樣作也存在潛在的問題,若是變化的class特別多也有可能致使合併dex時出現65535的錯誤)
解決第一個問題是加了一個可配置選項,默認是3個以上的源文件發生變化時觸發merge,這樣即不用每次都作代碼注入和merge操做,也能在源文件變化多的時候恢復狀態
這個dex merge工具是從freeline裏找到的,感興趣的話能夠把下載下來試着調用下
github.com/typ0520/fas…
java -jar fastdex-dex-merge.jar output.dex patch.dex merged-patch.dex複製代碼
在現階段的Android開發中,註解愈來愈流行起來,好比ButterKnife,EventBus等等都選擇使用註解來配置。按照處理時期,註解又分爲兩種類型,一種是運行時註解,另外一種是編譯時註解,運行時註解因爲性能問題被一些人所詬病。編譯時註解的核心依賴APT(Annotation Processing Tools)實現,原理是在某些代碼元素上(如類型、函數、字段等)添加註解,在編譯時編譯器會檢查AbstractProcessor的子類,而且調用該類型的process函數,而後將添加了註解的全部元素都傳遞到process函數中,使得開發人員能夠在編譯期進行相應的處理,例如,根據註解生成新的Java類,這也就是ButterKnife,EventBus等開源庫的基本原理。Java API已經提供了掃描源碼並解析註解的框架,你能夠繼承AbstractProcessor類來提供實現本身的解析註解邏輯
-- 引用自blog.csdn.net/industrious…
雖然能提升運行期的效率但也給開發帶來一些麻煩
AbstractProcessor這些類只有在編譯期纔會用到,運行期是用不到的,可是若是經過compile方式依賴的包,會把這些類都打包進dex中
以這個項目爲例(建議把代碼拉下來,後面好幾個地方會用到)
github.com/typ0520/fas…
app中依賴了butterknife7.0.1
dependencies {
compile 'com.jakewharton:butterknife:7.0.1'
}複製代碼
butterknife7.0.1中的註解生成器叫ButterKnifeProcessor
執行./gradlew app:assembleDebug
從上圖能夠看出ButterKnifeProcessor.class被打包進dex中了
app2中依賴了butterknife8.8.1
apply plugin: 'com.jakewharton.butterknife'
dependencies {
compile 'com.jakewharton:butterknife:8.8.1'
annotationProcessor 'com.jakewharton:butterknife-compiler:8.8.1'
}複製代碼
執行./gradlew app2:assembleDebug
從上圖能夠看出butterknife.compiler包下全部的代碼都沒有被打包進dex。雖然經過annotationProcessor依賴AbstractProcessor相關代碼有上述好處,可是會形成增量編譯不可用,簡單地說就是正常的項目執行compileDebugJavaWithJavac任務調用javac的時候只會編譯內容發生變化的java源文件,若是使用了annotationProcessor每次執行compileDebugJavaWithJavac任務都會把項目中全部的java文件都參與編譯,想象一下若是項目中有成百上千個java文件編譯起來那酸爽。咱們能夠作個測試,仍是使用這個項目
github.com/typ0520/fas…
annotation-generators包含三個子項目
app依賴7.0.1
compile 'com.jakewharton:butterknife:7.0.1'複製代碼
app2依賴8.8.1
dependencies {
compile 'com.jakewharton:butterknife:8.8.1'
annotationProcessor 'com.jakewharton:butterknife-compiler:8.8.1'
}複製代碼
這三個子工程都包含兩個java文件
com/github/typ0520/annotation_generators/HAHA.java
com/github/typ0520/annotation_generators/MainActivity.java
測試的思路是先檢查MainActivity.class文件的更新時間,而後修改HAHA.java執行編譯,最後在檢查MainActivity.class文件的更新時間是否和編譯以前的一致,若是一致說明增量編譯可用,反之不可用
經過increment_compile_test.sh這個shell腳原本作測試(使用windows的同窗能夠手動作測試V_V)
#!/bin/bash
sh gradlew assembleDebug
test_increment_compile() {
echo "========測試${1}是否支持增量, ${2}"
str=$(stat -x ${1}/build/intermediates/classes/debug/com/github/typ0520/annotation_generators/MainActivity.class | grep 'Modify')
echo $str
echo 'package com.github.typ0520.annotation_generators;' > ${1}/src/main/java/com/github/typ0520/annotation_generators/HAHA.java
echo 'public class HAHA {' >> ${1}/src/main/java/com/github/typ0520/annotation_generators/HAHA.java
echo " public long millis = $(date +%s);" >> ${1}/src/main/java/com/github/typ0520/annotation_generators/HAHA.java
echo '}' >> ${1}/src/main/java/com/github/typ0520/annotation_generators/HAHA.java
sh gradlew ${1}:assembleDebug > /dev/null
str2=$(stat -x ${1}/build/intermediates/classes/debug/com/github/typ0520/annotation_generators/MainActivity.class | grep 'Modify')
echo $str2
echo ' '
if [ "$str" == "$str2" ];then
echo "${1}只修改HAHA.java,MainActivity.class沒有發生變化"
else
echo "${1}只修改HAHA.java,MainActivity.class發生變化"
fi
}
test_increment_compile app "compile 'com.jakewharton:butterknife:7.0.1'"
test_increment_compile app2 "annotationProcessor 'com.jakewharton:butterknife-compiler:8.8.1'"
test_increment_compile app3 "沒有用任何AbstractProcessor"複製代碼
執行sh increment_compile_test.sh
日誌的輸出能夠證實上面所描述的
既然原生不支持那麼咱們就在自定義的java compile任務中來作這個事情,經過以前的快照模塊能夠對比出那些java源文件發生了變化,那麼就能夠本身拼接javac命令參數而後調用僅編譯變化的java文件
demo中寫了一個編譯任務方便你們理解這些參數都是怎麼拼接的,代碼太多了這裏就不貼出來了
github.com/typ0520/fas…
github.com/typ0520/fas…
能夠調用./gradlew mycompile1 或者 ./gradlew mycompile2看下最終拼接出來的命令
fastdex中對應模塊的代碼在
github.com/typ0520/fas…
解決的bug這塊原本是不許備說的,由於這塊最有價值的東西不是解決問題自己,而是怎麼發現和重現問題的,這塊確實不太好描述V_V,應簡友的要求仍是挑了一些相對比較有養分的問題說下,主要仍是說解決的方法,至於問題是怎樣定位和重現的只能盡力描述了。
致使這個問題的緣由是項目中原來的YtxApplication類被替換成了FastdexApplication,當在activity中執行相似於下面的操做時就會報ClassCastException
MyApplication app = (MyApplication) getApplication();複製代碼
解決的方法是在instant-run的源碼裏找到的,運行期把android api裏全部引用Application的地方把實例替換掉
public static void monkeyPatchApplication( Context context,
Application bootstrap,
Application realApplication,
String externalResourceFile) {
try {
// Find the ActivityThread instance for the current thread
Class<?> activityThread = Class.forName("android.app.ActivityThread");
Object currentActivityThread = getActivityThread(context, activityThread);
// Find the mInitialApplication field of the ActivityThread to the real application
Field mInitialApplication = activityThread.getDeclaredField("mInitialApplication");
mInitialApplication.setAccessible(true);
Application initialApplication = (Application) mInitialApplication.get(currentActivityThread);
if (realApplication != null && initialApplication == bootstrap) {
mInitialApplication.set(currentActivityThread, realApplication);
}
// Replace all instance of the stub application in ActivityThread#mAllApplications with the
// real one
if (realApplication != null) {
Field mAllApplications = activityThread.getDeclaredField("mAllApplications");
mAllApplications.setAccessible(true);
List<Application> allApplications = (List<Application>) mAllApplications
.get(currentActivityThread);
for (int i = 0; i < allApplications.size(); i++) {
if (allApplications.get(i) == bootstrap) {
allApplications.set(i, realApplication);
}
}
}
// Figure out how loaded APKs are stored.
// API version 8 has PackageInfo, 10 has LoadedApk. 9, I don't know. Class<?> loadedApkClass; try { loadedApkClass = Class.forName("android.app.LoadedApk"); } catch (ClassNotFoundException e) { loadedApkClass = Class.forName("android.app.ActivityThread$PackageInfo"); } Field mApplication = loadedApkClass.getDeclaredField("mApplication"); mApplication.setAccessible(true); Field mResDir = loadedApkClass.getDeclaredField("mResDir"); mResDir.setAccessible(true); Field mLoadedApk = null; try { mLoadedApk = Application.class.getDeclaredField("mLoadedApk"); } catch (NoSuchFieldException e) { // According to testing, it's okay to ignore this.
}
for (String fieldName : new String[]{"mPackages", "mResourcePackages"}) {
Field field = activityThread.getDeclaredField(fieldName);
field.setAccessible(true);
Object value = field.get(currentActivityThread);
for (Map.Entry<String, WeakReference<?>> entry :
((Map<String, WeakReference<?>>) value).entrySet()) {
Object loadedApk = entry.getValue().get();
if (loadedApk == null) {
continue;
}
if (mApplication.get(loadedApk) == bootstrap) {
if (realApplication != null) {
mApplication.set(loadedApk, realApplication);
}
if (externalResourceFile != null) {
mResDir.set(loadedApk, externalResourceFile);
}
if (realApplication != null && mLoadedApk != null) {
mLoadedApk.set(realApplication, loadedApk);
}
}
}
}
} catch (Throwable e) {
throw new IllegalStateException(e);
}
}複製代碼
具體能夠參考測試工程的代碼
github.com/typ0520/fas…
github.com/typ0520/fas…
@YuJunKui1995
這個錯誤的表現是若是項目裏包含baidumapapi_v2_0_0.jar,正常打包是沒問題的,只要使用fastdex就會報下面這個錯誤
Error:Error converting bytecode to dex:
Cause: PARSE ERROR:
class name (com/baidu/platform/comapi/map/a) does not match path (com/baidu/platform/comapi/map/A.class)
...while parsing com/baidu/platform/comapi/map/A.class複製代碼
通過分析使用fastdex打包時會有解壓jar而後在壓縮的操做,使用下面這段代碼作測試
github.com/typ0520/fas…
task gen_dex2<< {
File tempDir = project.file('temp')
tempDir.deleteDir()
project.copy {
from project.zipTree(project.file('baidumapapi_v2_0_0.jar'))
into tempDir
}
File baidumapJar = project.file('temp/baidu.jar')
project.ant.zip(baseDir: tempDir, destFile: baidumapJar)
ProcessBuilder processBuilder = new ProcessBuilder('dx','--dex',"--output=" + project.file('baidu.dex').absolutePath, baidumapJar.absolutePath)
def process = processBuilder.start()
InputStream is = process.getInputStream()
BufferedReader reader = new BufferedReader(new InputStreamReader(is))
String line = null
while ((line = reader.readLine()) != null) {
println(line)
}
reader.close()
int status = process.waitFor()
reader = new BufferedReader(new InputStreamReader(process.getErrorStream()));
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
reader.close();
try {
process.destroy()
} catch (Throwable e) {
}
}複製代碼
執行./gradlew gen_dex2
果不其然重現了這個問題,查了資料發現mac和windows同樣文件系統大小寫不敏感,若是jar包裏有A.class,解壓後有可能就變成a.class了,因此生成dex的時候會報不匹配的錯誤(相似的問題也會影響git,以前就發現改了一個文件名字的大小寫git檢測不到變化,當時沒有細想這個問題,如今看來也是一樣的問題)。知道問題是怎麼發生的那麼解決就簡單了,既然在文件系統操做jar會有問題,那就放在內存作,對應java的api就是ZipOutputStream和ZipInputStream。
對於mac下文件系統大小寫不敏感能夠在終端執行下面這段命令,體會下輸出
echo 'a' > a.txt;echo 'A' > A.txt;cat a.txt;cat A.txt複製代碼
github.com/typ0520/fas…
@dongzy
Error:Execution failed for task ':app:tinkerSupportProcess_360DebugManifest'.
java.io.FileNotFoundException: E:\newkp\kuaipiandroid\NewKp\app\src\main\java\com\dx168\fastdex\runtime\FastdexApplication.java (系統找不到指定的路徑。)複製代碼
出現這個錯誤的緣由是@dongzy的項目中使用了tinkerpatch的一鍵接入,tinkerpatch的gradle插件也有Application替換的功能,必須保證fastdexProcess{variantName}Manifest任務在最後執行才行
FastdexManifestTask manifestTask = project.tasks.create("fastdexProcess${variantName}Manifest", FastdexManifestTask)
manifestTask.fastdexVariant = fastdexVariant
manifestTask.mustRunAfter variantOutput.processManifest
variantOutput.processResources.dependsOn manifestTask
//fix issue#8
def tinkerPatchManifestTask = null
try {
tinkerPatchManifestTask = project.tasks.getByName("tinkerpatchSupportProcess${variantName}Manifest")
} catch (Throwable e) {}
if (tinkerPatchManifestTask != null) {
manifestTask.mustRunAfter tinkerPatchManifestTask
}複製代碼
這段不是解決問題的, 忍不住吐槽下這哥們,以爲浪費了他的時間,上來就是「親測無軟用,建議你們不要用什麼什麼的」,搞的我很是鬱悶,果斷用知乎上的一篇文章回應了過去
zhuanlan.zhihu.com/p/25768464
後來通過溝通發現這哥們在一個正常打包3秒的項目上作的測試,我也是無語了
說實在的真的但願你們對開源項目多一點尊重,以爲對本身有幫助就用。若是以爲很差,能夠選擇提建議,也能夠選擇默默離開,若是有時間有能力能夠參與進來優化,解決本身工做問題的同時也服務了你們。在這個快節奏的社會你們的時間都寶貴,你以爲測試一下浪費了時間就開始吐槽,有沒有想到開源項目的做者犧牲了大量的我的時間在解決一個一個問題、爲了解決新功能的技術點一個一個方案的作測試作對比呢?
注: 若是項目的dex生成小於10秒,建議不要使用fastdex,幾乎是感知不到效果的。
gradle編譯速度優化建議
少直接使用compile project(':xxx')依賴library工程,若是module比較多編譯開始的時候須要遍歷module根據build.gradle配置項目,另外每一個library工程都包含大量的任務每一個任務都須要對比輸入和輸出,這些小任務疊加到一塊的時間消耗也是很可觀的。 建議把library工程打成aar包丟到公司的maven服務器上,別和我說開發階段library常常改直接依賴方便,每次修改打包到maven服務器上沒有那麼麻煩。咱們團隊的項目都是隻有一個乾淨的application工程,library代碼全丟進了maven服務器,dex方法數在12w左右,使用fastdex修改了幾個java文件能穩定在8秒左右完成打包、發送補丁和app重啓
任何狀況都別在library工程裏使用flavor
具體能夠參考@依然範特稀西寫的這篇文章
Android 優化APP 構建速度的17條建議
github.com/typ0520/fas…
@junchenChow
[ant:javac] : warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
[ant:javac] /Users/zhoujunchen/as/xx/app/build/fastdex/DevelopDebug/custom-combind/com/xx/xx/xx/xx/CourseDetailActivity.java:229: 錯誤: -source 1.7 中不支持 lambda 表達式
[ant:javac] wrapperControlsView.postDelayed(() -> wrapperControlsView.initiativeRefresh(), 500L);
[ant:javac] ^
[ant:javac] (請使用 -source 8 或更高版本以啓用 lambda 表達式)
[ant:javac] /Users/zhoujunchen/as/android-donguo/app/build/fastdex/DevelopDebug/custom-combind/com/xx/xx/xx/xx/CourseDetailActivity.java:489: 錯誤: -source 1.7 中不支持方法引用
[ant:javac] .subscribe(conf -> ShareHelper.share(this, conf), Throwable::printStackTrace);
[ant:javac] ^
[ant:javac] (請使用 -source 8 或更高版本以啓用方法引用)
[ant:javac] 2 個錯誤
:app:fastdexCustomCompileDevelopDebugJavaWithJavac FAILED
有什麼選項沒開啓麼 不支持lambda?複製代碼
這個錯誤的緣由是以前自定義的編譯任務寫死了使用1.7去編譯,查閱gradle-retrolambda的源碼找到了這些代碼
github.com/evant/gradl…
https://github.com/evant/gradle-retrolambda/blob/master/gradle-retrolambda/src/main/groovy/me/tatarka/RetrolambdaPluginAndroid.groovy
private static configureCompileJavaTask(Project project, BaseVariant variant, RetrolambdaTransform transform) {
variant.javaCompile.doFirst {
def retrolambda = project.extensions.getByType(RetrolambdaExtension)
def rt = "$retrolambda.jdk/jre/lib/rt.jar"
variant.javaCompile.classpath = variant.javaCompile.classpath + project.files(rt)
ensureCompileOnJava8(retrolambda, variant.javaCompile)
}
transform.putVariant(variant)
}
private static ensureCompileOnJava8(RetrolambdaExtension retrolambda, JavaCompile javaCompile) {
javaCompile.sourceCompatibility = "1.8"
javaCompile.targetCompatibility = "1.8"
if (!retrolambda.onJava8) {
// Set JDK 8 for the compiler task
def javac = "${retrolambda.tryGetJdk()}/bin/javac"
if (!checkIfExecutableExists(javac)) {
throw new ProjectConfigurationException("Cannot find executable: $javac", null)
}
javaCompile.options.fork = true
javaCompile.options.forkOptions.executable = javac
}
}複製代碼
從這些代碼中咱們能夠得知如下信息
有了這些信息就能夠在自定義的編譯任務作處理了
if (project.plugins.hasPlugin("me.tatarka.retrolambda")) {
def retrolambda = project.retrolambda
def rt = "${retrolambda.jdk}${File.separator}jre${File.separator}lib${File.separator}rt.jar"
classpath.add(rt)
executable = "${retrolambda.tryGetJdk()}${File.separator}bin${File.separator}javac"
if (Os.isFamily(Os.FAMILY_WINDOWS)) {
executable = "${executable}.exe"
}
}
List<String> cmdArgs = new ArrayList<>()
cmdArgs.add(executable)
cmdArgs.add("-encoding")
cmdArgs.add("UTF-8")
cmdArgs.add("-g")
cmdArgs.add("-target")
cmdArgs.add(javaCompile.targetCompatibility)
cmdArgs.add("-source")
cmdArgs.add(javaCompile.sourceCompatibility)
cmdArgs.add("-cp")
cmdArgs.add(joinClasspath(classpath))複製代碼
具體能夠參考
github.com/typ0520/fas…
github.com/typ0520/fas…
@wsf5918 @ysnows @jianglei199212 @tianshaokai @Razhan
Caused by: java.lang.RuntimeException: ==fastdex jar input size is 117, expected is 1
at com.dx168.fastdex.build.transform.FastdexTransform.getCombinedJarFile(FastdexTransform.groovy:173)
at com.dx168.fastdex.build.transform.FastdexTransform$getCombinedJarFile.callCurrent(Unknown Source)
at com.dx168.fastdex.build.transform.FastdexTransform.transform(FastdexTransform.groovy:131)
at com.android.build.gradle.internal.pipeline.TransformTask$2.call(TransformTask.java:185)
at com.android.build.gradle.internal.pipeline.TransformTask$2.call(TransformTask.java:181)
at com.android.builder.profile.ThreadRecorder.record(ThreadRecorder.java:102)
at com.android.build.gradle.internal.pipeline.TransformTask.transform(TransformTask.java:176)
at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:73)
at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$IncrementalTaskAction.doExecute(DefaultTaskClassInfoStore.java:163)
at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.execute(DefaultTaskClassInfoStore.java:134)
at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.execute(DefaultTaskClassInfoStore.java:123)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:95)
at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:76)
... 78 more複製代碼
正常狀況下開啓multidex而且minSdkVersion < 21時會存在transformClassesWithJarMergingForDebug任務,用來合併全部的JarInput和DirectoryInput而且輸出到build/intermediates/transforms/jarMerging/debug/jars/1/1f/combined.jar,而這個錯誤的表現是丟失了jarMerging任務,因此走到dexTransform時原本指望只有一個combined.jar,可是因爲沒有合併因此jar input的個數是117。當時因爲一直沒法重現這個問題,因此就採用加標示的手段解決的,具體是當走到FastdexJarMergingTransform而且執行完成之後就把executedJarMerge設置爲true,走到dexTransform時判斷若是開啓了multidex而且executedJarMerge==false就說明是丟失了jarMerge任務,這個時候調用com.android.build.gradle.internal.transforms.JarMerger手動合併就能夠解決了,具體能夠參考GradleUtils的executeMerge方法
github.com/typ0520/fas…
後來在開發中發現了丟失jarMerging任務的規律以下
看到這裏第三點的表現是否是很奇怪,命令行和studio點擊run最終都是走gradle的流程,既然表現不同有多是傳的參數不同,把下面這段代碼放進build.gradle中
println "projectProperties: " + project.gradle.startParameter.projectProperties複製代碼
點擊studio的run按鈕選擇一個6.0的設備
獲得如下輸出
projectProperties: [android.injected.build.density:560dpi, android.injected.build.api:23, android.injected.invoked.from.ide:true, android.injected.build.abi:x86]複製代碼
使用上面的這些參數一個一個作測試,發現是android.injected.build.api=23這個參數影響的,咱們能夠用這個測試項目作下測試
github.com/typ0520/fas…
執行./gradlew clean assembleDebug -Pandroid.injected.build.api=23
注: gradle傳自定義的參數是以-P開頭
從上面的日誌輸出中能夠看出重現了丟失jarMerge任務,咱們再來總結下重現這個問題的條件
有告終論還沒完,之因此2.3.0是這個行爲是由於引入了build-cache機制,不合並是爲了作jar級別的dex緩存,這樣每次執行dex transform時只有第一次時第三方庫才參與生成,爲了提升效率也不會合並dex,若是項目比較大apk中多是出現幾十個甚至上百個dex
目前fastdex因爲作了jar合併至關於把這個特性禁掉了,後面會考慮再也不作合併使之能用dex緩存,這樣全量打包時的速度應該能夠提升不少,另外還能夠引入到除了debug別的build-type打包中,還有設備必須大於6.0問題也能夠處理下,理論上5.0之後系統就能夠加載多個dex了,不知道爲何這個閾值設置的是6.0而不是5.0
==========================
原本想一氣呵成把這幾個月作的功能和優化全在這篇一併說完的,寫着寫着簡書提示字數快超限了,無奈只能分篇寫了,下一篇主要講免安裝模塊和idea插件的實現。快到中秋節了提早祝你們中秋快樂。未完待續,後會有期。。。。。。
若是你喜歡本文就來給咱們star吧
github.com/typ0520/fas…
加快apk的構建速度,如何把編譯時間從130秒降到17秒
加快apk的構建速度,如何把編譯時間從130秒降到17秒(二)
Instant Run
Tinker
Freeline
安卓App熱補丁動態修復技術介紹
Android應用程序資源的編譯和打包過程分析
關鍵字:
加快apk編譯速度
加快app編譯速度
加快android編譯速度
加快android studio 編譯速度
android 加快編譯速度
android studio編譯慢
android studio編譯速度優化
android studio gradle 編譯慢
本文出自typ0520的簡書博客www.jianshu.com/p/53923d8f2…