Flutter ios安裝包size的裁剪一直是個備受關注的主題,年前字節跳動分享了一篇文章(juejin.im/post/5de8a3…),提到了ios分離AOT編譯產物,把裏面的數據段和資源提取出來以減小安裝包size,但文章裏面並無展開介紹如何實現,這篇文章會很詳細的分析如何分離AOT編譯產物。並給出工具,方便沒編譯flutter engine經驗的同窗也能夠快速的實現這功能。ios
本文主要分析App.framework裏面的生成流程,以及如何分離AOT編譯產物,App.framework的構成以下圖所示。c++
主要有App動態庫二進制文件、flutter_assets還有Info.plist三部分構成,而App動態庫二進制文件又由4部分構成,vm的數據段、代碼段和isolate的數據段、代碼段。其中flutter_assets、vm數據段、isolate數據段都是能夠不打包到ipa中,能夠從外部document中加載到,這就讓咱們有縮減ipa包的可能了。git
不少人確定會關心最終縮減的效果。咱們先給出一個真實線上項目,用官方編譯engine和用分離產物的engine生成的App.framework的對比圖。github
官方engine生成的App.framework構成以下,其中App動態庫二進制文件19.2M,flutter_assets有3.3M,共22.5M。shell
用分離產物的engine生成的App.framework構成以下,只剩App動態庫二進制文件14.8M。xcode
App.framework從22.5裁到14.8M,不一樣項目可能不同。緩存
每次xcode項目進行進行構建前都會運行xcode_backend.sh這個腳本進行flutter產物打包,咱們從xcode_backend.sh開始分析。從上文分析App.framework裏面總共有三個文件生成二進制文件App、資源文件flutter_assets目錄和Info.plist文件,這裏面咱們只關心二進制文件App和flutter_assets目錄是怎樣生成的。bash
分析xcode_backend.sh,咱們能夠發現生成App和flutter_assets的關鍵shell代碼以下服務器
# App動態庫二進制文件
RunCommand "${FLUTTER_ROOT}/bin/flutter" --suppress-analytics \
${verbose_flag} \
build aot \
--output-dir="${build_dir}/aot" \
--target-platform=ios \
--target="${target_path}" \
--${build_mode} \
--ios-arch="${archs}" \
${flutter_engine_flag} \
${local_engine_flag} \
${bitcode_flag}
.
.
.
RunCommand cp -r -- "${app_framework}" "${derived_dir}"
# 生成flutter_assets
RunCommand "${FLUTTER_ROOT}/bin/flutter" \
${verbose_flag} \
build bundle \
--target-platform=ios \
--target="${target_path}" \
--${build_mode} \
--depfile="${build_dir}/snapshot_blob.bin.d" \
--asset-dir="${derived_dir}/App.framework/${assets_path}" \
${precompilation_flag} \
${flutter_engine_flag} \
${local_engine_flag} \
${track_widget_creation_flag}
複製代碼
從上面的代碼能夠看到這裏調用了的遠行了 {FLUTTER_ROOT}/bin/flutter 裏面提到真正運行代碼的是架構
...
FLUTTER_TOOLS_DIR="$FLUTTER_ROOT/packages/flutter_tools"
SNAPSHOT_PATH="$FLUTTER_ROOT/bin/cache/flutter_tools.snapshot"
STAMP_PATH="$FLUTTER_ROOT/bin/cache/flutter_tools.stamp"
SCRIPT_PATH="$FLUTTER_TOOLS_DIR/bin/flutter_tools.dart"
DART_SDK_PATH="$FLUTTER_ROOT/bin/cache/dart-sdk"
DART="$DART_SDK_PATH/bin/dart"
PUB="$DART_SDK_PATH/bin/pub"
//真正的執行邏輯
"$DART" $FLUTTER_TOOL_ARGS "$SNAPSHOT_PATH" "$@"
//等價於下面的命令
/bin/cache/dart-sdk/bin/dart $FLUTTER_TOOL_ARGS "bin/cache/flutter_tools.snapshot" "$@"
複製代碼
就是說經過dart命令運行flutter_tools.snapshot這個產物
flutter_tools.snapshot的入口是
[-> flutter/packages/flutter_tools/bin/flutter_tools.dart]
import 'package:flutter_tools/executable.dart' as executable;
void main(List<String> args) {
executable.main(args);
}
複製代碼
import 'runner.dart' as runner;
Future<void> main(List<String> args) async {
...
await runner.run(args, <FlutterCommand>[
AnalyzeCommand(verboseHelp: verboseHelp),
AttachCommand(verboseHelp: verboseHelp),
BuildCommand(verboseHelp: verboseHelp),
ChannelCommand(verboseHelp: verboseHelp),
CleanCommand(),
ConfigCommand(verboseHelp: verboseHelp),
CreateCommand(),
DaemonCommand(hidden: !verboseHelp),
DevicesCommand(),
DoctorCommand(verbose: verbose),
DriveCommand(),
EmulatorsCommand(),
FormatCommand(),
GenerateCommand(),
IdeConfigCommand(hidden: !verboseHelp),
InjectPluginsCommand(hidden: !verboseHelp),
InstallCommand(),
LogsCommand(),
MakeHostAppEditableCommand(),
PackagesCommand(),
PrecacheCommand(),
RunCommand(verboseHelp: verboseHelp),
ScreenshotCommand(),
ShellCompletionCommand(),
StopCommand(),
TestCommand(verboseHelp: verboseHelp),
TraceCommand(),
TrainingCommand(),
UpdatePackagesCommand(hidden: !verboseHelp),
UpgradeCommand(),
VersionCommand(),
], verbose: verbose,
muteCommandLogging: muteCommandLogging,
verboseHelp: verboseHelp,
overrides: <Type, Generator>{
CodeGenerator: () => const BuildRunner(),
});
}
複製代碼
通過一輪調用後,真正編譯產物的類在 GenSnapshot.run,調用棧gityuan.com/2019/09/07/…這篇文章有詳細介紹,這裏就不細說了
[-> lib/src/base/build.dart]
class GenSnapshot {
Future<int> run({
@required SnapshotType snapshotType,
IOSArch iosArch,
Iterable<String> additionalArgs = const <String>[],
}) {
final List<String> args = <String>[
'--causal_async_stacks',
]..addAll(additionalArgs);
//獲取gen_snapshot命令的路徑
final String snapshotterPath = getSnapshotterPath(snapshotType);
//iOS gen_snapshot是一個多體系結構二進制文件。 做爲i386二進制文件運行將生成armv7代碼。 做爲x86_64二進制文件運行將生成arm64代碼。
// /usr/bin/arch可用於運行具備指定體系結構的二進制文件
if (snapshotType.platform == TargetPlatform.ios) {
final String hostArch = iosArch == IOSArch.armv7 ? '-i386' : '-x86_64';
return runCommandAndStreamOutput(<String>['/usr/bin/arch', hostArch, snapshotterPath]..addAll(args));
}
return runCommandAndStreamOutput(<String>[snapshotterPath]..addAll(args));
}
}
複製代碼
GenSnapshot.run具體命令根據前面的封裝,最終等價於:
//這是針對iOS的genSnapshot命令
/usr/bin/arch -x86_64 flutter/bin/cache/artifacts/engine/ios-release/gen_snapshot
--causal_async_stacks
--deterministic
--snapshot_kind=app-aot-assembly
--assembly=build/aot/arm64/snapshot_assembly.S
build/aot/app.dill
複製代碼
此處gen_snapshot是一個二進制可執行文件,所對應的執行方法源碼爲third_party/dart/runtime/bin/gen_snapshot.cc 這個文件是flutter engine裏面文件,須要拉取engine的代碼才能修改,編譯flutter engine 能夠參考文章手把手教你編譯Flutter engine,下文咱們也會介紹編譯完flutter engine ,怎麼拿到gen_snapshot編譯後的二進制文件。
Flutter機器碼生成gen_snapshot這篇文章對gen_snapshot流程作了詳細的分析,這裏我直接給出最後結論,生成數據段和代碼段的代碼在 AssemblyImageWriter::WriteText這個函數裏面
[-> third_party/dart/runtime/vm/image_snapshot.cc]
void AssemblyImageWriter::WriteText(WriteStream* clustered_stream, bool vm) {
Zone* zone = Thread::Current()->zone();
//寫入頭部
const char* instructions_symbol = vm ? "_kDartVmSnapshotInstructions" : "_kDartIsolateSnapshotInstructions";
assembly_stream_.Print(".text\n");
assembly_stream_.Print(".globl %s\n", instructions_symbol);
assembly_stream_.Print(".balign %" Pd ", 0\n", VirtualMemory::PageSize());
assembly_stream_.Print("%s:\n", instructions_symbol);
//寫入頭部空白字符,使得指令快照看起來像堆頁
intptr_t instructions_length = next_text_offset_;
WriteWordLiteralText(instructions_length);
intptr_t header_words = Image::kHeaderSize / sizeof(uword);
for (intptr_t i = 1; i < header_words; i++) {
WriteWordLiteralText(0);
}
//寫入序幕.cfi_xxx
FrameUnwindPrologue();
Object& owner = Object::Handle(zone);
String& str = String::Handle(zone);
ObjectStore* object_store = Isolate::Current()->object_store();
TypeTestingStubNamer tts;
intptr_t text_offset = 0;
for (intptr_t i = 0; i < instructions_.length(); i++) {
auto& data = instructions_[i];
const bool is_trampoline = data.trampoline_bytes != nullptr;
if (is_trampoline) { //針對跳牀函數
const auto start = reinterpret_cast<uword>(data.trampoline_bytes);
const auto end = start + data.trampline_length;
//寫入.quad xxx字符串
text_offset += WriteByteSequence(start, end);
delete[] data.trampoline_bytes;
data.trampoline_bytes = nullptr;
continue;
}
const intptr_t instr_start = text_offset;
const Instructions& insns = *data.insns_;
const Code& code = *data.code_;
// 1. 寫入 頭部到入口點
{
NoSafepointScope no_safepoint;
uword beginning = reinterpret_cast<uword>(insns.raw_ptr());
uword entry = beginning + Instructions::HeaderSize(); //ARM64 32位對齊
//指令的只讀標記
uword marked_tags = insns.raw_ptr()->tags_;
marked_tags = RawObject::OldBit::update(true, marked_tags);
marked_tags = RawObject::OldAndNotMarkedBit::update(false, marked_tags);
marked_tags = RawObject::OldAndNotRememberedBit::update(true, marked_tags);
marked_tags = RawObject::NewBit::update(false, marked_tags);
//寫入標記
WriteWordLiteralText(marked_tags);
beginning += sizeof(uword);
text_offset += sizeof(uword);
text_offset += WriteByteSequence(beginning, entry);
}
// 2. 在入口點寫入標籤
owner = code.owner();
if (owner.IsNull()) {
// owner爲空,說明是一個常規的stub,其中stub列表定義在stub_code_list.h中的VM_STUB_CODE_LIST
const char* name = StubCode::NameOfStub(insns.EntryPoint());
if (name != nullptr) {
assembly_stream_.Print("Precompiled_Stub_%s:\n", name);
} else {
if (name == nullptr) {
// isolate專有的stub代碼[見小節3.5.1]
name = NameOfStubIsolateSpecificStub(object_store, code);
}
assembly_stream_.Print("Precompiled__%s:\n", name);
}
} else if (owner.IsClass()) {
//owner爲Class,說明是該類分配的stub,其中class列表定義在class_id.h中的CLASS_LIST_NO_OBJECT_NOR_STRING_NOR_ARRAY
str = Class::Cast(owner).Name();
const char* name = str.ToCString();
EnsureAssemblerIdentifier(const_cast<char*>(name));
assembly_stream_.Print("Precompiled_AllocationStub_%s_%" Pd ":\n", name,
i);
} else if (owner.IsAbstractType()) {
const char* name = tts.StubNameForType(AbstractType::Cast(owner));
assembly_stream_.Print("Precompiled_%s:\n", name);
} else if (owner.IsFunction()) { //owner爲Function,說明是一個常規的dart函數
const char* name = Function::Cast(owner).ToQualifiedCString();
EnsureAssemblerIdentifier(const_cast<char*>(name));
assembly_stream_.Print("Precompiled_%s_%" Pd ":\n", name, i);
} else {
UNREACHABLE();
}
#ifdef DART_PRECOMPILER
// 建立一個標籤用於DWARF
if (!code.IsNull()) {
const intptr_t dwarf_index = dwarf_->AddCode(code);
assembly_stream_.Print(".Lcode%" Pd ":\n", dwarf_index);
}
#endif
{
// 3. 寫入 入口點到結束
NoSafepointScope no_safepoint;
uword beginning = reinterpret_cast<uword>(insns.raw_ptr());
uword entry = beginning + Instructions::HeaderSize();
uword payload_size = insns.raw()->HeapSize() - insns.HeaderSize();
uword end = entry + payload_size;
text_offset += WriteByteSequence(entry, end);
}
}
FrameUnwindEpilogue();
#if defined(TARGET_OS_LINUX) || defined(TARGET_OS_ANDROID) || \ defined(TARGET_OS_FUCHSIA)
assembly_stream_.Print(".section .rodata\n");
#elif defined(TARGET_OS_MACOS) || defined(TARGET_OS_MACOS_IOS)
assembly_stream_.Print(".const\n");
#else
UNIMPLEMENTED();
#endif
//寫入數據段
const char* data_symbol = vm ? "_kDartVmSnapshotData" : "_kDartIsolateSnapshotData";
assembly_stream_.Print(".globl %s\n", data_symbol);
assembly_stream_.Print(".balign %" Pd ", 0\n",
OS::kMaxPreferredCodeAlignment);
assembly_stream_.Print("%s:\n", data_symbol);
uword buffer = reinterpret_cast<uword>(clustered_stream->buffer());
intptr_t length = clustered_stream->bytes_written();
WriteByteSequence(buffer, buffer + length);
}
複製代碼
這裏是生成的是snapshot_assembly.S,後面在dart代碼還將對這個文件加工成App動態庫文件,咱們會在下文介紹,咱們要作代碼段和數據段分離修改的就是這個c++函數,首先改掉代碼不寫進snapshot_assembly.S,在另外的地方把二進制數據保存起來。後面經過修改engine的加載流程從外部加載這二進制數據,便可達到分離代碼段和數據段的目的。下面咱們繼續分析生成完snapshot_assembly.S後,在哪裏生成App動態庫二進制文件。
生成完snapshot_assembly.S後,再加工關鍵代碼在**[-> lib/src/base/build.dart]**
/// Builds an iOS or macOS framework at [outputPath]/App.framework from the assembly
/// source at [assemblyPath].
Future<RunResult> _buildFramework({
@required DarwinArch appleArch,
@required bool isIOS,
@required String assemblyPath,
@required String outputPath,
@required bool bitcode,
@required bool quiet
}) async {
final String targetArch = getNameForDarwinArch(appleArch);
if (!quiet) {
printStatus('Building App.framework for $targetArch...');
}
final List<String> commonBuildOptions = <String>[
'-arch', targetArch,
if (isIOS)
'-miphoneos-version-min=8.0',
];
const String embedBitcodeArg = '-fembed-bitcode';
final String assemblyO = fs.path.join(outputPath, 'snapshot_assembly.o');
List<String> isysrootArgs;
if (isIOS) {
final String iPhoneSDKLocation = await xcode.sdkLocation(SdkType.iPhone);
if (iPhoneSDKLocation != null) {
isysrootArgs = <String>['-isysroot', iPhoneSDKLocation];
}
}
//生成snapshot_assembly.o二進制文件
final RunResult compileResult = await xcode.cc(<String>[
'-arch', targetArch,
if (isysrootArgs != null) ...isysrootArgs,
if (bitcode) embedBitcodeArg,
'-c',
assemblyPath,
'-o',
assemblyO,
]);
if (compileResult.exitCode != 0) {
printError('Failed to compile AOT snapshot. Compiler terminated with exit code ${compileResult.exitCode}');
return compileResult;
}
final String frameworkDir = fs.path.join(outputPath, 'App.framework');
fs.directory(frameworkDir).createSync(recursive: true);
final String appLib = fs.path.join(frameworkDir, 'App');
final List<String> linkArgs = <String>[
...commonBuildOptions,
'-dynamiclib',
'-Xlinker', '-rpath', '-Xlinker', '@executable_path/Frameworks',
'-Xlinker', '-rpath', '-Xlinker', '@loader_path/Frameworks',
'-install_name', '@rpath/App.framework/App',
if (bitcode) embedBitcodeArg,
if (isysrootArgs != null) ...isysrootArgs,
'-o', appLib,
assemblyO,
];
//打包成動態庫
final RunResult linkResult = await xcode.clang(linkArgs);
if (linkResult.exitCode != 0) {
printError('Failed to link AOT snapshot. Linker terminated with exit code ${compileResult.exitCode}');
}
return linkResult;
}
複製代碼
這裏最終會調用xcrun cc命令和xcrun clang命令打包動態庫二進制文件。
根據上面的分析整個流程涉及dart代碼和c++代碼,dart代碼其實不在engine,屬於flutter項目,只須要用打開**[-> packages/flutter_tools]這個flutter 項目,直接修改就好,要注意一點,flutter_tools的編譯產物是有緩存的,緩存路徑是[-> bin/cache/flutter_tools.snapshot]**,每次咱們修改完dart代碼,都須要刪掉flutter_tools.snapshot從新生成才能生效。
那c++部分代碼呢,首先設計c++代碼都是須要從新編譯flutter engine, 能夠參考文章手把手教你編譯Flutter engine,編譯後engine的產物,以下圖
把編譯後的gen_snapshot文件拷貝到flutter目錄下,下圖的位置便可。
注意,engine是分架構的,arm64的gen_snapshot名字是gen_snapshot_arm64,armv7的gen_snapshot名字是gen_snapshot_armv7,完成替換後,咱們定製的代碼就能夠生效了。
至此,生成動態庫文件App的所有流程都介紹清楚了,關鍵部分就是修改4.1.4提到的c++函數,咱們修改完後的編譯產物以下。
提取到了4個文件,分別是arm64和armv7架構下的vm數據段和isolate數據段,能夠按需下發給數據段文件給應用,從而實現flutter ios 動態庫編譯產物的裁剪。
像4.1.1和4.1.2說的那樣,具體生成flutter_assets的代碼在BundleBuilder.dart文件
[-> packages/flutter_tools/lib/src/bundle.dart]
Future<void> build({
@required TargetPlatform platform,
BuildMode buildMode,
String mainPath,
String manifestPath = defaultManifestPath,
String applicationKernelFilePath,
String depfilePath,
String privateKeyPath = defaultPrivateKeyPath,
String assetDirPath,
String packagesPath,
bool precompiledSnapshot = false,
bool reportLicensedPackages = false,
bool trackWidgetCreation = false,
List<String> extraFrontEndOptions = const <String>[],
List<String> extraGenSnapshotOptions = const <String>[],
List<String> fileSystemRoots,
String fileSystemScheme,
}) async {
mainPath ??= defaultMainPath;
depfilePath ??= defaultDepfilePath;
assetDirPath ??= getAssetBuildDirectory();
printStatus("assetDirPath" + assetDirPath);
printStatus("mainPath" + mainPath);
packagesPath ??= fs.path.absolute(PackageMap.globalPackagesPath);
final FlutterProject flutterProject = FlutterProject.current();
await buildWithAssemble(
buildMode: buildMode ?? BuildMode.debug,
targetPlatform: platform,
mainPath: mainPath,
flutterProject: flutterProject,
outputDir: assetDirPath,
depfilePath: depfilePath,
precompiled: precompiledSnapshot,
trackWidgetCreation: trackWidgetCreation,
);
// Work around for flutter_tester placing kernel artifacts in odd places.
if (applicationKernelFilePath != null) {
final File outputDill = fs.directory(assetDirPath).childFile('kernel_blob.bin');
if (outputDill.existsSync()) {
outputDill.copySync(applicationKernelFilePath);
}
}
return;
}
複製代碼
這裏assetDirPath就是最終打包產生bundle產物的路徑,咱們只要修改這個路徑,不指向App.framework,指向其餘路徑,就能夠避免打包進app。
至此,咱們已經把AOT編譯產物裏面的動態庫文件App、flutter_assets,的生成流程解析清楚了,也把如何分離的方法介紹了,對咱們的demo作完修改後的產物跟分離前的產物對好比下圖所示
分離前
分離後
那下面咱們分析如何修改flutter engine的加載流程,使engine再也不加載App.framework裏面的資源(由於已經分離出來),去加載外部給予的資源
上面咱們已經成功從App.framework裏面分離出了數據段數據已經flutter_assets,如今須要修改加載流程,加載外部數據。
加載數據段的堆棧以下。
能夠看到實際上是用::dlsym從動態庫裏面讀出數據段的數據強轉成const uint8_t使用,咱們只要修改代碼,不從動態庫讀取,外部提供一個const uint8_t來代替就行了
我最終選擇在下圖的兩個地方修改
這裏我直接構造一個SymbolMapping返回,SymbolMapping的定義以下
class SymbolMapping final : public Mapping {
public:
SymbolMapping(fml::RefPtr<fml::NativeLibrary> native_library,
const char* symbol_name);
//新增一個構造函數直接傳如外部數據
SymbolMapping(const uint8_t * data);
~SymbolMapping() override;
// |Mapping|
size_t GetSize() const override;
// |Mapping|
const uint8_t* GetMapping() const override;
private:
fml::RefPtr<fml::NativeLibrary> native_library_;
const uint8_t* mapping_ = nullptr;
FML_DISALLOW_COPY_AND_ASSIGN(SymbolMapping);
};
複製代碼
修改了這裏,咱們就能夠完成外部數據段的加載了。
這個比較簡單,咱們直接上代碼,
只要改了settings.assets_path,改爲外部的路徑就行了。
到這裏,咱們已經成功分離好engine了,分離以後對於不少混編的項目就是,flutter並非必須的,就能夠吧數據段部分和flutter_assets不打包進ipa,按需的下載下來,從而實現ipa的減size,下午會給出編好的engine、gen_snapshot文件和demo。固然,有些業務甚至不但願下載,想調用流程徹底不變,也能夠減size,這個因爲篇幅有限,咱們後面再寫一篇專門給出方法和工具。
從上面的分析能夠看出,搞這個事情,要不少鋪墊,很麻煩,不少同窗並不想摸索這麼久才能在本身的項目進行實驗,看效果,爲了方便你們驗證,我直接把基於v1.12.13+hotfix.7編好的engine、gen_snapshot文件和demo放到github上,讓你們直接用.編出來的Flutter.framework是全架構支持的、通過優化的release版,能夠直接上線的。下面介紹下運行流程。
在github上下載demo,不作任何改動,用真機直接運行,能夠看到產物以下圖所示,App動態庫 5.5M,flutter_assets 715k,總大小 6.3M。
而後執行下面的操做,替換engine
把github上的Flutter.framework覆蓋掉[->/bin/cache/artifacts/engine/ios-release/Flutter.framework]這個目下的Flutter.framework
把github上的gen_snapshot_arm64覆蓋掉[->/bin/cache/artifacts/engine/ios-release/gen_snapshot_arm64]
把github上的gen_snapshot_armv7覆蓋掉[->/bin/cache/artifacts/engine/ios-release/gen_snapshot_armv7]
而後把github上的bundle.dart覆蓋掉[->packages/flutter_tools/lib/src/bundle.dart]目錄下的bundle.dart文件
而後刪掉[->bin/cache/flutter_tools.snapshot],這個文件是dart項目生成的二進制文件,刪除了新的bundle.dart才能生效
而後從新跑起項目,觀察編譯產物
能夠看到產物以下圖所示,只剩下4.6M的產物了,這是demo的壓縮效果。
目前使用這方案,能夠分離編譯產物和flutter_assets,但也須要app作必定的改動,就是從服務器下載數據段和flutter_assets,才能運行flutter。固然還有一個方法,直接對數據段進行壓縮,運行的時候解壓,這個也是可行的,但壓縮率就沒這麼高,後面咱們也會開源並給出文章介紹。