(基于android12分析和測試)
一、現存問題
Android早期是aot的方式先編譯成機器碼,然后再運行的,這樣會導致安裝時間變長,后面的版本改成jit編譯方式休建,在運行時編譯乍恐,這樣會導致運行速度較慢。android7.0之后的版本支持jit+oat的方式編譯测砂,支持app配置baseline-profile的方式設置熱點代碼茵烈。
二、理論基礎
1砌些、ART 執(zhí)行方式
- 最初安裝應用時不進行任何 AOT 編譯呜投。應用前幾次運行時,系統(tǒng)會對其進行解譯存璃,并對經常執(zhí)行的方法進行 JIT 編譯仑荐。
- 當設備閑置和充電時,編譯守護程序會運行纵东,以便根據在應用前幾次運行期間生成的配置文件對常用代碼進行 AOT 編譯粘招。
-
下一次重新啟動應用時將會使用配置文件引導型代碼,并避免在運行時對已經過編譯的方法進行 JIT 編譯偎球。在應用后續(xù)運行期間經過 JIT 編譯的方法將會添加到配置文件中洒扎,然后編譯守護程序將會對這些方法進行 AOT 編譯。
2衰絮、編譯選項
dex2oat工具可以將dex文件生成vdex袍冷,odex或者.art文件。其中:
.dex
.java->.class->.dex (apk解壓后的問題)
java被編譯成.class之后猫牡,使用d8工具(以前是dx)將class文件合成dex文件胡诗,dex一般是jar的50%大小。然后被打包成單個.apk
文件。.dex
文件可以通過自動轉換用 Java 編程語言編寫的編譯應用程序來創(chuàng)建乃戈。.odex一種文件格式
.java->.class->.dex->.oat
過 AOT 編譯的方法代碼,ART可以直接用的機器碼褂痰。
- .vdex
dex->.vdex
對dex文件進行初步優(yōu)化,調用dexOpt方法症虑,轉成vdex文件(文件名后綴依然是.dex)缩歪,只是小小的優(yōu)化了操作碼,
其中odex的文件是可以直接被運行的谍憔。生成那種類型的文件依賴dex2Oat工具匪蝙,dex2Oat依賴一個核心參數“編譯過濾器”
編譯過濾器:(android 官方sdk android8.0之后沒有再更新)
-
verify
:僅運行 DEX 代碼驗證。 -
quicken
:(從 Android 12 開始已移除)運行 DEX 代碼驗證习贫,并優(yōu)化一些 DEX 指令逛球,以獲得更好的解譯器性能。(我在12還是看到有這樣配置) -
speed
:運行 DEX 代碼驗證苫昌,并對所有方法進行 AOT 編譯颤绕。 -
speed-profile
:運行 DEX 代碼驗證,并對配置文件中列出的方法進行 AOT 編譯祟身。
實際上最新的android 12代碼中增加了一些奥务。
//android_12/art/libartbase/base/compiler_filter.h
enum Filter {
kAssumeVerified, // Skip verification but mark all classes as verified anyway.
kExtract, // Delay verication to runtime, do not compile anything.
kVerify, // Only verify classes.
kSpaceProfile, // Maximize space savings based on profile.
kSpace, // Maximize space savings.
kSpeedProfile, // Maximize runtime performance based on profile.
kSpeed, // Maximize runtime performance.
kEverythingProfile, // Compile everything capable of being compiled based on profile.
kEverything, // Compile everything capable of being compiled.
};
//android_12/art/libartbase/base/compiler_filter.cc
std::string CompilerFilter::NameOfFilter(Filter filter) {
switch (filter) {
case CompilerFilter::kAssumeVerified: return "assume-verified";
case CompilerFilter::kExtract: return "extract";
case CompilerFilter::kVerify: return "verify";
case CompilerFilter::kSpaceProfile: return "space-profile";
case CompilerFilter::kSpace: return "space";
case CompilerFilter::kSpeedProfile: return "speed-profile";
case CompilerFilter::kSpeed: return "speed";
case CompilerFilter::kEverythingProfile: return "everything-profile";
case CompilerFilter::kEverything: return "everything";
}
UNREACHABLE();
}
從這里可以得知,quicken被移除了袜硫,如果配置了重定向到verify 氯葬。配置speed可以將所有方法進行oat,從而加速代碼的運行速度婉陷。配置speed-profile帚称,可以選擇性的讓配置的方法進行AOT編譯。
3秽澳、強制編譯命令
系統(tǒng)支持用命令執(zhí)行odex編譯
命令:
adb shell cmd package compile
基于配置文件:
adb shell cmd package compile -m speed-profile -f my-package
全面編譯:
adb shell cmd package compile -m speed -f my-package
代碼執(zhí)行流程 :
命令執(zhí)行后會生成odex和vdex文件闯睹,放置在data/app/package/oat/arm[64]/xxx
重啟進程后可以使用命令查:
/proc/pid/maps/ |grep "odex"
這樣就看到應用加載了odex加載到了內存。
4担神、手動執(zhí)行dex2Oat
android系統(tǒng)是自帶dex2oat工具的瞻坝,直接在平臺執(zhí)行dex2oat 命令可以直接生成對應的文件,默認位speed編譯
dex2oat --dex-file=a.dex --oat-file=./oat/arm64/base.odex
不過我企圖生成odex去覆蓋原來的odex失敗了杏瞻。簡單看了ART執(zhí)行的邏輯所刀,應該是校驗不通過導致。(沒有仔細研究捞挥,只是簡單看看代碼+推測)浮创。
后面發(fā)現如果用PackageManagerService(pkms)去生成一個odex就可以用。于是我加了點log查看一樣的命令參數砌函,使用pkms同款編譯參數后就可以用了斩披。放上研究了一天的命令參數
dex2oat32 --dex-file=./data/app/xxx/xxx.dex --oat-file=/data/app/xxx/oat/arm/xxx.odex --classpath-dir=/data/app/xxx --class-loader-context=PCL[]{PCL[/system/framework/org.apache.http.legacy.jar]} --instruction-set=arm --instruction-set-features=default --instruction-set-variant=cortex-a7 --compiler-filter=speed --compilation-reason=cmdline --max-image-block-size=524288 --resolve-startup-const-strings=true --generate-mini-debug-info --runtime-arg -Xdeny-art-apex-data-files --runtime-arg -Xtarget-sdk-version:31 --runtime-arg -Xhidden-api-policy:enabled -j4 --runtime-arg -Xms64m --runtime-arg -Xmx512m --compile-individually
三溜族、PKMS 代碼執(zhí)行邏輯
1、pkms執(zhí)行邏輯
int performDexOpt(AndroidPackage pkg, @NonNull PackageSetting pkgSetting,
String[] instructionSets, CompilerStats.PackageStats packageStats,
PackageDexUsage.PackageUseInfo packageUseInfo, DexoptOptions options) {
if (PLATFORM_PACKAGE_NAME.equals(pkg.getPackageName())) {
throw new IllegalArgumentException("System server dexopting should be done via "
+ " DexManager and PackageDexOptimizer#dexoptSystemServerPath");
}
if (pkg.getUid() == -1) {
throw new IllegalArgumentException("Dexopt for " + pkg.getPackageName()
+ " has invalid uid.");
}
if (!canOptimizePackage(pkg)) {//過濾不允許oat的
return DEX_OPT_SKIPPED;
}
synchronized (mInstallLock) {
final long acquireTime = acquireWakeLockLI(pkg.getUid());
try {
return performDexOptLI(pkg, pkgSetting, instructionSets,
packageStats, packageUseInfo, options);
} finally {
releaseWakeLockLI(acquireTime);
}
}
}
// 進入performDexOptLI(pkg, pkgSetting, instructionSets, packageStats, packageUseInfo, options)
private int performDexOptLI(AndroidPackage pkg, @NonNull PackageSetting pkgSetting,
String[] targetInstructionSets, CompilerStats.PackageStats packageStats,
PackageDexUsage.PackageUseInfo packageUseInfo, DexoptOptions options) {
...
int result = DEX_OPT_SKIPPED;
for (int i = 0; i < paths.size(); i++) {
// Skip paths that have no code.
if (!pathsWithCode[i]) {
continue;
}
if (classLoaderContexts[i] == null) {
throw new IllegalStateException("Inconsistent information in the "
+ "package structure. A split is marked to contain code "
+ "but has no dependency listed. Index=" + i + " path=" + paths.get(i));
}
// Append shared libraries with split dependencies for this split.
String path = paths.get(i);
if (options.getSplitName() != null) {
// We are asked to compile only a specific split. Check that the current path is
// what we are looking for.
if (!options.getSplitName().equals(new File(path).getName())) {
continue;
}
}
String profileName = ArtManager.getProfileName(
i == 0 ? null : pkg.getSplitNames()[i - 1]); //找profile文件
...
final String compilerFilter = getRealCompilerFilter(pkg,
options.getCompilerFilter(), isUsedByOtherApps);//對一些過濾編譯器做調整
...
for (String dexCodeIsa : dexCodeInstructionSets) {
int newResult = dexOptPath(pkg, pkgSetting, path, dexCodeIsa, compilerFilter,
profileAnalysisResult, classLoaderContexts[i], dexoptFlags, sharedGid,
packageStats, options.isDowngrade(), profileName, dexMetadataPath,
options.getCompilationReason());
...
return result;
}
//進入dexOptPath
private int dexOptPath(AndroidPackage pkg, @NonNull PackageSetting pkgSetting, String path,
String isa, String compilerFilter, int profileAnalysisResult, String classLoaderContext,
int dexoptFlags, int uid, CompilerStats.PackageStats packageStats, boolean downgrade,
String profileName, String dexMetadataPath, int compilationReason) {
...
String oatDir = getPackageOatDirIfSupported(pkg,
pkgSetting.getPkgState().isUpdatedSystemApp());
...
mInstaller.dexopt(path, uid, pkg.getPackageName(), isa, dexoptNeeded, oatDir,
dexoptFlags, compilerFilter, pkg.getVolumeUuid(), classLoaderContext,
seInfo, false /* downgrade*/, pkg.getTargetSdkVersion(),
profileName, dexMetadataPath,
getAugmentedReasonName(compilationReason, dexMetadataPath != null)); //調用Installer 執(zhí)行dexOpt過程
...
return DEX_OPT_PERFORMED;
} catch (InstallerException e) {
Slog.w(TAG, "Failed to dexopt", e);
return DEX_OPT_FAILED;
}
}
調用performDexOpt需要傳入以下參數
performDexOpt(AndroidPackage pkg, @NonNull PackageSetting pkgSetting,
String[] instructionSets,
CompilerStats.PackageStats packageStats,
PackageDexUsage.PackageUseInfo packageUseInfo,
DexoptOptions options)
其中DexoptOptions 定義了編譯的可選項垦沉,其構造方法如下煌抒,有個比較重要的屬性compilationReason,構造方法2也是通過reason獲取compilerFilter的厕倍。
//構造1
public DexoptOptions(String packageName, String compilerFilter, int flags) {
this(packageName, /*compilationReason*/ -1, compilerFilter, /*splitName*/ null, flags);
}
//構造2
public DexoptOptions(String packageName, int compilationReason, int flags) {
this(packageName, compilationReason, getCompilerFilterForReason(compilationReason),
/*splitName*/ null, flags);
}
//構造3
public DexoptOptions(String packageName, int compilationReason, String compilerFilter,
String splitName, int flags) {
int validityMask =
DEXOPT_CHECK_FOR_PROFILES_UPDATES |
DEXOPT_FORCE |
DEXOPT_BOOT_COMPLETE |
DEXOPT_ONLY_SECONDARY_DEX |
DEXOPT_ONLY_SHARED_DEX |
DEXOPT_DOWNGRADE |
DEXOPT_AS_SHARED_LIBRARY |
DEXOPT_IDLE_BACKGROUND_JOB |
DEXOPT_INSTALL_WITH_DEX_METADATA_FILE |
DEXOPT_FOR_RESTORE;
if ((flags & (~validityMask)) != 0) {
throw new IllegalArgumentException("Invalid flags : " + Integer.toHexString(flags));
}
mPackageName = packageName;
mCompilerFilter = compilerFilter;
mFlags = flags;
mSplitName = splitName;
mCompilationReason = compilationReason;
}
通過reason獲取compilerFilter的寡壮。也就是從這個通過prop定義的值完成設置編譯過濾器。
public static String getCompilerFilterForReason(int reason) {
return getAndCheckValidity(reason);
}
/ Load the property for the given reason and check for validity. This will throw an
// exception in case the reason or value are invalid.
private static String getAndCheckValidity(int reason) {
String sysPropValue = SystemProperties.get(getSystemPropertyName(reason));
if (sysPropValue == null || sysPropValue.isEmpty()
|| !(sysPropValue.equals(DexoptOptions.COMPILER_FILTER_NOOP)
|| DexFile.isValidCompilerFilter(sysPropValue))) {
throw new IllegalStateException("Value \"" + sysPropValue +"\" not valid "
+ "(reason " + REASON_STRINGS[reason] + ")");
} else if (!isFilterAllowedForReason(reason, sysPropValue)) {
throw new IllegalStateException("Value \"" + sysPropValue +"\" not allowed "
+ "(reason " + REASON_STRINGS[reason] + ")");
}
return sysPropValue;
}
PKMS 中定義了14中編譯原因讹弯,對應了每種觸發(fā)dexOpt的原因况既,和執(zhí)行dexOpt定義的編譯過濾器。
// Compilation reasons.
public static final int REASON_FIRST_BOOT = 0;
public static final int REASON_BOOT_AFTER_OTA = 1;
public static final int REASON_POST_BOOT = 2;
public static final int REASON_INSTALL = 3;
public static final int REASON_INSTALL_FAST = 4;
public static final int REASON_INSTALL_BULK = 5;
public static final int REASON_INSTALL_BULK_SECONDARY = 6;
public static final int REASON_INSTALL_BULK_DOWNGRADED = 7;
public static final int REASON_INSTALL_BULK_SECONDARY_DOWNGRADED = 8;
public static final int REASON_BACKGROUND_DEXOPT = 9;
public static final int REASON_AB_OTA = 10;
public static final int REASON_INACTIVE_PACKAGE_DOWNGRADE = 11;
public static final int REASON_CMDLINE = 12;
public static final int REASON_SHARED = 13;
2组民、BackgroundDexOptService
BackgroundDexOptService 是一個JobService棒仍,用于定期執(zhí)行任務。在SystemServer#startOtherService()方法中啟動臭胜,啟動后執(zhí)行兩個任務莫其。
任務一:監(jiān)聽開機廣播,在開機10分鐘-60分鐘內完成post-boot的dex優(yōu)化
任務二:每隔一天執(zhí)行一次idle 場景下dex優(yōu)化耸三。
//SystemServer.java
private void startOtherServices(@NonNull TimingsTraceAndSlog t) {
...
t.traceBegin("StartBackgroundDexOptService");
try {
BackgroundDexOptService.schedule(context);
} catch (Throwable e) {
reportWtf("starting StartBackgroundDexOptService", e);
}
t.traceEnd();
...
}
// BackgroundDexOptService.java
public static void schedule(Context context) {
if (isBackgroundDexoptDisabled()) {//讀屬性"pm.dexopt.disable_bg_dexopt" ,目前是false
return;
}
final JobScheduler js = context.getSystemService(JobScheduler.class);
// Schedule a one-off job which scans installed packages and updates
// out-of-date oat files. Schedule it 10 minutes after the boot complete event,
// so that we don't overload the boot with additional dex2oat compilations.
context.registerReceiver(new BroadcastReceiver() {
@Override
public void onReceive(Context context, Intent intent) {
js.schedule(new JobInfo.Builder(JOB_POST_BOOT_UPDATE, sDexoptServiceName)//BackgroundDexOptService 的第一個任務
.setMinimumLatency(TimeUnit.MINUTES.toMillis(10)) //最短執(zhí)行時間10min
.setOverrideDeadline(TimeUnit.MINUTES.toMillis(60)) //最遲執(zhí)行時間:60mins
.build());
context.unregisterReceiver(this);
if (DEBUG) {
Slog.i(TAG, "BootBgDexopt scheduled");
}
}
}, new IntentFilter(Intent.ACTION_BOOT_COMPLETED));//監(jiān)聽開機啟動廣播
// Schedule a daily job which scans installed packages and compiles
// those with fresh profiling data.
js.schedule(new JobInfo.Builder(JOB_IDLE_OPTIMIZE, sDexoptServiceName) //BackgroundDexOptService 的第二個任務
.setRequiresDeviceIdle(true) //設備在idle狀態(tài)
.setRequiresCharging(true) //充電中
.setPeriodic(IDLE_OPTIMIZATION_PERIOD) //執(zhí)行周期 一天
.build());
if (DEBUG) {
Slog.d(TAG, "BgDexopt scheduled");
}
}
//關注第一個job:到點執(zhí)行進入onStartJob方法
public boolean onStartJob(JobParameters params) {
if (DEBUG) {
Slog.i(TAG, "onStartJob");
}
// NOTE: PackageManagerService.isStorageLow uses a different set of criteria from
// the checks above. This check is not "live" - the value is determined by a background
// restart with a period of ~1 minute.
PackageManagerService pm = (PackageManagerService)ServiceManager.getService("package");
if (pm.isStorageLow()) {//Environment.getDataDirectory().getUsableSpace() < getMemoryLowThreshold(); 為data空間的20%或者低于500MB
Slog.i(TAG, "Low storage, skipping this run");
return false;
}
final ArraySet<String> pkgs = pm.getOptimizablePackages();
if (pkgs.isEmpty()) {
Slog.i(TAG, "No packages to optimize");
return false;
}
mThermalStatusCutoff =
SystemProperties.getInt("dalvik.vm.dexopt.thermal-cutoff", THERMAL_CUTOFF_DEFAULT);//2
boolean result;
if (params.getJobId() == JOB_POST_BOOT_UPDATE) {
result = runPostBootUpdate(params, pm, pkgs);//進入這里
} else {
result = runIdleOptimization(params, pm, pkgs);
}
return result;
}
//執(zhí)行post-boot更新
private boolean runPostBootUpdate(final JobParameters jobParams,
final PackageManagerService pm, final ArraySet<String> pkgs) {
if (mExitPostBootUpdate.get()) {
// This job has already been superseded. Do not start it.
return false;
}
new Thread("BackgroundDexOptService_PostBootUpdate") {//新起一個線程執(zhí)行postBootUpdate
@Override
public void run() {
postBootUpdate(jobParams, pm, pkgs);
}
}.start();
return true;
}
//子線程運行執(zhí)行所有可優(yōu)化包的dex優(yōu)化
private void postBootUpdate(JobParameters jobParams, PackageManagerService pm,
ArraySet<String> pkgs) {
final BatteryManagerInternal batteryManagerInternal =
LocalServices.getService(BatteryManagerInternal.class);
final long lowThreshold = getLowStorageThreshold(this);
mAbortPostBootUpdate.set(false);
ArraySet<String> updatedPackages = new ArraySet<>();
for (String pkg : pkgs) {
if (mAbortPostBootUpdate.get()) {
// JobScheduler requested an early abort.
return;
}
if (mExitPostBootUpdate.get()) {
// Different job, which supersedes this one, is running.
break;
}
if (batteryManagerInternal.getBatteryLevelLow()) {//低電量不執(zhí)行
// Rather bail than completely drain the battery.
break;
}
long usableSpace = mDataDir.getUsableSpace();
if (usableSpace < lowThreshold) {//存儲不足
// Rather bail than completely fill up the disk.
Slog.w(TAG, "Aborting background dex opt job due to low storage: " +
usableSpace);
break;
}
if (DEBUG) {
Slog.i(TAG, "Updating package " + pkg);
}
// Update package if needed. Note that there can be no race between concurrent
// jobs because PackageDexOptimizer.performDexOpt is synchronized.
// checkProfiles is false to avoid merging profiles during boot which
// might interfere with background compilation (b/28612421).
// Unfortunately this will also means that "pm.dexopt.boot=speed-profile" will
// behave differently than "pm.dexopt.bg-dexopt=speed-profile" but that's a
// trade-off worth doing to save boot time work.
int result = pm.performDexOptWithStatus(new DexoptOptions(//進入PKMS
pkg,
PackageManagerService.REASON_POST_BOOT,//原因
DexoptOptions.DEXOPT_BOOT_COMPLETE));
if (result == PackageDexOptimizer.DEX_OPT_PERFORMED) {
updatedPackages.add(pkg);
}
}
notifyPinService(updatedPackages);
notifyPackagesUpdated(updatedPackages);
// Ran to completion, so we abandon our timeslice and do not reschedule.
jobFinished(jobParams, /* reschedule */ false);
}
進入PKMS 的performDexOptWithStatus方法乱陡。
/* package */ int performDexOptWithStatus(DexoptOptions options) {
return performDexOptTraced(options);
}
private int performDexOptTraced(DexoptOptions options) {
Trace.traceBegin(TRACE_TAG_PACKAGE_MANAGER, "dexopt");
try {
return performDexOptInternal(options);
} finally {
Trace.traceEnd(TRACE_TAG_PACKAGE_MANAGER);
}
}
private int performDexOptInternal(DexoptOptions options) {
AndroidPackage p;
PackageSetting pkgSetting;
synchronized (mLock) {
p = mPackages.get(options.getPackageName());
pkgSetting = mSettings.getPackageLPr(options.getPackageName());
if (p == null || pkgSetting == null) {
// Package could not be found. Report failure.
return PackageDexOptimizer.DEX_OPT_FAILED;
}
mPackageUsage.maybeWriteAsync(mSettings.getPackagesLocked());
mCompilerStats.maybeWriteAsync();
}
final long callingId = Binder.clearCallingIdentity();
try {
synchronized (mInstallLock) {
return performDexOptInternalWithDependenciesLI(p, pkgSetting, options); //進入這里
}
} finally {
Binder.restoreCallingIdentity(callingId);
}
}
private int performDexOptInternalWithDependenciesLI(AndroidPackage p,
@NonNull PackageSetting pkgSetting, DexoptOptions options) {
// System server gets a special path.
if (PLATFORM_PACKAGE_NAME.equals(p.getPackageName())) {
return mDexManager.dexoptSystemServer(options);//android 走系統(tǒng)
}
// Select the dex optimizer based on the force parameter.
// Note: The force option is rarely used (cmdline input for testing, mostly), so it's OK to
// allocate an object here.
PackageDexOptimizer pdo = options.isForce()
? new PackageDexOptimizer.ForcedUpdatePackageDexOptimizer(mPackageDexOptimizer)
: mPackageDexOptimizer; //如果有攜帶f就要強制編譯,無其他邏輯
// Dexopt all dependencies first. Note: we ignore the return value and march on
// on errors.
// Note that we are going to call performDexOpt on those libraries as many times as
// they are referenced in packages. When we do a batch of performDexOpt (for example
// at boot, or background job), the passed 'targetCompilerFilter' stays the same,
// and the first package that uses the library will dexopt it. The
// others will see that the compiled code for the library is up to date.
Collection<SharedLibraryInfo> deps = findSharedLibraries(pkgSetting);
final String[] instructionSets = getAppDexInstructionSets(
AndroidPackageUtils.getPrimaryCpuAbi(p, pkgSetting),
AndroidPackageUtils.getSecondaryCpuAbi(p, pkgSetting));
if (!deps.isEmpty()) {
DexoptOptions libraryOptions = new DexoptOptions(options.getPackageName(),
options.getCompilationReason(), options.getCompilerFilter(),
options.getSplitName(),
options.getFlags() | DexoptOptions.DEXOPT_AS_SHARED_LIBRARY);
for (SharedLibraryInfo info : deps) {
AndroidPackage depPackage = null;
PackageSetting depPackageSetting = null;
synchronized (mLock) {
depPackage = mPackages.get(info.getPackageName());
depPackageSetting = mSettings.getPackageLPr(info.getPackageName());
}
if (depPackage != null && depPackageSetting != null) {
// TODO: Analyze and investigate if we (should) profile libraries.
pdo.performDexOpt(depPackage, depPackageSetting, instructionSets, //先對依賴庫進行dex優(yōu)化
getOrCreateCompilerPackageStats(depPackage),
mDexManager.getPackageUseInfoOrDefault(depPackage.getPackageName()),
libraryOptions);
} else {
// TODO(ngeoffray): Support dexopting system shared libraries.
}
}
}
return pdo.performDexOpt(p, pkgSetting, instructionSets,//進入PackageDexOptimizer#performDexOpt流程吕晌,上面已經分析過蛋褥。
getOrCreateCompilerPackageStats(p),
mDexManager.getPackageUseInfoOrDefault(p.getPackageName()), options);
}