【Java】Agent初探

Java Agent是什么

在官方文檔上的描述中文摘錄如下：

“提供允許 Java 代理檢測運行在JVM 上的程序的服務吐辙。檢測機制是對方法的字節(jié)碼進行修改晴弃。”

翻譯過來的意思就是通過JVM提供的鉤子（類加載）免绿，允許開發(fā)者對目標類字節(jié)碼修改浸剩。
給我們帶來的好處就是虛擬機級別的AOP，無侵入字節(jié)碼增強娄涩。

Java Agent怎么用

按照文檔的操作指示窗怒，使用Java Agent需要以下幾個步驟:

準備Agent.jar 。代理Jar文件的MANIFEST.MF必須包含 Premain-Class 屬性蓄拣。代理Jar必須實現(xiàn)premain 方法
public static void premain(String agentArgs, Instrumentation inst);

應用端使用命令行啟動JavaAgent
-javaagent:jarpath=[= options]

Agent端準備premain()：

package com.example.agent;
import java.lang.instrument.Instrumentation;
public class TinyAgent {
    public static void premain(String agentArgs, Instrumentation instrument) {
        Class[] allLoadedClasses = instrument.getAllLoadedClasses();
        System.out.println("loadedClasses length:" + allLoadedClasses.length);
        //instrumentation.addTransformer(new TinyTransformer(agentArgs));
        System.out.println("TinyAgent premain method");
    }
}

配置清單：MANIFEST.MF

    <build>
        <finalName>agent</finalName>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-jar-plugin</artifactId>
                <version>3.2.0</version>
                <configuration>
                    <archive>
                       <manifestEntries>
                           <Premain-Class>
                               com.example.agent.TinyAgent
                           </Premain-Class>
                       </manifestEntries>
                    </archive>
                </configuration>
            </plugin>
        </plugins>
    </build>

應用端main()準備扬虚，其中Hello.say()簡單的打印hello字符串

package com.example.MiniApm;
import com.example.Hello;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class MiniApmApplication {
    public static void main(String[] args) {
        SpringApplication.run(MiniApmApplication.class, args);
        Hello h = new Hello();
        h.say();
    }
}

命令行啟動:
-javaagent:E:\workspace\MiniApm\agent\target\agent.jar

輸出結果：

loadedClasses length:488
TinyAgent premain method

  .   ____          _            __ _ _
 /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
 \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
  '  |____| .__|_| |_|_| |_\__, | / / / /
 =========|_|==============|___/=/_/_/_/
 :: Spring Boot ::        (v2.2.6.RELEASE)
 ............................................
2020-05-05 22:52:36.624  INFO 10800 --- [  restartedMain] o.s.b.w.embedded.tomcat.TomcatWebServer  : Tomcat started on port(s): 8080 (http) with context path ''
2020-05-05 22:52:36.639  INFO 10800 --- [  restartedMain] com.example.MiniApm.MiniApmApplication   : Started MiniApmApplication in 2.64 seconds (JVM running for 3.152)
-------------hello

我們看到控制臺輸出兩行文字

loadedClasses length:488 (1)
TinyAgent premain method (2)

這代表我們的Java Agent機制生效。由于我們啟動的是Spring boot應用球恤，第一行加載的類才488個辜昵，從數(shù)量上看也不匹配，而且debug觀察之后根本沒有org.springframework的類咽斧。

問題：spring相關類哪去了堪置，Agent機制到底生效沒有？

帶著這個問題去OpenJdk上尋找答案张惹，需要提前下載好源碼

Java Agent底層原理

在分析OpenJdk源碼之前先看下premain()舀锨，顧名思義在main方法之前執(zhí)行。

public static void premain(String agentArgs, Instrumentation instrument)

接口Instrumentation是Jvm傳遞過來的宛逗，它的實現(xiàn)類IDE點進去就能看到:

package sun.instrument;
public class InstrumentationImpl implements Instrumentation {
    // jvm invoke premain()
    private void loadClassAndCallPremain(String var1, String var2)  {
        this.loadClassAndStartAgent(var1, "premain", var2);
    }
    static {
        // step 1: 加載動態(tài)鏈接庫instrument.dll
        System.loadLibrary("instrument");
    }
}

靜態(tài)代碼塊先執(zhí)行坎匿，系統(tǒng)在加載動態(tài)鏈接庫D:\Java\jdk1.8.0_202\jre\bin\instrument.dll

動態(tài)庫被加載完成之后會尋找Agent入口方法Agent_OnLoad()，截取部分代碼如下：openjdk\jdk\src\share\instrument\InvocationAdapter.c

/*
 *  看下面的注釋就能大概了解這個方法是程序的入口
 *  This will be called once for every -javaagent on the command line.
 *  Each call to Agent_OnLoad will create its own agent and agent data.
 *
 *  The argument tail string provided to Agent_OnLoad will be of form
 *  <jarfile>[=<options>]. The tail string is split into the jarfile and
 *  options components. The jarfile manifest is parsed and the value of the
 *  Premain-Class attribute will become the agent's premain class. The jar
 *  file is then added to the system class path, and if the Boot-Class-Path
 *  attribute is present then all relative URLs in the value are processed
 *  to create boot class path segments to append to the boot class path.
 */
JNIEXPORT jint JNICALL
Agent_OnLoad(JavaVM *vm, char *tail, void * reserved) {
    JPLISInitializationError initerror  = JPLIS_INIT_ERROR_NONE;
    jint                     result     = JNI_OK;
    JPLISAgent *             agent      = NULL;

    // 創(chuàng)建JPLISAgent結構體 在JPLISAgent.c中
    initerror = createNewJPLISAgent(vm, &agent);
    premainClass = getAttribute(attributes, "Premain-Class");
    /*
     * Add to the jarfile
     */
    appendClassPath(agent, jarfile);
    bootClassPath = getAttribute(attributes, "Boot-Class-Path");
        switch (initerror) {
    case JPLIS_INIT_ERROR_NONE:
      result = JNI_OK;
      break;
    case JPLIS_INIT_ERROR_CANNOT_CREATE_NATIVE_AGENT:
      result = JNI_ERR;
      fprintf(stderr, "java.lang.instrument/-javaagent: cannot create native agent.\n");
      break;
    case JPLIS_INIT_ERROR_FAILURE:
      result = JNI_ERR;
      fprintf(stderr, "java.lang.instrument/-javaagent: initialization of native agent failed.\n");
      break;
    ......
}

這里面能看到主要就是創(chuàng)建JPLISAgent結構體以及讀取MANIFEST.MF配置信息

struct _JPLISAgent {
    JavaVM *          mJVM;                   
    JPLISEnvironment  mNormalEnvironment;     
    JPLISEnvironment  mRetransformEnvironment;
    ...
    jobject           mInstrumentationImpl; //sun/instrument/InstrumentationImpl
    jmethodID         mPremainCaller;         // loadClassAndCallPremain
    jmethodID         mTransform;             // transform
    char const *      mAgentClassName;        /* agent class name */
    char const *      mOptionsString;         /* -javaagent options */
};

createNewJPLISAgent的內部核心邏輯就在initializeJPLISAgent拧额，開啟VMInit事件注冊回調函數(shù)eventHandlerVMInit

JPLISInitializationError
initializeJPLISAgent(   JPLISAgent *    agent,
                        JavaVM *        vm,
                        jvmtiEnv *      jvmtienv) {
    ......
    /* now turn on the VMInit event */
    if ( jvmtierror == JVMTI_ERROR_NONE ) {
        jvmtiEventCallbacks callbacks;
        memset(&callbacks, 0, sizeof(callbacks));
        // 注冊VMInit事件回調 在InvocationAdapter.c
        callbacks.VMInit = &eventHandlerVMInit;

        jvmtierror = (*jvmtienv)->SetEventCallbacks( jvmtienv,
                                                     &callbacks,
                                                     sizeof(callbacks));
        check_phase_ret_blob(jvmtierror, JPLIS_INIT_ERROR_FAILURE);
        jplis_assert(jvmtierror == JVMTI_ERROR_NONE);
    }
    ....
}

Jvm事件回調有很多本文相關的主要有兩個回調VMInit和ClassFileLoadHook碑诉，枚舉如下：

typedef struct {
                              /*   50 : VM Initialization Event */
    jvmtiEventVMInit VMInit;
                              /*   51 : VM Death Event */
    jvmtiEventVMDeath VMDeath;
    ....
                              /*   54 : Class File Load Hook */
    jvmtiEventClassFileLoadHook ClassFileLoadHook;
                              /*   55 : Class Load */
    jvmtiEventClassLoad ClassLoad;
                              /*   56 : Class Prepare */
    jvmtiEventClassPrepare ClassPrepare;
                              /*   57 : VM Start Event */
    jvmtiEventVMStart VMStart;
    
} jvmtiEventCallbacks;

注冊完VMInit事件注冊回調函數(shù)eventHandlerVMInit就可以等待調度執(zhí)行了

/*
 *  JVMTI callback support
 *
 *  We have two "stages" of callback support.
 *  At OnLoad time, we install a VMInit handler.
 *  When the VMInit handler runs, we remove the VMInit handler and install a
 *  ClassFileLoadHook handler.
 */

void JNICALL
eventHandlerVMInit( jvmtiEnv *      jvmtienv,
                    JNIEnv *        jnienv,
                    jthread         thread) {
    JPLISEnvironment * environment  = NULL;
    jboolean           success      = JNI_FALSE;

    environment = getJPLISEnvironment(jvmtienv);

    /* process the premain calls on the all the JPL agents */
    if ( environment != NULL ) {
        jthrowable outstandingException = preserveThrowable(jnienv);
        // java進程啟動 in JPLISAgent.c
        success = processJavaStart( environment->mAgent,
                                    jnienv);
        restoreThrowable(jnienv, outstandingException);
    }

    /* if we fail to start cleanly, bring down the JVM */
    if ( !success ) {
        abortJVM(jnienv, JPLIS_ERRORMESSAGE_CANNOTSTART);
    }
}

執(zhí)行函數(shù)體方法processJavaStart()：

創(chuàng)建實現(xiàn)類完善JPLISAgent剩余的field，之前初始化有些字段為null
開啟ClassFileLoadHook事件侥锦，注冊回調函數(shù)eventHandlerClassFileLoadHook
執(zhí)行sun.instrument.InstrumentationImpl.loadClassAndCallPremain()
等待ClassFileLoadHook事件回調執(zhí)行

/*
 * If this call fails, the JVM launch will ultimately be aborted,
 * so we don't have to be super-careful to clean up in partial failure
 * cases.
 */
jboolean
processJavaStart(   JPLISAgent *    agent,
                    JNIEnv *        jnienv) {
    jboolean    result;
    result = initializeFallbackError(jnienv);
    jplis_assert(result);

    /*
     *  Now make the InstrumentationImpl instance.
     */
    if ( result ) {
        // 創(chuàng)建實現(xiàn)類 完善JPLISAgent 剩余的field
        result = createInstrumentationImpl(jnienv, agent);
        jplis_assert(result);
    }

    /*
     *  Then turn off the VMInit handler and turn on the ClassFileLoadHook.
     *  This way it is on before anyone registers a transformer.
     */
    if ( result ) {
        // 開啟ClassFileLoadHook事件,注冊回調eventHandlerClassFileLoadHook
        result = setLivePhaseEventHandlers(agent);
        jplis_assert(result);
    }

    /*
     *  Load the Java agent, and call the premain.
     */
    if ( result ) {
        result = startJavaAgent(agent, jnienv,
                                agent->mAgentClassName, agent->mOptionsString,
                                agent->mPremainCaller);//loadClassAndCallPremain
    }

    /*
     * Finally surrender all of the tracking data that we don't need any more.
     * If something is wrong, skip it, we will be aborting the JVM anyway.
     */
    if ( result ) {
        deallocateCommandLineData(agent);
    }

    return result;
}

jvm事件調度開始執(zhí)行eventHandlerClassFileLoadHook：

// 在字節(jié)碼加載之前攔截 委托給初始化的sun/instrument/InstrumentationImpl.transform()
void JNICALL
eventHandlerClassFileLoadHook(  jvmtiEnv *              jvmtienv,
                                JNIEnv *                jnienv,
                                jclass                  class_being_redefined,
                                jobject                 loader,
                                const char*             name,
                                jobject                 protectionDomain,
                                jint                    class_data_len,
                                const unsigned char*    class_data,
                                jint*                   new_class_data_len,
                                unsigned char**         new_class_data) {
    JPLISEnvironment * environment  = NULL;

    environment = getJPLISEnvironment(jvmtienv);

    if ( environment != NULL ) {
        jthrowable outstandingException = preserveThrowable(jnienv);
        // 委托給java的transform() in JPLISAgent.c
        transformClassFile( environment->mAgent,
                            jnienv,
                            loader,
                            name,
                            class_being_redefined,
                            protectionDomain,
                            class_data_len,
                            class_data,
                            new_class_data_len,
                            new_class_data,
                            environment->mIsRetransformer);
        restoreThrowable(jnienv, outstandingException);
    }
}

委托java的transform()執(zhí)行命令

void
transformClassFile(             JPLISAgent *            agent,
                                JNIEnv *                jnienv,
                                jobject                 loaderObject,
                                const char*             name,
                                jclass                  classBeingRedefined,
                                jobject                 protectionDomain,
                                jint                    class_data_len,
                                const unsigned char*    class_data,
                                jint*                   new_class_data_len,
                                unsigned char**         new_class_data,
                                jboolean                is_retransformer) {
            // 開始調用transform()
            transformedBufferObject = (*jnienv)->CallObjectMethod(
                                                jnienv,
                                                agent->mInstrumentationImpl,
                                                agent->mTransform,// transform
                                                loaderObject,
                                                classNameStringObject,
                                                classBeingRedefined,
                                                protectionDomain,
                                                classFileBufferObject,
                                                is_retransformer);
            errorOutstanding = checkForAndClearThrowable(jnienv);
            jplis_assert_msg(!errorOutstanding, "transform method call failed");
    ...
}

看到這里进栽，上面的問題就有答案了
“spring相關類哪去了，Agent機制到底生效沒有恭垦？”

我們想要增強的相關類在InstrumentationImpl.transform()里面由jvm類加載事件鉤子傳遞進來快毛，下面來證實一下格嗅。

Java Agent Transform

最上面的例子把注釋打開，調用方法追加到TransformerManager中唠帝，所以上面的c代碼必然有從管理器中獲取transformer的邏輯屯掖，這里就不細說了。

package com.example.agent;
import java.lang.instrument.Instrumentation;
public class TinyAgent {
    public static void premain(String agentArgs, Instrumentation instrument) {
        Class[] allLoadedClasses = instrument.getAllLoadedClasses();
        System.out.println("loadedClasses length:" + allLoadedClasses.length);
        // 添加到TransformerManager
        instrumentation.addTransformer(new TinyTransformer(agentArgs));
        System.out.println("TinyAgent premain method");
    }
}

TinyTransformer這個類比較復雜打算后面單獨寫一篇文章細說襟衰，這里只是做個簡單的實現(xiàn)：

package com.example.agent;

import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.lang.instrument.ClassFileTransformer;
import java.lang.instrument.IllegalClassFormatException;
import java.net.URL;
import java.security.ProtectionDomain;

public class TinyTransformer implements ClassFileTransformer {
    private final String args;

    public TinyTransformer(String args) {
        this.args = args;
    }

    @Override
    public byte[] transform(ClassLoader loader, String className, Class<?> classBeingRedefined, ProtectionDomain protectionDomain, byte[] classfileBuffer) throws IllegalClassFormatException {
        if (className.startsWith("com/example/Hello")) {
            String hello = "com/example/Hello.class";
            String agent = "E:/workspace/MiniApm/agent/target/classes/com/example/Hello.class";
            URL resource = loader.getResource(hello);
            System.out.println("resource in apm   :" + resource);
            System.out.println("resource in agent :" + agent);

            try {
                File file = new File(agent);
                InputStream in = new FileInputStream(file);
                byte[] bytes = new byte[in.available()];
                in.read(bytes);
                in.close();
                return bytes;
            } catch (IOException e) {
                e.printStackTrace();
            }
        }

        return classfileBuffer;
    }
}

這個實現(xiàn)邏輯是簡單的把com/example/Hello.class重寫成新的代理類實現(xiàn)贴铜，之后返回新的class數(shù)組，實現(xiàn)代碼的增強瀑晒。

輸出結果如下：

before say()
-------------agent hello
after say()

原始結果：

-------------hello

小結：

因為實現(xiàn)的比較簡單绍坝，就是class文件的簡單替換。真實項目中這種替換是無法實施的苔悦，市面上有許多開源的字節(jié)碼增強工具轩褐，比如bytebuddy，具體下一篇文章準備寫一些實戰(zhàn)玖详。

增強方法timer
增強Caller/Runner
增強某些開源軟件
插件化命令行Options