java动态编译——JavaFileManager详解

前言

使用过java内存编译功能的小伙伴应该了解，我们可以通过tools包提供的JavaCompiler模块在内存中对java代码进行编译，而我们经常使用的javac编译工具，底层也是借助tools.jar完成编译的。在java代码里，我们可以通过下面几行代码，就完成源文件到字节码的编译：

        JavaCompiler javaCompiler = ToolProvider.getSystemJavaCompiler();

        StandardJavaFileManager fileManager = javaCompiler.getStandardFileManager(null, null, null);
        Iterable<? extends JavaFileObject> fileObjects = fileManager.getJavaFileObjects(sourceCodeFilePath);
        // 指定编译的class文件的路径
        List<String> options = Arrays.asList("-d", compilePath);
        JavaCompiler.CompilationTask cTask = javaCompiler.getTask(null, fileManager, null, options, null, fileObjects);
        Boolean success = cTask.call();

        if (success) {
            // 从source code中读出包名
            File sourceFile = new File(sourceCodeFilePath);
            BufferedReader bufferedReader = new BufferedReader(new FileReader(sourceFile));
            String line;
            while ((line = bufferedReader.readLine()) != null) {
                if (line.startsWith("package")) {
                    break;
                }
            }

            String packageName = line;
            packageName = packageName.replaceAll("package", "").trim().replaceAll("\\.", "/");
            packageName = packageName.substring(0, packageName.length() - 1);

            // 获取类名（这里简单处理, 默认文件名与类名相同）
            String sourceFileName = sourceFile.getName();

            String[] split = sourceFileName.split("\\.");
            String simpleClassName = split[0] + "." + "class";

            return compilePath + "/" +  packageName + "/" + simpleClassName;
        }

编译完成的class文件。会被放在-d选项指定的目录下（保留包名），我们可以通过读取文件中保存的字节码做类的动态加载，动态增强等一系列看上去很酷的操作。但是通过javaCompiler的getStandardFileManager获取的默认JavaFileManager，可能无法满足我们的所有需求，比如我们想完全指定编译结果的输出， 比如不生成文件而是直接保存在内存字节数组中直接使用，而当编译出现问题时，往往都和使用的JavaFileManager有关，这时候排查问题就需要我们对其了解，而实际上网络上相关的有价值内容甚少，所以才有这篇博客。

从问题开始

对于JavacFileManager的探究起源于开发远端热部署功能遇到的问题，最开始的设计思路是在在本地对修改后的java文件进行编译，然后将编译好的class文件上传至服务器，通过agent机制加载更新后的字节码进行redefine，但是其它步骤都通过验证后，最终卡在了编译这一步，我发现用前言中贴出的那段代码，对毫无依赖的简单java文件（实际上最初测试内存编译时使用的就是最常见的HelloWorld）可以编译成功，但是一旦有依赖关系，编译就会失败，并提示:

line: 5 , message: 找不到符号
  符号:   类 xxx
  位置: 程序包 xxx.xxx.xxx
...  

有点眼熟，我们使用javac命令进行编译时也会碰到一样的提示，因为实际上底层都是通过tools包进行的编译，而javac命令提供了-classpath 选项在依赖找不到时手动指定classpath，但是这个指定的classpath会覆盖默认的classpath，且在我的使用场景下，在外部获取需要热部署的程序的classpath并不是一件轻松的事情，而之前使用arthas的经验告诉我，带依赖的内存编译是可行的，并且似乎可以不自己指定classpath，那么就clone下arthas的源码，找寻答案

arthas的内存编译指令是mc，那我们从指令入口开始，首先定位到MemoryCompilerCommand指令类，接下来分析类的process指令处理方法：

    @Override
    public void process(final CommandProcess process) {
        int exitCode = 0;
        RowAffect affect = new RowAffect();

        try {
            Instrumentation inst = process.session().getInstrumentation();
            ClassLoader classloader = null;
            if (hashCode == null) {
                classloader = ClassLoader.getSystemClassLoader();
            } else {
                classloader = ClassLoaderUtils.getClassLoader(inst, hashCode);
                if (classloader == null) {
                    process.write("Can not find classloader with hashCode: " + hashCode + ".\n");
                    exitCode = -1;
                    return;
                }
            }

            DynamicCompiler dynamicCompiler = new DynamicCompiler(classloader, new Writer() {
                @Override
                public void write(char[] cbuf, int off, int len) throws IOException {
                    process.write(new String(cbuf, off, len));
                }

                @Override
                public void flush() throws IOException {
                }

                @Override
                public void close() throws IOException {

                }

            });

            Charset charset = Charset.defaultCharset();
            if (encoding != null) {
                charset = Charset.forName(encoding);
            }

            for (String sourceFile : sourcefiles) {
                String sourceCode = FileUtils.readFileToString(new File(sourceFile), charset);
                String name = new File(sourceFile).getName();
                if (name.endsWith(".java")) {
                    name = name.substring(0, name.length() - ".java".length());
                }
                dynamicCompiler.addSource(name, sourceCode);
            }

            Map<String, byte[]> byteCodes = dynamicCompiler.buildByteCodes();

            File outputDir = null;
            if (this.directory != null) {
                outputDir = new File(this.directory);
            } else {
                outputDir = new File("").getAbsoluteFile();
            }

            process.write("Memory compiler output:\n");
            for (Entry<String, byte[]> entry : byteCodes.entrySet()) {
                File byteCodeFile = new File(outputDir, entry.getKey().replace('.', '/') + ".class");
                FileUtils.writeByteArrayToFile(byteCodeFile, entry.getValue());
                process.write(byteCodeFile.getAbsolutePath() + '\n');
                affect.rCnt(1);
            }

        } catch (Throwable e) {
            logger.warn("Memory compiler error", e);
            process.write("Memory compiler error, exception message: " + e.getMessage()
                            + ", please check $HOME/logs/arthas/arthas.log for more details. \n");
            exitCode = -1;
        } finally {
            process.write(affect + "\n");
            process.end(exitCode);
        }
    }

arthas自己实现了一个DynamicCompile，并且看上去”只需要“将读取的源文件的内容通过addSource方法添加到DynamicCompile中再执行buildByteCodes方法，就可以得到对应的编译结果。那我们重点分析DynamicCompile类

private final JavaCompiler javaCompiler = ToolProvider.getSystemJavaCompiler();
    private final StandardJavaFileManager standardFileManager;
    private final List<String> options = new ArrayList<String>();
    private final DynamicClassLoader dynamicClassLoader;

    private final Collection<JavaFileObject> compilationUnits = new ArrayList<JavaFileObject>();
    private final List<Diagnostic<? extends JavaFileObject>> errors = new ArrayList<Diagnostic<? extends JavaFileObject>>();
    private final List<Diagnostic<? extends JavaFileObject>> warnings = new ArrayList<Diagnostic<? extends JavaFileObject>>();

    private Writer writer;

    public DynamicCompiler(ClassLoader classLoader) {
        this(classLoader, null);
    }

    public DynamicCompiler(ClassLoader classLoader, Writer writer) {
        standardFileManager = javaCompiler.getStandardFileManager(null, null, null);

        options.add("-Xlint:unchecked");
        dynamicClassLoader = new DynamicClassLoader(classLoader);
        this.writer = writer;
    }

    public void addSource(String className, String source) {
        addSource(new StringSource(className, source));
    }

    public void addSource(JavaFileObject javaFileObject) {
        compilationUnits.add(javaFileObject);
    }
    
    public Map<String, byte[]> buildByteCodes() {

        errors.clear();
        warnings.clear();

        JavaFileManager fileManager = new DynamicJavaFileManager(standardFileManager, dynamicClassLoader);

        DiagnosticCollector<JavaFileObject> collector = new DiagnosticCollector<JavaFileObject>();
        JavaCompiler.CompilationTask task = javaCompiler.getTask(null, fileManager, collector, options, null,
                        compilationUnits);

        try {

            if (!compilationUnits.isEmpty()) {
                boolean result = task.call();

                if (!result || collector.getDiagnostics().size() > 0) {

                    for (Diagnostic<? extends JavaFileObject> diagnostic : collector.getDiagnostics()) {
                        switch (diagnostic.getKind()) {
                        case NOTE:
                        case MANDATORY_WARNING:
                        case WARNING:
                            warnings.add(diagnostic);
                            break;
                        case OTHER:
                        case ERROR:
                        default:
                            errors.add(diagnostic);
                            break;
                        }

                    }

                    if (!errors.isEmpty()) {
                        throw new DynamicCompilerException("Compilation Error", errors);
                    }
                }
            }

            return dynamicClassLoader.getByteCodes();
        } catch (ClassFormatError e) {
            throw new DynamicCompilerException(e);
        } finally {
            compilationUnits.clear();

        }

    }

不出意外，DynamicCompile也是对tools包JavaCompile的封装，和我们之前的用法不同的是使用了自定义的DynamicJavaFileManager代替了我们使用的默认的JavacFileManager：

public class DynamicJavaFileManager extends ForwardingJavaFileManager<JavaFileManager> {
    private static final String[] superLocationNames = { StandardLocation.PLATFORM_CLASS_PATH.name(),
            /** JPMS StandardLocation.SYSTEM_MODULES **/
            "SYSTEM_MODULES" };
    private final PackageInternalsFinder finder;

    private final DynamicClassLoader classLoader;
    private final List<MemoryByteCode> byteCodes = new ArrayList<MemoryByteCode>();

    public DynamicJavaFileManager(JavaFileManager fileManager, DynamicClassLoader classLoader) {
        super(fileManager);
        this.classLoader = classLoader;

        finder = new PackageInternalsFinder(classLoader);
    }

    @Override
    public JavaFileObject getJavaFileForOutput(JavaFileManager.Location location, String className,
                    JavaFileObject.Kind kind, FileObject sibling) throws IOException {

        for (MemoryByteCode byteCode : byteCodes) {
            if (byteCode.getClassName().equals(className)) {
                return byteCode;
            }
        }

        MemoryByteCode innerClass = new MemoryByteCode(className);
        byteCodes.add(innerClass);
        classLoader.registerCompiledSource(innerClass);
        return innerClass;

    }

    @Override
    public ClassLoader getClassLoader(JavaFileManager.Location location) {
        return classLoader;
    }

    @Override
    public String inferBinaryName(Location location, JavaFileObject file) {
        if (file instanceof CustomJavaFileObject) {
            return ((CustomJavaFileObject) file).binaryName();
        } else {
            /**
             * if it's not CustomJavaFileObject, then it's coming from standard file manager
             * - let it handle the file
             */
            return super.inferBinaryName(location, file);
        }
    }

    @Override
    public Iterable<JavaFileObject> list(Location location, String packageName, Set<JavaFileObject.Kind> kinds,
                                         boolean recurse) throws IOException {
        if (location instanceof StandardLocation) {
            String locationName = ((StandardLocation) location).name();
            for (String name : superLocationNames) {
                if (name.equals(locationName)) {
                    return super.list(location, packageName, kinds, recurse);
                }
            }
        }

        // merge JavaFileObjects from specified ClassLoader
        if (location == StandardLocation.CLASS_PATH && kinds.contains(JavaFileObject.Kind.CLASS)) {
            return new IterableJoin<JavaFileObject>(super.list(location, packageName, kinds, recurse),
                    finder.find(packageName));
        }

        return super.list(location, packageName, kinds, recurse);
    }
    
    ...
}

DynamicJavaFileManager继承了ForwardingJavaFileManager，并重写了父类的getJavaFileForOutput，getClassLoader，inferBinaryName和list方法，难道是因为重写了这几个方法所以arthas内存编译可以成功而我却编译失败，想要找寻原因只有继续阅读源码。但是首先我们需要知道什么是JavaFileManager，查看这个接口的java doc:

* File manager for tools operating on Java™ programming language
 * source and class files.  In this context, <em>file</em> means an
 * abstraction of regular files and other sources of data.

直译过来，JavaFileManager就是tools包中使用的，管理java源文件和class类文件，抽象不同来源的这些数据的管理工具，所以我们之前找不到依赖的原因就是使用默认的JavaFileManager没有找到对应的依赖的源文件或class类文件，那么接下来探寻的重点就是JavaFileManager寻找依赖文件的过程了。下面的源码分析涉及到部分编译原理相关知识，搭配编译原理知识相互印证效果更佳哦。

编译任务的入口是CompilationTask的call方法，我们从入口开始追寻
com.sun.tools.javac.api.JavacTaskImpl

public Boolean call() {
  return this.doCall().isOK();
}

public Result doCall() {
  if (!this.used.getAndSet(true)) {
      this.initContext();
      this.notYetEntered = new HashMap();
      this.compilerMain.setAPIMode(true);
      this.result = this.compilerMain.compile(this.args, this.classNames, this.context, this.fileObjects, this.processors);
      this.cleanup();
      return this.result;
  } else {
      throw new IllegalStateException("multiple calls to method 'call'");
  }
}  

public void compile(List<JavaFileObject> var1, List<String> var2, Iterable<? extends Processor> var3) {
     if (var3 != null && var3.iterator().hasNext()) {
         this.explicitAnnotationProcessingRequested = true;
     }

     if (this.hasBeenUsed) {
         throw new AssertionError("attempt to reuse JavaCompiler");
     } else {
         this.hasBeenUsed = true;
         this.options.put(Option.XLINT_CUSTOM.text + "-" + LintCategory.OPTIONS.option, "true");
         this.options.remove(Option.XLINT_CUSTOM.text + LintCategory.OPTIONS.option);
         this.start_msec = now();

         try {
             this.initProcessAnnotations(var3);
             this.delegateCompiler = this.processAnnotations(this.enterTrees(this.stopIfError(CompileState.PARSE, this.parseFiles(var1))), var2);
             this.delegateCompiler.compile2();
             this.delegateCompiler.close();
             this.elapsed_msec = this.delegateCompiler.elapsed_msec;
         } catch (Abort var8) {
             if (this.devVerbose) {
                 var8.printStackTrace(System.err);
             }
         } finally {
             if (this.procEnvImpl != null) {
                 this.procEnvImpl.close();
             }
         }
     }
 }  

最终会去执行JavaCompiler的compile方法，在compile方法中，会对Java文件描述对象（JavaFileObject）转换为编译单元进行后续处理，具体转换逻辑在JavaCompile类中的parseFiles和parse方法

public List<JCCompilationUnit> parseFiles(Iterable<JavaFileObject> var1) {
   if (this.shouldStop(CompileState.PARSE)) {
       return List.nil();
   } else {
       ListBuffer var2 = new ListBuffer();
       HashSet var3 = new HashSet();
       Iterator var4 = var1.iterator();

       while(var4.hasNext()) {
           JavaFileObject var5 = (JavaFileObject)var4.next();
           if (!var3.contains(var5)) {
               var3.add(var5);
               var2.append(this.parse(var5));
           }
       }

       return var2.toList();
   }
}

public JCCompilationUnit parse(JavaFileObject var1) {
    JavaFileObject var2 = this.log.useSource(var1);

    JCCompilationUnit var4;
    try {
        JCCompilationUnit var3 = this.parse(var1, this.readSource(var1));
        if (var3.endPositions != null) {
            this.log.setEndPosTable(var1, var3.endPositions);
        }

        var4 = var3;
    } finally {
        this.log.useSource(var2);
    }

    return var4;
}    

转换逻辑设计到编译原理中的词法分析和语法分析等概念，十分复杂也不在这篇博客的探究范围我们不细说，我们来看看转换后的JCCompilationUnit

public static class JCCompilationUnit extends JCTree implements CompilationUnitTree {
   public List<JCTree.JCAnnotation> packageAnnotations;
   public JCTree.JCExpression pid;
   public List<JCTree> defs;
   public JavaFileObject sourcefile;
   public PackageSymbol packge;
   public ImportScope namedImportScope;
   public StarImportScope starImportScope;
   public LineMap lineMap = null;
   public DocCommentTable docComments = null;
   public EndPosTable endPositions = null;

   protected JCCompilationUnit(List<JCTree.JCAnnotation> var1, JCTree.JCExpression var2, List<JCTree> var3, JavaFileObject var4, PackageSymbol var5, ImportScope var6, StarImportScope var7) {
       this.packageAnnotations = var1;
       this.pid = var2;
       this.defs = var3;
       this.sourcefile = var4;
       this.packge = var5;
       this.namedImportScope = var6;
       this.starImportScope = var7;
   }

 ...
}  

其中我们的依赖信息被保存在了编译单元的defs属性中。

接下来就是调用enterTrees方法解析这棵结构已经被分析出来的语法树，我们忽略一些和这篇博客无关的调用栈直接定位到我们想要分析的方法入口
com.sun.tools.javac.comp.Enter#visitTopLevel

    public void visitTopLevel(JCCompilationUnit var1) {
        JavaFileObject var2 = this.log.useSource(var1.sourcefile);
        boolean var3 = false;
        boolean var4 = var1.sourcefile.isNameCompatible("package-info", Kind.SOURCE);
        if (var1.pid != null) {
            var1.packge = this.reader.enterPackage(TreeInfo.fullName(var1.pid));
            if (var1.packageAnnotations.nonEmpty() || this.pkginfoOpt == PkgInfo.ALWAYS || var1.docComments != null) {
                if (var4) {
                    var3 = true;
                } else if (var1.packageAnnotations.nonEmpty()) {
                    this.log.error(((JCAnnotation)var1.packageAnnotations.head).pos(), "pkg.annotations.sb.in.package-info.java", new Object[0]);
                }
            }
        } else {
            var1.packge = this.syms.unnamedPackage;
        }

        var1.packge.complete();
        Env var5 = this.topLevelEnv(var1);
        if (var4) {
            Env var6 = this.typeEnvs.get(var1.packge);
            if (var6 == null) {
                this.typeEnvs.put(var1.packge, var5);
            } else {
                JCCompilationUnit var7 = var6.toplevel;
                if (!this.fileManager.isSameFile(var1.sourcefile, var7.sourcefile)) {
                    this.log.warning(var1.pid != null ? var1.pid.pos() : null, "pkg-info.already.seen", new Object[]{var1.packge});
                    if (var3 || var7.packageAnnotations.isEmpty() && var1.docComments != null && var1.docComments.hasComment(var1)) {
                        this.typeEnvs.put(var1.packge, var5);
                    }
                }
            }

            for(Object var9 = var1.packge; var9 != null && ((Symbol)var9).kind == 1; var9 = ((Symbol)var9).owner) {
                ((Symbol)var9).flags_field |= 8388608L;
            }

            Name var10 = this.names.package_info;
            ClassSymbol var8 = this.reader.enterClass(var10, var1.packge);
            var8.flatname = this.names.fromString(var1.packge + "." + var10);
            var8.sourcefile = var1.sourcefile;
            var8.completer = null;
            var8.members_field = new Scope(var8);
            var1.packge.package_info = var8;
        }

        this.classEnter(var1.defs, var5);
        if (var3) {
            this.todo.append(var5);
        }

        this.log.useSource(var2);
        this.result = null;
    }

我们重点分析堆栈中的reader.enterPackage，package.complete和classEnter方法

com.sun.tools.javac.jvm.ClassReader#enterPackage(com.sun.tools.javac.util.Name)

public PackageSymbol enterPackage(Name var1) {
        PackageSymbol var2 = (PackageSymbol)this.packages.get(var1);
        if (var2 == null) {
            Assert.check(!var1.isEmpty(), "rootPackage missing!");
            var2 = new PackageSymbol(Convert.shortName(var1), this.enterPackage(Convert.packagePart(var1)));
            var2.completer = this.thisCompleter;
            this.packages.put(var1, var2);
        }

        return var2;
    }

enterPackage根据传入的包名构建了一个PackageSymbol对象，然后将ClassReader的Completer设置给了新构建的packageSymbol的completer属性。

private final Completer thisCompleter = new Completer() {
    public void complete(Symbol var1) throws CompletionFailure {
        ClassReader.this.complete(var1);
    }
};

构造完成后会在visitTopLevel调用packageSymbol的complete方法（实际是委托给ClassReader处理），进行包下类元信息的读取
com.sun.tools.javac.code.Symbol#complete

public void complete() throws Symbol.CompletionFailure {
    if (this.completer != null) {
        Symbol.Completer var1 = this.completer;
        this.completer = null;
        var1.complete(this);
    }
}

private void complete(Symbol var1) throws CompletionFailure {
    if (var1.kind == 2) {
        ClassSymbol var2 = (ClassSymbol)var1;
        var2.members_field = new ErrorScope(var2);
        this.annotate.enterStart();

        try {
            this.completeOwners(var2.owner);
            this.completeEnclosing(var2);
        } finally {
            this.annotate.enterDoneWithoutFlush();
        }

        this.fillIn(var2);
    } else if (var1.kind == 1) {
        PackageSymbol var8 = (PackageSymbol)var1;

        try {
            this.fillIn(var8);
        } catch (IOException var7) {
            throw (new CompletionFailure(var1, var7.getLocalizedMessage())).initCause(var7);
        }
    }

    if (!this.filling) {
        this.annotate.flush();
    }

}

通过fillIn方法进行类元信息的读取
com.sun.tools.javac.jvm.ClassReader#fillIn(com.sun.tools.javac.code.Symbol.PackageSymbol)

    private void fillIn(PackageSymbol var1) throws IOException {
        if (var1.members_field == null) {
            var1.members_field = new Scope(var1);
        }

        String var2 = var1.fullname.toString();
        EnumSet var3 = this.getPackageFileKinds();
        this.fillIn(var1, StandardLocation.PLATFORM_CLASS_PATH, this.fileManager.list(StandardLocation.PLATFORM_CLASS_PATH, var2, EnumSet.of(Kind.CLASS), false));
        EnumSet var4 = EnumSet.copyOf(var3);
        var4.remove(Kind.SOURCE);
        boolean var5 = !var4.isEmpty();
        EnumSet var6 = EnumSet.copyOf(var3);
        var6.remove(Kind.CLASS);
        boolean var7 = !var6.isEmpty();
        boolean var8 = this.fileManager.hasLocation(StandardLocation.SOURCE_PATH);
        if (this.verbose && this.verbosePath && this.fileManager instanceof StandardJavaFileManager) {
            StandardJavaFileManager var9 = (StandardJavaFileManager)this.fileManager;
            List var10;
            Iterator var11;
            File var12;
            if (var8 && var7) {
                var10 = List.nil();

                for(var11 = var9.getLocation(StandardLocation.SOURCE_PATH).iterator(); var11.hasNext(); var10 = var10.prepend(var12)) {
                    var12 = (File)var11.next();
                }

                this.log.printVerbose("sourcepath", new Object[]{var10.reverse().toString()});
            } else if (var7) {
                var10 = List.nil();

                for(var11 = var9.getLocation(StandardLocation.CLASS_PATH).iterator(); var11.hasNext(); var10 = var10.prepend(var12)) {
                    var12 = (File)var11.next();
                }

                this.log.printVerbose("sourcepath", new Object[]{var10.reverse().toString()});
            }

            if (var5) {
                var10 = List.nil();

                for(var11 = var9.getLocation(StandardLocation.PLATFORM_CLASS_PATH).iterator(); var11.hasNext(); var10 = var10.prepend(var12)) {
                    var12 = (File)var11.next();
                }

                for(var11 = var9.getLocation(StandardLocation.CLASS_PATH).iterator(); var11.hasNext(); var10 = var10.prepend(var12)) {
                    var12 = (File)var11.next();
                }

                this.log.printVerbose("classpath", new Object[]{var10.reverse().toString()});
            }
        }

        if (var7 && !var8) {
            this.fillIn(var1, StandardLocation.CLASS_PATH, this.fileManager.list(StandardLocation.CLASS_PATH, var2, var3, false));
        } else {
            if (var5) {
                this.fillIn(var1, StandardLocation.CLASS_PATH, this.fileManager.list(StandardLocation.CLASS_PATH, var2, var4, false));
            }

            if (var7) {
                this.fillIn(var1, StandardLocation.SOURCE_PATH, this.fileManager.list(StandardLocation.SOURCE_PATH, var2, var6, false));
            }
        }

        this.verbosePath = false;
    }

fillIn方法会在不同条件下指定不同的location通过fileManager去获取PackageSymbol对应包下类的classSymbol。而获取包下类元信息（JavaFileObject）这一步是委托给JavaFileManager的list方法做的。而此时ClassReader中持有的fileManager就是我们自定义的DynamicJavaFileManager。

com.taobao.arthas.compiler.DynamicJavaFileManager#list

@Override
    public Iterable<JavaFileObject> list(Location location, String packageName, Set<JavaFileObject.Kind> kinds,
                                         boolean recurse) throws IOException {
        if (location instanceof StandardLocation) {
            String locationName = ((StandardLocation) location).name();
            for (String name : superLocationNames) {
                if (name.equals(locationName)) {
                    return super.list(location, packageName, kinds, recurse);
                }
            }
        }

        // merge JavaFileObjects from specified ClassLoader
        if (location == StandardLocation.CLASS_PATH && kinds.contains(JavaFileObject.Kind.CLASS)) {
            return new IterableJoin<JavaFileObject>(super.list(location, packageName, kinds, recurse),
                    finder.find(packageName));
        }

        return super.list(location, packageName, kinds, recurse);
    }

DynamicJavaFileManager会将寻找JavaFileObject的任务委托给finder和父类ForwardingJavaFileManager（父类ForwardingJavaFileManager委托给JavacFileManager的list方法），finder会根据包名去对应的jar包或文件路径下寻找class文件，处理成JavaFileObject返回。

分析至此可以着手解决开篇的疑惑，为什么使用arthas-memorycompiler模块提供的DynamicFileManager在attach的java进程中可以编译成功，而在外部程序使用JavacFileManager编译会报can’t find class symbol，其实看一下JavacFileManager list方法的源码就水落石出了

public Iterable<JavaFileObject> list(Location var1, String var2, Set<Kind> var3, boolean var4) throws IOException {
    nullCheck(var2);
    nullCheck(var3);
    Iterable var5 = this.getLocation(var1);
    if (var5 == null) {
        return List.nil();
    } else {
        RelativeDirectory var6 = RelativeDirectory.forPackage(var2);
        ListBuffer var7 = new ListBuffer();
        Iterator var8 = var5.iterator();

        while(var8.hasNext()) {
            File var9 = (File)var8.next();
            this.listContainer(var9, var6, var3, var4, var7);
        }

        return var7.toList();
    }
}

private void listContainer(File var1, RelativeDirectory var2, Set<Kind> var3, boolean var4, ListBuffer<JavaFileObject> var5) {
    JavacFileManager.Archive var6 = (JavacFileManager.Archive)this.archives.get(var1);
    if (var6 == null) {
        if (this.fsInfo.isDirectory(var1)) {
            this.listDirectory(var1, var2, var3, var4, var5);
            return;
        }

        try {
            var6 = this.openArchive(var1);
        } catch (IOException var8) {
            this.log.error("error.reading.file", new Object[]{var1, getMessage(var8)});
            return;
        }
    }

    this.listArchive(var6, var2, var3, var4, var5);
}

list方法会根据Location类型选择Location对应的File集合，然后通过listContainer方法去这些file下去寻找属于传入的包名下的class集合，如果在所有的File下都找不到对应的class，编译就会失败，那么Location分别对应哪些File集合呢？
先看看Location有哪些类型

public enum StandardLocation implements Location {

    /**
     * Location of new class files.
     */
    CLASS_OUTPUT,

    /**
     * Location of new source files.
     */
    SOURCE_OUTPUT,

    /**
     * Location to search for user class files.
     */
    CLASS_PATH,

    /**
     * Location to search for existing source files.
     */
    SOURCE_PATH,

    /**
     * Location to search for annotation processors.
     */
    ANNOTATION_PROCESSOR_PATH,

    /**
     * Location to search for platform classes.  Sometimes called
     * the boot class path.
     */
    PLATFORM_CLASS_PATH,

    /**
     * Location of new native header files.
     * @since 1.8
     */
    NATIVE_HEADER_OUTPUT;

我们这篇博客只关心在ClassReader的fillIn方法中出现的CLASS_PATH，SOURCE_PATH和PLATFORM_CLASS_PATH三种类型的Location

JavacFileManager持有了一个Locations对象，这个Locations对象会在JavacFileManager对象创建时同步被创建，JavacFileManager通过调用这个对象的getLocation方法获取不同Location对应的File集合，不同类型的Location对应不同的LocationHandler，而具体寻找File集合的任务是委托给这一个个具体的Location去做

public class Locations {
    private Log log;
    private Options options;
    private Lint lint;
    private FSInfo fsInfo;
    private boolean warn;
    private boolean inited = false;
    Map<Location, Locations.LocationHandler> handlersForLocation;
    Map<Option, Locations.LocationHandler> handlersForOption;

 	Collection<File> getLocation(Location var1) {
        Locations.LocationHandler var2 = this.getHandler(var1);
        return var2 == null ? null : var2.getLocation();
    }
    
    void initHandlers() {
        this.handlersForLocation = new HashMap();
        this.handlersForOption = new EnumMap(Option.class);
        Locations.LocationHandler[] var1 = new Locations.LocationHandler[]{new Locations.BootClassPathLocationHandler(), new Locations.ClassPathLocationHandler(), new Locations.SimpleLocationHandler(StandardLocation.SOURCE_PATH, new Option[]{Option.SOURCEPATH}), new Locations.SimpleLocationHandler(StandardLocation.ANNOTATION_PROCESSOR_PATH, new Option[]{Option.PROCESSORPATH}), new Locations.OutputLocationHandler(StandardLocation.CLASS_OUTPUT, new Option[]{Option.D}), new Locations.OutputLocationHandler(StandardLocation.SOURCE_OUTPUT, new Option[]{Option.S}), new Locations.OutputLocationHandler(StandardLocation.NATIVE_HEADER_OUTPUT, new Option[]{Option.H})};
        Locations.LocationHandler[] var2 = var1;
        int var3 = var1.length;

        for(int var4 = 0; var4 < var3; ++var4) {
            Locations.LocationHandler var5 = var2[var4];
            this.handlersForLocation.put(var5.location, var5);
            Iterator var6 = var5.options.iterator();

            while(var6.hasNext()) {
                Option var7 = (Option)var6.next();
                this.handlersForOption.put(var7, var5);
            }
        }

    }
   
    ...
}    

• CLASS_PATH LocationHandler

private class ClassPathLocationHandler extends Locations.SimpleLocationHandler {
        ClassPathLocationHandler() {
            super(StandardLocation.CLASS_PATH, Option.CLASSPATH, Option.CP);
        }

        Collection<File> getLocation() {
            this.lazy();
            return this.searchPath;
        }

        protected Locations.Path computePath(String var1) {
            String var2 = var1;
            if (var1 == null) {
                var2 = System.getProperty("env.class.path");
            }

            if (var2 == null && System.getProperty("application.home") == null) {
                var2 = System.getProperty("java.class.path");
            }

            if (var2 == null) {
                var2 = ".";
            }

            return this.createPath().addFiles(var2);
        }

        protected Locations.Path createPath() {
            return (Locations.this.new Path()).expandJarClassPaths(true).emptyPathDefault(new File("."));
        }

        private void lazy() {
            if (this.searchPath == null) {
                this.setLocation((Iterable)null);
            }

        }
    }

• PLATFORM_CLASS_PATH LocationHandler

 private class BootClassPathLocationHandler extends Locations.LocationHandler {
    private Collection<File> searchPath;
    final Map<Option, String> optionValues = new EnumMap(Option.class);
    private File defaultBootClassPathRtJar = null;
    private boolean isDefaultBootClassPath;

    BootClassPathLocationHandler() {
        super(StandardLocation.PLATFORM_CLASS_PATH, Option.BOOTCLASSPATH, Option.XBOOTCLASSPATH, Option.XBOOTCLASSPATH_PREPEND, Option.XBOOTCLASSPATH_APPEND, Option.ENDORSEDDIRS, Option.DJAVA_ENDORSED_DIRS, Option.EXTDIRS, Option.DJAVA_EXT_DIRS);
    }
    ```
}       

com.sun.tools.javac.main.Option片段

 BOOTCLASSPATH("-bootclasspath", "opt.arg.path", "opt.bootclasspath", Option.OptionKind.STANDARD, Option.OptionGroup.FILEMANAGER) {
        public boolean process(OptionHelper var1, String var2, String var3) {
            var1.remove("-Xbootclasspath/p:");
            var1.remove("-Xbootclasspath/a:");
            return super.process(var1, var2, var3);
        }

• SOURCE_PATH LocationHandler

new Locations.SimpleLocationHandler(StandardLocation.SOURCE_PATH, new Option[]{Option.SOURCEPATH})

com.sun.tools.javac.main.Option片段

SOURCEPATH("-sourcepath", "opt.arg.path", "opt.sourcepath", Option.OptionKind.STANDARD, Option.OptionGroup.FILEMANAGER),

至此问题的原因水落石出，我们最开始在外部程序编译时，外部程序的classpath中并不包含被编译程序的依赖，而为了编译成功将依赖全部引入外部程序显然也不是明智的做法，因此在attach的java进程内进行编译可以保证classpath包含被编译类的所有依赖是更好的做法，想明白了之后我们也可以不依赖阿里造的轮子了，使用tools.jar原生提供的JavacFileManager也能达到我们的目的。

JavaFileManager其它可以可以被重写的方法

除了关键的list方法，JavaFileManager接口还提供了一系列其它方法，使用者可以重写这部分方法达到自己的需求，比如DynamicFileManager就重写了以下方法:

1. 重写getClassLoader对所有Location都使用attach程序的应用类加载器

    @Override
    public ClassLoader getClassLoader(JavaFileManager.Location location) {
        return classLoader;
    }

2. 重写getJavaFileForOutput方法使用MemoryByteCode接收编译完成后的字节流

@Override
    public JavaFileObject getJavaFileForOutput(JavaFileManager.Location location, String className,
                    JavaFileObject.Kind kind, FileObject sibling) throws IOException {

        for (MemoryByteCode byteCode : byteCodes) {
            if (byteCode.getClassName().equals(className)) {
                return byteCode;
            }
        }

        MemoryByteCode innerClass = new MemoryByteCode(className);
        byteCodes.add(innerClass);
        classLoader.registerCompiledSource(innerClass);
        return innerClass;

    }

com.taobao.arthas.compiler.MemoryByteCode

public class MemoryByteCode extends SimpleJavaFileObject {
    private static final char PKG_SEPARATOR = '.';
    private static final char DIR_SEPARATOR = '/';
    private static final String CLASS_FILE_SUFFIX = ".class";

    private ByteArrayOutputStream byteArrayOutputStream;

    public MemoryByteCode(String className) {
        super(URI.create("byte:///" + className.replace(PKG_SEPARATOR, DIR_SEPARATOR)
                + Kind.CLASS.extension), Kind.CLASS);
    }

    public MemoryByteCode(String className, ByteArrayOutputStream byteArrayOutputStream)
            throws URISyntaxException {
        this(className);
        this.byteArrayOutputStream = byteArrayOutputStream;
    }

    @Override
    public OutputStream openOutputStream() throws IOException {
        if (byteArrayOutputStream == null) {
            byteArrayOutputStream = new ByteArrayOutputStream();
        }
        return byteArrayOutputStream;
    }

    public byte[] getByteCode() {
        return byteArrayOutputStream.toByteArray();
    }

    public String getClassName() {
        String className = getName();
        className = className.replace(DIR_SEPARATOR, PKG_SEPARATOR);
        className = className.substring(1, className.indexOf(CLASS_FILE_SUFFIX));
        return className;
    }

}

3. 重写inferBinaryName方法支持对自定义的CustomJavaFileObject的对binaryName解析，这个方法会在ClassReader解析完PackageSymbol后对包下ClassSymbol进行解析的includeClassFile方法中用到，用来或得类的全限定名

@Override
public String inferBinaryName(Location location, JavaFileObject file) {
   if (file instanceof CustomJavaFileObject) {
       return ((CustomJavaFileObject) file).binaryName();
   } else {
       /**
        * if it's not CustomJavaFileObject, then it's coming from standard file manager
        * - let it handle the file
        */
       return super.inferBinaryName(location, file);
   }
}

com.sun.tools.javac.jvm.ClassReader#includeClassFile

protected void includeClassFile(PackageSymbol var1, JavaFileObject var2) {
        if ((var1.flags_field & 8388608L) == 0L) {
            for(Object var3 = var1; var3 != null && ((Symbol)var3).kind == 1; var3 = ((Symbol)var3).owner) {
                ((Symbol)var3).flags_field |= 8388608L;
            }
        }

        Kind var10 = var2.getKind();
        int var4;
        if (var10 == Kind.CLASS) {
            var4 = 33554432;
        } else {
            var4 = 67108864;
        }

        String var5 = this.fileManager.inferBinaryName(this.currentLoc, var2);
        int var6 = var5.lastIndexOf(".");
        Name var7 = this.names.fromString(var5.substring(var6 + 1));
        boolean var8 = var7 == this.names.package_info;
        ClassSymbol var9 = var8 ? var1.package_info : (ClassSymbol)var1.members_field.lookup(var7).sym;
        if (var9 == null) {
            var9 = this.enterClass(var7, (TypeSymbol)var1);
            if (var9.classfile == null) {
                var9.classfile = var2;
            }

            if (var8) {
                var1.package_info = var9;
            } else if (var9.owner == var1) {
                var1.members_field.enter(var9);
            }
        } else if (var9.classfile != null && (var9.flags_field & (long)var4) == 0L && (var9.flags_field & 100663296L) != 0L) {
            var9.classfile = this.preferredFileObject(var2, var9.classfile);
        }

        var9.flags_field |= (long)var4;
    }

本文地址：https://blog.csdn.net/m0_37556444/article/details/109626847