Martin Desruisseaux (Geomatys), last edited in October 2023.
This page has been partially copied to Apache
SIS 1.4 release notes.
Since the release of Java 9 and the Java Platform Module System (JPMS, code-named "Jigsaw"),
the tendency in Maven or Gradle projects is to either ignore JPMS,
or to consider Java modules as equivalent to Maven modules or Gradle sub-projects:
each Maven module contains exactly one Java module,
and the same convention is applied to Gradle sub-projects.
This restriction is inherited from Java 8 days,
when the Java compiler accepted only a directory layout named Package Hierarchy
(directory names match exactly package names).
Few peoples are aware that since Java 9, most Java tools,
such as javac and javadoc,
accept an alternative layout named Module Hierarchy
(source: javac Directory Hierarchies).
The latter is used for example by the OpenJDK project itself.
Since Java 9, the javac and javadoc tools can process
many JPMS modules together.
Allowing that multiplicity in developer's project has some advantages described in this page.
To summarize, it gives to developers an extra level of flexibility for organizing their modules,
enables more compile-time checks of cross-module documentation,
facilitates aggregated javadoc and aggregated annotation processing,
facilitates the reuse of test fixtures between modules, and more.
However taking full advantage of JPMS
requires changes in the directory layout compared to Maven and Gradle conventions.
As of October 2023, both Maven and Gradle support only the directory layout known as Package Hierarchy.
This page describes how the alternative Module Source Hierarchy
can be applied to the Apache SIS project.
The layout described below is currently very difficult to apply with Maven,
so this page will provide code snippets for Gradle only.
Table of content:
Multiple Java modules in a single Gradle sub-project does not mean that Gradle sub-projects should be abandoned. Instead, Java modules introduces one new level of sub-division between Gradle sub-projects and Java packages, illustrated by the blue line below. Developers should be free to use it or not, and to choose what to group as a Gradle sub-project and what to group as a Java module.
Java provides some flexibility about the directory layout in a Module Source Hierarchy.
In this section, we assume that the developer wants to stay close to Maven conventions.
The names in green are directories taken unchanged from Maven conventions
and names in blue are new directories added for Module Source Hierarchy support.
The names in red are directories from Maven conventions that we propose to drop,
but this is an arbitrary proposal for Apache SIS restructuring
that does not need to be adopted (the java directory relevance is a separated debate).
In the figure below, the paths on the left side can become the paths on the right side,
where <module> shall be replaced by a
JPMS module name
such as org.apache.sis.storage.
| Current Maven layout | Module Source Hierarchy |
|---|---|
|
|
When compiling the sources illustrated on the right side above,
the root source directory given to the javac command shall be the src directory containing all
modules rather than the java directory containing the org/apache/sis/… hierarchy of packages.
The <module> directory names shall be identical to the module names declared in module-info.java.
An arbitrary amount of custom directories can be inserted between the <module> directory
and the start of the package directories (e.g. org/apache/sis/…).
This insertion of custom directories is the difference between Module Hierarchy
and Module Source Hierarchy in javac documentation.
The custom directories shall be declared with the --module-source-path option like below,
where the * character will be automatically replaced by
<module> by the javac compiler:
javac --module-source-path=/path/to/subproject/src/*/main/java
The options for compiling the tests are similar with main replaced by test (or anything else at user's choice) like below:
javac --module-source-path=/path/to/subproject/src/*/test/java
Users are free to add other sub-directories, for example resources,
under the <module> directory.
Everything that do not match the pattern given to the --module-source-path option will be ignored.
So the main sources, the tests and the resources can be all located under the same module directory,
in a way close to Maven convention if desired.
How to apply those options in a Gradle project will be discussed later on this page.
This section discusses a proposed restructuring of the Apache SIS project following the Module Source Hierarchy. It can been seen as an example of one possible approach for dispatching modules between JPMS modules and Gradle sub-projects. For more generic discussion about Gradle configuration, jump to the next section.
The tree on the left side shows the current directory layout as of May 2023, which follows Maven conventions. The tree on the right side shows a new directory layout proposal. The names in blue are new directory levels. The names in red are removed directory levels. The names in green are directory levels kept unchanged. The src, main, test and java directories are from Maven conventions. The other changes shown below are a proposed restructuring of the Apache SIS project. Resources are omitted in this discussion for simplicity.
| Current Maven layout | Module Source Hierarchy |
|---|---|
Apache SIS project root ├── pom.xml ├── core │ ├── pom.xml │ ├── sis-util │ │ ├── pom.xml │ │ └── src │ │ ├── main │ │ │ └── java │ │ │ └── org/apache/sis/… │ │ └── test │ │ └── java │ │ └── org/apache/sis/… │ ├── sis-metadata │ │ └── Same structure, omitted for brevity. │ ├── sis-referencing │ ├── sis-referencing-by-identifiers │ ├── sis-feature │ ├── sis-cql │ └── sis-portrayal ├── storage │ ├── pom.xml │ ├── sis-storage │ │ ├── pom.xml │ │ └── src │ │ ├── main │ │ │ └── java │ │ │ └── org/apache/sis/… │ │ └── test │ │ └── java │ │ └── org/apache/sis/… │ ├── sis-shapefile │ │ └── Same structure, omitted for brevity. │ ├── sis-xmlstore │ ├── sis-sqlstore │ ├── sis-netcdf │ ├── sis-geotiff │ └── sis-earth-observation ├── cloud │ ├── pom.xml │ └── sis-cloud-aws ├── profiles │ ├── pom.xml │ ├── sis-france-profile │ └── sis-japan-profile └── application ├── pom.xml ├── sis-console ├── sis-webapp ├── sis-openoffice └── sis-javafx |
Apache SIS project root ├── settings.gradle.kts ├── endorsed │ ├── build.gradle.kts │ └── src │ ├── org.apache.sis.util │ │ ├── main │ │ │ ├── module-info.java │ │ │ └── org/apache/sis/… │ │ └── test │ │ └── org/apache/sis/… │ ├── org.apache.sis.metadata │ │ └── Same structure, omitted for brevity. │ ├── org.apache.sis.referencing │ ├── org.apache.sis.referencing.gazetteer │ ├── org.apache.sis.feature │ ├── org.apache.sis.storage │ ├── org.apache.sis.storage.xml │ ├── org.apache.sis.storage.sql │ ├── org.apache.sis.storage.netcdf │ ├── org.apache.sis.storage.geotiff │ ├── org.apache.sis.storage.earthobservation │ ├── org.apache.sis.cloud.aws │ ├── org.apache.sis.portrayal │ ├── org.apache.sis.profile.france │ ├── org.apache.sis.profile.japan │ ├── org.apache.sis.console │ ├── org.apache.sis.openoffice │ └── org.apache.sis.test (new module, see below) │ └── test │ └── module-info.java ├── incubator │ ├── build.gradle.kts │ └── src │ ├── org.apache.sis.cql │ │ ├── main │ │ │ ├── module-info.java │ │ │ └── org/apache/sis/… │ │ └── test │ │ └── org/apache/sis/… │ ├── org.apache.sis.storage.shapefile │ │ └── Same structure, omitted for brevity. │ └── org.apache.sis.webapp └── optional ├── build.gradle.kts └── src └── org.apache.sis.gui └── Same structure, omitted for brevity. |
In the old layout (left side),
the Apache SIS project organized modules in some groups:
core, storage, cloud, profiles and application.
While this grouping can be useful for understanding the content
of the Apache SIS project,
it serves no purpose from the point of view of build management.
In the new layout (right side), that grouping is removed from the directory tree.
Such logical grouping can appear in JPMS
module names if desired, for example org.apache.sis.storage.*,
org.apache.sis.cloud.* and org.apache.sis.profile.*.
The old "build-irrelevant" grouping is replaced by a new grouping which is relevant to the build:
With the new layout, modules that are not ready for release can be easily excluded all together. By comparison, with the old layout the release manager had to manually exclude various modules scattered in the tree. Likewise, the optional modules can be included or excluded all together depending on license agreement. For example the GUI depends on JavaFX and can be included in the build only on acceptance of GPL terms. This new way of grouping modules will hopefully simplify Apache SIS releases.
In the new layout, the replacement of Maven modules by JPMS modules has a desirable side-effect. In the old layout, any Maven module could depend on any other Maven module as long as there is no cycle. Maven determines the modules build order regardless if modules belong to the same groups or not. So nothing (except cycles) prevented a module in the core group to depend on a module in incubation or subject to restrictive license terms. With the new layout, the endorsed, incubator and optional sub-projects are the finest level of grouping managed by the build system. The JPMS modules inside those sub-projects cannot be managed independently by the build system, which has advantages and inconvenient. An advantage for Apache SIS is that compile-time dependency of endorsed modules toward any incubator or optional modules become impossible (however, runtime dependency through Service Provider Interfaces is still possible).
For the Apache SIS restructuring, we propose to drop the java directory. The consequence would be that resources are no longer separated from Java source code. The Maven's convention putting resources in a separated directory hierarchy is considered a good practice by some, but this is not an universal opinion. NetBeans Ant projects and OpenJDK for example don't do that. The argument is similar to documentation, which was traditionally separated from the code in previous programming languages. The Java designers decided that the best place to put documentation (javadoc) was close to the code. Having resources close to the code has similar advantages. It makes more likely that the developer sees when a change in a class may require a change in a resource, and less tedious to open that file (no need to navigate through the exact same path in a separated directory hierarchy). Furthermore, the Maven's convention separating java and resources does not work well in a multi-languages project anyway, because it does not distinguish the Java resources to copy in a JAR file from the C/C++ or Python resources (for example). Mixing two languages in the same module happens when the module is a bridge between those two languages, such as PROJ-JNI (between Java and C/C++) and GeoAPI bridge (between Java and Python).
The new layout contains a module, named org.apache.sis.test,
that did not existed in the old layout. This module is local to the build and never deployed.
It has no main sub-directory,
only a test/module-info.java file.
This is a convenient way to declare dependencies that are needed by the tests
but not declared in any main/module-info.java file being compiled.
Actually our experiments suggest that the --add-reads option does not work well if the added
module does not appear in a requires clause of at least one module-info.java file.
The org.apache.sis.test module resolves that problem.
Maven has a different approach which allows the tests to overwrite the main module-info files.
We don't do that because those files are sometime a bit large,
and we want to avoid the risk of overwriting them with
test/module-info.java files that differ in unintended ways.
In our proposed approach, the org.apache.sis.test module information
is added to the module-info files of all modules to test instead of overwriting them.
It is possible to get Gradle to work to some extent without writing a custom plugin. The hacks are not very clean, but could be much better with a little bit of improvement from Gradle. The "Ideas for Gradle evolution" section provides some proposals. The key information that needs to be supplied are:
--module-path <paths to all dependencies>--module-source-path /path_to_sub_project/src/*/main/java--add-modules module_1,module_2,module_3,…
(list all modules of the project to compile, not dependencies)--module-path <paths to dependencies including the directory containing the output of above compilation of main modules>--module-source-path /path_to_sub_project/src/*/test/java--patch-module module_1=/path_to_sub_project/src/module_1/test/java
(repeat for each module)--add-modules <same as for compilation of main code>--add-reads module_1=dependency_A,dependency_B,… for all test-only dependencies such as JUnit
(repeat for each module)--module-path <paths to dependencies and compilation result of main modules, but excluding compilation result of test classes>--patch-module module_1=${buildDir}/classes/java/test/module_1
(repeat for each module)--add-modules <same as for compilation of main code>--add-reads (as needed for test-only dependencies)--add-opens (as needed for Jakarta or other libraries based on reflection)--add-exports (as needed for allowing JUnit to test private package)
A lot of configuration can be done automatically with loops.
The example below shows a small fragment of what a build.gradle.kts file
may look like when there is no plugin for making the task easier.
This example uses hard-coded Apache SIS module names,
but the principle is generalizable to other projects.
plugins {
`java-library`
}
var srcDir = file("src") // Root directory of all modules.
sourceSets {
main {
java {
setSrcDirs(listOf(file("src/org.apache.sis.util/main/java),
file("src/org.apache.sis.metadata/main/java),
file("src/org.apache.sis.referencing/main/java),
/* etc. We will use a plugin in Java for automation. /* )
}
}
test {
java {
setSrcDirs(listOf(file("src/org.apache.sis.util/test/java),
file("src/org.apache.sis.metadata/test/java),
file("src/org.apache.sis.referencing/test/java),
/* etc. We will use a plugin in Java for automation. /* )
}
}
}
fun addAllModules(args : List<String>) {
args.add("--add-modules")
args.add(srcDir.list().joinToString(separator=","))
}
tasks.compileJava {
options.compilerArgs.add("--module-source-path=${srcDir}/*/main/java")
options.compilerArgs.add("--module-path=${classpath.asPath}")
setClasspath(files()) // For avoiding duplication with the --module-path option.
addAllModules(options.compilerArgs)
}
tasks.compileTestJava {
options.compilerArgs.add("--module-source-path=${srcDir}/*/test/java")
options.compilerArgs.add("--module-path=${classpath.asPath}")
setClasspath(files())
srcDir.list().forEach {
options.compilerArgs.add("--patch-module")
options.compilerArgs.add("${it}=${srcDir}/${it}/test/java")
options.compilerArgs.add("--add-reads")
options.compilerArgs.add("${it}=org.apache.sis.test,junit")
}
addAllModules(options.compilerArgs)
// Add more --add-reads and --add-exports project-specific options as needed.
}
tasks.test {
useJUnitPlatform()
/*
* Create a module path which excludes the compiled test classes, because those
* classes will be specified using another JVM option (namely `--patch-module`).
*/
val mainpath = classpath.filter {
path : File ->
!path.getName().equals(test)
}
val args = mutableListOf("--module-path", mainpath.asPath)
srcDir.list().forEach {
args.add("--patch-module")
args.add("${it}=${buildDir}/classes/java/test/${it}")
args.add("--add-reads")
args.add("${it}=junit,ALL-UNNAMED")
}
addAllModules(args)
// Add more --add-reads and --add-exports project-specific options as needed.
setAllJvmArgs(args)
}
This complexity can be handled by a customized Gradle plugin in Java. Details about how to do so are given in the
Gradle documentation
and are not repeated here. Some parts that can be moved from build.gradle.kts script to Java code
are the full sourceSets configuration, together with the class-path and module-path settings,
and the --source-module-path, --add-modules and --patch-modules options.
Some (but not all) --add-exports options can also be managed by the plugin.
Source code of a plugin developed specifically for Apache SIS can be seen below.
It could be generalized to arbitrary projects if there is an interest.
Above configuration works but has the following problems with Gradle 8.2.
Workarounds for current Gradle version are presented in this section.
Proposed Gradle evolutions are presented in a later section.
The issue that caused the greatest difficulties for us is the Gradle automatic dispatching
of dependencies between the --class-path and --module-path options,
which is discussed first.
That black magic was very close to be a blocker issue because it does the wrong thing.
The other issues are less critical and could be summarized as
"Insufficient control on the options passed to the command".
The ease of use issue is discussed in a separated section.
Gradle uses a set of heuristic rules for deciding if a dependency should be declared on the class-path or on the module-path.
But heuristic rules tend to work well only in some specific contexts, which is currently restricted to package hierarchy.
As of Gradle 8.2.1, those heuristic rules do not recognize any module in our Source Module Hierarchy.
In particular, the automatic dispatching of dependencies is enabled only if Gradle believes that the
sub-project being compiled is itself a JPMS module,
and Gradle does not recognize Module Hierarchy as such. The Gradle's
ModularitySpec
class does not provide an option for forcing the activation of automatic dispatching.
The current workaround is to make explicit calls to methods of Gradle API such as
setClasspath(…) for overwriting the class-path and module-path defined by Gradle.
Note: above paragraph explains the issue when building the library project. In that case the burden of applying workaround falls on us, which we accept to do. However the same issue (wrong dispatching) hits also all external projects using the library, potentially breaking any JPMS library (not only Apache SIS) for all users of that library in a non-JPMS project. This is not a JPMS problem, this is a Maven 3.8.6 (or maybe Plexus) mishandling which is also replicated in Gradle 8.2.1. See Quasi-blocker Maven bug for details.
The Gradle ModularitySpec implementation could be improved for recognizing a larger set of hierarchies,
but even better automatic detection will not always work.
Sometime we want to force a dependency to be on the module-path no matter what Gradle thinks.
It happens for example when a dependency has no module-info.class file
and no Automatic-Module-Name entry in the MANIFEST.MF file,
but we still want to handle it as an automatic module.
Some may argue that this is bad practice, but this is sometime necessary for getting tools to work.
In Apache SIS case, all those automatic modules are optional dependencies.
We need a way to control whether a dependency should be considered as a module or not on a case-by-case basis.
In the Javadoc task, it is difficult to modify the class-path and module-path options
because the class-path is modified again by Gradle after our configuration.
So the class-path was incomplete at the time we copied its entries to the module-path,
and the class-path receives undesired new entries after we cleared it.
This behavior causes Javadoc generation to fail, with no workaround we could find so far.
However a manual workaround exists by opening the build/tmp/javadoc/javadoc.options
file in an editor, move the -classpath content to --module-path,
then run javadoc @build/tmp/javadoc/javadoc.options on the command line.
Because Javadoc are generated less often than compilation,
we think that this workaround is acceptable until a better solution become available.
Adding the --module-path in the compiler options cause the option to appear twice
in the debug output of Gradle 8.2. The two occurrences have the exact same path, which may be large.
We found no way to prevent that duplication, as it does not appear in the list returned by
CompilerOptions.getCompilerArgs().
Our current workaround is to do nothing,
as javac seems to work anyway with duplicated elements on the module path.
In the same way that --class-path and --module-path should specify mutually exclusive sets,
--source-path and --module-source-path should also be mutually exclusive options.
The --source-path option is considered rarely needed in modern builds and can be omitted.
But the Gradle's debug output seems to unconditionally provide the latter option at least with an empty string,
because the empty string has a different meaning than the default javac value.
Using the Gradle API for setting the source path to null does not help since Gradle
interprets that as an empty path.
We saw no API for telling Gradle to omit completely that option.
This is a problem since the --module-source-path option is necessary for specifying the
src/*/main/java pattern.
Consequently when launching javac on the command-line with the options shown by Gradle's debug output,
we get the following error:
error: cannot specify both --source-path and --module-source-pathOur current workaround is to do nothing. The Java compiler seems to work inside Gradle, even if it doesn't work on the command-line with Gradle's debug output.
The use of Module Source Hierarchy instead of Package Hierarchy has advantages and inconvenient. Advantages for the Apache SIS project are described below. The main inconvenient is the poor support in current build tools and IDE. However the latter is not a blocker, and we can try to contribute in improving the situation with proposals such as the "Ideas for Gradle evolution" section at the bottom of this page.
Most JDK tools are JPMS aware
and can process many modules in one invocation of each command-line tool.
In some cases, the same result can be obtained by invoking the same tool repetitively for each module, but not always.
The most obvious example where the result differs is javadoc.
When executed for a group of modules instead of invoked repetitively for each module,
javadoc can generate an aggregated API documentation
with a home page listing all modules, an index with entries from all modules,
hyper-links to modules beyond the boundary of what is declared in module-info
(for example lists of all implementations of each interface), etc.
Such aggregation does not fit naturally in the Maven directory layout.
Maven does support aggregated Javadoc, but this support requires hacks and may not be easily applicable to other tools.
An example of another tool for which aggregated execution is sometime useful is annotation processor.
More use cases may appear in future Java versions.
A native support of Module Source Hierarchy in Gradle would make easier to leverage those features
with less needs to resort to hacks.
Suppose that module B depends on module A.
Module B can have compile-time dependencies toward A (backward references),
but the converse (a forward reference from A to B)
is illegal for the compiler except in module-info.
However such forward references are perfectly legal in documentation,
and indeed the javadoc tool handles them well.
It is possible to write Javadoc {@link} and {@see} tags in module A with
forward references to some API in module B.
However doing so with Maven directory layout requires that we sacrifice a safety.
The javac tool offers the possibility to verify Javadoc {@link} and {@see}
tags at compile-time. This feature offers much faster error detections than waiting for Javadoc generation,
because the latter is done less frequently than compilation.
This verification can be enabled by passing the -Xdoclint:all option to javac,
in which case any invalid {@link} or {@see} tag causes a compilation error.
It works well with references to API in the same module
or in dependencies, but cannot work with forward references unless javac knows that those modules
exist and what they contain.
This is possible with Module Source Hierarchy, but not with Maven directory layout.
With the latter, all forward references are flagged as errors.
With the former, compile-time verification of {@link} or {@see} tags,
including forward references, works like a charm.
Another place where forward references are used is in module-info files.
The opens and exports statements can be qualified,
i.e., a package can be exported only to some specific modules.
Those modules are forward references, because they are dependents rather than dependencies.
But without Module Source Hierarchy, it is difficult for the compiler to know what those dependents are,
so any use of qualified exports generally produces warnings like below:
endorsed/src/org.apache.sis.metadata/main/module-info.java:164:warning: [module] module not found:org.apache.sis.gui
The test code for a module may create test fixtures, mocks or assertion methods
that we want to reuse in the test code of dependent modules.
With Maven, we have to package the test classes in an artifact of type test-jar.
With the module source hierarchy, this is no longer necessary if the test fixtures are reused
only inside the same sub-project. For example if test fixtures are provided in the
org.apache.sis.test package under the
test/java
sub-directory of the org.apache.sis.util module,
then all other modules in the same sub-project can access those text fixture by adding
the following script in the build.gradle.kts file,
with nothing to package or deploy:
tasks.compileTestJava {
(…snip…)
var allModules = file("src").list().joinToString(separator=",")
args.add("--add-exports")
args.add("org.apache.sis.util/org.apache.sis.test=${allModules}")
}
A side-effect of the proposal described in this page is that the tests of all modules in a Gradle sub-project
are executed together in the same JVM.
It may be considered against unit test principles, but actually this is controllable.
First we note that having all modules in the JVM
during test execution is not necessarily a bad thing.
It sometime happens that a test behaves differently when the module is alone in the JVM
compared to when all modules are present.
Because the latter scenario is more representative of production environment than the former,
some bugs can be unnoticed because of module isolation during tests.
Some peoples will argue that integration tests should have discovered such bugs,
but it is hard to have an extensive coverage for all kinds of tests.
With the Module Source Hierarchy, the easiest configuration is to let all tests
be executed in the same JVM.
However it is possible to filter which modules to load in the JVM
with options such as --limit-modules.
It should be possible to improve Gradle with options for making easy to run different
subsets of the tests with different subsets of modules loaded.
We have not done serious benchmarks, but compiling all modules from scratch seems a little bit faster when
javac is invoked once for all modules compared to invoking javac for each module.
Likewise, tests are also faster presumably because common dependencies are loaded only once and the caching
mechanisms of the tested application takes effect
(see the previous section for a discussion about running the tests of all modules together).
In Apache SIS case, the build time is reduced by about 30%.
However, speed is not the main criterion regarding the use (or not) of Module Source Hierarchy.
The Module Source Hierarchy is not best suited to every situations. This section describes some cases where the Maven hierarchy currently supported by Gradle may be better suited.
As of October 2023, neither Maven or Gradle provides out-of-the-box support for Module Source Hierarchy. A very good out-of-the-box support (actually the major source of inspiration for the proposal in this page) is provided by the NetBeans IDE when using the NetBeans Ant build system, but the NetBeans community itself encourages the use of Maven or Gradle instead of Ant. Nevertheless, the flexibility of Gradle compared to Maven makes possible to use Module Source Hierarchy with some efforts, but the task could be made much easier with a little bit of Gradle improvements such as the ones proposed at the end of this page.
As of Gradle 8.2, the ANTLR task does not work well with arbitrary source directories. We have to keep the default conventions of the ANTLR plugin even if those conventions do not fit well in Module Source Hierarchy. Another problem is that we didn't found the right compiler options for combining a directory of generated sources with the main sources in a Module Source Hierarchy, so we have to write the output directly in main source directory. In Apache SIS case this problem is hopefully temporary, because we plan to replace ANTLR generated code by hand-written code.
IntelliJ seems to open well the Apache SIS Gradle project with Module Source Hierarchy, but NetBeans has some glitches. NetBeans 18 can compile, execute and debug that project, but the project tab does not show the JPMS module names. NetBeans only shows "Source/Test Packages" with the directory name, which is the same for all modules.
However among the many forms of projects that NetBeans supports, one of them is "Java with Ant" → "Java Modular Project". That form uses Module Source Hierarchy and prove that this layout can be rendered well in NetBeans at least with Ant projects. Extending this support to Gradle projects is under investigation.
When invoking Java tools such as java, javac or javadoc,
the project dependencies can be put either on the class-path or on the module-path using the
command-line --class-path and --module-path options respectively.
Maven 3.8.6 and Gradle 8.2.1 use automatically the module-path if all the following conditions are true:
module-info.class file
or an Automatic-Module-Name attribute in MANIFEST.MF), and
Condition #1 is okay as a default, but #2 is problematic.
The fact that a dependency is declared on the class-path rather than the module-path
changes the way that java. discovers the provided services.
ServiceLoader scans the content
of META-INF/services directory.ServiceLoader uses the declarations
in module-info.class.
Even if condition #2 is false (i.e. a project is not modularized), modularized dependencies still
need to be declared on the module-path for allowing the dependency to discover its own services,
or the services of a transitive modularized dependency.
If a modularized dependency is put on the class-path instead,
it has consequence not only for the project using that dependency,
but also for the dependency itself, which become unable to use its own module-info.class.
This is demonstrated by a small test case on GitHub,
together with two java command-lines reproducing the Maven behavior followed by the desired behavior.
Unless Maven provides some configuration options that we did not see, the way that Maven decides what to
put on --class-path and what to put on --module-path is a quasi-blocker issue
for gradual modularisation of large projects. It is so because the consequences of dispatching JAR
files on class-path versus module-path is not limited to the project using those JAR files.
The consequences apply also to the libraries inside those JAR files themselves,
even if those libraries were fully built as JPMS modules.
There is various JDK methods that behave differently depending on whether the code invoking
those methods were inside a JAR file specified on the class-path or a JAR file
specified on the module-path. Those methods are identified by the @CallerSensitive annotation
in the JDK source code and include not only above-cited java.,
but also ClassLoader. and more.
The workaround for library developers is to declare all service providers in both
module-info file and META-INF/services/ directory, with the risk of inconsistencies.
This workaround may force developers to renounce to the usage of provider() static methods,
because that method works only for providers declared in module-info.
It means that developers must renounce to provide singleton instances of their service providers
(that problem can sometime be mitigated with wrappers).
Note that this workaround does not fix the real issue,
which is that dependencies are loaded as unnamed modules when they should not.
The workaround allows libraries and applications to find some service providers despite this problem,
but any other features that depend on named modules are still broken.
Even the service providers may not work as intended despite the META-INF/services duplication,
because of the impossibility to reproduce exactly the provider() method behavior.
Issue on Maven JIRA tracker: MNG-7855
Maven and Gradle uses "convention-over-configuration" approach to build JVM-based project. A problem with that approach is that it works well for a few years, but when the convention become no longer suited to the language evolution, it is very difficult to changes the habits. I suspect that Maven conventions are a reason for the slow JPMS adoption.
Full JPMS support,
i.e. having the possibility to use the javac Module Source Hierarchy when desired
instead of being restricted to the Package Hierarchy, has some advantages described in this page.
There is also inconvenient, but the intend is not to force developers to use Source Module Hierarchy.
The intend is to make easy to use either the package or source module hierarchy when desired.
Gradle could help in different ways, described below.
The most critical issue, almost a blocker, is the first one.
The current automatic dispatching of dependencies on class-path versus module-path does not work with Module Source Hierarchy.
Even if it was fixed, it would break again if a new hierarchy was introduced in the future.
Even if an automagic algorithm was able to work for every hierarchies, automatic dispatching is not always desirable.
Sometime the developer really wants a dependency to be declared on --module-path even if it is not modularized,
because the consequences on the modules that use this dependency are not the same.
We could complete or replace the inferModulePath property in
ModularitySpec
for making possible to force the automatic dispatching of dependencies between class-path and module-path
without relying on the conditions documented in the getInferModulePath() method.
Alternatively Gradle could be enhanced for recognizing the Source Module Hierarchy itself.
But in any cases, we would still need a way to force a dependency to be on the module-path no matter what Gradle thinks.
One possibility may be to expand the syntax of dependencyResolutionManagement.
For example (green elements are what currently exist,
blue element is tne proposal with exact syntax to be discussed):
dependencyResolutionManagement { versionCatalogs { create("libs") { library("jts.core", "org.locationtech.jts", "jts-core") { version { strictly("[1.15, 2.0[") prefer("1.19.0") } JPMS { type("module") // Other allowed values: "unnamed" or "auto" (the default) } } } } }
The type("module") element would mean
"always put this dependency on the --module-path option".
Conversely type("unnamed")
would mean to always put on the --class-path option,
and type("auto")
would be the current behavior.
The rational for putting this information in dependencyResolutionManagement is that,
like version numbers, the dependency type (module versus unnamed) should be consistent in all sub-projects.
This is because a larger project combining many sub-projects will be able to make only one "class-path versus module-path"
choice per dependency.
This type("module") proposal is the only thing
that we need for resolving the biggest difficulty encountered so far in using Gradle with Module Source Hierarchy.
However this new JPMS block could be extended in the future with more features.
For example we could define an
importAllPackages("org.apache.sis.*") method
meaning that when this dependency is used, then an --add-exports compiler option should be automatically added
for exporting to that dependency all non-exported packages of each module whose name starts with "org.apache.sis.".
It would be useful for testing with JUnit.
Similar methods could be added for --add-opens, --patch-module, etc.
Provide a way to set the all compiler options, including the ones managed by Gradle itself.
Currently the CompilerOptions
class has a getCompilerArgs() method providing a mutable list,
but that list is only for additional options appended after the options managed by Gradle.
There is also a getAllCompilerArgs() method, but that list is unmodifiable.
The reason why full control on all compiler options is sometime desired
is because latest Java releases may have new features that are incompatible with the options managed by Gradle.
An example of incompatible Java compiler options is --module-source-path versus --source-path.
Even if Gradle resolves this incompatibility, other incompatible options existed in the past
(for example -release versus -source and -target) and
we cannot know in advance what will be the next incompatible options in future Java releases.
We may also want to do cleanups such as removing the --module-path duplication.
The same argument applies to other tools as well.
Currently the lack of full control on javadoc options is a blocker issue for generating the Javadoc,
forcing us to launch javadoc directly on the command-line instead.
We accept this problem as a hopefully temporary inconvenience since Javadoc is not generated
as often as the JAR files.
java.lang.Class objects
The test classes are currently specified as a list of files.
Gradle infers the class name from the file name, then invoke Class.forName(String, ClassLoader).
But the class name inferred by Gradle is wrong when the project layout is not as expected by Gradle.
Other reasons why developers way want to instantiate the Class themselves may be if they need
some control on the ClassLoader or ModuleLayer to use for loading the classes.
--add-exports options
It is currently tedious to add --add-reads and --add-exports options.
Furthermore the same options need to be specified in different places, such as javac
and the java command for executing tests. A convenient place where some options could
be specified would be in the dependency declarations, because the options usually need to be the same
for all usage of a dependency. Example:
library("junit", "junit", "junit") { version { prefer("4.13") } JPMS { importAllPackages("org.apache.sis.*") } }
The importAllPackages(…) statement would cause the following actions
to happen when the dependency is added to the class-path or module-path:
"org.apache.sis.".--add-exports org.apache.sis.foo/org.apache.sis.bar1=junit --add-exports org.apache.sis.foo/org.apache.sis.bar2=junit etc.
The search for module names, package lists and exported packages can be done easily with java.lang.module.ModuleDescriptor.
A prototype (but not using above proposed syntax) is available in
ModularTest.
Test cases demonstrating the problem, and issues created on bug trackers: