pantsbuild / pants / 37100435339

93%

push

github

jvm: use argfiles for JVM and JVM-compiler process arguments (#23620)

## Problem

JVM processes that don't use nailgun — JUnit/ScalaTest test runs
(`use_nailgun=False`), `pants run`, and deploy-jar execution — invoke
the `java` binary with their full classpath and argument list passed
directly on the command line via `argv`. For components with many
dependencies or a long classpath, this can exceed OS-level command-line
length limits (e.g. `MAX_ARG_STRLEN`/`ARG_MAX` on Linux/macOS), causing
the process to fail with an "Argument list too long" style error.

Concrete case hit in practice: `pants test` on a JUnit 5 target with a
~250-jar classpath (Spark + Hadoop + Cassandra + Scala + Netty native
bundles) failed with `Os { code: 7, kind: ArgumentListTooLong, ... }`
when colon-joined into a single `-cp` argv element.

## Fix

- For Java 9+ JDKs, `JvmProcess` now writes the `java` invocation's
arguments (classpath plus JVM options and program args) to a `@argfile`
(`__jvm_args.txt`) instead of passing them on the command line, per the
[`java` command-line argument files
support](https://docs.oracle.com/javase/9/tools/java.htm#GUID-4856361B-8BFD-4964-AE84-121F5F6CF111)
added in JDK 9.
- This is gated on `not request.use_nailgun`: nailgun sends the
classpath and program arguments to the already-running server over a
socket rather than via argv, so nailgun-eligible processes (e.g.
compilation) were never subject to this limit and get no benefit from
the argfile — they keep using nailgun exactly as before this PR.
- Skip the argfile (fall back to inline argv) whenever a `{chroot}`
placeholder is present in the classpath or argv. That placeholder is
only substituted by the engine within a `Process`'s literal `argv` at
spawn time, never within file contents, so callers relying on it
(`run.py`, `run_deploy_jar.py`, which also run outside the sandbox via
`InteractiveProcess`) can't have those arguments moved into a file.

## Scoped out

An earlier version of thi... (continued)

48 of 53 new or added lines in 2 files covered. (90.57%)

95982 of 103295 relevant lines covered (92.92%)

3.66 hits per line