Execution default-cli of goal com.google.cloud.tools:appengine-maven-plugin:2.8.5:stage failed: A required class was missing while executing com.google.cloud.tools:appengine-maven-plugin:2.8.5:stage: org/apache/commons/lang3/SystemProperties

[ludoch]

See CI CD plugin update usage in GAE repository to latest plugin version 3.8.5:

https://productionresultssa11.blob.core.windows.net/actions-results/818ff19a-8a0a-43db-88c0-fb0e3bec9330/workflow-job-run-0a7a0e00-357d-5ffd-b4d1-21de564fdc12/logs/job/job-logs.txt?rsct=text%2Fplain&se=2025-11-17T17%3A47%3A09Z&sig=NuuAvQN5%2FiHcuPjice1s%2BBOqtfdP5wUv9QZaTgzIM3s%3D&ske=2025-11-18T02%3A57%3A49Z&skoid=ca7593d4-ee42-46cd-af88-8b886a2f84eb&sks=b&skt=2025-11-17T14%3A57%3A49Z&sktid=398a6654-997b-47e9-b12b-9515b896b4de&skv=2025-11-05&sp=r&spr=https&sr=b&st=2025-11-17T17%3A37%3A04Z&sv=2025-11-05

I see this error with the new GAE Maven plugin:

2025-11-17T05:00:21.3142942Z ,
2025-11-17T05:00:21.3143358Z , Downloaded from central: https://repo.maven.apache.org/maven2/com/google/inject/guice/5.1.0/guice-5.1.0.jar (778 kB at 3.0 MB/s)
2025-11-17T05:00:21.3143790Z , [INFO] Staging the application to: /home/runner/work/appengine-java-standard/appengine-java-standard/applications/springboot/target/appengine-staging
2025-11-17T05:00:21.3144021Z , [INFO] Detected App Engine appengine-web.xml based application.
2025-11-17T05:00:21.3144219Z , [INFO] ------------------------------------------------------------------------
2025-11-17T05:00:21.3144323Z , [INFO] BUILD FAILURE
2025-11-17T05:00:21.3144514Z , [INFO] ------------------------------------------------------------------------
2025-11-17T05:00:21.3144640Z , [INFO] Total time: 5.301 s
2025-11-17T05:00:21.3144757Z , [INFO] Finished at: 2025-11-17T05:00:21Z
2025-11-17T05:00:21.3144937Z , [INFO] ------------------------------------------------------------------------
2025-11-17T05:00:21.3146702Z , [ERROR] Failed to execute goal com.google.cloud.tools:appengine-maven-plugin:2.8.5:stage (default-cli) on project springboot: Execution default-cli of goal com.google.cloud.tools:appengine-maven-plugin:2.8.5:stage failed: A required class was missing while executing com.google.cloud.tools:appengine-maven-plugin:2.8.5:stage: org/apache/commons/lang3/SystemPropertiies

Link to the github action log:

https://productionresultssa11.blob.core.windows.net/actions-results/818ff19a-8a0a-43db-88c0-fb0e3bec9330/workflow-job-run-0a7a0e00-357d-5ffd-b4d1-21de564fdc12/logs/job/job-logs.txt?rsct=text%2Fplain&se=2025-11-17T17%3A47%3A09Z&sig=NuuAvQN5%2FiHcuPjice1s%2BBOqtfdP5wUv9QZaTgzIM3s%3D&ske=2025-11-18T02%3A57%3A49Z&skoid=ca7593d4-ee42-46cd-af88-8b886a2f84eb&sks=b&skt=2025-11-17T14%3A57%3A49Z&sktid=398a6654-997b-47e9-b12b-9515b896b4de&skv=2025-11-05&sp=r&spr=https&sr=b&st=2025-11-17T17%3A37%3A04Z&sv=2025-11-05

Link to the upgrade automated Mend renovate PR:
GoogleCloudPlatform/appengine-java-standard#434

Faliure is seen for all JDK targets we use in Google Actions:
https://github.com/GoogleCloudPlatform/appengine-java-standard/actions/runs/19418340906

I guess you are missing a org/apache/commons/lang3/SystemPropertiies ?

[ludoch]

Related to recent https://github.com/GoogleCloudPlatform/appengine-plugins/pull/1058/files ?

[ludoch]

Related mvn command used in the failing test:
https://github.com/GoogleCloudPlatform/appengine-java-standard/blob/a787472074f421662efad91a05753181ce64d462/runtime/test/src/test/java/com/google/apphosting/runtime/jetty9/SpringBootTest.java#L40

[ludoch]

So I tried locally on my mac to switch to the mend renovate branch and did a clean build and I did not see the error...

I will retry a github action run, but it seems to be only gh action centric... Hummm

[ludoch]

new gh action run launched at https://github.com/GoogleCloudPlatform/appengine-java-standard/actions/runs/19418340906

[ludoch]

While I cannot repro locally , I also cannot find the missing class in A required class was missing while executing com.google.cloud.tools:appengine-maven-plugin:2.8.5:stage: org/apache/commons/lang3/SystemProperties

  jar tvf org/apache/commons/commons-lang3/3.8.1/commons-lang3-3.8.1.jar |grep System

 10173 Wed Sep 19 11:24:04 PDT 2018 org/apache/commons/lang3/SystemUtils.class
  1098 Wed Sep 19 11:24:02 PDT 2018 org/apache/commons/lang3/text/StrLookup$SystemPropertiesStrLookup.class

So the github actions environment must be triggering some code which is different than a mac local build, and it needs org/apache/commons/lang3/SystemProperties

[ludoch]

Error log is really

2025-11-17T19:28:19.5241404Z , Downloaded from central: https://repo.maven.apache.org/maven2/org/apache/maven/resolver/maven-resolver-util/1.9.7/maven-resolver-util-1.9.7.jar (190 kB at 585 kB/s)
2025-11-17T19:28:19.5242469Z , [INFO] Staging the application to: /home/runner/work/appengine-java-standard/appengine-java-standard/applications/springboot/target/appengine-staging
2025-11-17T19:28:19.5242798Z , [INFO] Detected App Engine appengine-web.xml based application.
2025-11-17T19:28:19.5243090Z , [INFO] ------------------------------------------------------------------------
2025-11-17T19:28:19.5243241Z , [INFO] BUILD FAILURE
2025-11-17T19:28:19.5243515Z , [INFO] ------------------------------------------------------------------------
2025-11-17T19:28:19.5243705Z , [INFO] Total time:  6.350 s
2025-11-17T19:28:19.5243882Z , [INFO] Finished at: 2025-11-17T19:28:19Z
2025-11-17T19:28:19.5244151Z , [INFO] ------------------------------------------------------------------------
2025-11-17T19:28:19.5246258Z , [ERROR] Failed to execute goal com.google.cloud.tools:appengine-maven-plugin:2.8.5:stage (default-cli) on project springboot: Execution default-cli of goal com.google.cloud.tools:appengine-maven-plugin:2.8.5:stage failed: A required class was missing while executing com.google.cloud.tools:appengine-maven-plugin:2.8.5:stage: org/apache/commons/lang3/SystemProperties
2025-11-17T19:28:19.5246532Z , [ERROR] -----------------------------------------------------
2025-11-17T19:28:19.5246906Z , [ERROR] realm =    plugin>com.google.cloud.tools:appengine-maven-plugin:2.8.5
2025-11-17T19:28:19.5247339Z , [ERROR] strategy = org.codehaus.plexus.classworlds.strategy.SelfFirstStrategy
2025-11-17T19:28:19.5248475Z , [ERROR] urls[0] = file:/home/runner/.m2/repository/com/google/cloud/tools/appengine-maven-plugin/2.8.5/appengine-maven-plugin-2.8.5.jar
2025-11-17T19:28:19.5249298Z , [ERROR] urls[1] = file:/home/runner/.m2/repository/com/google/cloud/tools/appengine-plugins-core/0.13.5/appengine-plugins-core-0.13.5.jar
2025-11-17T19:28:19.5249826Z , [ERROR] urls[2] = file:/home/runner/.m2/repository/com/google/guava/guava/32.1.2-jre/guava-32.1.2-jre.jar
2025-11-17T19:28:19.5251038Z , [ERROR] urls[3] = file:/home/runner/.m2/repository/com/google/guava/listenablefuture/9999.0-empty-to-avoid-conflict-with-guava/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar
2025-11-17T19:28:19.5251707Z , [ERROR] urls[4] = file:/home/runner/.m2/repository/com/google/code/findbugs/jsr305/3.0.2/jsr305-3.0.2.jar
2025-11-17T19:28:19.5252312Z , [ERROR] urls[5] = file:/home/runner/.m2/repository/org/checkerframework/checker-qual/3.33.0/checker-qual-3.33.0.jar
2025-11-17T19:28:19.5252950Z , [ERROR] urls[6] = file:/home/runner/.m2/repository/com/google/j2objc/j2objc-annotations/2.8/j2objc-annotations-2.8.jar
2025-11-17T19:28:19.5253390Z , [ERROR] urls[7] = file:/home/runner/.m2/repository/org/yaml/snakeyaml/2.2/snakeyaml-2.2.jar
2025-11-17T19:28:19.5253876Z , [ERROR] urls[8] = file:/home/runner/.m2/repository/com/google/code/gson/gson/2.11.0/gson-2.11.0.jar
2025-11-17T19:28:19.5254639Z , [ERROR] urls[9] = file:/home/runner/.m2/repository/com/google/errorprone/error_prone_annotations/2.27.0/error_prone_annotations-2.27.0.jar
2025-11-17T19:28:19.5255140Z , [ERROR] urls[10] = file:/home/runner/.m2/repository/org/glassfish/javax.json/1.1.4/javax.json-1.1.4.jar
2025-11-17T19:28:19.5255803Z , [ERROR] urls[11] = file:/home/runner/.m2/repository/org/apache/commons/commons-compress/1.28.0/commons-compress-1.28.0.jar
2025-11-17T19:28:19.5256355Z , [ERROR] urls[12] = file:/home/runner/.m2/repository/commons-codec/commons-codec/1.19.0/commons-codec-1.19.0.jar
2025-11-17T19:28:19.5257099Z , [ERROR] urls[13] = file:/home/runner/.m2/repository/org/codehaus/plexus/plexus-utils/3.5.1/plexus-utils-3.5.1.jar
2025-11-17T19:28:19.5257833Z , [ERROR] urls[14] = file:/home/runner/.m2/repository/org/codehaus/plexus/plexus-sec-dispatcher/2.0/plexus-sec-dispatcher-2.0.jar
2025-11-17T19:28:19.5258691Z , [ERROR] urls[15] = file:/home/runner/.m2/repository/org/codehaus/plexus/plexus-cipher/2.0/plexus-cipher-2.0.jar
2025-11-17T19:28:19.5259393Z , [ERROR] urls[16] = file:/home/runner/.m2/repository/org/apache/maven/maven-builder-support/3.9.1/maven-builder-support-3.9.1.jar
2025-11-17T19:28:19.5260445Z , [ERROR] urls[17] = file:/home/runner/.m2/repository/org/apache/maven/resolver/maven-resolver-named-locks/1.9.7/maven-resolver-named-locks-1.9.7.jar
2025-11-17T19:28:19.5261164Z , [ERROR] urls[18] = file:/home/runner/.m2/repository/org/apache/maven/shared/maven-shared-utils/3.3.4/maven-shared-utils-3.3.4.jar
2025-11-17T19:28:19.5261902Z , [ERROR] urls[19] = file:/home/runner/.m2/repository/org/eclipse/sisu/org.eclipse.sisu.inject/0.3.5/org.eclipse.sisu.inject-0.3.5.jar
2025-11-17T19:28:19.5262389Z , [ERROR] urls[20] = file:/home/runner/.m2/repository/com/google/inject/guice/5.1.0/guice-5.1.0.jar
2025-11-17T19:28:19.5262872Z , [ERROR] urls[21] = file:/home/runner/.m2/repository/aopalliance/aopalliance/1.0/aopalliance-1.0.jar
2025-11-17T19:28:19.5263441Z , [ERROR] urls[22] = file:/home/runner/.m2/repository/com/google/guava/failureaccess/1.0.1/failureaccess-1.0.1.jar
2025-11-17T19:28:19.5264178Z , [ERROR] urls[23] = file:/home/runner/.m2/repository/org/codehaus/plexus/plexus-interpolation/1.26/plexus-interpolation-1.26.jar
2025-11-17T19:28:19.5265051Z , [ERROR] urls[24] = file:/home/runner/.m2/repository/org/codehaus/plexus/plexus-component-annotations/2.1.0/plexus-component-annotations-2.1.0.jar
2025-11-17T19:28:19.5265766Z , [ERROR] urls[25] = file:/home/runner/.m2/repository/org/apache/commons/commons-lang3/3.8.1/commons-lang3-3.8.1.jar
2025-11-17T19:28:19.5266324Z , [ERROR] urls[26] = file:/home/runner/.m2/repository/commons-io/commons-io/2.20.0/commons-io-2.20.0.jar

/home/runner/.m2/repository/org/apache/commons/commons-lang3/3.8.1/commons-lang3-3.8.1.jar seems very old as I see many newer versions under
https://mvnrepository.com/artifact/org.apache.commons/commons-lang3

[ludoch]

You did a fix for gradle https://github.com/GoogleCloudPlatform/appengine-plugins/pull/1058/files but there is no related update in Maven pom.xml Maven for common-lang3. Maybe this transitive dep is pulled via a very old version.
I think it would be better to add some direct deps and control their versions as opposed to depend on old unmaintained versions.

[ludoch]

Not sure if you are reguraly updating the deps via renovate, but here is all the new versions I could find on the Maven pom file:

--- a/appengine-plugins-core/pom.xml
+++ b/appengine-plugins-core/pom.xml
@@ -77,7 +77,7 @@
     <dependency>
       <groupId>com.google.guava</groupId>
       <artifactId>guava</artifactId>
-      <version>32.1.2-jre</version>
+      <version>33.5.0-jre</version>
       <exclusions>
         <exclusion>
           <!-- To avoid dependency convergence error with gson's errorprone -->
@@ -100,7 +100,7 @@
     <dependency>
       <groupId>com.google.code.gson</groupId>
       <artifactId>gson</artifactId>
-      <version>2.11.0</version>
+      <version>2.13.2</version>
     </dependency>
     <dependency>
       <groupId>org.glassfish</groupId>
@@ -123,7 +123,7 @@
     <dependency>
       <groupId>org.mockito</groupId>
       <artifactId>mockito-core</artifactId>
-      <version>5.5.0</version>
+      <version>5.20.0</version>
       <scope>test</scope>
     </dependency>
     <dependency>
@@ -145,7 +145,7 @@
       <plugin>
         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-jar-plugin</artifactId>
-        <version>3.2.2</version>
+        <version>3.4.2</version>
         <configuration>
           <archive>
             <!-- include manifest produced by maven-bundle-plugin -->
@@ -158,19 +158,19 @@
       <plugin>
         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-surefire-plugin</artifactId>
-        <version>3.1.2</version>
+        <version>3.5.4</version>
       </plugin>
 
       <plugin>
         <groupId>com.coveo</groupId>
         <artifactId>fmt-maven-plugin</artifactId>
-        <version>2.4.0</version>
+        <version>2.13</version>
         <!-- use fmt:format to auto format -->
         <dependencies>
           <dependency>
             <groupId>com.google.googlejavaformat</groupId>
             <artifactId>google-java-format</artifactId>
-            <version>1.7</version>
+            <version>1.31.0</version>
           </dependency>
         </dependencies>
         <configuration>
@@ -221,7 +221,7 @@
       <plugin>
         <groupId>org.jacoco</groupId>
         <artifactId>jacoco-maven-plugin</artifactId>
-        <version>0.8.10</version>
+        <version>0.8.14</version>
         <executions>
           <execution>
             <id>initialize</id>
@@ -242,12 +242,12 @@
        <plugin>
         <groupId>org.apache.maven.plugins</groupId>
         <artifactId>maven-enforcer-plugin</artifactId>
-        <version>3.0.0-M3</version>
+        <version>3.6.2</version>
         <dependencies>
           <dependency>
             <groupId>com.google.cloud.tools</groupId>
             <artifactId>linkage-checker-enforcer-rules</artifactId>
-            <version>0.2.0</version>
+            <version>0.3.0</version>
           </dependency>
         </dependencies>
         <executions>
@@ -290,7 +290,7 @@
         -->
         <groupId>com.google.cloud.artifactregistry</groupId>
         <artifactId>artifactregistry-maven-wagon</artifactId>
-        <version>2.2.3</version>
+        <version>2.2.5</version>
       </extension>
     </extensions>
   </build>
@@ -307,7 +307,7 @@
           <plugin>
             <groupId>org.apache.maven.plugins</groupId>
             <artifactId>maven-compiler-plugin</artifactId>
-            <version>3.11.0</version>
+            <version>3.14.1</version>
             <configuration>
               <!-- For Java 8, the plugin is overridden below, without errorprone -->
               <source>1.8</source>
@@ -395,7 +395,7 @@
           <plugin>
             <groupId>org.sonatype.plugins</groupId>
             <artifactId>nexus-staging-maven-plugin</artifactId>
-            <version>1.6.13</version>
+            <version>1.7.0</version>
             <extensions>true</extensions>
             <configuration>
               <serverId>sonatype-nexus-staging</serverId>
@@ -418,7 +418,7 @@
           <plugin>
             <groupId>org.apache.maven.plugins</groupId>
             <artifactId>maven-gpg-plugin</artifactId>
-            <version>3.2.7</version>
+            <version>3.2.8</version>
             <executions>
               <execution>
                 <id>sign-artifacts</id>
@@ -438,7 +438,7 @@
           <plugin>
             <groupId>org.apache.maven.plugins</groupId>
             <artifactId>maven-javadoc-plugin</artifactId>
-            <version>3.3.2</version>
+            <version>3.12.0</version>
             <configuration>
               <additionalparam>-Xdoclint:none</additionalparam>
             </configuration>
@@ -454,7 +454,7 @@
           <plugin>
             <groupId>org.apache.maven.plugins</groupId>
             <artifactId>maven-source-plugin</artifactId>
-            <version>3.3.0</version>
+            <version>3.3.1</version>
             <executions>
               <execution>
                 <id>attach-sources</id>
@@ -566,7 +566,7 @@
         <plugins>
           <plugin>
             <artifactId>maven-checkstyle-plugin</artifactId>
-            <version>3.3.0</version>
+            <version>3.6.0</version>
             <dependencies>
               <dependency>
                 <groupId>com.puppycrawl.tools</groupId>

I suggest you update these, and try to add a direct dep on lang3

[ludoch]

https://github.com/GoogleCloudPlatform/appengine-java-standard/actions/runs/19447031691

The update inside GAE did not help. I am inclined to ask you do the same in this repo.
Also, see latest version is 3.20 https://archive.apache.org/dist/commons/lang/RELEASE-NOTES.txt and played a lot in the SystemProperties area and for jdk25, so no harm to be up to date and secure everywhere

[ludoch]

2.8.6 seems to fix this issue.
https://github.com/GoogleCloudPlatform/appengine-java-standard/actions/runs/19452021155/job/55658452824?pr=434

Marking as closed.

Now, all my comments regarding obsolete versions seen via renovate should be address so that such issue does not happen again.

[ludoch]

For ex: #1022 should really be integrated or updated.