Hi all,
I upgraded a system to 2020.03.01 LTS and now my solution refuses to deploy on Tomcat 9 Ubuntu. I upload the WAR file via the Tomcat manager, but the manager page never refreshes after deployment. I can see the war has been unpacked when I look in the web apps directory.
The syslog shows:
05-Sep-2020 11:10:01.160 INFO [https-openssl-nio-8443-exec-5] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/var/lib/tomcat9/webapps/myappname.war]
05-Sep-2020 11:10:03.324 INFO [https-openssl-nio-8443-exec-5] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
The same happens on 2020.06.01…
I have re-deployed my 2020.03 for now while investigating the issue…
Check both tomcat logs and servoy logs if they were created (by default in [user_home_dir]/.servoy/server/[your_context_name]/…).
I have similar problems with 2020.6.1
- deploying war without solution works
- importing the solution seems to work, but can’t be opened
- tomcat-manager won’t respond
This is only the case with the ng-client-version of our app, the old web client works without problems.
I attached both log files.
servoy_log.txt (13.8 KB)
catalina.2020-09-08.txt (50 KB)
we need if possible stackdumps of that (if the admin page is accessible do it from there, else you need a tool like jstack or jvisualvm)
the only thing is see in the servoy log that index creation are going wrong (of the i18n message table we try to make an index called: i18n_messages_m_l)
(and that shouldn’t really matter, can be ignored if its already there)
I can’t get jstack to connect to the jvm on the ubuntu-system, but luckily I have a windows machine with the same symptoms. I hope that works as well…
stackdump.txt (28.2 KB)
that looks like this: https://support.servoy.com/browse/SVY-15236
is fixed for 2020.9_rc, please test this release to see if this helped.
for one that are on the lts release, the same patch is also applied to that branch, so who is on the latest lts branch installation can use : https://build.servoy.com/latest/servoy_ … update.zip
Thank you for looking into this, I tried patching my 2020.03.1 LTS by dropping in the files from https://build.servoy.com/latest/servoy_ … update.zip, but deploy failed again.
This may be my patching going wrong, I did not see anything like this in the catalina logs when my deploy was failing.
08-Sep-2020 21:13:20.880 INFO [https-openssl-nio-8443-exec-14] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/var/lib/tomcat9/webapps/mysolution.war]
08-Sep-2020 21:13:20.884 WARNING [https-openssl-nio-8443-exec-14] org.apache.catalina.startup.SetContextPropertiesRule.begin [SetContextPropertiesRule]{Context} Setting property 'clearReferencesStatic' to 'true' did not find a matching property.
08-Sep-2020 21:13:22.051 WARNING [https-openssl-nio-8443-exec-14] org.apache.tomcat.util.descriptor.web.WebXml.orderWebFragments Used a wrong fragment name [log4j] at web.xml absolute-ordering tag!
08-Sep-2020 21:13:22.800 SEVERE [https-openssl-nio-8443-exec-14] org.apache.catalina.startup.HostConfig.deployWAR Error deploying web application archive [/var/lib/tomcat9/webapps/mysolution.war]
java.lang.IllegalStateException: Error starting child
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:716)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:690)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:695)
at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:978)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:493)
at org.apache.catalina.startup.HostConfig.check(HostConfig.java:1643)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.tomcat.util.modeler.BaseModelMBean.invoke(BaseModelMBean.java:289)
at java.management/com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:809)
at java.management/com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
at org.apache.catalina.manager.ManagerServlet.check(ManagerServlet.java:1584)
at org.apache.catalina.manager.HTMLManagerServlet.upload(HTMLManagerServlet.java:294)
at org.apache.catalina.manager.HTMLManagerServlet.doPost(HTMLManagerServlet.java:212)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.filters.CsrfPreventionFilter.doFilter(CsrfPreventionFilter.java:136)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:200)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:607)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:668)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:834)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1415)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Catalina].StandardHost[localhost].StandardContext[/mysolution]]
at org.apache.catalina.util.LifecycleBase.handleSubClassException(LifecycleBase.java:441)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:198)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:713)
... 42 more
Caused by: java.lang.NoClassDefFoundError: org/json/JSONObject
at java.base/java.lang.Class.getDeclaredMethods0(Native Method)
at java.base/java.lang.Class.privateGetDeclaredMethods(Class.java:3166)
at java.base/java.lang.Class.getDeclaredMethods(Class.java:2309)
at org.apache.catalina.util.Introspection.getDeclaredMethods(Introspection.java:133)
at org.apache.catalina.startup.WebAnnotationSet.loadMethodsAnnotation(WebAnnotationSet.java:285)
at org.apache.catalina.startup.WebAnnotationSet.loadApplicationFilterAnnotations(WebAnnotationSet.java:110)
at org.apache.catalina.startup.WebAnnotationSet.loadApplicationAnnotations(WebAnnotationSet.java:68)
at org.apache.catalina.startup.ContextConfig.applicationAnnotationsConfig(ContextConfig.java:330)
at org.apache.catalina.startup.ContextConfig.configureStart(ContextConfig.java:771)
at org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:301)
at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5034)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
... 43 more
Caused by: java.lang.ClassNotFoundException: org.json.JSONObject
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1363)
at org.apache.catalina.loader.WebappClassLoaderBase.loadClass(WebappClassLoaderBase.java:1186)
... 56 more
08-Sep-2020 21:13:22.807 INFO [https-openssl-nio-8443-exec-14] org.apache.catalina.startup.HostConfig.deployWAR Deployment of web application archive [/var/lib/tomcat9/webapps/mysolution.war] has finished in [1,927] ms
08-Sep-2020 21:13:36.980 SEVERE [https-openssl-nio-8443-exec-23] org.apache.catalina.startup.ExpandWar.copy Error copying [/tmp/0-mysolution] to [/tmp/1-mysolution]
java.io.FileNotFoundException: /tmp/0-mysolution (No such file or directory)
at java.base/java.io.FileInputStream.open0(Native Method)
at java.base/java.io.FileInputStream.open(FileInputStream.java:219)
at java.base/java.io.FileInputStream.<init>(FileInputStream.java:157)
at org.apache.catalina.startup.ExpandWar.copy(ExpandWar.java:276)
at org.apache.catalina.startup.ContextConfig.antiLocking(ContextConfig.java:709)
at org.apache.catalina.startup.ContextConfig.beforeStart(ContextConfig.java:747)
at org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:303)
at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123)
at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:424)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:182)
at org.apache.catalina.manager.ManagerServlet.start(ManagerServlet.java:1409)
at org.apache.catalina.manager.HTMLManagerServlet.start(HTMLManagerServlet.java:698)
at org.apache.catalina.manager.HTMLManagerServlet.doPost(HTMLManagerServlet.java:223)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:660)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.filters.CsrfPreventionFilter.doFilter(CsrfPreventionFilter.java:136)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:200)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:607)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:139)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:668)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:74)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:408)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:834)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1415)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:834)
Is your Servoy installation on Win, Linux or Mac? (applying update zip on Mac is a bit special)
When you dropped the update zip on top of existing installation you did it so that it overwrites existing files right?
After you do that you have to re-export the war from developer.
Also try to clean/undeploy failed war deployments on that (Ubuntu) Tomcat before deploying the new war.
Andrei Costescu:
Is your Servoy installation on Win, Linux or Mac? (applying update zip on Mac is a bit special)
When you dropped the update zip on top of existing installation you did it so that it overwrites existing files right?
I unpacked and replaced the application_server beans, lib and plugins directories with the ones in the zip (Thinking now that maybe I have accidentally deleted some files which were not updated by doing it this way).
Then I dropped the developer/plugins into Eclipse.app/Contents/Eclipse/plugins.
I will try it again now in a way which does not overwrite directories.
Andrei Costescu:
After you do that you have to re-export the war from developer.
Yes, I did.
Andrei Costescu:
Also try to clean/undeploy failed war deployments on that (Ubuntu) Tomcat before deploying the new war.
I un-deployed, but did not check the actual directory or .war had been cleaned out.
My developer now says “2020.3.1.3563_LTS” after my second attempt at patching.
The solutions runs fine via Developer, but when deploying on Tomcat I get the same
09-Sep-2020 10:29:24.544 INFO [https-openssl-nio-8443-exec-28] org.apache.catalina.startup.HostConfig.deployWAR Deploying web application archive [/var/lib/tomcat9/webapps/mysolution.war]
09-Sep-2020 10:29:26.841 INFO [https-openssl-nio-8443-exec-28] org.apache.jasper.servlet.TldScanner.scanJars At least one JAR was scanned for TLDs yet contained no TLDs. Enable debug logging for this logger for a complete list of JARs that were scanned but no TLDs were found in them. Skipping unneeded JARs during scanning can improve startup time and JSP compilation time.
in the Catalina logs.
I’m thinking of restoring the 2020.03 version to get an update to my solution deployed.
We need a stack dump like the one of above
I have to check something - one more fix might be needed for that case. But a new stack dump can confirm where the problem is yes.
I tried upgrading to 2020.09rc, and my problems persisted.
I’m in contact with Patrick Ruhsert, he got the attached stackdumps + a backup of the repository database, he sad someone is looking into it, but I assume you are in contact?
Stackdump_2020-09rc_after_restart_tomcat.txt (25.9 KB)
Stackdump_2020-09rc_after_import.txt (105 KB)
Yes. I talked to Patrick. Thanks for the info.
Just to confirm, reverting to plain 2020.03 solves the issue and I can deploy as normal.
Since my Servoy never starts I cannot get a stack trace from the Developer and I don’t really have the expertise to do it via Java.
I can reproduce it now and I saw the changes that caused it.
I will post here once the bug is fixed.
Good to know and thanks for the the info.
It is fixed. See case SVY-15236 comment for more info.
Please confirm that it fixes you problems both on 2020.03.1 LTS (Christian) and on 2020.09 rc (Robert) using the new servoy_update.zip files from https://build.servoy.com/latest/servoy_lts/ and https://build.servoy.com/latest/servoy_release/.