Somehow, on starting up in the last day or so, all of the postgresql servers are disabled. I tried enabling them one by one by right-clicking. The small red “x” icons went away, but the problems tab still reports four instances of “server not accessible (first occurrence error)”.
If I right-click and go into the individual servers, and try checking the un-checked “Enabled” button, it gives me the “Connection refused. Check that the hostname and port are correct…” error.
There is a series of crash reports in ~user/library/logs/diagnosticReports, with titles like
postgres_2011-08-09-133540_Ladue-MacBook-Pro
I’ve attached one as an example.
Does anybody know what might be the problem, and why it just suddenly happened?
Can you give us more information? Is this the bundled PostgreSQL or the EnterpriseDB installed version ?
Did this happen after a reboot of the machine ? Was it rebooted before and did PostgreSQL come up then or was this the first time you rebooted after installation.
Do you see anything in the postgres log.
I had been working and everything seemed fine. Then just today, when Servoy started up, all the database entries were disabled. I tried re-booting the machine, to no avail. I also tried running Servoy 5.2, which is installed into a separate workspace (Jason Meunier had assisted me to install Servoy 6 into a completely separate area). The database servers for 5.2 are still working.
There is a file in Servoy6/application server/postgres_db named “postgres_log.txt”. The first few lines of it are,
FATAL: the database system is starting up
LOG: database system was shut down at 2011-08-03 12:39:26 PDT
LOG: autovacuum launcher started
LOG: database system is ready to accept connections
FATAL: role “djlapin” does not exist
… lines 5 through 63 are the same ("role ‘djlapin’ does not exist’).
Lines 64 through 115 look sort of like repeats of the following, but just on different days,
LOG: received smart shutdown request
LOG: autovacuum launcher shutting down
LOG: shutting down
LOG: database system is shut down
LOG: database system was shut down at 2011-08-03 12:40:33 PDT
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
The remainder of the file (through Line 172) looks like repeats of the following,
LOG: shutting down
LOG: database system is shut down
LOG: database system was shut down at 2011-08-06 00:41:58 PDT
LOG: could not open file “pg_xlog/000000010000000000000002” (log file 0, segment 2): No such file or directory
LOG: invalid primary checkpoint record
LOG: could not open file “pg_xlog/000000010000000000000002” (log file 0, segment 2): No such file or directory
LOG: invalid secondary checkpoint record
PANIC: could not locate a valid checkpoint record
I’ve got the same problem on a Mac OS 10.6.8. Installed Servoy 6 today. 3 databases wouldn’t start, 4 did. Got Aqua Data Studio to connect and built the database I needed and then seemed to work. After closing down this afternoon I tried starting it again tonight and now none of the databases will open including the servoy_repository. Here is my log:
2011-08-16 21:56:47,890 ERROR [Thread-4] com.servoy.j2db.server.Za.Zo - Final get connection failure for server repository_server in 6 times
2011-08-16 21:56:47,892 ERROR [Thread-4] com.servoy.j2db.util.Debug - Throwable
org.postgresql.util.PSQLException: FATAL: the database system is shutting down
at org.postgresql.core.v3.ConnectionFactoryImpl.doAuthentication(ConnectionFactoryImpl.java:291)
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:108)
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:66)
at org.postgresql.jdbc2.AbstractJdbc2Connection.<init>(AbstractJdbc2Connection.java:125)
at org.postgresql.jdbc3.AbstractJdbc3Connection.<init>(AbstractJdbc3Connection.java:30)
at org.postgresql.jdbc3.Jdbc3Connection.<init>(Jdbc3Connection.java:24)
at org.postgresql.Driver.makeConnection(Driver.java:393)
at org.postgresql.Driver.connect(Driver.java:267)
at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
at com.servoy.j2db.Za.Zo.createConnection(Zo.java:3)
at com.servoy.j2db.Za.Zd.createConnection(Zd.java:5)
at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1158)
at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
at com.servoy.j2db.Za.Za.getConnection(Za.java:68)
at com.servoy.j2db.server.Za.Zo.getRawConnection(Zo.java:642)
at com.servoy.j2db.server.Za.Zo.Zf(Zo.java:1532)
at com.servoy.j2db.server.Za.Zo.testConnection(Zo.java:1458)
at com.servoy.j2db.server.main.ApplicationServer.initStandalone(ApplicationServer.java:368)
at com.servoy.j2db.server.starter.StandaloneServerStarter.initFromDeveloper(StandaloneServerStarter.java:1)
at com.servoy.j2db.server.starter.EclipseServerStarter.startApplicationServer(EclipseServerStarter.java:5)
at com.servoy.eclipse.appserver.Activator.startAppServer(Activator.java:158)
at com.servoy.eclipse.core.ServoyModel.startAppServer(ServoyModel.java:478)
at com.servoy.eclipse.core.util.SplashHandler$1.run(SplashHandler.java:52)
at java.lang.Thread.run(Thread.java:680)
2011-08-16 21:56:48,676 ERROR [main] com.servoy.j2db.server.Za.Zo - org.postgresql.util.PSQLException: FATAL: the database system is shutting down
2011-08-16 21:56:48,676 ERROR [main] com.servoy.j2db.util.Debug - Throwable
com.servoy.j2db.persistence.RepositoryException: org.postgresql.util.PSQLException: FATAL: the database system is shutting down FATAL: the database system is shutting down
at com.servoy.j2db.server.Za.Zo.Za(Zo.java:18)
at com.servoy.j2db.server.Za.Zo.Zk(Zo.java:1359)
at com.servoy.j2db.server.Za.Zo.Za(Zo.java:675)
at com.servoy.j2db.server.Za.Zo.getTableNames(Zo.java:692)
at com.servoy.eclipse.core.repository.EclipseUserManager.setFormAndTableChangeAware(EclipseUserManager.java:212)
at com.servoy.eclipse.core.ServoyModel.initialize(ServoyModel.java:2522)
at com.servoy.eclipse.core.ServoyModelManager$1.run(ServoyModelManager.java:71)
at com.servoy.eclipse.core.ServoyModelManager.getServoyModel(ServoyModelManager.java:80)
at com.servoy.eclipse.designer.Activator.start(Activator.java:61)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl$1.run(BundleContextImpl.java:783)
at java.security.AccessController.doPrivileged(Native Method)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.startActivator(BundleContextImpl.java:774)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.start(BundleContextImpl.java:755)
at org.eclipse.osgi.framework.internal.core.BundleHost.startWorker(BundleHost.java:370)
at org.eclipse.osgi.framework.internal.core.AbstractBundle.start(AbstractBundle.java:284)
at org.eclipse.osgi.framework.util.SecureAction.start(SecureAction.java:417)
at org.eclipse.osgi.internal.loader.BundleLoader.setLazyTrigger(BundleLoader.java:265)
at org.eclipse.core.runtime.internal.adaptor.EclipseLazyStarter.postFindLocalClass(EclipseLazyStarter.java:106)
at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClass(ClasspathManager.java:453)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.findLocalClass(DefaultClassLoader.java:216)
at org.eclipse.osgi.internal.loader.BundleLoader.findLocalClass(BundleLoader.java:393)
at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:469)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:422)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:410)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultClassLoader.java:107)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Just an update. Restarted the computer and now 4 of the 5 databases started up fine but svy_framework did not which I created this afternoon. Below is the current error in the log. Any help greatly appreciated!
2011-08-16 22:34:09,607 ERROR [pool-2-thread-2] com.servoy.j2db.dataprocessing.Zb - Error flushing message buffer to client D78EC47C-4450-4475-9633-7DAEAC8BA227
java.util.concurrent.RejectedExecutionException
at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:1768)
at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:767)
at java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:215)
at java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:397)
at com.servoy.j2db.util.ServoyScheduledExecutor.schedule(ServoyScheduledExecutor.java:121)
at com.servoy.j2db.dataprocessing.Zb.Za(Zb.java:227)
at com.servoy.j2db.dataprocessing.ClientProxy.Zc(ClientProxy.java:117)
at com.servoy.j2db.dataprocessing.Zo.run(Zo.java:11)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
2011-08-16 22:36:51,449 ERROR [main] com.servoy.j2db.server.Za.Zo - org.postgresql.util.PSQLException: FATAL: database "svy_framework" does not exist
2011-08-16 22:36:51,453 ERROR [main] com.servoy.j2db.util.Debug - Throwable
com.servoy.j2db.persistence.RepositoryException: org.postgresql.util.PSQLException: FATAL: database "svy_framework" does not exist FATAL: database "svy_framework" does not exist
at com.servoy.j2db.server.Za.Zo.Za(Zo.java:18)
at com.servoy.j2db.server.Za.Zo.Zk(Zo.java:1359)
at com.servoy.j2db.server.Za.Zo.Za(Zo.java:675)
at com.servoy.j2db.server.Za.Zo.getTableNames(Zo.java:692)
at com.servoy.eclipse.core.repository.EclipseUserManager.setFormAndTableChangeAware(EclipseUserManager.java:212)
at com.servoy.eclipse.core.ServoyModel.initialize(ServoyModel.java:2522)
at com.servoy.eclipse.core.ServoyModelManager$1.run(ServoyModelManager.java:71)
at com.servoy.eclipse.core.ServoyModelManager.getServoyModel(ServoyModelManager.java:80)
at com.servoy.eclipse.designer.Activator.start(Activator.java:61)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl$1.run(BundleContextImpl.java:783)
at java.security.AccessController.doPrivileged(Native Method)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.startActivator(BundleContextImpl.java:774)
at org.eclipse.osgi.framework.internal.core.BundleContextImpl.start(BundleContextImpl.java:755)
at org.eclipse.osgi.framework.internal.core.BundleHost.startWorker(BundleHost.java:370)
at org.eclipse.osgi.framework.internal.core.AbstractBundle.start(AbstractBundle.java:284)
at org.eclipse.osgi.framework.util.SecureAction.start(SecureAction.java:417)
at org.eclipse.osgi.internal.loader.BundleLoader.setLazyTrigger(BundleLoader.java:265)
at org.eclipse.core.runtime.internal.adaptor.EclipseLazyStarter.postFindLocalClass(EclipseLazyStarter.java:106)
at org.eclipse.osgi.baseadaptor.loader.ClasspathManager.findLocalClass(ClasspathManager.java:453)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.findLocalClass(DefaultClassLoader.java:216)
at org.eclipse.osgi.internal.loader.BundleLoader.findLocalClass(BundleLoader.java:393)
at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:469)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:422)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:410)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultClassLoader.java:107)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at org.eclipse.osgi.internal.loader.BundleLoader.loadClass(BundleLoader.java:338)
at org.eclipse.osgi.framework.internal.core.BundleHost.loadClass(BundleHost.java:232)
at org.eclipse.osgi.framework.internal.core.AbstractBundle.loadClass(AbstractBundle.java:1197)
at org.eclipse.core.internal.registry.osgi.RegistryStrategyOSGI.createExecutableExtension(RegistryStrategyOSGI.java:174)
at org.eclipse.core.internal.registry.ExtensionRegistry.createExecutableExtension(ExtensionRegistry.java:904)
at org.eclipse.core.internal.registry.ConfigurationElement.createExecutableExtension(ConfigurationElement.java:243)
at org.eclipse.core.internal.registry.ConfigurationElementHandle.createExecutableExtension(ConfigurationElementHandle.java:55)
at org.eclipse.ui.internal.WorkbenchPlugin$1.run(WorkbenchPlugin.java:267)
at org.eclipse.swt.custom.BusyIndicator.showWhile(BusyIndicator.java:70)
at org.eclipse.ui.internal.WorkbenchPlugin.createExtension(WorkbenchPlugin.java:263)
at org.eclipse.ui.internal.registry.EditorDescriptor.createEditor(EditorDescriptor.java:235)
at org.eclipse.ui.internal.EditorManager.createPart(EditorManager.java:845)
at org.eclipse.ui.internal.EditorReference.createPartHelper(EditorReference.java:609)
at org.eclipse.ui.internal.EditorReference.createPart(EditorReference.java:465)
at org.eclipse.ui.internal.WorkbenchPartReference.getPart(WorkbenchPartReference.java:595)
at org.eclipse.ui.internal.PartPane.setVisible(PartPane.java:313)
at org.eclipse.ui.internal.presentations.PresentablePart.setVisible(PresentablePart.java:180)
at org.eclipse.ui.internal.presentations.util.PresentablePartFolder.select(PresentablePartFolder.java:270)
at org.eclipse.ui.internal.presentations.util.LeftToRightTabOrder.select(LeftToRightTabOrder.java:65)
at org.eclipse.ui.internal.presentations.util.TabbedStackPresentation.selectPart(TabbedStackPresentation.java:473)
at org.eclipse.ui.internal.PartStack.refreshPresentationSelection(PartStack.java:1254)
at org.eclipse.ui.internal.PartStack.handleDeferredEvents(PartStack.java:1222)
at org.eclipse.ui.internal.LayoutPart.deferUpdates(LayoutPart.java:400)
at org.eclipse.ui.internal.PartSashContainer.handleDeferredEvents(PartSashContainer.java:1409)
at org.eclipse.ui.internal.LayoutPart.deferUpdates(LayoutPart.java:400)
at org.eclipse.ui.internal.WorkbenchPage.handleDeferredEvents(WorkbenchPage.java:1420)
at org.eclipse.ui.internal.WorkbenchPage.deferUpdates(WorkbenchPage.java:1410)
at org.eclipse.ui.internal.WorkbenchPage.access$14(WorkbenchPage.java:1401)
at org.eclipse.ui.internal.WorkbenchPage$16.runWithException(WorkbenchPage.java:3304)
at org.eclipse.ui.internal.StartupThreading$StartupRunnable.run(StartupThreading.java:31)
at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:134)
at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:3593)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3286)
at org.eclipse.ui.application.WorkbenchAdvisor.openWindows(WorkbenchAdvisor.java:803)
at org.eclipse.ui.internal.Workbench$31.runWithException(Workbench.java:1567)
at org.eclipse.ui.internal.StartupThreading$StartupRunnable.run(StartupThreading.java:31)
at org.eclipse.swt.widgets.RunnableLock.run(RunnableLock.java:35)
at org.eclipse.swt.widgets.Synchronizer.runAsyncMessages(Synchronizer.java:134)
at org.eclipse.swt.widgets.Display.runAsyncMessages(Display.java:3593)
at org.eclipse.swt.widgets.Display.readAndDispatch(Display.java:3286)
at org.eclipse.ui.internal.Workbench.runUI(Workbench.java:2548)
at org.eclipse.ui.internal.Workbench.access$4(Workbench.java:2438)
at org.eclipse.ui.internal.Workbench$7.run(Workbench.java:671)
at org.eclipse.core.databinding.observable.Realm.runWithDefault(Realm.java:332)
at org.eclipse.ui.internal.Workbench.createAndRunWorkbench(Workbench.java:664)
at org.eclipse.ui.PlatformUI.createAndRunWorkbench(PlatformUI.java:149)
at org.eclipse.ui.internal.ide.application.IDEApplication.start(IDEApplication.java:115)
at org.eclipse.equinox.internal.app.EclipseAppHandle.run(EclipseAppHandle.java:196)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(EclipseAppLauncher.java:110)
at org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(EclipseAppLauncher.java:79)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:369)
at org.eclipse.core.runtime.adaptor.EclipseStarter.run(EclipseStarter.java:179)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.eclipse.equinox.launcher.Main.invokeFramework(Main.java:620)
at org.eclipse.equinox.launcher.Main.basicRun(Main.java:575)
at org.eclipse.equinox.launcher.Main.run(Main.java:1408)
2011-08-16 22:37:00,293 ERROR [Worker-3] com.servoy.j2db.server.Za.Zo - Final get connection failure for server svy_framework in 1 times
2011-08-16 22:49:53,480 ERROR [main] com.servoy.j2db.server.Za.Zo - Final get connection failure for server svy_framework in 1 times
You created this svy_framework database through Servoy or via ADS/PgAdmin ? Also can you check if that database is actually there or not using ADS/PgAdmin?
Thanks for your help! What happened was the first time we launched Servoy 6, three of the databases were ‘whited’ out. When I tried to enable them the message came back about failing. When I tried to create the svy_framework database from within Servoy that also failed. I then was able to create the svy_framework db from ADS and everything seemed fine. After shutting down this afternoon I then tried to relaunch Servoy and this time all the databases including servoy_repository failed. I then sent the first message. Restarted my mac and then all the databases were fine EXCEPT the svy_framework one. Do you think the problem is starting that database originally from outside Servoy? But if so why did it work originally and I was able set it up within Servoy as a new connection? Tried again to restart the Mac and now the same again: all databases fine except the svy_framework. And when I try to connect from ADS I get the same message: something like ‘FATAL: database does not exist’.
Oh and one other thing. When I try to create a new database in PG from Servoy I get the message “Could not create the database : null”. I think the only time I was able to create a database today from Servoy 6 was after I created svy_framework and restarted Servoy. Then I could connect to svy_framework and I tried to create a new database called ‘test’ which worked and I then deleted it.
It’s very odd that the database you created is no longer there. Are you sure you are running the same PostgreSQL instance as before ?
Are you using the Servoy bundled PostgreSQL or the EnterpriseDB version ?
Also to see what database cluster/instance you are using you can run ‘ps aux | grep postmaster’ (without quotes) in the terminal and check the path behind the -D switch, unless you are running Windows of course.
I believe I’m running the same instance. I’m simply starting servoy developer up as usual from the same location in my Servoy 6 folder. I’m using the bundled PostgreSQL. When I run that command I get the following output:
Not sure what/where the path is behind the -D switch. (I am running OS 10.6.. I got the same message about the 3 databases not being there when I ran the Servoy 6 install the first time earlier this afternoon. Visually they are ‘there’ but either ‘whited’ out or with a red ‘x’ through them. Currently it is whited out (disabled) and when I try to enable it, I get the message about the database not being there.
Servoy launches PostgreSQL differently so it shows up like this:
robert 1422 0.0 0.1 2482728 5140 ?? S 9:43AM 0:00.31 /Applications/Servoy6-PG/application_server/postgres_db/bin/postgres -D /Applications/Servoy6-PG/application_server/database -p 5434
So grepping on ‘postgres’ would show it.
Anyway, I just installed Servoy 6 again with Pg and it all works for me fine (and this is on Lion!). I did however use the installer from the USB stick that was given out on the Servoy 6 launchparty (BBQ). I am not sure if that is actually the same built as the download version (one would assume it is). I am running built 1217.
Did you added/updated some JDBC drivers or anything ?
ROCLASI:
I did however use the installer from the USB stick that was given out on the Servoy 6 launchparty (BBQ). I am not sure if that is actually the same built as the download version (one would assume it is). I am running built 1217
When Servoy is installed with the bundled postgres you restart Developer it will ‘softly’ quit postgres.
If you are connected to postgres with another client outside developer, postgres will not quit until that client is disconnected.
When you start developer again while postgres has not quit yet, you will get the error: ‘PSQLException: FATAL: the database system is shutting down’
And all your postgres servers in developer will be invalid.
So make sure that other clients are disconnected when you stop developer.
I think this can be solved by adding the ‘-m fast’ switch to pg_ctl. This will actively close any connection and shuts down PostgreSQL instead of waiting for clients to disconnect.
As a quickfix for anyone reading this you can change the ‘shutdown launcher’ property in Preferences > Servoy > Startup to :
Can you look in the postgres_log.txt, or even post it here (or mail it to me) to see if it shows anything about that database ?
You just created an empty database ? Or did it already contained any objects/data ?
I had been working and everything seemed fine. Then just today, when Servoy started up, all the database entries were disabled. I tried re-booting the machine, to no avail. I also tried running Servoy 5.2, which is installed into a separate workspace (Jason Meunier had assisted me to install Servoy 6 into a completely separate area). The database servers for 5.2 are still working.
There is a file in Servoy6/application server/postgres_db named “postgres_log.txt”. The first few lines of it are,
…
You have some serious issues here. It’s been a week now, do you still have this issue or did you do a reinstall?
If you still have this issue I can dive into this one a little deeper.
