Solution Corruption...

[Providence1]

Say you’re working diligently on your development environment and some intrepid phone technician pulls the power to your machine. You freak out initially but once you’ve had chance to check everything you see that all your methods were saved. Everything looks okay.

You don’t think anything about it. You push out a new solution. Then some users start to complain that methods that used to fire and perform a certain task are no longer performing that task. You monkey around with things. You see some strange behavior where things don’t fire, then they do fire, even though you haven’t done anything really except for troubleshoot with some application.outputs, watching some globals fill, etc.

But the strangeness continues randomly with methods that you had open during the crash. Would you then start to think that your solution was corrupt? Is it even possible to corrupt a solution by pulling the power on it?

[jcompagner]

that shouldnt matter as far as i know, but maybe there is one catch,
If you have open editors with changes you will loose it. But this will not corrupt anything, you just lost some work.
But maybe if you are in the middle of a save that that file can be corrupted.
And maybe if you do use save all (and you have many open changed editors) and just in the middle of that a solution could be a bit out of sync. Because parts are changed, some arent…
But both events (1 or more files) will not happen that easy, because what is the chance of the power cut just at that same moment you press save all?

Normally the solution should be in the saved state that you had before the power cut.

[david]

Providence1:
…Is it even possible to corrupt a solution by pulling the power on it?

Ahhh, I fondly remember the days of Servoy developer running off of a repository database. It was frickin’ bullet proof. Never, EVER had to worry about Servoy not working.

Fast forward to now. You have Eclipse, you have some 3rd party Servoy plugins, you have some 3rd party Eclipse plugins, and a bazillion solution files. All living on…guess what: a hard drive.

Pull the plug and let’s do the math: one Eclipse directory with a saved state, one workspace with a saved state, possible Servoy plugins that save additional files somewhere, possible Eclipse plugins that save files somewhere, and a whole bunch of files that define your solution (yes, this is one solution):

[attachment=0]Picture 3.png[/attachment]

The chances of something being weird when you plug back in…last time I checked there were whole IT industries dedicated to this possibility. It’s a file system, it’s Eclipse, it’s plugins all trying to work together…times the amount of files you are depending on to work. So saying it “shouldn’t matter” is a really big understatement in my opinion. Every time Servoy crashes on us we cross our fingers while Servoy restarts.

If you think I am being overly dramatic, I was just a couple of hours away from finishing a fairly intensive update this past saturday night when I decided to take a break and do some fun stuff. Fun stuff involved checking out a plugin on ServoyForge, pulling down the new Servoy Commons and Security frameworks, testing out PostgreSQL, and even testing out the new Sybase 12 (now that’s a winner saturday night for you…).

Early sunday morning before heading out for a mountain bike race, I fire up Servoy. Debugger hangs. It wasn’t until Tuesday evening, about 40 fresh servoy installs (it was like resetting a windows operating system nightmare), and definitely no bike race later before we got everything working again. And we’re the guys who wrote the book on Servoy “weirdness” on the mac.

Moral of the story: Servoy runs on a file system now manipulated by Eclipse and all that that is. Recommendation – don’t trust its stability. And if an “intrepid phone technician pulls the power” thing happens…it’s not a matter of “shouldn’t” – it’s a matter of “when”. And when the “when” happens, it really is not fun.

In my opinion the many benefits of Servoy running on Eclipse comes with a steep price sometimes. If it weren’t for Solution Model and subversion I think we would still be doing the bulk of our work with version 3.5 still. For all intents and purposes we completely skipped version 4. Shrug…not sure what the answer is. Maybe it has just been a bad Servoy week for me

[ROCLASI]

Providence is talking about Servoy 3.5.x (he didn’t say so in his post but I know his situation)
So this is WITH a repository database.

[jcompagner]

i never have problems with the actual data in the workspace in my eclipse. When it hangs or when it crashes (now and then it can happen that a real jvm crash did happen)
Eclipse always just starts up.
I never loose or corrupt java files (only not saved changes)

The same is true for servoy. everything that is saved like frm files or js files you name it, will not be corrupted…

My java eclipse or servoy if i would on purpose kill it, and start it up. I can assure you that for me it will not be corrupted in anyway.

Yes there are cases here on the forum that servoy couldnt be started again, but i pointed out (time and time again!!) that it is subversive svn plugin that does that and i pointed out time and time again that you should use subclipse to get rid of that.

Also your file system thing is even WORSE for a database (or could be worse depending on the db)
why? Because the DB is also on that same filesystem, and some used to write one big file (like sybase) what happens if you have power cut RIGHT at the time sybase tries to save data?
Your whole DB could be destroyed or in an invalid state!

For a workspace in eclipse this just cant be true. There are so many files that are written and stored on disk, and maybe only the one you just saved can corrupt itself.

On the case that the metadata of the workspace is really corrupt. Then yes you have a bit more work, but you will not loose any work.

For this, just make an export of your preferences (and copy some other stuff if you want)

Then delete in you corrupted workspace dir the .metadata dir.
Start servoy, point to that workspace dir. It will not see it as new, import your preferences then do file import all your projects again. And you are pretty much up and running again in a few minutes…

But a corrupt workspace i have to see, my oldest workspace that i use (i use quite a lot , lot of branch workspaces for servoy or wicket and so on) is years and years and years old
constantly upgraded from eclipse to eclipse…
And yes i had some hard crashes of eclipse on that workspace also…

[ROCLASI]

jcompagner:
Yes there are cases here on the forum that servoy couldnt be started again, but i pointed out (time and time again!!) that it is subversive svn plugin that does that and i pointed out time and time again that you should use subclipse to get rid of that.

That is my experience also.
I even made a small screencast to show how to move from subversive to subclipse.
http://guides.roclasi.com/subversive2subclipse/
Also don’t use SVNKit but use the native tools (on Mac and Linux this means you have to install the Subversion tools).

jcompagner:
Also your file system thing is even WORSE for a database (or could be worse depending on the db)
why? Because the DB is also on that same filesystem, and some used to write one big file (like sybase) what happens if you have power cut RIGHT at the time sybase tries to save data?
Your whole DB could be destroyed or in an invalid state!

Well if your database has a 100% ACID implementation then you would always end up with non-corrupted data. Of course the database can be in an invalid state. In that situation you simply ‘recover’ the database to put it in a proper state (with PostgreSQL this happens automatically when you start the database server again) and you are ready to go.
Mind you, with recover I don’t mean rebuilding the database but just finishing (when possible) or rolling back any unfinished writes.
A 100% ACID implementation makes sure that data is really and fully written onto the platter of the harddrive (or the silicon in an SSD).

Of course when hardware fails then you can support ACID all you want but you will loose data then anyway.

Johan, are you suggesting that Sybase iAnywhere is not fully ACID compliant ?

[jcompagner]

i have no idea, i was just saying that files can get corrupt, especially with 1 big file. (so it was just an example)

Problem with ACID is that the the OS really need to be sure that it is written to the disk
I dont know the current state of SATA but at least before (PATA/IDE) an OS couldnt force the disk that it should write everything.
So even the os could think everything is written, but the drive did still have small caches.

In those days it was always said, if you want that kind of level you have to use SCSI (that protocol has that build in)
But it could be that SATA did fix that, that you now can have guarantees of the write instead of semi onces…

when a real hard power cut does happen i think anything can go wrong thats why there are UPS

[ROCLASI]

From what I read on the web is that not all SATA drives support this. Serial Attached SCSI (SAS) and SCSI do support write-thru.
And of course it’s also up to the driver to support this.

So I guess we can conclude that the answer to the original question of this thread is, Yes it’s possible.
I guess providence should (in any case) rollback/delete the last pushed out release on the production server and then use a backup (or copy) from this production repository to continue developing on (of course on a development machine, not the production server).

[david]

Our experience with Servoy 3 basically mirrors what you are saying Robert. The only time we had issues was with outright hard drive failures using MySQL InnoDB storage engine set to a separate file per table (we still deploy Servoy 5 Server this way). Our experience over several years with Sybase on a Mac was that it tended to get a little wacky at times, sometimes for no apparent reason (that we could figure out anyways).

So I’m not convinced that Eclipse’s file-based system is better than the prior repository-based setup. Seems to me we’re one step closer to the failure point now and we have a lot more potential failure points.

But I agree with you Johan, it’s not like we lose data when an issue happens. The files are all still there and maybe a few unsaved changes are lost. It’s just getting everything working again that is the problem. Searching around and deleting .meta files really isn’t an intuitive process if you’re just used to things working all the time

The point about switching to subclipse is well taken. And we are way late in getting that done. So no more complaining from me until we get our act together on that score and work with that for a couple of months. If that irons out 90% of things, I’ll come back and say I was dead wrong.

After a month on 5.1.4, it is noticeably smoother than the 5.0.1 release we were deployed at. It was our last remaining 5.0.1 instance that I blew up over the weekend. And we literally couldn’t get it back up and running. Moving to 5.1.4 was the fix. So possible my complaining is on a dead issue already…my apologies if that is the case.

[Thomas_Parry]

[off topic]
@David

So no more complaining from me until we get our act together on that score and work with that for a couple of months.

The timer is ticking…

(I could not resist)

[jcompagner]

david:
The point about switching to subclipse is well taken. And we are way late in getting that done. So no more complaining from me until we get our act together on that score and work with that for a couple of months. If that irons out 90% of things, I’ll come back and say I was dead wrong.

or upgrade to 5.2
i think that will also fix it because that has the latest 3.5.2 code of eclipse and i take that by now they did fix the subversion bug…