Prepared statement with datetime value

Given the following piece of code

for (var _i=0; _i<100; _i++) {
	var _df1 = utils.dateFormat(_dt1, "yyyy-MM-dd HH:mm:ss")
	var _df2 = utils.dateFormat(_dt2, "yyyy-MM-dd HH:mm:ss")

	// find/search
	_fs.find()
	_fs.start_date = "<="  + _df2 + "|yyyy-MM-dd HH:mm:ss"
	_fs.end_date = ">="  + _df1 + "|yyyy-MM-dd HH:mm:ss"
	_fs.search()

	// getDataSetByQuery with prepared statement
	var _sql = "select glb_employee_working_time_id from glb_employee_working_time "
	var _where1 = "where start_date <= ? and end_date >= ? and owner_id ='" + globals.owner_id + "'"
	var _args1 = [_dt2, _dt1]
	databaseManager.getDataSetByQuery(globals.db_server, _sql+_where1, _args1, -1)

	// fill it in yourself
	var _where2 = "where start_date <= '"+_df2+"' and end_date >= '"+_df1+"' and owne_id ='" + globals.owner_id + "'"
	databaseManager.getDataSetByQuery(globals.db_server, _sql+_where2, null, -1)
}

I get the following performance data
[attachment=0]performance_data.png[/attachment]

Seems that getting datetime values in prepared statements is very time consuming. I’m using Servoy 5.2.18 and a postgresql 8.4.4 database. Is this behaviour very different in Servoy 7.4.x/postgresql 9.4.x?

You test this now in 1 method in the same loop (first by using prepared statement and right after that, filling it in manually)
Maybe postgreSQL is doing some optimization?

Can you try to test, by splitting the two ways up in two different methods?
fire them after eachother and than look at the performance page

Already did that. Code is just an example. I was triggered because my find/search was slow. After replacing the find search by a getDataSetByQuery with a prepared statement it was a bit less slow, after replacing this by a getDataSetByQuery filling in the arguments manually it was a lot faster.

When I execute this in a different order and in seperately run methods results are the same.
do it yourself:
[attachment=2]doityourself.png[/attachment]
getdatasetbyquery with prepared statement:
[attachment=1]getdatasetbyquery.png[/attachment]
find/search:
[attachment=0]findsearch.png[/attachment]

hmm, interesting…
Maybe Servoy could comment on this…

It’s discussed at long length all over the internet: the general idea is that PreparedStatement are faster when you run them multiple time, because they are compiled before execution. Simple Statement ate faster for one time execution.
It’s driver and database dependant of course but simple Statements (a String) are faster for one time deal because there’s no extra step to compile them, but if you run them multiple time (in a loop for example) then PreparedStatement will perform better because the database will have compiled the query plan and will only change the parameters… So in your case, the query performs better because it’s a one time deal. If you run it inside a loop, in the end the PreparedStatement will work best (how much time depends on the driver and database, so YMMV)…

Nobody said prepared statements are generally faster. I guess you can say that the more complex your query is, the more you benefit from a prepared statement performance wise. One advantage of a prepared statement is that the DB can prepare and keep an execution plan, but obviously, the execution plan has to be rather generic, since the arguments can be very different every time. So for simple statements, that advantage can be a disadvantage performance wise.

ptalbot:
…So in your case, the query performs better because it’s a one time deal. If you run it inside a loop, in the end the PreparedStatement will work best (how much time depends on the driver and database, so YMMV)…

This was run in a loop, not only the find/search fields were the same also the values were the same every iteration! So if I follow you we would expect the prepared statement (find/search) to be faster? It’s 20 times slower…

patrick:
… So for simple statements, that advantage can be a disadvantage performance wise.

So it has nothing to do with datetime values? I haven’t seen this behaviour with for instance integer fields (but have to admit that I haven’t researched that well enough yet)

I don’t think so, no.

Prepared statements tend to be faster because the query planner only have to plan them once (as both Patricks already mentioned).
Like I explained in my talk at ServoyWorld the planner uses meta-data about your data to see if you are looking for a common value or a more unique value (and other characteristics).
So using prepared statements the planner doesn’t have all the info on what you are searching for so you get a more generic and could be a less optimal plan.

Another thing prepared statements also give is the benefit of secure inputs. Just concatenating (possible) arbitrary values into your SQL might open you up for SQL injection. Of course all this depends on your use-case. And you could sanitize your inputs yourself before concatenation but that will take up some processing time as well.

To see what is really going on you might want to run some EXPLAINs on these queries.
You could run your SQL in a query editor and check what the query plan is for each value (and the query times) using the following syntax:

Using plain SQL:

EXPLAIN 
SELECT glb_employee_working_time_id 
FROM glb_employee_working_time 
WHERE start_date <= '<yourdateval1>' AND end_date >= '<yourdateval2>' AND owner_id='<yourownerid>'

For a prepared statement you need to do more steps. First you need to create the prepared statement like so:

PREPARE myplan (timestamp, timestamp, character varying) AS
SELECT glb_employee_working_time_id 
FROM glb_employee_working_time 
WHERE start_date <= $1 AND end_date >= $2 AND owner_id=$3;

So now we have the prepared statement with the name ‘myplan’ we can explain this using passed values like so:

EXPLAIN EXECUTE myplan ('<yourdateval1>','<yourdateval2>','<yourownerid>');
EXPLAIN EXECUTE myplan ('<yourdateval3>','<yourdateval4>','<yourownerid>');
-- etc.

Afterwards you may want to remove the prepared statement from memory like so:

DEALLOCATE myplan

(for more info on using prepared statements in your query editor see here)

Hope this helps.

One side-note about your 8.4.4. install. I hope this is not a production environment and just the Developer bundled Pg install.
First of all PostgreSQL 8.4 is EOL since July 2014 and the latest minor update was 8.4.22 (!) It’s very much recommended to keep your version up to date with the latest minor version because of security- and/or bug fixes.
See this link for the PostgreSQL version policy.

I know Servoy bundles PostgreSQL with Developer but doesn’t provide minor updates for it. It’s one of the reasons to actually run the community distribution instead. The installer allows you to do this using the EDB installer. Enterprise DB (EDB) has indeed a commercial Postgres product but they also employ some community members that create the community installers. EDB hosts these as well.
Updating this distribution is just a matter of running the installer and seconds later you have your PostgreSQL instance running the latest minor version.
Also be aware that Servoy will not start or shutdown this version since it will be running as a service.

I hope this clears things up.

besides that these timings are really still way way to fast to really tell something about it (it should be at least in the many seconds instead of just some milli seconds)

What happens if you first do a run that you don’t count, and then do another run (a larger one) that you do count?
So that everything is already cached and prepared?

jcompagner:
besides that these timings are really still way way to fast to really tell something about it…

In our application there is a delay of several seconds using the find/search and there is a delay of tenths of seconds using the self filled in getDataSetByQuery. This is a huge difference, the user will notice that…

Hi Michel,

You say there is a difference of seconds in your solution?
Are you also looping to query things like in your example code? Also your example code shows only a difference of 1 second over 100 queries between the fastest method and the slowest.
What are you doing differently than the above code example?

In our applications there are a lot more than 100 queries with for each query different arguments. There is also one argument more, an integer field glb_employee_id. Also end_date can be empty (or end_date is null) in the query.

Hi Michel,

So if I understand you correctly you are NOT using loops like in your example code but you want to make your solution faster overall. And that you see a (small) difference between the different query approaches.
In that case you need to do some profiling like I suggested in my first post in this thread. See if it’s in the SQL (and plan), if not then the slowdown is inside Servoy.

it seems that the difference is already in the prepared or not (the 2 dataset by query, 1 with arguments filled in 1 with arguments as parameters)

So i really would do an explain on both somehow (not sure if you can do an examplain with ? in query??)

jcompagner:
(not sure if you can do an examplain with ? in query??)

Yes, see my first post in this thread. ;)

PREPARE myplan (timestamp, timestamp) AS
SELECT glb_employee_working_time_id
FROM glb_employee_working_time
WHERE start_date <= $1 AND end_date >= $2;
EXPLAIN ANALYZE EXECUTE myplan ('2015-11-03', '2015-11-03');

[attachment=0]prepared.png[/attachment]

EXPLAIN ANALYZE SELECT glb_employee_working_time_id
FROM glb_employee_working_time
WHERE start_date <= '2015-11-03' AND end_date >= '2015-11-03';

[attachment=1]normal.png[/attachment]
I know this is the first time the prepared statement is called, so there is overhead. But could it be (in my case) that Servoy keeps preparing and dealllocating prepared statements?

Hi Michel,

Please run the queries 2-3 times and record the last one. That is better way to compare because then the data will be in shared memory.
Or else you are comparing apples and oranges.

Also please show the full output, the actuals are cut off.