I am currently looking into adding jsunit tests into our solutions. I am wondering if anyone has examples of how they have implemented jsunit tests in their solutions or suggestions for best practices regarding this? One specific question I have is if it is possible to run unit tests at the form level? Currently it looks as though you can only run tests from the solution level.
jwservoy:
One specific question I have is if it is possible to run unit tests at the form level? Currently it looks as though you can only run tests from the solution level.
Currently you can only run jsunit tests at solution level. But we will add the possibility to run them at form level as well - there is already a case for that.
You can find a lot of discussions/articles/blogs about “unit test guidelines/best practices” in general. Most of them (that are not tool-specific) apply to all unit testing situations.
In case of Servoy solutions: keep your tests separate from the tested solution if possible (by adding the tests to a new solution that uses the tested solution as a module). You will probably find “application.updateUI(timeout)” useful in many situations when you trigger something that will run async and you want to wait for it to happen so you can test the results (for example if you use plugins.rawSQL in tests).
Some general guidelines I would like to point out :
- each test method should be independent. They should not rely on being ran in a specific order; use “setUp()” and “tearDown()” methods to provide/cleanup the test environment for each method.
- add unit tests for assumptions you make about external API (when external documentation doesn’t specify exactly one aspect that you need/use) - if possible.
Looking forward to see what others will add here.
Hello,
currently I am also fighting with unit tests for a solution and I detected a very strange behaviour. As it is suggested for unit tests each unit test should run independent from the other unit tests. I followed your recommendation and created a seperate testing solution.
I added the following setUp method:
function setUp() {
application.output("Setting up test");
globals.loadSampleData();
//just to be sure that no transaction is started
databaseManager.rollbackTransaction();
}
The global function loadSampleData reads a file with sql statements.
function loadSampleData() {
var txt = plugins.file.readTXTFile('/.../testdata.sql', 'UTF-8');
var lines = txt.split("\n");
application.output("find " + lines.length + " lines of SQL Commands in the testdata");
plugins.rawSQL.executeSQL("localhost_mysql", "inventar", "START TRANSACTION;")
// execute each line
for (var i = 0; i < lines.length; ++i) {
//only execute line if it contains character
if (lines[i].length > 0) {
var sql = lines[i];
var ret = plugins.rawSQL.executeSQL("localhost_mysql", "test", sql)
if (ret == false) {
var msg = plugins.rawSQL.getException().getMessage(); //see exception node for more info about the exception obj
application.output('SQL exception: ' + msg);
jsunit.fail("Failed importing test data - SQL exception: " + msg);
}
}
}
plugins.rawSQL.executeSQL("localhost_mysql", "test", "COMMIT;")
// ### approaches to reload the data in the client- but not working ! #####
//allways reload all records
var server = plugins.maintenance.getServer("localhost_mysql");
var tables = server.getTableNames();
for (var i = 0; i < tables.length; i++) {
plugins.rawSQL.flushAllClientsCache("localhost_mysql", tables[i]);
}
var formlist = forms.allnames
for (var i = 0; i < formlist.length; i++) {
databaseManager.refreshRecordFromDatabase(forms[formlist[i]].foundset, -1)
var relations = forms[formlist[i]].allrelations;
for (var j = 0; j < relations.length; j++) {
databaseManager.refreshRecordFromDatabase(forms[formlist[i]].allrelations[j], -1)
}
}
}
The problem now is that as it can be read in the comment the client is not using the new imported data. Even more strange the different tests fail randomly for different reasons. After a little bit of investigation I think I could say that the problem is that the test sometimes still use the data which was in the tables before the tables were cleared and data reinserted by the testdata sql-script.
For sure I can say, that my tests run perfectly without reload the testdata before each test.
Any advice on this problem ?
Schoby
P.S.: Runing Servoy 5.2.2 on Linux, with MySQL, and InnoDB engine
When you alter table data via rawSQL, you must use either notifyDataChange or flushAllClientsCache to get Servoy to see the changes.
But even if you do (which you did), it does not happen at once, rather it tells Servoy that things have changed an to update them later (this explains why tests fail randomly).
In your case it’s about adding records. databaseManager.refreshRecordFromDatabase only updates records that are already in use by Servoy. So you can do one of 2 things:
- easy way: use foundset.clear() and foundset.loadAllRecords()
- data-broadcast like way: wait a bit for the changes notified by notifyDataChange/flushAllClientsCache to happen using application.updateUI(timeInMillis). Here you can either use some small value (100, 1000, …) but this can still cause random failures or you could approach it like this to allow it to wait longer, but only if needed (condition parameter could be a function checking foundset size > 0 for example):
/**
* Does application.updateUI(100) until the condition is fullfilled or the given time expires. If condition still doesn't pass it fails the test.
* @param description optional, describes the test done by -condition-
* @param maxTime the maximum time in ms to wait for the condition to pass
* @condition a <b>function</b> object that returns true/false - that tests for the expected test behavior.
*/
function assertTrueInMaxTime(description, maxTime, condition) {
// 'description' is optional
// for example assertTrueInMaxTime("Test description", 5000, function () { return x == 1); });
if (typeof condition == 'undefined') {
condition = maxTime;
maxTime = description;
description = null;
}
var b = condition();
var timeLimit = java.lang.System.currentTimeMillis() + maxTime;
while ((!b) && (java.lang.System.currentTimeMillis() <= timeLimit)) {
application.updateUI(100);
b = condition();
}
if (!b) {
if (description == null) jsunit.fail();
else jsunit.fail(description);
}
}
Hi Andrei,
thanks for your advice -it was leading me into the right direction.
I tried your foundset.clear and foundset.loadAllRecords approach.
So after executing all my SQL statements (which is not only adding data but also clearing all tables before) I do the following:
//allways reload all records
var server = plugins.maintenance.getServer("localhost_mysql");
var tables = server.getTableNames();
for (var i = 0; i < tables.length; i++) {
plugins.rawSQL.flushAllClientsCache("localhost_mysql", tables[i]);
}
//first update all form foundsets
var formlist = forms.allnames
for (var i = 0; i < formlist.length; i++) {
var frm = formlist[i];
application.output(frm);
var fs = forms[frm].foundset
if (forms[frm] != null && fs != null && fs.getDataSource() != null) {
forms[frm].foundset.clear();
forms[frm].foundset.loadAllRecords();
}
}
This works quite fine, but still I face some problems when running tests. All this problems are related to forms with portals and related foundset. As an example I create a new Inventar and test the count of al relates exemplars (which should be 0). This test fails:
jsunit.assertTrue("Error-Count:" + forms.frmInventar.inventar_to_exemplar.EX_INVENTAR_COUNT + "!",forms.frmInventar.inventar_to_exemplar.EX_INVENTAR_COUNT == 0);
The error says that forms.frmInventar.inventar_to_exemplar.EX_INVENTAR_COUNT has no value (“”)
So I tried to reload the data for all relations by doing this - but unfortunately without any success:
//now update all relations
for (var i = 0; i < tables.length; i++) {
var relations = solutionModel.getRelations("localhost_mysql", tables[i])
for (var j = 0; j < relations.length; j++) {
var rel = relations[j];
application.output(rel)
var fs = solutionModel.getRelation(rel)
if (fs != null && fs.getDataSource() != null) {
forms[frm].foundset.clear();
forms[frm].foundset.loadAllRecords();
}
}
}
Regards
Schoby
solutionModel.getRelation(…) returns a JSRelation type, not a foundset (so it doesn’t have getDataSource()) - see solution explorer.
You are also using forms[frm].foundset which is not the related foundset you want.
Did you try using the same approach as you first posted for relations? (based on forms[…].allrelations)
You might end up better using the second solution I posted (with updateUI) instead of hunting down all foundsets.
Hi Andrei,
sorry for the late reply - but I was a little busy the last week.
So somewho I was not able to iterate over all relations and I followed the second approach.
I used your function assertTrueInMaxTime but still I am facing some problems.
So perhaps my testcase is wrong and I want to describe my setup here in detail.
- assertTrueInMaxTime
/**
* Does application.updateUI(100) until the condition is fullfilled or the given time expires. If condition still doesn't pass it fails the test.
* @param description optional, describes the test done by -condition-
* @param maxTime the maximum time in ms to wait for the condition to pass
* @condition a <b>function</b> object that returns true/false - that tests for the expected test behavior.
*
* @properties={typeid:24,uuid:"1D05F438-3D4D-469D-8BA6-DF85723DEFB4"}
*/
function assertTrueInMaxTime(description, maxTime, condition) {
// 'description' is optional
// for example assertTrueInMaxTime("Test description", 5000, function () { return x == 1); });
if (typeof condition == 'undefined') {
condition = maxTime;
maxTime = description;
description = null;
}
var b = condition();
var timeLimit = java.lang.System.currentTimeMillis() + maxTime;
while ( (!b) && (java.lang.System.currentTimeMillis() <= timeLimit)) {
application.updateUI(100);
b = condition();
}
if (!b) {
if (description == null) jsunit.fail();
else jsunit.fail(description);
}
}
- My function for waiting/checking for the update (yes I know this could be written more compact):
function checkfrmInventarInventar_to_exemplarEX_INVENTAR_COUNT() {
if (forms.frmInventar.inventar_to_exemplar == 'undefined')
return false;
else if (forms.frmInventar.inventar_to_exemplar.EX_INVENTAR_COUNT == 'undefined')
return false;
else if (forms.frmInventar.inventar_to_exemplar.EX_INVENTAR_COUNT==null)
return false;
else if (forms.frmInventar.inventar_to_exemplar.EX_INVENTAR_COUNT.toString().length == 0)
return false;
else if (isNaN(forms.frmInventar.inventar_to_exemplar.EX_INVENTAR_COUNT))
return false;
else
return true
}
- The testcase which causes the problem:
function testDelete() {
//create a record, save it and then delete it
application.output("Start testDelete");
forms.frmMain.onClickbtnAdd();
//data for the new record
forms.frmInventar.in_inventar = "Test_Inventar_Delete";
forms.frmInventar.in_kategorie = globals.getValueList("Inventar_Kategorien", "Musik")
forms.frmInventar.in_lagerort = globals.getValueList("Lagerort", "Hauptgeschaefstelle")
//press save
forms.frmInventar.onClickbtnSave();
//save the record count
var recordcount = databaseManager.getFoundSetCount(forms.frmInventar.foundset);
//save the id of the currently added inventar
var id = forms.frmInventar.in_id;
//this inventar could not have any exemplars
//waiting for the related record count - this fails !!!!
globals.assertTrueInMaxTime("Problem testing EX_INVENTAR_COUNT with:", 5000, function () { return globals.checkfrmInventarInventar_to_exemplarEX_INVENTAR_COUNT(); });
jsunit.assertTrue(forms.frmInventar.inventar_to_exemplar.EX_INVENTAR_COUNT == 0);
//delete the record
forms.frmMain.onClickbtnDelete()
//should be one record less
jsunit.assertTrue(recordcount - 1 == databaseManager.getFoundSetCount(forms.frmInventar.foundset));
//the added record with the saved id shouldn't be in the recordset
jsunit.assertTrue(forms.frmInventar.foundset.selectRecord(id) == false);
}
I am not sure what I am doing wrong - remember if I don’t load the data from the SQL script before every test with the setUp() method every thing works fine !
Regards
Steffen
I think what you are trying to do is too complicated, and that you should wait for the updates immediately after executing the SQL, before starting to run actual tests based on that data.
So this is what I had in mind:
function setUp()
{
// execute SQL here and flush tables using rawSQL plugin
globals.loadSampleData(); // as your initial posted version, but only until plugins.rawSQL.flushAllClientsCache (without any refreshRecordFromDatabase/loadRecords code)
// wait 10 sec for the rawSQL data to become available
globals.assertTrueInMaxTime("Data not yet available", 10000, function () { return forms.aFormBasedOnLastTableFlushedViaRawSQL.foundset.getSize() > 0 });
}
Notice the 10 sec and the form called “aFormBasedOnLastTableFlushedViaRawSQL” . So you create a form for the last table that is flushed with rawSQL (and to which you added records via rawSQL) and you check to see when records appear in that form. If your SQL file is huge and takes long to execute / update you can incease the 10 sec time limit. Don’t forget to delete these in tearDown().
If you have many records and you just want them in place before testing you can do this only once on solution load for example instead of setUp() - especially if individual tests don’t use the same data.
Hi Andrei,
yes I was already following your approach. First executing the SQL script, then flushAllClientsCache and then checking for some conditions (e.g. foundset size of some forms or the correct value of the calculation of EX_INVENTAR_COUNT). But still in some cases I had problems with this calculated field (EX_INVENTAR_COUNT). I could reproduce this behaviour by adding many application.updateUI(1000) statements after each statement in my test cases. So I could see in the GUI what happens in each test case. And for some reasons - which seem not obvious to me - all data was reloaded from the SQL script and showed in the GUI perfectly as expected. But after adding a new record the recalculation of EX_INVENTAR_COUNT was not done and the field states blank no matter how long I was waiting.
But your advice with only executing the SQL script some times was worth. Now I load the SQL script only when the form is loaded and everything runs nice - I have no explanation for this behaviour - but as long as it’s working I don’t ask for one.
Thanks for your help.
Schoby
SVY-1711 was implemented in 6.1 (run tests only for a form/a method/…).