Foundset assigned to variable updating...

Classic Servoy
1

Having just updated a large system from 2.2.7 to 3.5.5, live usage has highlighted what seems to be serious issue.

My client is a mail order supplier who has a number of special customers that require the ability to order ‘special’ products that only exist virtually for them as multiples of real products. The clients needs to be able to enter special virtual product codes for these on an order and then when the order is complete, click a button to ‘expand’ the virtual products into the real products, but leaving the invoice totals the same and hiding the real products.
I created a method that would save the foundset of line_items, loop thru them and for each virtual product create 2 new hidden line_items, 1st a duplicate of the virtual product, but with a negative value for the same quantity, to cancel it out and 2nd a line_item of the real product with a quantity of the virtual product x real product quantity
e.g. if the order was for 2 x 6-pack virtual product this would create 2 more lines
2 x 6-pack virtual product [original visible line item]
-2 x 6-pack virtual product [hidden]
12 x real individual product [hidden]

In 2.2.7 you could do

//Get duplicate of current foundset, can be used by loadRecords again
var dupFoundset = controller.duplicateFoundSet();

//do some stuff, including
var my_count = controller.getMaxRecordIndex() ;
for ( var i = 1 ; i <= my_count ; i++ )
	{
           foundset.duplicateRecord(i) ;
	   quantity = quantity * -1 ;
	   line_hidden = 1 ;

// and

	//for each line real product, create a new lineItem
	controller.newRecord(false) ;
	quantity    = record.quantity * record.line_items_to_stock_vp_join_vp.qty ;
      }

//reload original foundset as we have done something
controller.loadRecords(dupFoundset);

This used to work perfectly, but now it seems that when doing
var dupFoundset = controller.duplicateFoundSet();
[or var dupFoundset = foundset.duplicateFoundSet(); in Servoy 3 code]
if you make any changes to the foundset, the primary keys for any new foundset records are added to the variable dupFoundset even though it should have been static (unless all my 25 years of programming have been wrong about variables staying as they are unless you explicitly change them 8-) ).
My guess is that dupFoundset is being assigned a pointer to the foundset array, not it’s own array of the foundset, so if the foundset changes, so does the variable! This is WRONG!

I hope this can be looked at urgently. I have managed to work around it (see below for original method and updated method) as I am at client and they needed it fixed immediately!

Thanks,
Rafi

Servoy Developer
Version 3.5.6-build 519
Java version 1.6.0_05-b13-52 (Mac OS X)
[but was happening in client of 3.5.5]

ORIGINAL METHOD

// expandVPs [lineItemsOrder]
//expands VPs to real items
//called from button on order_details

//make duplicate of foundset to allow looping to work if we add new lines
//Get duplicate of current foundset, can be used by loadRecords again
var dupFoundset = controller.duplicateFoundSet();

//loop thru all order lines
for ( var i = 1 ; i <= controller.getMaxRecordIndex() ; i++ )
	{
		//put line into a variable
		var record = foundset.getRecord(i);//reference is valid as long nothing else is done on the foundset
						
		//if line has an item
		if ( record.stock_id )
		{
			//check if item is virtual product
			if ( record.line_items_to_stock.vp_virtual_product == 'Yes' )
			{
				//reverse line so totals are ok
				foundset.duplicateRecord(i) ;
				quantity = quantity * -1 ;
				line_hidden = 1 ;
				
				//check if it has real stock assigned
				if (databaseManager.hasRecords(record.line_items_to_stock_vp_join_vp))
				{
					//loop thru all real products
					for ( var j = 1 ; j <= record.line_items_to_stock_vp_join_vp.getSize() ; j++ )
						{
							record.line_items_to_stock_vp_join_vp.setSelectedIndex(j) ;
							
							//for each line real product, create a new lineItem
							controller.newRecord(false) ;
							order_id    = record.order_id ;
							stock_id    = record.line_items_to_stock_vp_join_vp.stock_id ;
		
							//set FKs
							company_id  = record.line_items_to_orders.company_id ;
							contact_id  = record.line_items_to_orders.contact_id ;
					
							item_number = record.line_items_to_stock_vp_join_vp.c_stock_item ;
							description = record.line_items_to_stock_vp_join_vp.c_stock_desc ;
							quantity    = record.quantity * record.line_items_to_stock_vp_join_vp.qty ;
							media_code  = record.media_code ;
							vat_rate    = record.vat_rate ;
							//weight = 0 ;
							line_hidden = 1 ;
							//if it's a special item
							if (record.line_items_to_orders.price_band == 'Special')
							{
								unit_price_ex_vat  = ( record.unit_price_ex_vat / record.line_items_to_stock_vp_join_vp.qty );
								unit_price_inc_vat = ( record.unit_price_inc_vat / record.line_items_to_stock_vp_join_vp.qty ) ;
								unit_price_vat     = ( record.unit_price_vat / record.line_items_to_stock_vp_join_vp.qty );
							}
							else
							{
								choosePriceExpandVPs() ;
							}
						}
				}
				else
				{
					//no real items created yet...
				}
				//reload original foundset as we have added lines
				controller.loadRecords(dupFoundset);
			}
		}
	}

//save	
databaseManager.saveData();

UPDATED METHOD

// expandVPs [lineItemsOrder]
//expands VPs to real items
//called from button on order_details

controller.setSelectedIndex(1) ;

//make duplicate of foundset to allow looping to work if we add new lines
//Get duplicate of current foundset, can be used by loadRecords again
var dupFoundset = foundset.duplicateFoundSet();

// loop thru all order lines
var lv_found_size = foundset.getSize() ;

for ( var i = 1 ; i <= lv_found_size ; i++ )
	{
		//put line into a variable
		var record = foundset.getRecord(i);//reference is valid as long nothing else is done on the foundset
						
		foundset.setSelectedIndex(i) ;

		//if line has an item
		if ( record.stock_id )
		{
			//check if item is virtual product
			if ( record.line_items_to_stock.vp_virtual_product == 'Yes' )
			{
				//reverse line so totals are ok
				foundset.duplicateRecord(i,false,true) ;
				
				quantity    = quantity * -1 ;
				line_hidden = 1 ;
				
				// save	
				databaseManager.saveData();
				
				//check if it has real stock assigned
				if (databaseManager.hasRecords(record.line_items_to_stock_vp_join_vp))
				{
					//loop thru all real products
					var lv_vp_size = record.line_items_to_stock_vp_join_vp.getSize() ;
					
					for ( var j = 1 ; j <= lv_vp_size ; j++ )
						{
							record.line_items_to_stock_vp_join_vp.setSelectedIndex(j) ;
							
							//for each line real product, create a new lineItem
							foundset.newRecord(false,true) ;
							order_id    = record.order_id ;
							stock_id    = record.line_items_to_stock_vp_join_vp.stock_id ;
		
							//set FKs
							company_id  = record.line_items_to_orders.company_id ;
							contact_id  = record.line_items_to_orders.contact_id ;
					
							item_number = record.line_items_to_stock_vp_join_vp.c_stock_item ;
							description = record.line_items_to_stock_vp_join_vp.c_stock_desc ;
							quantity    = record.quantity * record.line_items_to_stock_vp_join_vp.qty ;
							media_code  = record.media_code ;
							vat_rate    = record.vat_rate ;
							//weight = 0 ;
							line_hidden = 1 ;
							//if it's a special item
							if (record.line_items_to_orders.price_band == 'Special')
							{
								unit_price_ex_vat  = ( record.unit_price_ex_vat  / record.line_items_to_stock_vp_join_vp.qty ) ;
								unit_price_inc_vat = ( record.unit_price_inc_vat / record.line_items_to_stock_vp_join_vp.qty ) ;
								unit_price_vat     = ( record.unit_price_vat     / record.line_items_to_stock_vp_join_vp.qty ) ;
							}
							else
							{
								choosePriceExpandVPs() ;
							}
						}
				}
				else
				{
					application.beep() ;
				}
				
				// save	
				databaseManager.saveData();
				
				//reload original foundset as we have added lines
				
				controller.loadRecords(dupFoundset);
			}
		}
	}

//save	
databaseManager.saveData();
2

Jan Blok:
As you know we have databroadcasting, which means that other (duplicated) foundsets objects receive updates about records belonging to there search criteria (which defines them as mutable objects)
I cannot explain why this not worked in 2.2.7, but the current implementation you are seing works as expected.

OK, I just looked up mutable objects & immutable objects. I think it might be in the best interests of Servoy users to explain this somewhere in the docs. and also to highlight what things will be created in Servoy as mutable or immutable.
I think it worked in 2.2.7 because the changes to the foundset were being added to the end of the foundset object, but that must have changed in 3.5.
My changes to get it working (I think) made sure all new records went to the end of the foundset object.
I guess that I expected to be able to assign a static foundset object to a variable & not have it change unless I make an assignment to it.
Is there a new command in 3.5 that I can use to assign a foundset before I make changes to it & have that variable NOT change so I can restore it?
(GrahamG emailed me some Servoy/SQL code that does something like this, but is there a single command I can use?)

Thanks,
Rafi

3

-by default when an object has methods which change the internal state, defines them as mutable.
-foundsets/datasets are mutable, foundset is a bit of an exception since it also actively listens(!) for changes and tries to keep its records (content) in sync with reality
-numbers/strings are not mutable