200 row limit

Discuss all feature requests you have for a new Servoy versions here. Make sure to be clear about what you want, provide an example and indicate how important the feature is for you

200 row limit

Postby lesouef » Mon May 10, 2010 8:19 am

On every controller action, the max lines you can get is 200. While this make senses for web access, this is very low on a LAN, especially in smart mode.
Could this default max value be a param for each form, one for smart, one for web client?
At the moment, I replace the default "search" and "find all" of the menu with my own methods.
lesouef
 
Posts: 420
Joined: Thu Oct 20, 2005 8:13 pm
Location: Strasbourg & Paris - France

Re: 200 row limit

Postby jcompagner » Mon May 10, 2010 2:57 pm

what is the real usecase.
let say you could make it 1000 and then what?
there is never a good number because i guess you want to set it to unlimited?
But thats a very bad idea.

And why do you want to see more? What do you want to see? Why would you go to the 589 record? (or to the last?)
Johan Compagner
Servoy
User avatar
jcompagner
 
Posts: 8214
Joined: Tue May 27, 2003 7:26 pm
Location: The Internet

Re: 200 row limit

Postby lesouef » Mon May 10, 2010 3:27 pm

the usecase...
when I search records for a given content, and I know that even if all records contain the searched pattern, it is not too much for the client to handle it, why would I scroll 50 times to inspect results?
With my current machine, I get 180klines in 3s from a given table. if I search a string in a field which returns 5000 lines and now want to check some other field content in the foundset to see if it is correct, why would I scroll endlessly instead of getting the whole lot (5 or 10k is instant)? Of course there is no standard limit, this is why I am asking for one we could adjust according to the context. I can say for instance that 100k when working locally, is no problem at all. So I would set it to say 500k to avoid to stay stuck for 5mn when calling a new form. unlimited could be used if no table is exceeding a reasonable limit. I have a table which 6M lines, I certainly want to keep a limit for this one! and same for web access where I would use lower figures of course.
lesouef
 
Posts: 420
Joined: Thu Oct 20, 2005 8:13 pm
Location: Strasbourg & Paris - France

Re: 200 row limit

Postby jcompagner » Mon May 10, 2010 3:53 pm

you are constantly talking about local.. But we take into account some different things

Normally we have 2 things: Smart clients over a network (can even be internet)
or webclients that are accessed the same.

For both we dont want default 180K pk go over the line for every query you do on that table.
That is a huge huge thing. For webclients this would mean that they hold loads of data in the server memory
For smart clients this also means way more memory usage if you are not careful (using different foundset on the same table and bang you have 2 or 3 times 180K lines in memory)

But again, why would you scroll to 9999 record? Why would you do that? What is what you are trying to do? Do you really inspect all those lines?
But if you do that 1 at a time then it is not a problem because then we load as you go.

But normally just search for that record or sort differently so that it easy on top.

How many people here go to page 10 or more if you search with google?
I think i never pass beyond page 5 or 6..
Johan Compagner
Servoy
User avatar
jcompagner
 
Posts: 8214
Joined: Tue May 27, 2003 7:26 pm
Location: The Internet

Re: 200 row limit

Postby lesouef » Mon May 10, 2010 6:09 pm

local = smart client on the LAN.
I am not asking you to change the defaults, but to be able to alter it when necessary.
why would I do this? we obviously have not been facing the same situations...
for instance, I am making stats on the content of a certain field. the result is wrong, so I must check what wrong codes have been entered (another app is taking care of data entry and makes no check on field contents), but I don't even know what kind of pattern to look for, so I need to read the table until I get a hint about what could have been badly entered. Say I want first to inspect the latest records manually, I need to scroll and scroll untill the 5000th? that's 25 clicks! and sorting will only sort the 1st 200, so no way to get the last first ... so unless I script everything, manually I have no excel like mode to inspect the table. when I use sql, I can say top 100000, if it takes too much memory, that's my problem.
Of course, all your good reasons are true to limit memory, bandwidth, but under some circumtances, it would be nice to view the whole table if I wanted it.
So the ideal is to keep your default value, but be able to alter it for let's call it "data inspection".
lesouef
 
Posts: 420
Joined: Thu Oct 20, 2005 8:13 pm
Location: Strasbourg & Paris - France

Re: 200 row limit

Postby pbakker » Mon May 10, 2010 6:24 pm

nd sorting will only sort the 1st 200, so no way to get the last first ...


That is not correct. If you want to get to the last record, you sort decending on whatever column that normally determines that it's the last record (PK, createdate or whatever).

When sorting, Servoy fires an SQL statement at the database to retrieve the sorted records, so all records are taken into account.

Paul
pbakker
 
Posts: 2822
Joined: Wed Oct 01, 2003 8:12 pm
Location: Amsterdam, the Netherlands

Re: 200 row limit

Postby lesouef » Mon May 10, 2010 6:32 pm

ah? in all modes? I am in table view and I don't see this at all, even using the key field to sort (most of the time I use the colomn header), but using the sort command is the same I guess...
lesouef
 
Posts: 420
Joined: Thu Oct 20, 2005 8:13 pm
Location: Strasbourg & Paris - France

Re: 200 row limit

Postby jcompagner » Tue May 11, 2010 9:04 am

doesnt matter in what mode or view you are
if you sort a foundset ascending you get rows 1 - 10K sorted like that and if you sort on the same column descending you get 10K -> 1 in reverse
So scrolling through thousands of records or jumping to the last is really a weird idea, you should search and or sort it.
Johan Compagner
Servoy
User avatar
jcompagner
 
Posts: 8214
Joined: Tue May 27, 2003 7:26 pm
Location: The Internet

Re: 200 row limit

Postby pbakker » Tue May 11, 2010 9:12 am

If you/your users are looking for odd values in. lets say 5000 records, find/search seems to me a lot better way to get to those values.

You know that the find/search syntax allows you to do "not equal" find/search? And omit records from the set?

Also, depending on how many different possible values there are for a certain column, you can better offer the user a combobox with all values in that column and filter based on that. That way he/she can quickly see which distinct values there are for the column, select that value and see the records that match it.

Paul
pbakker
 
Posts: 2822
Joined: Wed Oct 01, 2003 8:12 pm
Location: Amsterdam, the Netherlands

Re: 200 row limit

Postby lesouef » Tue May 11, 2010 10:05 am

we stop that debate. I realize I have a rock in front of me. just say 'no'.
last bit of info, yes I know I can search !x, and reduce foundsets, make dropdowns based on table fields, etc...
If I want to inspect data, it really is because in this case I have no choice. an information has been encoded by users in a field, then progresssively they applied a new encoding method, so I need to check which user and when each user has applied the new encoding method by reading data within the change period + detect typing mistakes, and this is a 6 month period in a direct marketing company, 180k sales per year... I wanted to convert them to servoy since they have a heterogenous environment db2 and mssql and thousands of excel macros, I probably wasted my time.
heve you ever asked youself why excel or filemaker are not limiting their foundset, why excel has removed the 64k limit? have you ever been in front of a table and even don't know what to search in it? you first need to see it before you can filter? obviously not.
I did not intend to open such a debate anyway, for just a parameter which is fixed and incredibly low (I have no table with so few records!) which should maybe changed according to my humble experience (only 33 years behind computers).
leave it the way it is, I have to work.
lesouef
 
Posts: 420
Joined: Thu Oct 20, 2005 8:13 pm
Location: Strasbourg & Paris - France

Re: 200 row limit

Postby jcompagner » Tue May 11, 2010 10:17 am

to set one thing straight: there is no limit, we dont impose a limit.
that 200 is just a window. nothing more.

And if you want to really load everything then a small global method will do that just fine:

Code: Select all
   var currentSelection = currentcontroller.getSelectedIndex();
   currentcontroller.setSelectedIndex(currentcontroller.getMaxRecordIndex());
   while (currentcontroller.getSelectedIndex() != currentcontroller.getMaxRecordIndex())
   {
      currentcontroller.setSelectedIndex(currentcontroller.getMaxRecordIndex());   
   }
   currentcontroller.setSelectedIndex(currentSelection);
Johan Compagner
Servoy
User avatar
jcompagner
 
Posts: 8214
Joined: Tue May 27, 2003 7:26 pm
Location: The Internet

Re: 200 row limit

Postby mboegem » Tue May 11, 2010 10:36 am

Just one more suggestion:

depending on the number of different entries that are in a field, you could think of a initial view which shows a grouped count of all different entries.
If you really have to go over 180K of records, there won't be single user being happy to do the job.
If you could group this 180K lines of data having 'only' 150 lines with different entries, data is even much better to inspect.
Then, when the user finds a group of records with a wrong entry, enable them to request those records simply by clicking a button.

The initial, grouped view can be a datasource dispayed on a form that's created with the solutionModel
The button requesting the actual records can simply do a find (might even be a custom pk query) and show the result on the form you are using right now.

As the solutionModel makes everything very flexible, you can even implement kind of a filter where the user can define which field to group on... (or even combination of fields)

If this whole debate is about user friendly interface, this is IMHO the way to go.
_____________________
Marc Boegem
Solutiative / JBS Group, Partner
• Servoy Certified Developer
• Servoy Valued Professional
• Freelance SAN Developer

Image
User avatar
mboegem
 
Posts: 1526
Joined: Sun Oct 14, 2007 1:34 pm
Location: Hoofddorp, The Netherlands

Re: 200 row limit

Postby lesouef » Sat May 29, 2010 8:25 pm

I have simply done my own global "search", "sort" and "show all" methods to replace the default form actions where I want the whole foundset to be shown at once without having to scroll. that does the job perfectly as long as I don't use it anywhere. Of course using this everywhere would be catastrophic for performance, but I was just asking to be able to change the default value where required, not making standard, but that seems to hurt the servoy philosophy.
lesouef
 
Posts: 420
Joined: Thu Oct 20, 2005 8:13 pm
Location: Strasbourg & Paris - France


Return to Discuss Feature Requests

Who is online

Users browsing this forum: No registered users and 2 guests