Marking records

[jcompagner]

marking records in a global can be done now (>RC2) with this:

Code: Select all

globals.mark += foundset;
forms.actions2.controller.loadRecords(databaseManager.convertToDataSet(globals.mark))


This will not filter the doubles!!
so if globals.mark already contains id 1 and the current foundset also has an id 1 it wil have 2 id 1's in the foundset (one for example on position 2 and other at 25)

[chris]

but this can't be done for 15000 records... that will not work (database dependend where the limit is)

This seems to be getting to my Need #1 / Example B, above: Marking records (NOT saving found sets - that is a different matter).

When a user "marks" a record, its ID is placed in a global. When a user marks the entire found set, all of the found set's IDs are placed in the global. We should be able to load the found set by loading the IDs from that global.

1) If it's true that Servoy globals have no practical character limit than we should indeed be able to mark > 15,000 records.

Ron, are my exampls close to your users' needs?

Team, are we on the right track?

[jcompagner]

you can mark (the foundset in a global) and store (put the foundset in a record dataprovider) the global can be as big as youre memory can hold so that is not a problem. The biggest problem is that you can store 15.000 id's in a globa. But you wil not be able to get a working foundset for that because of the query we have to generate for that can't handly that many input values.

[chris]

Not sure what you mean in your last sentence.

OK, what happens if i do this:

1) Found set is 15,000 records
2) I execute a method that uses "copy all records" to copy just the IDs
3) I paste the clipboard into a global.
4) I then play around and change the found set.
4) I can get the found set back by loading the found set from these IDs, right?

How do I load the found set from these IDs? I don't see this method in my methods window.

loadRecords(databaseManager.convertToDataSet(globals.mark))

I'm using the latest version of Servoy (clicked help>check for new version and it said there wasn't one).

Thanks.

[jcompagner]

1) Found set is 15,000 records

OK

2) I execute a method that uses "copy all records" to copy just the IDs
3) I paste the clipboard into a global.

those 2 steps are just a method that does this:

globals.mark += foundset;

4) I can get the found set back by loading the found set from these IDs, right?

No this will fail with that many id's
Why are such large foundsets needed?


all these things i discuss here are not in a build that is released.. so

globals.mark += foundset;
or
databaseManager.convertToDataSet(globals.mark))

will not work yet with RC2!!

Also it is still subject to change.. nothing is in stone yet.

[chris]

Hi Johan: some answers to your questions

1) WHY do we need to be able to mark such a large found set?
Because our users deal with membership databases that have several hundred thousand records. Say we need to make a targeted mailing. First we'll find everyone in a certain zip code. We'll mark them. Then we'll find everyone who has given the foundation money before. Then we mark them, etc. etc. Finally, we've marked everyone we want to mail to. So we click "view marked records" and our found set of approx. 15,000 people shows up. From there, we export to Excel and send off to a mailing house, or we print labels, whatever.

2) THIS MIGHT BE A PROBLEM...
We need to be sure that it's actually IDs being stored in the global. Here's why: Users need to view the list of people in list view. Next to each person there will be an "M" if that person is marked. The "M" will just be a calculation field that does a patterncount to see if that person's ID is in the global field. If it is, the "M" shows up.

PLEASE CONFIRM.

3) ANOTHER PROBLEM...
I'm concerned that your solution might leave out something important. Say we're looking in a form attached to a table that tracks all checks a foundation has received in the past 10 years. There should be thousands of checks. We find all checks received that were for a category, like "general operating fund." Then we want to view all people who gave these checks - that is, all people who gave in this category.

So, WE WANT TO BE ABLE TO VIEW ALL PEOPLE RELATED TO THE FOUND SET IN THE CONTRIBUTIONS TABLE. That is, in FM, I'd copy all of the personIDs into the global and then use go to related to go to these person records, or I'd import into the people file from the contributions file, matching on the personID. Note that personID is not the primary key for that table.

Your solution doesn't seem to specificy which IDs are copied to the global and therefore might leave out this important possibility.

Thank you.

[jcompagner]

the PK's are stored .. you can't make a foundset back if you don't have the pk's..

So if you store an id field or something that is not the pk in a record column or a global then you will not be able to make a foundset of it again.


for 3. If you have form that displays the foundset of youre checks (the check table has te person id yes??)
why aren't you just setting that foundset on a other form where you only display the related data of the person for every check and not the check columns itself.? (in a listview)

[chris]

First, is it clear now why it's necessary to be able to mark a nearly unlimited number of records? It's a core feature for my membership databases. Can the team find a way to do this? Thank you.

the PK's are stored .. you can't make a foundset back if you don't have the pk's..

So if you store an id field or something that is not the pk in a record column or a global then you will not be able to make a foundset of it again.

So, the answer to my number 2 about making a calc field that displays if a particular record is marked or not is YES it will work!?

for 3. If you have form that displays the foundset of youre checks (the check table has te person id yes??)
why aren't you just setting that foundset on a other form where you only display the related data of the person for every check and not the check columns itself.? (in a listview)

This won't work because we need to load the found set of people who gave the checks in a table/form attached to the people table. Only in the people table can the user send email to people, send formmail to people, fill out action logs for people, etc. In a membership database that tracks literally dozens of activities about people, we will always need to find a set of activities and then load a found set of people who performed those activities directly in the people table (it won't always work to perform the find in the people table b/c the finds are more complex than simply finding through related fields on a people-based find form)

Is that clear?

Otherwise we'd be replicating many things in each new module like sending email or mail and that's just not efficient from a development perspective.

[Jan Blok]

PLEASE CONFIRM.

3) ANOTHER PROBLEM...
I'm concerned that your solution might leave out something important. Say we're looking in a form attached to a table that tracks all checks a foundation has received in the past 10 years. There should be thousands of checks. We find all checks received that were for a category, like "general operating fund." Then we want to view all people who gave these checks - that is, all people who gave in this category.

So, WE WANT TO BE ABLE TO VIEW ALL PEOPLE RELATED TO THE FOUND SET IN THE CONTRIBUTIONS TABLE. That is, in FM, I'd copy all of the personIDs into the global and then use go to related to go to these person records, or I'd import into the people file from the contributions file, matching on the personID. Note that personID is not the primary key for that table.

We introduced databaseManager.convertFoundSet(foundset,relation) which has the the behavior you want, it makes it possible to view all people related to the contribution table.

so in the contribution form do the following:

Code: Select all

var convertedFoundSet = databaseManager.convertFoundSet(foundset,contributions_to_people);
forms.peopleform.controller.showRecords(convertedFoundSet);

will be availeble in Servoy 2.0 rc3

[chris]

That looks really great, thank you so much.

Especially as I just learned there is NO LIMIT on the number of records it can process.

I wonder if something from this approach can be used for the "marking records" examples we've discussed?

Thanks for your responsiveness!