Databroadcasting Questions

Classic Servoy
1

Databroadcasting Questions

  1. Is anything currently known to be broken/not working in the current iteration of the Servoy application server (either 5.1.4 or 5.2.1) as it pertains to databroadcasting?
  2. Can/does databroadcasting ever occur across solutions, and if so under what circumstances will it or won’t it occur?
  3. Can databroadcasts broadcast new records, deleted records and changes to existing records? I believe it can, but regarding creates and deletes, I don’t think clients “refresh” automatically to see new or deleted records except under certain situations. What are those circumstances?
  4. Is headless client any different from any other kind of client in regards to databroadcasting? Does it not do anything that smart and web do, or does it do anything differently than smart and web do [in regards to brroadcasting]?
  5. Is there any difference in the way stored calculations are broadcast vs other dataproviders that are not calculations?

Databroadcasting Scenario…

A smart-client solution (solution A) captures information and that information is inserted as new records in a database table.
A headless client (Solution B) polls the same table, in the same database looking for new records. When new records are encountered the headless client solution selects the data out of the table and processes it, inserting new records elsewhere in the database and at the same time recalculating a number of stored calcs.
A webclient solution (Solution C) is looking at a list of records from the same table that the headless client creates records in, and on that table view there are stored calcs as dataproviders. Those stored calcs have been updated by the headless client.

In this scenario, what will the Servoy application server do? What will be broadcast to the various solutions and clients and when?

Also where is all of this documented?

Thanks in advance

2

jbader:

  1. Is anything currently known to be broken/not working in the current iteration of the Servoy application server (either 5.1.4 or 5.2.1) as it pertains to databroadcasting?

no

jbader:
2. Can/does databroadcasting ever occur across solutions, and if so under what circumstances will it or won’t it occur?

solutions don’t matter, its all which client touches which tables of which servers. That is monitored.
So if client touches table X it will get all data broadcast of that table.

jbader:
3. Can databroadcasts broadcast new records, deleted records and changes to existing records? I believe it can, but regarding creates and deletes, I don’t think clients “refresh” automatically to see new or deleted records except under certain situations. What are those circumstances?

yes it broadcasts all of them, only new records are only updated in related foundsets, not “normal” foundsets, they don’t suddenly get those new records.
Especially if the foundset is fully loaded…
A new record could come in if that new pk is queried in the next bunch of pks when we get another 200 pk’s…

jbader:
4. Is headless client any different from any other kind of client in regards to databroadcasting? Does it not do anything that smart and web do, or does it do anything differently than smart and web do [in regards to brroadcasting]?

no there is no difference.

jbader:
5. Is there any difference in the way stored calculations are broadcast vs other dataproviders that are not calculations?

no they are normal columns for the databroadcast itself.

jbader:
A smart-client solution (solution A) captures information and that information is inserted as new records in a database table.
A headless client (Solution B) polls the same table, in the same database looking for new records. When new records are encountered the headless client solution selects the data out of the table and processes it, inserting new records elsewhere in the database and at the same time recalculating a number of stored calcs.
A webclient solution (Solution C) is looking at a list of records from the same table that the headless client creates records in, and on that table view there are stored calcs as dataproviders. Those stored calcs have been updated by the headless client.

In this scenario, what will the Servoy application server do? What will be broadcast to the various solutions and clients and when?

Solution C will see the updates if it shows the records
It will not directly see the new records if it is just a normal foundset, because a normal foundset doesnt check if the new records fall into its search criteria

3

Johan,

Thanks a lot for the quick reply.

So if I experience functionality dramatically different than this, I can assume that there is a bug in application server and I should submit a ticket + a sample solution correct?

Please advise.

4

if you have some situation that you say, i should have updated records there but that doesn’t happen
please make a reproduce able sample and attach it to a case.

5

Could case 295204 be made “public?” Is it’s content relevant to this thread?

6

jbader:
Could case 295204 be made “public?” Is it’s content relevant to this thread?

That databroadcasting bug is for inserts when there is a column who has default value non null but in the new record its value is null .

7

Thank you for the reply.

So are there any other known issues with databroadcasting at this time in Servoy 5.2.x?

I’m trying very hard to get a handle on databroadcasting and cannot. It’s the single most implicit feature I have ever had the pleasure of working with. I never know what it is doing, why it is doing it, when it will or won’t do it again and whether or not what I am seeing is expected behavior or a bug. It’s really quite impossible to understand and/or predict, seeing as it is mostly undocumented (besides this? http://wiki.servoy.com/display/public/D … oadcasting).

Is there more documentation available somewhere?

8

Don’t think there are other broadcasting issues. If you encounter something weird you can post a sample here and we’ll have a quick look. Or post some javascript code .

9

If you encounter something weird you can post a sample here and we’ll have a quick look. Or post some javascript code .

If I knew how to break it I could figure out how to fix it. If I had some code that always failed I would be elated, but I do not, and so I have nothing to post; no JS code, no sample solution. The same code in my solution may fail or work at will, anytime it chooses at the mercy of the app server.

I don’t know what databroadcasting is supposed to do and when b/c it is entirely undocumented. So as far as I am concerned it is all “weird.” Sometimes I get broadcasts, other times I don’t. There is no rhyme or reason that I can see at all, but if there is, and if I am missing that, that is because there is no documentation to tell me how it is supposed to work in the first place.

Can you address the documentation issue please? Is there any, anywhere?

10

I think Johan gives a pretty good descriptions here above, of how it works.

with foundset’s you will see: updates, delete’s (not (new) insert’s although you will receive a trigger in the solution property: onDataBroadCast, than you can decide what you want to do with it…)
with related foundset’s you will see updates, delete’s AND inserts…

11

Can you address the documentation issue please? Is there any, anywhere?

I would really appreciate some feedback on the above. Where can I find documentation on databroadcasting?

Thank you

12

jbader:
I would really appreciate some feedback on the above. Where can I find documentation on databroadcasting?

harjo summarized my stuff perfectly, that is exactly how databroadcast should work.
it something that just is there.

The only thing that you need to know is the normal foundsets will not suddenly insert new records when new records are inserted by others for the table the foundset is on.

13

Great! So let’s just point the Wiki to this exhaustive and comprehensive forum post, so that others can benefit from the wealth of knowledge that now resides here.

Most specifically…

it something that just is there.

Ever typed… ```
import this


I don't think that asking for feature documentation for commercial software that I pay money to use is unreasonable, nor should it be dismissed. The implicitness of this feature is causing me to spend time trying to understand and predict it. That time could be better spent elsewhere, and that is the whole idea with a RAD tool in the first place. I shouldn't have to think about commercial features of my RAD tool, and how they work, but if I need to understand how they work I should be able to/allowed to.

[frustrating as usual]
14

the thing is that you shouldnt need to predict it… It should just work out of the box, there shouldn’t be any question about it
It is not directly a feature that is for the servoy developer itself, because it is a feature that is needed for the servoy system itself…
Because we cache internally data and that data needs to be updated thats why we have data broadcasting…
So it is not a servoy developer feature… yes we have now a databroadcast event where you can hook into that also but for the first part it is a internal thing for servoy itself.

So it shouldn’t be a question for you (it should just work for you), if you really have a databroadcast problem because you do see rows not updating then we just need to get to the bottom of it and look what events dont get from the server to the client for you.

We know 1 issue that is when you insert new records that have db default column values, that you didnt set in the client, so it really should just be the default db value, then that new record will not be updated in related foundsets that should have that new row. (because the relation key values do map on that new row)
Updated or deleted records should always work over all foundsets no matter if they are related or normal foundsets…

So if for you that last part doesn’t work (so normal updates on existing records or deletes of records) then we need to figure out how that is possible, then it looks like your clients don’t get the databroadcast events
Can you tell me a bit more about your clients? Are webclients in this setup? Are there smart clients do you see it over multiply operating systems?

15

Hi Johan,

thank you very much for replying. Databroadcasting may work exactly as intended (out of the box), and I may not need to predict it, but in order to know I need to understand how it is intended to work in the first place. I’ve seen some very odd behavior that I cannot explain (mostly in webclient FYI) and I am trying to understand how to control it. I have a lot of questions, and would very much appreciate responses when you can find the time.

Here are a few of my questions (I’ll post others soon).

Databroadcasting Questions
1.Do all native Servoy components receive broadcasts? Meaning, for example, will a label whose value is set to %%dataProvider%% receive a broadcast when dataProvider is changed? Is a label different than a text field as far as databroadcasting is concerned (value vs reference)? Are some components not “broadcast aware?”
2.How are aggregates treated in terms of databroadcasts? Example, I have a related dataprovider placed on a form, which is an aggregate, and any of the related values that it aggregates change? Or new, related values to aggregate are created/deleted. Will the aggregate’s select query be automatically rerun, resulting in the aggregate receiving a “broadcast?” How about the same case except instead of placing a field I placed a label whose value is %%relation.aggregate%%. Will that receive a broadcast/rerun the select in the same way (per #1 above)?
3.Does the app server know which clients have stale data, or do the clients tell the server they have stale data? If the latter, is there a difference between smart and webclient here, or is the mechanism for determining stale data the same?
4.Are unstored calcs ever cached?
5.In webclient specifically, can/does the inherent asynchronicity of AJAX cause problems with broadcasts? Can they happen out of order? For example, if I had a calc (stored or unstored) on a parent table whose value was an aggregate through a relationship, and one of the columns, from one of the records/rows from the related foundset changed, could it ever happen that webclient attempt to first get/receive a broadcast for the calc on the parent table, which in that case would not yet have the related record’s changes? If so, this would mean that it’s data was indeed stale, and it did indeed got its broadcast, but the broadcast it got didn’t take into account the related change!? How are you making AJAX synchronous in this regard?

Again thank you

16

jbader:
Databroadcasting Questions
1.Do all native Servoy components receive broadcasts? Meaning, for example, will a label whose value is set to %%dataProvider%% receive a broadcast when dataProvider is changed? Is a label different than a text field as far as databroadcasting is concerned (value vs reference)? Are some components not “broadcast aware?”

components are not broadcast aware at all.
They are record changes aware…
Where the record change come from doesnt matter…

Its the same as you edit a field of a record on place X and then when you save place Y on a different form is also updated (this could even happen before you save when you just commit the value (tab out of field))

So databroadcasat is in the data layer not ui… The datalayer just gets a even that there is new or updated row data. The ui just recieves a value update event.

jbader:
2.How are aggregates treated in terms of databroadcasts? Example, I have a related dataprovider placed on a form, which is an aggregate, and any of the related values that it aggregates change? Or new, related values to aggregate are created/deleted. Will the aggregate’s select query be automatically rerun, resulting in the aggregate receiving a “broadcast?” How about the same case except instead of placing a field I placed a label whose value is %%relation.aggregate%%. Will that receive a broadcast/rerun the select in the same way (per #1 above)?

Ths is the same as #1 an row update event is generated by a databroadcast (but could also be again by a change of the client itself) then we clear the aggregates and those will be get again when the ui asks for them.

jbader:
3.Does the app server know which clients have stale data, or do the clients tell the server they have stale data? If the latter, is there a difference between smart and webclient here, or is the mechanism for determining stale data the same?

the app server only knows which client did touch what data (server/table) and then a client will receive databroadcast events for that server/table combination.
the client then looks if it has stale data.

jbader:
4.Are unstored calcs ever cached?

calcs are cached yes and calculated when depended data changes
some more info;
http://wiki.servoy.com/display/public/D … alculation

jbader:
5.In webclient specifically, can/does the inherent asynchronicity of AJAX cause problems with broadcasts? Can they happen out of order? For example, if I had a calc (stored or unstored) on a parent table whose value was an aggregate through a relationship, and one of the columns, from one of the records/rows from the related foundset changed, could it ever happen that webclient attempt to first get/receive a broadcast for the calc on the parent table, which in that case would not yet have the related record’s changes? If so, this would mean that it’s data was indeed stale, and it did indeed got its broadcast, but the broadcast it got didn’t take into account the related change!? How are you making AJAX synchronous in this regard?

that ajax poll just aks the webclient state on the server if there is any change where if it maybe.
and will render that. If not then it will take it the next time.
Databroadcast is not triggered or really called by the ajax poll, the ajax poll only updates the html of the ui state the webclient has on the server…
The ui state on the server is the one that gets the databroadcast and will mark all the ui components as changed so that they will be rendered the next time an ajax poll (or a request a user does by clicking a button or something) happens.