Servoy 8.0.0 beta 1

[david]

Back to the idea of making the session/websocket/foundset piece a discrete API so can plug Servoy’s realtime web in any type of web app.

Servoy’s code for this piece is here: https://github.com/Servoy/sablo. No license listed.

It turns out that the core functionality is implemented agnostic to any client-side framework and NG Client just consumes the functionality with some not-so-elegant client-side code. It gets the job done but some API-esque wiring would help clean up this code.

A weekend of poking and prodding later…

Basic mechanics

Server-side Servoy forms function to set up a model/foundset. Form has a datasource, the fields you put on the form are the allowable data fields the client can see.

Form items provide additional meta information to the client such as data formatters, calculations, i18n keys, tagging, UI parameters, etc.

Forms also provide services such as when a button is pushed, code on the server is run and a response is sent to the client. Foundset functions are automatically mirrored on the client. So something as simple as foundset.setSelectedIndex(foundset.getSelectedIndex() + 1); automatically returns the data for the next record for the fields you have on the form without you having to return the record yourself.

When an NG Client connects, a session is created and (all?) forms in your app have a state per session. Data broadcasting happens between sessions where the foundsets share the same visible foundset index(es). I assume all CRUD operations but edit is the only one I’ve tested so far.

All of this communication goes back and forth from clients and server via websockets as JSON. If a field is edited, only the data for that field is sent over the wire, not the whole record. Quite compact for most operations from what I’ve seen (initial connection response sends a lot of unnecessary Angular stuff is one exception).

[attachment=1]websockets.png[/attachment]

Solution

Figured out the basic wiring to establish websocket connections, send messages to the server, and receive server responses. Servoy’s JSON objects are consistent and easy to grok so mapping to client-side models of any particular framework will be easy.

Triggering form methods is straight forward. My guess is that only public methods are exposed on the client but on my list to test.

If a solution requires login, websocket endpoints are not created until you login.

A neat trick is that you can return any JS object from a form method (serialization to JSON happens automatically) and the client websocket.onmessage handler picks it right up. So not only can you trigger foundset methods but you return your own data structures over the pipe. (Note to self, test sending arguments to form methods.)

So there you have it — it is all possible to hook into Servoy’s realtime web special sauce without having to go through NG Client.

And yes, we are available for hire!

Cheers - David

[attachment=0]servoy_socket_test.png[/attachment]

[jcompagner]

juan.cristobo:
Are you planning to include sample webcomponents and sample NG solutions in the installer?

we already are shipping this, but thats inside our plugins, the problem is a bit how to do it other wise?
Because that would just mean that you have to do some import on the workspace that you already had or just newly create.

as an example this is a basic set of components that we made using bootstrap: https://github.com/Servoy/servoy-client … components

[jcompagner]

david:
Servoy’s code for this piece is here: https://github.com/Servoy/sablo. No license listed.

thats why we made this, thats for ngclient the “wicket” part
so that it is just a standalone thing that makes sure that the code is kept clean.

david:
It turns out that the core functionality is implemented agnostic to any client-side framework and NG Client just consumes the functionality with some not-so-elegant client-side code. It gets the job done but some API-esque wiring would help clean up this code.

which code is not so elegant!?

david:
Server-side Servoy forms function to set up a model/foundset. Form has a datasource, the fields you put on the form are the allowable data fields the client can see.

Form items provide additional meta information to the client such as data formatters, calculations, i18n keys, tagging, UI parameters, etc.

yes this is kind of just like wicket in the webclient, the ui component now really lives on the client so we need to push all the properties to the client so that we configure it like the webclient
So how the ui behaves,formats is now all done on the client.
For our default component set this is quite a bit of work, because we want to be as good backwards compatible as possible. But a clean new set of components are way simpler then our default component set
(only the properties like foreground,background and so on can be all gone by just 2 properties “style” and “class”)

david:
Forms also provide services such as when a button is pushed, code on the server is run and a response is sent to the client.

its not really forms, and services is a bit wrong, what you are saying we call handlers (services are singletons most of the time, our plugin that we make to map on the default servoy plugins like window or dialog those are services)
But a handler that is defined in the spec (like onaction) is configures by the serverside so that a client side can just call it “handlers.onaction()” then the attached servoy script on the server side is being executed.

david:
When an NG Client connects, a session is created and (all?) forms in your app have a state per session. Data broadcasting happens between sessions where the foundsets share the same visible foundset index(es). I assume all CRUD operations but edit is the only one I’ve tested so far.

only the forms that are touched are created, the forms are way lighter then in the webclient/wicket. because they are basically just maps of webcomponents and a webcomponent is basically a map of property values.
(because all the ui things are really on the client, the server doesn’t have any differences between a textfield and a combobox, both are just the same class/object instance (which is a map of values))

the thing is that you talk constantly just about foundset, but foundsets are not the only source of data, also globals or formvariables could be the data for the client side (or even just plain datasets if you want)
Also you seem to be saying i want the foundset as the thing that i want to have on the client, (as a proxy). But for servoy this is not the thing that we really talk to, servoy talks to forms and that has or can have a founset
but it is not directly just a plain foundset where you just talk to without having any form.

david:
All of this communication goes back and forth from clients and server via websockets as JSON. If a field is edited, only the data for that field is sent over the wire, not the whole record. Quite compact for most operations from what I’ve seen (initial connection response sends a lot of unnecessary Angular stuff is one exception).

which initial stuff are you really talking about? yes there is some initial request like getting the locale and timezone information from the browser, also a form is fully loaded (requestInitialData) and things like onshow of the form must also be taken care of (because the ui decides if a form is shown so it must tell us that)
Many services need to be instantiated.

david:
If a solution requires login, websocket endpoints are not created until you login.

thats not true, the websocket will be created directly, because the login could be a login form or login solution…
That state is only there when we create the client and load the solution that is requested.

david:
A neat trick is that you can return any JS object from a form method (serialization to JSON happens automatically) and the client websocket.onmessage handler picks it right up. So not only can you trigger foundset methods but you return your own data structures over the pipe. (Note to self, test sending arguments to form methods.)

i do wonder if that works directly out of the box currently
because i think you are talking about client to server communication right?
so the handler of a button like “onaction” event

i am not sure if we really do that currently and what we return, because if you do in client side code:

var x = $scope.handlers.someactionreturnssomethingfromtheserver();

that then x is not directly the value that you return, because thats impossible everything is async, i guess we can return something (if we don’t already) but that is the an promise ($q api)
where you need to register a method on in the then(xxx) function.

the other way already works like this, so server to client calls, like elements.textfield.getSelectedText()

that really does a directly call over the websocket to the actual ui component and ask for the selected text and that then is returned an the code on the server gets the value and moves on.
So the executing of that script really stops for a while so that the async call can execute completely

This is also why modal dialogs or plugins.dialog.showInput() now work again just like the smart client, scripting is just blocked and will move on when the result is there.

[david]

Lot’s of good info Johan, thanks.

If you are creating the endpoints on app initiation but before login, they are at least not accessible — which is a good thing for security purposes.

Form methods can and do return JSON when rolling your own websocket(s). Not sure if this is a side effect of your implementation but it is quite handy. I did make a note to test if it was async or not on the server. If not, need to see if can connect up headless client plugin functionality to websocket traffic. Of course the client is non-blocking and the responses can be queued up and handled in any fashion.

With a form acting as the server-side proxy, it’s just a matter of observing the websocket traffic you are generating to eventually build a full picture of what is happening. This we can use to build our own client-side API.

Regarding cleanup: referring to servoy_app.js > sablo_app.js > websocket.js.

[juan.cristobo]

jcompagner:
as an example this is a basic set of components that we made using bootstrap: https://github.com/Servoy/servoy-client … components

Interesting!!