Resources concept

Hi all,

I would understand better how to use the resoruces.

I have two top solutions: “business” and “magazine”.
Each of them has as module the servoy framework: “svy_framework”.

“business” goes on the db “businessdb”.
“magazine” goes on the db “magazinedb”.

Now, I woundn’t have one big “resource_all” in wich I have to put all .dbi of all possible databases,
so I’ve maked one “resource_business” and one “resource_magazine”.

The problem is that I get a problem with this strategy, because the “svy_framework” have to has a resource equals to the the top solution,
so,in this case, one time “resource_business” and one time “resource_magazine”.
Each time I develop a solution, I have to change (one by one) the resource of svy_framework and it’s modules.

So my question is: What is the right way to organize my resources in this scenario to don’t change every time all the resource’s reference?

Thanks

Riccardo

I have all my solution under a source safe (SVN)

http://www.data-mosaic.com/sutras/cms/d … e-projects

Thanks David,

I’ve taked a look!

It would be more usable if we can have something like a “scope” in wich we can put alle the resources list.
When you create a new dbi, you could specify for one, or a group, in what resource you want to it.

However, thanks again for the informations.

Riccardo

rn:
It would be more usable if we can have something like a “scope” in wich we can put alle the resources list.
When you create a new dbi, you could specify for one, or a group, in what resource you want to it.

Or each module could just have a resources section in addition to the install scoped resources project.

I am not a fan of this scopes thing Servoy introduced with 6.x. At best it is a duct tape solution to much deeper issues with setting up large solutions in Servoy. Extending it further to apply to resources would be a bad idea.

The real fix is to turn Servoy’s module chain upside down – instead of parent modules “seeing” included modules, child modules “see” parent modules. This would give you automatic scoping and name spacing without having to come up with a contrived “scope” node. Available resources would flow naturally down the chain from most general solution available resources to specific module specific resources. You could specify various resources as public or private. Style classes in style sheets down the chain would override identically named styles up the chain. And so on.

Servoy’s form inheritance model works this way. Module inheritance is opposite. Main reason why it is so easy to screw up module organization in large solutions.

sorry but that would be a bit ridiculous, modules are always things that extend functionality for the thing that include that module, not the other way around. Thats just weird.

Scoping is not a duct tape if you say that then you really didn’t the point, the point is that you have functionality that belong to each other in 1 file. If you want you can compare it with the various js libs you have flying around like the YUI libs
Those have a all kinds of different js files doing specific stuff (drag n drop, animations, you name it)
A scope in servoy is just a javascript file by its name.

But i do agree scopes have nothing to do with dbi/resources project that , its just javascript/coding.

A resource project should be seen as a server, its just that. Its a server representation. So if both solutions do run on the same server then it would be just wise to have 1 resource project, if they don’t then you should have 2 resource projects.
Solutions don’t supply a resource project, but are build against a resource project.

jcompagner:
sorry but that would be a bit ridiculous, modules are always things that extend functionality for the thing that include that module, not the other way around. Thats just weird.

The current implementation does lead to contortions on how to organize and include modules in each other without getting cycles. My thinking is that being able to write things at the top level that all sub modules can reference would simplify large solutions.

jcompagner:
Scoping is not a duct tape if you say that then you really didn’t the point, the point is that you have functionality that belong to each other in 1 file. If you want you can compare it with the various js libs you have flying around like the YUI libs
Those have a all kinds of different js files doing specific stuff (drag n drop, animations, you name it)
A scope in servoy is just a javascript file by its name.

I get this. But, there is no “build” process to bind them all together like in JS libraries. They are just javascript files hanging out there. Currently, just an easier way to specify a namespace which can already be done as a variable in the global file. No help in constructing more extensive javascript API libraries. And again, since there is no place to put scopes that apply to all modules in a solution…

jcompagner:
A resource project should be seen as a server, its just that. Its a server representation. So if both solutions do run on the same server then it would be just wise to have 1 resource project, if they don’t then you should have 2 resource projects.
Solutions don’t supply a resource project, but are build against a resource project.

This is a big flaw with Servoy’s organization. Modules can have their own individual projects but when you combine modules into a solution, they all take on the resource project of the solution module. Ideally, the final resource project for the solution should be a combination of all the included resource projects.

The article linked above is the only way we’ve seen that successfully solves architecting large solutions in Servoy with common modules that can be applied to many client projects while keeping everything managed on SVN without tangling with each other. Adding scopes in Servoy 6 did not simplify anything from this perspective.

You really revert it…
Modules are EXACTLY like plugins,jar files you name it java, that is working fine for miljoens of java developers…

You “top level” thing is a module that all modules use.
Yes that “top level” module can’t then access the things in the modules because that doesn’t make sense, its the toplevel module that defines all common things that all other modules use.

How come i never have a problem with cycles in my java projects/eclipse plugins???
It only goes always one way, and yes i have “top level” things, but that is the project that every other project (via via) references.

So i think you really have a wrong definition of what a top level thing is.

What “build” process are you talking about for example in YUI? I just include a bunch of javascript files (same with jquery) and thats it.

If you have a certain defined functionality in side a scope, that you want to access in all other modules… Then just have that in a module (your top level thing) and include that module in all the modules that want to use that.
Looks pretty simple to me

About resources, if a solution or module doesn’t touch certain tables then there is now problem what so ever, but again the resource project really targets the server, so a workspace (with 1 resource project) is really a workspace for a specific server, thats how you should work. Else it would be come one big mess, what happens if you have multiply resource projects but accessing the same kind of server/tables? Then you define on column level X in one resource and Y in another then what?

I think by now this thread is mixing several discussions:
1- scopes
2- topdown/bottomup module inclusion
3- resource projects

As for the first, yes, you already could create your own scopes inside globals, what was added in Servoy 6.1 for that was just a better way to organize it in separate files

As for 2: I think the problem you’re trying to solve would not be solved by a top down module system. The "issue’ you’re talking about is more about managing dependencies between modules, in such a way that you can more easily combine 3rd party or framework modules with custom development and/or extensions. Am I right?

As for 3: the “issue” with resources projects and working with 3rd party and/or framework modules when developing solutions in Servoy and how the monolitic resources project approach in Servoy makes this cumbersome is something that we’re aware of and are looking at to see how we can better support the scenario where you’re building solutions that utilize 3rd party and framework modules.

I think both 2 and 3 are “issues” you run into when developing solutions utilizing 3rd party modules/frameworks and as I mentioned under 3 it’s something we are aware of and are looking into.

Paul

  1. About Riccardo’s initial problem - with the mention that “svy_framework” could be any 3rd party module - I would recommend that you have 2 workspaces:
  • one workspace would contain “business” and “svy_framework” both using “resource_business”.
  • the other workspace would contain “magazine” and “svy_framework” both using “resource_magazine”.

You would no longer need to change resources projects in this setup.
Once you get a new version of “svy_framework” you must import it in both workspaces (so having that ‘3rd party’ solution and module resources project linked to the creator’s team provider location is a no-go currently, because of this need for different resources projects which Paul/David talked about). This import will update the 2 resources projects as needed for “svy_framework”.

But if you are going to deploy both main solutions on the same Servoy Application Server you can safely use the same resources project - “resources_all” (maybe even should use, in order not to get in a situation where you have separate things like sequences/custom values and so on different in the common things for the 2 resources projects which is not possible on the same server). It makes more sense to do this separation if you will (or might some day) deploy the 2 main solutions on separate Servoy Application Servers, each server having access only to the needed sub-set of databases. If you do this separation you might also want to consider starting the developer with different servoy.properties configuration, in order to have the correct sub-set of databases used at design-time as well.

Having multiple resources projects in the same workspace would make sense if the set of solutions that use each of them is completely independent (no solutions need to use both one and the other - equivalent to the same solution needing to be deployed to separate Servoy Application Servers).

  1. About David’s module problem. Is there a concrete scenario where what Johan mentioned (if you want something accessible from many modules, that must be present in a child module of all of those) would not work? It seems to me as well that it’s the same problem no matter how you think of the dependencies (reversed or not). If you want a module to offer some functionality based on a state from the parent solution/modules, the module must define a way for that state to be provided to it (an abstract state - like an interface - that one can provide and the other can use; the module itself or a separate sub-module of the 2 could be used for defining this state), similar to what Johan mentioned about jar dependencies.

I think there was also another discussion somewhere about how one can get rid of the dependency cycles.

BTW where is the dependency of a module to a resource project stored?

I’m asking because I’m building a dependency checker tool right now and I cannot find it in any of the solution files (rootmedatada.obj, solution_settings.obj, .buildpath and .project)…

Found it! In fact it’s an Eclipse project dependency, to be found in .project

See viewtopic.php?f=15&t=18988

pbakker:
I think by now this thread is mixing several discussions:
1- scopes
2- topdown/bottomup module inclusion
3- resource projects

I think both 2 and 3 are “issues” you run into when developing solutions utilizing 3rd party modules/frameworks and as I mentioned under 3 it’s something we are aware of and are looking into.

Right on Paul. Reusing modules and updating them independently of particular client solutions and servers is a PITA right now with Servoy. Not something I’ve seen other techs completely solve yet either so tough nut to crack I think.

Some thoughts for your “looking into” process:

Real solution layer
Aggregate resources
Can public anything
Scoping at the module level

REAL SOLUTION LAYER

Replace the “active module equals the solution” concept with a real solution node to give us a clear top-down scope chain: Database > Server > Solution > Module > Object.

The current way of “faking” a solution by flattening out a bunch of modules that are included in each other is overly complex and makes creating a new client workspace/project with modules coming from everywhere a work of art to achieve.

With a real solution node, a new client project would instead simply entail: 1) create a new solution; 2) include various stand alone modules into the solution (navigation, layout, access and control, code, a cms, a crm, document manager, etc); 3) create custom modules for the client.

The solution object would have most of the properties (ie. “firstForm”) and events (ie. “onOpen”) modules currently have. Modules would have their own properties and events (ie. “moduleType”, “requires”, etc).

A solution by definition requires at least one module.

AGGREGATE RESOURCES

jcompagner:
About resources, if a solution or module doesn’t touch certain tables then there is now problem what so ever, but again the resource project really targets the server, so a workspace (with 1 resource project) is really a workspace for a specific server, thats how you should work. Else it would be come one big mess, what happens if you have multiply resource projects but accessing the same kind of server/tables? Then you define on column level X in one resource and Y in another then what?

I think the answer is easier than it looks – Servoy has already solved the tougher piece to this puzzle: syncing/listening workspace .dbi files with database tables.

One additional piece is needed: implement a way for the workspace .dbi files to sync/listen to table definitions included at the module level.

This would require a specific format for specifying table definitions at the module level. Musing out loud I would suggest that the workspace .dbi files remain the default approach. Then at the module level functions could be created to modify the default workspace .dbi’s. Servoy checks through these functions when a client is started, builds up the workspace .dbi’s, and throws various warnings for duplicate columns, dbi’s out of sync with database, etc.

I had this discussion of building up resources (including data connections) programmatically per solution/project last week at a conference with some very smart people types and it turns out to not be an uncommon idea/approach in the JS stack world.

However it’s done, the ability for modules to carry around their .dbi information needs to be solved to be able to plug and play modules anywhere we like with zero fuss.

EVERYTHING CAN BE TAGGED AS “PUBLIC”

Independent module design depends on the ability to include and export to other modules (see http://www.commonjs.org). Recent Servoy versions have been going in the right direction with being able to tag things as “public”. Forms got this recently via an encapsulation property and functions have had it for a while.

But that leaves out other objects like value lists and relations. Because of this we almost always have to split major modules into two modules – one module for all the api stuff that we want to share (forms, relations, calcs, value lists, code), and one module for the specific private functionality. The api module gets included into the private module and then can also be included in any other module as needed.

This situation basically negates the whole public/private capabilities Servoy currently has. You might as well just tag modules as public or private.

So the obvious question now is: what about potential naming conflicts with making say – a relation – public? Glad you asked…

SCOPING AT THE MODULES LEVEL

Rip out current globals and scopes. There I said it.

Instead, namespace modules. Maybe with a module shortname property. The only thing a developer needs to handle then is to make sure all modules in a solution have unique shortnames.

Now, within a module all your naming conventions and referencing remains the same. However, another module referencing any public object does so with “shortname + DOT + object” notation. Examples:

barcode.forms.thisGreatForm.controller.someFunction()
(barcode.globals.globalSheetName.someProperty == "Version 3.02")

Notice I do have a “globals” object still so I lied a little at the start of this section. What I’d rather see is a reworking of globals/scopes to make things less wordy:

  • globals node is no longer a js file of it’s own, just a place to put javascript files.
  • there is automatically a js file in the globals node per module that has the same name as the module shortname.

A real bonus would be to have a special globals node on the solution as well. The solution object would have an implicit identifier (“Servoy?”, “solution?”) to reference global jsFiles.functions() (ie. “solution.brilliantCodeSheet.someFunctionAllSolutionModulesSeeByDefault()”).

Finally, objects within modules have to have unique names but not within a solution. Same form name in two modules in a solution? No problem. (Finally….)

SUMMARY

The ultimate goal is complete module portability. Modules need to be name spaced, carry everything they need to function with them (resources), public/private completely contained, and a solution object to plug them into.

so in a modules js file (form or scope/global.js)
if you want to access a form of that module
do you need to know that you have to prefix it, so in other words in any solution you are everything is prefixed?

so in solution Y
everything you access
solutiony.forms.aforminsolutiony.controller.show()

or do you want to program inside a solution y as

forms.aforminsolutiony.controller.show()

So inside a solution itself you don’t want to prefix with its own name? Or do you also want to prefix it then?

Second way – no prefix if referencing objects in same module.

That would mean a huge huge change in servoy, that will not happen
because “forms” is a top level thing, that is how javascript works, you have vars locally and through scoping but things all other things just go into the toplevel scope
Modules are only really modules in the developer not at runtime (just like java and includes jars), there we don’t have any concept of a module, but when we would change that, that forms and scopes is specific to a module everything needs to change
Developer, but also the runtime(s), not to mention that it would break all existing solutions.

@David

Talking great sense as usual! Very nicely put.

Hi David,

Tnx for you (elaborate) input.

Paul

jcompagner:
That would mean a huge huge change in servoy, that will not happen
because “forms” is a top level thing, that is how javascript works, you have vars locally and through scoping but things all other things just go into the toplevel scope
Modules are only really modules in the developer not at runtime (just like java and includes jars), there we don’t have any concept of a module, but when we would change that, that forms and scopes is specific to a module everything needs to change
Developer, but also the runtime(s), not to mention that it would break all existing solutions.

Just how huge of a change would it be? I’m not sure but it seems to me that it would be mostly Servoy developer behavior changes. At the server, a solution and associated modules would still be flattened out so I don’t think the repository structure would change much.

the huge part (which is also breaking for all solutions) is that modules now have no meaning what so ever in the client itself only in developer
But in what you describe then suddenly modules are really affecting also the clients because the whole scripting “tree” (Servoy Object Model) is different.

they are not just flattened out because you want to program access to solutions like : solutionname.forms.xxx right?
(and not for just inside the module itself there forms.xxx just works and works only for its forms, and duplicates between forms are then on different levels)

Or maybe i didn’t understand it right what you want… But if you want to prefix access to module stuff by putting the module name in front of it and having the ability to have duplicate form names accross modules
that really would mean that the runtime must change big time