How to implement Web Speech API in NG Titanium?

Questions
1

Hi everyone,

I am looking for guidance on implementing the Web Speech API (Speech Recognition) within a Servoy NG Titanium application and capturing the recognized text.

My Environment:

  • OS: Windows

  • Browser: Google Chrome

  • Servoy Version: 2024.03 (NG Titanium)

The Issue:
I have been exploring clientutils.generateBrowserFunction, but I am struggling with two specific parts of the workflow:

  1. How to correctly trigger the client-side speech recognition process from the server.

  2. How to pass the recognized text back from the browser to my Servoy solution once the recognition is complete.

Does anyone have a code snippet or a recommended pattern for handling this type of client-side event and callback in NG Titanium?

This is some code:

function iniVoiceRecognition() {
    var jsCode = "var SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition;" +
                 "if (!SpeechRecognition) {" +
                 "  alert('Your webBrowser not support Voice Recognition');" +
                 "  return;" +
                 "}" +
                 "var recognition = new SpeechRecognition();" +
                 "recognition.lang = 'es-ES';" +
                 "recognition.continuous = false;" +
                 "recognition.interimResults = false;" +
                 "recognition.onresult = function(event) {" +
                 "  var transcript = event.results[0][0].transcript;" +
                 "  callback(transcript);" +
                 "};" +
                 "recognition.onerror = function(event) {" +
                 "  alert('Error: ' + event.error);" +
                 "};" +
                 "recognition.start();";
    
    clientutils.generateBrowserFunction(jsCode);
    
}

function updateText(_recognizedText) {
    // Asignar el texto al campo
    myField = _recognizedText;
}

Thanks in advance for your help!

2

Hi everyone,

Does anyone know the correct way to execute code injected via clientutils.generateBrowserFunction(jsCode) in NG Titanium?

In the old WebClient, we could simply use WebClientUtils.executeClientSideJS(jsToExecute) to trigger immediate execution. However, in NG Titanium, the behavior seems different.

Is there a specific “trick” or a different service I should be using to invoke these client-side functions once they have been generated? I am looking for the NG Titanium equivalent of the direct execution we had in WebClient.

Any insights or best practices would be greatly appreciated.

Un Saludo

3

generateBrowserFunction() is used when injecting a client side function into another component or service. There’s no way to execute it directly outside an angular component/service.

AFAIK the only way to do this is to create your own angular service which is registered as a plugin in Servoy.

i.e., The toastr plugin is an angular service that you could use for inspiration.

4

Hi everyone,

Following up on the previous suggestions, I have successfully implemented a Web Component for Speech Recognition in NG Titanium. I am attaching it here for the community to use or improve.

Download here speechrecognition

API Overview:

The component is quite robust and includes a full suite of functions to manage the recognition lifecycle:

1. Recognition Control

  • startRecognition(language, onResult, ...): Initializes the Web Speech API. Supports BCP-47 tags, auto-restart (keepListening), and custom silence timeouts.

  • stopRecognition(): Immediately terminates the session and prevents restarts.

  • pauseRecognition() / resumeRecognition(): Allows releasing the microphone temporarily without losing the current configuration.

  • isRecording(): Returns the active listening state.

  • getStatus(): Returns the current state (idle, listening, paused, or stopped).

2. Configuration & Filtering

  • setLanguage(language): Changes the target language for the next session.

  • setConfidenceThreshold(value): Discards results below a specific accuracy level (0-1).

  • setNoiseFilter(enabled): A specialized filter that discards short, low-confidence results—ideal for noisy environments like warehouses.

  • setMaxAlternatives(n): Configures the browser to return up to 5 transcription alternatives.

3. Advanced Callbacks

  • onInterimResult: Captures real-time partial results while the user is still speaking.

  • onStatusChange: Monitors transitions between states.

  • setOnSilenceDetected(): Fires just before the session stops due to silence, allowing for UI updates (like dimming a recording icon).

  • setOnAlternatives(): Captures the full array of transcription options.

4. Voice Commands & History

  • addVoiceCommand(command, callback): Registers specific keywords (e.g., “save”, “next”) to trigger Servoy methods directly.

  • setSynonyms(command, synonyms): Links multiple phrases to a single command.

  • getHistory() / getLastResult(): Accesses the session log of recognized text and confidence scores.

Compatibility Note:

This component currently relies on the Web Speech API, which is primarily supported in Google Chrome and Microsoft Edge.

Question regarding NGDesktop:

Does anyone know if there is a way to make this work within NGDesktop? Since NGDesktop is based on Electron/Chromium, I expected it to work out of the box, but I am facing some challenges with microphone permissions and API availability within the desktop container.

Has anyone successfully implemented voice recognition in the desktop client?

Thanks!

1 Like
5

Nice. The zip is just the distributable right? Could you also make the project source available on github or similar?

6

Hi,

Thanks for the feedback!

You are correct, the ZIP file I shared earlier is the distributable version.

As requested, I have now made the full source code available on GitHub for the community to review, fork, or improve. You can find the repository, including the component logic, manifest, and build scripts, here:

https://github.com/juanetec/speechrecognition-servoy

I would love to see this project grow, so please feel free to submit pull requests or open issues if you find any bugs or have ideas for new features.

Regarding my previous question, I am still looking for insights on NGDesktop compatibility. Since it’s an Electron-based container, there might be a specific way to handle microphone permissions or API flags to make the Web Speech API work there as it does in Chrome/Edge.

Has anyone in the community successfully tackled microphone access within NGDesktop?

Best regards,

2 Likes
7

Hi Juan,

thanks for sharing the GitHub link, much appreciated.

Regarding your Electron question: I’m not sure if this directly applies to your setup, but I ran into quite a few issues when my server was running over plain HTTP. In particular, I had recurring permission problems with things like webcam access and native browser/electron notifications.

After switching the server to HTTPS, most of those permission-related issues disappeared.

Maybe this is something worth checking on your side as well.

8

Hi,

Thanks for the feedback regarding the HTTPS setup!

I have already tested the component using both HTTP and HTTPS protocols, but unfortunately, the issue persists in NGDesktop. Interestingly, some parts of the component are working—for example, calling plugins.speechrecognition.getStatus() returns the correct state. This leads me to believe the core issue is specifically related to Electron’s microphone permissions or the way the container handles the Speech Recognition service.

In the Servoy console, I am seeing the following error when attempting to start recognition:

[SpeechRecognition] Error: network      
at o.buildStackMessage ([https://www.etecsoft.net:8443/servoyServer/chunk-75KVMOCP.js:455:3886](https://www.etecsoft.net:8443/servoyServer/chunk-75KVMOCP.js:455:3886))     
at Object.apply ([https://www.etecsoft.net:8443/servoyServer/chunk-75KVMOCP.js:455:726](https://www.etecsoft.net:8443/servoyServer/chunk-75KVMOCP.js:455:726))     
at e.recognition.onerror ([https://www.etecsoft.net:8443/servoyServer/chunk-75KVMOCP.js:784:26780](https://www.etecsoft.net:8443/servoyServer/chunk-75KVMOCP.js:784:26780))

Since NGDesktop is an Electron-based container, it likely requires an explicit permission handler to allow media access. Based on some research, I suspect we might need to implement something like this in the main process:

session.defaultSession.setPermissionRequestHandler((webContents, permission, callback) => {
    if (permission === 'microphone' || permission === 'media') {
        return callback(true); // Grant permission
    }
    callback(false);
});

Does anyone know if there is a way to configure these defaultSession permissions within the NGDesktop framework, or if there’s a specific flag we should be setting in the ngdesktopfile or ngdesktoputils plugins to authorize the microphone and ensure the speech service can connect?

I’m looking forward to any insights from the NGDesktop experts!

Best regards,

9

Hi Juan,

good point.

Some time ago, I implemented native browser notifications inside a Servoy service. As part of that, I added a small function to explicitly request notification permissions from the browser (i.e., whether notifications are allowed or denied).

It looked roughly like this:

requestPermission() {
    if (typeof window === "undefined" || !Notification) {
        return;
    }
    Notification.requestPermission().then(permission => {
        return permission;
    });
}

This function was called once during initialization. After the user granted permission, notifications worked reliably in ngDesktop & Browser.

Maybe this approach helps in your case as well.

For reference, here’s the test project I created back then:
GitHub - r4inX/svyNotifications: test servoy angular project / wpm

10

Hi,

Thanks for the suggestion and for sharing your project!

While the Notification.requestPermission() approach is great for alerts, the Web Speech API handles permissions differently. It typically triggers the permission prompt automatically when recognition.start() is called, or requires checking navigator.permissions for the 'microphone' descriptor.

I suspect my issue is deeper because of the specific Error: network I am seeing. In the Web Speech API documentation, a “network” error often means that the browser’s speech recognition engine (which in Chromium-based browsers like the one in NGDesktop, usually relies on Google’s cloud services) is unable to communicate with the recognition servers.

This could be happening because Electron Sandbox: The NGDesktop container might be restricting the outbound connection required by the recognition engine.

Best regards,

1 Like