Audio Input Bean

I am getting further out of my depth, but ill try anything once!

Trying to create a bean/applet/plugin to get the audio input into servoy and then to store it into my database.

So far and by some miracle i have managed to compile a java program which takes the input from my mic and stores it in a .wav file.

I suppose the next move is to try and work out how to create that java program as bean so that can show the GUI as an object on my form and return the data to a variable in Servoy.

From then on i think Marcel’s data streaming plugin might be needed.

Therefore i think i need to take the code below and strip out the file creation bits and return the input back to servoy.

Is this quite simple or should i give up now!!!

Are there any good pointers on creating beans?

David

One of Servoy’s beans includes the source code. Just check that out, it’s very simple.

Patrick,

Cant see any demo source code.

marcel suggests i use a plugin.

I have got up to the stage of getting a ‘hello world’ plugin to work.

Is there a way that i can simply wrap an externally working java class with its interface within a plugin,

all i want the plugin to do is run my application and then as it generates a freestanding audio file, i could then get servoy to read that file into the database as a separate entity.

I suppose a more sophisticate way of doing it would be to stop my application writing the file and get it to store the contents of the received audio into a variable and then return the variable back to servoy.

i attach the code so you can see what i am trying to do. I jst dont know whether i am just so out of my depth or whether actually i am quite close.

I must say i was fairly pleased with myself when i got the java script to compile and work.

I just see a lot of interfaces other than the Iscript object and wonder whether one istn their to be able to refer my raw code?

Many thanks for any further pointers

David

/*File AudioRecorder02.java
Copyright 2003, Richard G. Baldwin

This program demonstrates the capture of audio
data from a microphone into an audio file.

A GUI appears on the screen containing the
following buttons:
  Capture
  Stop

In addition, five radio buttons appear on the
screen allowing the user to select one of the
following five audio output file formats:

  AIFC
  AIFF
  AU
  SND
  WAVE

When the user clicks the Capture button, input
data from a microphone is captured and saved in
an audio file named junk.xx having the specified
file format.  (xx is the file extension for the
specified file format.  You can easily change the
file name to something other than junk if you
choose to do so.)

Data capture stops and the output file is closed
when the user clicks the Stop button.

It should be possible to play the audio file
using any of a variety of readily available
media players, such as the Windows Media Player.

Not all file types can be created on all systems.
For example, types AIFC and SND produce a "type
not supported" error on my system.

Be sure to release the old file from the media
player before attempting to create a new file
with the same extension.

Tested using SDK 1.4.1 under Win2000
************************************************/

import javax.swing.*;
import java.awt.*;
import java.awt.event.*;
import java.io.*;
import javax.sound.sampled.*;

public class AudioRecorder02 extends JFrame{

  AudioFormat audioFormat;
  TargetDataLine targetDataLine;

  final JButton captureBtn =
                          new JButton("Capture");
  final JButton stopBtn = new JButton("Stop");

  final JPanel btnPanel = new JPanel();
  final ButtonGroup btnGroup = new ButtonGroup();
  final JRadioButton aifcBtn =
                        new JRadioButton("AIFC");
  final JRadioButton aiffBtn =
                        new JRadioButton("AIFF");
  final JRadioButton auBtn =//selected at startup
                     new JRadioButton("AU",true);
  final JRadioButton sndBtn =
                         new JRadioButton("SND");
  final JRadioButton waveBtn =
                        new JRadioButton("WAVE");

  public static void main( String args[]){
    new AudioRecorder02();
  }//end main

  public AudioRecorder02(){//constructor
    captureBtn.setEnabled(true);
    stopBtn.setEnabled(false);

    //Register anonymous listeners
    captureBtn.addActionListener(
      new ActionListener(){
        public void actionPerformed(
                                  ActionEvent e){
          captureBtn.setEnabled(false);
          stopBtn.setEnabled(true);
          //Capture input data from the
          // microphone until the Stop button is
          // clicked.
          captureAudio();
        }//end actionPerformed
      }//end ActionListener
    );//end addActionListener()

    stopBtn.addActionListener(
      new ActionListener(){
        public void actionPerformed(
                                  ActionEvent e){
          captureBtn.setEnabled(true);
          stopBtn.setEnabled(false);
          //Terminate the capturing of input data
          // from the microphone.
          targetDataLine.stop();
          targetDataLine.close();
        }//end actionPerformed
      }//end ActionListener
    );//end addActionListener()

    //Put the buttons in the JFrame
    getContentPane().add(captureBtn);
    getContentPane().add(stopBtn);

    //Include the radio buttons in a group
    btnGroup.add(aifcBtn);
    btnGroup.add(aiffBtn);
    btnGroup.add(auBtn);
    btnGroup.add(sndBtn);
    btnGroup.add(waveBtn);

    //Add the radio buttons to the JPanel
    btnPanel.add(aifcBtn);
    btnPanel.add(aiffBtn);
    btnPanel.add(auBtn);
    btnPanel.add(sndBtn);
    btnPanel.add(waveBtn);

    //Put the JPanel in the JFrame
    getContentPane().add(btnPanel);

    //Finish the GUI and make visible
    getContentPane().setLayout(new FlowLayout());
    setTitle("Copyright 2003, R.G.Baldwin");
    setDefaultCloseOperation(EXIT_ON_CLOSE);
    setSize(300,120);
    setVisible(true);
  }//end constructor

  //This method captures audio input from a
  // microphone and saves it in an audio file.
  private void captureAudio(){
    try{
      //Get things set up for capture
      audioFormat = getAudioFormat();
      DataLine.Info dataLineInfo =
                          new DataLine.Info(
                            TargetDataLine.class,
                            audioFormat);
      targetDataLine = (TargetDataLine)
               AudioSystem.getLine(dataLineInfo);

      //Create a thread to capture the microphone
      // data into an audio file and start the
      // thread running.  It will run until the
      // Stop button is clicked.  This method
      // will return after starting the thread.
      new CaptureThread().start();
    }catch (Exception e) {
      e.printStackTrace();
      System.exit(0);
    }//end catch
  }//end captureAudio method

  //This method creates and returns an
  // AudioFormat object for a given set of format
  // parameters.  If these parameters don't work
  // well for you, try some of the other
  // allowable parameter values, which are shown
  // in comments following the declarations.
  private AudioFormat getAudioFormat(){
    float sampleRate = 44100.0F;
    //8000,11025,16000,22050,44100
    int sampleSizeInBits = 16;
    //8,16
    int channels = 1;
    //1,2
    boolean signed = true;
    //true,false
    boolean bigEndian = true;
    //true,false
    return new AudioFormat(sampleRate,
                           sampleSizeInBits,
                           channels,
                           signed,
                           bigEndian);
  }//end getAudioFormat
//=============================================//

//Inner class to capture data from microphone
// and write it to an output audio file.
class CaptureThread extends Thread{
  public void run(){
    AudioFileFormat.Type fileType = null;
    File audioFile = null;

    //Set the file type and the file extension
    // based on the selected radio button.
    if(aifcBtn.isSelected()){
      fileType = AudioFileFormat.Type.AIFC;
      audioFile = new File("junk.aifc");
    }else if(aiffBtn.isSelected()){
      fileType = AudioFileFormat.Type.AIFF;
      audioFile = new File("junk.aif");
    }else if(auBtn.isSelected()){
      fileType = AudioFileFormat.Type.AU;
      audioFile = new File("junk.au");
    }else if(sndBtn.isSelected()){
      fileType = AudioFileFormat.Type.SND;
      audioFile = new File("junk.snd");
    }else if(waveBtn.isSelected()){
      fileType = AudioFileFormat.Type.WAVE;
      audioFile = new File("junk.wav");
    }//end if

    try{
      targetDataLine.open(audioFormat);
      targetDataLine.start();
      AudioSystem.write(
            new AudioInputStream(targetDataLine),
            fileType,
            audioFile);
    }catch (Exception e){
      e.printStackTrace();
    }//end catch

  }//end run
}//end inner class CaptureThread
//=============================================//

}//end outer class AudioRecorder02.java

As always i am replying to myself, because servoy is just so damn brilliant.

I have now managed to write my own simple plugin which allows an audio file to be created in the root of servoy and the recording started and stopped from within Servoy!!!

and i really dont understand this level of JAVA, but it is a testimony to Servoy that from the simple tutorial on creating a hello world plugin one can add some snippets of someone elses code to achieve all this.

David

dpearce:
I am getting further out of my depth, but ill try anything once!

Trying to create a bean/applet/plugin to get the audio input into servoy and then to store it into my database.

So far and by some miracle i have managed to compile a java program which takes the input from my mic and stores it in a .wav file.

I suppose the next move is to try and work out how to create that java program as bean so that can show the GUI as an object on my form and return the data to a variable in Servoy.

From then on i think Marcel’s data streaming plugin might be needed.

Therefore i think i need to take the code below and strip out the file creation bits and return the input back to servoy.

Is this quite simple or should i give up now!!!

Are there any good pointers on creating beans?

David

David

This is super simple, I managed it !! ;) The solution is already done for you with Marcels Media Manager Bean.

Basically I used it for a radio commercial producer who needed to get all of their production and the resultant commercials to radio stations across the UK. Hence the need for lots of audio formats (MP3, WAV, etcetc ) together with pdfs of the scripts, word docs, excel files etc all stored with the audio.

The process was - Servoy solution for the client in his office connected to our server at the data centre (hence streaming). The staff upload copy, audio etc against each job file. The client can then trac the jobs and download the media using his web site with a login to their work. We used Lasso for the site and toyed with flash etc for streaming audio playback prior to download. Lasso serves the audio without a problem. This was all done prior to the web client being available and I don’t know if that will serve the audio.

To make this a reality take a standard install of Servoy (no cute java stuff) install Marcel’s media manager bean. Decide where you want to store your audio, file system or blob. IF your using a blob make sure the db can take the file size (mySQL is limited to 1mb in standard format so change the config) Stick the bean on the screen where you want your user to upload the file and then drag/drop the files onto the bean. Marcel as usual does an excellent demo and it works out of the bag so to speak - so give that a try.

Playback in Servoy is even easier - Pop up a URL with a flash audio player in it and it will stream the audio back to your user with a nice flashed up interface - if your in the mood for spending money with Marcel ! get the browser bean and you can have your window in your solution - very cool for Audio and Video.

Gotchas:

  1. Make sure your blob can take the file size !!
  2. Before you use the db to store your media consider how you will back it up the get HUGE
  3. Decide how your user is going to move the files you can ether script them from a > b or use drag and drop (personally I prefer drag/drop)

Hope this helps

Gordon

Thanks Gordon,

my real issue is that i want this for medical consultants to use and simply dictate into 8-10 different sections for a report they are creating.

It needs to be very simple, so i was looking to have a button on the screen for them to record into, rahter than record a file and then drage it to the screen and then stream it.

Where i have got to (very proud!) is that i now have designed a plugin which will allow servoy to start and stop an audio input and store the results as a file (i will work on it just being a variable as a next step).

The next problem is that JAVA doesnt seem to have much native audio compression, so even a 8000 freq of recording the files remain probably too large.

I have found a JAVA MP3 encoder that will work client side, although it does need a DLL installing along with the JARS. If i can get this to work then i will have a plugin that allows the input from a microphone to be started and stopped, converted to MP3 and then probably again using marcels media manager bean streamed down to the server.

Its all a bit above my head, but slowly i think i am bodging my way through it.

Thanks for the input. Any other suggestions welcome.

david

dpearce:
Thanks Gordon,

my real issue is that i want this for medical consultants to use and simply dictate into 8-10 different sections for a report they are creating.

It needs to be very simple, so i was looking to have a button on the screen for them to record into, rahter than record a file and then drage it to the screen and then stream it.

Where i have got to (very proud!) is that i now have designed a plugin which will allow servoy to start and stop an audio input and store the results as a file (i will work on it just being a variable as a next step).

The next problem is that JAVA doesnt seem to have much native audio compression, so even a 8000 freq of recording the files remain probably too large.

I have found a JAVA MP3 encoder that will work client side, although it does need a DLL installing along with the JARS. If i can get this to work then i will have a plugin that allows the input from a microphone to be started and stopped, converted to MP3 and then probably again using marcels media manager bean streamed down to the server.

Its all a bit above my head, but slowly i think i am bodging my way through it.

Thanks for the input. Any other suggestions welcome.

david

Thats an interesting idea, it has a number of uses for phone call monitoring too which I am very keen on, but don’t have the ability to code so let me know if you get it cracked!! If your feeling really adventurous a phone dialer would be good too !!

Cheers and good luck!!!
Gordon

Gordon,

I dont know about a phone dialer!! I think feeling my way in the dark is a bit of an understatement here.

The good news is that i have go this far:

  1. Two commands can be sent to the plugin Start Record, and Stop Record.
  2. The plugin then creates a local WAV file.
  3. The plugin then converts this to a .spx file on Stop.

The .spx file is amazingly small compared to the WAV file. MP3 was a bit dfficult as it needed dll’s etc. the JSpeex Codec was with a lot of trouble quite easy to install into my classpath. (I assume that when i put it in my Servoy LIB it will go into the classpath of the Servoy client.

So a 30 second dictation of 2.5MB is now 80K.

I will let you have a test version shortly.
I need to try and get the SPX player integrated into Servoy as well.

David

Bit of a cross post but:

The plugin and source code of David’s Audio plugin are now available on the ServoyFORGE website.

WHy it doesnt work in client!

I have just picked up the reason this never worked in client. I never wrote the jnlp file!

I am not really sure how to do this, but i am sure one of the clever servoy guys will do it if someone really needs it.

David

Whomever writes it, just send it my way and I will put it up the same website the plugin is hosted.

dpearce:
WHy it doesnt work in client!

I have just picked up the reason this never worked in client. I never wrote the jnlp file!

I am not really sure how to do this, but i am sure one of the clever servoy guys will do it if someone really needs it.

David

You cannot just add the jars to the lib folder, you have to create a folder in the plugins folder and write a jnlp file to tell java webstart which files have to be downloaded.
You can take this as a starting point:

<?xml version="1.0" encoding="utf-8"?>
<jnlp spec="1.0+"
	codebase="%%serverURL%%"
	href="/servoy-client/plugins/sintproScannerPlugin.jar.jnlp">
	<information>
		<title>Sintpro Scanner Plugin</title>
		<vendor>Servizi Integrati Professionali s.a.s.</vendor>
		<homepage href="http://www.sintpro.com/scannerplugin.php"/>
		<description>SintPro Scanner Plugin</description>
		<offline-allowed/>
	</information>
	<resources>
		<jar href="/plugins/sintproScannerPlugin.jar" download="eager" version="1.0.1"/>
		<jar href="/plugins/sintproScannerPlugin/morena.jar" download="eager"/>
		<jar href="/plugins/sintproScannerPlugin/gif-plugin-0.1.jar" download="eager"/>
	</resources>
	<resources os="Windows">
		<jar href="/plugins/sintproScannerPlugin/morena_windows.jar" download="eager"/>
	</resources>
	<resources os="Mac OS X">
		<jar href="/plugins/sintproScannerPlugin/morena_osx.jar" download="eager"/>
	</resources>
	<component-desc/>
</jnlp>

Remember to sign all your jars otherwise java web start will prevent the download, use google for java webstart signing.