Speech Recognition and Virtual Assistant (Nuance)

• Sep 09, 2024
• 3 minutes to read

This topic explains how to use the Nuance Dragon SpeechKit to enable speech recognition and voice-activated workflows in the DevExpress Rich Text Editor.

Prerequisites

• A Nuance Dragon license
• References to Nuance resources in the header section

Register Controls in SpeechKit’s Configuration Method

Declare a NUSA_configure method. The SpeechKit framework calls this method so you can register controls that need voice-enabled functionality. Once you register a RichEdit control, it can process default SpeechKit commands.

Use the nusaSettings property to access the RichEdit’s Nuance-related settings.

JS

window.NUSA_configure = function() {
    // ...
    // register every RichEdit control that should work with SpeechKit functionality
    richEdit1.nusaSettings.register();
    richEdit2.nusaSettings.register();
}

Add Support for the “Go To” Command (Input Focus Switch)

Specify the conceptName option in the register method to allow users to focus a RichEdit with the go to voice command.

JS

function NUSA_configure() {
    // ...
    // register every RichEdit control that should work with SpeechKit functionality
    richEdit1.nusaSettings.register({ conceptName: 'patient description' });
    richEdit2.nusaSettings.register({ conceptName: 'medical history' });
}

The method registers the RichEdit control and its container with the “DevExpressRichEdit” and “DevExpressRichEditContainer” names respectively. You can use the customControlType and customContainerType parameters of the NusaRegisterOptions interface to specify custom type names.

Enable Commands Specific to RichEdit

Rich Text Editor implements the following custom commands:

Name	Phrase	Description	Phrase Example
DxReFontColor	‘set <DxReColorName> font color’	Set font color	‘set red font color’
DxReFontSize	‘font size <standard:cardinal0-100>’	Change font size	‘font size twelve’

Call the registerCommands method to register these commands. Note that this method should be called only once regardless of the number of RichEdit controls on the page.

JS

function NUSA_configure() {
    // ...
    richEdit1.nusaSettings.registerCommands();
}

Customize the Registered Commands

The registerCommands method accepts an optional parameter that allows you to modify command settings. For instance, you can change a command description that is displayed in the SpeechKit’s personalization & help window.

JS

richEdit.nusaSettings.registerCommands(function(commandSets/*: NusaCommandSet[]*/, placeholders: /*NusaPlaceholder[]*/) {
    commandSets[0].commands.find(function(cmd){
        return cmd.commandId == "DxReFontColor"}).description = 'Custom description';
});

Custom User Commands

You can add custom RichEdit commands in the SpeechKit as follows:

1. Create custom command sets (NusaCommandSet).
2. Create custom placeholders (NusaPlaceholder).
3. Cal the registerCommands method to register the command sets and placeholders.
4. Implement the NUSA_onCommandRecognized function that is called when an application command is recognized. Add a command handler for every registered command.

JS

function NUSA_onCommandRecognized(cmdId, spokenText, content, placeholderIds, placeholderValues) {
    // your custom commands
    switch(cmdId) {
        case 'customName1': {
            // command handler
        }
        case 'customName2': {
            // command handler
        }
    // ...
}

In the event handler you can use the getFocusedRichEdit method to access the focused RichEdit control.

JS

/** @type {DevExpress.RichEdit.RichEdit} */
const focusedRichEdit = richEdit.nusaSettings.getFocusedRichEdit();

Modify the RichEdit’s SpeechKit-Related Settings

The NusaCustomControlType and NusaCustomContainerType classes allow you to modify the RichEdit’s SpeechKit-related settings (for instance, insert two paragraphs with the new paragraph voice command). You can use the getCustomControl and getCustomContainer methods to access the current instances of these classes.

JS

function NUSA_configure() {
    richEdit.nusaSettings.register();
    var customControl = richEdit.nusaSettings.getCustomControl();
    customControl.paragraphFormat = DevExpress.RichEdit.Characters.ParagraphBreak + 
        DevExpress.RichEdit.Characters.ParagraphBreak;
}