Selecting Excel Cells using Interop and Speech Recognition with C#

I’ve been investigating the idea of manipulating Excel spreadsheets by using voice commands. One task is to select a range of cells along the lines of “Select cells A1 to E7”. The trick to get this to work is to use an Excel add-in (for some UI), Excel interop (to manipulate a worksheet programmatically), and speech recognition (to input a command).


So, there are a lot of prerequisite needed and they are quite version-sensitive. I was using Excel 2013 with Visual Studio 2012. To create the add-in I installed the Microsoft Office Developer Tools for Visual Studio 2012 ENU. This gave me an Excel add-in template so when I did File -> New -> Project, I was able to select the C# -> Office Add-ins -> Excel 2013 Add-in template. Apparently this also gave me the ability to do interop because the template generated code contained a ” using Excel = Microsoft.Office.Interop.Excel;” statement.

To use speech, I installed three 32-bit (64-bit doesn’t work with add-ins) Microsoft Speech Platform 11 packages: the SDK (to create speech recognition in VS), the Runtime (to use speech on my client machine), and English Language pack MSSpeech_SR_en-US_TELE.msi for recognition. Because I was not using speech synthesis, that is, making the computer talk, I did not download and install MSSpeech_TTS_en-US_Helen.msi (“text-to-speech”).

I named my demo Project ExcelAddInTest. In the Solution Explorer window, I right-clicked on the Project and selected Add -> New Item, User Control and renamed the associated file from UserControl1.cs to MyUserControl.cs. This brought me to a design surface with what is essentially a WinForm. I added a ListBox control to display diagnostic messages.

In file ThisAddIn.cs, I added this code:

public partial class ThisAddIn
  public MyUserControl muc;
  public CustomTaskPane ctp;

  private void ThisAddIn_Startup(object sender, System.EventArgs e)
    muc = new MyUserControl();
    ctp = this.CustomTaskPanes.Add(muc, "MyControl");
    ctp.DockPosition = 
    ctp.Visible = true;

  // template code here, no changes

I did a quick sanity check by Building the Solution and then launching an Excel 2013 spreadsheet to verify I could see a ListBox named MyControl. I also did an Excel File -> Options -> Add-Ins -> Manage: COM Add-Ins, Go to verify the add-in was active and that I didn’t have any other old test add-ins.

I added the speech recognition functionality in the MyUserControl code, located in file MyUserControl.cs. First I added a reference to Microsoft.Speech.dll which was located at C:\ Program Files (x86)\ Microsoft SDKs\ Speech\ v11.0\ Assembly\ on my machine (spaces added to the path for readability. Then I added a using statement: “using Microsoft.Speech.Recognition;” at the top of the code.

I got the speech set up like so:

public partial class MyUserControl : UserControl
  static SpeechRecognitionEngine sre;

  public MyUserControl()

      System.Globalization.CultureInfo ci =
 new System.Globalization.CultureInfo("en-us");
      sre = new SpeechRecognitionEngine(ci);
      sre.SpeechRecognized +=
 new EventHandler(sre_SpeechRecognized);

      // TODO
. . .

I wanted a speech template to be like, “Select cells A1 E7” so I need to recognize letters and numbers. I prepared to recognize letters like this:

Choices ch_Letters = new Choices();
string[] letters = new string[26];
for (int i = 0; i < 26; ++i)
  char cLetter = Convert.ToChar(i + 65);
  string sLetter = cLetter.ToString();
  letters[i] = sLetter;

Next I prepared to recognize numbers like so:

Choices ch_Numbers = new Choices();
string[] numbers = new string[20];
for (int n = 1; n <= 20; ++n)
  numbers[n-1] = n.ToString();

I just recognize 1 through 20; I found that there’s a real impact on the recognition confidence as you increase the number of numbers. Next I set up the beginning of the phrase:

Choices ch_SelectCommand = new Choices();
ch_SelectCommand.Add("Select cells");

And then I constructed the rest of the phrase to recognize, loaded the resulting Grammar, and activated speech recognition:

GrammarBuilder gb_SelectRange =
  new GrammarBuilder();
Grammar g_SelectRange = 
  new Grammar(gb_SelectRange);

And then I wired up the speech-recognized event handler:

void sre_SpeechRecognized(object sender,
  SpeechRecognizedEventArgs e)
    string txt = e.Result.Text;
    double conf = e.Result.Confidence;
    if (conf = 0 &&
      txt.IndexOfAny(new char[] { 'A', 'B', 'C', 'D',
 'E', 'F', 'G', 'H' }) >= 0)
      // "Select cells A 1 B 2" 
      string[] toks = txt.Split(' ');
      string s1 = toks[2] + toks[3];
      string s2 = toks[4] + toks[5]; 
      Excel.Worksheet active =
      Excel.Range rangeToSelect = active.get_Range(s1, s2);
  catch (Exception ex)

I built the project and tested it by using speech to select cells A1 to E7 in a dummy spreadsheet as shown in the image above.

This entry was posted in Machine Learning. Bookmark the permalink.