The new concept of “Talking to Your TV”

To begin with, what is Voice Interaction?

Yokohagi: As its name suggests, it's a function that lets you operate the TV with your voice. The user simply says a command word or a search word into the special Touch Pad Controller or a smartphone.* It's a very simple and convenient operating style. * “TV Remote 2” must be installed for smartphone use.

- Content Search
- Text Input
- Moving Around the Web and Verbally Reading Out Text
- Conventional Operations Are Also Possible.
(Direct Operation of Volume Up/Down, Channel Change, Etc.)

What got you started in its development?

Yokohagi: It began with our Voice Guidance function for the Japanese market in 2010. The function would verbally tell the viewer things like the channel that was selected and the name of the TV program that was being viewed. It was developed to make using TV more convenient for visually impaired users.

Starting with the new 2013 models, we expanded this function to cover more than 20 languages. Around this time, we had meetings to consider our next-generation user's interface. The convenience of Voice Interaction was brought up at these meetings, and it was decided that, rather than simply offering Voice Guidance, we would propose an entirely new style of using TVs where ordinary users would also operate the TV by voice.

As we proceeded to make the concept more concrete, we were faced with questions like, “What can users do with Voice interaction?” and “What exactly should the user say?” Since we were drastically changing the concepts of both the TV and the remote control by using command words to do everything that was previously done by pressing remote control buttons, we found ourselves in a new and unknown world.

What are the features of the voice recognition engine?

Koganei: To put it simply, our engine has two main brains.
Our Voice function’s strong point is that those two main brains co-work nicely.

For example, there are fixed expressions for basic TV operation, like “Volume Up.” We call these “fixed words” and they are stored in one brain. And there are words that the user can freely use. They are convenient for cases like searching. We call them “free words” and those words are stored in another brain. The main feature of our engine is that it's able to start up both of these -- one part with a dictionary of fixed words, and the other part with a cloud service recognizing free words. In other words, it's a hybrid engine. This lets it recognize the words that are spoken by the user from a massive data bank.

Ordinarily, when the user wants to perform a TV operation like “Volume Up,” only the fixed word dictionary engine built into the TV is started up. Similarly, when the user wants to input a search word, only the free word recognition engine is started up. However, this means that a key word from a TV program being watched cannot be retrieved from the free word
recognition engine in a single speaking step. That isn't very convenient.

A) “Web Search” -› [Please say keyword] -› “Panasonic”
B) “Search for Panasonic by web.”

With our engine, the fixed word dictionary engine and free word recognition engine are both driven, so it recognizes either a TV operating command or a free word searching phrase. We took this kind of fine performance into consideration throughout our development to make the function as convenient as possible for the user.

Konuma: For example, when you want to look at “flower photos,” it's inconvenient to always have to say “search” first. We wanted to create a system that would more intuitively recognize what you're looking for by simply saying the word. As a result, all you have to do with our system is to say the word “flower” into the mic. After saying “flower,” the TV responds by asking additional conditions, such as, “Do you want photos or videos?”

Since you didn't have anyone who could understand the languages of 20 countries, how did you verify them?

Yokohagi: It's a little tricky, but we would display the control command for each language on a PC and have the PC read it out. This was input into a development-stage TV for verification. Finally, we asked local Panasonic staff in each country to verify the results, and received feedback from them.

Koganei: This involved a huge amount of data, and required an incredible amount of work, but it cannot be avoided if you're going to use voice recognition as a user's interface. So we repeated fine adjustments over and over again to raise the precision.

Wasn't it difficult to add the microphone in its originally minimal shape, and keep such a high level of quality?

Sekito: The evaluation point for the mic function was the voice recognition rate. This was evaluated using three performance parameters:

- Wireless performance (interruption-free communication)
- Mic performance (sound collecting function)
- Recognition engine performance (TV side capability to understand what the sound means.）

First, hardware design was concentrated on building solid performance for the wireless function and the mic, and software design aimed to fine-tune the recognition engine in the TV. The key to success was to raise the performance as much as possible given the limited space that we had to work with.

But raising performance wasn't the only aim. It was also important to strike a balance. For example, when you increase the sensitivity of the mic, it also picks up sounds that shouldn't be included, and this leads to operating errors. It's important to only pick up the necessary sounds, for example, inside a noisy store or when a group of people are making noise while watching an exciting movie. On the contrary, if the range of sound-collection precision is excessively narrowed, the user will have to speak very loudly. So we had to aim for an ideal level with a fine balance.

Imai: While refining a few hundred parameters through trial and error, we found some of the optimal parameters for the TV side. After repeatedly verifying a number of these, we applied the parameters that had the highest recognition rate.

We were confident about building the remote control, but it was our first attempt to integrate the mic in the remote control so it was an entirely new challenge. With no precedent to follow, it was difficult to decide the specification base for verification results. We also took a variety of approaches to make the shape easier to use.

In this way, we worked on making improvements in every area, including the shape, and tested a prototype internally about 3-4 months before the product release. This was also the first time that we had guests who had come for business discussions use this new function. We received mainly favorable comments, such as, “This is great!” and “Very convenient.” Encouraged by this, we were highly confident that it would be a success.

However, while the “ability to operate by voice” was highly evaluated, there were still some areas where precision had to be boosted. With all members working together, we devoted our utmost efforts to complete the design right up to the production timing.

How has the market response been since the launch?

Imai: The response has been excellent, with people saying that the precision of Panasonic's voice recognition is extremely good compared to the competitors. Particularly, customers in Japan have complimented us, saying things like, "Voice interaction is great. I want to try using for more operations."