Use Olami SDK Voice control for an iOS program that supports HomeKit smart home

Objective

HomeKit is an apple-released smart home platform. With the HomeKit component, users can control smart bulbs, fans, air conditioners and other smart homes that support HomeKit through iphone, ipad and ipod touch, especially with Siri for voice control.
But there's a big problem with Siri for voice control, which is that Siri-supported corpora can't be expanded freely, and there's no way to add more. The Olami SDK can be extended freely through OSL (Olami syntax Description Language Olami Syntax Language, abbreviated as: OSL), and the ability to expand intelligent conversations becomes very easy to pass. The Olami server makes semantic parsing of the input corpus and obtains the desired control results. In this way, we can expand the corpus and the statement, enrich the function of the operation.

Preparatory work

• HomeKit some basic concepts about smart devices and the installation of simulators

  As the number of devices supporting HomeKit is relatively expensive, Apple provides a HomeKit simulator to help programmers develop HomeKit devices. This demo is a simulation of an intelligent air conditioner that functions through the voice control of the air conditioning switch and adjusts the temperature.

  HomeKit Simulator is not installed by default, to download the installation
  Taget->capabilities->homekit can be downloaded in one item, see

Once installed, you can open and use it. See

There are several concepts in the HomeKit, here is a brief introduction of the app used, online has more detailed introduction, but also can refer to my reproduced this blog:
http://blog.csdn.net/dfman1978/article/details/72179458
HomeKit allows users to add multiple home, but only one home, called Primaryhome, is used. Each home can have multiple rooms. Smart devices called accessory belong to home, but each accessory can be equipped with a room. If no guest is specified, it is assigned to the default Roomforentirehome returned by default. Each accessory can have multiple service, each service can have more than one attribute (hmcharacteristic). Some features are read-only, such as the current temperature of the air conditioning, and some features are readable and writable, such as air conditioning switches. The app controls the smart device by controlling the value of these features.

A accessory is added to the program, and the name is air-conditioned. Then add two service to it: One is the switch, the other is the temperature of the temperature control air conditioning. Through the app, but to achieve the air conditioning switch and temperature adjustment.

• Olami and configuration of grammar rules for smart devices
  Users say, the app how to know what to mean, how to understand it? This involves the OSL syntax description language. To use the speech and semantic understanding APIs provided by the Olami platform, first write a set of grammars based on the rules of the Olami Syntax Description Language (Olami Syntax Language, abbreviation: OSL). OSL Introduction of the following Web site has a detailed introduction https://cn.olami.ai/wiki/?mp=osl&content=osl1.html
  With the NLI natural language semantic interaction system provided by the Olami platform, you can learn how to write a set of grammar rules for your own business application.

Natural Language Semantic interaction (Natural Language Interaction, abbreviated as: NLI) management system is a set of online semantic analysis management tools, NLI system using Olami Syntax Description Language (Olami Syntax Language, Abbreviation: OSL) replaces complex coding programming so that users without software development backgrounds can easily and quickly maintain intelligent conversation flows that include semantic extensions and answers.

More detailed information can be learned at the following URL
Https://cn.olami.ai/wiki/?mp=nli&content=nli1.html
Conveniently, the Olami platform has provided some well-written grammatical rules for many areas, which are called modules in Olami. Which about the smart device has been written, the following step-by-step configuration.
First to go to the Olami platform registration, register to enter the interface

Click "Create App" to go to this page

Fill in the application name, application description, and after the application is introduced, you can create it. Go back to the previous page and you'll see the app you created.

Click on "Enter NLI system" to enter the module page

In the official website has built up many fields of grammar. On the module page, click the "Import" button to view the existing domain module

Select a to use, for example I want to import "Smarthome" This module, first select it, click the "Import" button

Then go into the Smarthome module and you'll see the example.

However, this time is still not available and needs to be released first. Click the "Publish" button at the top of the page to go to the publishing page

Click "Publish" button to publish successfully

Finally, go back to the "My Apps" screen and click on the "Configure Nli Module" button to associate your own created apps and modules.

The smarthome syntax file is configured so that it can be used.

Implementation of the Code

1. Get the home and accessory objects

- (void) Viewwillappear: (BOOL) Animated {[SuperViewwillappear:animated]; Self. Accessories= [NsmutablearrayArray];if( Self. Homemanager&& Self. Homemanager. Primaryhome) { for(Hmaccessory *accessory in Self. Homemanager. Primaryhome. Accessories) {            [ Self. AccessoriesInsertobject:accessory Atindex:0]; Accessory. Delegate= Self; [ Self. TableViewReloaddata]; }    }if(_currenthome) {_currenthomelabel. Text= [NSStringstringwithformat:@"Current home:%@", _currenthome. Name]; }}

In the viewwillappear time to get the current home object and name, as well as the accessory object, and save it, with a tableview to show the program just started when the interface, this time because there is no add home, so nothing

Click "Add Home" button to pop up a dialog box, you can fill in the name of a home. The corresponding code

- (IBAction)addHomeBtnClicked:(id)sender

The next step is to add accessory, which is not done in the code. is to add smart air conditioners via the "home" app provided by iOS.
Click on the "Home" App to pop up the page

Click "Get Started" and pop up

Click "Add Accessory" and the app will automatically search for the air conditioner that you just added in the simulator, follow the prompts step by step, and finally show the air conditioner's two service:switch and temperature

Back in the app, this time the app will show the search accessory: Air Conditioning

Click "Air Conditioning" to enter the Voice control page

On this page, the TableView above shows some of the properties of the air conditioning.
The following textview are used to display some of the results of ASR
Circular button for recording

2.. go to URL https://cn.olami.ai/wiki/?mp=sdk&content=sdk_and_sample.html download Olami SDK. Includes two files, one of which is the static function library of Olami, and one is its header file
The first step is to initialize the Olami speech recognition object and set the proxy

olamiRecognizer= [[OlamiRecognizer alloc] init];olamiRecognizer.delegateself;

3. Call the Setauthorization function to authorize

setAuthorization:@"d13bbcbef2a4460dbf19ced850eb5d83"    api:@"asr" appSecret:@"3b08b349c0924a79869153bea334dd86" cusid:OLACUSID];

The description of the parameters of this function is described in Olamirecognizer, and you can go to the online API description to see
Https://cn.olami.ai/wiki/?mp=sdk&content=sdk/ios/reference.html

The parameters are obtained when the app was created just now.

4. Set up language

[olamiRecognizer setLocalization:LANGUAGE_SIMPLIFIED_CHINESE];

You must set it up before recording, otherwise you will not get the result. Currently only supports Simplified Chinese (language_simplified_chinese)

4. Start recording
Call the start () interface to start recording

start];

5. Get the text and semantics of the recording and deal with it
By calling the Stop () function or stopping automatically, you will get the result of the recorded text and the semantic analysis of it.
Implement the Olamirecognizerdelegate Onresult function to get the result, the result is a JSON string callback, the string to parse, you can get the desired number. For example, the microphone said "turn on the air conditioning", the results are as follows

{    "Data":{        "ASR":{"result": "turn on air conditioning", "speech_status": 0, "Fina L": true,"status": 0        },        "Nli":[            {                "Desc_obj":{"status": 0 },                "Semantic":[                    {                        "app":"Smarthome",                        "input":"Turn on the air conditioner",                        "Slots":[{"name": "Device", "value":  "air conditioner" }],                        "modifier":[ "open" ],                        "Customer":"58df685e84ae11f0bb7b4893" }                ],                "type":"Smarthome"            }        ]},    "Status":"OK"}

This is a set of rules that are defined by the OSL Syntax description language, and the results are returned. A description of this result is shown on the Https://cn.olami.ai/wiki/?mp=api_nlu&content=api_nlu3.html website.
6. Get all Accessory objects

forin _accessory.services) {        setforKey:service.name];        NSLog(@"service.name is %@",service.name);    }

Saved in a dictionary, the name of the service is key, and the pointer to the object is value

Description of the 7.onResult function
In the whole program, one of the main functions is the Onresult function, which is to deal with the results passed. In this function, three functions are called to handle the three more important nodes in the JOSN format, respectively.

- (void)processASR:(NSDictionary*)asrDic 

This is used to process the ASR node to obtain the result of speech recognition, and if there is no result, a dialog box is displayed to prompt

- (void)processSemantic:(NSDictionary*)semanticDic

This is used to process the semantic node, which contains the value of the slot and the value of the modifier. The slots in the OSL syntax description Language can be understood as variables in semantics that are used to pass and extract information and are the source of data processed by code. For this procedure, is to control the air conditioning of various actions, such as open, off. There is also the adjusted temperature value. The value of the slot can refer to https://cn.olami.ai/wiki/?mp=osl&content=osl_slot.html, which is explained in detail

- (void)processModify:(NSString*) str 

This is used to deal with speech and semantic results. This function mainly deals with the modifier nodes in the JSON string. The modifier syntax Description rule is a OSL syntax description language, in addition to the slot of a built-in information transfer mechanism, generally used to denote semantic purposes, but also can be understood as a way of commenting on semantics, so that the application's developers know the corresponding intention grammar represents. Detailed description Reference
Https://cn.olami.ai/wiki/?mp=osl&content=osl_regex.html#11, through modifier, can we know what the program is intended for? For example, to turn the air conditioner on and off. or adjust the temperature.

In the code we processed three modifier: "Open", "close", and "control_temperature". Then further processing is performed based on the value of the slot.
Take the action that handles the "off" air conditioning to make a note

if([Str isequaltostring:@"Close"]){//Turn off air conditioningHmservice *tmpservice = _servicedic[@"Switch"]; Hmcharacteristic *characteristic = Tmpservice. Characteristics[1];if([Characteristic. CharacteristictypeIsequaltostring:hmcharacteristictypetargetlockmechanismstate] | | [Characteristic. CharacteristictypeIsequaltostring:hmcharacteristictypepowerstate] | | [Characteristic. CharacteristictypeIsequaltostring:hmcharacteristictypeobstructiondetected]) {[characteristic writevalue: @NO completionHandler:^ (Nserror*error) {if(Error = =Nil) {Dispatch_async(Dispatch_get_main_queue (), ^ {_asrtextview. Text= @"Air conditioning is off";                }); }Else{NSLog(@"Error in writing characterstic:%@", error); _asrtextview. Text= @"The air conditioning failed to shut down, please try again!";        }            }]; }    }

If modifier equals "close", then this is the property of a switch service. Get the service pointer through "switch" and get the properties of the switch. And then through

- (void)writeValue:(nullable id)value completionHandler:(void (^)(NSError * __nullable error))completion;

function to modify the value of this property. Because the air conditioning is off, so write directly to @no.
Because this function is returned asynchronously, the return is not necessarily a successful operation, so the return result should be processed.

Code download

Code can be downloaded to GitHub
Https://github.com/lym-ay/SmartHome

Use Olami SDK Voice control for an iOS program that supports HomeKit smart home