[IOS and EV3 hybrid robot programming series] What does iOS do? EV3 can be cooler !, Iosev3

[IOS and EV3 hybrid robot programming series] What does iOS do? EV3 can be cooler !, Iosev3
Lego Mindstorm EV3 intelligent robot (EV3)
Since its presentation at CES (Consumer Electronics Show USA Consumer Electronics Show) in 2013,
It attracted the attention of robots around the world!
The biggest improvement of EV3 over NXT is its hardware.
In addition to more powerful ARM processors and Linux operating systems,
The EV3 is also equipped with Bluetooth and supports external USB and external WiFi.
With so many hardware improvements, EV3's coolest feature is
EV3 supports iOS devices !!!
This allows us to use iOS devices such as iPhone to control EV3!
Lego officially launched the control App Commander on iOS, as shown in:

We can use Commander to control various robot forms built by EV3 family edition through Bluetooth!
With the iPhone, you can control a robot. It is exciting for every robot enthusiast!
However, the official App Commander has its own limitations! That is, it can only perform extremely simple control on the EV3 robot. The most important thing is to control the rotation of the motor on the robot. We cannot add any control logic.
It can be said that Commander only turns the iOS device into an EV3 remote control!
As a robot enthusiast, how can this satisfy us?
IPhone and iPad, as the world's most advanced mobile phones and tablets, can only be used as one remote control?
The answer must be No!
We need to combine iOS and EV3 perfectly and then do things far beyond what EV3 robots can do!

As mentioned above, the most attractive part of EV3 is that it can connect to iOS devices and use iOS devices to control the EV3 robot!
However, the limitations of the official App allow us to use iOS devices as a robot remote control, which is not what we want to see.
We hope iOS and EV3 can make robots cool!
Then the problem arises: how should we combine iOS and EV3?

= Comparison between iOS devices and EV3 =

From the comparison above, we can see that apart from the EV3 support many sensors not available for the iPhone, EV3 is not an order of magnitude as iPhone 5S in other aspects! At the same time, the iPhone does not have the sensor EV3! Obviously, the EV3 robot and iPhone are generally not compared. After all, the two are completely different products of the same concept! However, from the technical point of view, the hardware configuration of EV3 is really very low, completely behind the times! Although many robot enthusiasts see that EV3 is just a single-chip microcomputer, it has improved a lot!
After reading the comparison above, do you have an idea:
If the EV3 robot absorbs all the hardware of the iPhone, how powerful the robot will be!
Yes, that's right! Therefore, for robotics entrepreneurs, there are two good idea worthy of reference:
1. Design a set of robot kits with iPhone performance and all the sensors on the iPhone in addition to the basic EV3 sensor!
2. Use the iPhone to design an EV3 extended hardware so that the EV3 sensor can be directly connected to the iPhone through this hardware, so that the EV3 control block is not needed and the iPhone is directly used for control!
But we haven't seen such a product yet. What can we do?
Connect iOS and EV3 through wireless communication! And use iOS to control EV3!
= Combination of iOS and EV3 =
We will analyze the combination of iOS and EV3 one by one:
1) programming and Control Methods
On EV3, we can only use graphical languages for programming, which is of course a good choice for the majority of friends who have never been familiar with programming. However, when we are familiar with the graphic language and have a certain understanding of programming, we will find the disadvantages of the graphic language, that is, even if we write a slightly more complex program, writing in a graphical language looks messy. You just need to take a look at the EV3 family edition Gyro Boy graphical program will have a deep understanding.
Programming in graphical languages limits the programming of the EV3 Robot Control Program. Therefore, if we want to use the EV3 robot to do something cooler, we must consider programming in a general text language.
At WWDC2014, Apple released the new programming language Swift. the concise and Modern Language Style makes programming for iOS and Mac easier. The Objective C language can still be used as the iOS development language for a long time. The Swift language is completely seamlessly connected to the Objective C language.
So can we consider moving the graphical program originally placed in the EV3 control block to the iOS device and then using iOS to control the EV3 robot?
1) under normal circumstances, we use the EV3 software on the computer to write graphical language programs, and then write the program into the EV3 control block to run. The EV3 control block controls the sensors connected to each port of EV3 according to the program output to run the entire program!
2) After joining the iOS device, we use Swift or Objective C language on the computer to write a program to control the EV3 robot. After writing the program, we run it on the iOS device! After running, the iOS device first connects to the EV3 control block through WiFi or Bluetooth, and then the iOS device outputs control commands to the EV3 control block according to the program, the EV3 control block controls the sensors connected to each port of EV3 according to the control commands of the received iOS device, in addition, the EV3 control block transmits sensor data to iOS devices in real time so that programs on iOS devices can obtain sensor data for effective program running.
In general, we should use iOS devices to replace the EV3 control block, so that the EV3 control block is only used for data exchange and Output Control Transfer stations! Here we will not talk about the specific implementation methods, which will be introduced in the future chapter.
2) Hardware usage
From the above comparison, we can also see that there are good hardware on iOS devices. We should consider applying hardware on iOS devices to the EV3 robot. Here we actually have two ways to use iOS devices:
(1) The iOS device is in our hands. The iOS device is both the brain of the EV3 robot and the Controller of the EV3 robot.
(2) iOS devices are directly bound to the EV3 robot. hardware on iOS devices works together with sensors on EV3. We can consider adding an iOS device to our hands as a controller.
The sensor on EV3 is not used here. Let's take a look at how the hardware on iOS devices can be used?
= Hardware usage of iOS devices =
The hardware that can be combined with EV3 on iOS devices can be divided into the following categories:
1) camera
2) motion sensor (including Three-Axis Gyroscope sensor and three-axis acceleration sensor)
3) GPS and compass Sensors
4) Speech Recognition
5) multi-touch Retina screen
6) WiFi and Bluetooth Communication Module
For the Wi-Fi and Bluetooth communication modules, the function is to connect EV3 for data communication. Other types of hardware are useful in the EV3 robot.
1) camera
The EV3 robot has an eye in an instant!
The most basic method is to bind the iPhone to the EV3 robot, and then let the camera on the iPhone take video in real time and transmit the video to another handheld iOS device, in this way, we can watch the images seen by the EV3 robot remotely! Is it cool?
The advanced method is image recognition. With the camera, the EV3 robot can implement image recognition through programming and use the recognition results for special behavior. For example, automatic walking turns an EV3 robot into a self-driving car, automatically avoids obstacles, and automatically identifies route walking without human intervention! If this is done, the EV3 robot is too powerful, isn't it?
2) motion sensor
With motion sensors, we can use the following two methods:
(1) On the EV3 robot. The EV3 Education Edition is equipped with a gyroscope sensor, but it only has a single axis, that is, it can only detect the rotation in one direction. The gyroscope and accelerometer on iOS devices are all three axes, which means that the motion of space X, Y, and Z can be detected. Binding the iPhone to EV3 and using motion sensors on the iPhone will greatly enhance the performance of the EV3 robot. For example, if a motion sensor is added to the Gyro Boy model of EV3 family edition, we can not only detect the front and back pitch of Gyro Boy, but also detect whether Gyro Boy is uphill or downhill while moving, whether to turn left or right. We can easily know the specific rotation angle. Based on the motion sensor data and servo motor data, we can even calculate the Motion Track of the entire EV3 robot and then directly display it on the screen of our handheld device. This increases the level of the EV3 robot.
2) as a handheld controller. Using motion sensors, we can turn the iPhone in our hands into a somatosensory controller. Based on the motion sensor data, we can know the specific attitude of the iPhone and output the specific control amount based on the attitude. Here is an example. For example, if we set up an EV3 manipulator, how can we control it before that? The official EV3 example only allows the manipulator to be moved by itself, but this is not what we want to see! With the iPhone, we have a cool way to play: we use the iPhone to control the EV3 arm. The iPhone is in our hands, the EV3 arm is in our hands, and the iPhone is in our hands, the EV3 manipulator is turning down... This will be one of our final challenges and will detail the implementation methods in the subsequent chapters.
3) GPS and compass Sensors
With GPS and compass sensors, we will be more comfortable with EV3 robot control. Because we have clearly understood where the EV3 robot is. Combined with cameras, motion sensors, and even networks, we can achieve super-Remote Control of the EV3 robot. We don't need to see that the robot itself can control it through network transmission, this makes the EV3 robot a real robot! How can we create EV3 as a robot Walle?
4) Speech Recognition
Do you want to use the iPhone to control the EV3 robot through speech recognition? We can do this.
(Add Siri Introduction)
The iPhone's Siri is slowly opening up. In the future iOS version, we may be able to directly use Siri to control the EV3 robot.
So, even if Siri is not available for the moment, we can still use the voice recognition technology of Tencent! By embedding the Tencent voice recognition SDK In the iOS app, we can recognize the speech in the app and then make judgments based on the speech recognition content to output control!
5) multi-touch screen
If we don't want to use EV3 to control the ugly black/white dot matrix screen on the block, we can simply replace it with the iPhone's Retina screen. Viewing the information of each sensor connected to EV3 on the iPhone is obviously more important. We can also test the working status of each sensor on the iPhone, or even record the data.
As you can see, with the iOS device, it is simply adding several powerful wings to the EV3 robot, which can make the EV3 robot stand taller and create an unimaginable EV3 robot.
Many friends may have such doubts: Can we make such a powerful EV3 robot?

[This article is an original article. If you need to reprint it, please indicate the source. Thank you !]