Create an audio/video application based on Android devices and FMS

Through the next study, we will understand the following three key knowledge points.

In the original Flash program, the client creates a connection with the Flash Media Server through the RTMP protocol, and manages the list of online users on the Flash Media Server through mongodobject, communication between clients uses the Netconnection Call method to send video invitation-related information.

Requirements

You must first install the following products:

Flash Professional CS5

• Trial
• Purchase

Flash Media Server 4

• Trial
• Purchase

AIR 2.5 for Android development plug-in on Flash Professional CS5

• Download

Example file:

• Original Flash Audio/Video Program
• Final code

Prerequisites

• ActionScript 3.0 intermediate programming level
• Will use Flash Pro and Flash Media Server Development Projects

The client creates a RTMFP connection with the flash media server.

To create a P2P application, we first need to connect to the server to create the RTMFPReal Time Media Flow Protocol. We can directly change the RTMP protocol in the original code to RTMFP, but the RTMFP protocol is based on UDP, not all network environments support UDP. Therefore, for the availability of our applications, we still retain the RTMP connection method. In the Communicator class responsible for communication, modify the connect function:

 
 

  
  
Public function connect (rtmp: Boolean = false): void
  
  
{
  
  
// Add a Boolean parameter to the function and use the parameter to switch the connection mode to the UDF server.
  
  
// By default, we use the rtmfp connection.
  
  
Var url = (rtmp? "Rtmp:": "rtmfp:") + "//" + serverURL + "/vchat"
  
  
Nc. connect (url, userName );
  
  

  
  
// Judge whether a connection time counter is added when the rtmfp protocol is used for connection.
  
  
// Use the connection timeout value set by this calculator to determine whether the rtmfp connection is successful
  
  
If (! Rtmp)
  
  
{
  
  
RtmfpTimer = new Timer (connectTimeout, 1 );
  
  
RtmfpTimer. addEventListener (TimerEvent. TIMER_COMPLETE, rtmfpTimeoutHandler)
  
  
RtmfpTimer. start ();
  
  
}
  
  
}
 
 

Add the event listening function. When the RTMFP connection fails, close the original connection and create an RTMP connection.

 
 

  
  
private function rtmfpTimeoutHandler(e:TimerEvent):void 
  
  
{ 
  
  
    isRTMFP=false; 
  
  
    rtmfpTimer.stop(); 
  
  
    rtmfpTimer = null; 
  
  
    nc.close(); 
  
  
    connect(true); 
  
  
} 
 
 

Add the code to the ncStatusHandler function of the original Netconnection connection status listener:

 
 

  
  
if (rtmfpTimer) 
  
  
{ 
  
  
    rtmfpTimer.stop(); 
  
  
    rtmfpTimer = null; 
  
  
} 
 
 

In this way, when the Netconnection connection responds, close the counter.

At this point, we have changed the client connection to a P2P-compatible RTMFP connection method. Now let's take a look at the Flash Media Server code.

Manage and allocate P2P connections in flash media server 4

When the RTMFP connection is successfully created between the client and Flash Media Server 4, the Server creates a unique 256-bit ID for each client connection. Through this ID, the client can perform P2P connections.

To create a P2P Audio/Video Stream, we need to use Flash Media Server 4 to send the unique ID corresponding to each client to the communication peer.

When the client successfully connects to the FMS, the unique ID of the connection is saved in the attribute value named farID. Open the Flash Media Server script main. asc in the examples of fms> vchat. In the onConnect function, the unique ID is saved in the parameter named peerID.

 
 

  
  
application.onConnect = function(client, userName) 
  
  
{ 
  
  
    …… 
  
  
    clientclient.peerID=client.farID; 
  
  
    …… 
  
  
} 
 
 

Add the following code to the videoChat function of the video invitation:

 
 

  
  
case "accept": 
  
  
    var myPeerID="" 
  
  
    var otherPeerID="" 
  
  
    if(userList[myname].protocol=="rtmfp" && userList[othername].protocol=="rtmfp") 
  
  
    { 
  
  
        myPeerID=userList[myname].peerID 
  
  
        otherPeerID=userList[othername].peerID 
  
  
    } 
  
  
    userList[othername].call("vc", null, stat, myname,myPeerID); 
  
  
    userList[myname].call("vc", null, stat,othername,otherPeerID); 
 
 

The code above indicates that when one party receives the video chat invitation from the other party, we determine whether the connection protocol between the two parties and the FMS is RTMFP. If yes, the peer IDs are transmitted to the other party.

After modifying the Server script, do not forget to copy the vchat directory to the applications subdirectory of Flash Media Server.

Create P2P-compatible audio/video interaction on the client

Now let's go back to the client code Communicator. as for editing. In the initialization and Publishing Audio and Video Stream Function initOutStream, add the transfer parameter peerID: String to receive the peer peerID sent by the FMS server.

 
 

  
  
initOutStream(user:String,peerID:String) 
 
 

Then add the Code:

 
 

  
  
if(isRTMFP && peerID!="") 
  
  
{ 
  
  
    outStreamDirect = new NetStream(nc, NetStream.DIRECT_CONNECTIONS); 
  
  
    var c:Object = new Object(); 
  
  
    c.onPeerConnect = function(peer:NetStream):Boolean 
  
  
    { 
  
  
        if (outStream) 
  
  
        { 
  
  
            outStream.close(); 
  
  
            outStream = null; 
  
  
        } 
  
  
    outStreamDirect.attachAudio(Microphone.getMicrophone()); 
  
  
    outStreamDirect.attachCamera(cam); 
  
  
    return true; 
  
  
} 
  
  
outStreamDirect.client = c; 
  
  
outStreamDirect.publish(mySteamName+"_direct"); 
  
  
} 
 
 

In the above Code, we first determine the connection between ourselves and the FMS server and whether the other party is using the RTMFP protocol. If so, we create a publish stream outStreamDirect through P2P direct connection, different from the original release stream, we have created a new name "_ direct" for this new release stream ".

Please note that we keep the original release stream based on FMS at the same time because, even if both audio and video chat parties have created the RTMFP protocol with the FMS server, due to different network environments, such as firewalls, vro restrictions may prevent both parties from creating point-to-point connections. Therefore, we use onPeerConnect to determine whether the P2P connection is successful. When the P2P connection is successful, disable the original RTMP-based publish stream outStream.

With the modification of the above Code, our application can truly adapt to users in various network environments.

In the same way, we use the initInStream function to play audio and video streams simultaneously.

 
 

  
  
if(isRTMFP && peerID!="") 
  
  
{ 
  
  
    inStreamDirect=new NetStream(nc,peerID) 
  
  
    inStreamDirect.addEventListener(NetStatusEvent.NET_STATUS, inStream_NetStatusHandler); 
  
  
    inStreamDirect.play(user+"_direct"); 
  
  
} 
 
 

Note that the newly created stream inStreamDirect in the Code specifies the peer peerID of the stream to be played. This unique ID comes from the same as the UDF server program we just modified.

Because we play two streams at the same time, we add the following event listening function inStream_NetStatusHandler to the newly created P2P direct connection stream, so that when the P2P stream is successfully played, disable the original RTMP protocol stream and switch the currently displayed video to this stream.

 
 

  
  
private function inStream_NetStatusHandler(event:NetStatusEvent) : void 
  
  
{ 
  
  
    if (event.info.code == "NetStream.Play.Start") 
  
  
    { 
  
  
        if(inStream) 
  
  
        { 
  
  
            inStream.close(); 
  
  
            inStream = null; 
  
  
        } 
  
  
        VideoChat.instance._chatScreen.showUserVideo(inStreamDirect) 
  
  
    } 
  
  
} 
 
 

Finally, in order to ensure that the Android Application we created is disconnected from the FMS server when the user clicks the return key or the program is closed during the actual operation of the mobile device, in the main program class VideoChat. add code in

 
 

  
  
if(Capabilities.cpuArchitecture=="ARM") 
  
  
{ 
  
  
    NativeApplication.nativeApplication.addEventListener(Event.ACTIVATE, 
  
  
    handleActivate, false, 0, true); 
  
  
    NativeApplication.nativeApplication.addEventListener(Event.DEACTIVATE, 
  
  
    handleDeactivate, false, 0, true); 
  
  
    NativeApplication.nativeApplication.addEventListener(KeyboardEvent.KEY_DOWN, 
  
  
    handleKeys, false, 0, true); 
  
  
} 
  
  
private function handleActivate(event:Event):void 
  
  
{ 
  
  
    NativeApplication.nativeApplication.systemIdleMode = SystemIdleMode.KEEP_AWAKE; 
  
  
} 
  
  
private function handleDeactivate(event:Event):void 
  
  
{ 
  
  
    communicator.onExit() 
  
  
    NativeApplication.nativeApplication.exit(); 
  
  
} 
  
  
private function handleKeys(event:KeyboardEvent):void 
  
  
{ 
  
  
    if(event.keyCode == Keyboard.BACK) 
  
  
    { 
  
  
        communicator.onExit() 
  
  
        NativeApplication.nativeApplication.exit(); 
  
  
    } 
  
  
} 
 
 

Add the onExit function to the Communicator class.

 
 

  
  
public function onExit():void 
  
  
{ 
  
  
    nc.close() 
  
  
    nc=null; 
  
  
} 
 
 

In this way, when you click the return key or exit the program, the connection to the FMS is automatically closed.

So far, we have modified the Code related to supporting P2P Audio and video streams by using the client and FMS4. Next we will release this program as an application based on Android mobile devices.

Use the latest flash pro AIR2.5 plug-in to publish ANDROID mobile applications

Since the original code is based on Flash Professional, we can download the latest AIR 2.5 for Flash Pro CS5 plug-in from the Adobe lab to easily publish an Android-based mobile device application ,: http://labs.adobe.com/technologies/flashpro_extensionforair/

After installing this plug-in, open the Flash release settings, select AIR Android, and click open settings.

In General settings) panel, we can set whether the application is full screen, the release name, and so on.

In the Deployment panel, create your application certificate. In the post-release options, if your computer is connected to Android mobile devices, you can select release to automatically install the application on the device and run it automatically.

In the Permissions panel, set the access permission for this application. Because the application we are using requires to connect to the network and use the Camera, we need to select the Internet and Camera options.

Release our final modified program. Now you can use Android mobile devices for audio and video communication. :

For the effect of running the program on the Samsung Galaxy Tab.

About the author

Peter huang)

Adobe Platform Evangelist, one of the earliest Flash developers in China, Rich Interactive EntertainmentRIE®) Founder. He has been engaged in the development of Internet interactive applications and Flash games for nearly 10 years and has rich experience in Flash game, video and mobile application development. Currently, it is an Adobe platform technology missionary who is committed to the application and promotion of Flash technology in games and videos.