Android displaymanager service resolution 1

Address: http://blog.csdn.net/goohong/article/details/8536102

Android 4.2 and later versions provide a display management service displaymanagerservice, which supports image display of Multiple display types, including built-in display types (local) HDMI display type and support for WiFi display protocol (miracast) to display images of local devices on a remote display.

The architecture class diagram is as follows:

The display Management Service provides external interfaces through displaymanager. The interfaces provided include:

1. Public display getdisplay (INT displayid)

Obtains information about a logical Display Based on the displayid parameter.

2. Public display [] getdisplays ()

Obtain a list of all currently valid logical displays

3. Public void registerdisplaylistener (displaylistener listener, Handler handler)

Registers a display listener object to monitor the addition, removal, or change notification events of the monitor.

4. Public void unregisterdisplaylistener (displaylistener listener)

Cancels a display listener object previously registered

5. scanwifidisplays ()

Start the scan of the WiFi monitor.

6. Public void connectwifidisplay (string deviceaddress)

Connect to the WiFi Monitor Based on the device address

7. Public void disconnectwifidisplay ()

Disconnect Wi-Fi display

8. Public void renamewifidisplay (string deviceaddress, string alias)

Name a Wi-Fi display

9. Public void forgetwifidisplay (string deviceaddress)

Cancel Wi-Fi display with previous memories

10. Public wifidisplaystatus getwifidisplaystatus ()

Get the current WiFi display status

The management system also interacts with other systems, the discovery of wifi display, registration of wifi display in the window system, display of window content on WiFi display (acquisition, encryption, and encoding of source image data, decoding and playback of image data received by the sink.

Through the wifip2pmanager interface and wifip2pservice interaction, through the WIFI-DIRECT to realize the WiFi display automatic discovery.

The window management service displays the listener object of the management service. The window management service registers with displaymanagerservice through the displaymanager interface. When the WiFi display is found and connected successfully, and the WiFi display is disconnected and its status changes, events are sent to the window management service through callback. The corresponding callback functions ondisplayadded, ondisplaychanged, and ondisplayremoved Of The Window Management Service are called, it is used to register a Wi-Fi display in the window system, cancel the registration, and change the status.

In addition, displaymanagerservice also uses the window management interface of windowmanagerfuncs to directly call the Window Management Service (windowmanagerservice is the implementation of this interface) function to refresh the window content. Similarly, displaymanagerservice also uses the inputmanagerfuncs interface to directly call the setdisplayviewports function of the input management service to set the display view information of the display required by the input system.

The display management system also interacts with the mediaplayerservice through the imediaplayerservice interface.

For example, call the listenforremotedisplay function of the mediaplayerservice service to instantiate the remotedisplay local proxy object of a remote display in the media service. The display system calls the media service through the iremotedisplay interface, currently, the iremotedisplay interface only has one dispose interface function, which is used to disconnect the remote display and stop listening for new connections.

The wifidisplaysource object of the display management system also calls the makehdcp function of the mediaplayerservice service to instantiate an HDCP object and returns an ihdcp interface to the display system to implement the HDCP encryption service.

The system sink tunnelrenderer object also calls the create function of the mediaplayerservice in its initplayer to create a mediaplayer object, and returns an imediaplayer interface for the display system, this function is used to play back the image data received by the sink.

After the image data obtained from the management system source is audio/video encoded (h264), HDCP encryption, PES packetization, and TS Streaming (converted to TS Stream) are performed, finally, the RTP packet is packaged and sent to the sink through the UDP channel. The sink must go through the opposite processing process and receive the RTP packet from the UDP channel, then, the TS parsing and PES are de-packetization and HDCP decryption, and finally sent to the decoder for decoding. The decoded data is sent to the player's Renderer for rendering.

Both the source and sink audio and video codec interfaces interact with the underlying multimedia framework through the iomx interface to implement the audio and video codec function. The object OMX corresponding to the iomx interface is also instantiated on the mediaplayerservice server. In the client object omxclient's connect function, the iomx interface corresponding to OMX is returned by calling the getomx function of the mediaplayerservice. The OMX object is an encapsulation of the Multimedia Framework openomx.

The source audio/video encoding, TS Streaming, HDCP encryption, and RTP packaging and sending processes all involve management and scheduling of playbacksession threads. A surfacemediasource object is instantiated during the playbacksession class initialization, the surfacemediasource object instantiates a bufferqueue object (bufferqueue is derived from isurfacetexure ). After establishing a connection with the sink, pass the bufferqueue object to the Java layer through the ondisplayconnected callback function of the iremotedisplayclient interface, after receiving the ondisplayconnected event, the Java-layer wifidisplayadapter object calls the createdisplay function of the surface class to register a virtual display in the surfaceflinger service, call the setdisplaysurface function of the surface class to pass bufferqueue to the displaydevicestate variable corresponding to the surfaceflinger service virtual display .. Therefore, playbacksession can use a bufferqueue object to read data from the surfaceflinger service.

The data received by the wifidisplaysink object on the sink end is decoded by RTP (the rtpsink object is responsible for) and then presented to the tunnelrenderer object. The tunnelrenderer object instantiates a playerclient playing client in the initplayer function, the imediaplayerservice interface is used to call the create function of the mediaplayerservice service to create a mediaplayer object and return the tunnelrenderer object imediaplayer interface,
The tunnelrenderer object uses the imediaplayer interface to play and present the received image data. The tunnelrenderer object is also instantiated through the surfacecomposerclient object in the initplayer function to obtain a surface object, call its getsurfacetexture function to obtain the isurfacetexture corresponding to the surface object, and call the setvideosurfacetexture function of the imediaplayer interface to assign the isurfacetexture value to the player, so that the display data decoded by the player is sent to.

The wifidisplaysink object at the sink end and the source wifidisplaysource object are responsible for handling the WiFi display interaction protocol. Both objects contain an anetworksession object, which is responsible for the network interaction session process between the two objects.

The following figure shows the framework of the WiFi display protocol.

The android Open Source Code provides the specific implementation of the WiFi display protocol, but the sink is only executed as a local test command, and does not provide the user input function shown in the figure, the WFD source end provides complete implementation code, which is mainly because the Android system is mainly used as the mobile tablet operating system.

The implementation of wifi display in android4.2 includes Java and C ++. Java functions mainly correspond to displaymanagerservice. displaymanagerservice instantiates and registers a wifidisplayadapter object (derived from displayadapter) for WiFi display, which is used to establish a connection with the corresponding display device, each displayadapter corresponds to a display device displaydevice class. Each display device is in the displaymanagerservice
Corresponds to a logicaldisplay object.

All the requests initiated by displaymanager to wifi display are forwarded by displaymanagerservice to the wifidisplayadapter object for processing. To achieve Asynchronous interaction and avoid deadlocks, the wifidisplayadapter object starts a thread for each request to actually process the request, and instantiates a wifidisplaycontroller object to encapsulate the call of these requests. The wifidisplayadapter object also instantiates a wifidisplaycontroller. Listener object to listen on the Request status.
The wifidisplaycontroller object instantiates a wifip2pmanager object to initiate a WI-FI direct request to the wifip2pservice service through its interface. When you connect to the WiFi display, instantiate a remotedisplay object to start the connection event listening to the underlying WiFi display.

The C ++ part is located at the media framework layer and under the WiFi-display directory in the libstagefright directory, the WiFi-display directory contains two sub-directories, sink and source, which correspond to the sink and source programs of wifi Display respectively. the WiFi-display directory also contains the common programs of source and sink.

It mainly includes the following files:

The sink directory mainly contains wifidisplaysink. CPP, rtpsink. CPP, tunnelrenderer. c ++ files such as CPP and the corresponding header files are responsible for the Protocol protocol of the sink, RTP receiving, and image data presentation processes. The ts parsing and PES de-packetization and decoding processes are completed by the player nuplayer. As the sink is only a test routine, the HDCP decoding process is not provided yet.

The source directory mainly includes wifidisplaysource. CPP, playbacksession. CPP, mediapuller. CPP, converter. CPP, tspacketizer. CPP, sender. c ++ files such as CPP and corresponding header files are responsible for protocol interaction at the source end, session process management, image media reading, encoding, TS packaging, and RTP packaging and sending. The converter object contains a mediacodec object that is responsible for the encoding process. The mediacodec object actually uses the acodec object to call the iomx interface and uses the openomx media framework to complete the image data encoding.

The common files used by sink and source are anetworksession. cpp, corresponding header files, and WFD. CPP, the main program that starts wifidisplaysink, which is used to generate the WFD command.

The main process of the entire Protocol includes the connection process of the WiFi display device and the process of packaging, sending, and receiving the image data. The specific process is described in the next blog.