17th Article--android Displaymanager Service analysis One

Android starts with version 4.2 to provide a display management Service Displaymanagerservice, which supports multiple display types for mirroring display, including built-in display type (local), HDMI display type, and support for the WiFi displays protocol ( MIRACAST) to enable the local device to be mirrored on the remote display.

The entire schema class diagram is as follows:

The Display Management Service provides an external interface via Displaymanager, which includes the following interfaces:

1. Public Display getdisplay (int Displayid)

Get information for a logical display based on the Displayid parameter

2. Public display[] Getdisplays ()

Get a list of all currently active logical displays

3. public void Registerdisplaylistener (Displaylistener listener, Handler Handler)

Registers a display listener to monitor new, removed, or changed notification events.

4. public void Unregisterdisplaylistener (Displaylistener listener)

Cancels a previously registered display listener object

5, Scanwifidisplays ()

Initiates a scan of the WiFi monitor.

6. public void Connectwifidisplay (String deviceaddress)

Connect WiFi monitor to device address

7. public void Disconnectwifidisplay ()

Disconnecting the WiFi monitor

8. public void Renamewifidisplay (string deviceaddress, string alias)

Name the WiFi monitor

9. public void Forgetwifidisplay (String deviceaddress)

Cancel a previously remembered WiFi display

10. Public Wifidisplaystatus Getwifidisplaystatus ()

Get the status of your current WiFi monitor

The display management system also interacts with other systems to realize the discovery of the WiFi display, the registration of the WiFi Monitor in the window system, the display of the window content in the WiFi display (source-side image data acquisition, encryption, encoding, decoding and playback of the mirrored data received by the sink side), etc.

Through the Wifip2pmanager interface and Wifip2pservice interaction, through the wifi-direct to realize the auto-discovery of WiFi display.

The Window Management Service is the listener that displays the management Service, and the Window Management Service registers with the Displaymanagerservice via the Displaymanager interface, and after the WiFi monitor is discovered and connected successfully, and the WiFi monitor disconnects and the status changes, The event is sent through a callback to the window Management Service, and the corresponding callback function of the window Management Service, ondisplayadded, ondisplaychanged, ondisplayremoved, is invoked to register the WiFi display in the window system and to unregister it, Status changes, and other processing.

In addition, the Displaymanagerservice service also directly invokes the Window Management Service (Windowmanagerservice is the implementation of the interface) through the Windowmanagerfuncs window management function interface to refresh the contents of the window. The same Displaymanagerservice service also directly invokes the function Setdisplayviewports of the input management Service through the Inputmanagerfuncs interface to set the display view information for the display required by the input system.

The display management system also interacts with the Mediaplayerservice service through the Imediaplayerservice interface.

If you call the Listenforremotedisplay function of the Mediaplayerservice service to instantiate a remote display's local proxy object Remotedisplay in the media service, The display system calls the media service via the Iremotedisplay interface, and currently the Iremotedisplay interface has only dispose of an interface function to disconnect the remote monitor and stop listening for new connections.

The Wifidisplaysource object that displays the management system also invokes the MAKEHDCP function of the Mediaplayerservice service to instantiate an HDCP object and return to the display system a IHDCP interface for HDCP Cryptographic services.

The Tunnelrenderer object that displays the sink side of the system also invokes the CREATE function of the Mediaplayerservice service in its initplayer, creating a MediaPlayer object. and returns a Imediaplayer interface for use by the display system to enable playback of the mirrored data received by the sink side.

Display the image data obtained from the source side of the management system after the audio and video encoding (H264), and then for HDCP encryption and PES packetization and TS fluidization (converted to TS Stream), finally packaged into RTP packets through the UDP channel sent to the sink side, The sink end is going through the opposite process, receiving RTP packets from the UDP channel, then TS parsing and PEs de-packetization and HDCP decryption, and finally decoding the decoder. The decoded data is sent to the player's renderer for rendering.

The audio and video codec of the source and sink end are interactive through the IOMX interface and the underlying multimedia frame, which realizes the audio-video codec function. The IOMX interface corresponding to the object OMX is also instantiated on the Mediaplayerservice server, In the Connect function of the client object omxclient, the Iomx interface of OMX is returned by invoking the GETOMX function of the Mediaplayerservice service. The OMX object is the encapsulation of the multimedia framework OPENOMX.

Audio and video coding at the source, TS streaming, HDCP encryption, RTP package sending process all have playbacksession thread class management and scheduling, Playbacksession class initialization when instantiating a Surfacemediasource object, An Bufferqueue object (Bufferqueue derived from isurfacetexure) is instantiated inside the Surfacemediasource object. After establishing a connection to the sink end, the Bufferqueue object is passed to the Java layer by ondisplayconnected the callback function of the Iremotedisplayclient interface. The Java layer's Wifidisplayadapter object receives the Ondisplayconnected event and calls the Createdisplay function of the surface class to register a virtual display in the Surfaceflinger service. and call the Setdisplaysurface function of the surface class to pass the Bufferqueue to the displaydevicestate variable corresponding to the Surfaceflinger service virtual display: Therefore playbacksession can use the Bufferqueue object to read the data to be mirrored from the Surfaceflinger service.

After the data received by the sink-side Wifidisplaysink object is decoded by RTP (by the Rtpsink object), it is sent to the Tunnelrenderer object for rendering. The Tunnelrenderer object instantiates a playerclient playback client in the Initplayer function, A MediaPlayer object is created by invoking the CREATE function of the Mediaplayerservice service through the Imediaplayerservice interface and the Tunnelrenderer object Imediaplayer interface is returned. The Tunnelrenderer object uses the Imediaplayer interface to connect to the received mirrored data for playback and rendering, The Tunnelrenderer object also instantiates and obtains a surface object in the Initplayer function through the Surfacecomposerclient object, and call its getsurfacetexture function to get the corresponding isurfacetexture for the surface object, and call the Setvideosurfacetexture function of the Imediaplayer interface to assign the Isurfacetexture to the player, so that the display data decoded by the player is displayed to the Surfaceflinger display service.

The Wifidisplaysink object on the sink side and the object Wifidisplaysource the source end are responsible for the processing of the WiFi Display interaction protocol, and two objects contain a Anetworksession object responsible for the network interaction session between the two.

The following is a frame diagram of the WiFi Display protocol.

ANDROID4.2 's Open source code provides a specific implementation of the WiFi Display protocol, but only executes as a local test command on the sink side, and does not provide user input functionality in the diagram, but provides a complete implementation code for the WFD source side. This is mainly because Android system is mainly used as a mobile phone tablet operating system.

The implementation of ANDROID4.2 's wifidisplay includes the Java and C + + sections. The Java section functions mainly correspond to displaymanagerservice services, The Displaymanagerservice service instantiates and enlists a Wifidisplayadapter object (derived from DisplayAdapter) for Wifidisplay, which is used to establish a connection to the corresponding display device. Each displayadapter corresponds to a display device Displaydevice class one by one. Each display device corresponds to a Logicaldisplay object in the Displaymanagerservice service.

Displaymanager initiated requests for WiFi display are forwarded by the Displaymanagerservice service to the Wifidisplayadapter object processing, in order to achieve asynchronous interaction and avoid deadlocks, The Wifidisplayadapter object initiates a thread for each request to actually complete the request processing and instantiates a Wifidisplaycontroller object to encapsulate the invocation of those requests. A Wifidisplaycontroller.listener object is also instantiated inside the Wifidisplayadapter object to listen for the state of the request. The Wifidisplaycontroller object internally instantiates a Wifip2pmanager object used to initiate a Wi-Fi direct request to the Wifip2pservice service through its interface. Initiates a connection event that listens for the underlying WiFi display by instantiating a Remotedisplay object when connected to the WiFi display.

The C + + section is located in the media framework layer, under the Wifi-display directory of the Libstagefright directory, and the Wifi-display directory contains two sink and source two subdirectories (corresponding to the sink and source of the WiFi display, respectively) corresponding program), the Wifi-display directory also contains the programs that are common to source and sink.

Mainly includes the following files:

Sink directory mainly contains WifiDisplaySink.cpp, RTPSink.cpp, TunnelRenderer.cpp and other C + + class files and corresponding header files, respectively, is responsible for the sink-side Protocol protocol, RTP receive and mirror data rendering process. While TS parsing and PEs de-packetization and decoding process have the player nuplayer complete, because the sink is only a test routine, so there is no HDCP decoding process.

The source directory mainly includes WifiDisplaySource.cpp, PlaybackSession.cpp, MediaPuller.cpp, Converter.cpp, TSPacketizer.cpp, Sender.cpp such as C + + class files and corresponding header files, respectively, is responsible for the source side of the protocol interaction, session management, Image Media reading, encoding, TS Packaging and RTP package delivery process. The Converter object includes a Mediacodec object that is responsible for the encoding process, and the Mediacodec object actually calls the Iomx interface through the Acodec object to complete the encoding of the mirrored data using the OPENOMX media framework.

The common class files used by sink and source are primarily ANetworkSession.cpp and corresponding header files, and a Wifidisplaysink main program wfd.cpp, which is used to generate command WFD.

The main process of the protocol includes the connection process of the WiFi display device and the package sending and receiving process of the mirrored data, which is described in the next blog post.

17th Article--android Displaymanager Service analysis One