Brief introduction of Wayland and Weston

Original address: http://blog.csdn.net/jinzhuojun/article/details/46794479

To put it simply, Wayland is a set of communication protocols between the display server (Wayland compositor) and the client, and Weston is the reference implementation of Wayland compositor. Its official website is http://wayland.freedesktop.org/. They are positioned to replace the X graphics system on Linux. The X-graphic system has been around for about 30 years, and its design seems to be a bit dated today. In x systems, x server serves as a central service, connecting Clien and hardware as well as compositor. But today, many of the things that have been done in X server have been moved to kernel or a separate library, so x server is a bit cumbersome. Wayland removes the middle tier from the architecture, compositor as the display server, enabling the client to communicate directly with the compositor, allowing for greater flexibility and performance than predecessors.

Wayland can be used for both traditional desktops and mobile devices, and has been used for commercial operating systems such as Tizen,sailfish OS, while more and more windows and graphics systems are beginning to be compatible with Wayland protocols. Wayland A library of display server-to-client communication based on the domain socket (simple example-based introduction can be found in the http://blog.csdn.net/jinzhuojun/article/ details/40264449), and defines a set of extensible communication protocols in XML form. This protocol is divided into Wayland Core protocol and Extension Protocol (located in Weston). Weston as a reference implementation of Wayland Compositor, General and Wayland synchronous release. Other Wayland Compositor implementation also like Mutter, Kwin, Lipstick, Enlightenment, Clayland and so on.

Below respectively from the architecture, the source code and the module structure, renders the pipeline, the window and the input management several aspects introduced the next Wayland/weston realization.

Architecture

Wayland's system architecture can be found in the official documentation, no longer described. Weston from the internal architecture, mainly divided into window management (shell), synthesizer (compositor) and input management of several parts. It can be seen that if you use Android as an analogy, it is functionally equivalent to Inputmanagerservice,windowmanagerservice and Surfaceflinger. From a general process point of view, the Input management module accepts user input, and then on the one hand the shell makes the appropriate window management operations (such as window stack changes, focus changes, etc.), on the other hand the input event passed to the client registered the corresponding input events. When the client receives it, it does the corresponding action in handler, such as adjusting the view and redrawing it. If a redraw occurs, the client passes its handle to the server after the new buffer rendering is complete, then the server generates a Z-order-ordered window list, which is then compositor synthesized with renderer, and the final output ( such as to framebuffer).

Weston is the main service process, and its event-handling model uses a typical reactor model. According to the principle of everything in Linux, the main loop waits for a series of file FD through the epoll mechanism. Unlike thread-based binders, this model is a serial event-handling model. Procedure calls on this model are asynchronous without additional synchronization mechanisms. The advantage is that there will be no competition problems, and data synchronization overhead is small. The disadvantage is that when there is an event processing time-consuming or waiting for IO, it is possible to degrade the performance of the entire system or respond in a timely manner.

Several core FD waits on the main loop include:
? Server/client Communication: Listener FD is established at Weston startup and listens for new client connections. A client connection will establish a pair of domain Socket,wayland with Weston to communicate based on it.
? Input processing: On the one hand, add and delete events are monitored through the udev monitor device. On the other hand, if a new device is added, the device will open and listen to the FD to get the input event.
? Other: Listen for events such as a timer (for scenarios such as a sleep lock screen) and signal (such as receiving SIGINT, SIGTERM, and exiting the main loop when sigquit). The timer and signal can be expressed by FD using TIMERFD and SIGNALFD respectively. In addition, there are logind dbus connections and so on.
In addition to this, an idle list is maintained in the event loop. Operations that require asynchronous processing in Weston can be placed in it. Each cycle checks to see if there is a task, and some words are taken out.

Below is a look at the run-time process model of Weston. Weston is designed to run as a normal user, but it needs to be started with Weston-launch. When you need to do some work that requires root authority, such as about DRM, TTY, input device, it is left to Weston-launch.

Weston will have some sub-processes at startup or on demand, which are essentially Weston's clients, and they will do some system application work through a dedicated protocol. such as System Application Weston-desktop-shell is responsible for some system global interface, such as panel, background, cursor, app launcher, lock screen and so on. It does not serve as part of the Weston service itself, but as a client. It works a bit like the Systemui in Android. This makes it easy to replace the custom interface. The Weston-keyboard is a soft keyboard panel. Weston-screenshooter and Weston-screensaver are used for screenshots and screensavers respectively, and they are all activated on demand, Weston. The former starts when the screenshot shortcut is pressed, and the latter is activated when a lock screen is required.

In addition, Weston will read the Weston.ini configuration file at startup, which can configure desktop, animation and backend information. See http://manpages.ubuntu.com/manpages/raring/man5/weston.ini.5.html for detailed configuration.

Source code and module structure

Wayland/weston/libinput and other projects source code is located under Http://cgit.freedesktop.org/wayland.

The Wayland protocol is defined in the Protocol directory, and the communication protocol is implemented in the SRC directory. It mainly compiles three libraries, Libwayland-client,libwayland-server and Libwayland-cursor. The first is the client-side implementation of the Protocol, and the second is the server-side implementation of the Protocol. The third is the implementation of mouse-related processing in the protocol. At compile time, the Wayland-scanner executable is compiled first, which uses the expat library to parse the XML file. Generate the appropriate wayland-protocol.c,wayland-client-protocol.h and wayland-server-protocol.h for the wayland.xml. They will be used by the Wayland client and server at compile time. At the same time, Wayland-scanner also needs to play the same role in the Wayland extension protocol that generates Weston.

Wayland relies primarily on two libraries, one of the expat protocols mentioned above, and the other libffi is used to generate a springboard code for the corresponding calling convention in a cross-process procedure call based on the function description.

The main implementation of Weston is in the SRC directory. Similar to Wayland, the Protocol directory is decentralized with the Wayland protocol definition. In the clients directory are some examples of clients, as well as examples of core clients such as Desktop-shell and keyboard, and also include Simple-egl, SIMPLE-SHM, Simple-touch and other targeted simple use cases. Several Backend:shell backend, render backend and compositor backend are loaded separately during the Weston startup process. They are used for window management, compositing rendering, and composite content output, respectively.

Since these back ends can have different implementations, for logical independence and structural flexibility, they are compiled into a dynamic-link library that can be loaded when Weston is initialized. This method is widely used in Weston, and some functions, such as screen sharing, are loaded in this form.

For example, Compositor backend mainly determines how the results of the compositor after synthesis are disposed of. From the data structure, the weston_output is an abstraction of the output device, and the following backend implements the specific output device.
? Fbdev: A framebuffer device that outputs directly to Linux. Interface is common.
? Headless: In conjunction with Noop-renderer, the logic can be tested on a machine without a windowing system, such as a server.
? RPI: For Raspberry Pi platform.
? RDP: The composition is transmitted via RDP to the RDP peer display for Remote Desktop.
? Drm:direct redering Manager, this is generally used on the desktop.
? X11:wayland Compositor as the client of X server. It allows the Wayland client to run on X11.
? Wayland:wayland COMPOSIOTR is also a client of another Wayland compositor as a server. For nested compositor.

Renderer backend is mainly used for the synthesis of compositor, except Noop-renderer, there are two kinds of gl-renderer and pixman-renderer. The former is GPU hardware rendering, and the latter is software rendering. Shell backend is used to implement specific window management. The corresponding implementations are in the Desktop-shell,fullscreen-shell and Ivi-shell directories respectively.

The Wayland/weston runtime relies on a library that has the following main interrelationships, which are generally as follows.

? LIBEGL, Libgles: The glue Layer,mesa of the local windowing system and graphics driver provides an open source version of the implementation.
? LIBDRM: Package Kms,gem and other graphics-related interfaces. Platform-related.
? Libffi: Used to describe the build function springboard and call at runtime based on the calling interface.
? Pixman: A library for pixel operations, including region, box, and more. Many platform-related optimizations were used.
? Cairo: Software rendering library, similar to Skia. There are also OpenGL backend.
? Libinput: Input processing, relying on Mtdev, Libudev, Libevdev and other libraries.
? Libxkbcommon: Mainly used for keyboard processing.
? Libjpeg, libpng, LIBWEBP: Used to load a variety of picture files, like wallpapers, panels and mouse are needed.

Rendering pipeline

A Wayland client to render memory to the screen, first request a graphic buffer, and then pass it to Wayland Compositor and notify it to redraw. Wayland Compositor collects all Wayland client requests and then synthesizes all submitted graphic buffer in a certain period of time. The output is finished after compositing. Essentially, the client needs to draw the window content onto a buffer that is shared with compositor. This buffer can be common shared memory, or it can be a GBM in DRM or a graphic buffer provided by Gralloc for hardware (such as GPU) operations. On most mobile platforms, there is no dedicated video memory, so they end up coming from system memory, except that the graphics acceleration hardware generally requires a physically continuous and aligned memory. If it is common shared memory, it is usually physically discontinuous, most of which is used for software rendering. Some graphics drivers also support hardware acceleration with physical discontinuous memory, but are less efficient. Depending on the type of buffer, the client can choose to draw on its own, either through Cairo,opengl, or at a higher level such as a qt,gtk+ widget library. After drawing, the client passes the handle of buffer to the server and the area to redraw. On the server side, compositor the buffer to texture (if shared memory uses glteximage2d to upload textures, hardware acceleration buffer generates external textures with gl_oes_egl_image_external extensions). Finally, it is composited with its other window contents. The following is an abstract flowchart.

Note The default buffer in the Wayland design is assigned from the client side. This is different from the strategy of allocating buffer on the server side of Android. This allows the client to design its own management strategy for buffer. Theoretically, the client can always use only a block of buffer, but because this buffer is competing for simultaneous client and server access, the general client will implement the buffer queue. The key link in the pipeline is the transfer of buffer across processes, that is, the efficient transfer between client and server. The buffer cannot be transmitted by copy, so it will only transmit handle, which is essentially the FD that transmits buffer. We know that FD is per-process. The main IPC mechanisms that can pass FD are binder, domain socket and pipe. The Wayland is a domain socket, so it can be used to transmit FD.

On this line, you can see that the client and server side will be drawn. The client draws the local window content, and the server side is used primarily for compositing-time rendering. Note that both sides can be individually selected for rendering with software or hardware. Today's commercial devices are mostly hardware-accelerated rendering. Like the Surfaceflinger on Android and Mir on Ubuntu, Wayland is also based on the EGL interface. EGL is used to correlate the local windowing system with OpenGL, similar to WGL, GLX, except that it is used for embedded platform. In the Wayland/weston system, Wayland defines the window abstraction for EGL as the glue layer of the EGL stack (which is the manufacturer's graphics driver) and the Wayland protocol. It expands EGL to include interfaces such as EGLBINDWAYLANDDISPLAYWL, EGLUNBINDWAYLANDDISPLAYWL, EGLQUERYWAYLANDBUFFERWL, etc. The Wayland-friendly EGL library should provide their implementation, meaning to provide Wayland EGL platform, such as Mesa (SRC/EGL/MAIN/EGLAPI.C). Another approach is to add these support (Hybris/egl/platforms/common/eglplatformcommon.cpp) in the same way as Eglplatform in Libhybris, through EGL wrapper. At the same time, EGL stack needs to provide vendor-related protocol extensions to enable client to share buffer with compositor. The WAYLAND-EGL library provides the role of surface and EGL bonding in the Wayland. An EGL window to be used for hardware acceleration can be created based on Wayland's surface, that is, through the interface provided by WAYLAND-EGL to create Wl_egl_window. The Wl_egl_window structure contains the wl_surface and then Wl_egl_window can be passed into the Eglcreatewindowsurface () interface of the EGL. This links the Wayland surface to the EGL stack.

Window Management

As mentioned earlier, buffer needs surface as a carrier, and this surface can be understood as a drawing surface of a window. If a Wayland client window is managed by the window Manager (shell), you need to create the appropriate shell surface for that surface. Here are some of the core concepts involved: Surface,view,shell surface. First, surface is one of the core data structures in Weston. Surface represents a drawing surface of the Wayland client. The Client updates the window by attach the painted buffer to surface, so that surface is the vector of the buffer. In Weston, the shell is the window manager, so a surface to be managed by the window manager needs to create the appropriate shell surface. At the same time, shell surface is actually a view of surface. View is a view of surface. In other words, the same surface can theoretically have multiple view, so there is a list of view in the weston_surface structure. The concept here and our logical window is the view recently, because it corresponds to a window that the user sees. When surface and view are 1:1 relationships (most of the time), surface can be similar to Windows. Their data structures are interrelated on the server side.

If from the server/client angle, they correspond to each other as follows:

In addition, subsurface can be used as an accessory drawing surface for surface, which remains associated with the parent surface, but has a separate drawing surface similar to the Surfaceview in Android. Mainly used for Camera,video and other applications.

Window management is not directly managed view, but divided into two tiers for management. Layer is the middle layer, the layer in the system generally has the following, in order from top to bottom:
? Fade Layer
? Lock Layer
? Cursor Layer
? Input Panel Layer
? Fullscreen layer
? Panel Layer
? Workspace Layers
? Background Layer

Where the workspace layer is an array, the default is one, there can be multiple, and its number can be specified in the Weston.ini. Most of the common application windows are in this layer. These layers are strung into lists. Each time a composition is to be made, the view of these layers is first merged into a view list in order. This way, compositor only needs to access the view list during the composition process.

As you can see, this is the process of synthesizing an ordered list of two-level ordered lists. This is similar to how the WMS in Android works by defining a section of z-axis for each type of window. Each type of window pair in the WMS is given a cardinality (defined in Windowmanager.java), which is multiplied by the multiplier plus the offset to get the area boundary on the z axis. The difference is that Weston is not a numeric value but a sequence table represents Z-order. To combine specific data structures:

Input management

To improve the reusability and modularity of the input Management section. Weston separates the processing of input devices (keyboards, mice, touch screens, etc.) into a single library, the Libinput. In this way, other graphics processing systems can share this part, such as X.Org Server and Mir. Specifically, it provides basic functions such as device detection, device handling, input event handling, and similar to the Eventhub in Android. It also features pointer acceleration,touchpad support and gesture recognition. Libinput is more like a framework that integrates the functionality of several lower-level libraries. It relies primarily on the following libraries:
? Mtdev:multi-touch device processing, such as it converts protocol a without Tracking ID to protocol B.
? Libevdev: Handles Evdev module docking in kernel.
? Libudev: used primarily for communication with UDEVD, thereby acquiring increased deletion events for the device. can also be obtained from kernel.

Weston in the input management module and Libinput docking, it realizes two parts of the function: first, the input device maintenance, and the second is the processing of input events. Input events are handled both in Weston and to the corresponding client. From the event-handling model, Libinput's main loop monitors Udev Monitor FD, which is mainly used to monitor the addition and deletion events of the device. If a device is added, the device is turned on and the FD is added to the libinput main loop. On the other hand, Libinput's Epoll FD is added to the main loop in Weston. This cascade of epoll, whether it is udev Monitor or input device FD has an event, will notify the main loop of Weston and Libinput. These events are stored through the event buffer queue in Libinput, while Weston takes the event as a consumer and processes it based on the event type.

The Weston supports three input devices, namely keyboard, touch and mouse. A set of input devices belongs to a seat (strictly speaking, the seat includes a set of input). As a result, the weston_seat contains three structures for Weston_keyboard,weston_pointer and Weston_touch. There can be multiple seat in the system, and their structure is strung in the Weston_compositor seat_list linked list. The corresponding data structure is as follows.

As you can see, for focus processing, each device has its own focus, which points to the focus window for drag-and-drop and input. The member Focus_resource_list contains the resource object that corresponds to the input device proxy in the client of the focus window. In this list, it means that the corresponding event can be received.

Finally, Wayland/weston as a project being actively developed, there are many other features that have been added or are being added. This article is only a bold introduction of the main structure and function of Wayland/weston, and then expand the details.

Copyright NOTICE: This article for Bo Master original article, without Bo Master permission not reproduced.

Brief introduction of Wayland and Weston