Lcui Design Principle

Lcui design document

2012-2013 Liu Chao (lc-soft@live.cn)
2012-2013 lcui Development Team (www.lcui.org)

This document describes the implementation methods and principles of each lcui function. If you want to hack or expand this function library, read it carefully. Layer Involved source files: Src/lcui_graph.c Src/lcui_graphlayer.c The graphical interface in lcui consists of several parts. Each part has a layer used to store the bitmap of the part. The drawing operation on the part is performed on the part layer. Layers form the entire GUI through top and bottom overlays, as shown in: In the memory, a layer is a piece of data in the memory space. This memory space is used to record each attribute of the layer (such as size and coordinates) and the rgba value of each pixel. RGB represents red, green, and blue. These three colors are called "primary colors ", A primary color is a "Basic Color" that cannot be obtained by mixing other colors. A primary color can be mixed in different proportions to produce other new colors. For example, when RGB is 0, it indicates pure black. You can open the color editor to test the effect, as shown in: In rgba, A indicates Alpha, that is, the transparency of the pixel. It is usually obtained when the image is superimposed and mixed, but this Alpha is useless when the final image is output. By default, layers in lcui use the rgb888 color mode to store RGB data, that is, R, G, and B each occupy 1 byte (8 bits), and then include Alpha, the data of a pixel occupies 4 bytes in total. Assume that there is a layer of X with alpha channel, the memory occupied by pixel data is 4x1280x800 = 4096000 bytes = 3.9 kb ≈ MB. Because each primary color occupies 1 byte, 1 word can be saved to 0 ~ The number in the range of 255. Therefore, each primary color has 256 kinds of brightness. In this way, each pixel can display 256*256*256 = 16777216 colors, however, these colors can only be fully displayed on a 24-or 32-bit screen. For 18-bit (rgb666), 16-bit (rgb565), and 15-bit (rgb555) screens, the color is distorted. In the future, we will consider adding a color palette for the layer and supporting other color modes to reduce memory usage. The lcui_graph struct used to save the layer pixel data may also be adjusted. Graphic Output Involved source files: Src/output/graph_display.c Src/output/framebuffer. c Src/output/win32.c During initialization, lcui obtains screen information, such as the size and color mode, and creates a root layer that is equivalent to the screen size, the layer of the widget created by the program is its child layer by default. The image update on the screen is completed by a separate GUI thread, which is created during lcui initialization. The workflow of this thread is: first update the coordinates of the mouse cursor displayed on the screen, then process the part message, and then transfer the invalid areas recorded in each part to the root record, finally, the images displayed on the screen are repainted Based on the invalid area of the record. After repainting, the invalid area record is deleted. If there is no invalid region record after processing the part message, the delay will be several milliseconds to reduce the CPU usage of the program, which is executed cyclically. At present, it seems that this solution to reduce CPU usage is somewhat inappropriate, and other better solutions will be considered in the future. When the GUI thread re-draws an image in an invalid area, it first records the layers that appear in the area, and then removes the layers that are completely blocked by other layers, at last, the layers are stacked according to the stack sequence. After the layers are stacked, the final image is obtained and finally written to the frame buffer to redraw the invalid area. This process can be called "rendering ". Note: The dotted line indicates that the rectangle is an invalid area (dirty rectangle). The GUI thread only overlays the layers that intersect the area. How does lcui display images on the screen? In Linux, lcui directly writes frame buffering to display images. What is frame buffering? You can refer to the following introduction (from the Internet ): Frame Buffer is an interface provided by Linux for display devices. It abstracts the video into a device, it allows upper-layer applications to directly read and write the display buffer in graphic mode. Such operations are abstract and unified. Users do not have to worry about the location of Physical video memory, page feed mechanism, and other details. These are all driven by the framebuffer device. In essence, Linux framebuffer only provides hardware abstraction for graphics devices. In the developer's view, framebuffer is a display cache. Writing data in a specific format to the display cache means outputting content to the screen. So framebuffer is a whiteboard. For example, for a 16-bit framebuffer, the two bytes in framebuffer represent a point on the screen, from top to bottom, from left to right, the linear relationship between screen position and memory address is sequential. The frame cache can be anywhere in the system memory. The Video Controller refreshes the screen by accessing the frame cache. Frame cache is also called frame buffer or refresh buffer. The frame here refers to the entire screen range. The frame cache has an address in the memory. By constantly writing data to framebuffer, the display controller automatically retrieves data from the frame buffer and displays the data. All images share the same frame cache in memory. After reading the above content, you should have a certain understanding of frame buffering. In this article, we will not detail the operations related to frame buffering. For details, refer to relevant information on the Internet, the following are the general steps for frame buffering: Call the OPEN function to open the/dev/FB device file. Call ioctrl to obtain the parameters of the current display screen, such as the screen resolution, the number of bits of each pixel, and calculate the size of the screen buffer according to the screen parameters. Call the MMAP function to map the screen buffer to the user space. After ing, you can directly read and write the screen buffer for graphic display. How can I write data into the frame buffer to see what I want? For example: The screen size is 320x240, and the number of pixels is 32. Fill the area with the coordinates (0, 0) and the size is 5x5 in white. The code segment is as follows: Int point_x = 0; int point_y = 0; int rect_w = 5; int rect_h = 5; int screen_w = 320; int screen_h = 240; unsigned char * fb_mem = frame buffer first address; for (y = point_y; y <rect_h; ++ y) {for (x = point_x; x <rect_w; ++ X) {// because it is 32 bits, divide by 8 to 4 (bytes). // y * screen_w + x converts two-dimensional coordinates (x, y) to one-dimensional coordinates. // Multiply this one-dimensional coordinate by 4, it is equal to the first address of the pixel in the memory of the current coordinate. fb_mem [(y * screen_w + x) * 4] = 255; fb_mem [(y * screen_w + x) * 4 + 1] = 255; fb_mem [(y * screen_w + x) * 4 + 2] = 255 ;}} The initialization code of the frame buffer is omitted. In Windows, lcui will first open up a piece of memory space during initialization. As a frame buffer, the write mode of pixel data is the same as that in Linux. However, because the image is displayed in a Windows window, you need to submit the pixel data in the frame buffer to the Windows window after the invalid area is re-painted, that is, call the functions provided by the Windows system to draw the data in the frame buffer into the window. Why not draw a graph directly in a window? Because we have not found a method for directly operating bitmap's pixel data using pointers, we need to call the setbitmapbits function to overwrite the bitmap's pixel data. Instead of calling the setbitmapbits function each time the invalid area is re-painted, and then calling the bitblt function for plotting, it is better to call the setbitmapbits function and the bitblt function to re-draw images in the window after all invalid areas are re-painted. Although I have tried dibsection before, it can save the call of the setbitmapbits function, but there is a problem: the graphic display is abnormal. You can see the "shadow" of the interface, but it does not work properly. The graphic output efficiency on the Win32 platform is not high. Every time the screen content is re-painted, the image in the Windows window is re-painted. Currently, the lcui graph is directly output to the Windows window. The idea is to let the lcui graphic interface get rid of the Windows window, however, the problem is that you do not know how to implement the layer display and layer control functions in windows. We plan to allow lcui to support xwindows in Linux, so that we do not have to switch to the console mode to experience the graphic interface. If you are interested in this function module and have good ideas, you can provide technical support to this project. Invalid Region Involved source files: Src/MISC/rect. c Invalid areas are mainly generated by parts. When you call a function to move, resize, update, or redraw a part, the areas to be repainted are recorded as invalid areas, to allow the GUI thread to update the image in the region. In order to reduce repeated invalid regions, when adding invalid regions, the system checks whether the areas and recorded regions have the intersection, inclusion, and inclusion relationships. If yes, partitions are separated to delete the overlapping areas. If it is contained, the region in the record is deleted. If it is included, the region is not added. Is the process of splitting three invalid regions. Add the first region (red) without processing. Add the second area (green). The gray dotted line shows the split line, which splits into three subareas and removes a subarea that overlaps the first area. Add the Third Region (blue). The process is the same as the previous one, but several more judgments and segmentation operations are performed. In Windows, the processing of invalid regions is better than that of lcui. When a program calls the beginpaint () function, several closely related areas are merged into a large region. Gui Components Involved source files: Src/GUI/widget_base.c Src/GUI/widget_msg.c Src/GUI/widget_event.c Src/GUI/widget_style.c The graphic interface is mainly used for graphical human-computer interaction. users input data to the computer with the mouse and keyboard, and the computer returns results to the user through the graphic interface. In lcui, components are the basic elements of the graphic interface. Different types of components have different ways to interact with users. Components have their own attributes. These attributes can be pre-set by the developer, or after the program runs, the program can set the corresponding attributes based on the user's operations. Part message Most of the operations on a part are asynchronous. When you set the coordinates, dimensions, or re-drawing parts, or update data of the part, these operations are added to the component's message queue in the form of messages. Each component has a message queue. The number of messages in the queue is limited, newly Added messages overwrite existing messages. This is to avoid the endless loop caused by messages nested and trigger, and also to reduce redundant messages of the same type. These messages are finally processed by the GUI thread. When the component is updated, it traverses the message queue in the component from the parent node to the child node, retrieves the messages in the queue, and then calls the corresponding function based on the message number to implement the operation. The basic messages of lcui are processed and executed by the GUI thread. By default, all messages have corresponding processing methods. For custom components, other processing is required, however, you do not want to share the default message. You can customize a new message, which is a new message number. You can establish a connection between the callback function and the message, to respond to custom messages. For example, the text content setting function of the label component is implemented by using custom messages. Considering that some part operations take a long time, in order to avoid delay in processing other parts, custom messages are not directly processed by the GUI thread, but transferred to the program task queue, let the main thread process in the main loop. The widget has its own mutex lock to ensure the integrity of the widget data during operation. When processing custom messages in the main loop, if you want to modify the part, we recommend that you lock the mutex lock. If the lock has been locked by the GUI thread, it will block and wait until it is unlocked, after using the components, remove the mutex lock. When a GUI thread updates, redraws, or destroys a part, it attempts to lock the mutex lock. If it cannot lock the mutex lock, it will skip this update/re-painting process on the part, the movement, size adjustment, display, and hiding of parts will not be affected by the mutex lock. Part events Lcui provides an event mechanism for parts to respond to user input, window dragging, Button clicking, text input in the text box, and so on. What is the difference between a message and an event? In fact, there is no big difference. As long as the event or message response function is registered, this function can be called when an event or message is generated. In other guis, the response function of an event returns a value to determine whether the event is processed. Other tasks are performed based on the returned value. However, this requirement is unavailable in lcui. Widget Style The style of a widget can be defined in an external file, so that you can adjust the style of each widget without modifying the source code and re-compiling. When running the program, you need to call the corresponding function to load the specified style file first. The components created later will apply the style in the style file. The part style is described in text and adopts CSS-like syntax, but its functions are not complete yet. It only supports defining the coordinates, alignment mode, size, background image, background color, and border of the part, other advanced functions have not yet been implemented. If you are interested in this function, you can participate in the improvement of this function. Graphic Processing Involved source files: Src/lcui_graph.c Src/draw/line. c Src/draw/border. c Src/draw/smooth. c Src/draw/rotate. c Src/BMP. c Src/BMP/JPEG. c Src/BMP/load_image.c Src/BMP/PNG. c Lcui provides basic graphic processing functions, such as cropping, overlays, and overwriting. In addition to these functions, the src/draw/directory also provides source code for drawing other images. However, the author does not spend much energy on Graph processing, in the future, we will consider using other graphics processing function libraries for advanced graphics processing. Lcui supports loading image data from external image files. Currently, the supported formats include BMP, PNG, and JPEG, among them, the decoding of PNG and JPEG images is supported by the libpng and JPEG function libraries. When you want to use an image in a specified area of the image, you do not want to use additional memory space to save the image captured from the image. You can use the image reference function, when you call a function to overlay, crop, and overwrite an image, this function operates pixel data in the reference area of the source image. Font Involved source files: Src/font/fontlibrary. c Src/font/bitmapfont. c The default font rendering engine of lcui is FreeType. During lcui initialization, the font database is initialized. This database is used to record the font file information and font bitmap. By default, built-in font bitmaps are added; if the font file path is specified in the environment variable, the font file under the path is loaded. If the file is loaded successfully, the font is set to the default font. When drawing text on a part, lcui first tries to obtain the ready-made font bitmap from the font database. If the acquisition fails, the new font bitmap will be loaded from the font file, and record it to the database. All components with the text display function use the font bitmap in the library. The data in the library is destroyed only when the library exits lcui, and the destruction of the parts does not affect the records in the library. In this way, the redundant font bitmap occupies excessive memory space. Character Set Involved source files: Src/font/charset. c The encoding method of the source file determines the encoding method of the String constant in the source code. The character encoding conversion function of lcui is not complete yet. in Linux, The libiconv function library is used for encoding conversion, but only provides gb2312 to unicode encoding; while in windows, only the lcui built-in UTF-8 to Unicode function, therefore, if you compile the source file of gb2312 encoding in Widnows, when running the program, the text displayed on the page may be garbled. We recommend that the source file be encoded in UTF-8. Memory Management The memory management module has not been added in version 0.14.0. It is planned to be added in later versions. It mainly encapsulates the memory allocation and release functions and records the information of each memory block, such as size and purpose, you can calculate the total occupied space of memory blocks for each purpose, so that you can know where memory overhead needs to be reduced. To maintain high efficiency, you need to think about the appropriate data structure and algorithms. If you can, this module can provide the memory pool function. Thread Involved source files: Src/thread/pthread/thread. c Src/thread/pthread/mutex. c Src/thread/Win32/thread. c Src/thread/Win32/mutex. c Src/thread/lcui_thread.c Lcui encapsulates the thread operation functions on Linux and Win32 platforms. The usage of the functions is roughly the same as that of the pthread thread library. The thread operation functions are divided into two types: one class is prefixed with _ lcuithread _ and the other class is prefixed with lcuithread. The former can be called directly and functions can be normally implemented without the need to initialize lcui first. The latter needs to initialize lcui, and the final function implementation will still call the former. The latter records the parent-child relationship of each thread. When the ID of the lcui program is obtained in other subthreads, the program id is obtained by querying this record. However, at present, the lcui program only records a single program. This function does not play a major role. In the future, there may be lcui programs that require multiple lcui instances, for example, you can use an lcui program as a "platform" to run multiple lcui programs in the form of threads on the platform. Before lcui exits and releases the resource, it will first cancel the thread in the record and then release the resource, because the resource to be released may be used by other threads, if it is released directly, the program may run abnormally. In the future, we may consider adding a thread pool for this module. Input Device Involved source files: Src/input/keyboard. c Src/input/mouse. c Src/input/touchscreen. c In the Console mode of the Linux environment, lcui sets the console to non-buffered and non-echo mode during initialization. After pressing the key, the key value in the input buffer is obtained directly, however, the keyboard events are incomplete. You can only detect which key is pressed and which key is released. You have used one method before, you can obtain the key status by reading the data in/dev/input/event3. However, some devices do not necessarily have/dev/input/event3, and the numbers behind them are uncertain and need to be set manually. After porting the test program of this method to another device, it can only detect which key is pressed, which may be a keyboard driver problem of the device. By reading data from the mouse device/dev/input/mice, lcui can obtain the relative coordinates of the mouse and the status of the left and right keys. When the data changes, a mouse event occurs. However, the mouse event is a bit incomplete and the intermediate key and scroll wheel status cannot be obtained. In Win32 environments, keyboard and mouse events are generated by lcui Based on the ready-made messages provided by the system. Event Involved source files: Src/kernel/event. c The event mechanism is mainly used for asynchronous task processing. If there is a task that needs to be processed but you do not want to delay the next task because of this task, you can use the event mechanism. First, set an ID for a task that is not in urgent need of processing, used to identify it, and then add it as an event to the event queue, waiting for the event thread to process; to enable the event to be correctly handled by the event thread, you also need to associate the function used to process the operation with the event ID. In this way, when the event thread processes the event, this function is called. What is asynchronous task processing? When processing a synchronization task, you need to wait for the task to be processed before other tasks can be processed. asynchronous task processing is the opposite. Other tasks can be performed without waiting for the task to be processed, these tasks are eventually processed. For example: Asynchronous task processing means that you call your friend for dinner. Your friend says you know, and will come to dinner later. You can do other things now. Synchronization task processing means that you call your friends for dinner, your friends are busy, and you are waiting there all the time. When your friends are busy, you can eat together, you can do other things only after eating. Timer Involved source files: Src/kernel/Timer. c The timer is mainly used for tasks that need to be executed after a specified period of time. At first, when developing activebox components for lcui to implement animation playback, this function allows you to update at least one frame at a time. This function allows you to update the animation frames of all activebox parts by one thread, later, the code of this function was released independently as the implementation code of the lcui timer. The main workflow of the timer thread is: Obtain an enabled timer from the timer list and pause it according to the time needed. Update other timers in the timer list, and subtract the time they need to be paused from the time they have been paused. Reset the time of the current timer. Sort the timer pause time in the list from short to long. Send the task in the timer to the program task queue for execution in the main loop. Main cycle Involved source files: Src/kernel/Main. c The main loop is generally executed in the main thread. When the lcui_main function is called, this function creates a main loop, which is mainly used to process the response of part events/messages and the response of the timer. On the Win32 platform, the main loop is also responsible for processing Win32 messages and generating lcui events based on messages. Input Method Involved source files: Src/lcui_inputmethod.c Not complete, To be continued ......

Last Updated: 2013-06-13