Search Image Information on the Internet

Two phases of Image Retrieval Technology

With the rapid development of the Internet, multimedia information on the Internet has also increased dramatically. Therefore, the demand for Multimedia Information Retrieval has also increased. Traditional Information Retrieval mainly focuses on text retrieval, and there are not many researches on multimedia. Multimedia on the internet is dominated by images, so image retrieval has become a hot topic of research.

Internet image retrieval goes through two stages: the first stage is keyword-based retrieval. The second stage is based on the image content.

In a keyword-based image search system, you must first mark all images with keywords before using full-text search technology to search for images. This method has two problems: first, This method requires a lot of manual participation, and it is difficult to implement as the number of images increases; the second problem is that the image contains a large amount of information, and different people have different understandings of the same image. As a result, there is no unified standard for image tagging, therefore, the search results cannot meet your needs.

Content-based retrieval is different from keyword-based retrieval. It does not require too much manual participation, but uses the image's own features (such as color, texture, shape, and so on) for retrieval, strong objectivity. However, because these features do not represent the true semantic information of the image, content-based retrieval results are often unsatisfactory. Therefore, most systems are still keyword-based searches, such as AltaVista and Yahoo! And ditto.

To this end, we propose a new method for image retrieval on the Internet, which combines keyword-based retrieval with content-based retrieval, user feedback is introduced to optimize the search results. Next we will introduce how to collect images, create indexes, and search images on the Internet. This section describes how to improve the search results by using relevant feedback based on user interaction. Finally, we will give a summary of image search.

Image search on the Internet

To establish an image search system on the Internet, three problems need to be solved. The first is how to obtain images from the Internet, the second is how to index the acquired images, and finally how to retrieve images in the image database based on user needs.

1. Image Acquisition

There are a variety of images on the Internet. We need to collect representative images that are most interesting to users for use. First, establish the corresponding image classification hierarchy based on the classification of some popular search engines. Then, select some popular and representative sites for each category as candidates. For example, in sports http://www.nba.com, political http://www.whitehouse.gov, entertainment http://www.disney.com, news http://www.cnn.com and other sites are selected as download images site.

Then, an efficient software tool (crawler) is designed to automatically collect images for the selected representative sites. All pages on the site are sent to the page analyzer for analysis. All images on the page are stored in the corresponding database as links. Meanwhile, some heuristic information, such as the size, file type, file name, and color histogram of the image, will be used for simple classification of the image, the image of the AD bar, background, icon, button and other non-semantic information is different from the image that the user really needs for the user to query.

2. Image Feature Extraction and Indexing

Feature Extraction is required for the collected images and corresponding indexes are created to improve the retrieval efficiency. There are two types of image features: the lower-layer features of the image, the color, texture, and shape of the image. Another feature is the semantic feature of the image.

For low-level features of an image, the color, texture, shape, and other features of the image are used. Color features are irrelevant to the image size and direction, and are not sensitive to the background color. Therefore, color features are widely used in image retrieval. Color Features include color histograms, color-related graphs, and color moment. Texture features represent the visual pattern of an object. They contain the organizational structure of the object surface and the relationship between the object and the surrounding environment. Common methods include the correlation matrix method, texture representation methods such as roughness and contrast, and wavelet transformation. There are two types of shape features: boundary-based shape features and region-based shape features. The most successful representation methods include Fu Liye transformation and immutable moment. These low-level features are extracted by various methods and form a set of feature vectors. Relevant indexes are created and stored in the database.

Because low-layer features do not directly represent the semantic information of the image, we will also extract the semantic features of the image. We use text information related to images on the webpage to characterize the semantic features of images. The information used here is:

The file name of the image and its URL are directly connected to the image content, such as redflower.jpg?cat.jpg=clinton.jpg. The image content is directly reflected in the file name. At the same time, the image URL information also provides some relevant semantic information, such.

The alternative text of an image is commonly used to represent the semantic information of an image on a webpage. It is also the most accurate feature. However, not all authors are willing to provide this information.

Surrounding text around the image is the most likely to express all the content contained in the image on the webpage. Although some texts may not be related to the image, however, these words still express the semantic information of the image to a certain extent, so they are selected as one of the semantic features.

The title of the page where the image is located. Some images are used to enhance the author's intention. Therefore, the content of some images is directly related to the Title content of the page. The page title becomes one of the semantic features.

The hyperlink information of an image is related to the image content to a certain extent. Therefore, semantic features can be calculated through hyperlink analysis.

Link-structure: analyzes links between a webpage and a webpage ), the semantic similarity between images contained in a webpage can be calculated to some extent. This information can be used to enhance the effect of image search.

All these features will be automatically extracted from the web page through the page analyzer and assigned different importance, and the semantic feature vectors of images will be established according to the traditional text information retrieval technology. Each component of a vector corresponds to a keyword. Its value depends on the distribution of the keyword in the webpage related to the image. If a keyword appears many times in a web page, the corresponding weight will be larger: On the other hand, if this keyword appears in many web pages, the corresponding weight will be smaller. This method is widely used in text search and is also suitable for image search.

3. Image Retrieval

A user-submitted query can be a keyword query or an image that the user is interested in, find some of the most similar images in the Image Database and return them to the user. The submitted query will first be converted into a vector that combines low-level features with high-level features, and then calculate the similarity with the vectors of the images in the database. Similarity calculation is completed in two steps: one is to calculate the similarity of low-level features, the other is to calculate the similarity of high-level semantic features, and then use a linear combination method to obtain the final similarity. Images with high similarity become retrieval results.

Feedback to improve search results

Although the combination of lower-level features and higher-level semantic features improves the image search performance to a certain extent, the search system performance is still unsatisfactory, mainly due to the following reasons:

1. According to some popular search engines, the average length of queries submitted by users is 2 ~ Three keywords. This short query is difficult to fully express the user's needs, resulting in a large difference between the search results and the user's needs.

2. indexes stored in the database are created based on the various texts of the collected images. These texts are described from the author's perspective, there are some differences with the words used by users.

3. because there is a lot of information in an image, and different users have different understandings of the same image, this makes even the same query, the expected results vary greatly from user to user.

4. Because the low-level features do not reflect the true semantic information of the image, it is difficult for the system to find the image you really want when you submit an image as a query.

These problems cause unsatisfactory results in automatic image retrieval. Therefore, many systems introduce human interaction and gradually improve the retrieval result through user feedback, that is, selecting correct/wrong examples as feedback. Based on the text information retrieval method, we also introduced feedback in the system to modify the query submitted by users, so that the modified query gradually approaches the real needs of users and improves the system performance.

Through relevant feedback on the modification of the query submitted by the user, the search performance has been improved to a certain extent than previously. However, most of the related feedback does not have the memory capability. The results after each feedback can only improve the query results. Therefore, we introduced the semantic network to record the results of each feedback to the semantic network, so that the system effect gradually increases with the increase in the number of times of use.

Distributed Development Trend

Image retrieval technology provides users with an effective means to search for images of interest on the Internet. It not only utilizes the features of the image itself, such as color, texture, and shape, it also uses the text information related to the image in the webpage as the semantic feature of the image to improve the image retrieval effect. In addition, the system also introduces user feedback to Improve the Quality of user queries, so that the query results are closer to the user's needs. Finally, the system records the user query process through the semantic network to provide guidance for other users.

Because traditional search engines have limitations in design (centralized architecture that collects information through network crawlers), they cannot provide database services with high accuracy and real-time search updates. Therefore, developing a new search framework that supports "point-to-point" and hierarchical distributed search will become the development trend of search engines in the future. This new search framework includes three layers: personal documents, lan, and Internet. When you search for a LAN or the Internet, the search is distributed. At the same time, the system will automatically distribute user queries to similar users or the most suitable search engine on the internet for better results.