Introduction to clustering: Vector Quantization

This article from: http://blog.pluskid.org /? P = 57

This article is part 1 of the "talk about clustering series". For more information, see other articles in this series.

Before talking about other clustering algorithms, let's introduce a bit of a problem: vector quantization. This technology is widely used in signal processing, data compression, and other fields. In fact, in JPEG, MPEG-4 and other multimedia compression formats have VQ this step.

The name of Vector Quantization sounds a little mysterious. In fact, it is not so advanced. As we all know, analog signals are continuous values, while computers can only process discrete digital signals. When converting analog signals into digital signals, we can replace an interval with a value in the interval. For example, all values on [0, 1) are 0, [1, 2) and so on. This is a VQ process. A more formal definition is that VQ uses a finite subset of vertices in a vector space for encoding.

A typical example is image encoding. In the simplest case, consider a grayscale image. 0 is black, 1 is white, and the value of each pixel is a real number on [0, 1. Now we need to encode it into a grayscale image of level 256. The simplest way is to encode the value of each pixel.xMaps to an integer.floor(x*255). Of course, the original data space is not necessarily continuous. For example, if you want to compress the image, each pixel is only stored in 4 bits (instead of the original 8 bits). Therefore, use [0, 15] As the integer in the original [0,255] range.
Encode the integer on. A simple ing scheme isx*15/255.

VQ 2

However, such a ing scheme is quite a bit naive, although it can reduce the number of colors to play the effect of compression, but if the original color is not evenly distributed, the image quality may not be good. For example, if a 256-level grayscale image is completely composed of 0 and 13 colors, the above ing will produce a black image, because both colors are mapped to 0. A better way is to combine clustering to select representative points.

The actual solution is to treat each pixel as a data, and run K-means to obtain K centroids, then, these centroids pixel values are used to replace the pixel values of all vertices in the corresponding cluster. For a color image, the same method can be used. For example, for an RGB image, each pixel is treated as a point in a three-dimensional vector space.

Let's start with the rechard Stallman photo at the beginning of this article for an experiment. Three images, VQ 2, VQ 10, and VQ 100, show the results when the number of clusters is 2, 10, and 100, respectively, we can see that VQ 100 is very close to the source image. After many of the original color values are replaced by centroids, the total number of colors is reduced, and repeated colors are increased. This redundancy is the favorite of the compression algorithm. Consider the simplest compression method: Separate storage (for example
100) centroids color information, and then each pixel stores the centroid index instead of the color information value. If an RGB color value requires 24 bits, each (less than 128) the index value only needs 7 bits to be stored, so the compression effect is achieved.

VQ 100

VQ 10

The implementation code is very simple and directly useskmeansAndvqFunction. Python image library is used for reading and writing images:

1234567891011121314151617181920

#!/usr/bin/python from scipy.cluster.vq import kmeans, vqfrom numpy import array, reshape, zerosfrom mltk import image vqclst = [2, 10, 100, 256] data = image.read('example.jpg')(height, width, channel) = data.shape data = reshape(data, (height*width, channel))for k in vqclst: print 'Generating vq-%d...' % k (centroids, distor) = kmeans(data, k) (code, distor) = vq(data, centroids) print 'distor: %.6f' % distor.sum() im_vq = centroids[code, :] image.write('result-%d.jpg' % k, reshape(im_vq, (height, width, channel)))

Of course, Vector Quantization does not have to be implemented using k-means. All kinds of clustering methods can be used, but K-means is usually the simplest and usually enough.