Numpy
Numpy是Python科學計算的核心庫。它提供了高效能多維陣列對象,以及使用這些數組的工具。如果你已經熟悉MATLAB,你可以找到這個教程來開始使用Numpy。
Arrays
一個numpy的數組(array)是一個由相同類型數值構成的網路(grid),並且被非負整數的元組索引。維數是數組的rank;而數組的shape是一個整數元組,它給出了數組每一維度大小。
我們可以使用嵌套的Python lists初始化numpy數組,並使用方括弧來訪問元素。
import numpy as npa = np.array([1, 2, 3]) # Create a rank 1 arrayprint type(a) # Prints "<type 'numpy.ndarray'>"print a.shape # Prints "(3,)"print a[0], a[1], a[2] # Prints "1 2 3"a[0] = 5 # Change an element of the arrayprint a # Prints "[5, 2, 3]"b = np.array([[1,2,3],[4,5,6]]) # Create a rank 2 arrayprint b.shape # Prints "(2, 3)"print b[0, 0], b[0, 1], b[1, 0] # Prints "1 2 4"
Numpy也提供了很多函數來建立數組。
import numpy as npa = np.zeros((2,2)) # Create an array of all zerosprint a # Prints "[[ 0. 0.] # [ 0. 0.]]" b = np.ones((1,2)) # Create an array of all onesprint b # Prints "[[ 1. 1.]]"c = np.full((2,2), 7) # Create a constant arrayprint c # Prints "[[ 7. 7.] # [ 7. 7.]]"d = np.eye(2) # Create a 2x2 identity matrixprint d # Prints "[[ 1. 0.] # [ 0. 1.]]" e = np.random.random((2,2)) # Create an array filled with random valuesprint e # Might print "[[ 0.91940167 0.08143941] # [ 0.68744134 0.87236687]]"
你可以在官方文檔中找到更多的數組建立的方法。
Array indexing
Numpy提供了一些方法來索引數組。
Slicing:與Python lists類似,numpy 數組可以被切分。數組可能是多維的,你必須為每一維度確定一個切分。
import numpy as np# Create the following rank 2 array with shape (3, 4)# [[ 1 2 3 4]# [ 5 6 7 8]# [ 9 10 11 12]]a = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])# Use slicing to pull out the subarray consisting of the first 2 rows# and columns 1 and 2; b is the following array of shape (2, 2):# [[2 3]# [6 7]]b = a[:2, 1:3]# A slice of an array is a view into the same data, so modifying it# will modify the original array.print a[0, 1] # Prints "2"b[0, 0] = 77 # b[0, 0] is the same piece of data as a[0, 1]print a[0, 1] # Prints "77"
你可以混合使用整數索引和切分索引。然而,這樣做會產生一個低階的數組。這與MATLAB中運算元組切分的方式是完全不同的。
import numpy as np# Create the following rank 2 array with shape (3, 4)# [[ 1 2 3 4]# [ 5 6 7 8]# [ 9 10 11 12]]a = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])# Two ways of accessing the data in the middle row of the array.# Mixing integer indexing with slices yields an array of lower rank,# while using only slices yields an array of the same rank as the# original array:row_r1 = a[1, :] # Rank 1 view of the second row of a row_r2 = a[1:2, :] # Rank 2 view of the second row of aprint row_r1, row_r1.shape # Prints "[5 6 7 8] (4,)"print row_r2, row_r2.shape # Prints "[[5 6 7 8]] (1, 4)"# We can make the same distinction when accessing columns of an array:col_r1 = a[:, 1]col_r2 = a[:, 1:2]print col_r1, col_r1.shape # Prints "[ 2 6 10] (3,)"print col_r2, col_r2.shape # Prints "[[ 2] # [ 6] # [10]] (3, 1)"
Integerarray indexing: 當使用slicing索引到numpy 數組內部的時候,所得的數組是原始數組的子數組。相反,整數形式的數組索引允許你使用其他數組的資料來建立任意數組。樣本如下:
import numpy as npa = np.array([[1,2], [3, 4], [5, 6]])# An example of integer array indexing.# The returned array will have shape (3,) and print a[[0, 1, 2], [0, 1, 0]] # Prints "[1 4 5]"# The above example of integer array indexing is equivalent to this:print np.array([a[0, 0], a[1, 1], a[2, 0]]) # Prints "[1 4 5]"# When using integer array indexing, you can reuse the same# element from the source array:print a[[0, 0], [1, 1]] # Prints "[2 2]"# Equivalent to the previous integer array indexing exampleprint np.array([a[0, 1], a[0, 1]]) # Prints "[2 2]"
關於整數索引的一個有用的技巧是選擇或者改變來自矩陣每行的一個元素。
import numpy as np# Create a new array from which we will select elementsa = np.array([[1,2,3], [4,5,6], [7,8,9], [10, 11, 12]])print a # prints "array([[ 1, 2, 3], # [ 4, 5, 6], # [ 7, 8, 9], # [10, 11, 12]])"# Create an array of indicesb = np.array([0, 2, 0, 1])# Select one element from each row of a using the indices in bprint a[np.arange(4), b] # Prints "[ 1 6 7 11]"# Mutate one element from each row of a using the indices in ba[np.arange(4), b] += 10print a # prints "array([[11, 2, 3], # [ 4, 5, 16], # [17, 8, 9], # [10, 21, 12]])
Booleanarray indexing: 布爾型數組索引允許你挑選出數組的任意元素。這種索引方式用於選擇數組中滿足特定條件的元素。樣本如下:
import numpy as npa = np.array([[1,2], [3, 4], [5, 6]])bool_idx = (a > 2) # Find the elements of a that are bigger than 2; # this returns a numpy array of Booleans of the same # shape as a, where each slot of bool_idx tells # whether that element of a is > 2. print bool_idx # Prints "[[False False] # [ True True] # [ True True]]"# We use boolean array indexing to construct a rank 1 array# consisting of the elements of a corresponding to the True values# of bool_idxprint a[bool_idx] # Prints "[3 4 5 6]"# We can do all of the above in a single concise statement:print a[a > 2] # Prints "[3 4 5 6]"
為了簡潔,關於numpy數組索引的很多細節沒有介紹;如果想瞭解更多,可以閱讀官方文檔。
Datatypes
每一個numpy數組是一個由相同類型元素組成的網路。Numpy提供大量的數實值型別,你可以用它們來構建數組。在你建立數組時候,Numpy嘗試猜測資料類型,但是建立數組的函數通常會包含一個可選的參數來確定資料類型。這裡有一個樣本:
import numpy as npx = np.array([1, 2]) # Let numpy choose the datatypeprint x.dtype # Prints "int64"x = np.array([1.0, 2.0]) # Let numpy choose the datatypeprint x.dtype # Prints "float64"x = np.array([1, 2], dtype=np.int64) # Force a particular datatypeprint x.dtype # Prints "int64"
你也可以在官方文檔中閱讀所有的numpy資料類型。
Array math
基本的數學函數在數組上進行元素級操作,在運算子多載和numpy模組的函數中都是可用的:
import numpy as npx = np.array([[1,2],[3,4]], dtype=np.float64)y = np.array([[5,6],[7,8]], dtype=np.float64)# Elementwise sum; both produce the array# [[ 6.0 8.0]# [10.0 12.0]]print x + yprint np.add(x, y)# Elementwise difference; both produce the array# [[-4.0 -4.0]# [-4.0 -4.0]]print x - yprint np.subtract(x, y)# Elementwise product; both produce the array# [[ 5.0 12.0]# [21.0 32.0]]print x * yprint np.multiply(x, y)# Elementwise division; both produce the array# [[ 0.2 0.33333333]# [ 0.42857143 0.5 ]]print x / yprint np.divide(x, y)# Elementwise square root; produces the array# [[ 1. 1.41421356]# [ 1.73205081 2. ]]print np.sqrt(x)
注意,與MATLAB中不同,“*”表示元素乘法,不是矩陣乘法。我們使用dot函數來計算向量內積,向量與矩陣相乘,已經矩陣乘法。dot 既可以作為numpy模組中的一個函數,也可以作為數組對象的一個樣本方法:
import numpy as npx = np.array([[1,2],[3,4]])y = np.array([[5,6],[7,8]])v = np.array([9,10])w = np.array([11, 12])# Inner product of vectors; both produce 219print v.dot(w)print np.dot(v, w)# Matrix / vector product; both produce the rank 1 array [29 67]print x.dot(v)print np.dot(x, v)# Matrix / matrix product; both produce the rank 2 array# [[19 22]# [43 50]]print x.dot(y)print np.dot(x, y)
Numpy提供了很多有用的函數來在數組上做運算;最有用的一個是sum:
import numpy as npx = np.array([[1,2],[3,4]])print np.sum(x) # Compute sum of all elements; prints "10"print np.sum(x, axis=0) # Compute sum of each column; prints "[4 6]"print np.sum(x, axis=1) # Compute sum of each row; prints "[3 7]"
你可以在官方文檔中找到numpy提供的數學函數的列表。
除了在數學函數中使用數組,我們經常還會需要對數組中的資料進行操作或者重構。這種操作最簡單的例子就是轉置一個矩陣;要轉置一個矩陣,可以使用數組對象的T屬性。
import numpy as npx = np.array([[1,2], [3,4]])print x # Prints "[[1 2] # [3 4]]"print x.T # Prints "[[1 3] # [2 4]]"# Note that taking the transpose of a rank 1 array does nothing:v = np.array([1,2,3])print v # Prints "[1 2 3]"print v.T # Prints "[1 2 3]"
Numpy提供了非常多的運算元組的函數;你可以在官方文檔中找到它們。
Broadcasting
廣播是一個強大的機制,它允許numpy在進行算數操作的時候可以使用不同的形狀的數組。經常會有這種情況,我們有一個小數組和一個大數組,我們相用小數組多次在大數組上進行操作。
例如,設想我們要往矩陣的每一行上加一個常數向量。我們可以這樣做:
import numpy as np# We will add the vector v to each row of the matrix x,# storing the result in the matrix yx = np.array([[1,2,3], [4,5,6], [7,8,9], [10, 11, 12]])v = np.array([1, 0, 1])y = np.empty_like(x) # Create an empty matrix with the same shape as x# Add the vector v to each row of the matrix x with an explicit loopfor i in range(4): y[i, :] = x[i, :] + v# Now y is the following# [[ 2 2 4]# [ 5 5 7]# [ 8 8 10]# [11 11 13]]print y
這是可以正常工作的;但是當矩陣x非常大的時候,計算一個顯式迴圈在Python中是非常慢的。須知,將一個向量v加到矩陣x的每一行上,等效於通過垂直儲存v的多個副本來構建一個矩陣vv,然後對x和vv執行元素級求和。我們可以這樣執行這個方法:
import numpy as np# We will add the vector v to each row of the matrix x,# storing the result in the matrix yx = np.array([[1,2,3], [4,5,6], [7,8,9], [10, 11, 12]])v = np.array([1, 0, 1])vv = np.tile(v, (4, 1)) # Stack 4 copies of v on top of each otherprint vv # Prints "[[1 0 1] # [1 0 1] # [1 0 1] # [1 0 1]]"y = x + vv # Add x and vv elementwiseprint y # Prints "[[ 2 2 4 # [ 5 5 7] # [ 8 8 10] # [11 11 13]]"
Numpy廣播允許我們執行這個操作,但是不必真得建立v的多個副本。考慮使用廣播的版本:
import numpy as np# We will add the vector v to each row of the matrix x,# storing the result in the matrix yx = np.array([[1,2,3], [4,5,6], [7,8,9], [10, 11, 12]])v = np.array([1, 0, 1])y = x + v # Add v to each row of x using broadcastingprint y # Prints "[[ 2 2 4] # [ 5 5 7] # [ 8 8 10] # [11 11 13]]"
即使x的shape是(4,3),而v的shape是(3,),y = x + v 仍然可以工作,這多虧了廣播;這一行工作時好像v的shape是(4,3),每一行是v的一個副本,求和是以元素層級執行的。
一起廣播兩個數組遵循這些規則:
(1) 如果數組沒有相同的階數,預先考慮低階數組,直到兩個shapes有相同的長度。
(2) 如果他們在一個維度上具有相同的尺寸,或者其中一個在該維度上的size是1,就說它們是相容的。
(3)如果它們在所有的維度上是相容的,就可以一起廣播。
(4) 廣播後,每一個數組的shape表現得像是與兩個數組的shape中最大那個元素相同。
(5) 在任何一個維度,如果一個數組的size是1,而另一個大於1,則第一個數組表現得像是在該維度上進行了拷貝。
如果這個解釋不起作用,可以閱讀官方文檔或者這裡的解釋。
支援廣播的函數成為通用函數。你可以在官方文檔中找到通用函數的列表。 這裡是關於廣播的一些應用:
import numpy as np# Compute outer product of vectorsv = np.array([1,2,3]) # v has shape (3,)w = np.array([4,5]) # w has shape (2,)# To compute an outer product, we first reshape v to be a column# vector of shape (3, 1); we can then broadcast it against w to yield# an output of shape (3, 2), which is the outer product of v and w:# [[ 4 5]# [ 8 10]# [12 15]]print np.reshape(v, (3, 1)) * w# Add a vector to each row of a matrixx = np.array([[1,2,3], [4,5,6]])# x has shape (2, 3) and v has shape (3,) so they broadcast to (2, 3),# giving the following matrix:# [[2 4 6]# [5 7 9]]print x + v# Add a vector to each column of a matrix# x has shape (2, 3) and w has shape (2,).# If we transpose x then it has shape (3, 2) and can be broadcast# against w to yield a result of shape (3, 2); transposing this result# yields the final result of shape (2, 3) which is the matrix x with# the vector w added to each column. Gives the following matrix:# [[ 5 6 7]# [ 9 10 11]]print (x.T + w).T# Another solution is to reshape w to be a row vector of shape (2, 1);# we can then broadcast it directly against x to produce the same# output.print x + np.reshape(w, (2, 1))# Multiply a matrix by a constant:# x has shape (2, 3). Numpy treats scalars as arrays of shape ();# these can be broadcast together to shape (2, 3), producing the# following array:# [[ 2 4 6]# [ 8 10 12]]print x * 2
廣播能夠使代碼更加簡潔和快速,你應該在可能的地方盡量使用它。
Numpy Documentation
這個簡要的概述涉及了很多你應該瞭解的關於numpy的重要內容,但是遠遠不夠完整。查看Numpy索引來找到更多的關於Numpy的內容。
Scipy
Numpy提供了一個高效能多維度數組和計算、操作這些數組的基本工具。SciPy在此基礎上構建而成,提供大量的在numpy數組上操作的函數,可以用於不同類型的科學和工程應用。
最好的熟悉SciPy的方式是瀏覽文檔。我們會重點強調一些對本課程有用的部分。
Image operations SciPy提供了一些基本的函數來處理映像。例如,把映像從硬碟讀入numpy數組,將numpy數組以映像格式寫到硬碟,重新設定映像的大小。樣本如下:
from scipy.misc import imread, imsave, imresize# Read an JPEG image into a numpy arrayimg = imread('assets/cat.jpg')print img.dtype, img.shape # Prints "uint8 (400, 248, 3)"# We can tint the image by scaling each of the color channels# by a different scalar constant. The image has shape (400, 248, 3);# we multiply it by the array [1, 0.95, 0.9] of shape (3,);# numpy broadcasting means that this leaves the red channel unchanged,# and multiplies the green and blue channels by 0.95 and 0.9# respectively.img_tinted = img * [1, 0.95, 0.9]# Resize the tinted image to be 300 by 300 pixels.img_tinted = imresize(img_tinted, (300, 300))# Write the tinted image back to diskimsave('assets/cat_tinted.jpg', img_tinted)
---------------------------------------------------------------------------------------
Left: The original image. Right: The tintedand resized image.
---------------------------------------------------------------------------------------
MATLAB files
函數scipy.io.loadmat 和 scipy.io.savemat 允許你讀寫MATLAB檔案,你可以從官方文檔中瞭解到它們。
Distance between points
SciPy定義了一些有用的函數來計算點之間的距離。
函數scipy.spatial.distance.pdist 計算給定集合中所有點對的距離。
import numpy as npfrom scipy.spatial.distance import pdist, squareform# Create the following array where each row is a point in 2D space:# [[0 1]# [1 0]# [2 0]]x = np.array([[0, 1], [1, 0], [2, 0]])print x# Compute the Euclidean distance between all rows of x.# d[i, j] is the Euclidean distance between x[i, :] and x[j, :],# and d is the following array:# [[ 0. 1.41421356 2.23606798]# [ 1.41421356 0. 1. ]# [ 2.23606798 1. 0. ]]d = squareform(pdist(x, 'euclidean'))print d
你可以從官網文檔中擷取該函數更多的細節。
一個類似的函數(scipy.spatial.distance.cdist)計算兩個集合中所有點對之間的距離;你可以在這裡閱讀它們。
Matplotlib
Matplotlib是一個繪圖庫。這個部分主要介紹matplotlib.pyplot模組,它提供了一個與MATLAB類型的繪圖系統。
Plotting
matplotlib中最重要的函數是plot,它讓我們可以繪製2D資料。這有一個簡單的例子:
import numpy as npimport matplotlib.pyplot as plt# Compute the x and y coordinates for points on a sine curvex = np.arange(0, 3 * np.pi, 0.1)y = np.sin(x)# Plot the points using matplotlibplt.plot(x, y)plt.show() # You must call plt.show() to make graphics appear.
運行代碼可以獲得下面的圖形。
-------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------
做一點額外的工作,我們就能一次繪製多條線,添加標題、圖例和座標軸標籤。
import numpy as npimport matplotlib.pyplot as plt# Compute the x and y coordinates for points on sine and cosine curvesx = np.arange(0, 3 * np.pi, 0.1)y_sin = np.sin(x)y_cos = np.cos(x)# Plot the points using matplotlibplt.plot(x, y_sin)plt.plot(x, y_cos)plt.xlabel('x axis label')plt.ylabel('y axis label')plt.title('Sine and Cosine')plt.legend(['Sine', 'Cosine'])plt.show()
-------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------
你可以在官方文檔中閱讀到更多的關於plot的內容。
Subplots
可以在同一個圖中繪製不同的東西,需要使用subplot函數。這裡是一個例子:
import numpy as npimport matplotlib.pyplot as plt# Compute the x and y coordinates for points on sine and cosine curvesx = np.arange(0, 3 * np.pi, 0.1)y_sin = np.sin(x)y_cos = np.cos(x)# Set up a subplot grid that has height 2 and width 1,# and set the first such subplot as active.plt.subplot(2, 1, 1)# Make the first plotplt.plot(x, y_sin)plt.title('Sine')# Set the second subplot as active, and make the second plot.plt.subplot(2, 1, 2)plt.plot(x, y_cos)plt.title('Cosine')# Show the figure.plt.show()
-------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------
同樣,官方文檔提供了更多的關於subplot的內容。
Images
你可以使用imshow函數來展示圖片。樣本如下:
import numpy as npfrom scipy.misc import imread, imresizeimport matplotlib.pyplot as pltimg = imread('assets/cat.jpg')img_tinted = img * [1, 0.95, 0.9]# Show the original imageplt.subplot(1, 2, 1)plt.imshow(img)# Show the tinted imageplt.subplot(1, 2, 2)# A slight gotcha with imshow is that it might give strange results# if presented with data that is not uint8. To work around this, we# explicitly cast the image to uint8 before displaying it.plt.imshow(np.uint8(img_tinted))plt.show()
-------------------------------------------------------------------------------------------
-------------------------------------------------------------------------------------------