Python Cookbook Learning Notes (i)

data structures and algorithms

Python provides a number of built-in data structures, including lists, collections, and dictionaries. Using these data structures in most cases is straightforward. However, we often encounter common problems such as querying, sorting, filtering, and so on. Therefore, the purpose of this chapter is to discuss these more common problems and algorithms. In addition, we will also give a way to manipulate these data structures in the collections of the collection module.

1. The decompression sequence is assigned to multiple variables

Problem

Now that there is a tuple of n elements or a sequence, how do you extract the values in it and assign them to n variables at the same time?

Solve

Any sequence can be extracted to multiple variables by a simple assignment statement, only if the number of variables needs to be the same as the number of elements in the sequence.

code example:

Discuss

In fact, this decompression assignment can be used on any iterator object, not just a list or tuple. Includes strings, file objects, iterators, and generators.

code example:

Sometimes, you might just want to unzip a part and discard other values. For this scenario, Python does not provide a special syntax. But you can use any variable name to take a place, and then throw away the variables.

code example:

It is important to ensure that the names of the selected placeholder variables are not used elsewhere.

2. Unzip an iterative object to assign values to multiple variables

Problem

If the number of elements of an iterative object exceeds the number of variables, a valueerror is thrown. So how can you extract N elements out of this iterative object?

Solve

The asterisk expression in Python can be used to solve this problem. For example, when studying a course, at the end of the semester, want to count the average performance of homework, but exclude the first and the last score. If you have only four points, you may go straight to a simple manual assignment, but what if there are 24? The asterisk expression comes in handy:

It is worth noting that the variable extracted by the asterisk expression is always a list type.

An asterisk expression can also be used at the beginning of the list. For example, there is a sequence of sales data for the first 8 months of a company, but you want to see a comparison of the last one months with the average of the previous 7 months. You can do this:

Discuss

The extended iterative decompression syntax is specifically designed to decompress an indeterminate number or an arbitrary number of elements in an iterative object. Typically, the element structure of these iterative objects has certain rules (for example, the 1th element is followed by a phone number), and the asterisk expression allows developers to easily use these rules to extract elements. Instead of getting these associated element values through some more sophisticated means. It is important to note that an asterisk expression is useful when iterating over a sequence of elements that are variable-length tuples. For example, here is a list of tuples with labels:

Sometimes you want to extract some elements and discard them, you can't simply use *, but you can use a generic deprecated name, such as _ or IGN.

code example:

3. Keep the last N elements

Problem

How do you keep a history of only the last finite number of elements when iterating or doing something else?

Solution Solutions

Keeping a limited history is the time to Collections.deque. For example, the following code makes a simple text match on multiple lines and returns the first N rows of the row that match:

Discuss

When we write the code for a query element, we typically use the generator function that contains the yield expression, which is what we have in the example code above. This allows you to decouple the search process code and use the search result code.

Using the Deque (Maxlen=n) constructor creates a new fixed-size queue. When a new element is added and the queue is full, the oldest element is automatically removed.

code example:

Although you can do this manually on a list (such as add, delete, and so on). But the queue scheme here will be more elegant and run faster. More generally, the Deque class can be used in any situation where only a simple queue data structure is needed. If you do not set the maximum queue size, you will get an infinite size queue that can be used to add and eject elements at both ends of the queue.

code example:

The time complexity of inserting or deleting elements at both ends of the queue is O (1), and the time complexity of inserting or deleting elements at the beginning of the list is O (N).

4. Find the largest or smallest N elements

Problem

How do I get the largest or smallest list of N elements from a collection?

Solution Solutions

The HEAPQ module has two functions: Nlargest () and nsmallest () can solve this problem perfectly.

Two functions can accept a keyword parameter for more complex data structures:

The above code compares each element with the value of price.

Discuss

These functions provide good performance if you want to find the smallest or largest n elements in a collection, and N is less than the number of collection elements. Because in the underlying implementation, the collection data is first sorted into a list:

The most important feature of a heap data structure is that heap[0] is always the smallest element. And the remaining elements can be easily obtained by calling the Heapq.heappop () method, which pops the first element out and then replaces the popup element with the next smallest element (the operation time Complexity is only O (log N) and N is the heap size). For example, if you want to find a minimum of 3 elements, you can do this:

Functions Nlargest () and nsmallest () are appropriate when the number of elements to find is relatively small. If you just want to find the only minimum or maximum (n=1) element, then using the min () and Max () functions will be faster. Similarly, if the size of N is close to the set size, it is usually faster to sort the collection before using the slice operation (sorted (items) [: n] or sorted (items) [-N:]). The functions Nlargest () and nsmallest () need to be used in the right places to take advantage of them (if N is close to the collection size, it would be better to use sort operations). Although you don't necessarily have to use the method here, the implementation of the heap data structure is a fun and rewarding thing to learn. Basically as long as there are data structures and algorithms in the book will be mentioned. The HEAPQ module's official documentation also provides detailed information on the implementation details of the heap data structure underlying.

5. Implementing a priority queue

Problem

How do I implement a priority-ordered queue? And on top of this queue each pop operation always returns the highest priority element?

Solution Solutions

The following class uses the HEAPQ module to implement a simple priority queue:

Here's how it's used:

A closer look reveals that the first pop () operation returns the element with the highest priority. Also note that if two elements with the same priority (Foo and Grok), the pop operation is returned in the order in which they were inserted into the queue.

Discuss

This section focuses on the use of the HEAPQ module. The function Heapq.heappush () and Heapq.heappop () respectively insert and delete the first element on queue queues, and the queue of queues guarantees that the first element has a minimum priority. The Heappop () function always returns the "smallest" element, which is the key to ensure that the queue pop operation returns the correct element. In addition, because the push and pop operation time complexity is O (log N), where n is the size of the heap, so even when N is very large, they are still running fast. In the above code, the queue contains a tuple (-priority, index, item). The goal of a negative priority is to make the element sort from highest to lowest priority. This sort of heap is the opposite of regular sort by priority from low to high. The role of the index variable is to ensure that the same priority elements are sorted correctly. By saving an ever-increasing index subscript variable, you can ensure that the elements are sorted in the order in which they are inserted. Also, the index variable plays an important role when comparing the same priority elements.

To illustrate these, it is assumed that the Item instance is not supported for sorting:

If you use a tuple (priority, item), you can compare the precedence of two elements if they are different. However, if the two elements have the same precedence, then the comparison operation will be as error as before:

By introducing another index variable into the ternary group (priority, index, item), it is possible to avoid the error above because it is impossible to have two elements with the same index value. In the case of a tuple comparison, if the previous comparison and the results can be determined, the subsequent comparison operation will not occur:

Python Cookbook Learning Notes (i)