0. Reply: ctypes. Someone who had worked overtime for several months in the past
------------------------ Labor Day supplement -----------------------------
Answer @ Yu crisp
1. ipython + ctypes: Interactive runtime environment for debugging/testing Linux APIs
Ipython is the best REPL! (I like Python, and at least 30% of my views come from ipython)
The benefits of REPL are self-evident, greatly improving efficiency in development and debugging. In particular, quick API verification is required.
I first used gdb to do some REPL tasks, but after all, operations are complicated and interactive functions are limited. Ctypes is much easier to use. ctypes can extract functions from so directly. It can be used directly at the Python layer with a little packaging, without the need to compile/connect and maintain the running status, the results are directly analyzed using Python ...... It's pretty beautiful.
Real scenarios:
Our operating environment has hundreds of thousands of Hosts. Apart from the compiling environment, one day we have doubts about the returned results of a system call. As a result, write a sample program on the development machine, compile the program, copy the program to the operating machine, run the program, and execute the process repeatedly.
With ctypes, You can debug it directly in the REPL of python/ipython. It is not easy to leave suspicious executable programs.
2. ctypes as glue
Ctypes enhances python's ability as the glue language, from process calling/unified protocol-level dehydration to binary-level dehydration. In this way, C ++'s compatibility with C seems less important.
Real scenarios:
An execution framework provided by the plug-in the form of so, so provides a fixed function entry. During the refactoring, we intend to remove language coupling and use the process call method to call the plug-in (similar to cgi server, this can reduce the restrictions on plug-in writing, and the plug-in itself is easier to test, prevent the so crash and cause the overall crash of the Framework ). However, many plug-ins have left, so you only need to add an additional framework to call the function from so to perform smooth upgrade.
3. ctypes and System Programming
As a lightweight and built-in C Language "proxy", ctypes greatly enhances python's system programming capability.
From then on, the system programming code can become more elegant.
Real scenarios:
The sdn/vpc solution requires a lot of adjustments to the kernel protocol stack. In terms of management, the network configuration is centrally controlled and distributed. Therefore, there is a daemon on the host. On the one hand, we need to accept the zookeeper configuration change notification, and on the other hand, we need to resolve the configuration and communicate with the kernel through netlink.
Almost no one in this daemon will use python. However, I can see that ctypes is used in iotop to encapsulate the netlink interface. It is amazing that python is more suitable for parsing and processing configurations. I tried to use python to implement this daemon, And the debugging was smooth. Then I reduced my overtime work for several months.
Answer me: Cheng Dian, Brother @ Wei yixiao
Cffi has not been used in any production environment. It has been used on its own computer before. It is not as simple as ctypes. Of course, the main reason why I didn't use it was that I didn't want to include too many things during deployment. What we can see on Quora is not a black technology, but it feels interesting. turtle is a built-in library.
========================================================== ====================================
Add one that was recently seen:
You need to assign values to the key nesting in the dictionary. The solution when the key does not exist:
import collectionstree = lambda: collections.defaultdict(tree)some_dict = tree()some_dict["colors"]["favourite"] = "yellow"
Use contextmanager to limit the execution timeout of a block:
with timeout(seconds=10): balabala()
Pandas... _ slots _ A website saved several GB of memory in that year. You can check this. It's actually not a black magic.
Difflib, which is an official module used to compare the similarity of strings.
(Difflib) [difflib-Compare sequences
],
In addition, common functools and collections are also attractive to Python.
The theft of many third-party databases is not an option.
In addition, doctest is also good at unit testing.
25.2. doctest
Speaking of python black magic, you must mention the underlying implementation of python's third-party coroutine gevent-greenlet.
Greenlet directly switches between different coroutines at the memory level by saving and replacing the running stack of the Python process.
This switch is transparent to the python interpreter. If the python interpreter is aware of the environment, it may feel like a person walking on the road one second before the coroutine switch, the next second suddenly appeared on the subway.
For common python users, they can directly operate the runtime stack of python. This means they dance on the tip of the knife, which requires 100% precise memory operations. Any errors may cause the python process to crash!
Then how does the author ensure correctness? In addition to familiarizing yourself with the underlying mechanisms of python, operating systems, compilers, and so on, and clarifying the design scheme, you also need to adapt to different systems and hardware environments. When we use python, we feel the same in different systems or hardware, because python itself shields us from the underlying details. When we do this kind of python underlying hack, obviously it's not that easy.
For example, there are many types of CPUs, such as i386, x86_64, and arm. Each type of CPU is designed differently, therefore, the author writes corresponding assembly operation commands for each CPU to save and replace the stack. These operations are highly bound to the operating system and hardware.
Although the implementation of greenlet is so bt, some people have done it. With the gevent encapsulation, it is much easier to use than the built-in coroutine of python.
I think any child shoes familiar with python will be curious about how it is implemented when I first came into contact with gevent. After learning more about its underlying greenlet implementation mechanism, I will not be amazed at its magical power.
This is the kind of things you may never think of without being told by others. Pip has everything, such as the itchat interface.
pip install itchat
- PEP 0302 -- New Import Hooks
