Use Python to create your own Shell (top)

I'd like to know how a shell (like bash,csh, etc.) works inside. So to satisfy my curiosity, I used Python to implement a shell called Yosh (Your Own Shell). The concepts described in this article can also be applied to other programming languages.

(Hint: You can find the source code used by Ben Boven here, and the code is published as an MIT license.) On Mac OS X 10.11.5, I tested it with Python 2.7.10 and 3.4.3. It should be able to run in other Unix-like environments, such as Linux and Cygwin on Windows. Let's get started.

Step 0: Project Structure

For this project, I used the following project structure.

Yosh_project |--Yosh     |--__init__.py     

For the project root directory (you can also name it simply as Yosh). Yosh is the package directory, and __init__.py can make it a package with the same name as the package's directory (you can ignore it if you don't write it in Python). Shell.py is our main script file.

Step 1:shell Cycle

When you start a shell, it displays a command prompt and waits for your command input. After you have received the input command and executed it (a later article will explain it in detail), your shell will be back here again and wait for the next instruction. In shell.py, we start with a simple main function that calls the Shell_loop () function, as follows:

Def shell_loop ():   # Start The loop here def Main ():   shell_loop () if __name__ = = "__main__":  

Then, in Shell_loop (), we use a status flag in order to indicate whether the loop continues or stops. At the beginning of the loop, our shell will display a command prompt and wait for the command input to be read.

Import Sysshell_status_run = 1shell_status_stop = 0def shell_loop ():    status = Shell_status_run while    status = = SH Ell_status_run:        # # shows command prompt        sys.stdout.write (' > ')        sys.stdout.flush ()        # # # Read command input        cmd = Sys.stdin.readline ()

After that, we slice the command (tokenize) and execute it (execute) (we are about to implement the Tokenize and execute functions). Therefore, our shell_loop () will be as follows:

Import Sysshell_status_run = 1shell_status_stop = 0def shell_loop ():    status = Shell_status_run while    status = = Shell_status_run:        # # shows command prompt        sys.stdout.write (' > ')        sys.stdout.flush ()        # # # Read command input        cmd = Sys.stdin.readline ()        # # # Shard Command input        cmd_tokens = tokenize (cmd)        # # # Execute the command and get the new state status        = Execute (Cmd_tokens)

This is our entire shell loop. If we launch our shell using Python shell.py, it will display a command prompt. However, if we enter the command and press ENTER, it throws an error because we haven't defined the Tokenize function yet. In order to exit the shell, you can try entering ctrl-c. I'll explain later how to exit the shell in an elegant form.

Step 2: command slicing

() When the user enters a command in our shell and presses the ENTER key, the command will be a long string containing the command name and its arguments. Therefore, we must slice the string (splitting a string into multiple tuples). It seems very simple at first glance. We might be able to use Cmd.split () to split the input with a space. It works on commands like Ls-a My_folder because it splits commands into a list [' ls ', '-a ', ' my_folder '] so that we can handle them easily.

However, there are cases where echo "Hello World" or echo ' Hello World ' refers to arguments in single or double quotes. If we use cmd.spilt, we will get a list of 3 tags [' echo ', ' ' Hello ', ' world '] instead of the list of 2 tags [' echo ', ' Hello World ']. Fortunately, Python provides a library called Shlex, which helps us to split commands magically. (Tip: We can also use regular expressions, but it's not the focus of this article.) ）

Import Sysimport shlex...def tokenize (String):    return Shlex.split (String) ...

We then send these tuples to the execution process.

Step 3: Perform

This is a central and interesting part of the shell. What happens when the shell executes the mkdir test_dir? (Hint: mkdir is an executor with the Test_dir parameter that creates a directory named Test_dir. EXECVP is the first function required for this step. Before we explain what EXECVP is doing, let's look at its actual effect.

Import Os...def Execute (cmd_tokens):    # # # Execute command    OS.EXECVP (cmd_tokens[0], Cmd_tokens)    # # # Return status to tell in Shell_ Loop waiting for the next command    return Shell_status_run ...

Try running our shell again, enter the mkdir test_dir command, and then press ENTER. After we hit the ENTER key, the problem is that our shell will exit directly instead of waiting for the next command. However, the catalog was created correctly. So, what did EXECVP actually do?

EXECVP is a variant of the system call exec. The first parameter is the name of the program. V indicates that the second parameter is a list of program parameters (variable number of parameters). P indicates that the environment variable PATH will be used to search for the given program name. In our last attempt, it will look for the MKDIR program based on our PATH environment variable. (There are other exec variants, such as EXECV, EXECVPE, Execl, EXECLP, execlpe; You can get more information from Google.) Exec replaces the current memory of the calling process with the new process that is about to run. In our case, our shell process memory is replaced with the mkdir program. Next, mkdir becomes the master process and creates the Test_dir directory. Finally, the process exits.

The point here is that our shell process has been replaced by the mkdir process. This is why our shell disappears and does not wait for the next command. Therefore, we need other system calls to solve the problem: fork. Fork allocates new memory and copies the current process to a new process. We call this new process a child process, the caller process as the parent process. The child process memory is then replaced with the program being executed. As a result, our shell, the parent process, can be protected from memory substitution.
Let's look at the modified code. ...

When our parent process calls Os.fork (), you can imagine that all of the source code is copied to the new sub-process. At this moment, the parent and child processes see the same code and run in parallel. If the code you are running belongs to a child process, the PID will be 0. Otherwise, if the running code belongs to the parent process, the PID will be the process ID of the child process.

When OS.EXECVP is called in a child process, you can imagine that all the source code of the child process is replaced with the code of the program being called. However, the code of the parent process is not changed. When the parent process finishes waiting for the child process to exit or terminate, it returns a status indicating that the shell loop continues.

Run now, you can try to run our shell and enter mkdir Test_dir2. It should be able to execute correctly. Our main shell process still exists and waits for the next command. Try to execute LS, and you can see the directory you have created.

However, there are still some questions:
First, try to execute the CD TEST_DIR2 and then execute the LS. It should go into an empty Test_dir2 directory. However, you will see that the directory has not changed to TEST_DIR2.
Second, we still have no way to gracefully exit our shell.

This article reprinted address: http://www.linuxprobe.com/python-shell-first.html

Free to provide the latest Linux technology tutorials Books, for open-source technology enthusiasts to do more and better: http://www.linuxprobe.com/

Use Python to create your own Shell (top)