Use Python to create your own Shell:part I

Use Python to create your own Shell:part I

[TOC]

Original link and description

1. https://hackercollider.com/articles/2016/07/05/create-your-own-shell-in-python-part-1/
2. This translation document is originally selected from Linux China , the translation of documents copyright belongs to linux China all

I'd like to know how a shell (like bash,csh, etc.) works inside. To satisfy my curiosity, I used Python to implement a Shell named Yosh (Your Own Shell). The concepts described in this article can also be applied to other programming languages.

(Hint: You can find the source code used by Ben Boven here, and the code is published as an MIT license.) On Mac OS X 10.11.5, I tested it with Python 2.7.10 and 3.4.3. It should be able to run in other Unix-like environments, such as Linux and Cygwin on Windows. ）

Let's get started.

Step 0: Project structure

For this project, I used the following project structure.

yosh_project|-- yosh   |-- __init__.py   |-- shell.py

yosh_projectAs the project root directory (you can also name it simply yosh ).

yoshAs a package directory and __init__.py can make it a package with the same name as the package directory (if you don't write Python, you can ignore it.) ）

shell.pyIs our main script file.

Step 1:shell Cycle

When you start a shell, it displays a command prompt and waits for your command input. After receiving the input command and executing it (a later article will explain it in detail), your shell will go back to the loop and wait for the next instruction.

In shell.py , we will start with a simple Mian function that calls the Shell_loop () function, as follows:

def shell_loop():    # Start the loop heredef main():    shell_loop()if__name__=="__main__":    main()

Then, in shell_loop() order to indicate whether the loop continues or stops, we use a status flag. At the beginning of the loop, our shell will display a command prompt and wait for the command input to be read.

import=1=0def shell_loop():    = SHELL_STATUS_RUN    while== SHELL_STATUS_RUN:        # Display a command prompt        sys.stdout.write('> ')        sys.stdout.flush()        # Read command input        = sys.stdin.readline()

After that, we slice the command input and execute it (we're about to implement 命令切分 and 执行 function).

Therefore, our shell_loop () will be as follows:

import  sysshell_status_ RUN =  1  shell_status_stop =  0  def  shell_loop (): Status =  shell_status_run while<        /span> status ==  shell_status_run: # Display a command prompt  Sys.stdout.write ( > ' ) sys.stdout.flush () # Read command input  cmd =  sys.stdin.readline () # tokenize the command input cmd_tokens =  tokenize (cmd) # Execute the command and retrieve N EW status  status =  execute (cmd_tokens)  

This is our entire shell loop. If we use python shell.py launch our shell, it will display a command prompt. However, if we enter the command and press ENTER, it throws an error because we haven't defined the 命令切分 function yet.

In order to exit the shell, you can try entering ctrl-c. I'll explain later how to exit the shell in an elegant form.

Step 2: Command slicing

When the user enters a command in our shell and presses the ENTER key, the command will be a long string containing the command name and its arguments. Therefore, we must slice the string (splitting a string into multiple tokens).

It seems very simple at first glance. We might be able to use cmd.split() a space to split the input. It works on similar ls -a my_folder commands because it splits the commands into a list [‘ls‘, ‘-a‘, ‘my_folder‘] so that we can handle them easily.

However, there are echo "Hello World" echo ‘Hello World‘ cases where arguments are referred to or quoted in single or double quotes. If we use cmd.spilt, we will get a list of 3 tokens instead of [‘echo‘, ‘"Hello‘, ‘World"‘] 2 [‘echo‘, ‘Hello World‘] .

Fortunately, Python provides a library of names that shlex can help us to partition commands like God. (Tip: We can also use regular expressions, but it's not the focus of this article.) ）

import sysimport shlex...def tokenize(string):    return shlex.split(string)...

We then send these tokens to the execution process.

Step 3: Perform

This is the core and interesting part of the shell. What exactly happened when the shell was executed mkdir test_dir ? (Hint: mkdir is an test_dir executor with parameters that creates a test_dir directory named.) ）

execvpis the first function that involves this step. Before we explain what we've execvp done, let's look at its actual effect.

import os...def execute(cmd_tokens):    # Execute command    os.execvp(cmd_tokens[0], cmd_tokens)    # Return status indicating to wait for next command in shell_loop    return SHELL_STATUS_RUN...

Try running our shell again, enter the mkdir test_dir command, and then press ENTER.

After we hit the ENTER key, the problem is that our shell will exit directly instead of waiting for the next command. However, the goal is correctly created.

So execvp what has actually been done?

execvpIs exec a variant of the system call. The first parameter is the name of the program. vindicates that the second parameter is a list of program parameters (variable parameters). pindicates that the environment variable PATH will be used to search for a given program name. In our last attempt, it will find the program based on our PATH environment variables mkdir .

(There are other exec variants, such as EXECV, EXECVPE, Execl, EXECLP, execlpe; You can get more information from Google.) ）

execReplaces the current memory of the calling process with the new process that is about to run. In our example, our shell process memory is replaced with a mkdir program. Next, mkdir become the master process and create the test_dir directory. Finally, the process exits.

The point here is that our shell process has been mkdir replaced by the process. This is why our shell disappears and does not wait for the next command.

Therefore, we need other system calls to solve the problem: fork .

forkwill open up new memory and copy the current process to a new process. We call this new process a child process , the caller process as the parent process . The child process memory is then replaced with the program being executed. As a result, our shell, the parent process, can be protected from memory substitution.

Let's look at the modified code.

...defExecute (cmd_tokens):# Fork A child shell process    # IF The current process was a child process, it's ' PID ' is set to ' 0 '    # Else the current process is a parent process and the value of ' PID '    # is the process ID of IT child process.Pid=Os.fork ()ifPid== 0:# Child Process        # Replace The child shell process with the program called with execOS.EXECVP (cmd_tokens[0], Cmd_tokens)elifPid> 0:# Parent Process         while True:# Wait Response status from it child process (identified with PID)Wpid, status=Os.waitpid (PID,0)# Finish Waiting if it child process exits normally            # or is terminated by a signal            ifOs. wifexited (status)orOs. Wifsignaled (status): Break    # Return status indicating to wait for next command in Shell_loop    returnShell_status_run ...

When our parent process is called os.fork() , you can imagine that all the source code is copied to the new subprocess. At this moment, the parent and child processes see the same code and run in parallel.

If the running code belongs to a child process, the pid 0 . Otherwise, if the running code belongs to the parent process, it pid will be the process ID of the child process.

When os.execvp called in a child process, you can imagine that all the source code of the child process is replaced with the code of the program being called. However, the code of the parent process is not changed.

When the parent process finishes waiting for the child process to exit or terminate, it returns a status indicating that the shell loop continues.

Run

Now, you can try running our shell and typing mkdir test_dir2 . It should be able to execute correctly. Our main shell process still exists and waits for the next command. Try to execute ls and you can see the directory you have created.

However, there are still many problems here.

First, try to execute cd test_dir2 , then execute ls . It should go into an empty test_dir2 directory. However, you will see that the directory has not changed test_dir2 .

Second, we still have no way to gracefully exit our shell.

We will solve such problems in part 2.

via:https://hackercollider.com/articles/2016/07/05/create-your-own-shell-in-python-part-1/

Use Python to create your own Shell:part I