PaaS Sandbox implementation principle analysis

I,

Cloud computing is very popular, and various cloud implementation methods are also divided into many schools. However, no matter how it changes, there are some basic types, mainly divided into SaaS, PaaS, and IaaS. PaaS (similar to online development platforms such as GAE and SAES) are the first choice for developers and security personnel. However, you are not familiar with the sandbox implementation methods supported by various languages. You just need to modify the source code of the language to make the restrictions. As for how to restrict it, it is enough to modify it. It is not clear to everyone. Therefore, whether it is a sandbox breakthrough or writing your own program on PaaS, it is often around for half an hour due to some inexplicable restrictions, but I don't know why. Therefore, the purpose of this article is to introduce the seemingly mysterious content based on the author's experience and use the power as a science popularization.
All PaaS environments described in this article are built on Linux or similar systems. The Win platform is not involved for the moment (I have never seen a PaaS building with win ?)

II,

First of all, we need to clarify what is necessary for a PaaS. I have introduced this in my previous AppEngine security testing ideas. Since the cloud platform sells resources, it is very sensitive to resource usage and usage. Any computing, storage, and network resources are a good Sandbox. In addition, as a cloud platform, the code of many different programs is hosted on it. Therefore, sandbox may need to ensure that these managed codes do not harm the platform itself and other different codes. As a result, sandbox has two tasks:
1. resource restrictions
2. Border Control (or security control, whatever)

After understanding this, let's take a look at how the two need to be implemented.
PaaS builds an overall environment that includes the operating system, network environment, Webserver, and CGI script parser. Its structure is roughly as follows:

[CGI language]-> [Webserver]-> [POSIX Environment]-> [system kernel]-> [network]-> [other resources (databases, etc.)]
As you can see, this is basically a layer-by-layer. Therefore, you can control resources at any layer (until the system kernel layer. But in most cases, most PaaS sandbox restrictions are implemented at the CGI scripting language layer. There are two reasons. One is that the goal is clear. If you impose restrictions on other underlying layers, it may mistakenly hurt other processes, which is too broad. The other is that the fine-grained implementation at the language level is more comprehensive, with omissions, making positioning easier and safer. After all, it is the first layer of interaction with users.
However, if sandbox is implemented at this layer, the original api of CGI is often changed to implement resource restrictions. For example, you may need to disable some functions in php, and use python to remove some multithreading modules (in fact, there are still many ). These will greatly affect the usage habits of some users, and some open-source programs cannot be used as a result.
Therefore, many cloud providers use another sandbox Model. Most of the control is implemented on the POSIX environment and system kernel layer. At present, some cloud vendors outside China do this, such as appfog and zend cloud (both of which are deployed on Amazon's IaaS ). At this layer, learning costs are low and users do not need to change their habits. But the disadvantage is also obvious: because security control is removed from the second line to the operating system layer, the boundaries of each of his apps are expanded, and often only one virtual machine can run only one app, otherwise, they may affect each other. Therefore, the cost will be very high (one app and one virtual machine may be more)
In the second model, I prefer IaaS instead of PaaS. Therefore, this article discusses the sandbox Implementation of PaaS, and focuses on the first implementation method. This is also the method used by many cloud vendors in China.

III,

Since we know how to implement the CGI scripting language layer, how can we implement it?
This is basically a matter of understanding and familiarity with the language itself. Some languages have considered similar scenarios, such as disable_function and safe_mode of php, such as Java securityManager, such as bash's "-r. Some languages require you to cut some content yourself. For example, python. Of course, reasonable use of the two is a better way to achieve.
The implementations of these languages seem to be numerous, but there are still some traces to be found. The general script language source code is generally divided into three parts:

One is the Syntax Parsing and memory structure of the Language Ontology.

One is the language's internal underlying api

The other is the programming standard library carried by the language itself.
Take the php source code as an example. The zend directory contains the language structure and other components. The main directory contains some internal APIs, while the ext directory contains the content of the PHP standard library; in Python source code, Grammar, Parser, and Objects are Syntax Parsing and memory structures respectively. The Python directory contains internal APIs, and Modules and Lib are standard libraries written in C and Py respectively.
In general, all resource-related operations cannot exist in the language structure and Syntax Parsing section. This is concentrated in the content of the standard library. The standard library usually calls language internal APIs centrally at the underlying layer. Therefore, you only need to modify the internal api to control resource access.
However, because open-source software may be somewhat nonstandard, the standard library may read and write files and operate resources on its own. If your resources are specific, such as Access Control for mysql and memcache, the content irrelevant to these languages must be modified and controlled using the source code of the standard library.
What needs to be modified and paid attention?

1. resource restrictions: posix api calls, file read/write, network read/write, thread process creation, memory cpu usage, etc.
2. Border Control: privileged code loading, Introduction, modification, code control for shared areas, memory structure accessibility, initialization code protection, etc.
For the first point, there are different restrictions on different resources:
1. Call posix api. That is, common system calls. Generally, languages are outsourced to a layer provided in the standard library. These must be permanently disabled. For example, exec, system, pcntl _ *, pthread _ *, and dl. Disabling this content mainly prevents users from bypassing our resource control by calling these basic APIs.
2. Read and Write files. In fact, most of these calls have been removed when posix is disabled. However, the language also needs to access files. These codes are basically in the underlying internal api of the language. These cannot be banned, greatly affecting ease of use. Unexpected problems may occur. The best way to deal with this is to find a unified underlying api. Here, we will make some paths and permission restrictions to only allow read and write content under the specified path. For example, you can only load py files in the standard library and write temporary session files to the tmp directory. By default, the user's home directory prefix is added to all fopen operations.
3. network access. For network access, the most important control content is the protocol, source, and destination. That is to say, it only allows sending and accessing the tcp/http protocol (php stream needs to be castrated), does not allow forging ip addresses (rawsocket is not allowed), and the target must also be controlled, not the Intranet, it cannot be a local machine, but an external network (identify the target ip address ). You may also need to control the port range. In addition to the necessary standard library, the control scheme here is generally implemented with a unified proxy. The advantage of using a proxy is that I only need to modify the unified code for accessing the network by language and add it to the proxy by default, so that I can implement unified access control (such as Intranet Control) on the proxy ), at the same time, we can also have log records for all the content, with a high access volume. It is also convenient to expand it into a cluster.

In addition, there are a series of network resource restrictions, such as various databases, distributed systems, remote rpc, and so on. Modify the unified underlying api and send it to a proxy for distribution and logging. It is transparent, scalable, and auditable.
4. Create a process thread. There is nothing to say about it. It has been restricted in the posix part.
5. Memory cpu usage. This is actually a difficult issue. Fortunately, the linux system supports the control of a single process. For example, when each fastcgi or httpd process is started, the setrlimit series of APIs are used to perform cpu on the process during initialization, the maximum limit of memory usage. But in fact, this does not solve the impact of a single user occupying too much resources on other users when multiple users are in a single process. If you want to implement more in-depth and detailed control, you can only modify the operating system kernel. Currently, most paas do not do this layer.
To address the second point, you can simply put it in one sentence: Pay attention to the control of shared code or process space content. No one is allowed to make modifications.
For example, you can modify the initialization code or php configuration during each request initialization. For example, you can modify the shared library code that paas will be executed by each user by default, for example, you can load a c module or dll file into the current process space, and bypass all language-level restrictions to execute privileged code. All of this requires control at the source code or system level.
Is this enough? Basically, sandbox is similar, but in fact there is not much control. One level of control may be able to withstand today, but it is never guaranteed to be able to withstand tomorrow. PaaS security is built on a more basic underlying system. Therefore, system security and cluster network access control must be improved gradually.

References: SAE architecture ppt, BAE source code, Cloudfoundry source code
________________________________________
Gainover (">_< '/& \ Have you ever seen any cross-site characters?) |
Wikipedia checked and made a note.
Infrastructure as a service (IaaS)
Platform as a service (PaaS)
Software as a service (SaaS)
Storage as a service (STaaS)
Security as a service (SECaaS)
Data as a service (DaaS)
Test environment as a service (TEaaS)
Desktop as a service (DaaS)
API as a service (APIaaS)
________________________________________
From: http://zone.wooyun.org/content/1012