How Small Businesses Build Their Own Free Cloud WAF

Overview

WEB attack is more than a decade of hacking the mainstream technology, the domestic manufacturers have long regarded WAF as a standard security infrastructure, the market there are many security vendors to provide WAF products or cloud WAF services.

For the lack of their own security team, but also suffer from sql injection, xss, cc and other WEB attacks in the small and medium enterprises, the demand for WAF is also very urgent.

The ways to get WAF are as follows:

Buy security vendors WAF products

Use the cloud waf service, set their own domain name DNS server provided by the cloud waf manufacturers, or will need to access the domain name cwt cloud waf in the past

Or from the Internet to find some free or open source waf use

Homemade WAF

There is nothing wrong with using paid products or services for well-paid companies, but some companies do not plan to use paid products or services due to budget, data privacy (cloud waf captures the content of all traffic requests and responses).

This case can only use the free waf, or according to business needs for a self-made cloud WAF.

I will elaborate on how to use a week's time to make a simple and easy to use cloud WAF, the following is completed WAF documentation and github address:

Project site: https://waf.xsec.io/

Github Address: https://github.com/xsec-lab

Cloud WAF Architecture Design

Physical architecture

The WAF also has different architectures based on business scenarios or needs, such as:

Integration into local web containers as modules, such as mod_security, Naxsi

Reverse proxy mode

Hardware Products WAF

Agent + detect cloud mode

The cloud WAF implemented in this paper uses an architecture of reverse proxy mode

waf can deploy one or more servers, if the business is larger, a waf performance has been unable to meet the business needs, can be used in front of waf LVS, haproxy, nginx build load balancing, VIP front-end request distribution To the back of the waf

Back-end app server to provide normal business web server, the user's request will be filtered through waf first, if it is a malicious attack request, it will block in the waf level, if it is a normal request will be forwarded to the back-end server

Logical architecture

x-waf by x-waf itself and the web management background x-waf-admin, of which:

x-waf based on openresty + lua development

waf management background: the use of golang + xorm + macrom developed to support the binary form of deployment

x-waf implementation

I spent two companies have independently developed cloud waf, the framework designed from the beginning designed for large-scale business systems, installation, deployment, operation and maintenance are more complex, inconvenient rapid deployment of small businesses, so in reference to github In the existing open source several waf, redesigned a lightweight.

x-waf implementation process

openresty does not execute the lua script by default and needs to be configured in nginx.conf as follows:

# Specify lua file search path

lua_package_path "/usr/local/openresty/nginx/conf/x-waf/?.lua;/usr/local/lib/lua/?.lua;;";

# Define two lua shared dict variables were limit and badGuys, the allocation of memory size 100M

lua_shared_dict limit 100m; lua_shared_dict badGuys 100m;

# Enable Lua code cache function

lua_code_cache on;

# Let nginx execute the lua code in the init.lua file during the init phase

init_by_lua_file /usr/local/openresty/nginx/conf/x-waf/init.lua;

# Let nginx execute the lua code in the access.lua file for each http request's access phase

access_by_lua_file /usr/local/openresty/nginx/conf/x-waf/access.lua;

openresty init stage will be based on the location specified in the configuration file into json format rules to the global lua table, different rules on different tables to speed up the regular matching speed

waf = require ("waf") waf_rules = waf.load_rules ()

waf.load_rules will load and read all the json format rules according to the path specified in the configuration file and load it into a different table and then encapsulate a get_rule function so that each http entry can be obtained directly from the lua table Type of rule:

local _M = {RULES = {}}

function _M.load_rules () _M.RULES = util.get_rules (config.config_rule_dir)

return _M.RULES end

function _M.get_rule (rule_file_name) ngx.log (ngx.DEBUG, rule_file_name)

return _M.RULES [rule_file_name] end

util.get_rules will save the rules in the specified file to the lua table by rule name for the waf.get_rule function to get the rules when needed:

function _M.get_rules (rules_path)

local rule_files = _M.get_rule_files (rules_path)

if rule_files == {} then return nil end

for rule_name, rule_file in pairs (rule_files) do local t_rule = {}

local file_rule_name = io.open (rule_file)

local json_rules = file_rule_name: read ("* a") file_rule_name: close ()

local table_rules = cjson.decode (json_rules)

if table_rules ~ = nil then

for _, table_name in pairs (table_rules) do table.insert (t_rule, table_name ["RuleItem"]) end end _M.RULE_TABLE [rule_name] = t_rule end

return (_M.RULE_TABLE) end

Waf will filter ip whitelist, ip blacklist, user_agent, cc-attack, url whitelist, url blacklist, cc-attack, cookies, get and post parameters in the order in which each request comes in. Any one will be the appropriate treatment (output prompt or jump), then will not continue to determine whether other types of attacks.

function _M.check ()

if_M.white_ip_check () then elseif _M.black_ip_check () then elseif _M.user_agent_attack_check () then elseif _M.white_url_check () then elseif _M.url_attack_check () then elseif _M.cc_attack_check () then elseif _M.cookie_attack_check () then elseif _M.url_args_attack_check () then elseif _M.post_attack_check () then else return end

end

Judgment for each parameter type of each request is to first get the content of the parameter, and then re-match with the regular rule of such parameters, if matching is considered as an attack request, the following is a function of filtering the post parameter :

- deny post function _M.post_attack_check ()

if config.config_post_check == "on" then ngx.req.read_body () local POST_RULES = _M.get_rule ('post.rule')

for _, rule in pairs (POST_RULES) do local POST_ARGS = ngx.req.get_post_args () or {}

for_, v in pairs (POST_ARGS) do local post_data = "" if type (v) == "table" then post_data = table.concat (v, ",")

else post_data = v

end if rule ~ = "" and rulematch (post_data, rule, "jo") then util.log_record ('Deny_USER_POST_DATA', post_data, "-", rule)

if config.config_waf_enable == "on" then util.waf_output ()

return true end end end end return false

end

waf management background x-waf-admin implementation

Waf rules is based on the string in the JSON format, manually maintain the capacity of error, the other waf waf will have more than one waf work, if the man-made waf back-end host management, synchronization rules and host configuration synchronization operation and maintenance Work, it is very easy to make mistakes or omissions, it is necessary to provide an automated management, synchronization configuration management background.

waf management background functional requirements

Convenient deployment, just before starting to do a simple configuration, the first start, x-waf-admin will generate a default administrator in mysql and the default waf rules;

User management, support administrator account increase, change, delete;

waf rules management, support waf rules increase, change, delete, and policy synchronization to all waf server functions;

Back-end site management, support for access waf site increase, change, delete, and separate or synchronous full synchronization back-end site functions.

Program structure

In order to facilitate the deployment, x-waf-admin did not use python, php and other need to build a runtime environment or rely on third-party package language, but with go directly compiled into executable file language, the specific technical stack go Language + macron + xorm.

The project structure is as follows:

hartnett at hartnett-notebook in / data / code / golang / src / xsec-waf / x-waf-admin (master ●) $ tree -L 2 ├── conf │ └── app.ini ├── models │ ├ - models.go │ ├── rules.go │ ├── site.go │ └── user.go ├── modules │ └── util ├── public │ ├── css ├── README.md ├── routers │ ├── admin.go │ ├── index.go │ ├── rules.go │ ├── site.go │ └── user.go ├── server ├── server.go ├ ── setting │ └── setting.go └── templates

conf for the configuration file directory

models directory orm file

modules are function module components

public and templates are static resources and template files are located in the directory

routers directory for the routing files

setting directory for the configuration file processing files

server.go for the program entrance

Implementation of rules management functions

User management, back-end site management and rules management functions to achieve similar, are similar to flask, martini, tornado, django and other MTV WEB framework applications, in order to reduce space, this article only write back-end site management functions to achieve a complete code See github.

ORM implementation of backend site management

First define the site's struct with xorm, and then provide methods for adding, changing, deleting, viewing, etc. These methods are called by the site file in the router module:

// Due to space is too long, omit part of the code, the detailed code, see github

debuglevel: debug, info, notice, warn, error, crit, alert, emerg

// ssl: on, off

type Site struct {Id int64 SiteName string `xorm:" unique "` Port int BackendAddr [] string Ssl string `xorm:" varchar (10) notnull default 'off' "DebugLevel string` xorm: "varchar (10) notnull default 'error' "` LastChange time.Time `xorm:" updated "` Version int `xorm:" version "` // Optimistic locking

}

func ListSite () (sites [] Site, err error) {sites = make ([] Site, 0) err = Engine.Find (& sites)

log.Println (err, sites)

return sites, err}

func NewSite (siteName string, Port int, BackendAddr [] string, SSL string, DebugLevel string) (err error) {

if SSL == "" {SSL = "off"}

if DebugLevel == "" {DebugLevel = "error"} _, err = Engine.Insert (& Site {SiteName: siteName, Port: Port, BackendAddr: BackendAddr, Ssl: SSL, DebugLevel: DebugLevel})

return err}

Back-end site management routing

First import the appropriate package, and then write the following processor:

Increase the site get and post request processor (NewSite, DoNewSite)

Modify the site's get and post request processor (EditSite, DoEditSite)

According to ID delete site get processor (DelSite)

Sync Site Configuration Processor (SyncSite)

The processor of the API of the synchronization site configuration and the processor of the API configured according to the ID synchronization site (SyncSiteApi, SyncSiteById)

// Due to space is too long, omit part of the code, the detailed code, see github

func NewSite (ctx * macaron.Context, sess session.Store, x csrf.CSRF) {if sess.Get ("uid")! = "" {ctx.Data ["csrf_token"] = x.GetToken () ctx. HTML (200, "newSite")} else {ctx.Redirect ("/ login /")}}

func DoNewSite (ctx * macaron.Context, sess session.Store) {

if sess.Get ("uid")! = nil {

log.Println (sess.Get ("uid")) siteName: = ctx.Req.Form.Get ("sitename") port: = ctx.Req.Form.Get ("port") Port, _: = strconv. Backendaddr: = strings.Split (backaddr, "\ r \ n") BackendAddr: = make ([] string, 0) Atoi (port) backaddr: = ctx.Req.Form.Get ("backendaddr"

for _, v: = range backendaddr {

if v == "" {

continue} v = strings.TrimSpace (v) BackendAddr = append (BackendAddr, v)} ssl: = ctx.Req.Form.Get ("ssl") debugLevel: = ctx.Req.Form.Get ("debuglevel")

log.Println (siteName, BackendAddr, ssl, debugLevel) models.NewSite (siteName, Port, BackendAddr, ssl, debugLevel) ctx.Redirect ("/ admin / site / list /")} else {ctx.Redirect ("/ login / ")}}

model initialization

We must note that, although the use of mysql, but did not require manual import into the table or insert initialized sql script before use, this is for God?

Because we use the ORM, ORM will help us to automatically do the above operation, as shown in the following code:

// Due to space is too long, omit part of the code, the detailed code, see github

var (Engine * xorm.Engine err error)

func init () {

// Obtain the database configuration information from conf / app.ini sec: = setting.Cfg.Section ("database")

// Connect to the database Engine, err = xorm.NewEngine ("mysql", fmt.Sprintf ("% s:% s @ tcp (% s) /% s? Charset = utf8", sec.Key ("USER"). String (), sec.Key ("PASSWD"). String (), sec.Key ("HOST"). String (), sec.Key ("NAME"). String

if err! = nil {

log.Panicf ("Faild to connect to database, err:% v", err)}

// Create new site, user, and rules tables Engine.Sync2 (new (Site)) Engine.Sync2 (new (User)) Engine.Sync2 (new (Rules))

// If the user table is empty, create a new default account ret, err: = Engine.IsTableEmpty (new (User))

if err == nil && ret {

log.Printf ("create new user:% v, password:% v \ n", "admin", "x@xsec.io") NewUser ("admin", "x@xsec.io")}

// If the rule is empty, insert the default initialization rules ret, err = Engine.IsTableEmpty (new (Rules))

if err == nil && ret {

log.Println ("Insert default waf rules") Engine.Exec (DefaultRules)}}

Configure the route

When the ORM, routing-related code can be written in the program after the import route, the URL and the routing controller corresponding to the following:

// Due to space is too long, omit part of the code, the detailed code, see github

m.Group ("/ admin", func () {

m.Get ("/ index /", routers.Admin)

m.Group ("/ site /", func () {

m.Get ("", routers.Admin)

m.Get ("/ list /", routers.Admin)

m.Get ("/ new /", routers.NewSite)

m.Post ("/ new /", csrf.Validate, routers.DoNewSite)

m.Get ("/ edit /: id", routers.EditSite)

m.Post ("/ edit /: id", csrf.Validate, routers.DoEditSite)

m.Get ("/ del /: id", routers.DelSite)

m.Get ("/ sync /", routers.SyncSite)

m.Get ("/ sync /: id", routers.SyncSiteById)

m.Get ("/ json /", routers.SiteJSON)

})})

m.Group ("/ api", func () {

m.Get ("/ site / sync /", routers.SyncSiteApi)

m.Get ("/ rule / sync /", routers.SyncRuleApi)})

log.Printf ("xsec waf admin% s", setting.AppVer)

log.Printf ("Run mode% s", strings.Title (macaron.Env))

log.Printf ("Server is running on% s", fmt.Sprintf ("0.0.0.0:%v", setting.HTTPPort))

log.Println (http.ListenAndServe (fmt.Sprintf ("0.0.0.0:%v", setting.HTTPPort), m))

Interaction issues

Once upon a time there was a mountain, there was a temple in the mountain, there was a gray hat Xiaoming student in the temple, and he once got a website rebound shell through some unspeakable means. Although it is a root privilege, the method of utilization is not stable.

At this point Xiao Ming found that the server's network card running on a root 权限 redis, but added a password, I saw Xiao Ming tiger body startled, suddenly with ideas: Leave a webshell, later through the webshell to perform redis rebound shell exp.

But when he finished reading nginx configuration and then a tight chrysanthemum, because this site only ran lua web application:

# Omit part of the configuration

http {include mime.types; default_type application / octet-stream; lua_package_path "/data0/www/abcom/?.lua/?.lua;/usr/local/lib/lua/?.lua ;;"; lua_code_cache off; init_by_lua_file /data0/www/abcom/init.lua; sendfile on; keepalive_timeout 65; server {listen 80; server_name abcom; location / api / {content_by_lua_file /data0/www/abcom/web.lua;}}}

Please listen to questions

▼▼▼

Seven trees on the tree, a monkey on the ground, how to stay a lua webshell Xia, please paste ideas and code.