Code Automation Packaging System "original"

I. Background of demand

XX project needs to develop a set of front-end component packaging system to handle user requests:

1. Users apply for application through the platform;

2. Select the required components;

3. The relevant front-end files of the components such as js,CSS,HTML extraction, merging, compression, packaging;

4. The package link address is returned to the user, and the user downloads the compressed package.

For the above requirements, we choose a more popular front-end Code packaging tool grunt.

ii. introduction of Grunt

What is grunt?

The official website gives it an explanation of the JavaScript Task Runner. Grunt is a project building tool based on node. js that allows tasks such as compressing, compiling, and unit testing of project files to be automatically executed with grunt commands through Gruntfile configuration, saving most of the tedious work and time.

Why Choose Grunt?

Because Grunt has a wealth of plug-ins, can meet the needs of packaging the merger, compression, zip package and other functions.

third, the system operating environment

The operation of the packaging system requires support for the following environment, which requires a one by one installation before development.

Node:JavaScript Run Environment (runtime). It actually encapsulates the Google V8 engine.

NPM: Full name Node Package Manager, is a Nodejs pack management and distribution tool

PM2: Process Manager for node app with load balancer

Dnode: Implements the communication between PHP and node, providing two-way remote method call class library

Grunt and its plugins :Grunt-contrib-clean,grunt-contrib-concat,grunt-contrib-copy,grunt-contrib-cssmin ,grunt-contrib-less,grunt-contrib-uglify,grunt-contrib-watch,grunt-zip, Load-grunt-tasks,Gurnt relies on these plugins to complete the code merge, compression, zip package and other functions

Four, packaging system implementation

Let's take a look at the flowchart

Process is a number of modified scenarios, during which many problems are encountered, mainly focused on the call between PHP and grunt and parameter delivery. Issues that need to be addressed:

1. How to extract the corresponding files from the source code files according to the selected components;

2. How to invoke the Grunt command in a PHP program

3. Dnode can be used as a bridge for PHP to invoke the grunt command,how PHP calls Grunt commands synchronously

1. gruntfile File Usage

The user first chooses the component to download the compressed package, passing the grunt command to gruntfile the list of components that need to be packaged,Gruntfile contains all the processing logic of grunt.

Loading the Grunt plugin

To load the required plugins, write them in the Package.json file:

   "Devdependencies": { "~0.4.0",  "Grunt-contrib-clean":  "^0.6.0",  "Grunt-contrib-cssmin":  "^0.6.1",  " Grunt-contrib-uglify ": " ^0.9.1 ", " Grunt-contrib-watch ":  "^0.3.1",  "^1.0.0",  "Load-grunt-tasks":  "Grunt-zip":      

Source file configuration file:

Here is the configuration file for the loading component,HTML and some general JS,css (less) written directly in the gruntfile file

{  "app": "loading",  "less": ["src/app/loading/loading.less"],  "js": ["src/vendor/common/pxloader.js", "src/app/loading/loading.js"]}

Gruntfile parameter Receive:

var Apparr = grunt.option (' app '). Split (', '); (The app is acomponent parameter string, which is a combination of component names and names separated by commas) For example:' Register,login,slider '.

Gruntfile reads the component configuration file based on the list of components obtained.

for (var i in appArr) {    var confName = ‘grunt_conf/‘ + appArr[i] + ‘.json‘; confArr[i] = grunt.file.readJSON(confName);};

Gruntfile gets the component name according to the parameter app, then gets the component configuration file according to the component name, through the configuration file, you can get the file list of the source file, then combine, merge, compress the source files of these components, and finally generate the compressed package

2. php call grunt

Because the production environment PHP.ini disable_functions the exec,shell_exec, andsystem functions that can execute Linux commands are disabled, but Node can be called,Dnode can implement the communication between PHP and node. This allows us to implement the PHP program to invoke the Grunt command.

Node Packaging Server

var PORT = 7083;var dnode = require(‘dnode‘);var cp = require(‘child_process‘);var server = dnode({ pack: function (params, callback) { var ls = cp.exec("grunt  --app=" + params.coms, [], function(error, stdout, stderr){ if(error != null || stdout.indexOf(‘without errors‘) < 0){ callback(‘error‘); }else{ callback(‘success‘); } }); } });server.listen(PORT);

PHP Synchronous Call

Why synchronous calls instead of async? Because after packing to upload to the compressed package, before uploading must ensure that the compression package exists, so we use the callback function, and can be based on the return value of the callback function to determine whether the package is normal, but also to ensure that the next upload normal.

/** * Source File Package *@param string $components *@param string $path *@param string $fileName *@throws \h5eexception * *PrivateStaticfunctionSourcepack($components) {$loop =New \react\eventloop\streamselectloop ();$dnode =New \dnode\dnode ($loop);$port =7083;Self::$params =Array ' coms ' =  $components);  $dnode->connect ( $port, function ( $remote ,  $connection) { $remote->pack (packservice::  $params, function  $ret) use  ( $connection) { span class= "keyword" >if ( "success"! =  $ret) { Throw new \h5eexception ( $connection->end ();}); });  $loop->run ();}            

3. Maintain node Packaging Server's continuous operation

Node starts the package server process, and after a while it will somehow hang up,PM2 as node daemon is a good solution to this problem.

4. Compress package Upload

The package is based on the version of the plan, when the version changes, the user download the package will need to repackage, but the same version of the source file is not necessary to re-hit the package, so we upload the package to the resource server, and then save the resource link in the database, the next time just from the database query to the link can be, This eliminates the need to repeat packaging, which improves the speed of user downloads and saves server resources.

Five, late improvement

Add node log, currently missing node log, if there is an exception, it is difficult to locate the problem;

Node Packaging Server to receive the parameters of strict validation;

Six, the security question ponder

A Control the frequency of access, compared with ordinary data interface, the packaging interface takes a long time, consumes more server resources, if the interface is malicious frequent request, may affect the server performance, and cause the normal packaging failure, it is necessary to limit the frequency of access;

B In the PHP layer and node layer are to carry out strict calibration parameters, can effectively prevent the problem caused by the parameters of the accident;

C Node Code Packaging Server runs a port that is not external and prevents users from accessing the port directly through the extranet.

Code Automation Packaging System "original"