MySQL Database request

In the previous period, a large department of the new set up a promotion of Baidu OCR, word recognition, image recognition and other scientific and technological capabilities in the financial sector of the application of the sub-sector. Because the departments have just been established, the infrastructure and manpower are lacking. At that time, the task of our department was to deploy a person to do the front-end development work of the new Department master station. Originally said is only responsible for the page development work. When I participated in the requirements of the quality audit, I learned that the new Department of Manpower shortage, and my main task this year is to do in our department based on the front end of the nodejs separation of the architecture upgrade work.

Before that, I wrote two internal systems with Nodejs and had no major online web development experience and wanted to take advantage of this opportunity to exercise. Then he took the initiative to discuss the matter with the boss, the boss is very supportive. And then with the new Department of product consultation, the people who are already short of hands they are also very happy that I undertake more development tasks.

This article and the style of their previous article will be very different, will not write some specific technical points and problems encountered specific solutions, the main talk is my entire development process encountered some problems and think about how to solve their methods.

Table of Contents

Thinking of technology selection
Related Service Request
Front-end Engineering
Technical goals
Web Security
Intranet Machine Access External network
Send mail
Network optimization
Harvest
Thinking of technology selection

In the article at the beginning of the introduction of the background is probably said that the site backend using Nodejs development. In order to highlight the scientific and technological capabilities, the website requires some special effects. Because I want to use CSS3 to write these special effects, with the product PK after the result is browser compatibility is IE8.0 above, special effects to meet most mainstream browser can. Then the other technologies based on Nodejs are as follows:

The following are the reasons for selecting these technologies:

YOG2 (click to reach Homepage)) is a relatively mature nodejs Web framework based on express developed by Baidu Company. The ability to provide the company's internal basic services (with the computer room access, operations, logs, etc.), and some departments have been on-line large-scale use, if encountered problems can have a lot of experience can learn from.
Swig is the template engine supported by the YOG2 default.
Because Nodejs's syntax is followed by the CMD specification. And in Baidu's internal promotion is also the use of FIS3 and mod.js. So I chose the company's internal FIS3 (click to reach Home page) and Mod.js (click to reach the homepage). FIS3 not only Baidu's own internal use, there are a lot of external companies in use, such as drip and so on.
Because the interaction requirements are compatible with some low-version browsers and some wonderful homemade browsers. In order to ensure the development of the time to control the choice of their own familiar jquery;
Application of related services

After determining the technology selection is to start the application services, mainly including the following related content:

Domain Name Application
Server request
MySQL Database request
BOS Storage Service (using the BOS storage of the Baidu Cloud)
The above is the company's internal process of walking, the specific is not introduced. Mainly introduces the role of some services. A website running on the network must need a domain name, can make the website run up is definitely need the online server. The database is required to store the user's registration data. Because face recognition is performed using OCR, it is satisfying to recognize multiple faces on a single image. Is the need to crop the user's picture. Because the site is deployed on more than one machine, it must not be stored on the server running on the site needs to be trimmed pictures stored on a dedicated storage server, and return to the site image link,

Front-end Engineering

With the development of Nodejs, the concept of the front-end engineering may be more extensive and will involve NODEJS related engineering. This part is introduced in two parts:

1. Front End

Goal:
-Use SASS to do the modular management of CSS, and compile it into CSS in real time, generate map files for local debugging;
-Package programming of components written with the CMD specification and modular code to be referenced by the business code of the page;
-Automatically add vendor prefixes to CSS attributes that require a vendor prefix;
-Ability to deploy code to test environments in real time to facilitate QA testing;

All of these goals can be achieved using FIS3 and related plugins.

2. Back end

Because the Nodejs version of our online mass use is version 6.x. However, asynchronous and await are used in the development process to handle asynchrony. So it is necessary to compile these ES7 syntax into 6.x supported syntax with the help of the compilation engine.

In addition, the Process.env.NODE_ENV can be used to read the characteristics of environment variables, to distinguish between configuration of some on-line and offline configuration, such as:

Const YOG_DEBUG = Process.env.YOG_DEBUG;
Const PANSHI_DEBUG = Process.env.PANSHI_DEBUG;

Let mysqlconf;

if (panshi_debug = = = ' true ') {
mysqlconf = {
Host: ' 10.00.00.00 ',
User: ' Ppui ',
Password: ' Ppui ',
Database: ' Excel ',
Port: ' 5003 '
};
} else if (yog_debug = = = ' true ') {
mysqlconf = {
Host: ' 127.0.0.1 ',
User: ' Root ',
Password: ",
Database: ' Pass_panshi ',
Port: ' 3306 '

Technical goals

Here we talk about the technical goals of the front end

1. Separation of style display and DOM operations

The situation I often encountered during the development process was that I needed the style of the HTML node and accidentally changed the class name. And JS is precisely using this class to manipulate the DOM. This time the page will be running when the error, increase the maintenance cost of the project.

There are two ways to solve this problem effectively, the first is to add a custom attribute, such as <div class= "section" node-type= "Pagesecond" ></div> When I need to manipulate the DOM, I use jquery's property selector to manipulate the DOM instead of using class. This will not affect the JS code when I adjust the style and need to change the class name. The second is based on what you often say about using-to do the HTML class name of the connector, and we have a specification is to use an underscore (_) to mark I want to manipulate the name of the DOM node, such as <div class= "section _pagesecond" ></ Div>.

Both of these ways, if it is in the development of multi-person maintenance projects are all need to pre-book specifications, I am using the former in the project.

2. Separation of business code and function code

As described earlier, the use of the CMD specification to organize front-end code. For example, to be able to satisfy my need to use the property selector as an operational dom. I have specifically encapsulated some code snippets, such as a piece of code in the Base.js file:

/**
* Get node information according to Node-type
*
* @param {any} params get node element
* @param {any} context environment
* @returns
*/
Exports.nodetypedom = function (params, context) {
If (context && context!== ') {
return $ (' [node-type= ' + params + '] ', $ (' [node-type= ' + context + '] '));
} else {
return $ (' [node-type= ' + params + '] ');

I need to use this snippet in other files, just like this.

var Basejs = require ('.. /libjs/base ');
var node = basejs.nodetypedom;

Where you need to select the DOM, pass in the value of the custom attribute directly

Node (' Pagesecond '). xxxx

In addition to some common code snippets such as encapsulation, some components are encapsulated in this way. For example: Carousel diagram component, File Upload component, form Check Component, tab scrolling component.

The advantages of the above two methods can greatly improve the maintainability of the code, reading.

Web Security

The Web security I'm focusing on in the development process is primarily

SQL injection
Interface attack
1. Preventing SQL injection

SQL injection is simply referring to some illegal users splicing a special user name or password, because we want to insert the user name and password into the database, will certainly be based on this user name and password stitching an SQL statement. This special user name statement of the offending user is likely to erase all data from our database.

Because the MySQL database is used. The Nodejs module is also the most used MySQL module on NPM. itself this module has provided the ability to access MySQL cluster and anti-injection capabilities.

Specific methods can refer to the official document click here to direct

2. Guard against interface attacks

What we have to do here is some illegal users to get our interface, write a cycle of frequent access to our interface. In order to prevent some illegal users is to add token to the request. is to return a token to the front end when a request is made to the server, with the token on the backend of the front-end request. If token is passed on the back-end check, the token is destroyed. There are, for example, the authentication request of the source IP, it is important to note that when we verify the IP should be obtained is the HTTP protocol header field of the X-forwarded-for property value. (These two methods can be used together)

But later from the back End Rd learned that the company has specialized services can be used to do anti-cheating, and more comprehensive strategy. Currently in the study ready for access.

Intranet Machine Access External network

On the cross-room access, with the computer room access and intranet access to the network, these will basically involve the topic of operation and maintenance. Baidu has a ready-made service access document inside. Different companies may offer their capabilities in different ways. Not much to introduce here.

Here's a little bit of detail. Let's look at one of the following pictures:

Bottom line: When a request arrives at the access layer, it is not known to access the server in the intranet environment of the computer room. On the opposite intranet, if there is a link to the extranet, for example: http://weibo.com. An extranet server needs to be accessed through a proxy.

Access interface I use the request module. With promise NPM There are request-promise by name and we know his every method or what is returned by the call result. This module already provides the configuration of the proxy parameters by default. Please refer to the documentation for more information.

There are more knowledge involved, such as proxy tunneling, HTTPS request proxy. Search for some keywords while reading the official configuration document to learn some other relevant knowledge.

If there are related requirements, you can refer to my configuration, if my configuration does not solve your problem, please read the official document HA. 、

Let options = {
' URL ': Params.url,
' Encoding ': ' binary ',
' rejectunauthorized ': false//Cancel verification of HTTPS certificate
};

To resolve the behavior of proxy HTTPS requests the test machine needs to configure environment variables Panshi_https_proxy
if (Process.env.PANSHI_DEBUG!== ' true ' | | Panshi_https_proxy) {
Options.tunnel = false;
Options.proxy = ' http://www.huazongyule.com ';

Send mail

The introduction of development is complete here. Here are some of the functional development of operations and product requirements. Each day the registered user is sent to the appropriate responsible person.

A mail server is required if this feature is to be met. This is common within the company and can be easily found. The other is to configure the service crontab timed Execution script query database send mail.

The Nodejs module Nodemailer is mainly used here. Specific configuration and methods of sending mail can refer to the official document configuration click to direct

Network optimization

Static file CDN deployment;
Merge static files;
cache static files;
Icon using Base64
The above list is a few more typical points. For example, like CSS put head tag header, the script tag is placed at the bottom of the body tag. These should belong to a front-end engineer's common sense.

Static file Deployment CDN This is not much introduction, each company will own a set of methods. Here is the main introduction to merge static files and cache static files.

1. Merging static files

The default FIS3 is a plug-in that supports merging static files. Because I developed more pages this time (a total of 11 master pages), and because of the use of block development loading modules and static files. If you do not merge, a page loads a request that requires 10-20 static files. Affects how quickly the page loads.

When I am ready to use FIS3 plug-ins to merge static files, it is still a bit cumbersome to find a page to configure the static files to package the merge. Finally asked the other Department of colleagues to use our Access layer server provided by the comb function, by the server to help us merge static files (in fact, Nginx concat module provides the functionality). Here also do not do too much introduction, self-search articles to understand on it.

2. Caching static files

Let's take a look at the next picture.

The red box is the field of the HTTP protocol associated with the static file cache. If the concept of these fields is blurred, you can read this article to deepen the impression of the "HTTP cache" click Direct

Whether using express or KOA (KOA can use Koa-static-cache middleware), the ability to configure these fields is provided by the middleware of the corresponding static file service. Express can be configured in a way (specific can read the Express document)

Const EXPRESS = www.fanku1.com require (' Express ')

Configure parameters related to static files
Express.static (' xxxxx ')

Harvest

The last is to talk about the harvest of this development

This project development on-line, just to the big department of the annual summary meeting, because of their own independent responsible for the development of the front and back, won the large department of "Shining Star" award and some material rewards (although not seen the shadow

MySQL Database request