MongoDB build a picture server for Rails projects

1 Installation and Configuration Nginx-gridfs

Our request to access the picture first passes through the Nginx and then through the Nginx-gridfs to MongoDB, so configuring Nginx-gridfs is an important step.

If the server does not have Nginx installed, we refer to the https://github.com/mdirolf/nginx-gridfs#installation steps to install, if the Nginx has been installed, but not installed Nginx-gridfs module, then we can install only the Nginx-gridfs module.

1.1 Installing Nginx and Nginx-gridfs

For simplicity's sake, let's assume that the Nginx is installed for the first time and is installed through source code.

1, download Nginx source code, and decompression

wget http://nginx.org/download/nginx-1.10.0.tar.gz
TAR-XVF nginx-1.10.0.tar.gz
2, clone Nginx-gridfs to local

git clone git@github.com:mdirolf/nginx-gridfs.git
CD Nginx-gridfs
git checkout v0.8
Git submodule init
git submodule update
3, Build

Cd/path/to/nginx-source
./configure--add-module=/path/to/nginx-gridfs/source/
Make
Make install
1.2 Configuration Nginx-gridfs

This step is actually in the Nginx conf file to add Gridfs related content.

The official configuration document on the Nginx-gridfs is too simple to be suitable for production environments. Our next configuration will add two servers in the Nginx conf file, 1th server will listen for 4444 ports, the server will interact directly with MongoDB, and the 2nd server will listen for 5555 ports, this serv The ER adds a configuration such as a picture cache, and forwards the request to the 1th server after the cache expires, with the following configuration:

$ cat nginx.conf

Worker_processes 1;

Events {
Worker_connections 1024;
}

HTTP {
Default_type Application/octet-stream;
Sendfile on;

Keepalive_timeout 65;

Include Mime.types;

Proxy_temp_path/data/nginx_cache/nginx_temp;
Proxy_cache_path/data/nginx_cache/nginx_cache levels=1:2 keys_zone=cache_one:4000m inactive=2d max_size=10g;

server{
Listen 4444;
Location/{
Gridfs imgdb root_collection=fs field=filename type=string;
MONGO 127.0.0.1:27017;
}
}
Upstream My_server_pool {
Server 127.0.0.1:4444 weight=1;
}
server {
Listen 5555;
location/upload/{
Proxy_cache Cache_one;
Proxy_cache_valid 304 2d;
Proxy_cache_valid 302 1m;
Proxy_cache_valid any 1s;
Proxy_cache_key $host $uri$is_args$args;
Proxy_set_header Host $host;
Proxy_set_header x-forwarded-for $remote _addr;
Proxy_pass http://my_server_pool/;
Add_header X-cache hit-lt;
Expires Max;
}
}
}
So we're done with the Nginx-gridfs configuration.

Here you should pay attention to the/DATA/NGINX_CACHE/NGINX_TEMP access rights issue, if the Nginx error log appears similar to failed (13:permission denied) while reading Upstrea M error, you can use the Chown command to modify the owner of/data/nginx_cache/nginx_temp.

We analyze the details of the 4444 ports, which are the configurations that interact with the Mongo Gridfs:

server{
Listen 4444;
Location/{
Gridfs imgdb root_collection=fs field=filename type=string;
MONGO 127.0.0.1:27017;
}
}
Gridfs:nginx identify the name of the plug-in;
IMGDB: Database name;
Root_collection: Choose collection, such as Root_collection=images, Mongod will go to find Images.files, the default is FS;
Field: Query fields, support _id, filename, can be omitted, the default is _id, where we use filename is the name of the picture to find pictures;
Type: Interpret field data type, support Objectid, int, string, omit, default is int, if we use filename in field, then we need to set to string;
User: Username, can be omitted;
Pass: password, can be omitted;
The host and Port of the MONGO:MONGODB;
2 start Nginx and MongoDB, and test the normal access to the picture

Start Nginx:

/usr/local/nginx/sbin/nginx-c/usr/local/nginx/conf/nginx.conf
The MongoDB configuration file is as follows:

port=27017
dbpath=/data/db
Logpath=/var/mongodb/log/mongodb.log
Logappend=true
Fork=true
Start MongoDB:

Mongod-f/path/to/mongodb.conf
Some permissions and directories do not exist when Nginx and MongoDB are started, and can be resolved using Chown and mkdir-p. The configuration steps above are operated on the Mac OS.

Test upload Picture:

--host localhost--port 27017--db mongofiles imgdb put--local=/path/to/dog.jpg
You can access the uploaded pictures through the browser:

3 integration with Rails projects

3.1 Demo Project Introduction

When I wrote this demo, I was going to use the code from my company's project to do the upload, but as the rails version used by the company was low, the associated gem was a bit of a match for the newer rails version (the rails version of this demo is 4.2.6). Many problems, and the related configuration is too cumbersome and complex, I decided to use only MongoDB's official Ruby driver to achieve the image upload function.

The following gems are used in the company's projects to achieve picture uploads:

# Gemfile
Gem ' Carrierwave '
Gem ' carrierwave-mongoid '
Gem ' mongoid '
Using only the official Ruby Driver:mongo of MongoDB,

# Gemfile
Gem ' Mongo ', ' ~> 2.2 '
In this demo, the picture uploads the business process can be simple to say in a word clearly:

Use to generate a unique name for the picture, and then submit the picture to the picture server, and if the picture server saves the picture successfully, store the picture name in the database for application to take pictures from the picture server in the future.

3.2 Demo Design

Of course in the actual application also involves picture compression, cut, transform, add watermark business logic, but these business logic should be and picture upload isolation should not be coupled together, from this point of view, I hate carrierwave and the like upload the image of the library to upload, cut, The transformation and so on work all put into a Uploader to realize.

As a programmer, there are two benefits to translating your ideas into intuitive graphics:

Easy to write your own code

Convenient for others to write code

So we still draw a picture of the seemingly simple function of uploading images:

Through the analysis of the above diagram, we can 4 steps to achieve the image upload function:

Application, we need to generate image params, such as image name, image entity and other parameters submitted to the image Server client;

Picture Server client side, we need to write server configuration information, and connect to the server;

Picture Server client side, we need to submit picture parameters to the server;

Application, we need to deal with the image server response, if the response is successful, we need to record the name of the picture and other parameters;

3.3 Demo Implementation

3.3.1 Image Server Client configuration and initialization

First Write configuration file:

# CONFIG/MONGO.YML

Development
Host:localhost
port:27017
Database:imgdb

Test
Host:localhost
port:27017
Database:imgdb

Staging
Host:localhost
port:27017
Database:imgdb

Production
Host:localhost
port:27017
Database:imgdb
Initialize client:

# CONFIG/INITIALIZERS/MONGO.RB

MC = Yaml.load_file (Rails.root.join (' config ', ' mongo.yml ')) [rails.env]
Db_url = "mongodb://#{mc[' host ']}/#{mc[' database '}"
$mongo = Mongo::client.new (Db_url)
So we have a client that can be accessed globally: $mongo

3.3.2 Write picture upload service

Image upload Service logic is very simple: accept a file parameter, return filename and Content_Type

# APP/SERVICES/UPLOAD_FILE_SERVICE.RB

Class Uploadfileservice

Def initialize (file)
@file = File
End

def call
Grid = $mongo. Database.fs
Grid.upload_from_stream (filename, @file)
res = {
Filename:filename,
Content_Type: @file. Content_Type
}
End

Private

def filename
return @filename if @filename. Present?
ext = File.extname (@file. Original_filename)
@filename = "#{securerandom.uuid}#{ext}"
End

End
Using Uploadfileservice is also very simple, you can refer to the relevant controller code:

# APP/CONTROLLERS/AVATARS_CONTROLLER.RB

Class Avatarscontroller < Applicationcontroller

DEF create
@avatar = Avatar.new
FILE = Params[:avatar][:attachment_file_name]

# Call image upload service upload picture to image server
# and store the name of the picture in the database
res = uploadfileservice.new (file). Call
@avatar. Attachment_file_name = Res[:filename]
@avatar. Attachment_content_type = Res[:content_type]
If @avatar. Save
Redirect_to action: ' Index '
Else
Render:new
End
End

End