This is a creation in Article, where the information may have evolved or changed.
Optimize configuration structure and implement image upload
Project Address: Https://github.com/EDDYCJY/go ...
If it helps, you're welcome to a Star.
Objective
One day, the product manager suddenly told you the article list, no cover map, not beautiful enough! ) &¥! &) #&¥! Add one, for a few minutes.
You open your program and analyze a wave to write a list:
- Optimize the configuration structure (because of the increasing number of configuration items)
- Extracting the File from the original logging is convenient for public (logging, upload each one is not appropriate)
- Implement upload Image interface (need to limit file format, size)
- Modify the article interface (need to support the cover page address parameter)
- Add database fields for Blog_article (article)
- Implement HTTP. Fileserver
Well, if you find it better, you need to tweak some of the application structure, because there are more functions and the original design is going to keep pace.
That is, in due time, optimize
Optimize configuration structure
First, explain
In the previous chapters, the use of direct read key to store configuration items, and the need to increase the picture configuration items, the overall is a bit redundant
We use the following workaround:
- Map structure: Use Mapto to set configuration parameters
- Configuration: All configuration items are in setting
Map structure Body (example)
In Go-ini, you can map structs in a mapto way, for example:
type Server struct { RunMode string HttpPort int ReadTimeout time.Duration WriteTimeout time.Duration}var ServerSetting = &Server{}func main() { Cfg, err := ini.Load("conf/app.ini") if err != nil { log.Fatalf("Fail to parse 'conf/app.ini': %v", err) } err = Cfg.Section("server").MapTo(ServerSetting) if err != nil { log.Fatalf("Cfg.MapTo ServerSetting err: %v", err) }}
In this code, you can note that serversetting took the address, why Mapto must address the parameter?
// MapTo maps section to given struct.func (s *Section) MapTo(v interface{}) error { typ := reflect.TypeOf(v) val := reflect.ValueOf(v) if typ.Kind() == reflect.Ptr { typ = typ.Elem() val = val.Elem() } else { return errors.New("cannot map to non-pointer struct") } return s.mapTo(val, false)}
An error that typ.Kind() == reflect.Ptr
is bound to be returned if a pointer must be used in Mapto cannot map to non-pointer struct
. This is a superficial reason.
More inward probing, which can be thought field.Set
of as the cause, when executed val := reflect.ValueOf(v)
, the function is created by passing the v
copy, but the change does not change the original, and the change that is to be made val
val
v
val
will have to be v
passed v
The address
Obviously Go-ini also contains the function of modifying the original value, what do you think is the reason?
Configuration of the governs
In previous versions, models and file configurations were parsed in their own files, while others were in setting.go, so we needed to unify them in setting
You might want to copy and paste the configuration items directly into Setting.go's init, and you're done.
But you're thinking, there's more than one init function in the previous code, there's a problem with the order of execution, and you can't meet our requirements, you can try
(Here is a basic point of knowledge)
In Go, when there are multiple init functions, the Order of execution is:
- init functions under the same package: order of execution by source file compilation (default sort by file name)
- init functions under different packages: determining the order of dependencies by package import
So to avoid multiple init situations, try to initialize the sequence by the program control
Second, the implementation
Modifying a configuration file
Open Conf/app.ini To modify the configuration file to the big hump name, in addition we added 5 configuration items for uploading the image function, 4 file log aspects of the configuration items
[app]PageSize = 10JwtSecret = 233RuntimeRootPath = runtime/ImagePrefixUrl = http://127.0.0.1:8000ImageSavePath = upload/images/# MBImageMaxSize = 5ImageAllowExts = .jpg,.jpeg,.pngLogSavePath = logs/LogSaveName = logLogFileExt = logTimeFormat = 20060102[server]#debug or releaseRunMode = debugHttpPort = 8000ReadTimeout = 60WriteTimeout = 60[database]Type = mysqlUser = rootPassword = rootrootHost = 127.0.0.1:3306Name = blogTablePrefix = blog_
Optimized configuration read and set initialization order
The first step
Delete the configuration that is scattered in other files, unify the processing in setting , and Modify the init function as the Setup method
Open the Pkg/setting/setting.go file and modify the following:
Package Settingimport ("Log", "Time" "Github.com/go-ini/ini") type App struct {jwtsecret string PageSize in T Runtimerootpath string Imageprefixurl string Imagesavepath string imagemaxsize int imageallowexts []strin G Logsavepath String Logsavename string Logfileext string timeformat string}var appsetting = &app{}type Se RVer struct {runmode string httpport int readtimeout time. Duration writetimeout time. Duration}var serversetting = &server{}type Database struct {type string User string Password string Host String Name string tableprefix string}var databasesetting = &database{}func Setup () {CFG, err: = ini. Load ("Conf/app.ini") if err! = Nil {log. Fatalf ("Fail to Parse ' Conf/app.ini ':%v", err)} err = Cfg.section ("app"). Mapto (appsetting) if err! = Nil {log. Fatalf ("Cfg.mapto appsetting err:%v", err)} appsetting.imagemaxsize = appsetting.imagemaxsize * 1024 * 1024Err = cfg.section ("server"). Mapto (serversetting) if err! = Nil {log. Fatalf ("Cfg.mapto serversetting err:%v", err)} serversetting.readtimeout = Serversetting.readtimeout * time. Second serversetting.writetimeout = serversetting.readtimeout * time. Second err = cfg.section ("database"). Mapto (databasesetting) if err! = Nil {log. Fatalf ("Cfg.mapto databasesetting err:%v", Err)}}
Here are a few things we did:
- Write structures that are consistent with configuration items (App, Server, Database)
- To map a configuration item to a struct using mapto
- To re-assign some configuration items that require special settings
- Modify the init function of Models.go, Setting.go, Pkg/logging/log.go to the Setup method
- Delete the Models/models.go independently read DB configuration item and read the setting
- Remove the Pkg/logging/file separate LOG configuration item and read it uniformly setting
After a few comparison basis, and did not post out, I believe you can solve, if there is a problem, you can turn right project address
Step Two
In this step we will set the initialization process, open the Main.go file, and modify the content:
func main() { setting.Setup() models.Setup() logging.Setup() endless.DefaultReadTimeOut = setting.ServerSetting.ReadTimeout endless.DefaultWriteTimeOut = setting.ServerSetting.WriteTimeout endless.DefaultMaxHeaderBytes = 1 << 20 endPoint := fmt.Sprintf(":%d", setting.ServerSetting.HttpPort) server := endless.NewServer(endPoint, routers.InitRouter()) server.BeforeBegin = func(add string) { log.Printf("Actual pid is %d", syscall.Getpid()) } err := server.ListenAndServe() if err != nil { log.Printf("Server err: %v", err) }}
Once the modification is complete, the initialization function of the multi-module is successfully put into the initiation process (it can be controlled in order)
Verify
So far, the configuration optimization for this requirement is complete, and you need to go run main.go
verify that your functionality is working properly.
Incidentally, let's leave a basic question and we can think about it.
ServerSetting.ReadTimeout = ServerSetting.ReadTimeout * time.SecondServerSetting.WriteTimeout = ServerSetting.ReadTimeout * time.Second
What happens if you delete these two lines in the Setting.go file, and what is the problem?
Pulling away from File
In previous versions, we used some of the OS methods in Logging/file.go, which we discovered through pre-planning, which can be reused in the upload image function.
The first step
Create a new file/file.go in the PKG directory and write the following file:
package fileimport ( "os" "path" "mime/multipart" "io/ioutil")func GetSize(f multipart.File) (int, error) { content, err := ioutil.ReadAll(f) return len(content), err}func GetExt(fileName string) string { return path.Ext(fileName)}func CheckExist(src string) bool { _, err := os.Stat(src) return os.IsNotExist(err)}func CheckPermission(src string) bool { _, err := os.Stat(src) return os.IsPermission(err)}func IsNotExistMkDir(src string) error { if exist := CheckExist(src); exist == false { if err := MkDir(src); err != nil { return err } } return nil}func MkDir(src string) error { err := os.MkdirAll(src, os.ModePerm) if err != nil { return err } return nil}func Open(name string, flag int, perm os.FileMode) (*os.File, error) { f, err := os.OpenFile(name, flag, perm) if err != nil { return nil, err } return f, nil}
Here we have a total of 7 methods encapsulated
- GetSize: Get File size
- Getext: Get file suffix
- Checkexist: Check if the file exists
- Checkpermission: Check file permissions
- Isnotexistmkdir: Create a new folder if it does not exist
- MkDir: New Folder
- Open: Opening file
Here we use the mime/multipart
package, it mainly implements the MIME multipart parsing, mainly for HTTP and common browser-generated multipart body
Multipart is what, rfc2388 Multipart/form-data understand
Step Two
We have re-encapsulated the file in the first step, and in this step we will fix the original logging package.
1. Open the Pkg/logging/file.go file and modify the contents of the file:
Package Loggingimport ("FMT" "OS" "Time" "github.com/eddycjy/go-gin-example/pkg/setting" "Github.com/eddyc Jy/go-gin-example/pkg/file ") func Getlogfilepath () string {return FMT. Sprintf ("%s%s", setting. Appsetting.runtimerootpath, setting. Appsetting.logsavepath)}func getlogfilename () string {return FMT. Sprintf ("%s%s.%s", setting. Appsetting.logsavename, time. Now (). Format (setting. Appsetting.timeformat), setting. Appsetting.logfileext,)}func openlogfile (FileName, FilePath string) (*os. File, error) {dir, err: = OS. GETWD () if err! = Nil {return nil, fmt. Errorf ("OS. GETWD err:%v ", err)} src: = dir +"/"+ filePath perm: = file. Checkpermission (src) if perm = = true {return nil, fmt. Errorf ("file. Checkpermission Permission denied src:%s ", src)} err = file. Isnotexistmkdir (SRC) if err! = Nil {return nil, fmt. Errorf ("file. Isnotexistmkdir src:%s, err:%v ", SRC, err)} F, err: = file. Open (src + FIlename, OS. O_append|os. O_create|os. O_wronly, 0644) if err! = Nil {return nil, fmt. Errorf ("Fail to OpenFile:%v", err)} return F, nil}
We're going to change the quote to the method in the File/file.go package.
2. Open the Pkg/logging/log.go file and modify the contents of the file:
package logging...func Setup() { var err error filePath := getLogFilePath() fileName := getLogFileName() F, err = openLogFile(fileName, filePath) if err != nil { log.Fatalln(err) } logger = log.New(F, DefaultPrefix, log.LstdFlags)}...
Since the original method parameters have changed, the OpenLogFile also needs to be adjusted
Implement upload Image interface
This section, we begin to implement some of the previous image-related methods and functions
You first need to add a field to the blog_article in the cover_image_url
formatvarchar(255) DEFAULT '' COMMENT '封面图片地址'
0th Step
Generally does not directly expose the uploaded image name, so we MD5 the image name to achieve this effect
Create a new md5.go in the Util directory and write the contents of the file:
package utilimport ( "crypto/md5" "encoding/hex")func EncodeMD5(value string) string { m := md5.New() m.Write([]byte(value)) return hex.EncodeToString(m.Sum(nil))}
The first step
We've already encapsulated the underlying approach in the first place, which is the process logic for encapsulating image
Create a new Upload/image.go file in the pkg directory and write the contents of the file:
Package Uploadimport ("OS" "Path" "Log" "FMT" "Strings" "Mime/multipart" "github.com/eddycjy/go-gin- Example/pkg/file "" Github.com/eddycjy/go-gin-example/pkg/setting "" github.com/eddycjy/go-gin-example/pkg/logging "Github.com/eddycjy/go-gin-example/pkg/util") Func Getimagefullurl (name string) string {return setting. Appsetting.imageprefixurl + "/" + Getimagepath () + Name}func getimagename (name string) String {ext: = path. EXT (name) FileName: = Strings. Trimsuffix (name, ext) fileName = util. EncodeMD5 (filename) return filename + ext}func Getimagepath () string {return setting. Appsetting.imagesavepath}func Getimagefullpath () string {return setting. Appsetting.runtimerootpath + getimagepath ()}func checkimageext (FileName string) bool {ext: = file. Getext (FileName) for _, Allowext: = Range setting. appsetting.imageallowexts {if strings. ToUpper (allowext) = = Strings. ToUpper (EXT) {return true}} return FalsE}func checkimagesize (f multipart. File) bool {size, err: = file. GetSize (f) if err! = Nil {log. PRINTLN (Err) logging. Warn (ERR) return false} return size <= setting. Appsetting.imagemaxsize}func checkimage (src string) error {dir, err: = OS. GETWD () if err! = Nil {return FMT. Errorf ("OS. GETWD err:%v ", err)} err = file. Isnotexistmkdir (dir + "/" + src) if err! = Nil {return FMT. Errorf ("file. Isnotexistmkdir err:%v ", Err)} perm: = file. Checkpermission (src) if perm = = true {return FMT. Errorf ("file. Checkpermission Permission denied src:%s ", SRC)} return nil}
Here we have implemented 7 methods, as follows:
- Getimagefullurl: Get picture full access URL
- Getimagename: Get Picture name
- Getimagepath: Get Picture path
- Getimagefullpath: Get picture full path
- Checkimageext: Check image suffix
- Checkimagesize: Check the image size
- Checkimage: Checking pictures
This is basically two times the underlying code package, in order to more flexible processing some of the image-specific logic, and easy to modify, not directly exposed to the lower layer
Step Two
This step will write the business logic to upload the picture, create a new Upload.go file in the Routers/api directory, write the contents of the file:
Package Apiimport ("Net/http" "Github.com/gin-gonic/gin" "github.com/eddycjy/go-gin-example/pkg/e" "github.co M/eddycjy/go-gin-example/pkg/logging "" Github.com/eddycjy/go-gin-example/pkg/upload ") func UploadImage (c *gin. Context) {code: = e.success Data: = Make (map[string]string) file, image, Err: = C.request.formfile ("image") I F Err! = Nil {logging. Warn (ERR) code = E.error C.json (http. Statusok, Gin. h{"Code": Code, "MSG": E.getmsg (Code), "Data": Data,})} if image = = Nil {code = E.invalid_params} else {imageName: = upload. Getimagename (image. Filename) FullPath: = Upload. Getimagefullpath () Savepath: = Upload. Getimagepath () src: = FullPath + imageName if! Upload. Checkimageext (imageName) | | ! Upload. Checkimagesize (file) {code = E.error_upload_check_image_format} else {err: = UPLOAD. Checkimage (FullPath) IF Err! = Nil {logging. Warn (ERR) code = E.error_upload_check_image_fail} else If err: = C.saveuploadedfile (IMAGE, SRC) ; Err! = Nil {logging. Warn (ERR) code = E.error_upload_save_image_fail} else {data["image_url"] = Uplo Ad. Getimagefullurl (imageName) data["image_save_url"] = Savepath + ImageName}}} c.js On (HTTP. Statusok, Gin. h{"Code": Code, "MSG": E.getmsg (Code), "Data": Data,})}
The error code involved (to be added in Pkg/e/code.go, msg.go):
// 保存图片失败ERROR_UPLOAD_SAVE_IMAGE_FAIL = 30001// 检查图片失败ERROR_UPLOAD_CHECK_IMAGE_FAIL = 30002// 校验图片错误,图片格式或大小有问题ERROR_UPLOAD_CHECK_IMAGE_FORMAT = 30003
In this large segment of business logic, we do the following things:
- C.request.formfile: Gets the uploaded picture (the first file that returns the provided table one-touch)
- Checkimageext, checkimagesize Check the image size, check the image suffix
- Checkimage: Check the upload image required (permissions, folders)
- Saveuploadedfile: Save Picture
In general, it is the application process of the check-save
Step Three
Open the Routers/router.go file to add routes r.POST("/upload", api.UploadImage)
such as:
func InitRouter() *gin.Engine { r := gin.New() ... r.GET("/auth", api.GetAuth) r.GET("/swagger/*any", ginSwagger.WrapHandler(swaggerFiles.Handler)) r.POST("/upload", api.UploadImage) apiv1 := r.Group("/api/v1") apiv1.Use(jwt.JWT()) { ... } return r}
Verify
Finally, we request to upload the image interface, test the function written
Check if the directory contains files (attention to permissions issues)
$ pwd$GOPATH/src/github.com/EDDYCJY/go-gin-example/runtime/upload/images$ ll... 96a3be3cf272e017046d1b2674a52bd3.jpg... c39fa784216313cf2faa7c98739fc367.jpeg
Here we have returned a total of 2 parameters, one is the full access URL, the other is to save the path
Implement HTTP. Fileserver
After we have completed the previous section, we also need to have the front-end access to the picture, which is generally as follows:
In the company, CDN or self-built distributed file system is mostly, do not need too much attention. In practice, the words must be built locally, Go itself has a good support for this, and Gin is again encapsulated a layer, only need to add a line of code in the route can be
Open the Routers/router.go file to add routes r.StaticFS("/upload/images", http.Dir(upload.GetImageFullPath()))
such as:
func InitRouter() *gin.Engine { ... r.StaticFS("/upload/images", http.Dir(upload.GetImageFullPath())) r.GET("/auth", api.GetAuth) r.GET("/swagger/*any", ginSwagger.WrapHandler(swaggerFiles.Handler)) r.POST("/upload", api.UploadImage) ...}
When the $HOST/upload/images is accessed, the files under $GOPATH/src/github.com/eddycjy/go-gin-example/runtime/upload/images will be read
And what does this line of code do, let's look at the method prototype
// StaticFS works just like `Static()` but a custom `http.FileSystem` can be used instead.// Gin by default user: gin.Dir()func (group *RouterGroup) StaticFS(relativePath string, fs http.FileSystem) IRoutes { if strings.Contains(relativePath, ":") || strings.Contains(relativePath, "*") { panic("URL parameters can not be used when serving a static folder") } handler := group.createStaticHandler(relativePath, fs) urlPattern := path.Join(relativePath, "/*filepath") // Register GET and HEAD handlers group.GET(urlPattern, handler) group.HEAD(urlPattern, handler) return group.returnObj()}
First, the use of * and: symbols is forbidden in the exposed URL, and by createStaticHandler
creating a static file service, the actual final call or fileServer.ServeHTTP
some processing logic
Below you can see urlPattern := path.Join(relativePath, "/*filepath")
, /*filepath
who you are, what you use here, you are the product of Gin?
The semantics are known to be the processing logic of the route, and the Gin route is based on the Httprouter, and the following information can be obtained by consulting the document.
Pattern: /src/*filepath /src/ match /src/somefile.go match /src/subdir/somefile.go match
filepath will match all file paths, and filepath must be at the end of the Pattern
Verify
Re go run main.go
-Execute, go to access the image address just obtained on the upload interface, check http. FileSystem is normal
Modify the article Interface
Next, you need to modify Routers/api/v1/article.go's addarticle, editarticle two interfaces
- New, updated article interface: support for the entry Cover_image_url
- New, updated article interface: increased non-empty, maximum length check for Cover_image_url
The previous article says that if you have a question, you can refer to the code of the project
Summarize
In this chapter, we briefly analyze the requirements, make a small plan for the application and implement
Complete the function points and optimizations in the checklist, which is a common scenario in the actual project, and I hope you can savor them and learn more about them.
Reference
Sample code for this series
Catalog of this series
- Gin Practice Serial One Golang introduction and environment installation
- Gin Practice Two Build blog API ' s (i)
- Gin Practice three Build blog API ' s (ii)
- Gin Practice Four Build blog API ' s (iii)
- Gin Practice Five Build blog API ' s (iv)
- Gin Practice Six Build blog API ' s (v)
- Gin Practice serial Seven Golang graceful restart HTTP service
- Gin practice serial Eight for it plus swagger
- Gin practice nine deploying Golang applications to Docker
- Gin Practice serial 10 Custom GORM callbacks
- Gin Practice Serial 11 Cron timed tasks
- Gin Practice Golang Cross-compiling