Embed data in a go executable file

This is a creation in Article, where the information may have evolved or changed.

Original Address https://scene-si.org/2017/08/22/embedding-data-in-go-executables/

If you've been watching me for a while, you should know that I'm developing the pendulum editor as the #100daysofcode Challenge of coding for at least one hour a day. Pendulum is a web-based editor that is ideal for editing simple text and markdown files.

In fact, this article is written with it. It consists of the go backend and the Vuejs front end. I want it to be easy to use and provide a single executable file with everything, so the user doesn't need to download the installer and unzip the file. I need to find a solution that can pack everything together. I decided to use Go-bindata to add all the data through the go build to the executable in a code-generation way.

Code generation?

Of course it's simple. For example, the Go-bindata tool can help us public_html generate a corresponding. go file from the directory. This is excellent for my application scenario. But why use a bash script or makefile to generate it? Because then we just go build need to execute the code generation tool with go before we execute it go generate . If you're not familiar with code generation, you just need to add a simple comment somewhere in your code, for main.go example:

package main//go:generate echo "Hello world"func main() {}

go generatewhen executed, you can see the output of "Hello world". This is not the actual need for you to generate code with go generate. Everything you //go:generate write in the back will be executed. You can even do it if you want go build .

package main//go:generate echo "Hello world"//go:generate go run main.gofunc main() {    println("Hello world from Go")}

Running this will have the expected output:

%go generateHello worldHello world from Go

Hit Go generate is very interesting. The node program babel makes node ES5 run-time compatible with ES6/ES7 syntax. People are trying to use a similar approach to provide go beyond the current features of the language.

For example, Genny is primarily for strong-type code generation, so you no longer need to copy and paste manually. However, these projects are closer to Babel's handling of node-providing the language to go. I am not aware of other attempts at this stage that are more appealing. But the discussion of Go2 and generics seems to be more interesting.

This is a bit tedious for our scenario, and we just need to pack some data into the program. Then the gossip, the book:

//go:generate go-bindata -prefix front/src -o assets/bindata.go -pkg assets -nomemcopy front/src/dist/...

This line is a little bit long, just split it up to see:

• //go:generate-To go generate make a note
• go-bindata-The main command to execute
• -prefix front/src-Exclude "FRONT/SRC" package
• -o assets/bindata.go-Specify Output file
• -pkg assets-The package name to be generated
• -nomemcopy-Optimized for memory consumption
• front/src/dist/...-The place to pack

This creates a package in the app directory that is simple to app/assets import assets , which corresponds to the app application directory.

Provides an embedded file service over HTTP

It's a little bit complicated. But after looking at the document, it's easy. If you want to provide services based on a local file, you need roughly the following lines of code like this:

folder := http.Dir("/")server := http.FileServer(folder)http.Handle("/", server)

In fact, the GO-BINDATA-ASSETFS package already provides an HTTP. Fileserver implementation. This is simple enough to use:

import "github.com/elazarl/go-bindata-assetfs"import "app/assets"// ...func main() {    // ...    files := assetfs.AssetFS{        Asset:     assets.Asset,        AssetDir:  assets.AssetDir,        AssetInfo: assets.AssetInfo,        Prefix:    "dist",    }    server := http.FileServer(&files)    // ...}

There is a small problem. I'm using the Vuejs app with the pushhistory enabled. This means that users will see similar generic links without the release companion (hash, #) /blog/about.md . The link content that needs to be processed by the app does not exist in asset.

The problem is not difficult to solve. A assetfs.AssetFS struct has a AssetsInfo method (equivalent os.Stat ) and a Asset method (a bit like ioutil.ReadFile ). This makes it possible to check if a file exists in asset and, if it does not exist, to output another file:

// Serves index.html in case the requested file isn't found// (or some other os.Stat error)func serveIndex(serve http.Handler, fs assetfs.AssetFS) http.HandlerFunc {    return func(w http.ResponseWriter, r *http.Request) {        _, err := fs.AssetInfo(path.Join(fs.Prefix, r.URL.Path))        if err != nil {            contents, err := fs.Asset(path.Join(fs.Prefix, "index.html"))            if err != nil {                http.Error(w, err.Error(), http.StatusNotFound)                return            }            w.Header().Set("Content-Type", "text/html")            w.Write(contents)            return        }        serve.ServeHTTP(w, r)    }}

If a file is found, replace my own implementation with the preset Servehttp method. This approach requires only a slight adjustment to the handler we have previously defined:

http.HandleFunc("/", serveIndex(server, assets))

serveIndexThe function returns one http.HandlerFunc , and this line is the corresponding modification. This provides a complete implementation of the data services you add to your app with go generate and go-bindata. If you want to skip //go:generate links, it's also possible to put these in a CI script.

In view of this I implemented a single executable release version of Pendulum. You can get and try from the GitHub publishing page.

EDIT: Improved Serveindex example Thanks @rdihipone

When you see this ...

It would be great if you could buy a book of my own:

• API Foundations in Go
• Factor Apps with Docker and Go
• The SaaS Handbook (work in Progress)

I promise you ' ll learn a lot more if you buy one. Buying a copy supports me writing more about similar topics. Say Thank you and buy my books. Feel free to send me an e-mail if you want to book my time for consultancy/freelance services. I ' m great at APIs, Go, Docker, Vuejs and scaling services, among many other things.