Go Pack Management's past life

This is a creation in Article, where the information may have evolved or changed.

To tell the truth, Golang is really unfriendly to a newcomer because of the concept of getting started. You look at others Java, come up a project mvn install a bit to finish, hurriedly neat. But Golang is in trouble, you have to know what Gopath is. When I first contacted Golang really started to do the project, only know to configure the environment variables as required, the real understanding of Gopath may be a few months later things. Plainly, or because of laziness. The person who really does the project, how many have the patience to chop wood to grind a knife, appear a thing to study half a day ah, we just want copy-paste only.

But I have to admit, for today's discussion of the topic of Go package management, if you want to understand Golang package management mechanism, even gopath do not want to fully understand, it may really not need to read this article, the next time I met a follow the readme step-by-step honest to the line. What if I can't find the Readme document? Ask the project maintainer for bugs!

Gopath

Back to the Gopath, or return to the understanding. So, what is Gopath? What's the use? Essentially Gopath is a system environment variable, which is where the go language is used to store code dependencies.

Many people do not know the difference between Gopath, goroot, in fact, there is no need to understand the complex. When the Go language installation package has just been downloaded, you unzip it or directly installed in the directory, is the Goroot directory, you need to do some additional configuration, goroot this environment variable set. Of course, the corresponding bin directory you have to set up, or the operating system can not find the executable file of go.

For example, after you unzip the installation to the/opt/go/directory, or under Windows you installed to the C-disk c:\\go, are OK, the difference is only different operating system settings in different ways. I won't go into the details of how to set it up.

So far, you can forget about the goroot, because it has solved all the things related to it. But we still have to explain why this thing is set up. The reason is very simple, golang many of the default mechanisms are like to read from the environment variables, set the GOROOT environment variable, equivalent to tell all read this variable program we golang source location, easy to reference the code. Can be understood as goroot is the primary color, it can be combined with a lot of different colors, is the most initial code dependence. Many of the code inside the Goroot are system drivers and system calls.

So we have three primary colors, want to match more colors, we set the color of the process of combining the color, that is, the new code to write out where the package? You must have guessed that this is the role of the Gopath directory. Any code other than the Golang installation package, whether you write it yourself or a third party's incredibly mature package, needs to be placed under Gopath.

So you do not ask why I directly git clone down the code how all kinds of error can not find the dependency, you set up the Gopath?

So how do you set Gopath? This is based on personal tastes, very casual. Some people choose to use the hidden directory, such as ~/.go/as Gopath, but also some people set up in Goroot next door, a new directory called/opt/gopath, for Windows users, Of course, you can set the environment variable to put yourself under the D disk and create a directory called Gopath D:\\gopath, and then set the environment variable. I personally prefer the/home/sunjianbo/gopath directory, which is ~/gopath.

So can I have multiple gopath directories? Of course, you can set the system environment variable when you can put more than one value.

You can set export gopath=~/gopath1:~/gopath2:~/gopath3 entirely. In the configuration of these directories, the GO program will look for the corresponding dependency package in turn.

So are some readers already trying to understand the original package management approach?

The answer is to make a gopath for each project.

In particular, suppose we have a project called Tastego, in which we write a footstep that reads as follows:

Export gopath= $PWD/tastego: $GOPATH

As long as one line, simple to wayward. Of course, it is best to add another line, the bin path under the Gopath, so go install out of the content can be out of the box.

Export path= $PATH: $PWD/tastego/bin

For each project's dependence, categorized in the corresponding source code directory.

tastego/    src/        github.com/            wonderflow/...        qiniu/...    golang.org/... bin/    pkg/

So, you may not believe that the first go official does not provide any package management mechanism at the time, a lot of go language players, are playing in this way. Still playing with the wind, tens of millions of lines of code is a cinch.

But it is worth mentioning that we have observed gopath= $PWD/logkit: $GOPATH This structure is a recursive structure, not only to write this, in a different way gopath= $GOPATH: $PWD/logkit is also effective. But the pros and cons of these two formulations are different. First of all, I don't recommend either of these two ways.

• Gopath= $PWD/logkit: $GOPATH This is the current project directory in front, so encountered a new project, always at the front of Gopath. The big benefit is that when you encounter multiple package names exactly the same, the default is the front-most package, so the probability of packet collisions is minimized. The downside is also obvious, you go get down the code package, will go to the front of this directory, resulting in the location of every go get is indeterminate, especially when the environment variables are inadvertently clean, you even need to go back to get again.
• Gopath= $GOPATH: $PWD/logkit This way to ensure the stability of the order, later the package must be in the back, but the drawback is that you will soon appear the issue of package version conflict.

Of course, one of the big drawbacks of this approach is that when your project is big, you might have to put half of GitHub's code under your single code base (Repo).

Blossom

Finally, in the Golang official face gopath management of all kinds of chaos as always indifferent, the community can not look down, have appeared various package management tools.

In the 2013, the famous GEDEP tools were born. It was the first package management tool in the community and was warmly sought after by everyone. The principle is very simple, that is, the way we talked about in the last section, through the script to achieve. Save all the source code to a directory called godeps/_workspace/, and then use this godeps/_workspace/directory as the only Gopath. Provides a range of quick and easy to save all dependent packages to godeps/_workspace/directory, including GODEP save, GODEP restore, while quickly recovering gopath through GODEP Equivalent to the Gopath and source version of a snapshot. Of course, it doesn't solve your git clone need to download a half github puzzle.

But this year, Docker started to fire, so a lot of Golang projects build dockerfile to test the starting-up is:

ADD. /app/xxx/env Gopath/app/xxx/godep/_workspace

All of a sudden all depends on all have, indeed very convenient.

But at the time (2013-2014 years), this tool still did not solve my version problem, still did not solve my dependency on the package if also rely on a lot of other packages to do the problem. So by the year 2014, the Glide Package management tool, known as the package management tool for other languages * * Best practices, was born.

So what is it about claiming to be a best practice? Its best embodiment in its package management philosophy, once you install the completion of glide, a glide create command, will completely handle the problem of all package dependencies. As for the construction of multiple versions can be used, depending on how to choose, choose which problem, it will help you fix.

What if I want to specify the package version myself? Of course there is no problem, just fill in the Glide.yaml configuration file with your corresponding package version, or version of the scope, glide will be selected within this range.

So for a new user who builds a project from scratch, glide must be a good choice, and it helps you handle all the package dependencies. But for a project that already has some historical baggage, it may be awkward to use glide. In particular, there are some conflict-dependent issues in your project, and glide still has to help you determine which dependency to use, leading to a final run.

So someone asked, "Can not absolutely give me the best practice, put my reliance on all in, step by step?" "The official said," Our philosophy of design is that every step under glide management is executable, not just half a step! "But you don't have to solve the problem, eventually you can not do it, it is very stiff ...

Perhaps it is the different ideas, so glide known as "best practice" but did not unified lakes and rivers.

This period of time, such as gopkg.in, Gom and other package management tools were born, the idea is more advanced than the original GODEP, you say with glide than no more features, have added the concept of package version.

Vendor

Time went to 2015 years, Golang official finally can not see, in the launch of go1.5 version of the same time, the first experimental nature of the addition of the vendor mechanism function. Of course, this feature, after all, is experimental and closed by default, causing most users to actually not use it at all. Until 2016, when the official launch of the go1.6 version, the vendor mechanism was turned on by default.

So what exactly is a vendor mechanism? In layman's words, you include a vendor folder in your project, and the Go language defaults to a gopath. So you can put your dependency library in it.

For example, suppose your project hosted on GitHub is like this, and the project name is Tastego:

tastego/    main.go    common/        common.go    util/        util.go

In addition to the MAIN.GO main function, also contains 2 of their own written library (package), one is common, one is util, then in order for the project to compile properly, the 2 libraries should be in Gopath, then actually in the GOPATH structure, Your project directory is like this (Wonderflow is my github ID):

$GOPATH/    src/        github.com/            wonderflow/                tastego/                    main.go                    common/                        common.go                    util/                        Util.go

After adding the vendor mechanism, you add a vendor folder under your project directory, which can be placed on other dependencies, in the form of:

$GOPATH/    src/        github.com/            wonderflow/                tastego/                    main.go                    common/common.go util/                        util.go                    vendor/                        github.com/                        qiniu/                            ...                    golang.org/...

Let's take a look at the Gopath-based package management approach that we've just started with this article:

tastego/    src/        github.com/            wonderflow/...        qiniu/...    golang.org/... bin/    pkg/

Looks very similar, only has the vendor, has the official rectification! And you no longer need to manually (semi-manual) modify the Gopath, the shape of the project has been unified with the previous, the benefits are obvious.

But is the problem really solved? In fact, not all the problem, but because in the subsequent 2016, 2017, the vendor mechanism became a formal go rule, the problem is increasingly serious.

1. Nested Vendor Directory issues: What about the vendor directory in the vendor directory below the project?
2. The vendor mechanism itself does not have a version concept, and the issue of type incompatibility between different versions persists.
3. Conflict with the package init function under other Gopath: The same package appears, what about the repeating init () function?

So Golang team members also held the General Assembly, very much agree with the community of various package management tools concept, it is necessary to package management to put forward a uniform rule to solve the above problems. But the problem is not that there are no rules, but too many rules. is often a disagreement, a new tool to kill a sudden. There are 15 types of package management tools that are only officially recommended.

• Dep
• Manul-vendor packages using Git submodules.
• Godep
• Govendor
• Godm
• Vexp
• Gv
• Gvt-recursively Retrieve and vendor packages.
• Govend-manage dependencies like go get and For/vendor.
• Glide-manage packages like composer, npm, bundler, or other languages.
• Vendetta
• Trash
• Gsv
• Gom
• Rubigo-golang Vendor Utility and package Manager

Please refer to the official wiki page for details: https://github.com/golang/go/wiki/PackageManagementTools

So go officials are also trying to make package management one of the go Tools toolchain, and the official package management tool is DEP but the project is still immature and not yet included in the toolchain.

But the official advice is already obvious, so let's try to use package management tools to introduce dependencies, but it's best to use the standard library as much as possible, and on the other hand use the existing package management tools instead of creating a set of rules yourself.

So, we also come to learn how to use the package management tools, after several comparative research, I recommend

Govendor tools, so take this as an example.

Govendor

Govendor is essentially a copy of the source code to the vendor directory, by maintaining a Vendor.json file in the vendor directory, specifying the package version to use. The entire directory is structured so that when syncing to GitHub, you can either include the code directly in the project or use the. Gitignore to ignore dependent libraries and sync via Govendor sync.

Installation

Install just like all Golang tools, go get.

Go get-u Github.com/kardianos/govendor

Initialize Project

For an existing project that has not used any package management tools, it is very easy to start using Govendor.

In the project directory, perform the initialization:

CD "My Project in Gopath" Govendor Init

Add existing dependencies to the current project's vendor management.

Govendor Add +external

At this point, you have successfully switched the existing project to Govendor management.

Common commands in the project process

After the initialization project is complete, it is time for the project General Management phase, which typically requires the following scenarios.

Add dependency

If you already exist in your local gopath, use Govendor add

# Specify version of commit, package name followed by @ symbol Plus commit Idgovendor add golang.org/x/net/context@a4bbce9fcae005b22ae5443f6af064d80a6f5a55# Specify the version name, the package name followed by the @ symbol plus the version name or branch name Govendor add golang.org/x/net/context@v1   # Get latest v1.*.* tag or branch.

If you do not exist locally, use Govendor Fetch, the other specified version is the same as Govendor add

Loading all the dependencies of all projects at once is the command we introduced when we initialized.

Govendor Add +external

Remove dependencies

Remove a dependency

Govendor Remove Golang.org/x/net/context

Remove dependencies that are not used by all projects

Govendor Remove +unused

Update dependencies

Of course you can choose Govendor Remove and then Govendor Add.

You can also update packages that have been updated in the local gopath directly using Govendor update.

If the local is not current or does not exist, please update with Govendor Fetch.

Govendor Update Golang.org/x/net/context@a4bbce9fcae005b22ae5443f6af064d80a6f5a55

Synchronizing GOVENODR Packages

In general, the collaboration process of open source projects, other people update the project Govendor, then you have to synchronize, directly using the following command.

Govendor Sync

View local dependencies and package status

Usually you get a project that you might want to intuitively understand what dependencies he has, and use the Govendor list to view it.

Pure use of the Govendor list is of little value.

Interestingly, Govendor to Golang's dependency package added a status description, combined with various vendor state parameters for various types of operation is very interesting.

There are several types of states:

    +local    (L) There are only packages in the project    +external (e) in Gopath, but packages that are not in the project    +vendor   (v) package vendor in the project +STD directory      (s) Use the Golang standard library    +excluded (x) project does not contain and explicitly declare packages to be excluded    +unused   (U) in the vendor directory, but the actual project does not use the package    +missing  (M A package that is used but not found in the project (Govendor Fetch is required)    +program  (P) package    +outside All external package combinations with the main function entry  , including (+external + Missing)    +all     List all packages

These state information can be used with other commands, such as

Govendor Add +external

Govendor Remove +unused

The coolest thing is that the state can also do logic and/or operations, such as:

+local,program (Local and program) indicates that the package in the project is also the main function entry +local +vendor (local OR Vendor) represents the package in the project and the package vendor in +vendor,program + The STD ((vendor and program) OR std) means that the package in vendor is accompanied by the main function entry, plus the package +vendor,^program (vendor and not program) of the standard library represents the package in the vendor directory. But does not contain a package with the main function entry

To view dependencies between packages

Use the-v parameter to see which packages are dependent on a package:

Govendor list-v

So in turn, you might want to know which packages a package depends on. This is the method provided in the Go tool chain, using the Go list directly

Like what:

Go list-f ' {{. Imports} ' Github.com/wonderflow/tastego

View the actual path of the package

With the-p parameter, you can see the actual file path of the package, which is different from the path filled in when import, and the actual path can quickly find the location of the package you refer to.

To view the packages for all current projects:

Govendor list-p-no-status +local

At this point, we, Govendor's commonly used commands have been introduced, I believe that the master of these, in the future project management of a variety of dependency packages, you must be at ease.

Thank Xuchuan for the review of this article.

To contribute to or participate in content translation for Infoq Chinese station, please email to editors@cn.infoq.com. You are also welcome to follow us through Sina Weibo (@InfoQ, @ Ding), (No.: Infoqchina).