Go dependency management mechanism

This is a creation in Article, where the information may have evolved or changed.

In any language, dependency management is a more complex issue. The dependency management mechanism in the go language is still a bit disappointing. Before the 1.6 version, the official only put the dependency on Gopath, and there is no multi-version management mechanism; the 1.6 version (1.5 version is experimental feature) introduces the vendor mechanism, which is package-dependent management for an important attempt. He remains a hot topic of contention in the go ecosystem and has yet to think of a perfect solution.

See other

Let's take a look at how other languages are solved, with examples of two typical management methods:

Java

Development, you can edit dependent inventory lists/scripts through Maven and Gradle tools, specify the location/version of dependent libraries, which can help you solidify your project to a state where you can compile and publish it at the right time. These tools are elegant and effective enough for me. But there are also annoying issues such as the internal dependency version conflict of different dependencies in Maven. Especially in large-scale projects, the dependency transfer problem, if the team members of the MAVEN mechanism is not enough to understand, relying on the abuse of scope, will make the entire project engineering the dependency tree becomes particularly large and inefficient each compilation. Running state, Java also does not have a good dependency on the management mechanism, although there are classloader can do some isolation, but like the strict version of OSGi management, will let users into multiple versions of conflicting mire.

node. js

NPM is the preferred Module dependency management tool for node. js. NPM describes the dependencies of the module through a Package.json file in the current directory, in which you can define your app name, app description (description), keywords (keywords), version number, and so on. NPM downloads the current project dependency module into a folder called Node_modules in your project. Unlike Maven/gradle, MAVEN will eventually parse the dependency tree and flatten the same software by default to the highest version. and NPM supports nested dependency tree. Nested dependency tree is a module in each module that relies on its own directory node_modules, which avoids dependency conflicts, but consumes more space and time. Since JavaScript is a source code release, the development state and the running state depend on NPM, which takes precedence from its own node_modules to search for dependent modules.

Go get

Go to package management must have their own understanding. For a package fetch, it is pulled from the remote code base (GitHub, BitBucket, Google Code, LAUNCHPAD) with the go Get command. The advantage of this is that directly skip the constraints of the Central Library of the package management, so that the code is pulled directly based on the version control Library, and everyone's collaborative management is based on this version of the library to interact. The benefit of this design is that it removes redundancy and directly re-uses the most basic code infrastructure. Golang's doing so greatly reduces the developer's burden of understanding the complex concepts of package management and is cleverly designed.

Of course, the go get command is still too simple. For developers in the real world, there is still a painful place to be:

• There is a lack of explicitly displayed versions. Teams develop different projects that are easy to import differently, each time get the latest code. Especially as our open source software management is very strict, open source applications are almost impossible to implement.
• Third-party packages do not have content security audits, access to the latest code is easy to introduce code new bugs, the subsequent run out of the bug needs to be resolved, and version tracking management.
• Dependency integrity can not be verified, domain-based package name, domain name changes or sub-path changes, will result in the inability to download the normal dependency. We are using the process to find that there are still a lot of indirect dependency on the package name has been invalidated (does not exist, or fork into a new project, the old has not been saved maintenance updates).

The go official's advice on such issues is to copy externally-dependent code into your source repository for management. The introduction of third-party code into your own code base is still a compromise approach, which is unrealistic for the software development process like our own:

• Open source scanning will scan out is similar code, if license is not loose, then involves the legal risk, if loose, open source Scan certification Confirmation work is also very cumbersome.
• How to upgrade the version, the code is copied over, the source of the project code can vary greatly, no obvious version verification, the use of tools or scripts to upgrade will also bring a lot of work.
• The copy of the code has started to become private, third-party code bugs can only be solved by themselves, it is difficult to contribute code to fix bugs, or to promote the community to solve.
• A common procedural problem may not be a big problem, at best, a compile-time dependency. But if you're writing a Lib library for someone else to use, introducing this library would be a hassle. How do you manage the code dependencies of your library, which is referenced by multiple people?

Fortunately, the power of open source is big, go official do not think clearly of version management issues, the community will be someone to solve, we can find a lot of good solutions, may wish to refer to the official recommendations.

Vendor mechanism

Vendor is a dependency management feature that 1.5 introduced for the experience, officially released in 1.6. The Go team has done a long-time research on Golang-dev group before launching vendor. Eventually Russ Cox was improved on the basis of Keith Rarick's proposal, forming the vendor in Go 1.5:

• No, rewrite Gopath.
• Go tool to solve
• Go get compatible
• Can reproduce building process

and gives the "4 lines" interpretation of the vendor mechanism:

If There is a source directory D/vendor, then, when compiling a source file within the subtree rooted at D, import "P" is Interpreted as import "d/vendor/p" if that exists.

When there is multiple possible resolutions,the most specific (longest) path wins.

The short form must always be used:no import path can contain "/vendor/" explicitly.

Import comments is ignored in vendored packages.

To sum up and explain:

• Vendor is a special directory, and the Go Doc tool ignores it in the application's source directory.
• The vendor mechanism supports nested Vendor,vendor in a third-party package that can also contain vendor directories.
• If the same package exists under different levels of vendor, the compilation lookup path first searches for the presence of vendor under the current Pakcage, and if there is no further vendor search under parent pacakge (x/y/z as Parentpath input, Search path: X/y/z/vendor/path->x/y/vendor/path->x/vendor/path->vendor/path)
• In the use of vendor this path of the existence, how to import the package on how to import, do not appear import "d/vendor/p" situation. Vendor is implicitly handled by the Go tool.
• Does not verify that the package's import path in vendor is consistent with the canonical import path.

The vendor mechanism looks like the node_modules of node. JS, supports nested vendor, and if a project is in a two-version package, it can be placed under different levels of vendor:

• Pros: Different versions may be resolved depending on conflict issues, different levels of vendor are stored in different vendor.
• Disadvantage: As the go package is organized by path, at compile time, the same packages in different levels of vendor will compile two times, link two parts, program files become larger, run time is to execute different code logic. can cause problems, if the global initialization in package init, may duplicate the initial problem, may also be initialized to a different variable (in memory), cannot share the fetch. Like before we met GPRC similar problems are different levels of the same package repeat Init caused, see community feedback.

So Russ Cox expects everyone to design a good layout, as Lib's package does not carry vendor better, a project of all vendor are concentrated in the top vendor inside.