Go modules: Best Practices

Go’s dependency management was very tricky from the very beginning and as a result, a lot of third-party tools like dep, govendor, glide came into existence. Each of these dependency management tools tried to solve the various set of problems but still, some void has to be filled.

Go module was released back in 2018 and was an experimental feature till Go 1.13. A lot of improvements and bug fixes have been done to make it production-ready. In the nutshell, go module itself takes care of a lot of things and gives developers a very rare chance to make a mistake. In this blog I will share the best practices while working with go modules.

Creating go.mod file:

Run go mod init github.com/org/repo inside the root folder of your project. It will create a go.mod file in the same directory with the module name and go version. If you are moving from any of the dependency management tools as mentioned here, It automatically translates the required information from Godeps.json,Gopkg.lock or other supported formats and puts it in go.mod file as require directive.

Note: The go.mod tab in GoCenter displays a suggested go.mod file for the projects that have not adopted go modules yet like given below:

Adding dependencies to go.mod:

Run go mod tidy to download all the dependencies transitively imported by packages in your module. It also removes the requirements that do not provide any imported dependencies, and populates go.mod and go.sum files with dependencies as require directives and checksums for each library at a specific version respectively.

Downloading dependencies:

Import a package in your project and run go build to build the binary and it will automatically pull the required dependencies for your package and put it as require directive in go.mod file and the checksum of the dependencies and their go.mod file into go.sum file (more on this later). By default, it pulls the latest version of that dependency but if you want to use any specific version of a dependency, edit the version of your choice in go.mod file.

Note: Though go build download the required dependencies but it is recommended to use go get -v to download the dependencies as it provides a more detailed error message than go build.

Checksum database:

When you run go build or go mod tidy, go.sum file is created if it doesn’t already exist, which stores a list of checksums of source code and go.mod files of each dependency respectively when it is downloaded. The go command uses this to verify the correctness of the dependencies from the checksum database sum.golang.org to ensure that the code you just downloaded wasn’t tampered with while building the binaries and hence you will get reproducible builds every time.

Upgrading/Downgrading dependencies:

To upgrade a dependency to the latest version run go get example.com/your/dependency It will automatically update the version of the dependency in go.mod file. If you want to update all the dependencies of a dependency run go get -u example.com/your/dependency

By default go get downloads the latest version of direct and indirect dependencies. When upgrading, consider using -u=patch

go get also supports upgrades or downgrades to a specific version by adding version or commit or branch suffix to the package argument. For example:

go get example.com/your/dependency@v1.2.0

go get example.com/your/dependency@ksjdbcdbciuwb

go get example.com/your/dependency@development

Note: Go modules follow semver but if it is not able to resolve to semver tag will be recorded as pseudo-version. Please have a look at this blog to learn more about pseudo versions.

Replace dependencies:

Sometimes you may feel the need to test only any particular version of the dependency or forked version of that dependency to investigate or fix bugs in your dependency, replace directive is very helpful in such cases. As stated here

The replace directive allows you to supply another import path that might be another module located in VCS (GitHub or elsewhere), or on your local filesystem with a relative or absolute file path. The new import path from the replace directive is used without needing to update the import paths in the actual source code.

There are three cases where you can leverage replace directive

• You can make use of replace directive to use example.com/some/repo v3.2.1 as given below:

module example.com/your/module
go 1.14
require (
    github.com/some/repo v1.2.3
)
replace github.com/some/repo => github.com/some/repo v3.2.1

• If you want to use forked repo you can modify replace directive as given below:

replace github.com/some/repo => github.com/forked/repo v2.3.4

Note: If you are not sure about the version then use v0.0.0 in the require directive and go client will take care of the things. require directive is not required in go 1.12+ versions

• If you want to use the dependency residing in your local file-system modify replace directive with the absolute path to that repo as given below:

replace github.com/some/repo => ../path

Note: if the right-hand side of a replace directive is a filesystem path, then the target must have a go.mod file at that location. If the go.mod file is not present, you can create one with go mod init

Vendoring and Proxy:

Vendoring is a very popular way of sharing the source code and dependencies among others and guarantees reproducible builds. Other dependency tools such as dep download the dependencies and put under the vendor directory. This entire process takes a good amount of time and there are various tradeoffs like taking up space, availability of dependencies, and availability of sources from where you want to download the dependencies.

• Run go mod vendor to download the dependencies under vendor directory and run go build -mod=vendor to consume the vendor while building the binary.

Go has added proxy support since go 1.11, which lets you download the dependencies from private and publicly hosted proxies. Not only this helps you in saving your time in downloading the dependencies but also guarantees its immutability, this means you don’t need to vendor the dependencies, go client will pull the dependencies from the proxy servers when required instead of VCS like Github.

To start using Go module proxy, set Golang environment variable like given below:

export GOPROXY=https://gocenter.io

This change will be reflected while downloading the dependencies as it will be redirected to GoCenter. It provides lots of information about a dependency such as security vulnerabilities, dependencies, used by and other metrics such as downloads, forks, stars, etc which will be very helpful in choosing the dependencies for your project.

Prune dependencies:

Run go mod tidy to remove any dependencies from go.mod that is no longer needed. It is a best practice to run go mod tidy before publishing the module to VCS.

Conclusion

Go modules solve a variety of problems for the developers and it gets your job done in most of the cases, you rarely get into a situation where manual intervention is required.
replace directive solves the very tricky problems that you may encounter during the development and GOPROXY provides immutability, security, and speeds up the build process. These are some of the very cool features of go modules which will be very useful in the development process.

That’s all folks! As always thanks for reading. 😊

~ उत्कर्ष_उवाच