Go, without package scoped variables

This is a creation in Article, where the information may have evolved or changed.

This was a thought experiment, what would Go look like if we could no longer declare variables at the package level? What would is the impact of removing package scoped variable declarations, and what could we learn about the design of Go Programs?

I ' M only talking on expunging var , the other five top level declarations would still is permitted as they are effecti vely constant at compile time. You can, for course, continue to declare variables at the function or block scope.

Why is the package scoped variables?

But first, why is the package scoped variables? Putting aside the problem of globally visible mutable state in a heavily concurrent language, package scoped variables is Fundamentally singletons, used to smuggle state between unrelated concerns, encourage tight coupling and makes the code T Hat relies on them hard to test.

As Peter Bourgon wrote recently:

Tl;dr:magic is bad; Global State was magic→[therefore, you want] no package level vars; No func init.

Removing package scoped variables, in practice

To put the the "idea" to the test I surveyed the most popular Go code base in existence; The scoped variables were used, and assessed the effect applying this experiment Woul D has.

Errors

One of the most frequent uses of public package level var declarations is errors; io.EOF
sql.ErrNoRows, crypto/x509.ErrUnsupportedAlgorithm , and so on. Removing the use of the package scoped variables would remove the ability to use public variables for sentinel error values. But what could is used to replace them?

I ' ve written previously that's should prefer behaviour over type or identity when inspecting errors. Where that isn ' t possible, declaring error constants removes the potential for modification which retaining their identity Semantics.

The remaining error variables is private declarations which give a symbolic name to an error message. These error values is unexported so they cannot is used for comparison by callers outside the package. Declaring them at the "level", rather than at the "point" they occur inside a function negates the opportunity to add Additional context to the error. Instead I recommend using something like to pkg/errors capture a stack trace at the point the error occurs.

Registration

A registration pattern is followed by several packages in the class library such as net/http , database/sql , flag , and to a Lesse R extent log . It commonly involves a package scoped private map or struct which are mutated by a public function-a textbook singleton.

Not being able to create a package scoped placeholder for this state would remove the side effects in image the, database/sql an D crypto packages to register image decoders, database drivers and cryptographic schemes. However, this was precisely the magic that Peter was referring to–importing a package for the side effect of changing some g Lobal State of your program was truly spooky action at a distance.

Registration also promotes duplicated business logic. net/http/pprofthe package registers itself, via a side effect with net/http.DefaultServeMux , which are both a potential security issue- Other Code cannot use the default MUX without exposing the pprof Endpoints-and makes it difficult to convince the package net/http/pprof To register it handlers with another mux.

If package scoped variables were no longer used, packages like net/http/pprof could provide a function this registers routes on a s Upplied http.ServeMux , rather than relying on side effects to altering global state.

Removing the ability to apply the registry pattern would also solve the issues encountered when multiple copies of the SAM E package is imported in the final binary and try to register themselves during startup.

Interface Satisfaction Assertions

The interface Satisfaction idiom

var _ someinterface = new (SomeType)

occurred at least times on the standard library. In my opinion these assertions is tests. They don ' t need to being compiled, only to being eliminated, every time you build your package. Instead they should is moved to the corresponding _test.go file. But if we ' re prohibiting package scoped variables, this prohibition also applies to tests, so how can we keep this test?

One option is to move the declaration from package scope to function scope, which would still fail to compile ifSomeType stop implementing SomeInterface

Func testsometypeimplementssomeinterface (t *testing. T) {       //won ' t compile if SomeType does not implement Someinterface       var _ someinterface = new (SomeType)}

But, as this is actually a test, it's not the hard-to-rewrite this idiom as a standard Go test.

Func testsometypeimplementssomeinterface (t *testing.  T) {       var i interface{} = new (SomeType)       If _, OK: = i (someinterface);!ok {               t.fatalf ("expected%t to implement Someinterface ", i)       }}

As a side note, because the spec says that assignment to the blank identifier must fully evaluate the right hand side of T He expression, there is probably a few suspicious package level initialisation constructs hidden in those var Declarati Ons.

It ' s not all beer and skittles

The previous sections showed that avoiding package scoped variables might is possible, but there is some areas of the STA Ndard Library which has proved more difficult to apply the this idea.

Real singletons

While I think the singleton pattern was generally overplayed, especially in its registration form, there Me real singleton values in every program. A Good example of this is and os.Stdout friends.

Package OS var (        Stdin  = NewFile (UIntPtr (syscall. Stdin), "/dev/stdin")        Stdout = NewFile (UIntPtr (syscall). Stdout), "/dev/stdout")        Stderr = NewFile (UIntPtr (syscall). Stderr), "/dev/stderr"))

There is a few problems with this declaration. Firstly Stdin , Stdout and is of Stderr type *os.File , not their respective io.Reader or io.Writer interfaces. This makes replacing them with alternatives problematic. However the notion of replacing them is exactly the kind of magic, this experiment seeks to avoid.

As the previous constant error example showed, we can retain the singleton nature of the standard IO file descriptors, suc h that packages like log fmt and can address them directly, but avoid declaring them as mutable public variables with Something like this:

Package Mainimport (        "FMT"        "Syscall") type READFD IntFunc (r readfd) Read (buf []byte) (int, error) {        return sys Call. Read (int (r), buf)}type writefd intfunc (w writefd) Write (buf []byte) (int, error) {        return syscall. Write (Int (w), buf)}const (        Stdin  = READFD (0)        Stdout = WRITEFD (1)        Stderr = WRITEFD (2)) Func main () {        FMT. fprintf (Stdout, "Hello World")}

Caches

The second most common use of unexported package scoped variables is caches. These come in the forms; Real caches made out of maps (see the registration pattern above) sync.Pool and, and quasi constant-variables that ameliorate The cost of a compilation.

As a example the package have crypto/ecsda a zr type whose Read method zeros any buffer passed to it. The package keeps a single instance of zr around because it's embedded in other structs as an io.Reader , potentially ESCAP ing to the heap each time it is instantiated.

Package ECDSA type ZR struct {        io. reader}//read replaces the contents of DST with Zeros.func (Z-*ZR) Read (DST []byte) (n int, err error) {for        I: = Ran GE DST {                dst[i] = 0        }        return Len (DST), Nil}var zeroreader = &zr{}

However zr doesn ' t embed an io.Reader , it 's io.Reader A, so the unused zr.Reader field could be eliminated, giving zr A width of zero. In my testing the modified type can be created directly where it is used without performance regression.

        CSPRNG: = cipher. streamreader{                r:zr{},                s:cipher. NEWCTR (block, []byte (Aesiv)),        }

Perhaps some of the caching decision could be revisited as the inlining and escape analysis options available to the Compi Ler has improved significantly since the standard library is first written.

Tables

The last major use of common use of the private package scoped variables are for tables, as seen in the unicode , crypto/* , and packages. These tables either encode constant data in the form of arrays of an integer types, or less commonly simple structs and maps.

Replacing package scoped variables with constants would require a language change along the lines of #20443. So, fundamentally, providing there is no-i-modify those tables at run time, they was probably a reasonable exception To this proposal.

A Bridge Too Far

Even though this post was just a thought experiment, it's clear that forbidding all package scoped variables is t Oo draconian to be workable as a language precept. Addressing the bespoke uses of private var usage may prove impractical from a performance standpoint, would is AK Pinning a "kick me" sign to ones back and inviting all the Go haters to take a free swing.

However, I believe there is a few concrete recommendations that can is drawn from this exercise, without going to the EXT Reme of changing the language spec.

• Firstly, public var declarations should is eschewed. This is a controversial conclusion, and not one, that's unique to Go. The singleton pattern is discouraged, and a unadorned public variable so can be changed at any time by all party that K Nows its name should is a design, and concurrency, Red flag.
• Secondly, where public package var declarations is used, the type of those variables should is carefully constructed to Expose as little surface area as possible. It should not being the default to take a type expected to being used on a per instance basis, and assign it to a package scoped Variable.

Private variable declarations is more nuanced, but certain patterns can be observed:

• Private variables with public setters, which I labelled registries, has the same effect on the overall program Design as their public counterparts. Rather than registering dependencies globally, they should instead BES passed in during declaration using a constructor fun ction, compact literal, config structure, or option function.
• Caches of []byte VARs can often be expressed as const s at no performance cost.  don ' t f Orget The compiler is pretty good at avoiding  string ([]byte) conversions where they don ' t escape the fun Ction call.
• Private variables The tables, like the Unicode  package, is an unavoidable consequence of the Lack of a constant array type. As long as they is unexported, and do not expose any mutate them, they can is considered effectively constant for The purpose of this discussion.

The bottom line; Think long and hard about adding package scoped variables that is mutated during the operation of your program. It May is a sign of that you ' ve introduced Magic Global state.

Related Posts:

1. On declaring variables
2. How to include C code in your Go package
3. A Whirlwind tour of Go ' s runtime environment variables
4. Stack traces and the errors package