embed: go:embed produces error when "go.mod" has go version < 1.16

[lestrrat]

What version of Go are you using (go version)?

$ go version
go version go1.16rc1 darwin/arm64

Does this issue reproduce with the latest release?

with the rc release? yes.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
go version go1.16rc1 darwin/arm64
lestrrat@finch jwk % go env
GO111MODULE="on"
GOARCH="arm64"
GOBIN=""
GOCACHE="/Users/lestrrat/Library/Caches/go-build"
GOENV="/Users/lestrrat/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="arm64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/lestrrat/dev/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/lestrrat/dev"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_arm64"
GOVCS=""
GOVERSION="go1.16rc1"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/lestrrat/dev/src/github.com/lestrrat-go/jwx/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/vb/gf1mcqdd2s509731hrnnvkjm0000gn/T/go-build3140323037=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Using github.com/lestrrat-go/jwx@09b6dea (note that it's at a tracking branch)

lestrrat@finch jwx % cd jwk
lestrrat@finch jwk % go test               

What did you expect to see?

lestrrat@finch jwx % cd jwk
lestrrat@finch jwk % go test   
PASS
ok  	github.com/lestrrat-go/jwx/jwk	5.437s
lestrrat@finch jwk % 

What did you see instead?

lestrrat@finch jwx % cd jwk
lestrrat@finch jwk % go test               
# github.com/lestrrat-go/jwx/jwk_test [github.com/lestrrat-go/jwx/jwk.test]
./fs_test.go:13:3: go:embed requires go1.16 or later (-lang was set to go1.13; check go.mod)
FAIL	github.com/lestrrat-go/jwx/jwk [build failed]
lestrrat@finch jwk % 

Discussion

My go.mod contains the following

lestrrat@finch jwk % cat ../go.mod
module github.com/lestrrat-go/jwx

go 1.13

require (
	github.com/goccy/go-json v0.3.4
	github.com/lestrrat-go/backoff/v2 v2.0.7
	github.com/lestrrat-go/codegen v1.0.0
	github.com/lestrrat-go/httpcc v1.0.0
	github.com/lestrrat-go/iter v1.0.0
	github.com/lestrrat-go/option v1.0.0
	github.com/lestrrat-go/pdebug/v3 v3.0.1
	github.com/pkg/errors v0.9.1
	github.com/stretchr/testify v1.6.1
	golang.org/dl v0.0.0-20210128152142-7744cb1878f1 // indirect
	golang.org/x/crypto v0.0.0-20201217014255-9d1352758620
	golang.org/x/mod v0.4.1 // indirect
	golang.org/x/tools v0.0.0-20210114065538-d78b04bdf963
)

If I change the line go 1.13 to go 1.16, then it works on go1.16rc1

lestrrat@finch jwk % go test
# github.com/lestrrat-go/jwx/jwk_test [github.com/lestrrat-go/jwx/jwk.test]
./fs_test.go:13:3: go:embed requires go1.16 or later (-lang was set to go1.13; check go.mod)
FAIL	github.com/lestrrat-go/jwx/jwk [build failed]
lestrrat@finch jwk % go mod edit -go 1.16
lestrrat@finch jwk % go test             
PASS
ok  	github.com/lestrrat-go/jwx/jwk	5.648s
lestrrat@finch jwk % 

go1.16beta1 worked just fine.

lestrrat@finch jwk % go install golang.org/dl/go1.16beta1
lestrrat@finch jwk % go1.16beta1 download
go1.16beta1: already downloaded in /Users/lestrrat/sdk/go1.16beta1
lestrrat@finch jwk % go1.16beta1 test .
ok  	github.com/lestrrat-go/jwx/jwk	6.066s
lestrrat@finch jwk % 

Shouldn't features like this just depend on the runtime version, and not contents of go.mod?

[ianlancetaylor]

This is intentional, because a //go:embed comment will only work correctly with Go 1.16 or later. A //go:embed comment requires importing the "embed" package, which is only available in Go 1.16. So compiling a package with an earlier language version will fail.

[lestrrat]

Even with "// +build go1.16" protecting it? -- just making sure, because while I understand go:embed not working with go < 1.16, I would think that if the source code was not eligible for compilation via build tags, it should be ignored.

And if so, I would appreciate a doc explaining that go will not consider build tags in that case. My skimming through the release notes or the embed docs didn't produce such explanation. I can certainly provide a patch if need be.

[zigo101]

By my understanding,

• "// +build go1.16" specify the exact minimum Go Toolchain version needed to compile a source file.
• "go 1.N" directive in a "go.mod" file specifies the features the containing module could use (https://golang.org/ref/mod#go-mod-file-go). It likes the "-lang=go1.N" compilation option, but for the source files in a specified module. So
  • it doesn't specify the minimum Go Toolchain version to compile the module. Higher or lower Toolchain versions may be used if the desired features are supported these Toolchain versions.
  • it doesn't cover other modules in a program. https://play.golang.org/p/YlMYLWsP_0S

[lestrrat]

I have learned that go mod tidy on 1.15 which uses io/fs, regardless of build tags, will fail. Fair enough! Closing.

[mvdan]

This is intentional, because a //go:embed comment will only work correctly with Go 1.16 or later. A //go:embed comment requires importing the "embed" package, which is only available in Go 1.16. So compiling a package with an earlier language version will fail.

@ianlancetaylor while I get the current check against -lang, I'm not sure I understand why it was done this way. If a module declares go 1.16 in its go.mod, and then only imports the "embed" package in a Go file guarded by // +build go1.16, why should that fail?

[ianlancetaylor]

@mvdan Fair point. I'll reopen.

[mvdan]

I made a typo, by the way. I meant that the go.mod declares an older go language version, like go 1.15.

[ianlancetaylor]

Thanks, understood.

[zikaeroh]

I have learned that go mod tidy on 1.15 which uses io/fs, regardless of build tags, will fail. Fair enough! Closing.

For reference, that's #40067, and it even affects users of modules that try and use new standard library packages.

No module can successfully guard its use of a new standard library package with a build tag and have it not still break downstream users with older Go versions. (I expect this to start happening pretty often now that everyone is excited for embed, unlike when maphash was added.)