runtime: Pinner doesn't work for global struct variables

[strager]

Go version

go version go1.24.1 darwin/arm64; go version go1.24.1 windows/arm64

Output of go env in your module/workspace:

AR='ar'
CC='clang'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='clang++'
GCCGO='gccgo'
GO111MODULE=''
GOARCH='arm64'
GOARM64='v8.0'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/Users/traduality/Library/Caches/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/Users/traduality/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/l4/p1_djmc55xq5rxlpmdxqfb8c0000gp/T/go-build2536227112=/tmp/go-build -gno-record-gcc-switches -fno-common'
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMOD='/Users/traduality/Projects/Traduality/go.mod'
GOMODCACHE='/Users/traduality/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/traduality/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/nix/store/jp3z5jp9gaxsw7fdzbqn29aabmrxq62j-go-1.24.1/share/go'
GOSUMDB='sum.golang.org'
GOTELEMETRY='on'
GOTELEMETRYDIR='/Users/traduality/Library/Application Support/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/nix/store/jp3z5jp9gaxsw7fdzbqn29aabmrxq62j-go-1.24.1/share/go/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.24.1'
GOWORK=''
PKG_CONFIG='pkg-config'

What did you do?

Run this program (requires CGo):

package main

import (
	"runtime"
)

/*
typedef struct {
   void *p;
} Engine;

void foo(Engine* e) {
   e->p = e;
}
*/
import "C"

type State struct {
	engine C.Engine
}

var state *State = &State{}

func main() {
	var pinner runtime.Pinner = runtime.Pinner{}
	pinner.Pin(state)
	defer pinner.Unpin()

	C.foo(&state.engine)
	C.foo(&state.engine)
}

What did you see happen?

The program crashes:

$ go run ./test/
panic: runtime error: cgo argument has Go pointer to unpinned Go pointer

goroutine 1 [running]:
main.main.func2(...)
        /Users/traduality/Projects/Traduality/beedo/beedovsm/cmd/test/main.go:30
main.main()
        /Users/traduality/Projects/Traduality/beedo/beedovsm/cmd/test/main.go:30 +0xcc
exit status 2

What did you expect to see?

I expect no crash.

If I move the initialization of state ( = &State{}) into main() (or init()), the crash goes away.

My guess at what's happening (probably wrong): &State{} in the variable initialization does not allocate the memory on the heap; it instead uses .data memory. runtime.Pinner.Pin() does nothing for .data memory, but runtime.cgoCheckArg() thinks that .data memory is unpinned.

[gabyhelp]

Related Issues

• runtime: CGo requires unrelated memory to be pinned #74117
• runtime: issue confused about the use of pin #64735 (closed)
• runtime: cgoCheckPointer fatal error: can't happen #70016 (closed)
• cmd/cgo: sometimes when passing a struct to C containing a go pointer to a struct field, the entire go struct is checked for go pointers instead of just the field #63460 (closed)
• cgo: wrapping a C pointer as uintptr gives "cgo argument has Go pointer to Go pointer" after some time #24269 (closed)
• runtime: way to check if pointer satisfies (*Pinner).Pin #62356 (closed)
• runtime: frame pointer check fails on arm64 for C-to-Go calls #59401
• x/tools/go/packages: nil pointer dereference #69524 (closed)
• runtime: runtime crashes, reproducer same as #20427 #36371 (closed)
• cmd/compile: value assigned to global fails to escape #29000 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[mknyszek]

Hm. The easy thing to do would be to consider linker-allocated objects as implicitly pinned, but this may cause headaches for someone who forgets to pin, but then later does not pass a linker-allocated object. That suggests we probably do want true pinner tracking for linker-allocated objects, which is doable but more work.

[mknyszek]

On the other hand, if it's always considered pinned, the problem will be quickly flagged by cgocheck when heap-allocated memory is passed through. So maybe it's not a big deal.

[mknyszek]

Oops, I misremembered. cgocheck just calls isPinned which returns true for linker-allocated objects already. So that's not the problem.

[mknyszek]

Ah hah! This is not actually based on the result of isPinned, but a failing of cgocheck. Because of the self-reference we get down to checking cgoCheckUnknownPointer which is conservative with respect to linker-allocated data. This is because we do not have any metadata about linker-allocated object boundaries. The GC doesn't need it, because it always scans all linker-allocated memory. This is the relevant code: https://cs.opensource.google/go/go/+/master:src/runtime/cgocall.go;l=744;drc=577bb3d0ce576b2ca311e58dd942f189838b80fc.

Without adding in these object boundaries and a table to look them up, I think cgocheck is right to be conservative here. This is fixable but not without a non-trivial amount of compiler work, probably.