Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime: decorate anonymous memory mappings #71546

Closed
L3n41c opened this issue Feb 3, 2025 · 5 comments
Closed

runtime: decorate anonymous memory mappings #71546

L3n41c opened this issue Feb 3, 2025 · 5 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FixPending Issues that have a fix which has not yet been reviewed or submitted. Implementation Issues describing a semantics-preserving change to the Go implementation. NeedsFix The path to resolution is known, but the work has not been done.
Milestone
Go1.25

Comments

@L3n41c
Copy link
Contributor

L3n41c commented Feb 3, 2025 •
edited
Loading

Linux 5.17 has introduced an API to name the anonymous memory areas: prctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME, …).

This API is already leveraged by the glibc to give an insight of what each anonymous memory area is used for.
See:

Thanks to that, /proc/<pid>/maps and /proc/<pid>/smaps provide information about what each anonymous memory area is used for:

lenaic@coruscant:~$ GLIBC_TUNABLES=glibc.mem.decorate_maps=1 cat /proc/self/maps 
603d00bf8000-603d00bfa000 r--p 00000000 00:21 47929385                   /usr/bin/cat
603d00bfa000-603d00bff000 r-xp 00002000 00:21 47929385                   /usr/bin/cat
603d00bff000-603d00c01000 r--p 00007000 00:21 47929385                   /usr/bin/cat
603d00c01000-603d00c02000 r--p 00008000 00:21 47929385                   /usr/bin/cat
603d00c02000-603d00c03000 rw-p 00009000 00:21 47929385                   /usr/bin/cat
603d01d05000-603d01d26000 rw-p 00000000 00:00 0                          [heap]
7091f2800000-7091f2aec000 r--p 00000000 00:21 48077669                   /usr/lib/locale/locale-archive
7091f2cd1000-7091f2cd4000 rw-p 00000000 00:00 0                          [anon: glibc: loader malloc]
7091f2cd4000-7091f2cf8000 r--p 00000000 00:21 48076955                   /usr/lib/libc.so.6
7091f2cf8000-7091f2e69000 r-xp 00024000 00:21 48076955                   /usr/lib/libc.so.6
7091f2e69000-7091f2eb8000 r--p 00195000 00:21 48076955                   /usr/lib/libc.so.6
7091f2eb8000-7091f2ebc000 r--p 001e3000 00:21 48076955                   /usr/lib/libc.so.6
7091f2ebc000-7091f2ebe000 rw-p 001e7000 00:21 48076955                   /usr/lib/libc.so.6
7091f2ebe000-7091f2ec6000 rw-p 00000000 00:00 0 
7091f2ec6000-7091f2ec8000 rw-p 00000000 00:00 0                          [anon: glibc: loader malloc]
7091f2ed2000-7091f2f14000 rw-p 00000000 00:00 0                          [anon: glibc: malloc]
7091f2f14000-7091f2f16000 r--p 00000000 00:00 0                          [vvar]
7091f2f16000-7091f2f18000 r--p 00000000 00:00 0                          [vvar_vclock]
7091f2f18000-7091f2f1a000 r-xp 00000000 00:00 0                          [vdso]
7091f2f1a000-7091f2f1b000 r--p 00000000 00:21 48076946                   /usr/lib/ld-linux-x86-64.so.2
7091f2f1b000-7091f2f44000 r-xp 00001000 00:21 48076946                   /usr/lib/ld-linux-x86-64.so.2
7091f2f44000-7091f2f4f000 r--p 0002a000 00:21 48076946                   /usr/lib/ld-linux-x86-64.so.2
7091f2f4f000-7091f2f51000 r--p 00035000 00:21 48076946                   /usr/lib/ld-linux-x86-64.so.2
7091f2f51000-7091f2f52000 rw-p 00037000 00:21 48076946                   /usr/lib/ld-linux-x86-64.so.2
7091f2f52000-7091f2f53000 rw-p 00000000 00:00 0 
7ffe15321000-7ffe15342000 rw-p 00000000 00:00 0                          [stack]
ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0                  [vsyscall]

See the lines with [anon: glibc: …]

It could be valuable to have something similar for Go programs.
One use-case that comes to my mind is profiling multi-languages programs that are mixing Go and native C libraries.
With a pure 100% Go program, knowing where the memory allocations are coming from can be done with pprof.
But pprof profiles only the Go code.
In a program mixing Go and C libraries, we first need to know if the memory allocation we’re hunting is coming from the Go part or the C part.
Labeling the anonymous memory regions would help for that.

I have already drafted a proof of concept in CL #646095: https://go-review.googlesource.com/c/go/+/646095 that makes the following Go program:

package main

import (
        "fmt"
        "log"
        "os"
)

/*
#include <stdlib.h>

void f(void)
{
  (void)malloc(1024*1024*1024);
}
*/
import "C"

func main() {
        C.f()

        data, err := os.ReadFile("/proc/self/maps")
        if err != nil {
                log.Fatal(err)
        }
        fmt.Println(string(data))
}

produce this output:

lenaic@coruscant:~/doc/devel/perso/test_vma_name$ GLIBC_TUNABLES=glibc.mem.decorate_maps=1 ~/doc/devel/open-source/go/bin/go run .
00400000-00402000 r--p 00000000 00:47 738                                /tmp/go-build3615860867/b001/exe/test_vma_name
00402000-004a4000 r-xp 00002000 00:47 738                                /tmp/go-build3615860867/b001/exe/test_vma_name
004a4000-00574000 r--p 000a4000 00:47 738                                /tmp/go-build3615860867/b001/exe/test_vma_name
00574000-00575000 r--p 00173000 00:47 738                                /tmp/go-build3615860867/b001/exe/test_vma_name
00575000-00580000 rw-p 00174000 00:47 738                                /tmp/go-build3615860867/b001/exe/test_vma_name
00580000-005a4000 rw-p 00000000 00:00 0 
05105000-05126000 rw-p 00000000 00:00 0                                  [heap]
c000000000-c000400000 rw-p 00000000 00:00 0                              [anon: Go: map]
c000400000-c004000000 ---p 00000000 00:00 0                              [anon: Go: reserve]
774f2c000000-774f2c021000 rw-p 00000000 00:00 0                          [anon: glibc: malloc arena]
774f2c021000-774f30000000 ---p 00000000 00:00 0 
774f34000000-774f34021000 rw-p 00000000 00:00 0                          [anon: glibc: malloc arena]
774f34021000-774f38000000 ---p 00000000 00:00 0 
774f38000000-774f38021000 rw-p 00000000 00:00 0                          [anon: glibc: malloc arena]
774f38021000-774f3c000000 ---p 00000000 00:00 0 
774f3c000000-774f3c021000 rw-p 00000000 00:00 0                          [anon: glibc: malloc arena]
774f3c021000-774f40000000 ---p 00000000 00:00 0 
774f433ff000-774f43400000 ---p 00000000 00:00 0 
774f43400000-774f43c00000 rw-p 00000000 00:00 0                          [anon: glibc: pthread stack: 4120]
774f44000000-774f44021000 rw-p 00000000 00:00 0                          [anon: glibc: malloc arena]
774f44021000-774f48000000 ---p 00000000 00:00 0 
774f4ac4b000-774f4ac4c000 ---p 00000000 00:00 0 
774f4ac4c000-774f4b44c000 rw-p 00000000 00:00 0                          [anon: glibc: pthread stack: 4122]
774f4b44c000-774f4b44d000 ---p 00000000 00:00 0 
774f4b44d000-774f4bc4d000 rw-p 00000000 00:00 0                          [anon: glibc: pthread stack: 4121]
774f4bc4d000-774f4bc8d000 rw-p 00000000 00:00 0                          [anon: Go: persistent alloc]
774f4bc8d000-774f4bc8e000 ---p 00000000 00:00 0 
774f4bc8e000-774f4c48e000 rw-p 00000000 00:00 0                          [anon: glibc: pthread stack: 4119]
774f4c48e000-774f4c4ce000 rw-p 00000000 00:00 0                          [anon: Go: persistent alloc]
774f4c4ce000-774f4c4cf000 ---p 00000000 00:00 0 
774f4c4cf000-774f4cccf000 rw-p 00000000 00:00 0                          [anon: glibc: pthread stack: 4118]
774f4cccf000-774f4ccdf000 rw-p 00000000 00:00 0                          [anon: Go: arena]
774f4ccdf000-774f4ccef000 rw-p 00000000 00:00 0                          [anon: Go: span]
774f4ccef000-774f4cdef000 rw-p 00000000 00:00 0                          [anon: Go: page alloc]
774f4cdef000-774f4ce00000 rw-p 00000000 00:00 0                          [anon: Go: persistent alloc]
774f4ce00000-774f4ee00000 rw-p 00000000 00:00 0                          [anon: Go: heap arena map]
774f4ee00000-774f5ef80000 ---p 00000000 00:00 0                          [anon: Go: reserve]
774f5ef80000-774f5ef81000 rw-p 00000000 00:00 0                          [anon: Go: map]
774f5ef81000-774f7ef80000 ---p 00000000 00:00 0                          [anon: Go: reserve]
774f7ef80000-774f7ef81000 rw-p 00000000 00:00 0                          [anon: Go: map]
774f7ef81000-774f90e30000 ---p 00000000 00:00 0                          [anon: Go: reserve]
774f90e30000-774f90e31000 rw-p 00000000 00:00 0                          [anon: Go: map]
774f90e31000-774f93206000 ---p 00000000 00:00 0                          [anon: Go: reserve]
774f93206000-774f93207000 rw-p 00000000 00:00 0                          [anon: Go: map]
774f93207000-774f93600000 ---p 00000000 00:00 0                          [anon: Go: reserve]
774f93605000-774f93685000 ---p 00000000 00:00 0                          [anon: Go: reserve]
774f93685000-774f93686000 rw-p 00000000 00:00 0                          [anon: Go: map]
774f93686000-774f93705000 ---p 00000000 00:00 0                          [anon: Go: reserve]
774f93705000-774f93725000 rw-p 00000000 00:00 0                          [anon: Go: map]
774f93725000-774f93728000 rw-p 00000000 00:00 0                          [anon: glibc: loader malloc]
774f93728000-774f9374c000 r--p 00000000 00:21 48076955                   /usr/lib/libc.so.6
774f9374c000-774f938bd000 r-xp 00024000 00:21 48076955                   /usr/lib/libc.so.6
774f938bd000-774f9390c000 r--p 00195000 00:21 48076955                   /usr/lib/libc.so.6
774f9390c000-774f93910000 r--p 001e3000 00:21 48076955                   /usr/lib/libc.so.6
774f93910000-774f93912000 rw-p 001e7000 00:21 48076955                   /usr/lib/libc.so.6
774f93912000-774f9391a000 rw-p 00000000 00:00 0 
774f9391a000-774f9391c000 rw-p 00000000 00:00 0                          [anon: glibc: loader malloc]
774f93928000-774f93968000 rw-p 00000000 00:00 0                          [anon: Go: persistent alloc]
774f93968000-774f9396a000 r--p 00000000 00:00 0                          [vvar]
774f9396a000-774f9396c000 r--p 00000000 00:00 0                          [vvar_vclock]
774f9396c000-774f9396e000 r-xp 00000000 00:00 0                          [vdso]
774f9396e000-774f9396f000 r--p 00000000 00:21 48076946                   /usr/lib/ld-linux-x86-64.so.2
774f9396f000-774f93998000 r-xp 00001000 00:21 48076946                   /usr/lib/ld-linux-x86-64.so.2
774f93998000-774f939a3000 r--p 0002a000 00:21 48076946                   /usr/lib/ld-linux-x86-64.so.2
774f939a3000-774f939a5000 r--p 00035000 00:21 48076946                   /usr/lib/ld-linux-x86-64.so.2
774f939a5000-774f939a6000 rw-p 00037000 00:21 48076946                   /usr/lib/ld-linux-x86-64.so.2
774f939a6000-774f939a7000 rw-p 00000000 00:00 0 
7fff8a80a000-7fff8a82b000 rw-p 00000000 00:00 0                          [stack]
ffffffffff600000-ffffffffff601000 --xp 00000000 00:00 0                  [vsyscall]

See the lines with [anon: Go: …]

Thanks to CL #646095, we can know track which memory has been allocated by the Go runtime vs. by the C part.

Rather than labeling all anonymous areas with the same Go label, I decided to mimic the glibc behavior and to provide more information about which part of the Go runtime allocated them. But the exact labels can probably be improved.
@gopherbot gopherbot added this to the Proposal milestone Feb 3, 2025
@gopherbot
Copy link
Contributor

Change https://go.dev/cl/646095 mentions this issue: runtime: decorate anonymous memory mappings

@gabyhelp
Copy link

gabyhelp commented Feb 3, 2025

Related Issues

Related Code Changes

(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)

@gabyhelp gabyhelp added the Implementation Issues describing a semantics-preserving change to the Go implementation. label Feb 3, 2025
@ianlancetaylor
Copy link
Member

I don't think this has to go through the proposal process.

@ianlancetaylor ianlancetaylor changed the title proposal: runtime: decorate anonymous memory mappings runtime: decorate anonymous memory mappings Feb 3, 2025
@ianlancetaylor ianlancetaylor added compiler/runtime Issues related to the Go compiler and/or runtime. and removed Proposal labels Feb 3, 2025
@ianlancetaylor ianlancetaylor modified the milestones: Proposal, Backlog Feb 3, 2025
@mknyszek mknyszek added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. help wanted labels Feb 5, 2025
@dmitshur dmitshur modified the milestones: Backlog, Go1.25 Feb 20, 2025
@dmitshur dmitshur added NeedsFix The path to resolution is known, but the work has not been done. FixPending Issues that have a fix which has not yet been reviewed or submitted. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Feb 20, 2025
@dmitshur dmitshur moved this from Todo to All-But-Submitted in Go Compiler / Runtime Feb 20, 2025
@github-project-automation github-project-automation bot moved this from All-But-Submitted to Done in Go Compiler / Runtime Mar 4, 2025
@prattmic
Copy link
Member

prattmic commented Mar 7, 2025

@L3n41c Do you know which Linux distros actually set CONFIG_ANON_VMA_NAME to enable this feature?

I found that neither Debian nor Container-Optimized OS (which most of our Linux CI builders run) enable this feature, so PR_SET_VMA_ANON_NAME simply returns EINVAL.

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/655895 mentions this issue: internal/godebugs: add decoratemappings as an opaque godebug setting

gopherbot pushed a commit that referenced this issue Mar 10, 2025
@prattmic @gopherbot
internal/godebugs: add decoratemappings as an opaque godebug setting
c40a373 
This adds a new godebug to control whether the runtime applies the
anonymous memory mapping annotations added in https://go.dev/cl/646095.
It is enabled by default.

This has several effects:

* The feature is only enabled by default when the main go.mod has go >=
  1.25.
* This feature can be disabled with GODEBUG=decoratemappings=0, or the
  equivalents in go.mod or package main. See https://go.dev/doc/godebug.
* As an opaque setting, this option will not appear in runtime/metrics.
* This setting is non-atomic, so it cannot be changed after startup.

I am not 100% sure about my decision for the last two points.

I've made this an opaque setting because it affects every memory mapping
the runtime performs. Thus every mapping would report "non-default
behavior", which doesn't seem useful.

This setting could trivially be atomic and allow changes at run time,
but those changes would only affect future mappings. That seems
confusing and not helpful. On the other hand, going back to annotate or
unannotate every previous mapping when the setting changes is
unwarranted complexity.

For #71546.

Change-Id: I6a6a636c5ad551d76691cba2a6f668d5cff0e352
Reviewed-on: https://go-review.googlesource.com/c/go/+/655895
Reviewed-by: Michael Knyszek <mknyszek@google.com>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
LUCI-TryBot-Result: Go LUCI <golang-scoped@luci-project-accounts.iam.gserviceaccount.com>
Auto-Submit: Michael Pratt <mpratt@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FixPending Issues that have a fix which has not yet been reviewed or submitted. Implementation Issues describing a semantics-preserving change to the Go implementation. NeedsFix The path to resolution is known, but the work has not been done.
Projects
Go Compiler / Runtime
Status: Done
Milestone
Go1.25
Development

No branches or pull requests
7 participants
@mknyszek @prattmic @L3n41c @dmitshur @ianlancetaylor @gopherbot @gabyhelp