otel: Add multi-standard propagator and fix gcp observability (#240)

* otel: Add gcp propagator

* tests

* go mod

* add debug logging

* Adds span force flushing on close

Ensures all spans are exported before shutdown to prevent data loss, especially in environments like Cloud Run where containers can terminate abruptly.

Introduces a `ForceFlush` method to the `Observer` interface and its implementations, which immediately exports all pending spans.
The `Close` method now calls `ForceFlush` before shutting down the tracer provider.
A default timeout for force flushing is introduced.

* Adds ForceFlush method to Sentry provider

This change adds a `ForceFlush` method to the Sentry provider.
This method allows for explicitly flushing the Sentry client, ensuring
that any pending events are sent to Sentry. This is useful in scenarios
where immediate event delivery is required, such as before application shutdown.

* Adds simple span export option

Allows users to enable immediate span export without batching, providing real-time visibility at the cost of increased network overhead.

This introduces a `use_simple_span` configuration option, enabling spans to be exported immediately when they finish, instead of being batched.

* linting issue

* Improves hostname detection for service identification

Enhances hostname detection by prioritizing GCP metadata service,
falling back to OS hostname, and finally using a default identifier.
This ensures reliable service identification in multi-server and
cloud environments.

Removes unnecessary logging from span start and finish.

* Enhances hostname detection for services

Improves service identification by prioritizing environment variables for hostname detection.

This change replaces reliance on GCP metadata service lookups with a more robust approach using environment variables. It constructs a meaningful hostname based on variables such as service name, revision, and project ID, improving accuracy and avoiding unnecessary external requests. It falls back to system hostname or a default "unknown-host" if no suitable environment variables are found. The hostname initialization is performed only once to cache results.

* Adds span counting feature

Introduces span counting to track and log the number of spans within a trace.

This enables better observability by providing insights into trace complexity and performance.
It introduces a new `SpanCountingProcessor` that wraps the existing span processor.
The feature can be enabled or disabled via configuration.

Adds a YAML parsing test for `SpanCountingConfig`.

Updates hostname logic to avoid duplication when the revision already contains the service name.

* Enables GCP sampler for cloud environments

Configures the observability provider to prioritize local sampling decisions in cloud environments like Cloud Run to prevent trace loss.

Logs additional debug information for configuration values.

* Enables trace propagation configuration

Adds support for configuring trace propagation formats (W3C, B3, Jaeger).

This allows users to specify which propagation formats should be
used when sending and receiving traces.  Defaults to W3C if no
formats are specified. This ensures interoperability with
different tracing systems.

Also includes integration tests and examples to verify proper
propagation of trace context with different formats.

* Configures propagation

Configures OpenTelemetry propagation based on user-defined formats or defaults to W3C.

Adds debug logging to track propagation configuration and span counting settings, aiding in troubleshooting and verification.

* Refactors observability configuration and hostname detection

This commit streamlines the observability configuration by removing the span counting feature and simplifying the propagation defaults.

It introduces a centralized hostname detection mechanism to improve service identification across different GCP environments. The previous hostname detection logic was moved to `pkg/zobservability/hostname.go` for reuse and consistency.

The propagation default is now B3 because it's the only one supported by GCP+Signoz.

* Clean up

* clean up

Author: lucaslopezf

go.mod

@@ -25,6 +25,8 @@ require (
 	github.com/uptrace/opentelemetry-go-extra/otelgorm v0.3.2
 	go.opentelemetry.io/contrib/bridges/otelzap v0.12.0
 	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.62.0
+	go.opentelemetry.io/contrib/propagators/b3 v1.37.0
+	go.opentelemetry.io/contrib/propagators/jaeger v1.37.0
 	go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc v0.13.0
 	go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploghttp v0.13.0
 	go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.37.0

go.sum

@@ -297,6 +297,10 @@ go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.6
 go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.61.0/go.mod h1:snMWehoOh2wsEwnvvwtDyFCxVeDAODenXHtn5vzrKjo=
 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.62.0 h1:Hf9xI/XLML9ElpiHVDNwvqI0hIFlzV8dgIr35kV1kRU=
 go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.62.0/go.mod h1:NfchwuyNoMcZ5MLHwPrODwUF1HWCXWrL31s8gSAdIKY=
+go.opentelemetry.io/contrib/propagators/b3 v1.37.0 h1:0aGKdIuVhy5l4GClAjl72ntkZJhijf2wg1S7b5oLoYA=
+go.opentelemetry.io/contrib/propagators/b3 v1.37.0/go.mod h1:nhyrxEJEOQdwR15zXrCKI6+cJK60PXAkJ/jRyfhr2mg=
+go.opentelemetry.io/contrib/propagators/jaeger v1.37.0 h1:pW+qDVo0jB0rLsNeaP85xLuz20cvsECUcN7TE+D8YTM=
+go.opentelemetry.io/contrib/propagators/jaeger v1.37.0/go.mod h1:x7bd+t034hxLTve1hF9Yn9qQJlO/pP8H5pWIt7+gsFM=
 go.opentelemetry.io/otel v1.37.0 h1:9zhNfelUvx0KBfu/gb+ZgeAfAgtWrfHJZcAqFC228wQ=
 go.opentelemetry.io/otel v1.37.0/go.mod h1:ehE/umFRLnuLa/vSccNq9oS1ErUlkkK71gMcN34UG8I=
 go.opentelemetry.io/otel/exporters/otlp/otlplog/otlploggrpc v0.13.0 h1:z6lNIajgEBVtQZHjfw2hAccPEBDs+nx58VemmXWa2ec=

pkg/zobservability/config.go

@@ -4,6 +4,11 @@ import (
 	"fmt"
 )
 
+// PropagationConfig controls trace propagation formats
+type PropagationConfig struct {
+	Formats []string `yaml:"formats" mapstructure:"formats"` // ["w3c", "b3", "b3-single", "jaeger"]
+}
+
 // Config holds configuration for all observability features (tracing, logging, metrics)
 type Config struct {
 	Provider     string            `yaml:"provider" mapstructure:"provider"`
@@ -15,6 +20,7 @@ type Config struct {
 	SampleRate   float64           `yaml:"sample_rate" mapstructure:"sample_rate"` // Common sampling rate
 	Middleware   MiddlewareConfig  `yaml:"middleware" mapstructure:"middleware"`
 	Metrics      MetricsConfig     `yaml:"metrics" mapstructure:"metrics"`             // Metrics configuration
+	Propagation  PropagationConfig `yaml:"propagation" mapstructure:"propagation"`     // Trace propagation configuration
 	CustomConfig map[string]string `yaml:"custom_config" mapstructure:"custom_config"` // Provider-specific configuration
 }
 
@@ -60,4 +66,9 @@ func (c *Config) SetDefaults() {
 	if c.Metrics.Provider == "" {
 		c.Metrics = DefaultMetricsConfig()
 	}
+
+	// Set propagation defaults
+	if len(c.Propagation.Formats) == 0 {
+		c.Propagation.Formats = []string{PropagationB3} // Default to B3 because is the only one supported by GCP+Signoz
+	}
 }

pkg/zobservability/config_test.go

@@ -308,3 +308,203 @@ func TestConfig_WhenSetDefaultsMultipleTimes_ShouldBeIdempotent(t *testing.T) {
 	assert.Equal(t, firstSampleRate, config.SampleRate)
 	assert.Equal(t, firstCaptureErrors, config.Middleware.CaptureErrors)
 }
+
+// Tests for PropagationConfig
+
+func TestPropagationConfig(t *testing.T) {
+	tests := []struct {
+		name           string
+		config         PropagationConfig
+		expectedLength int
+		expectedFirst  string
+	}{
+		{
+			name: "single format",
+			config: PropagationConfig{
+				Formats: []string{PropagationB3},
+			},
+			expectedLength: 1,
+			expectedFirst:  PropagationB3,
+		},
+		{
+			name: "multiple formats",
+			config: PropagationConfig{
+				Formats: []string{PropagationB3, PropagationW3C, PropagationJaeger},
+			},
+			expectedLength: 3,
+			expectedFirst:  PropagationB3,
+		},
+		{
+			name: "empty formats",
+			config: PropagationConfig{
+				Formats: []string{},
+			},
+			expectedLength: 0,
+		},
+		{
+			name: "nil formats",
+			config: PropagationConfig{
+				Formats: nil,
+			},
+			expectedLength: 0,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			assert.Len(t, tt.config.Formats, tt.expectedLength)
+			if tt.expectedLength > 0 {
+				assert.Equal(t, tt.expectedFirst, tt.config.Formats[0])
+			}
+		})
+	}
+}
+
+func TestConfig_WhenSetDefaultsPropagationFormats_ShouldSetB3Default(t *testing.T) {
+	tests := []struct {
+		name           string
+		initialConfig  Config
+		expectedLength int
+		expectedFirst  string
+	}{
+		{
+			name: "empty propagation formats get B3 default",
+			initialConfig: Config{
+				Propagation: PropagationConfig{
+					Formats: []string{},
+				},
+			},
+			expectedLength: 1,
+			expectedFirst:  PropagationB3,
+		},
+		{
+			name: "nil propagation formats get B3 default",
+			initialConfig: Config{
+				Propagation: PropagationConfig{
+					Formats: nil,
+				},
+			},
+			expectedLength: 1,
+			expectedFirst:  PropagationB3,
+		},
+		{
+			name: "existing formats are preserved",
+			initialConfig: Config{
+				Propagation: PropagationConfig{
+					Formats: []string{PropagationB3, PropagationJaeger},
+				},
+			},
+			expectedLength: 2,
+			expectedFirst:  PropagationB3,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			config := tt.initialConfig
+			config.SetDefaults()
+
+			assert.Len(t, config.Propagation.Formats, tt.expectedLength)
+			if tt.expectedLength > 0 {
+				assert.Equal(t, tt.expectedFirst, config.Propagation.Formats[0])
+			}
+		})
+	}
+}
+
+func TestConfig_WhenValidateWithPropagation_ShouldSucceed(t *testing.T) {
+	tests := []struct {
+		name        string
+		config      Config
+		expectError bool
+	}{
+		{
+			name: "valid config with propagation",
+			config: Config{
+				Provider:    ProviderSigNoz,
+				Enabled:     true,
+				Environment: "test",
+				Address:     "localhost:4317",
+				Propagation: PropagationConfig{
+					Formats: []string{PropagationB3},
+				},
+			},
+			expectError: false,
+		},
+		{
+			name: "valid config with multiple propagation formats",
+			config: Config{
+				Provider:    ProviderSigNoz,
+				Enabled:     true,
+				Environment: "test",
+				Address:     "localhost:4317",
+				Propagation: PropagationConfig{
+					Formats: []string{PropagationB3, PropagationW3C, PropagationJaeger},
+				},
+			},
+			expectError: false,
+		},
+		{
+			name: "valid config with empty propagation formats",
+			config: Config{
+				Provider:    ProviderSigNoz,
+				Enabled:     true,
+				Environment: "test",
+				Address:     "localhost:4317",
+				Propagation: PropagationConfig{
+					Formats: []string{},
+				},
+			},
+			expectError: false,
+		},
+		{
+			name: "disabled config with propagation doesn't validate",
+			config: Config{
+				Provider:    ProviderSigNoz,
+				Enabled:     false,
+				Environment: "",
+				Address:     "",
+				Propagation: PropagationConfig{
+					Formats: []string{PropagationB3},
+				},
+			},
+			expectError: false,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			err := tt.config.Validate()
+			if tt.expectError {
+				assert.Error(t, err)
+			} else {
+				assert.NoError(t, err)
+			}
+		})
+	}
+}
+
+func TestConfig_WhenPropagationIntegration_ShouldWorkEndToEnd(t *testing.T) {
+	// Test that a full configuration works end-to-end
+	config := Config{
+		Provider:    ProviderSigNoz,
+		Enabled:     true,
+		Environment: "test",
+		Release:     "1.0.0",
+		Address:     "localhost:4317",
+		SampleRate:  1.0,
+		Propagation: PropagationConfig{
+			Formats: []string{PropagationB3, PropagationW3C},
+		},
+	}
+
+	// Set defaults and validate
+	config.SetDefaults()
+	err := config.Validate()
+	assert.NoError(t, err)
+
+	// Check that propagation formats are preserved
+	assert.Len(t, config.Propagation.Formats, 2)
+	assert.Equal(t, PropagationB3, config.Propagation.Formats[0])
+	assert.Equal(t, PropagationW3C, config.Propagation.Formats[1])
+}

pkg/zobservability/constants.go

@@ -6,6 +6,14 @@ const (
 	ProviderSigNoz = "signoz"
 )
 
+// Propagation format constants
+const (
+	PropagationW3C      = "w3c"
+	PropagationB3       = "b3"
+	PropagationB3Single = "b3-single"
+	PropagationJaeger   = "jaeger"
+)
+
 // Environment constants for observability configuration
 const (
 	EnvironmentProduction  = "production"

pkg/zobservability/factory/factory.go

@@ -94,6 +94,12 @@ func newSigNozObserver(config *zobservability.Config, serviceName string) (zobse
 		ignoreParentSampling = strings.ToLower(ignoreParentStr) == signoz.ConfigKeyTrueValue
 	}
 
+	// Parse SimpleSpan configuration
+	useSimpleSpan := false
+	if simpleSpanStr, ok := config.CustomConfig[signoz.ConfigKeyUseSimpleSpan]; ok {
+		useSimpleSpan = strings.ToLower(simpleSpanStr) == signoz.ConfigKeyTrueValue
+	}
+
 	// Parse advanced batch configuration if present
 	// BatchConfig controls performance and batching behavior
 	batchConfig, err := parseBatchConfig(config.CustomConfig)
@@ -124,6 +130,8 @@ func newSigNozObserver(config *zobservability.Config, serviceName string) (zobse
 		BatchConfig:          batchConfig,          // Optional: nil means use defaults
 		ResourceConfig:       resourceConfig,       // Optional: nil means use defaults
 		IgnoreParentSampling: ignoreParentSampling, // Critical for Google Cloud Run deployments
+		UseSimpleSpan:        useSimpleSpan,        // Enable immediate span export
+		Propagation:          config.Propagation,   // Copy propagation configuration
 	}
 
 	return signoz.NewObserver(signozConfig)