Skip to content

Commit 32251e3

Browse files
PROFeNoMbwoebi
PROFeNoM
and
bwoebi
authored
Re-enable code coverage using codecov (#2464)
* Explicitly use `CODECOV_TOKEN` * Verify that `CODECOV_TOKEN` is set * Use an intermediary parameter * Use an intermediary env var * Use an intermediary env var * Fix syntax error * Setup CODECOV_TOKEN Environment Variable * Pass `CODECOV_TOKEN` to the docker image * Add initial `.codecov.yml` * Remove debug logs * Update `codecov.yml` * Rename codecov.yml * Validate codecov.yml * Add graph to the comment layout * [appsec] Install CodeCov Uploader Dependencies * Install missing `sudo` dependency * Set blocking coverage targets * App appsec & tracer extension flags * Create gpg file before valdation * [appsec] Install gnupg * Create `/root/.gnupg` * Add slug env var (See Notes) Notes: https://github.com/codecov/uploader/blob/6ccf3c3b145904308d38cdef1ad615df2ff6f9e0/src/ci_providers/provider_circleci.ts#L33-L49 * Detect CircleCI as the CI provider * Run coverage while running the integration[s]/web tests * fix: Only keep the last entry of xdebug.so files * Merge coverage reports * Add back again test_integrations_phpredis5 to the PHP 8.0 test suite * Only include the Integrations/ directory * Reset TEST_EXTRA_ENV after usage * Only run test coverage for PHP 7.4+ * Fix circleci syntax * Fix circleci syntax * Upgrade phpredis5 scenario to redis-5.3.7 * Revert phpredis5 changes * Revert phpredis5 changes * Add gpg to the base buster container * Use php-config to find the php extension directory Co-authored-by: Bob Weinand <[email protected]> * Merge coverage reports * Add back again test_integrations_phpredis5 to the PHP 8.0 test suite * Only include the Integrations/ directory * Only run test coverage for PHP 7.4+ * Upgrade phpredis5 scenario to redis-5.3.7 * Revert phpredis5 changes * Revert phpredis5 changes * Merge master * fix: Iteration over null * fix: missing coverage_file definition * fix: Undefined variable `data` * fix: Unique codecov/upload upload_name * fix: Unique codecov/upload upload_name * Exclude `dogstatsd/` --------- Co-authored-by: Bob Weinand <[email protected]>
1 parent ba0cf68 commit 32251e3

File tree

7 files changed

+195
-21
lines changed

7 files changed

+195
-21
lines changed

‎.circleci/continue_config.yml‎

Lines changed: 114 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -494,6 +494,11 @@ commands:
494494
-e BASH_ENV=/home/circleci/bashenv/bash.sh \
495495
-e CIRCLE_SHA1 \
496496
-e CIRCLE_BRANCH \
497+
-e CODECOV_TOKEN \
498+
-e CI \
499+
-e CIRCLECI \
500+
-e CIRCLE_PROJECT_USERNAME \
501+
-e CIRCLE_PROJECT_REPONAME \
497502
-v $(pwd):/home/circleci/datadog \
498503
-v /tmp/bashenv:/home/circleci/bashenv \
499504
-v /rust:/rust \
@@ -982,7 +987,8 @@ jobs:
982987
sudo apt install -y gpg
983988
- codecov/upload:
984989
file: tmp/coverage.info
985-
upload_name: "PHP<< parameters.php_major_minor >>.dd-trace-php"
990+
upload_name: "PHP<< parameters.php_major_minor >>.extension.dd-trace-php"
991+
flags: tracer-extension
986992
- run:
987993
command: |
988994
mkdir -p /tmp/artifacts/core_dumps
@@ -1135,7 +1141,7 @@ jobs:
11351141
command: |
11361142
export DEBIAN_FRONTEND=noninteractive
11371143
apt update
1138-
apt install -y wget git g++ gcc gcovr cmake make curl libcurl4-gnutls-dev clang clang-tidy clang-format git php-dev php-cgi cargo
1144+
apt install -y wget sudo git g++ gcc gcovr cmake make curl libcurl4-gnutls-dev clang clang-tidy clang-format git php-dev php-cgi cargo
11391145
- run: git config --global --add safe.directory /home/circleci/datadog/appsec/third_party/libddwaf
11401146
- run:
11411147
name: CMake
@@ -1168,9 +1174,17 @@ jobs:
11681174
pattern: "^[^.]+$"
11691175
value: << parameters.resource_class >>
11701176
steps:
1177+
- run:
1178+
name: Install CodeCov Uploader Dependencies
1179+
command: |
1180+
sudo apt update
1181+
sudo apt install -y gpg gnupg
1182+
sudo mkdir -p /root/.gnupg
1183+
sudo touch /root/.gnupg/trustedkeys.gpg
11711184
- codecov/upload:
11721185
file: appsec/coverage.xml
11731186
upload_name: "appsec-coverage >>"
1187+
flags: appsec-extension
11741188

11751189
lint_appsec:
11761190
parameters:
@@ -1353,6 +1367,9 @@ jobs:
13531367
disable_runner_distributed_tracing:
13541368
type: boolean
13551369
default: false
1370+
coverage:
1371+
type: boolean
1372+
default: false
13561373
<<: *BARE_DOCKER_MACHINE
13571374
environment:
13581375
COMPOSER_PROCESS_TIMEOUT: 0
@@ -1392,6 +1409,28 @@ jobs:
13921409
- run:
13931410
name: Run tests
13941411
command: DD_TRACE_AGENT_TIMEOUT=1000 <<# parameters.disable_runner_distributed_tracing >> DD_DISTRIBUTED_TRACING=false <</ parameters.disable_runner_distributed_tracing >> DD_TRACE_TEST_SAPI=<< parameters.sapi >> make << parameters.make_target >> PHPUNIT_OPTS="--log-junit test-results/php-composer/results.xml"
1412+
- when:
1413+
# codecov uploader only on amd64
1414+
condition:
1415+
and:
1416+
- equal: [ true, << parameters.coverage >> ]
1417+
- matches:
1418+
pattern: "^[^.]+$"
1419+
value: << parameters.resource_class >>
1420+
steps:
1421+
- run:
1422+
name: Merge coverage reports
1423+
command: |
1424+
make merge_coverage_reports
1425+
- run:
1426+
name: Install CodeCov Uploader Dependencies
1427+
command: |
1428+
sudo apt update
1429+
sudo apt install -y gpg
1430+
- codecov/upload:
1431+
file: reports/coverage.xml
1432+
upload_name: "PHP<< parameters.php_major_minor >>.<< parameters.make_target >>.dd-trace-php"
1433+
flags: tracer-integrations
13951434
- run:
13961435
command: |
13971436
mkdir -p /tmp/artifacts
@@ -1423,6 +1462,9 @@ jobs:
14231462
disable_runner_distributed_tracing:
14241463
type: boolean
14251464
default: false
1465+
coverage:
1466+
type: boolean
1467+
default: false
14261468
resource_class: << parameters.resource_class >>
14271469
executor:
14281470
name: with_integrations
@@ -1462,6 +1504,28 @@ jobs:
14621504
- run:
14631505
name: Run tests
14641506
command: DD_TRACE_AGENT_TIMEOUT=1000 <<# parameters.disable_runner_distributed_tracing >> DD_DISTRIBUTED_TRACING=false <</ parameters.disable_runner_distributed_tracing >> DD_TRACE_TEST_SAPI=<< parameters.sapi >> make << parameters.make_target >> RUST_DEBUG_SYMBOLS=1 PHPUNIT_OPTS="--log-junit test-results/php-composer/results.xml"
1507+
- when:
1508+
# codecov uploader only on amd64
1509+
condition:
1510+
and:
1511+
- equal: [ true, << parameters.coverage >> ]
1512+
- matches:
1513+
pattern: "^[^.]+$"
1514+
value: << parameters.resource_class >>
1515+
steps:
1516+
- run:
1517+
name: Merge coverage reports
1518+
command: |
1519+
make merge_coverage_reports
1520+
- run:
1521+
name: Install CodeCov Uploader Dependencies
1522+
command: |
1523+
sudo apt update
1524+
sudo apt install -y gpg
1525+
- codecov/upload:
1526+
file: reports/coverage.xml
1527+
upload_name: "PHP<< parameters.php_major_minor >>.<< parameters.make_target >>.dd-trace-php"
1528+
flags: tracer-integrations
14651529
- run:
14661530
command: |
14671531
mkdir -p /tmp/artifacts
@@ -4152,16 +4216,29 @@ workflows:
41524216
- '7.1'
41534217
- '7.2'
41544218
- '7.3'
4219+
make_target:
4220+
- test_composer
4221+
- test_integration
4222+
- test_distributed_tracing
4223+
- test_auto_instrumentation
4224+
4225+
- integration:
4226+
requires: [ 'Prepare Code' ]
4227+
resource_class: medium+
4228+
coverage: true
4229+
matrix:
4230+
parameters:
4231+
php_major_minor:
41554232
- '7.4'
41564233
- '8.0'
41574234
- '8.1'
41584235
- '8.2'
41594236
- '8.3'
41604237
make_target:
4161-
- test_composer
4162-
- test_integration
4163-
- test_distributed_tracing
4164-
- test_auto_instrumentation
4238+
- test_composer_coverage
4239+
- test_integration_coverage
4240+
- test_distributed_tracing_coverage
4241+
- test_auto_instrumentation_coverage
41654242

41664243
- integration_snapshots:
41674244
requires: [ 'Prepare Code' ]
@@ -4174,14 +4251,26 @@ workflows:
41744251
- '7.1'
41754252
- '7.2'
41764253
- '7.3'
4254+
make_target:
4255+
- test_web
4256+
- test_integrations
4257+
4258+
- integration_snapshots:
4259+
requires: [ 'Prepare Code' ]
4260+
# Due to Symfony OOM during composer update
4261+
resource_class: xlarge
4262+
coverage: true
4263+
matrix:
4264+
parameters:
4265+
php_major_minor:
41774266
- '7.4'
41784267
- '8.0'
41794268
- '8.1'
41804269
- '8.2'
41814270
- '8.3'
41824271
make_target:
4183-
- test_web
4184-
- test_integrations
4272+
- test_web_coverage
4273+
- test_integrations_coverage
41854274

41864275
- integration:
41874276
requires: [ 'Prepare Code' ]
@@ -4195,11 +4284,6 @@ workflows:
41954284
- '7.1'
41964285
- '7.2'
41974286
- '7.3'
4198-
- '7.4'
4199-
- '8.0'
4200-
- '8.1'
4201-
- '8.2'
4202-
- '8.3'
42034287
make_target:
42044288
- test_distributed_tracing
42054289

@@ -4223,6 +4307,23 @@ workflows:
42234307
make_target:
42244308
- test_integrations_phpredis5
42254309

4310+
- integration:
4311+
requires: [ 'Prepare Code' ]
4312+
resource_class: medium+
4313+
sapi: fpm-fcgi
4314+
disable_runner_distributed_tracing: true
4315+
coverage: true
4316+
matrix:
4317+
parameters:
4318+
php_major_minor:
4319+
- '7.4'
4320+
- '8.0'
4321+
- '8.1'
4322+
- '8.2'
4323+
- '8.3'
4324+
make_target:
4325+
- test_distributed_tracing_coverage
4326+
42264327
- xdebug_tests:
42274328
requires: [ 'Prepare Code' ]
42284329
name: "PHP 70 Xdebug tests"

‎Makefile‎

Lines changed: 41 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ TEST_OPCACHE_FILES = $(shell find tests/opcache -name '*.php*' -o -name '.gitkee
3636
TEST_STUB_FILES = $(shell find tests/ext -type d -name 'stubs' -exec find '{}' -type f \; | awk '{ printf "$(BUILD_DIR)/%s\n", $$1 }' )
3737
INIT_HOOK_TEST_FILES = $(shell find tests/C2PHP -name '*.phpt' -o -name '*.inc' | awk '{ printf "$(BUILD_DIR)/%s\n", $$1 }' )
3838
M4_FILES = $(shell find m4 -name '*.m4*' | awk '{ printf "$(BUILD_DIR)/%s\n", $$1 }' ) $(BUILD_DIR)/config.m4
39+
XDEBUG_SO_FILE = $(shell find $(shell php-config --extension-dir) -type f -name "xdebug*.so" -exec basename {} \; | tail -n 1)
3940

4041
all: $(BUILD_DIR)/configure $(SO_FILE)
4142

@@ -463,17 +464,20 @@ cores:
463464
REQUEST_INIT_HOOK := -d ddtrace.request_init_hook=$(REQUEST_INIT_HOOK_PATH)
464465
ENV_OVERRIDE := $(shell [ -n "${DD_TRACE_DOCKER_DEBUG}" ] && echo DD_AUTOLOAD_NO_COMPILE=true) DD_TRACE_CLI_ENABLED=true
465466
TEST_EXTRA_INI ?=
467+
TEST_EXTRA_ENV ?=
466468

467469
### DDTrace tests ###
468470
TESTS_ROOT = ./tests
469471
COMPOSER = $(if $(ASAN), ASAN_OPTIONS=detect_leaks=0) COMPOSER_MEMORY_LIMIT=-1 composer --no-interaction
470472
COMPOSER_TESTS = $(COMPOSER) --working-dir=$(TESTS_ROOT)
471473
PHPUNIT_OPTS ?=
472474
PHPUNIT = $(TESTS_ROOT)/vendor/bin/phpunit $(PHPUNIT_OPTS) --config=$(TESTS_ROOT)/phpunit.xml
475+
PHPUNIT_COVERAGE ?=
473476
PHPBENCH_OPTS ?=
474477
PHPBENCH_CONFIG ?= $(TESTS_ROOT)/phpbench.json
475478
PHPBENCH_OPCACHE_CONFIG ?= $(TESTS_ROOT)/phpbench-opcache.json
476479
PHPBENCH = $(TESTS_ROOT)/vendor/bin/phpbench $(PHPBENCH_OPTS) run
480+
PHPCOV = $(TESTS_ROOT)/vendor/bin/phpcov
477481

478482
TEST_INTEGRATIONS_70 := \
479483
test_integrations_deferred_loading \
@@ -938,14 +942,30 @@ TEST_WEB_83 := \
938942

939943
FILTER := .
940944

945+
define run_tests_without_coverage
946+
$(TEST_EXTRA_ENV) $(ENV_OVERRIDE) php $(TEST_EXTRA_INI) $(REQUEST_INIT_HOOK) $(PHPUNIT) $(1) --filter=$(FILTER)
947+
endef
948+
949+
define run_tests_with_coverage
950+
$(TEST_EXTRA_ENV) $(ENV_OVERRIDE) php -d zend_extension=$(XDEBUG_SO_FILE) -d xdebug.mode=coverage $(TEST_EXTRA_INI) $(REQUEST_INIT_HOOK) $(PHPUNIT) $(1) --filter=$(FILTER) --coverage-php reports/cov/$(coverage_file)
951+
endef
952+
953+
# Note: The condition below only checks for existence - i.e., whether PHPUNIT_COVERAGE is set to anything.
941954
define run_tests
942-
$(ENV_OVERRIDE) php $(TEST_EXTRA_INI) $(REQUEST_INIT_HOOK) $(PHPUNIT) $(1) --filter=$(FILTER)
955+
$(eval coverage_file := $(shell echo $(1) | tr '[:upper:]' '[:lower:]' | tr '/=' '_' | tr -d '-').cov) \
956+
$(if $(PHPUNIT_COVERAGE),$(call run_tests_with_coverage,$(1)),$(call run_tests_without_coverage,$(1)))
943957
endef
944958

945959
define run_tests_debug
946-
(set -o pipefail; { DD_TRACE_DEBUG=1 $(call run_tests,$(1)) 2>&1 >&3 | tee >(grep -vE '\[ddtrace\] \[debug\]|\[ddtrace\] \[info\]' >&2) | { ! (grep -E '\[error\]|\[warning\]|\[deprecated\]' >/dev/null && echo $$'\033[41m'"ERROR: Found debug log errors in the output."$$'\033[0m'); }; } 3>&1)
960+
$(eval TEST_EXTRA_ENV=$(TEST_EXTRA_ENV) DD_TRACE_DEBUG=1)
961+
(set -o pipefail; { $(call run_tests,$(1)) 2>&1 >&3 | \
962+
tee >(grep -vE '\[ddtrace\] \[debug\]|\[ddtrace\] \[info\]' >&2) | \
963+
{ ! (grep -E '\[error\]|\[warning\]|\[deprecated\]' >/dev/null && \
964+
echo $$'\033[41m'"ERROR: Found debug log errors in the output."$$'\033[0m'); }; } 3>&1)
965+
$(eval TEST_EXTRA_ENV=)
947966
endef
948967

968+
949969
define run_benchmarks
950970
$(ENV_OVERRIDE) php $(TEST_EXTRA_INI) $(REQUEST_INIT_HOOK) $(PHPBENCH) --config=$(1) --filter=$(FILTER) --report=all --output=file --output=console
951971
endef
@@ -997,17 +1017,25 @@ test_unit: global_test_run_dependencies
9971017

9981018
test_integration: global_test_run_dependencies
9991019
$(call run_tests,--testsuite=integration $(TESTS))
1020+
test_integration_coverage:
1021+
PHPUNIT_COVERAGE=1 $(MAKE) test_integration
10001022

10011023
test_auto_instrumentation: global_test_run_dependencies
10021024
$(call run_tests,--testsuite=auto-instrumentation $(TESTS))
10031025
# Cleaning up composer.json files in tests/AutoInstrumentation modified for TLS during tests
10041026
git checkout $(TESTS_ROOT)/AutoInstrumentation/**/composer.json
1027+
test_auto_instrumentation_coverage:
1028+
PHPUNIT_COVERAGE=1 $(MAKE) test_auto_instrumentation
10051029

10061030
test_composer: global_test_run_dependencies
10071031
$(call run_tests,--testsuite=composer-tests $(TESTS))
1032+
test_composer_coverage:
1033+
PHPUNIT_COVERAGE=1 $(MAKE) test_composer
10081034

10091035
test_distributed_tracing: global_test_run_dependencies
10101036
$(call run_tests,--testsuite=distributed-tracing $(TESTS))
1037+
test_distributed_tracing_coverage:
1038+
PHPUNIT_COVERAGE=1 $(MAKE) test_distributed_tracing
10111039

10121040
test_metrics: global_test_run_dependencies
10131041
$(call run_tests,--testsuite=metrics $(TESTS))
@@ -1028,7 +1056,9 @@ benchmarks_opcache: benchmarks_run_dependencies
10281056
test_opentelemetry_1: global_test_run_dependencies
10291057
rm -f tests/.scenarios.lock/opentelemetry1/composer.lock
10301058
$(MAKE) test_scenario_opentelemetry1
1031-
$(shell [ $(PHP_MAJOR_MINOR) -ge 81 ] && echo "OTEL_PHP_FIBERS_ENABLED=1" || echo "") DD_TRACE_OTEL_ENABLED=1 DD_TRACE_GENERATE_ROOT_SPAN=0 $(call run_tests,--testsuite=opentelemetry1 $(TESTS))
1059+
$(eval TEST_EXTRA_ENV=$(shell [ $(PHP_MAJOR_MINOR) -ge 81 ] && echo "OTEL_PHP_FIBERS_ENABLED=1" || echo '') DD_TRACE_OTEL_ENABLED=1 DD_TRACE_GENERATE_ROOT_SPAN=0)
1060+
$(call run_tests,--testsuite=opentelemetry1 $(TESTS))
1061+
$(eval TEST_EXTRA_ENV=)
10321062

10331063
test_opentracing_beta5: global_test_run_dependencies
10341064
$(MAKE) test_scenario_opentracing_beta5
@@ -1045,6 +1075,11 @@ test_opentracing_10: global_test_run_dependencies
10451075
test_integrations: $(TEST_INTEGRATIONS_$(PHP_MAJOR_MINOR))
10461076
test_web: $(TEST_WEB_$(PHP_MAJOR_MINOR))
10471077

1078+
test_web_coverage:
1079+
PHPUNIT_COVERAGE=1 $(MAKE) test_web
1080+
test_integrations_coverage:
1081+
PHPUNIT_COVERAGE=1 $(MAKE) test_integrations
1082+
10481083
test_integrations_amqp2: global_test_run_dependencies
10491084
$(MAKE) test_scenario_amqp2
10501085
$(call run_tests_debug,tests/Integrations/AMQP)
@@ -1280,6 +1315,9 @@ test_web_custom: global_test_run_dependencies
12801315
test_scenario_%:
12811316
$(Q) $(COMPOSER_TESTS) scenario $*
12821317

1318+
merge_coverage_reports:
1319+
$(PHPCOV) merge --clover reports/coverage.xml reports/cov
1320+
12831321
### Api tests ###
12841322
API_TESTS_ROOT := ./tests/api
12851323

‎codecov.yml‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Validate using: cat codecov.yml | curl --data-binary @- https://codecov.io/validate
2+
codecov:
3+
require_ci_to_pass: false # See coverage on CI test failures
4+
5+
coverage:
6+
range: 90..100
7+
round: down
8+
precision: 2
9+
status: # Blocking coverage targets
10+
project: # Each PR must increase coverage
11+
default:
12+
target: auto
13+
patch: # New code must be at least 90% tested
14+
default:
15+
target: 90%
16+
17+
18+
comment: # See https://docs.codecov.com/docs/pull-request-comments
19+
behavior: default
20+
layout: "header,reach,diff,flags,files,footer"
21+
require_changes: true
22+
require_base: false
23+
require_head: false

‎dockerfiles/ci/buster/Dockerfile‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@ ENV DEVLIBS \
3535
zlib1g-dev \
3636
libasan5 \
3737
gnupg \
38+
gpg \
3839
unixodbc-dev \
3940
unixodbc
4041

‎src/Integrations/Integrations/Curl/CurlIntegration.php‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -189,11 +189,17 @@ public function init()
189189
$data = ObjectKVStore::get($hook->args[0], "span");
190190
} elseif ($lastMh[0] == (int)$hook->args[0]) {
191191
$data = $lastMh[1];
192+
} else {
193+
$data = null;
192194
}
193195

194196
list(, $spans) = $data;
195197

196198
if (!isset($hook->returned["result"]) || $hook->returned["result"] == CURLE_OK) {
199+
if (empty($spans)) {
200+
return;
201+
}
202+
197203
foreach ($spans as $requestSpan) {
198204
list($ch, $requestSpan) = $requestSpan;
199205
if ($ch === $handle) {

0 commit comments

Comments
 (0)