Releases: DataDog/datadog-lambda-extension
v97
# AWS Commercial
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:97
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-FIPS:97
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:97
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM-FIPS:97
# AWS GovCloud
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:97
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-FIPS:97
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:97
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM-FIPS:97What's Changed
- fix: update Cargo.lock to fix cargo audit vulnerabilities by @lym953 in #1190
- fix: bump rustls-webpki to 0.103.13 to fix RUSTSEC-2026-0104 by @lym953 in #1209
- fix: disable DD_TRACE_STATS_COMPUTATION_ENABLED by default by @eunskin in #1186
- feat: set durable_function.execution_status on END platform logs by @lym953 in #1188
- fix(appsec): Go runtime
_dd.appsec.enabledmissing onaws.lambdaby @RomainMuller in #1213 - fix(otel): Update Operation Name to Match Frontend Usage by @zarirhamza in #1033
- refactor(stats): take
stats_urlas aStatsFlusher::newparameter by @lucaspimentel in #1210 - chore(deps): bump libdatadog to db05e1f and adapt to HttpClientTrait API by @duncanista in #1218
- perf(deps): drop aws-lc-rs from non-FIPS builds, use ring backend by @duncanista in #1219
- fix(logs): bound retries on non-success HTTP status by @litianningdatadog in #1220
New Contributors
Full Changelog: v96...v97
v96
# AWS Commercial
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:96
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-FIPS:96
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:96
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM-FIPS:96
# AWS GovCloud
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:96
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-FIPS:96
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:96
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM-FIPS:96What's Changed
- feat(logs): Hold logs and add durable context to durable function logs by @lym953 in #1053
- feat(logs): add aws_lambda.durable_function.first_invocation to START/END/REPORT logs by @lym953 in #1158
- feat(auth): add AWS delegated authentication support by @jchrostek-dd in #1112
- fix(lifecycle): prune sorted_reparenting_info on context release to stop warning flood by @litianningdatadog in #1161
- fix(tags): Fix SnapStart cold_start tag using restore_time by @jchrostek-dd in #1139
- feat(traces): reintroduce
base_servicefor Service Remapping by @zarirhamza in #1157 - fix(logging): Lower log level for some span deduper logs to debug by @lym953 in #1124
- revert(traces): revert Datadog-Client-Computed-Stats header support by @duncanista in #1176
New Contributors
- @chouetz made their first contribution in #1128
- @kathiehuang made their first contribution in #1144
- @happynancee made their first contribution in #1160
Full Changelog: v95...v96
v95
Warning
There is an issue regarding trace metrics on this version, users can expect a loss in trace metrics.
This does not affect any other functionality regarding custom or enhanced metrics.
# AWS Commercial
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:95
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-FIPS:95
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:95
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM-FIPS:95
# AWS GovCloud
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:95
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-FIPS:95
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:95
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM-FIPS:95What's Changed
- fix(deps): update Cargo.lock to resolve cargo audit vulnerabilities by @lym953 in #1122
- [SVLS-8741] fix: Improve log messages and add retries for proxy flusher by @lym953 in #1123
- feat(otlp): add gRPC protocol support for trace ingestion by @jchrostek-dd in #1105
- feat(triggers): handle MSK events by @joeyzhao2018 in #1066
- feat(traces): [SVLS-8734] respect Datadog-Client-Computed-Stats header by @lym953 in #1118
- feat(traces): migrate trace propagation to dd-trace-rs by @duncanista in #1089
- chore: consolidate agent code for otlp by @duncanista in #1132
- fix(tests): remove stale _dd.compute_stats:1 assertion from logs integration test by @litianningdatadog in #1136
New Contributors
- @chouetz made their first contribution in #1128
- @kathiehuang made their first contribution in #1144
Full Changelog: v94...v95
v94
# AWS Commercial
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:94
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-FIPS:94
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:94
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM-FIPS:94
# AWS GovCloud
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:94
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-FIPS:94
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:94
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM-FIPS:94What's Changed
- fix(lifecycle): raise axum body limit to 6 MB for large Lambda payloads by @litianningdatadog in #1044
- fix(traces): downgrade handle_proxy body-read failure from ERROR to WARN by @litianningdatadog in #1046
- [SVLS-8581] feat: Add durable_function:true tag to enhanced metrics by @lym953 in #1048
- fix(logs): update
leveltostatuson datadog owned logs by @duncanista in #1055 - chore(dogstatsd): use pre built client by @duncanista in #1061
- fix(http): consolidate clients by @duncanista in #1062
- feat(http): allow skip ssl validation by @duncanista in #1064
- fix: handle_end_invocation race condition when extracting payload by @pablomartinezbernardo in #1024
- fix(lifecycle): release invocation context after platform report to prevent memory leak by @litianningdatadog in #1050
- feat: [SVLS-8390] drop sampled-out traces from backend when compute_trace_stats_on_extension is enabled by @lym953 in #1060
- fix: oversize payload causing universal instrumentation failures by @joeyzhao2018 in #1067
- fix(otlp): allow json payloads by @jchrostek-dd in #1069
- fix(traces): update payload size after dropping traces by @lym953 in #1091
- fix(http): disable connection pooling to prevent stale connections in Lambda by @jchrostek-dd in #1094
- fix(otlp): accept flexible timestamp formats in JSON payloads by @jchrostek-dd in #1108
- fix(traces): add retry strategy by @duncanista in #1116
New Contributors
- @pablomartinezbernardo made their first contribution in #1024
- @juliendoutre made their first contribution in #1104
Full Changelog: v93...v94
v93
# AWS Commercial
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:93
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-FIPS:93
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:93
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM-FIPS:93
# AWS GovCloud
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:93
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-FIPS:93
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:93
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM-FIPS:93What's Changed
Major Features
- feat: Add ap-east-2 and ap-southeast-6 to release config by @lym953 in #1001
- feat: add check to extract trace context present within event.request.headers by @Dogbu-cyber in #1011
Major Fixes
- fix: lowercase DD_SERVICE on processors by @duncanista in #1013
- avoid setting cold start on managed instances mode by @duncanista in #1008
- fix(config): ensure logs intake URL is correctly prefixed by @shreyamalpani in #1021
- publish image to correct repos by @jchrostek-dd in #1022
- fix(vuln): cargo update by @duncanista in #1014
- fix(logs): format log level into correct status by @duncanista in #1036
- fix(deps): upgrade libdatadog by @duncanista in #1039
- fix: pin rustls-native-certs to <0.8.3 by @duncanista in #1047
Minor Fixes
- chore: Fix cargo-audit error by @lym953 in #992
- docs: remove reference to code and docs being in different repos by @egracer in #1017
- chore(flushing): refactor flushing by @duncanista in #1016
- chore(flushing): standardize code with refactoring on some flushers and retries by @duncanista in #1018
- chore(deps): upgrade dogstatsd by @duncanista in #1020
- chore(deps): upgrade rust to v1.93.1 by @duncanista in #1034
- upgrade time@0.3.47 by @duncanista in #1035
- fix(logs): stop logging payload on debug by @duncanista in #1037
- perf(logger): replace serde_json::json! with zero-allocation write! in logger by @litianningdatadog in #1042
- perf(deps): bump opentelemetry-proto to 0.30.0 and tonic-types to 0.13 by @litianningdatadog in #1043
New Contributors
- @campaigner-prod[bot] made their first contribution in #1002
- @egracer made their first contribution in #1017
- @Dogbu-cyber made their first contribution in #1011
Full Changelog: v92...v93
v92
# AWS Commercial
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:92
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-FIPS:92
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:92
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM-FIPS:92
# AWS GovCloud
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:92
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-FIPS:92
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:92
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM-FIPS:92What's Changed
Major Features:
- Support TLS certificate for logs/proxy flusher by @lym953 in #979
- Support TLS certificate for dogstatsd metrics flusher by @lym953 in #984
Major fixes:
- Fix AWS API Gateway endpoints correlation HTTP span tags by @e-n-0 in #967
- Fix OTLP response format by @jchrostek-dd in #969
- Fix log requestId for LMI by @jchrostek-dd in #973
- Add stats flusher to continuous mode by @litianningdatadog in #977
Minor fixes:
- Add debug log for flush strategy by @jchrostek-dd in #974
- Add timeout for requests to span_dedup_service by @lym953 in #986
New Contributors
Full Changelog: v91...v92
v91
# AWS Commercial
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:91
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-FIPS:91
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:91
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM-FIPS:91
# AWS GovCloud
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:91
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-FIPS:91
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:91
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM-FIPS:91What's Changed
- fix: [SVLS-8072] run apt-get update and apt-get install in same layer by @duncanpharvey in #949
- chore: Remove committed RIE binaries and download at build time by @litianningdatadog in #950
- [SVLS-8054] add integration testing by @jchrostek-dd in #946
- fix(config): support colons in tag values (URLs, etc.) by @litianningdatadog in #953
- docs: Add Lambda Managed Instance mode documentation by @litianningdatadog in #951
- fix(github): update vulnerability scans by @duncanista in #941
- fix(vulnerabilities): update rust dependencies by @duncanista in #955
- fix integ test cleanup by @jchrostek-dd in #956
- add integration tests for node and java by @jchrostek-dd in #958
- chore: Add a timer to avoid repeated debug logs by @lym953 in #954
- APPSEC-60188: gracefully accept
nullin APIGW response by @RomainMuller in #960 - build(deps): bump anchore/scan-action from 7.2.1 to 7.2.2 by @dependabot[bot] in #962
- fix build layer script usages by @ojproductions in #931
- APPSEC-60188: downgrade integration response parsing log to debug by @RomainMuller in #959
- remove run bottlecap script by @astuyve in #966
- [SVLS-7934] feat: Support TLS certificate for trace/stats flusher by @lym953 in #961
- chore: Upgrade libdatadog by @lym953 in #964
- v91 by @shreyamalpani in #968
New Contributors
- @ojproductions made their first contribution in #931
Full Changelog: v90...v91
v90
# AWS Commercial
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:90
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-FIPS:90
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:90
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM-FIPS:90
# AWS GovCloud
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:90
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-FIPS:90
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:90
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM-FIPS:90What's Changed
Features
- Lambda Managed Instance Support: Introduced the support for AWS Lambda Managed Instance mode (#947 by @astuyve, @duncanista, @jchrostek-dd, @litianningdatadog, @shreyamalpani)
- Span Deduplication: Added span deduplication by span_id to fix trace stats overcount issue in Node.js runtime when using continuous flush strategy (#939 by @purple4reina)
- VPC Proxy Support for Trace Stats: Enabled sending trace stats to Datadog when Lambda is in a VPC with a proxy (#927 by @purple4reina)
- Extended Debugger Proxy Support: Added proxy support for /debugger/v2/input and /debugger/v1/diagnostics endpoints to enable Exception Replay (#918 by @purple4reina)
- DD_LOGS_ENABLED Environment Variable: Added DD_LOGS_ENABLED as an alias for DD_SERVERLESS_LOGS_ENABLED with OR logic for backward compatibility (#928 by @litianningdatadog)
Improvements
- libdatadog Upgrade: Upgraded libdatadog dependency with updated crate names and new HTTP client API for trace submission (#917 by @purple4reina)
- CI Stability: Set 10-minute timeout and automatic retries for bottlecap GitLab job to prevent false failures (#945 by @purple4reina)
Full Changelog: v89...v90
v89
Read about the Next Generation Datadog Lambda Extension.
# AWS Commercial
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:89
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-FIPS:89
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:89
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM-FIPS:89
# AWS GovCloud
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:89
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-FIPS:89
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:89
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM-FIPS:89What's Changed
Features
-
DogStatsD Improvements
- Added support for DD_STATSD_METRIC_NAMESPACE environment variable to prefix custom metrics with a namespace (#920, #921 by @jchrostek-dd and @litianningdatadog)
-
SnapStart Support
- Added automatic creation of SnapStart spans and metrics when Lambda SnapStart is enabled (#911 by @jchrostek-dd)
- Replaces init spans with restore spans for SnapStart invocations
-
Trace Agent Improvements
- Increased trace payload limit from 2 MiB to 50 MiB to prevent connection errors with large trace payloads (#902 by @lym953)
- Implemented libdatadog's native concentrator for trace stats aggregation (#887 by @lym953)
- Removed hostname from ClientStatsPayload for proper stats aggregation (#905 by @lym953)
-
Secrets Management
-
SSM Parameter Store: Add support for retrieving Datadog API keys from AWS Systems Manager Parameter Store via the DD_API_KEY_SSM_ARN environment variable. Supports both String and SecureString parameter types. (#924 by @jchrostek-dd)
-
Tag Handling
- Updated DD_TAGS parsing to support both comma (,) and space ( ) delimiters, improving compatibility with container images (#916 by @jchrostek-dd)
Bug Fixes
-
Logging
- Fixed log processing rules incorrectly filtering orphan logs (extension logs and error logs) when exclude patterns were configured (#912 by @lym953)
- Add URL information to proxy flusher error logs for easier identification of failing endpoints (#933 by @litianningdatadog)
-
Metrics
- Fix an issue where negative CPU utilization values could be reported due to timing differences between /proc/uptime and /proc/stat (#929 by @shreyamalpani and @litianningdatadog)
- Fix an issue where the tmp_free metric was missing after the enhanced metrics refactor. (#937 by @shreyamalpani)
Performance
- Removed invocation lock and replaced with aggregator service pattern (#894 by @jchrostek-dd )
- Refactored enhanced metrics monitoring to use long-running task instead of spawning new tasks per invocation (#901 by @shreyamalpani)
- Migrated trace aggregation to channel-based approach to avoid locking (#909 by @shreyamalpani)
Error Handling
Known Issues
- Trace stats may show up to 9% under/over-counting in high-volume, high-concurrency scenarios (under investigation)
Full Changelog: v88...v89
v88
Read about the Next Generation Datadog Lambda Extension.
Caution
This v88 release no longer contains the Compatibility Mode (Datadog Go Agent).
DD_EXTENSION_VERSION=compatibility will no longer work to fallback. Yet you should not have any problems, this will not disturb your Lambdas or their execution.
Warning
.NET 6 users with DD_SERVERLESS_APPSEC_ENABLED should avoid upgrading, as an issue with the AWS Lambda Runtime conflicts with the HTTP2 header standard. An alternative is upgrading to the .NET 8 runtime.
# AWS Commercial
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension:88
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-FIPS:88
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM:88
arn:aws:lambda:<AWS_REGION>:464622532012:layer:Datadog-Extension-ARM-FIPS:88
# AWS GovCloud
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension:88
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-FIPS:88
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM:88
arn:aws-us-gov:lambda:us-gov-<AWS_REGION>:002406178527:layer:Datadog-Extension-ARM-FIPS:88What's Changed
- strip symbols too not just debug info by @astuyve in #876
- fix: dupe copy of saluki fixed by @astuyve in #877
- chore: [Trace Stats] Rename env var DD_COMPUTE_TRACE_STATS by @lym953 in #875
- fix: setting last_flush to 0 makes us flush on the first invocation by @astuyve in #843
- chore(extension): allow register with name by @duncanista in #881
- chore: Add a space in debug log for duration by @lym953 in #880
- chore(gitlab): remove Go agent from build by @duncanista in #795
- fix(logs): use channel-based aggregator by @shreyamalpani in #879
- feat: remove failover to go by @astuyve in #882
- chore: add prefix to some logs by @duncanista in #884
- chore: cleanup code for main agent creation by @duncanista in #885
- fix: tombstone event to be sent properly by @duncanista in #886
- feat: Aggregate tasks into chunks sooner, then drop the lock by @astuyve in #889
- allow native certs by @astuyve in #892
- fix: use None if propagation style is invalid by @astuyve in #895
- feat: Support periodic reload for api key secret by @lym953 in #893
Full Changelog: v87...v88