Skip to content

Commit 0d765ab

Browse files
committed
Document currency code capability, in README and example
1 parent 403ecdc commit 0d765ab

6 files changed

Lines changed: 112 additions & 2 deletions

File tree

‎README.md‎

Lines changed: 23 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -151,8 +151,8 @@ For a batch transform, use `Client`. You should only need the following methods:
151151
* `find_file_path(file_name)` – Resolve a bundled file (placed under `payload/files/`) to a `pathlib.Path` that exists. Works the same locally and inside Data Cloud — see [Bundled file resolution](#bundled-file-resolution) below for the full lookup order. Raises `FileNotFoundError` if the file isn't found.
152152
* `read_dlo(name)` – Read from a Data Lake Object by name
153153
* `read_dmo(name)` – Read from a Data Model Object by name
154-
* `write_to_dlo(name, spark_dataframe, write_mode)` – Write to a Data Model Object by name with a Spark dataframe
155-
* `write_to_dmo(name, spark_dataframe, write_mode)` – Write to a Data Lake Object by name with a Spark dataframe
154+
* `write_to_dlo(name, spark_dataframe, write_mode)` – Write to a Data Lake Object by name with a Spark dataframe
155+
* `write_to_dmo(name, spark_dataframe, write_mode)` – Write to a Data Model Object by name with a Spark dataframe
156156

157157
For a streaming (delta) transform, use `StreamingClient`, which exposes the streaming counterparts:
158158
* `read_dlo_deltas()` – Read the streaming change feed (deltas) of a Data Lake Object as a streaming DataFrame.
@@ -176,6 +176,27 @@ client.write_to_dlo('output_DLO')
176176
> [!WARNING]
177177
> Currently we only support reading from DMOs and writing to DMOs or reading from DLOs and writing to DLOs, but they cannot mix.
178178
179+
### Writing Currency fields
180+
181+
A DLO or DMO with a Currency field has a system field, `cdp_sys_record_currency__c`, that holds each record's ISO currency code. Data Explorer UI uses it to display currency values, and shows a blank value if it's null (Query Editor still shows it).
182+
183+
Code Extensions don't set `cdp_sys_record_currency__c` for you, since only your script knows what currency its amounts are in. When writing to an object with Currency fields, set it yourself:
184+
185+
```python
186+
from pyspark.sql.functions import lit
187+
188+
# Either source has Currency fields: keep its cdp_sys_record_currency__c in your output
189+
df = source.select("id__c", "amount__c", "cdp_sys_record_currency__c")
190+
191+
# OR amounts are in a known currency
192+
df = df.withColumn("cdp_sys_record_currency__c", lit("EUR"))
193+
```
194+
195+
- Use an ISO code that exists in your org's configured currency rates.
196+
- When aggregating, group by the currency code so amounts in different currencies aren't summed together.
197+
198+
See [`docs/examples/script/currency_totals`](docs/examples/script/currency_totals) for a complete example.
199+
179200
### Streaming (delta) transforms
180201

181202
Streaming BYOC transforms process a Data Lake Object's Change Data Feed continuously instead of reading a bounded snapshot. Use a `StreamingClient` and its `*_deltas` methods in place of the batch `Client` read/write methods:
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Example: Totals that keep their currency code
2+
3+
A Code Extension for Batch Transform that sums `Payment__dll` amounts per customer and writes them to `Customer_Payment_Totals__dll`, a DLO with a Currency field, setting the `cdp_sys_record_currency__c` system field so Data Explorer UI can display the totals.
4+
5+
## What it demonstrates
6+
7+
- **Carrying `cdp_sys_record_currency__c` through** from the source rows to the output. Code Extensions don't auto-populate `cdp_sys_record_currency__c`. And if it's null, Data Explorer shows the Currency value as blank. See [Writing Currency fields](../../../../README.md#writing-currency-fields).
8+
- **Grouping by currency** so EUR and USD amounts are never summed together. Customer `C1` gets one EUR row and one USD row.
9+
- If all amounts are in one known currency, one can also use `lit("EUR")` instead of copying a column.
10+
11+
## Prerequisites
12+
13+
Input DLO `Payment__dll` in the target dataspace:
14+
15+
| Column | Type | Notes |
16+
|------------------------------|----------|------------------------------------------------------------------------|
17+
| `id__c` | text | Primary key. |
18+
| `customer_id__c` | text | |
19+
| `amount__c` | Currency | |
20+
| `cdp_sys_record_currency__c` | text | System field, added with the Currency field. From CSV `currency_code`. |
21+
22+
Data should be loaded via `sample_data/payment.csv`. See [loading sample data](../README.md#loading-sample-data).
23+
24+
Output DLO `Customer_Payment_Totals__dll` in the same dataspace:
25+
26+
| Column | Type | Notes |
27+
|------------------------------|----------|--------------------------------------------------|
28+
| `id__c` | text | Primary key (`<customer_id>_<currency>`). |
29+
| `customer_id__c` | text | |
30+
| `total_amount__c` | Currency | |
31+
| `cdp_sys_record_currency__c` | text | System field, added with the Currency field. |
32+
33+
The ISO codes in `sample_data/payment.csv` must exist in your org's currency rates.
Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"sdkVersion": "6.1.0",
3+
"entryPoint": "entrypoint.py",
4+
"dataspace": "default",
5+
"permissions": {
6+
"read": {
7+
"dlo": ["Payment__dll"]
8+
},
9+
"write": {
10+
"dlo": ["Customer_Payment_Totals__dll"]
11+
}
12+
}
13+
}
Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
from pyspark.sql.functions import (
2+
col,
3+
concat_ws,
4+
sum as _sum,
5+
)
6+
7+
from datacustomcode.client import Client
8+
from datacustomcode.io.writer.base import WriteMode
9+
10+
11+
def main():
12+
client = Client()
13+
14+
payments = client.read_dlo("Payment__dll").select(
15+
"customer_id__c", "amount__c", "cdp_sys_record_currency__c"
16+
)
17+
18+
totals = (
19+
payments.groupBy("customer_id__c", "cdp_sys_record_currency__c")
20+
.agg(_sum("amount__c").alias("total_amount__c"))
21+
.select(
22+
concat_ws(
23+
"_", col("customer_id__c"), col("cdp_sys_record_currency__c")
24+
).alias("id__c"),
25+
col("customer_id__c"),
26+
col("total_amount__c"),
27+
col("cdp_sys_record_currency__c"),
28+
)
29+
)
30+
31+
client.write_to_dlo("Customer_Payment_Totals__dll", totals, WriteMode.OVERWRITE)
32+
33+
34+
if __name__ == "__main__":
35+
main()
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
# Packages required for the custom code
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
id,customer_id,amount,currency_code
2+
P1,C1,100.00,EUR
3+
P2,C1,250.50,EUR
4+
P3,C1,40.00,USD
5+
P4,C2,1200.00,USD
6+
P5,C2,300.00,USD
7+
P6,C3,75.25,GBP

0 commit comments

Comments
 (0)