Halyna R.

.gitignore

@@ -162,3 +162,11 @@ dist
 vite.config.js.timestamp-*
 vite.config.ts.timestamp-*
 
+
+# Local secrets and dbt generated files
+.env
+profiles.yml
+dbt_packages/
+target/
+logs/
+grader_output.txt

.user.yml

@@ -0,0 +1 @@
+id: be40aff5-745e-4ac9-9496-e9d5fb6c1776

AI_ASSIST.md

@@ -4,33 +4,52 @@ Document one place you used an LLM during this assignment.
 
 ## The problem
 
-<!-- TODO: describe the specific problem you asked an LLM about.
-     Example: "My safe_divide macro compiled but returned NULL for all rows even
-     when tip_amount and fare_amount were both non-zero." -->
-
-TODO
+I used an LLM while debugging my Week 10 dbt assignment after I ran into several errors that blocked the project from running correctly. The main issue was not writing the assignment from scratch, but understanding and fixing setup/runtime problems: VS Code showed false SQL syntax errors for dbt/Jinja syntax, `dbt deps` failed on Windows because `dbt_packages` was locked by another process, YAML parsing failed in `_fct_daily_borough_stats.yml`, and `dbt build` failed because the local `profiles.yml` connection settings were incorrect.
 
 ## The prompt
 
-<!-- TODO: paste the exact prompt you sent to the LLM. -->
+I asked the LLM several debugging questions while I was trying to get the dbt project to run correctly. The main prompt was:
+
+```text
+I am working on a Week 10 dbt assignment. I already have the dbt models and YAML files, but I am getting several errors while trying to run and validate the project.
+
+Please help me debug the errors step by step. I want to understand what is wrong, which files need to be fixed, and how to verify that the project works correctly.
 
-TODO
+The errors include:
+- VS Code shows SQL syntax errors near dbt/Jinja syntax such as {{ ref(...) }}, {{ source(...) }}, and {% macro %}.
+- dbt deps fails on Windows with a file-locking error in dbt_packages.
+- dbt compile fails with: Env var required but not provided: 'PG_HOST'.
+- A YAML file fails with: mapping values are not allowed in this context.
+- dbt build fails because of PostgreSQL profile/connection settings.
+
+Please explain which errors are real dbt problems, which ones are only editor/linter warnings, and give the exact commands I should run to verify the project.
+```
 
 ## The response
 
-<!-- TODO: summarise or paste what the LLM returned. -->
+The LLM explained that many red VS Code “SQL syntax” errors were false positives because VS Code was parsing dbt/Jinja files as plain PostgreSQL. It advised me to validate the project with dbt compile and dbt build instead of relying only on the Problems panel.
+
+For the real errors, the LLM identified concrete fixes:
 
-TODO
+add dbt_utils correctly in packages.yml;
+clean and reinstall dbt_packages after the Windows file-lock error;
+fix _fct_daily_borough_stats.yml by using a YAML multiline description with description: |;
+use a local profiles.yml only for running dbt, but keep profiles.yml.example for the repo;
+make sure the profile points to the correct database, schema, SSL mode, and password environment variable;
+run the final validation with:
+dbt deps
+dbt debug --profiles-dir .
+dbt compile --profiles-dir .
+dbt build --profiles-dir . --select +fct_daily_borough_stats
 
 ## Reflection
 
-<!-- TODO: what did you change, keep, or discard after reviewing the LLM's answer?
-     Be specific: "I kept the NULLIF suggestion but changed the column alias from
-     'ratio' to 'tip_pct' to match the assignment schema." -->
+I used the LLM output as debugging guidance, not as a blind code generator. I kept the advice that dbt/Jinja syntax should be checked with dbt compile rather than by the generic SQL linter in VS Code. I also applied the YAML fix using description: |, corrected the package setup, and checked the dbt project with dbt build.
 
-TODO
+I did not paste any real password or private connection string into the LLM. I reviewed the suggested changes before applying them and kept the assignment logic focused on the required dbt models, tests, documentation, and business-answer queries.
 
 ---
 
 > Remember: never paste real connection strings, passwords, or PII into an LLM.
 > The NYC TLC dataset is public so sample rows are safe here, but practise the habit.
+```

macros/safe_divide.sql

@@ -3,8 +3,5 @@
 -- Use for tip_pct = tip_amount / fare_amount and similar ratio columns.
 
 {% macro safe_divide(numerator, denominator) %}
-    -- TODO: implement the macro body.
-    -- Use NULLIF(denominator, 0) to avoid division-by-zero errors.
-    -- Return only the SQL expression (no SELECT, no semicolon).
-    NULL
-{% endmacro %}
+    {{ numerator }} / nullif({{ denominator }}, 0)
+{% endmacro %}

models/marts/_fct_daily_borough_stats.yml

@@ -2,23 +2,55 @@ version: 2
 
 models:
   - name: fct_daily_borough_stats
-    description: >
-      TODO: state the grain (one row per ___), the source lineage
-      (built from ___ and ___), and at least one known caveat
-      (e.g. rows dropped in staging, any WARN-severity tests).
-    # TODO: Task 5 -- add the compound uniqueness test on the mart's primary
-    # key (pickup_borough, pickup_date). You need the dbt_utils package for
-    # this: declare it in packages.yml and run `dbt deps` first.
+    description: |
+      Daily borough-level mart for NYC green taxi pickup activity.
+
+      Grain: one row per pickup_borough and pickup_date.
+
+      Source lineage: built from stg_trips joined with stg_zones.
+      stg_trips reads nyc_taxi.raw_trips.
+      stg_zones reads nyc_taxi.raw_zones.
+
+      Known caveats: stg_trips drops rows with missing pickup_location_id
+      or negative fare_amount. The mart uses an inner join to drop trips
+      whose pickup_location_id does not match a known zone. Real TLC labels
+      such as EWR, Unknown, and dirty NaN borough values remain in the mart.
+      The avg_tip_pct singular test uses WARN severity because very small
+      borough/day groups can legitimately exceed 1 due to outliers.
+
+    tests:
+      - dbt_utils.unique_combination_of_columns:
+          combination_of_columns:
+            - pickup_borough
+            - pickup_date
+
     columns:
       - name: pickup_borough
-        description: "TODO: explain what this column contains and where it comes from"
+        description: Borough label from stg_zones.borough for the trip pickup zone.
+        tests:
+          - not_null
+
       - name: pickup_date
-        description: "TODO: explain units and how it is derived"
+        description: Calendar date derived from pickup_datetime. Unit is one local pickup day.
+        tests:
+          - not_null
+
       - name: trip_count
-        description: "TODO: explain what is counted (unit: number of trips)"
+        description: Number of trips picked up in this borough on this date. Unit is trips.
+        tests:
+          - not_null
+
       - name: total_fare
-        description: "TODO: explain units (USD) and what fare_amount represents"
+        description: Sum of fare_amount for this borough and date. Unit is US dollars.
+        tests:
+          - not_null
+
       - name: avg_tip_pct
-        description: "TODO: explain the ratio (tip_amount / fare_amount)"
+        description: Average tip percentage for this borough and date. Calculated as tip_amount divided by fare_amount.
+        tests:
+          - not_null
+
       - name: avg_trip_distance
-        description: "TODO: explain units (miles, from TLC source data)"
+        description: Average trip distance for this borough and date. Unit is miles.
+        tests:
+          - not_null

models/marts/fct_daily_borough_stats.sql

@@ -14,25 +14,15 @@ zones AS (
 )
 
 SELECT
-    -- TODO: join trips to zones on pickup_location_id = location_id.
-    --   Use an INNER JOIN: a few trips have a pickup_location_id with no matching
-    --   zone (e.g. 999). INNER JOIN drops those so pickup_borough is never NULL and
-    --   can serve as part of the mart's primary key (your not_null test needs this).
-    -- TODO: aggregate to grain (pickup_borough, pickup_date)
-    -- Required output columns:
-    --   pickup_borough   TEXT     - z.borough
-    --   pickup_date      DATE     - pickup_datetime::date
-    --   trip_count       BIGINT   - count(*)
-    --   total_fare       NUMERIC  - sum(fare_amount)
-    --   avg_tip_pct      NUMERIC  - avg(tip_pct)
-    --   avg_trip_distance NUMERIC - avg(trip_distance)
-    NULL AS pickup_borough,
-    NULL AS pickup_date,
-    NULL AS trip_count,
-    NULL AS total_fare,
-    NULL AS avg_tip_pct,
-    NULL AS avg_trip_distance
-
+    z.borough as pickup_borough,
+    t.pickup_datetime::date as pickup_date,
+    count(*)::bigint as trip_count,
+    sum(t.fare_amount)::numeric as total_fare,
+    avg(t.tip_pct)::numeric as avg_tip_pct,
+    avg(t.trip_distance)::numeric as avg_trip_distance
 FROM trips t
--- TODO: add JOIN to zones here
--- TODO: add GROUP BY here
+inner join zones z
+    on t.pickup_location_id = z.location_id
+group by
+    z.borough,
+    t.pickup_datetime::date

models/staging/_sources.yml

@@ -2,9 +2,9 @@ version: 2
 
 sources:
   - name: nyc_taxi
-    schema: nyc_taxi     # TODO: confirm this matches the schema where raw_trips and raw_zones live
+    schema: nyc_taxi     
     tables:
       - name: raw_trips
-        description: "TODO: one sentence on what this table contains and its grain"
+        description: One row per raw NYC green taxi trip record from the loaded January 2024 dataset.
       - name: raw_zones
-        description: "TODO: one sentence on what this table contains"
+        description: One row per TLC taxi zone, mapping location IDs to borough and zone metadata.

models/staging/_stg_trips.yml

@@ -2,19 +2,28 @@ version: 2
 
 models:
   - name: stg_trips
-    description: "TODO: state the grain (one row per ___) and what source this reads from"
+    description: One row per cleaned NYC green taxi trip, read from nyc_taxi.raw_trips. Rows with missing pickup_location_id or negative fare_amount are removed before aggregation.
     columns:
       - name: pickup_datetime
-        description: "TODO"
-        # TODO: Task 5 -- add not_null tests on every column used as a join or
-        # group-by key. See the chapter's dbt Tests section for the syntax.
+        description: Wall-clock timestamp when the trip started; used to derive pickup_date in the mart.
+        tests:
+          - not_null
       - name: pickup_location_id
-        description: "TODO"
+        description: TLC pickup zone identifier; join key to stg_zones.location_id.
+        tests:
+          - not_null
+          - relationships:
+              to: ref('stg_zones')
+              field: location_id
+              severity: warn   
       - name: fare_amount
-        description: "TODO"
+        description: Metered trip fare in USD before tip; negative fares are filtered out in this staging model.
+        tests:
+          - not_null
       - name: tip_amount
-        description: "TODO"
+        description: Tip amount in USD recorded for the trip.
       - name: trip_distance
-        description: "TODO"
+        description: Trip distance in miles recorded for the trip. 
       - name: tip_pct
-        description: "TODO"
+        description: Ratio tip_amount / fare_amount; NULL when fare_amount is zero.
+

models/staging/_stg_zones.yml

@@ -2,11 +2,15 @@ version: 2
 
 models:
   - name: stg_zones
-    description: "TODO: state the grain and what source this reads from"
+    description: One row per TLC taxi zone from nyc_taxi.raw_zones, used as the borough lookup for pickup_location_id.
     columns:
       - name: location_id
-        description: "TODO"
-        # TODO: Task 5 -- this column is the join key. Which two generic tests
-        # guarantee a clean one-to-many join from stg_trips?
+        description: TLC taxi zone identifier; primary key of the zone lookup and join target for stg_trips.pickup_location_id.
+        tests:
+          - unique
+          - not_null
       - name: borough
-        description: "TODO"
+        description: Borough or special TLC borough label for the zone, including Manhattan, Brooklyn, Queens, Bronx, Staten Island, EWR, Unknown, and dirty NaN labels.
+        tests:
+          - not_null:
+              severity: warn

models/staging/stg_trips.sql

@@ -3,11 +3,13 @@
 -- Downstream: fct_daily_borough_stats joins this to stg_zones.
 
 SELECT
-    -- TODO: select the columns you need for the mart:
-    --   pickup_datetime, pickup_location_id, fare_amount, tip_amount, trip_distance
-    --
-    -- TODO: add tip_pct using {{ safe_divide('tip_amount', 'fare_amount') }}
-    --
-    -- TODO: filter out rows where pickup_location_id IS NULL or fare_amount < 0
-
+    pickup_datetime,
+    pickup_location_id,
+    fare_amount,
+    tip_amount,
+    trip_distance,
+    {{ safe_divide('tip_amount', 'fare_amount') }} as tip_pct
 FROM {{ source('nyc_taxi', 'raw_trips') }}
+where pickup_location_id is not null
+  and fare_amount >= 0
+

models/staging/stg_zones.sql

@@ -2,6 +2,6 @@
 -- Exposes location_id and borough for use as a lookup in the mart.
 
 SELECT
-    -- TODO: select location_id and borough from {{ source('nyc_taxi', 'raw_zones') }}
-
+    location_id,
+    borough
 FROM {{ source('nyc_taxi', 'raw_zones') }}

package-lock.yml

@@ -0,0 +1,5 @@
+packages:
+  - name: dbt_utils
+    package: dbt-labs/dbt_utils
+    version: 1.4.1
+sha1_hash: e6424ba9e5a22487e47f023803aa4f0411946808

packages.yml

@@ -1,5 +1,7 @@
-# TODO: Task 5 -- declare the dbt-labs/dbt_utils package here, then run
+
 # `dbt deps` to install it. You need it for the compound uniqueness test
 # on the mart. See https://hub.getdbt.com/dbt-labs/dbt_utils/latest/
 # for the package block syntax.
-packages: []
+packages: 
+  - package: dbt-labs/dbt_utils
+    version: [">=1.1.0", "<2.0.0"]

profiles.yml.example

@@ -3,14 +3,11 @@ nyc_taxi_borough_daily:
   outputs:
     dev:
       type: postgres
-      host: "{{ env_var('PG_HOST') }}"
+      host: hyf-data-pg.postgres.database.azure.com
       port: 5432
-      user: "{{ env_var('PG_USER') }}"
+      user: hyfadmin
       password: "{{ env_var('PG_PASSWORD') }}"
-      dbname: "{{ env_var('PG_DBNAME', 'postgres') }}"
-      schema: "dev_<your_name>"   # TODO: replace <your_name> with your first name (the schema you already own)
-      threads: 1
-
-# Copy this file to profiles.yml (same directory), fill in your name, and ensure
-# PG_HOST, PG_USER, PG_PASSWORD, and PG_DBNAME are set in your environment.
-# profiles.yml is git-ignored — never commit it with a real password.
+      dbname: team1
+      schema: dev_<your_name>
+      sslmode: require
+      threads: 4