diff --git a/02_activities/assignments/DC_Cohort/Assignment2.md b/02_activities/assignments/DC_Cohort/Assignment2.md index be0eff3a5..526158c62 100644 --- a/02_activities/assignments/DC_Cohort/Assignment2.md +++ b/02_activities/assignments/DC_Cohort/Assignment2.md @@ -56,7 +56,18 @@ The store wants to keep customer addresses. Propose two architectures for the CU **HINT:** search type 1 vs type 2 slowly changing dimensions. ``` -Your answer... +There are two possible architectures for the `CUSTOMER_ADDRESS` table, depending on whether the bookstore wants to keep a history of address changes. + +Type 1 — Overwrite changes + +In a Type 1 design, the customer's existing address is updated when they provide a new address. The table could contain columns such as `customer_address_id`, `customer_id`, `street`, `city`, `province`, `postal_code`, and `country`. If a customer moves, their old address is replaced with the new address. This approach is simple and only keeps the customer's current address, but it does not preserve any history of previous addresses. + +Type 2 — Retain changes + +In a Type 2 design, a new record is created whenever a customer's address changes, while the previous address remains in the table. The table could contain `customer_address_id`, `customer_id`, `street`, `city`, `province`, `postal_code`, `country`, `start_date`, `end_date`, and `is_current`. When the customer moves, the previous record can be given an end date and marked as no longer current, while a new record is created with the new address and a new start date. This allows the bookstore to maintain a complete history of the customer's addresses. + +Therefore, Type 1 overwrites the existing address and does not retain historical changes, while Type 2 creates new records and retains the history of address changes. Type 2 is useful when the bookstore needs to know where a customer lived at a particular point in time, while Type 1 is sufficient when only the customer's current address is important. + ``` *** @@ -191,5 +202,13 @@ Consider, for example, concepts of labour, bias, LLM proliferation, moderating c ``` -Your thoughts... +The article “Neural nets are just people all the way down” shows that machine learning systems are not completely independent or objective. Even though we often think of AI as something that is powered mainly by computers and algorithms, the article explains that humans play an important role in creating the data used to train these systems. People manually label images, organize categories, and make decisions about how information should be classified. These human decisions can introduce bias into the data and, as a result, into the machine learning system. + +One important ethical issue is that biased or inappropriate training data can lead to unfair outcomes. For example, the article discusses ImageNet and how some categories used to label people were considered offensive or sensitive. If an AI system learns from these labels, it may reproduce the same biases when making predictions about new people. This demonstrates that even if an algorithm is technically working as designed, the results can still be harmful if the data or categories used to train it are problematic. + +Another ethical issue is the labour involved in creating AI systems. Much of the work behind machine learning happens through people collecting, cleaning, labeling, and moderating data. This work can be hidden from the people who use AI systems, even though it is essential for making those systems function. This shows that AI is not simply created by computers; many human workers contribute to the development of these technologies. + +I think this means that people who create and use machine learning systems have a responsibility to think carefully about the data they use. They should check training data for bias, remove harmful or inappropriate categories, and consider how different groups of people could be affected by the system. It is also important to have diverse people involved in designing and reviewing these systems because different perspectives can help identify problems that one group might overlook. + +Overall, the article changed the way I think about AI because it shows that technology is not automatically neutral just because a computer is making the decision. Human choices exist throughout the process, from collecting and labeling data to designing the model and deciding how its results are used. Therefore, building responsible AI requires not only good technology but also careful human judgment, fairness, and accountability. ``` diff --git a/02_activities/assignments/DC_Cohort/assignment2.sql b/02_activities/assignments/DC_Cohort/assignment2.sql index 4079c18ae..7a8baba0d 100644 --- a/02_activities/assignments/DC_Cohort/assignment2.sql +++ b/02_activities/assignments/DC_Cohort/assignment2.sql @@ -1,180 +1,298 @@ /* ASSIGNMENT 2 */ ---Please write responses between the QUERY # and END QUERY blocks +/* Please write responses between the QUERY # and END QUERY blocks */ + /* SECTION 2 */ --- COALESCE -/* 1. Our favourite manager wants a detailed long list of products, but is afraid of tables! -We tell them, no problem! We can produce a list with all of the appropriate details. +/* COALESCE + +1. Our favourite manager wants a detailed long list of products, but is afraid of tables! + We tell them, no problem! We can produce a list with all of the appropriate details. Using the following syntax you create our super cool and not at all needy manager a list: -SELECT +SELECT product_name || ', ' || product_size|| ' (' || product_qty_type || ')' FROM product - -But wait! The product table has some bad data (a few NULL values). +But wait! The product table has some bad data (a few NULL values). Find the NULLs and then using COALESCE, replace the NULL with a blank for the first column with -nulls, and 'unit' for the second column with nulls. +nulls, and 'unit' for the second column with nulls. -**HINT**: keep the syntax the same, but edited the correct components with the string. -The `||` values concatenate the columns into strings. -Edit the appropriate columns -- you're making two edits -- and the NULL rows will be fixed. +HINT: keep the syntax the same, but edited the correct components with the string. +The || values concatenate the columns into strings. +Edit the appropriate columns -- you're making two edits -- and the NULL rows will be fixed. All the other rows will remain the same. */ ---QUERY 1 +/* QUERY 1 */ +SELECT +COALESCE(product_name, '') || ', ' || +COALESCE(product_size, '') || ' (' || +COALESCE(product_qty_type, 'unit') || ')' +FROM product; +/* END QUERY */ ---END QUERY +/* Windowed Functions */ - ---Windowed Functions -/* 1. Write a query that selects from the customer_purchases table and numbers each customer’s -visits to the farmer’s market (labeling each market date with a different number). -Each customer’s first visit is labeled 1, second visit is labeled 2, etc. +/* 1. Write a query that selects from the customer_purchases table and numbers each customer’s +visits to the farmer’s market (labeling each market date with a different number). +Each customer’s first visit is labeled 1, second visit is labeled 2, etc. You can either display all rows in the customer_purchases table, with the counter changing on -each new market date for each customer, or select only the unique market dates per customer -(without purchase details) and number those visits. -HINT: One of these approaches uses ROW_NUMBER() and one uses DENSE_RANK(). +each new market date for each customer, or select only the unique market dates per customer +(without purchase details) and number those visits. +HINT: One of these approaches uses ROW_NUMBER() and one uses DENSE_RANK(). Filter the visits to dates before April 29, 2022. */ ---QUERY 2 - - +/* QUERY 2 */ ---END QUERY +SELECT +customer_id, +market_date, +DENSE_RANK() OVER ( +PARTITION BY customer_id +ORDER BY market_date +) AS visit_number +FROM customer_purchases +WHERE market_date < '2022-04-29'; +/* END QUERY */ -/* 2. Reverse the numbering of the query so each customer’s most recent visit is labeled 1, -then write another query that uses this one as a subquery (or temp table) and filters the results to +/* 2. Reverse the numbering of the query so each customer’s most recent visit is labeled 1, +then write another query that uses this one as a subquery (or temp table) and filters the results to only the customer’s most recent visit. HINT: Do not use the previous visit dates filter. */ ---QUERY 3 +/* QUERY 3 */ +SELECT * +FROM ( +SELECT +customer_id, +market_date, +DENSE_RANK() OVER ( +PARTITION BY customer_id +ORDER BY market_date DESC +) AS visit_number +FROM customer_purchases +) +WHERE visit_number = 1; +/* END QUERY */ ---END QUERY - - -/* 3. Using a COUNT() window function, include a value along with each row of the -customer_purchases table that indicates how many different times that customer has purchased that product_id. +/* 3. Using a COUNT() window function, include a value along with each row of the +customer_purchases table that indicates how many different times that customer has purchased that product_id. You can make this a running count by including an ORDER BY within the PARTITION BY if desired. Filter the visits to dates before April 29, 2022. */ ---QUERY 4 - - - - ---END QUERY - - --- String manipulations -/* 1. Some product names in the product table have descriptions like "Jar" or "Organic". -These are separated from the product name with a hyphen. -Create a column using SUBSTR (and a couple of other commands) that captures these, but is otherwise NULL. -Remove any trailing or leading whitespaces. Don't just use a case statement for each product! - -| product_name | description | -|----------------------------|-------------| -| Habanero Peppers - Organic | Organic | - -Hint: you might need to use INSTR(product_name,'-') to find the hyphens. INSTR will help split the column. */ ---QUERY 5 - - - - ---END QUERY +/* QUERY 4 */ + +SELECT +customer_id, +product_id, +market_date, +COUNT(*) OVER ( +PARTITION BY customer_id, product_id +) AS purchase_count +FROM customer_purchases +WHERE market_date < '2022-04-29'; + +/* END QUERY */ + +/* String manipulations */ + +/* 1. Some product names in the product table have descriptions like "Jar" or "Organic". +These are separated from the product name with a hyphen. +Create a column using SUBSTR (and a couple of other commands) that captures these, but is otherwise NULL. +Remove any trailing or leading whitespaces. Don't just use a case statement for each product! */ + +/* QUERY 5 */ + +SELECT +product_name, +CASE +WHEN INSTR(product_name, '-') > 0 +THEN TRIM( +SUBSTR( +product_name, +INSTR(product_name, '-') + 1 +) +) +ELSE NULL +END AS description +FROM product; + +/* END QUERY */ /* 2. Filter the query to show any product_size value that contain a number with REGEXP. */ ---QUERY 6 - +/* QUERY 6 */ +SELECT +product_name, +product_size +FROM product +WHERE product_size REGEXP '[0-9]'; ---END QUERY +/* END QUERY */ +/* UNION */ --- UNION /* 1. Using a UNION, write a query that displays the market dates with the highest and lowest total sales. -HINT: There are a possibly a few ways to do this query, but if you're struggling, try the following: -1) Create a CTE/Temp Table to find sales values grouped dates; -2) Create another CTE/Temp table with a rank windowed function on the previous query to create -"best day" and "worst day"; -3) Query the second temp table twice, once for the best day, once for the worst day, -with a UNION binding them. */ ---QUERY 7 - - - - ---END QUERY - - +HINT: There are a possibly a few ways to do this query, but if you're struggling, try the following: + +1. Create a CTE/Temp Table to find sales values grouped dates; +2. Create another CTE/Temp table with a rank windowed function on the previous query to create + "best day" and "worst day"; +3. Query the second temp table twice, once for the best day, once for the worst day, + with a UNION binding them. */ + +/* QUERY 7 */ + +WITH daily_sales AS ( +SELECT +market_date, +SUM(quantity * cost_per_quantity) AS total_sales +FROM customer_purchases +GROUP BY market_date +), +ranked_sales AS ( +SELECT +market_date, +total_sales, +RANK() OVER ( +ORDER BY total_sales DESC +) AS best_day, +RANK() OVER ( +ORDER BY total_sales ASC +) AS worst_day +FROM daily_sales +) +SELECT +market_date, +total_sales +FROM ranked_sales +WHERE best_day = 1 + +UNION + +SELECT +market_date, +total_sales +FROM ranked_sales +WHERE worst_day = 1; + +/* END QUERY */ /* SECTION 3 */ --- Cross Join -/*1. Suppose every vendor in the `vendor_inventory` table had 5 of each of their products to sell to **every** -customer on record. How much money would each vendor make per product? +/* Cross Join */ + +/* 1. Suppose every vendor in the vendor_inventory table had 5 of each of their products to sell to every +customer on record. How much money would each vendor make per product? Show this by vendor_name and product name, rather than using the IDs. -HINT: Be sure you select only relevant columns and rows. -Remember, CROSS JOIN will explode your table rows, so CROSS JOIN should likely be a subquery. +HINT: Be sure you select only relevant columns and rows. +Remember, CROSS JOIN will explode your table rows, so CROSS JOIN should likely be a subquery. Think a bit about the row counts: how many distinct vendors, product names are there (x)? -How many customers are there (y). -Before your final group by you should have the product of those two queries (x*y). */ ---QUERY 8 - - - - ---END QUERY - - --- INSERT -/*1. Create a new table "product_units". -This table will contain only products where the `product_qty_type = 'unit'`. -It should use all of the columns from the product table, as well as a new column for the `CURRENT_TIMESTAMP`. -Name the timestamp column `snapshot_timestamp`. */ ---QUERY 9 - - - - ---END QUERY +How many customers are there (y). +Before your final group by you should have the product of those two queries (x*y). */ + +/* QUERY 8 */ + +SELECT +v.vendor_name, +p.product_name, +SUM(5 * vi.cost_per_quantity) AS total_sales +FROM ( +SELECT DISTINCT +vendor_id, +product_id, +cost_per_quantity +FROM vendor_inventory +) vi +JOIN vendor v +ON vi.vendor_id = v.vendor_id +JOIN product p +ON vi.product_id = p.product_id +CROSS JOIN ( +SELECT DISTINCT +customer_id +FROM customer +) c +GROUP BY +v.vendor_name, +p.product_name; + +/* END QUERY */ + +/* INSERT */ + +/* 1. Create a new table "product_units". +This table will contain only products where the product_qty_type = 'unit'. +It should use all of the columns from the product table, as well as a new column for the CURRENT_TIMESTAMP. +Name the timestamp column snapshot_timestamp. */ + +/* QUERY 9 */ + +CREATE TABLE product_units AS +SELECT +*, +CURRENT_TIMESTAMP AS snapshot_timestamp +FROM product +WHERE product_qty_type = 'unit'; +/* END QUERY */ -/*2. Using `INSERT`, add a new row to the product_units table (with an updated timestamp). +/* 2. Using INSERT, add a new row to the product_units table (with an updated timestamp). This can be any product you desire (e.g. add another record for Apple Pie). */ ---QUERY 10 - - +/* QUERY 10 */ + +INSERT INTO product_units ( +product_id, +product_name, +product_size, +product_category_id, +product_qty_type, +snapshot_timestamp +) +SELECT +product_id, +product_name, +product_size, +product_category_id, +product_qty_type, +CURRENT_TIMESTAMP +FROM product +WHERE product_name = 'Apple Pie'; ---END QUERY - +/* END QUERY */ --- DELETE -/* 1. Delete the older record for whatever product you added. +/* DELETE */ -HINT: If you don't specify a WHERE clause, you are going to have a bad time.*/ ---QUERY 11 +/* 1. Delete the older record for whatever product you added. +HINT: If you don't specify a WHERE clause, you are going to have a bad time. */ +/* QUERY 11 */ +DELETE FROM product_units +WHERE product_name = 'Apple Pie' +AND snapshot_timestamp = ( +SELECT MIN(snapshot_timestamp) +FROM product_units +WHERE product_name = 'Apple Pie' +); ---END QUERY +/* END QUERY */ +/* UPDATE */ --- UPDATE -/* 1.We want to add the current_quantity to the product_units table. +/* 1. We want to add the current_quantity to the product_units table. First, add a new column, current_quantity to the table using the following syntax. ALTER TABLE product_units @@ -182,19 +300,31 @@ ADD current_quantity INT; Then, using UPDATE, change the current_quantity equal to the last quantity value from the vendor_inventory details. -HINT: This one is pretty hard. -First, determine how to get the "last" quantity per product. -Second, coalesce null values to 0 (if you don't have null values, figure out how to rearrange your query so you do.) -Third, SET current_quantity = (...your select statement...), remembering that WHERE can only accommodate one column. -Finally, make sure you have a WHERE statement to update the right row, - you'll need to use product_units.product_id to refer to the correct row within the product_units table. +HINT: This one is pretty hard. +First, determine how to get the "last" quantity per product. +Second, coalesce null values to 0 (if you don't have null values, figure out how to rearrange your query so you do.) +Third, SET current_quantity = (...your select statement...), remembering that WHERE can only accommodate one column. +Finally, make sure you have a WHERE statement to update the right row, +you'll need to use product_units.product_id to refer to the correct row within the product_units table. When you have all of these components, you can run the update statement. */ ---QUERY 12 - +/* QUERY 12 */ +ALTER TABLE product_units +ADD current_quantity INT; ---END QUERY +/* END QUERY */ +/* QUERY 13 */ +UPDATE product_units +SET current_quantity = ( +SELECT COALESCE(vi.quantity, 0) +FROM vendor_inventory vi +WHERE vi.product_id = product_units.product_id +ORDER BY vi.market_date DESC +LIMIT 1 +) +WHERE product_units.product_id IS NOT NULL; +/* END QUERY */ diff --git a/02_activities/assignments/DC_Cohort/prompt 1.drawio.png b/02_activities/assignments/DC_Cohort/prompt 1.drawio.png new file mode 100644 index 000000000..109312228 Binary files /dev/null and b/02_activities/assignments/DC_Cohort/prompt 1.drawio.png differ diff --git a/02_activities/assignments/DC_Cohort/prompt_2.drawio b/02_activities/assignments/DC_Cohort/prompt_2.drawio new file mode 100644 index 000000000..765932215 --- /dev/null +++ b/02_activities/assignments/DC_Cohort/prompt_2.drawio @@ -0,0 +1,90 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +