diff --git a/01_materials/slides/slides_03.pdf b/01_materials/slides/slides_03.pdf index f52e94d95..4885e33a4 100644 Binary files a/01_materials/slides/slides_03.pdf and b/01_materials/slides/slides_03.pdf differ diff --git a/02_activities/assignments/DC_Cohort/Assignment1.md b/02_activities/assignments/DC_Cohort/Assignment1.md index 1e8903b24..f890d3fe4 100644 --- a/02_activities/assignments/DC_Cohort/Assignment1.md +++ b/02_activities/assignments/DC_Cohort/Assignment1.md @@ -127,7 +127,7 @@ Steps to complete this part of the assignment: #### WHERE 1. Write a query that returns all customer purchases of product IDs 4 and 9. Limit to 25 rows of output. -2. Write a query that returns all customer purchases and a new calculated column 'price' (quantity * cost_to_customer_per_qty), filtered by customer IDs between 8 and 10 (inclusive) using either: +2. Write a query that returns all customer purchases and a new calculated column 'price' (quantity * cost_per_quantity), filtered by customer IDs between 8 and 10 (inclusive) using either: 1. two conditions using AND 2. one condition using BETWEEN
Limit to 25 rows of output. @@ -182,7 +182,7 @@ To insert the new row use VALUES, specifying the value you want for each column: Limit to 25 rows of output. -2. Using the previous query as a base, determine how much money each customer spent in April 2022. Remember that money spent is `quantity*cost_to_customer_per_qty`. +2. Using the previous query as a base, determine how much money each customer spent in April 2022. Remember that money spent is `quantity*cost_per_quantity`. **HINTS**: you will need to AGGREGATE, GROUP BY, and filter...but remember, STRFTIME returns a STRING for your WHERE statement... AND be sure you remove the LIMIT from the previous query before aggregating!! @@ -209,5 +209,18 @@ Consider, for example, concepts of fariness, inequality, social structures, marg ``` -Your thoughts... +# + +Rida Qadri’s article about Pakistan’s National Database and Registration Authority (NADRA) shows that databases are not just technical systems used to store information. They are also built based on decisions about what information matters and how people should be categorized. One thing that stood out to me was how NADRA’s database had a specific idea of what a family should look like. People had to fit into the relationships and categories that were already designed in the system. When someone’s real-life situation did not match those categories, the problem was not with the person’s information; the problem was that the database was not designed to represent their reality. + +This made me realize that database design is not always neutral. The choices made when creating tables, fields, and relationships can reflect the values and assumptions of the people who build the system. For example, if a database only allows certain options for family relationships, gender, or legal status, it may unintentionally exclude people who do not fit those options. Once these choices become part of a system used by organizations or governments, they can affect people’s access to important services and opportunities. + +I can see similar examples in everyday systems I interact with. Many forms I complete for school, banking, healthcare, or other services have fixed categories. For example, a form may ask for marital status, gender, or an emergency contact relationship using a limited list of choices. These categories are usually created to make data easier to organize, but they may not represent everyone’s situation. People who do not fit the available options may have to choose an inaccurate answer, leave information incomplete, or spend extra time explaining their circumstances. + +The farmers market database we worked with in this course also shows how database structures involve decisions. For example, the `customer_purchases` table assumes that each purchase is connected to a specific customer through a `customer_id`. The `vendor_booth_assignments` table assumes a clear relationship between vendors, booths, and market dates. These choices make the database organized and useful, but they also represent decisions about how the real world is being modeled. Someone has to decide what information should be stored and how different parts of the data connect. + +The biggest lesson I took from Qadri’s article is that the impact of a database is not only measured by whether it works technically. We also need to think about who is included, who might be left out, and whose experiences are not represented. A system can be efficient and still create problems if it is based on assumptions that do not apply to everyone. + +As someone learning data science, this is something I want to keep in mind. Building databases and models is not only about writing correct queries or creating efficient systems. It is also about understanding that the way we organize data can influence real people. Even small design choices, like what categories exist in a table or what relationships are allowed, can have meaningful consequences. + ``` diff --git a/02_activities/assignments/DC_Cohort/assignment1.sql b/02_activities/assignments/DC_Cohort/assignment1.sql index 2ec561e2a..f8919cec4 100644 --- a/02_activities/assignments/DC_Cohort/assignment1.sql +++ b/02_activities/assignments/DC_Cohort/assignment1.sql @@ -6,20 +6,18 @@ --SELECT /* 1. Write a query that returns everything in the customer table. */ --QUERY 1 - - - - +SELECT * +FROM customer; --END QUERY /* 2. Write a query that displays all of the columns and 10 rows from the customer table, sorted by customer_last_name, then customer_first_ name. */ --QUERY 2 - - - - +SELECT * +FROM customer +ORDER BY customer_last_name ASC, customer_first_name ASC +LIMIT 10; --END QUERY @@ -27,25 +25,26 @@ sorted by customer_last_name, then customer_first_ name. */ /* 1. Write a query that returns all customer purchases of product IDs 4 and 9. Limit to 25 rows of output. */ --QUERY 3 - - - - +SELECT * +FROM customer_purchases +WHERE product_id IN (4, 9) +LIMIT 25; --END QUERY -/*2. Write a query that returns all customer purchases and a new calculated column 'price' (quantity * cost_to_customer_per_qty), +/*2. Write a query that returns all customer purchases and a new calculated column 'price' (quantity * cost_per_quantity), filtered by customer IDs between 8 and 10 (inclusive) using either: 1. two conditions using AND 2. one condition using BETWEEN Limit to 25 rows of output. */ --QUERY 4 - - - - +SELECT *, + quantity * cost_per_quantity AS price +FROM customer_purchases +WHERE customer_id BETWEEN 8 AND 10 +LIMIT 25; --END QUERY @@ -55,10 +54,14 @@ Using the product table, write a query that outputs the product_id and product_n columns and add a column called prod_qty_type_condensed that displays the word “unit” if the product_qty_type is “unit,” and otherwise displays the word “bulk.” */ --QUERY 5 - - - - +SELECT + product_id, + product_name, + CASE + WHEN product_qty_type = 'unit' THEN 'unit' + ELSE 'bulk' + END AS prod_qty_type_condensed +FROM product; --END QUERY @@ -66,10 +69,18 @@ if the product_qty_type is “unit,” and otherwise displays the word “bulk. add a column to the previous query called pepper_flag that outputs a 1 if the product_name contains the word “pepper” (regardless of capitalization), and otherwise outputs 0. */ --QUERY 6 - - - - +SELECT + product_id, + product_name, + CASE + WHEN product_qty_type = 'unit' THEN 'unit' + ELSE 'bulk' + END AS prod_qty_type_condensed, + CASE + WHEN LOWER(product_name) LIKE '%pepper%' THEN 1 + ELSE 0 + END AS pepper_flag +FROM product; --END QUERY @@ -78,10 +89,12 @@ contains the word “pepper” (regardless of capitalization), and otherwise out vendor_id field they both have in common, and sorts the result by market_date, then vendor_name. Limit to 24 rows of output. */ --QUERY 7 - - - - +SELECT * +FROM vendor +INNER JOIN vendor_booth_assignments +ON vendor.vendor_id = vendor_booth_assignments.vendor_id +ORDER BY market_date ASC, vendor_name ASC +LIMIT 24; --END QUERY @@ -92,10 +105,11 @@ Limit to 24 rows of output. */ /* 1. Write a query that determines how many times each vendor has rented a booth at the farmer’s market by counting the vendor booth assignments per vendor_id. */ --QUERY 8 - - - - +SELECT + vendor_id, + COUNT(*) AS booth_rentals +FROM vendor_booth_assignments +GROUP BY vendor_id; --END QUERY @@ -105,10 +119,17 @@ of customers for them to give stickers to, sorted by last name, then first name. HINT: This query requires you to join two tables, use an aggregate function, and use the HAVING keyword. */ --QUERY 9 - - - - +SELECT + customer.customer_id, + customer_first_name, + customer_last_name, + SUM(quantity * cost_per_quantity) AS total_spent +FROM customer +INNER JOIN customer_purchases +ON customer.customer_id = customer_purchases.customer_id +GROUP BY customer.customer_id +HAVING total_spent > 2000 +ORDER BY customer_last_name ASC, customer_first_name ASC; --END QUERY @@ -124,10 +145,13 @@ When inserting the new vendor, you need to appropriately align the columns to be VALUES(col1,col2,col3,col4,col5) */ --QUERY 10 +CREATE TEMP TABLE new_vendor AS +SELECT * +FROM vendor; - - - +INSERT INTO new_vendor +VALUES +(10, 'Thomass Superfood Store', 'Fresh Focused', 'Thomas', 'Rosenthal'); --END QUERY @@ -138,22 +162,27 @@ HINT: you might need to search for strfrtime modifers sqlite on the web to know and year are! Limit to 25 rows of output. */ --QUERY 11 - - - - +SELECT + customer_id, + STRFTIME('%m', market_date) AS month, + STRFTIME('%Y', market_date) AS year +FROM customer_purchases +LIMIT 25; --END QUERY /* 2. Using the previous query as a base, determine how much money each customer spent in April 2022. -Remember that money spent is quantity*cost_to_customer_per_qty. +Remember that money spent is quantity*cost_per_quantity. HINTS: you will need to AGGREGATE, GROUP BY, and filter... but remember, STRFTIME returns a STRING for your WHERE statement... AND be sure you remove the LIMIT from the previous query before aggregating!! */ --QUERY 12 - - - - +SELECT + customer_id, + SUM(quantity * cost_per_quantity) AS total_spent +FROM customer_purchases +WHERE STRFTIME('%m', market_date) = '04' +AND STRFTIME('%Y', market_date) = '2022' +GROUP BY customer_id; --END QUERY diff --git a/02_activities/assignments/DC_Cohort/assignment1_image.png b/02_activities/assignments/DC_Cohort/assignment1_image.png new file mode 100644 index 000000000..8c0ae7341 Binary files /dev/null and b/02_activities/assignments/DC_Cohort/assignment1_image.png differ diff --git a/02_activities/assignments/Microcredential_Cohort/Assignment1.md b/02_activities/assignments/Microcredential_Cohort/Assignment1.md index 5483663c1..45f0f5250 100644 --- a/02_activities/assignments/Microcredential_Cohort/Assignment1.md +++ b/02_activities/assignments/Microcredential_Cohort/Assignment1.md @@ -126,7 +126,7 @@ Steps to complete this part of the assignment: #### WHERE 1. Write a query that returns all customer purchases of product IDs 4 and 9. Limit to 25 rows of output. -2. Write a query that returns all customer purchases and a new calculated column 'price' (quantity * cost_to_customer_per_qty), filtered by customer IDs between 8 and 10 (inclusive) using either: +2. Write a query that returns all customer purchases and a new calculated column 'price' (quantity * cost_per_quantity), filtered by customer IDs between 8 and 10 (inclusive) using either: 1. two conditions using AND 2. one condition using BETWEEN
Limit to 25 rows of output. @@ -177,11 +177,11 @@ To insert the new row use VALUES, specifying the value you want for each column: #### Date 1. Get the customer_id, month, and year (in separate columns) of every purchase in the customer_purchases table. -**HINT**: you might need to search for strfrtime modifers sqlite on the web to know what the modifers for month and year are! +**HINT**: you might need to search for strfrtime modifiers sqlite on the web to know what the modifiers for month and year are! Limit to 25 rows of output. -2. Using the previous query as a base, determine how much money each customer spent in April 2022. Remember that money spent is `quantity*cost_to_customer_per_qty`. +2. Using the previous query as a base, determine how much money each customer spent in April 2022. Remember that money spent is `quantity*cost_per_quantity`. **HINTS**: you will need to AGGREGATE, GROUP BY, and filter...but remember, STRFTIME returns a STRING for your WHERE statement... AND be sure you remove the LIMIT from the previous query before aggregating!! diff --git a/02_activities/assignments/Microcredential_Cohort/assignment1.sql b/02_activities/assignments/Microcredential_Cohort/assignment1.sql index 2ec561e2a..518415744 100644 --- a/02_activities/assignments/Microcredential_Cohort/assignment1.sql +++ b/02_activities/assignments/Microcredential_Cohort/assignment1.sql @@ -35,7 +35,7 @@ Limit to 25 rows of output. */ -/*2. Write a query that returns all customer purchases and a new calculated column 'price' (quantity * cost_to_customer_per_qty), +/*2. Write a query that returns all customer purchases and a new calculated column 'price' (quantity * cost_per_quantity), filtered by customer IDs between 8 and 10 (inclusive) using either: 1. two conditions using AND 2. one condition using BETWEEN @@ -134,7 +134,7 @@ VALUES(col1,col2,col3,col4,col5) -- Date /*1. Get the customer_id, month, and year (in separate columns) of every purchase in the customer_purchases table. -HINT: you might need to search for strfrtime modifers sqlite on the web to know what the modifers for month +HINT: you might need to search for strfrtime modifiers sqlite on the web to know what the modifiers for month and year are! Limit to 25 rows of output. */ --QUERY 11 @@ -146,7 +146,7 @@ Limit to 25 rows of output. */ /* 2. Using the previous query as a base, determine how much money each customer spent in April 2022. -Remember that money spent is quantity*cost_to_customer_per_qty. +Remember that money spent is quantity*cost_per_quantity. HINTS: you will need to AGGREGATE, GROUP BY, and filter... but remember, STRFTIME returns a STRING for your WHERE statement... diff --git a/03_instructional_team/markdown_slides/images/03_subquery_where.png b/03_instructional_team/markdown_slides/images/03_subquery_where.png index 2c9a523c9..2f0d66767 100644 Binary files a/03_instructional_team/markdown_slides/images/03_subquery_where.png and b/03_instructional_team/markdown_slides/images/03_subquery_where.png differ diff --git a/04_this_cohort/custom_slides/markdown/imgs/03_subquery_where.png b/04_this_cohort/custom_slides/markdown/imgs/03_subquery_where.png index 2c9a523c9..2f0d66767 100644 Binary files a/04_this_cohort/custom_slides/markdown/imgs/03_subquery_where.png and b/04_this_cohort/custom_slides/markdown/imgs/03_subquery_where.png differ diff --git a/04_this_cohort/custom_slides/markdown/xaringan-themer.css b/04_this_cohort/custom_slides/markdown/xaringan-themer.css index 6ca56a367..31881c1f1 100644 --- a/04_this_cohort/custom_slides/markdown/xaringan-themer.css +++ b/04_this_cohort/custom_slides/markdown/xaringan-themer.css @@ -45,7 +45,7 @@ /* Colors */ --text-color: #657b83; --header-color: #dc322f; - --background-color: #000000; + --background-color: #d3d2d9; --link-color: #b58900; --text-bold-color: #d33682; --code-highlight-color: #268bd2; @@ -56,7 +56,7 @@ --title-slide-background-color: #000000; --title-slide-text-color: #fdf6e3; --header-background-color: #dc322f; - --header-background-text-color: #000000; + --header-background-text-color: #d3d2d9; } html { @@ -190,7 +190,7 @@ th, td { .remark-slide table:not(.table-unshaded) thead, .remark-slide table:not(.table-unshaded) tfoot, .remark-slide table:not(.table-unshaded) tr:nth-child(even) { - background: #2C2D26; + background: #000000; } table.dataTable tbody { background-color: var(--background-color); @@ -200,7 +200,7 @@ table.dataTable.display tbody tr.odd { background-color: var(--background-color); } table.dataTable.display tbody tr.even { - background-color: #2C2D26; + background-color: #000000; } table.dataTable.hover tbody tr:hover, table.dataTable.display tbody tr:hover { background-color: rgba(255, 255, 255, 0.5); diff --git a/04_this_cohort/custom_slides/pdf/slides_03.pdf b/04_this_cohort/custom_slides/pdf/slides_03.pdf index 99dfc81d4..74ab946e1 100644 Binary files a/04_this_cohort/custom_slides/pdf/slides_03.pdf and b/04_this_cohort/custom_slides/pdf/slides_03.pdf differ diff --git a/04_this_cohort/live_code/module_2/CASE.sql b/04_this_cohort/live_code/module_2/CASE.sql new file mode 100644 index 000000000..4c6fc12ef --- /dev/null +++ b/04_this_cohort/live_code/module_2/CASE.sql @@ -0,0 +1,41 @@ +/* MODULE 2 */ +/* CASE */ + + +SELECT * +/* 1. Add a CASE statement declaring which days vendors should come */ +,CASE WHEN vendor_type = 'Fresh Focused' THEN 'Wednesday' + WHEN vendor_type = 'Eggs & Meats' THEN 'Thursday' + ELSE 'Saturday' + END as day_of_specialty + +/* 2. Add another CASE statement for Pie Day */ +,CASE WHEN vendor_name = "Annie's Pies" -- double quotes okay here + THEN 'Annie is the best' + END as pie_day + + +/* 3. Add another CASE statement with an ELSE clause to handle rows evaluating to False */ +,CASE WHEN vendor_name LIKE '%pie%' + THEN 'Wednesday' -- this came first for Annie even though she sells Prepared Foods too + WHEN vendor_type = 'Prepared Foods' + THEN 'Thursday' + ELSE 'Friday' + END as pies_special_day + + +FROM vendor; + +/* 4. Experiment with selecting a different column instead of just a string value */ +SELECT * +,CASE WHEN cost_per_quantity < 1.00 + THEN cost_per_quantity*5 + ELSE cost_per_quantity + END as inflation + +FROM customer_purchases + + + + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_2/DISTINCT.sql b/04_this_cohort/live_code/module_2/DISTINCT.sql new file mode 100644 index 000000000..478ed258c --- /dev/null +++ b/04_this_cohort/live_code/module_2/DISTINCT.sql @@ -0,0 +1,43 @@ +/* MODULE 2 */ +/* DISTINCT */ + + +/* 1. Compare how many customer_ids are in the customer_purchases table, one select with distinct, one without */ + +-- 4221 rows +SELECT customer_id FROM customer_purchases; + +SELECT DISTINCT customer_id FROM customer_purchases; + + + +/* 2. Compare the difference between selecting market_day in market_date_info, with and without distinct: + what do these difference mean?*/ + + -- market is open for 150 days +SELECT market_day +FROM market_date_info; + +-- market is only open 2 days, saturday and wednesday +SELECT DISTINCT market_day +FROM market_date_info; + + +/* 3. Which vendor has sold products to a customer */ +SELECT DISTINCT vendor_id +FROM customer_purchases; +-- only 3 vendors have sold anything at the farmers market :( + + +/* 4. Which vendor has sold products to a customer ... and which product was it */ +SELECT DISTINCT vendor_id, product_id +FROM customer_purchases; + + +/* 5. Which vendor has sold products to a customer +... and which product was it? +... AND to whom was it sold*/ +SELECT DISTINCT vendor_id, product_id, customer_id +FROM customer_purchases; + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_2/INNER_JOIN.sql b/04_this_cohort/live_code/module_2/INNER_JOIN.sql new file mode 100644 index 000000000..aa9ca9f2e --- /dev/null +++ b/04_this_cohort/live_code/module_2/INNER_JOIN.sql @@ -0,0 +1,44 @@ +/* MODULE 2 */ +/* INNER JOIN */ + + +/* 1. Get product names (from product table) alongside customer_purchases + ... use an INNER JOIN to see only products that have been purchased */ + +-- without table aliases +SELECT product_name, -- coming from the product TABLE +vendor_id, -- rest of these are coming from customer_purchases table +market_date, +customer_id, +customer_purchases.product_id, +product.product_id + + +FROM product +INNER JOIN customer_purchases + ON customer_purchases.product_id = product.product_id; + + + + +/* 2. Using the Query #5 from DISTINCT earlier + (Which vendor has sold products to a customer AND which product was it AND to whom was it sold) + + Add customers' first and last names with an INNER JOIN */ + +-- using table aliases +SELECT DISTINCT +vendor_id, -- coming from cp +product_id, -- coming from cp +c.customer_id, -- coming from c (and below) +customer_first_name, +customer_last_name + +FROM customer_purchases AS cp +INNER JOIN customer AS c + ON cp.customer_id = c.customer_id + + + + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_2/LEFT_JOIN.sql b/04_this_cohort/live_code/module_2/LEFT_JOIN.sql new file mode 100644 index 000000000..9a8286979 --- /dev/null +++ b/04_this_cohort/live_code/module_2/LEFT_JOIN.sql @@ -0,0 +1,63 @@ +/* MODULE 2 */ +/* LEFT JOIN */ + + +/* 1. There are products that have been bought +... but are there products that have not been bought? +Use a LEFT JOIN to find out*/ +SELECT DISTINCT +p.product_id +,cp.product_id as [cp_product_id] +,product_name + +FROM product as p +LEFT JOIN customer_purchases as cp + ON p.product_id = cp.product_id; + + +/* 2. Directions of LEFT JOINs matter ...*/ +SELECT DISTINCT +p.product_id +,cp.product_id as [cp_product_id] +,product_name + +FROM customer_purchases as cp +LEFT JOIN product as p + ON p.product_id = cp.product_id; + + +/* 3. As do which values you filter on ... */ +SELECT DISTINCT +p.product_id +,cp.product_id as [cp_product_id] +,product_name + +FROM product as p +LEFT JOIN customer_purchases as cp + ON p.product_id = cp.product_id +WHERE cp.product_id BETWEEN 1 AND 6; -- if we pick p product 6 rows (1-6)...but if we pick cp customer_purchases only 5 rows + + + +/* 4. Without using a RIGHT JOIN, make this query return the RIGHT JOIN result set +...**Hint, flip the order of the joins** ... + +SELECT * + +FROM product_category AS pc +LEFT JOIN product AS p + ON pc.product_category_id = p.product_category_id + ORDER by pc.product_category_id + +...Note how the row count changed from 24 to 23 +*/ + +SELECT * + +FROM product AS p +LEFT JOIN product_category AS pc + ON pc.product_category_id = p.product_category_id + +ORDER by pc.product_category_id + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_2/SELECT.sql b/04_this_cohort/live_code/module_2/SELECT.sql new file mode 100644 index 000000000..3b5e586a4 --- /dev/null +++ b/04_this_cohort/live_code/module_2/SELECT.sql @@ -0,0 +1,33 @@ +/* MODULE 2 */ +/* SELECT */ + + +/* 1. Select everything in the customer table */ +SELECT * FROM customer; + +/* 2. Use sql as a calculator */ +SELECT 1+1 AS addition, 10*5 as multiplication, pi() as pi; + + +/* 3. Add order by and limit clauses */ +SELECT * +FROM customer +ORDER BY customer_first_name DESC +LIMIT 10; + + +/* 4. Select multiple specific columns */ +SELECT +customer_last_name, +customer_first_name, -- comma between column names +customer_postal_code + +FROM customer; + + +/* 5. Add a static value in a column */ +SELECT 2026 as this_year, 'July' as this_month, customer_id +FROM customer + + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_2/WHERE.sql b/04_this_cohort/live_code/module_2/WHERE.sql new file mode 100644 index 000000000..2f480d35a --- /dev/null +++ b/04_this_cohort/live_code/module_2/WHERE.sql @@ -0,0 +1,48 @@ +/* MODULE 2 */ +/* WHERE */ + +/* 1. Select only customer 1 from the customer table */ +SELECT * +FROM customer +WHERE customer_id = 1; + + +/* 2. Differentiate between AND and OR */ +SELECT * +FROM customer +WHERE customer_id = 1 +AND customer_id = 2; -- OR is two rows, AND is 0 rows + + +/* 3. IN */ +SELECT * +FROM customer +WHERE customer_id IN (3,5,6,7); + + + +/* 4. LIKE */ +-- find all the peppers +SELECT * FROM product +WHERE product_name LIKE '%pepper%'; + + +/* 5. Nulls and Blanks*/ +SELECT * FROM product +WHERE product_size IS NULL +OR product_size = ''; -- blank, two single quotes NOT one double quote, different from NULL + + +/* 6. BETWEEN x AND y */ +SELECT * +FROM customer +WHERE customer_id BETWEEN 1 AND 20 +ORDER BY customer_id ASC; + +--dates +SELECT * +FROM market_date_info +WHERE market_date BETWEEN '2022-10-01' AND '2022-10-31' + + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_2/module_2.sqbpro b/04_this_cohort/live_code/module_2/module_2.sqbpro index 0719e69a4..05ca33f4c 100644 --- a/04_this_cohort/live_code/module_2/module_2.sqbpro +++ b/04_this_cohort/live_code/module_2/module_2.sqbpro @@ -1,23 +1,33 @@ -
/* MODULE 2 */ +
/* MODULE 2 */ /* SELECT */ /* 1. Select everything in the customer table */ -SELECT +SELECT * FROM customer; /* 2. Use sql as a calculator */ - +SELECT 1+1 AS addition, 10*5 as multiplication, pi() as pi; /* 3. Add order by and limit clauses */ - +SELECT * +FROM customer +ORDER BY customer_first_name DESC +LIMIT 10; /* 4. Select multiple specific columns */ +SELECT +customer_last_name, +customer_first_name, -- comma between column names +customer_postal_code +FROM customer; /* 5. Add a static value in a column */ +SELECT 2026 as this_year, 'July' as this_month, customer_id +FROM customer -------------------------------------------------------------------------------------------------------------------------------------------- @@ -27,26 +37,45 @@ SELECT /* 1. Select only customer 1 from the customer table */ SELECT * FROM customer -WHERE +WHERE customer_id = 1; /* 2. Differentiate between AND and OR */ - +SELECT * +FROM customer +WHERE customer_id = 1 +AND customer_id = 2; -- OR is two rows, AND is 0 rows /* 3. IN */ +SELECT * +FROM customer +WHERE customer_id IN (3,5,6,7); /* 4. LIKE */ - +-- find all the peppers +SELECT * FROM product +WHERE product_name LIKE '%pepper%'; /* 5. Nulls and Blanks*/ - +SELECT * FROM product +WHERE product_size IS NULL +OR product_size = ''; -- blank, two single quotes NOT one double quote, different from NULL /* 6. BETWEEN x AND y */ +SELECT * +FROM customer +WHERE customer_id BETWEEN 1 AND 20 +ORDER BY customer_id ASC; + +--dates +SELECT * +FROM market_date_info +WHERE market_date BETWEEN '2022-10-01' AND '2022-10-31' -------------------------------------------------------------------------------------------------------------------------------------------- @@ -56,20 +85,36 @@ WHERE SELECT * /* 1. Add a CASE statement declaring which days vendors should come */ - +,CASE WHEN vendor_type = 'Fresh Focused' THEN 'Wednesday' + WHEN vendor_type = 'Eggs & Meats' THEN 'Thursday' + ELSE 'Saturday' + END as day_of_specialty /* 2. Add another CASE statement for Pie Day */ - +,CASE WHEN vendor_name = "Annie's Pies" -- double quotes okay here + THEN 'Annie is the best' + END as pie_day /* 3. Add another CASE statement with an ELSE clause to handle rows evaluating to False */ +,CASE WHEN vendor_name LIKE '%pie%' + THEN 'Wednesday' -- this came first for Annie even though she sells Prepared Foods too + WHEN vendor_type = 'Prepared Foods' + THEN 'Thursday' + ELSE 'Friday' + END as pies_special_day - -FROM vendor +FROM vendor; /* 4. Experiment with selecting a different column instead of just a string value */ +SELECT * +,CASE WHEN cost_per_quantity < 1.00 + THEN cost_per_quantity*5 + ELSE cost_per_quantity + END as inflation +FROM customer_purchases @@ -82,27 +127,40 @@ FROM vendor /* 1. Compare how many customer_ids are in the customer_purchases table, one select with distinct, one without */ -- 4221 rows -SELECT customer_id FROM customer_purchases +SELECT customer_id FROM customer_purchases; + +SELECT DISTINCT customer_id FROM customer_purchases; /* 2. Compare the difference between selecting market_day in market_date_info, with and without distinct: what do these difference mean?*/ + + -- market is open for 150 days +SELECT market_day +FROM market_date_info; +-- market is only open 2 days, saturday and wednesday +SELECT DISTINCT market_day +FROM market_date_info; /* 3. Which vendor has sold products to a customer */ - +SELECT DISTINCT vendor_id +FROM customer_purchases; +-- only 3 vendors have sold anything at the farmers market :( /* 4. Which vendor has sold products to a customer ... and which product was it */ - +SELECT DISTINCT vendor_id, product_id +FROM customer_purchases; /* 5. Which vendor has sold products to a customer ... and which product was it? ... AND to whom was it sold*/ - +SELECT DISTINCT vendor_id, product_id, customer_id +FROM customer_purchases; -------------------------------------------------------------------------------------------------------------------------------------------- /* MODULE 2 */ @@ -113,6 +171,17 @@ SELECT customer_id FROM customer_purchases ... use an INNER JOIN to see only products that have been purchased */ -- without table aliases +SELECT product_name, -- coming from the product TABLE +vendor_id, -- rest of these are coming from customer_purchases table +market_date, +customer_id, +customer_purchases.product_id, +product.product_id + + +FROM product +INNER JOIN customer_purchases + ON customer_purchases.product_id = product.product_id; @@ -123,6 +192,17 @@ SELECT customer_id FROM customer_purchases Add customers' first and last names with an INNER JOIN */ -- using table aliases +SELECT DISTINCT +vendor_id, -- coming from cp +product_id, -- coming from cp +c.customer_id, -- coming from c (and below) +customer_first_name, +customer_last_name + +FROM customer_purchases AS cp +INNER JOIN customer AS c + ON cp.customer_id = c.customer_id + @@ -134,15 +214,37 @@ SELECT customer_id FROM customer_purchases /* 1. There are products that have been bought ... but are there products that have not been bought? Use a LEFT JOIN to find out*/ +SELECT DISTINCT +p.product_id +,cp.product_id as [cp_product_id] +,product_name +FROM product as p +LEFT JOIN customer_purchases as cp + ON p.product_id = cp.product_id; -/* 2. Directions of LEFT JOINs matter ...*/ +/* 2. Directions of LEFT JOINs matter ...*/ +SELECT DISTINCT +p.product_id +,cp.product_id as [cp_product_id] +,product_name +FROM customer_purchases as cp +LEFT JOIN product as p + ON p.product_id = cp.product_id; /* 3. As do which values you filter on ... */ +SELECT DISTINCT +p.product_id +,cp.product_id as [cp_product_id] +,product_name +FROM product as p +LEFT JOIN customer_purchases as cp + ON p.product_id = cp.product_id +WHERE cp.product_id BETWEEN 1 AND 6; -- if we pick p product 6 rows (1-6)...but if we pick cp customer_purchases only 5 rows @@ -159,10 +261,16 @@ LEFT JOIN product AS p ...Note how the row count changed from 24 to 23 */ +SELECT * + +FROM product AS p +LEFT JOIN product_category AS pc + ON pc.product_category_id = p.product_category_id +ORDER by pc.product_category_id -------------------------------------------------------------------------------------------------------------------------------------------- -/* MODULE 2 */ +/* MODULE 2 */ /* Multiple Table JOINs */ @@ -170,7 +278,22 @@ LEFT JOIN product AS p (Which vendor has sold products to a customer AND which product was it AND to whom was it sold) Replace all the IDs (customer, vendor, and product) with the names instead*/ - +SELECT DISTINCT +--vendor_id, +vendor_name, +--product_id, +product_name, +--customer_id +customer_first_name, +customer_last_name + +FROM customer_purchases as cp +INNER JOIN vendor as v + ON v.vendor_id = cp.vendor_id +INNER JOIN product as p + ON p.product_id = cp.product_id +INNER JOIN customer as c + ON c.customer_id = cp.customer_id; /* 2. Select product_category_name, everything from the product table, and then LEFT JOIN the customer_purchases table @@ -178,4 +301,17 @@ LEFT JOIN product AS p Why do we have more rows now?*/ +SELECT +product_category_name, +p.*, +cp.product_id as product_id_in_cust_purchases_tbl + +FROM product_category as pc +INNER JOIN product as p -- will give us product_name, product_size, product_qty_type + ON pc.product_category_id = p.product_category_id +LEFT JOIN customer_purchases as cp -- by making this a left join, we are joining products that have not been sold + ON cp.product_id = p.product_id + +ORDER BY cp.product_id + diff --git a/04_this_cohort/live_code/module_2/multiple_table_joins.sql b/04_this_cohort/live_code/module_2/multiple_table_joins.sql new file mode 100644 index 000000000..38d2a8a1e --- /dev/null +++ b/04_this_cohort/live_code/module_2/multiple_table_joins.sql @@ -0,0 +1,43 @@ +/* MODULE 2 */ +/* Multiple Table JOINs */ + + +/* 1. Using the Query #5 from DISTINCT earlier + (Which vendor has sold products to a customer AND which product was it AND to whom was it sold) + + Replace all the IDs (customer, vendor, and product) with the names instead*/ +SELECT DISTINCT +--vendor_id, +vendor_name, +--product_id, +product_name, +--customer_id +customer_first_name, +customer_last_name + +FROM customer_purchases as cp +INNER JOIN vendor as v + ON v.vendor_id = cp.vendor_id +INNER JOIN product as p + ON p.product_id = cp.product_id +INNER JOIN customer as c + ON c.customer_id = cp.customer_id; + + +/* 2. Select product_category_name, everything from the product table, and then LEFT JOIN the customer_purchases table +... how does this LEFT JOIN affect the number of rows? + +Why do we have more rows now?*/ + +SELECT +product_category_name, +p.*, +cp.product_id as product_id_in_cust_purchases_tbl + +FROM product_category as pc +INNER JOIN product as p -- will give us product_name, product_size, product_qty_type + ON pc.product_category_id = p.product_category_id +LEFT JOIN customer_purchases as cp -- by making this a left join, we are joining products that have not been sold + ON cp.product_id = p.product_id + +ORDER BY cp.product_id diff --git a/04_this_cohort/live_code/module_3/Arithmitic.sql b/04_this_cohort/live_code/module_3/Arithmitic.sql new file mode 100644 index 000000000..78c22e095 --- /dev/null +++ b/04_this_cohort/live_code/module_3/Arithmitic.sql @@ -0,0 +1,24 @@ +/* MODULE 3 */ +/* Arithmitic */ + + +/* 1. power, pi(), ceiling, division, integer division, etc */ +SELECT power(4,2) as power, pi() as pi, sin(1) as sin; + +SELECT 10.0 / 3.0 as division, +CAST(10.0 as INT) / CAST(3.0 as INT) as integer_division; + +SELECT DISTINCT cost_per_quantity, cast(cost_per_quantity as INT) / 3 -- what if we do this with a column instead? +FROM customer_purchases; + + +/* 2. Every even vendor_id with modulo */ +SELECT * FROM vendor +WHERE vendor_id % 2 = 0; + + +/* 3. What about every third? */ +SELECT * FROM vendor +WHERE vendor_id % 3 = 0; + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_3/COUNT.sql b/04_this_cohort/live_code/module_3/COUNT.sql new file mode 100644 index 000000000..694eeddaa --- /dev/null +++ b/04_this_cohort/live_code/module_3/COUNT.sql @@ -0,0 +1,39 @@ +/* MODULE 3 */ +/* COUNT */ + + +/* 1. Count the number of products */ + SELECT + COUNT(product_id) as num_of_products + + FROM product; + + +/* 2. How many products per product_qty_type */ +SELECT +product_qty_type +,COUNT(product_id) as num_of_products + +FROM product +GROUP BY product_qty_type; + + + +/* 3. How many products per product_qty_type and per their product_size */ +SELECT +product_size +,product_qty_type +,COUNT(product_id) as num_of_products + +FROM product +GROUP BY product_size, product_qty_type; + + +/* COUNT DISTINCT + 4. How many unique products were bought */ + + SELECT COUNT(DISTINCT product_id) as bought_products + FROM customer_purchases; + + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_3/CTEs.sql b/04_this_cohort/live_code/module_3/CTEs.sql new file mode 100644 index 000000000..3da5cbc40 --- /dev/null +++ b/04_this_cohort/live_code/module_3/CTEs.sql @@ -0,0 +1,40 @@ +/* MODULE 3 */ +/* Common Table Expression (CTE) */ + + +/* 1. Calculate sales per vendor per day */ +;WITH vendor_daily_sales AS ( +SELECT +md.market_date +,market_day +,market_week +,market_year +,vendor_name +,SUM(quantity*cost_per_quantity) as sales + +FROM customer_purchases as cp +INNER JOIN market_date_info as md + ON cp.market_date = md.market_date +INNER JOIN vendor as v + ON cp.vendor_id = v.vendor_id + +GROUP BY md.market_date, v.vendor_id +) + +-- adding another CTE +, some_other_CTE_name AS +( +SELECT * FROM product +) + +/* ... re-aggregate the daily sales for each WEEK instead now */ +SELECT +market_year +,market_week +,vendor_name +,SUM(sales) as sales + +FROM vendor_daily_sales +GROUP BY market_year, market_week, vendor_name + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_3/DATES.sql b/04_this_cohort/live_code/module_3/DATES.sql new file mode 100644 index 000000000..6e39893ed --- /dev/null +++ b/04_this_cohort/live_code/module_3/DATES.sql @@ -0,0 +1,43 @@ +/* MODULE 3 */ +/* Date functions */ + + +/* 1. now */ +SELECT +DATE('now','localtime') as [now] +,DATETIME('now') as [UTC-now] + +/* 2. strftime */ +,strftime('%Y/%m','now') as year_month +,strftime('%Y %m %d','now','+50 days') as [the_future]; + +SELECT DISTINCT +market_date +,strftime('%m-%d-%Y', market_date,'+50 days','-1 year') as the_past + +FROM market_date_info; + + +/* 3. adding dates, e.g. last date of the month */ +SELECT +market_date +,DATE(market_date, 'start of month','-1 day') as end_of_prev_month +,DATE(market_date,'start of month','-1 day','start of month') as start_of_prev_month + +FROM market_date_info; + + + +/* 4. difference between dates, + a. number of days between now and each market_date + b. number of YEARS between now and market_date + c. number of HOURS bewtween now and market_date + */ + + SELECT market_date + ,julianday('now') - julianday(market_date) as datesbetweenmarketdate + ,(julianday('now') - julianday(market_date)) / 365.25 as yearsbetweenmktdate + ,(julianday('now') - julianday(market_date)) * 24 as hrsbetweenmktdate + + FROM market_date_info + diff --git a/04_this_cohort/live_code/module_3/HAVING.sql b/04_this_cohort/live_code/module_3/HAVING.sql new file mode 100644 index 000000000..4b0cc7f70 --- /dev/null +++ b/04_this_cohort/live_code/module_3/HAVING.sql @@ -0,0 +1,31 @@ +/* MODULE 3 */ +/* HAVING */ + + +/* 1. How much did a customer spend on each day? +Filter to customer_id between 1 and 5 and total_spend > 50 +... What order of execution occurs?*/ + SELECT + market_date + ,customer_id + ,SUM(quantity*cost_per_quantity) as total_spend + + FROM customer_purchases + WHERE customer_id BETWEEN 1 AND 5 + GROUP BY market_date,customer_id + HAVING total_spend > 50; + + +/* 2. How many products were bought? +Filter to number of purchases between 300 and 500 */ + +SELECT +count(product_id) as num_of_prod +,product_id + +FROM customer_purchases +GROUP BY product_id +HAVING count(product_id) BETWEEN 300 AND 500; + + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_3/MIN_MAX.sql b/04_this_cohort/live_code/module_3/MIN_MAX.sql new file mode 100644 index 000000000..f5b5a6015 --- /dev/null +++ b/04_this_cohort/live_code/module_3/MIN_MAX.sql @@ -0,0 +1,60 @@ +/* MODULE 3 */ +/* MIN & MAX */ + + +/* 1. What is the most expensive product +...pay attention to how it doesn't handle ties very well +*/ +SELECT +product_name, max(original_price) as most_expensive + +FROM vendor_inventory as vi +INNER JOIN product as p + ON p.product_id = vi.product_id; + + + +/* 2. Prove that max is working */ +SELECT DISTINCT +product_name, -- product name from product +original_price + +FROM vendor_inventory as vi +INNER JOIN product as p + ON p.product_id = vi.product_id +ORDER BY original_price DESC; + + + +/* 3. Find the minimum price per each product_qty_type */ +SELECT +product_qty_type -- coming from product TABLE +,MIN(original_price) as lowest_priced + +FROM vendor_inventory vi +INNER JOIN product as p + ON p.product_id = vi.product_id + +GROUP BY product_qty_type; + +/* 4. Prove that min is working */ +SELECT DISTINCT +product_name +,product_qty_type -- coming from product TABLE +,original_price as lowest_priced + +FROM vendor_inventory as vi +INNER JOIN product as p + ON p.product_id = vi.product_id + +ORDER BY product_qty_type,original_price; + + +/* 5. Min/max on a string +... not particularly useful? */ + +SELECT max(product_name) +FROM product; + + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_3/SUBQUERY_WHERE.sql b/04_this_cohort/live_code/module_3/SUBQUERY_WHERE.sql new file mode 100644 index 000000000..51d9eaca9 --- /dev/null +++ b/04_this_cohort/live_code/module_3/SUBQUERY_WHERE.sql @@ -0,0 +1,41 @@ +/* MODULE 3 */ +/* Subquery WHERE */ + + +/* 1. How much did each customer spend at each vendor for each day at the market WHEN IT RAINS */ +SELECT +market_date +,customer_id +,vendor_id +,SUM(quantity*cost_per_quantity) as total_spent + +FROM customer_purchases + +--filter by rain_flag +-- "what dates was it raining?" +WHERE market_date IN +( + SELECT market_date + FROM market_date_info + WHERE market_rain_flag = 1 +) +GROUP BY +market_date +,customer_id +,vendor_id; + + +/* 2. What is the name of the vendor who sells pie */ +SELECT DISTINCT vendor_name -- coming from vendor + +FROM customer_purchases as cp -- that something was sold +INNER JOIN vendor as v + ON cp.vendor_id = v.vendor_id + +WHERE product_id IN ( + SELECT product_id + FROM product + WHERE product_name LIKE '%pie%' +) + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_3/SUM_AVG.sql b/04_this_cohort/live_code/module_3/SUM_AVG.sql new file mode 100644 index 000000000..c0383da19 --- /dev/null +++ b/04_this_cohort/live_code/module_3/SUM_AVG.sql @@ -0,0 +1,28 @@ +/* MODULE 3 */ +/* SUM & AVG */ + + +/* 1. How much did customers spend each day */ +SELECT +market_date +,customer_id +,SUM(quantity*cost_per_quantity) as total_spend + +FROM customer_purchases +GROUP BY market_date,customer_id; + + +/* 2. How much does each customer spend on average */ +SELECT +customer_first_name +,customer_last_name +,ROUND(AVG(quantity*cost_per_quantity),2) as avg_spend + +FROM customer_purchases as cp +INNER JOIN customer as c + ON c.customer_id = cp.customer_id + +GROUP BY c.customer_id; -- why customer_id and not customer_last_name/customer_first_name -- in case of duplicate names!! + + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_3/Subquery_FROM_JOIN.sql b/04_this_cohort/live_code/module_3/Subquery_FROM_JOIN.sql new file mode 100644 index 000000000..b389fc31b --- /dev/null +++ b/04_this_cohort/live_code/module_3/Subquery_FROM_JOIN.sql @@ -0,0 +1,39 @@ +/* MODULE 3 */ +/* Subquery FROM */ + + +/*1. Simple subquery in a FROM statement, e.g. for inflation +...we could imagine joining this to a more complex query perhaps */ +SELECT DISTINCT +product_id +,inflation + +FROM ( + SELECT product_id + ,cost_per_quantity + ,CASE WHEN cost_per_quantity < 1.00 THEN cost_per_quantity*5 + ELSE cost_per_quantity END as inflation + + FROM customer_purchases +); + + +/* 2. What is the single item that has been bought in the greatest quantity?*/ +--outer query +SELECT +product_name -- coming from product table +,MAX(quantity_purchased) as most_purchased + + +FROM product as p +INNER JOIN ( +--inner query + SELECT product_id + ,count(quantity) as quantity_purchased + + + FROM customer_purchases + GROUP BY product_id +) AS x ON p.product_id = x.product_id + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_3/module_3.sqbpro b/04_this_cohort/live_code/module_3/module_3.sqbpro index 35cd56eb0..449f3e82b 100644 --- a/04_this_cohort/live_code/module_3/module_3.sqbpro +++ b/04_this_cohort/live_code/module_3/module_3.sqbpro @@ -1,120 +1,266 @@ -
/* MODULE 3 */ +/* MODULE 3 */ /* COUNT */ /* 1. Count the number of products */ + SELECT + COUNT(product_id) as num_of_products + FROM product; /* 2. How many products per product_qty_type */ +SELECT +product_qty_type +,COUNT(product_id) as num_of_products + +FROM product +GROUP BY product_qty_type; /* 3. How many products per product_qty_type and per their product_size */ +SELECT +product_size +,product_qty_type +,COUNT(product_id) as num_of_products +FROM product +GROUP BY product_size, product_qty_type; /* COUNT DISTINCT 4. How many unique products were bought */ + + SELECT COUNT(DISTINCT product_id) as bought_products + FROM customer_purchases; -------------------------------------------------------------------------------------------------------------------------------------------- -/* MODULE 3 */ +/* MODULE 3 */ /* SUM & AVG */ /* 1. How much did customers spend each day */ +SELECT +market_date +,customer_id +,SUM(quantity*cost_per_quantity) as total_spend +FROM customer_purchases +GROUP BY market_date,customer_id; /* 2. How much does each customer spend on average */ +SELECT +customer_first_name +,customer_last_name +,ROUND(AVG(quantity*cost_per_quantity),2) as avg_spend + +FROM customer_purchases as cp +INNER JOIN customer as c + ON c.customer_id = cp.customer_id + +GROUP BY c.customer_id; -- why customer_id and not customer_last_name/customer_first_name -- in case of duplicate names!! -------------------------------------------------------------------------------------------------------------------------------------------- -/* MODULE 3 */ +/* MODULE 3 */ /* MIN & MAX */ /* 1. What is the most expensive product ...pay attention to how it doesn't handle ties very well */ +SELECT +product_name, max(original_price) as most_expensive + +FROM vendor_inventory as vi +INNER JOIN product as p + ON p.product_id = vi.product_id; + /* 2. Prove that max is working */ +SELECT DISTINCT +product_name, -- product name from product +original_price + +FROM vendor_inventory as vi +INNER JOIN product as p + ON p.product_id = vi.product_id +ORDER BY original_price DESC; /* 3. Find the minimum price per each product_qty_type */ +SELECT +product_qty_type -- coming from product TABLE +,MIN(original_price) as lowest_priced +FROM vendor_inventory vi +INNER JOIN product as p + ON p.product_id = vi.product_id +GROUP BY product_qty_type; /* 4. Prove that min is working */ +SELECT DISTINCT +product_name +,product_qty_type -- coming from product TABLE +,original_price as lowest_priced + +FROM vendor_inventory as vi +INNER JOIN product as p + ON p.product_id = vi.product_id +ORDER BY product_qty_type,original_price; /* 5. Min/max on a string ... not particularly useful? */ +SELECT max(product_name) +FROM product; + -------------------------------------------------------------------------------------------------------------------------------------------- -/* MODULE 3 */ +/* MODULE 3 */ /* Arithmitic */ /* 1. power, pi(), ceiling, division, integer division, etc */ -SELECT +SELECT power(4,2) as power, pi() as pi, sin(1) as sin; +SELECT 10.0 / 3.0 as division, +CAST(10.0 as INT) / CAST(3.0 as INT) as integer_division; -/* 2. Every even vendor_id with modulo */ +SELECT DISTINCT cost_per_quantity, cast(cost_per_quantity as INT) / 3 -- what if we do this with a column instead? +FROM customer_purchases; +/* 2. Every even vendor_id with modulo */ +SELECT * FROM vendor +WHERE vendor_id % 2 = 0; + /* 3. What about every third? */ +SELECT * FROM vendor +WHERE vendor_id % 3 = 0; -------------------------------------------------------------------------------------------------------------------------------------------- -/* MODULE 3 */ +/* MODULE 3 */ /* HAVING */ /* 1. How much did a customer spend on each day? -Filter to customer_id between 1 and 5 and total_cost > 50 +Filter to customer_id between 1 and 5 and total_spend > 50 ... What order of execution occurs?*/ + SELECT + market_date + ,customer_id + ,SUM(quantity*cost_per_quantity) as total_spend + FROM customer_purchases + WHERE customer_id BETWEEN 1 AND 5 + GROUP BY market_date,customer_id + HAVING total_spend > 50; /* 2. How many products were bought? Filter to number of purchases between 300 and 500 */ +SELECT +count(product_id) as num_of_prod +,product_id + +FROM customer_purchases +GROUP BY product_id +HAVING count(product_id) BETWEEN 300 AND 500; + -------------------------------------------------------------------------------------------------------------------------------------------- -/* MODULE 3 */ +/* MODULE 3 */ /* Subquery FROM */ /*1. Simple subquery in a FROM statement, e.g. for inflation ...we could imagine joining this to a more complex query perhaps */ +SELECT DISTINCT +product_id +,inflation - +FROM ( + SELECT product_id + ,cost_per_quantity + ,CASE WHEN cost_per_quantity < 1.00 THEN cost_per_quantity*5 + ELSE cost_per_quantity END as inflation + + FROM customer_purchases +); /* 2. What is the single item that has been bought in the greatest quantity?*/ - +--outer query +SELECT +product_name -- coming from product table +,MAX(quantity_purchased) as most_purchased + + +FROM product as p +INNER JOIN ( +--inner query + SELECT product_id + ,count(quantity) as quantity_purchased + + + FROM customer_purchases + GROUP BY product_id +) AS x ON p.product_id = x.product_id -------------------------------------------------------------------------------------------------------------------------------------------- -/* MODULE 3 */ +/* MODULE 3 */ /* Subquery WHERE */ /* 1. How much did each customer spend at each vendor for each day at the market WHEN IT RAINS */ - - +SELECT +market_date +,customer_id +,vendor_id +,SUM(quantity*cost_per_quantity) as total_spent + +FROM customer_purchases + +--filter by rain_flag +-- "what dates was it raining?" +WHERE market_date IN +( + SELECT market_date + FROM market_date_info + WHERE market_rain_flag = 1 +) +GROUP BY +market_date +,customer_id +,vendor_id; /* 2. What is the name of the vendor who sells pie */ +SELECT DISTINCT vendor_name -- coming from vendor + +FROM customer_purchases as cp -- that something was sold +INNER JOIN vendor as v + ON cp.vendor_id = v.vendor_id +WHERE product_id IN ( + SELECT product_id + FROM product + WHERE product_name LIKE '%pie%' +) -------------------------------------------------------------------------------------------------------------------------------------------- -/* MODULE 3 */ +/* MODULE 3 */ /* Temp Tables */ @@ -130,30 +276,59 @@ DROP TABLE IF EXISTS temp.new_vendor_inventory; CREATE TABLE temp.new_vendor_inventory AS -- definition of the table - - - +SELECT * +,original_price*5 as inflation +FROM vendor_inventory; /* 2. put the previous table into another temp table, e.g. as temp.new_new_vendor_inventory */ +DROP TABLE IF EXISTS temp.new_new_vendor_inventory; +CREATE TABLE temp.new_new_vendor_inventory AS +SELECT * +,inflation*2 as SUPER_INFLATION +FROM temp.new_vendor_inventory -------------------------------------------------------------------------------------------------------------------------------------------- -/* MODULE 3 */ +/* MODULE 3 */ /* Common Table Expression (CTE) */ /* 1. Calculate sales per vendor per day */ +;WITH vendor_daily_sales AS ( SELECT - - - - +md.market_date +,market_day +,market_week +,market_year +,vendor_name +,SUM(quantity*cost_per_quantity) as sales + +FROM customer_purchases as cp +INNER JOIN market_date_info as md + ON cp.market_date = md.market_date +INNER JOIN vendor as v + ON cp.vendor_id = v.vendor_id + +GROUP BY md.market_date, v.vendor_id +) + +-- adding another CTE +, some_other_CTE_name AS +( +SELECT * FROM product +) /* ... re-aggregate the daily sales for each WEEK instead now */ +SELECT +market_year +,market_week +,vendor_name +,SUM(sales) as sales - +FROM vendor_daily_sales +GROUP BY market_year, market_week, vendor_name -------------------------------------------------------------------------------------------------------------------------------------------- /* MODULE 3 */ @@ -162,13 +337,27 @@ SELECT /* 1. now */ SELECT - +DATE('now','localtime') as [now] +,DATETIME('now') as [UTC-now] /* 2. strftime */ +,strftime('%Y/%m','now') as year_month +,strftime('%Y %m %d','now','+50 days') as [the_future]; + +SELECT DISTINCT +market_date +,strftime('%m-%d-%Y', market_date,'+50 days','-1 year') as the_past +FROM market_date_info; /* 3. adding dates, e.g. last date of the month */ +SELECT +market_date +,DATE(market_date, 'start of month','-1 day') as end_of_prev_month +,DATE(market_date,'start of month','-1 day','start of month') as start_of_prev_month + +FROM market_date_info; @@ -177,4 +366,12 @@ SELECT b. number of YEARS between now and market_date c. number of HOURS bewtween now and market_date */ + + SELECT market_date + ,julianday('now') - julianday(market_date) as datesbetweenmarketdate + ,(julianday('now') - julianday(market_date)) / 365.25 as yearsbetweenmktdate + ,(julianday('now') - julianday(market_date)) * 24 as hrsbetweenmktdate + + FROM market_date_info + diff --git a/04_this_cohort/live_code/module_3/temp_tables.sql b/04_this_cohort/live_code/module_3/temp_tables.sql new file mode 100644 index 000000000..1d5741a27 --- /dev/null +++ b/04_this_cohort/live_code/module_3/temp_tables.sql @@ -0,0 +1,31 @@ +/* MODULE 3 */ +/* Temp Tables */ + + +/* 1. Put our inflation query into a temp table, e.g. as temp.new_vendor_inventory*/ + +/* some structural code */ +/* ...heads up, sometimes this query can be finnicky -- it's good to try highlighting different sections to help it succeed...*/ + +-- if a table named new_vendor_inventory exists, delete it, otherwise do NOTHING +DROP TABLE IF EXISTS temp.new_vendor_inventory; + +--make the table +CREATE TABLE temp.new_vendor_inventory AS + +-- definition of the table +SELECT * +,original_price*5 as inflation +FROM vendor_inventory; + + +/* 2. put the previous table into another temp table, e.g. as temp.new_new_vendor_inventory */ +DROP TABLE IF EXISTS temp.new_new_vendor_inventory; +CREATE TABLE temp.new_new_vendor_inventory AS + +SELECT * +,inflation*2 as SUPER_INFLATION +FROM temp.new_vendor_inventory + + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_4/FULL_OUTER_JOIN_UNION.sql b/04_this_cohort/live_code/module_4/FULL_OUTER_JOIN_UNION.sql new file mode 100644 index 000000000..349b36ae2 --- /dev/null +++ b/04_this_cohort/live_code/module_4/FULL_OUTER_JOIN_UNION.sql @@ -0,0 +1,45 @@ +/* MODULE 4 */ +/* UNION */ + +/* 1. Emulate a FULL OUTER JOIN with a UNION */ +DROP TABLE IF EXISTS temp.store1; +CREATE TEMP TABLE IF NOT EXISTS temp.store1 +( +costume TEXT, +quantity INT +); + +INSERT INTO temp.store1 +VALUES("tiger",6), + ("elephant",2), + ("princess", 4); + + +DROP TABLE IF EXISTS temp.store2; +CREATE TEMP TABLE IF NOT EXISTS temp.store2 +( +costume TEXT, +quantity INT +); + +INSERT INTO temp.store2 +VALUES("tiger",2), + ("dancer",7), + ("superhero", 5); + +SELECT s1.costume, s1.quantity as store1_quantity, s2.quantity as store2_quantity +FROM store1 as s1 +LEFT JOIN store2 as s2 + ON s1.costume = s2.costume + +UNION ALL + +SELECT s2.costume, s1.quantity, s2.quantity + +FROM store2 s2 +LEFT JOIN store1 s1 + ON s1.costume = s2.costume +WHERE s1.costume IS NULL + + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_4/INTERSECT_EXCEPT.sql b/04_this_cohort/live_code/module_4/INTERSECT_EXCEPT.sql new file mode 100644 index 000000000..9e080f9e9 --- /dev/null +++ b/04_this_cohort/live_code/module_4/INTERSECT_EXCEPT.sql @@ -0,0 +1,31 @@ +/* MODULE 4 */ +/* INTERSECT & EXCEPT */ + +/* 1. Find products that have been sold (e.g. are in customer purchases AND product) */ + +SELECT product_id FROM product +INTERSECT +SELECT product_id FROM customer_purchases; + + +/* 2. Find products that have NOT been sold (e.g. are NOT in customer purchases even though in product) */ +SELECT product_id FROM product +EXCEPT +SELECT product_ID FROM customer_purchases; + + +/* 3. Directions matter... if we switch the order here: +products that do not exist, because no products purchased are NOT in the product table (e.g. are NOT in product even though in customer purchases)*/ +SELECT product_id FROM customer_purchases +EXCEPT +SELECT product_id FROM product; + + +/* 4. We can remake the intersect with a WHERE subquery for more details ... */ +SELECT * FROM product +WHERE product_id IN + ( + SELECT product_id FROM product + INTERSECT + SELECT product_id FROM customer_purchases + ) diff --git a/04_this_cohort/live_code/module_4/NTILE.sql b/04_this_cohort/live_code/module_4/NTILE.sql new file mode 100644 index 000000000..dad99c3e3 --- /dev/null +++ b/04_this_cohort/live_code/module_4/NTILE.sql @@ -0,0 +1,32 @@ +/* MODULE 4 */ +/* Windowed functions: NTILE */ + + +/* 1. Calculate quartile, quntiles, and percentiles from vendor daily sales */ +SELECT * +,NTILE(4) OVER (PARTITION BY vendor_name ORDER BY sales ASC) as [quartile] +,NTILE(5) OVER (PARTITION BY vendor_name ORDER BY sales ASC) as [quintile] +,NTILE(100) OVER (PARTITION BY vendor_name ORDER BY sales ASC) as [percentile] +--,PERCENT_RANK OVER(...) -- maybe better for field of corn, not having 100 buckets to split into + +FROM ( + +-- vendor daily sales + SELECT + md.market_date + ,market_day + ,market_week + ,market_year + ,vendor_name + ,SUM(quantity*cost_per_quantity) AS sales + + FROM customer_purchases AS cp + JOIN market_date_info AS md + ON cp.market_date = md.market_date + JOIN vendor AS v + ON v.vendor_id = cp.vendor_id + + GROUP BY cp.market_date, v.vendor_id + +) x +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_4/NULLIF_budget.sql b/04_this_cohort/live_code/module_4/NULLIF_budget.sql new file mode 100644 index 000000000..8815ed03a --- /dev/null +++ b/04_this_cohort/live_code/module_4/NULLIF_budget.sql @@ -0,0 +1,44 @@ +/* MODULE 4 */ +/* NULLIF Budget (example from the slides) */ + +/* The following example creates a budgets table to show a department (dept) +...its current budget (current_year) and its previous budget (previous_year). + +For the current year, NULL is used for departments with budgets that have not changed from the previous year, +and 0 is used for budgets that have not yet been determined. + +To find out the average of only those departments that receive a budget and to include the budget value +from the previous year (use the previous_year value, where the current_year is NULL), +combine the NULLIF and COALESCE functions. */ + +DROP TABLE IF EXISTS temp.budgets; +CREATE TEMP TABLE IF NOT EXISTS temp.budgets ( +dept STRING +,current_year INT +,previous_year INT +); + + +INSERT INTO temp.budgets VALUES +('software',1000,1000) +, ('candles',NULL,500) +, ('coffee', 400, 200) +, ('pencils',0, 50); + + +/*examine each of these columns */ +SELECT +NULLIF(current_year, previous_year) +--,NULLIF(COALESCE(current_year, previous_year), 0.00) +--, +--AVG(NULLIF(COALESCE(current_year, previous_year), 0.00)) +FROM budgets + + +/* more NULLIF here: +https://learn.microsoft.com/en-us/sql/t-sql/language-elements/nullif-transact-sql?view=sql-server-ver17 +*/ + + + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_4/NULL_management.sql b/04_this_cohort/live_code/module_4/NULL_management.sql new file mode 100644 index 000000000..077ebb3d6 --- /dev/null +++ b/04_this_cohort/live_code/module_4/NULL_management.sql @@ -0,0 +1,43 @@ +/* MODULE 4 */ +/* NULL Management */ + + +/* 1. IFNULL: Missing product_size, missing product_qty_type */ +SELECT * +,IFNULL(product_size, 'Unknown') as new_product_size +,IFNULL(product_size,product_qty_type) as silly_replacement + +/* 2. Coalesce */ +,coalesce(product_size,product_qty_type,'missing') as more_replacement -- if the first value is null, then replace with the second value, if that is null, replace with the word "missing" +,coalesce(product_qty_type,product_size,'missing') as even_more_replacements +,IFNULL(IFNULL(product_qty_type,product_size),'missing') as same_as_above + +FROM product; + + +/* 3. NULLIF +finding values in the product_size column that are "blank" strings and setting them to NULL if they are blank */ +SELECT * +,IFNULL(product_size,'unknown') as not_both_nulls +,NULLIF(product_size,'') as both_nulls -- two single quotes +,coalesce(NULLIF(product_size,''),'unknown') as replaced_both + +FROM product; + + +/* 4. NULLIF +filtering which rows are null or blank */ + +SELECT * +,NULLIF(product_size,'') -- criteria we are filtering on + +FROM product +WHERE NULLIF(product_size,'') IS NULL; -- how many blanks or nulls are there, in the same line + + +-------------------------------------------------------------------------------------------------------------------------------------------- + +SELECT * +,CASE WHEN product_id < 5 THEN NULL ELSE product_id END + +FROM product \ No newline at end of file diff --git a/04_this_cohort/live_code/module_4/ROW_NUMBER.sql b/04_this_cohort/live_code/module_4/ROW_NUMBER.sql new file mode 100644 index 000000000..2ff3e1c87 --- /dev/null +++ b/04_this_cohort/live_code/module_4/ROW_NUMBER.sql @@ -0,0 +1,39 @@ +/* MODULE 4 */ +/* Windowed functions: row_number */ + + +/* 1. What product is the highest price per vendor */ +--outer query +SELECT x.* +,product_name + +FROM +-- inner QUERY +( + SELECT + vendor_id + ,product_id + ,original_price + ,ROW_NUMBER() OVER(PARTITION BY vendor_id ORDER BY original_price DESC) as price_rank + + FROM vendor_inventory +) x +INNER JOIN product p + on x.product_id = p.product_id + +WHERE x.price_rank = 1 + + +/* See how this varies from using max due to the group by +SELECT vendor_id, +--product_id, +MAX(original_price) + +FROM vendor_inventory +GROUP BY vendor_id--,product_id + +*/ + + + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_4/UNION_UNION_ALL.sql b/04_this_cohort/live_code/module_4/UNION_UNION_ALL.sql new file mode 100644 index 000000000..8c218225f --- /dev/null +++ b/04_this_cohort/live_code/module_4/UNION_UNION_ALL.sql @@ -0,0 +1,47 @@ +/* MODULE 4 */ +/* UNION */ + +/* 1. Find the most and least expensive product by vendor with UNION (and row_number!) */ + +--outer query +SELECT +vendor_id +,original_price +--,'Max' as max_or_min +,rn_max as [row_number] + +FROM ( +--inner query + SELECT vendor_id + ,product_id + ,original_price + ,ROW_NUMBER() OVER(PARTITION BY vendor_id ORDER BY original_price DESC) as rn_max + + FROM vendor_inventory +) + +WHERE rn_max = 1 + +UNION -- union returned 5 rows, UNION ALL returned 6 rows because vendor #4 is duplicated + +SELECT +vendor_id +,original_price +--,'Min' as max_or_min +,rn_min + +FROM ( +--inner query + SELECT vendor_id + ,product_id + ,original_price + ,ROW_NUMBER() OVER(PARTITION BY vendor_id ORDER BY original_price ASC) as rn_min + + FROM vendor_inventory +) + +WHERE rn_min = 1 + + + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_4/module_4.sqbpro b/04_this_cohort/live_code/module_4/module_4.sqbpro index 55fe883e1..32b8cc5fb 100644 --- a/04_this_cohort/live_code/module_4/module_4.sqbpro +++ b/04_this_cohort/live_code/module_4/module_4.sqbpro @@ -1,26 +1,46 @@ -
/* MODULE 4 */ +/* MODULE 4 */ /* NULL Management */ /* 1. IFNULL: Missing product_size, missing product_qty_type */ - +SELECT * +,IFNULL(product_size, 'Unknown') as new_product_size +,IFNULL(product_size,product_qty_type) as silly_replacement /* 2. Coalesce */ +,coalesce(product_size,product_qty_type,'missing') as more_replacement -- if the first value is null, then replace with the second value, if that is null, replace with the word "missing" +,coalesce(product_qty_type,product_size,'missing') as even_more_replacements +,IFNULL(IFNULL(product_qty_type,product_size),'missing') as same_as_above +FROM product; /* 3. NULLIF finding values in the product_size column that are "blank" strings and setting them to NULL if they are blank */ +SELECT * +,IFNULL(product_size,'unknown') as not_both_nulls +,NULLIF(product_size,'') as both_nulls -- two single quotes +,coalesce(NULLIF(product_size,''),'unknown') as replaced_both +FROM product; /* 4. NULLIF filtering which rows are null or blank */ +SELECT * +,NULLIF(product_size,'') -- criteria we are filtering on + +FROM product +WHERE NULLIF(product_size,'') IS NULL; -- how many blanks or nulls are there, in the same line -------------------------------------------------------------------------------------------------------------------------------------------- -/* MODULE 4 */ + +SELECT * +,CASE WHEN product_id < 5 THEN NULL ELSE product_id END + +FROM product/* MODULE 4 */ /* NULLIF Budget (example from the slides) */ /* The following example creates a budgets table to show a department (dept) @@ -69,7 +89,25 @@ https://learn.microsoft.com/en-us/sql/t-sql/language-elements/nullif-transact-sq /* 1. What product is the highest price per vendor */ +--outer query +SELECT x.* +,product_name +FROM +-- inner QUERY +( + SELECT + vendor_id + ,product_id + ,original_price + ,ROW_NUMBER() OVER(PARTITION BY vendor_id ORDER BY original_price DESC) as price_rank + + FROM vendor_inventory +) x +INNER JOIN product p + on x.product_id = p.product_id + +WHERE x.price_rank = 1 /* See how this varies from using max due to the group by @@ -110,7 +148,15 @@ VALUES (7, 230000), (8, 100000), (9, 165000), -(10, 100000); +(10, 100000), +(11, 90000); + +SELECT * +,ROW_NUMBER() OVER(ORDER BY salary DESC) as [row_number] +,RANK() OVER(ORDER BY salary DESC) as [rank] +,DENSE_RANK() OVER(ORDER BY salary DESC) as [dense_rank] + +FROM row_rank_dense @@ -121,7 +167,13 @@ VALUES /* 1. Calculate quartile, quntiles, and percentiles from vendor daily sales */ +SELECT * +,NTILE(4) OVER (PARTITION BY vendor_name ORDER BY sales ASC) as [quartile] +,NTILE(5) OVER (PARTITION BY vendor_name ORDER BY sales ASC) as [quintile] +,NTILE(100) OVER (PARTITION BY vendor_name ORDER BY sales ASC) as [percentile] +--,PERCENT_RANK OVER(...) -- maybe better for field of corn, not having 100 buckets to split into +FROM ( -- vendor daily sales SELECT @@ -140,31 +192,77 @@ VALUES GROUP BY cp.market_date, v.vendor_id - +) x -------------------------------------------------------------------------------------------------------------------------------------------- -/* MODULE 4 */ +/* MODULE 4 */ /* String Manipulations */ /* 1. ltrim, rtrim, trim*/ -SELECT +SELECT +LTRIM(' THOMAS ROSENTHAL ') as [ltrim] +,RTRIM(' THOMAS ROSENTHAL ') as [rtrim] +,TRIM(' THOMAS ROSENTHAL ') as [trim] +,LTRIM(RTRIM(' THOMAS ROSENTHAL ')) as [both]; /* 2. replace*/ +SELECT +customer_first_name +,REPLACE(customer_first_name,'a','e') as cust_a_to_e +,REPLACE(customer_first_name,'a',customer_last_name) as chaos + +FROM customer; /* 3. upper, lower*/ +SELECT customer_first_name +,UPPER(customer_first_name) as [upper] +,LOWER(customer_first_name) as [lower] + +FROM customer; + + /* 4. concat with || */ +SELECT * +,customer_first_name || ' ' || customer_last_name as customer_name +,LOWER(customer_first_name) || ' ' || UPPER(customer_last_name) || ' ' || customer_postal_code as customer_details +,NULL || 'thomas' as always_null + +FROM customer; + /* 5. substr */ +SELECT +customer_last_name +,substr(customer_last_name,4) as [4] -- any length from the 4th character +,substr(customer_last_name,4,2) as [4,2] -- length of 2 from the 4th character +,substr(customer_last_name,-5,4) as count_right -- counting from the right 5 characters in 4 out +FROM customer; + /* 6. length */ +SELECT +customer_last_name, length(customer_last_name) as length + +FROM customer; /* 7. unicode, char */ +SELECT UNICODE('b') as b; -/* 8. REGEXP in a WHERE statement */ +SELECT +'THOMAS +ROSENTHAL' as bad_spaced_name +,replace('THOMAS + +ROSENTHAL',char(10),' ') as better_linebreak; -- removes all instances of line break from the string + +/* 8. REGEXP in a WHERE statement */ + +SELECT * FROM customer +WHERE customer_last_name REGEXP '(a)$' -- filtering to customer last names ending in a -------------------------------------------------------------------------------------------------------------------------------------------- /* MODULE 4 */ @@ -196,6 +294,45 @@ SELECT /* 1. Find the most and least expensive product by vendor with UNION (and row_number!) */ +--outer query +SELECT +vendor_id +,original_price +--,'Max' as max_or_min +,rn_max as [row_number] + +FROM ( +--inner query + SELECT vendor_id + ,product_id + ,original_price + ,ROW_NUMBER() OVER(PARTITION BY vendor_id ORDER BY original_price DESC) as rn_max + + FROM vendor_inventory +) + +WHERE rn_max = 1 + +UNION -- union returned 5 rows, UNION ALL returned 6 rows because vendor #4 is duplicated + +SELECT +vendor_id +,original_price +--,'Min' as max_or_min +,rn_min + +FROM ( +--inner query + SELECT vendor_id + ,product_id + ,original_price + ,ROW_NUMBER() OVER(PARTITION BY vendor_id ORDER BY original_price ASC) as rn_min + + FROM vendor_inventory +) + +WHERE rn_min = 1 + -------------------------------------------------------------------------------------------------------------------------------------------- @@ -228,6 +365,19 @@ VALUES("tiger",2), ("dancer",7), ("superhero", 5); +SELECT s1.costume, s1.quantity as store1_quantity, s2.quantity as store2_quantity +FROM store1 as s1 +LEFT JOIN store2 as s2 + ON s1.costume = s2.costume + +UNION ALL + +SELECT s2.costume, s1.quantity, s2.quantity + +FROM store2 s2 +LEFT JOIN store1 s1 + ON s1.costume = s2.costume +WHERE s1.costume IS NULL -------------------------------------------------------------------------------------------------------------------------------------------- @@ -236,18 +386,30 @@ VALUES("tiger",2), /* 1. Find products that have been sold (e.g. are in customer purchases AND product) */ +SELECT product_id FROM product +INTERSECT +SELECT product_id FROM customer_purchases; /* 2. Find products that have NOT been sold (e.g. are NOT in customer purchases even though in product) */ - +SELECT product_id FROM product +EXCEPT +SELECT product_ID FROM customer_purchases; /* 3. Directions matter... if we switch the order here: products that do not exist, because no products purchased are NOT in the product table (e.g. are NOT in product even though in customer purchases)*/ - +SELECT product_id FROM customer_purchases +EXCEPT +SELECT product_id FROM product; /* 4. We can remake the intersect with a WHERE subquery for more details ... */ - - +SELECT * FROM product +WHERE product_id IN + ( + SELECT product_id FROM product + INTERSECT + SELECT product_id FROM customer_purchases + ) diff --git a/04_this_cohort/live_code/module_4/row_rank_dense.sql b/04_this_cohort/live_code/module_4/row_rank_dense.sql new file mode 100644 index 000000000..1dacdf014 --- /dev/null +++ b/04_this_cohort/live_code/module_4/row_rank_dense.sql @@ -0,0 +1,39 @@ +/* MODULE 4 */ +/* Windowed functions: dense_rank, rank, row_number */ + + +/* 1. Compare dense_rank, rank, and row_number */ + +DROP TABLE IF EXISTS TEMP.row_rank_dense; + +CREATE TEMP TABLE IF NOT EXISTS TEMP.row_rank_dense +( +emp_id INT, +salary INT +); + +INSERT INTO temp.row_rank_dense +VALUES +(1,200000), +(2,200000), +(3, 160000), +(4, 120000), +(5, 125000), +(6, 165000), +(7, 230000), +(8, 100000), +(9, 165000), +(10, 100000), +(11, 90000); + +SELECT * +,ROW_NUMBER() OVER(ORDER BY salary DESC) as [row_number] +,RANK() OVER(ORDER BY salary DESC) as [rank] +,DENSE_RANK() OVER(ORDER BY salary DESC) as [dense_rank] + +FROM row_rank_dense + + + + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_4/string_manipulation.sql b/04_this_cohort/live_code/module_4/string_manipulation.sql new file mode 100644 index 000000000..2a4332fb8 --- /dev/null +++ b/04_this_cohort/live_code/module_4/string_manipulation.sql @@ -0,0 +1,71 @@ +/* MODULE 4 */ +/* String Manipulations */ + + +/* 1. ltrim, rtrim, trim*/ +SELECT +LTRIM(' THOMAS ROSENTHAL ') as [ltrim] +,RTRIM(' THOMAS ROSENTHAL ') as [rtrim] +,TRIM(' THOMAS ROSENTHAL ') as [trim] +,LTRIM(RTRIM(' THOMAS ROSENTHAL ')) as [both]; + +/* 2. replace*/ +SELECT +customer_first_name +,REPLACE(customer_first_name,'a','e') as cust_a_to_e +,REPLACE(customer_first_name,'a',customer_last_name) as chaos + +FROM customer; + +/* 3. upper, lower*/ + +SELECT customer_first_name +,UPPER(customer_first_name) as [upper] +,LOWER(customer_first_name) as [lower] + +FROM customer; + + +/* 4. concat with || */ +SELECT * +,customer_first_name || ' ' || customer_last_name as customer_name +,LOWER(customer_first_name) || ' ' || UPPER(customer_last_name) || ' ' || customer_postal_code as customer_details +,NULL || 'thomas' as always_null + +FROM customer; + + +/* 5. substr */ + +SELECT +customer_last_name +,substr(customer_last_name,4) as [4] -- any length from the 4th character +,substr(customer_last_name,4,2) as [4,2] -- length of 2 from the 4th character +,substr(customer_last_name,-5,4) as count_right -- counting from the right 5 characters in 4 out +FROM customer; + +/* 6. length */ +SELECT +customer_last_name, length(customer_last_name) as length + +FROM customer; + +/* 7. unicode, char */ + +SELECT UNICODE('b') as b; + +SELECT +'THOMAS + +ROSENTHAL' as bad_spaced_name + +,replace('THOMAS + +ROSENTHAL',char(10),' ') as better_linebreak; -- removes all instances of line break from the string + +/* 8. REGEXP in a WHERE statement */ + +SELECT * FROM customer +WHERE customer_last_name REGEXP '(a)$' -- filtering to customer last names ending in a + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_5/CROSS_JOIN.sql b/04_this_cohort/live_code/module_5/CROSS_JOIN.sql new file mode 100644 index 000000000..e5c6bb1bf --- /dev/null +++ b/04_this_cohort/live_code/module_5/CROSS_JOIN.sql @@ -0,0 +1,39 @@ +/* MODULE 5 */ +/* CROSS JOIN */ + + +/* 1. CROSS JOIN sizes with product*/ + +DROP TABLE IF EXISTS TEMP.sizes; +CREATE TEMP TABLE IF NOT EXISTS TEMP.sizes (size TEXT); + +INSERT INTO TEMP.sizes +VALUES('small'), +('medium'), +('large'); + +SELECT * FROM TEMP.sizes; + +DROP TABLE IF EXISTS TEMP.colours; +CREATE TEMP TABLE IF NOT EXISTS TEMP.colours (colour TEXT); + +INSERT INTO TEMP.colours +VALUES('red'), +('green'), +('orange'), +('purple'); + +SELECT * FROM TEMP.sizes; +SELECT * FROM TEMP.colours; + +SELECT product_name,product_qty_type,size--,colour +FROM product -- 23 rows +CROSS JOIN temp.sizes -- 3 ROWS +-- 3*23 = 69 rows for the cartesian product +--CROSS JOIN temp.colours -- 4 ROWS -- add colour if you would like +-- 69*4 = 276 rows for the cartesian product + + + + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_5/INSERT_UPDATE_DELETE.sql b/04_this_cohort/live_code/module_5/INSERT_UPDATE_DELETE.sql new file mode 100644 index 000000000..26376f6a7 --- /dev/null +++ b/04_this_cohort/live_code/module_5/INSERT_UPDATE_DELETE.sql @@ -0,0 +1,31 @@ +/* MODULE 5 */ +/* INSERT UPDATE DELETE */ + + +DROP TABLE IF EXISTS temp.product_expanded; +CREATE TEMP TABLE product_expanded AS + SELECT * FROM product; + +--SELECT * FROM product_expanded + +/* 1. add a product to the temp table */ +--INSERT +INSERT INTO product_expanded +VALUES(24,'Almonds','1 lb',3,'lbs'); + + +/* 2. change the product_size for THAT product */ +--UPDATE +UPDATE product_expanded +SET product_size = '.5 kg', product_qty_type = 'kg' +WHERE product_id = 24; + +/* 3. delete the newly added product */ +--DELETE + +DELETE FROM product_expanded +--SELECT * FROM product_expanded -- helping you determine which rows you are looking at, are they the right rows before you delete it? +WHERE product_id = 24; + + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_5/JSON_to_table.sql b/04_this_cohort/live_code/module_5/JSON_to_table.sql new file mode 100644 index 000000000..eecb06225 --- /dev/null +++ b/04_this_cohort/live_code/module_5/JSON_to_table.sql @@ -0,0 +1,35 @@ +--JSON to TABLE + +-- create a temp TABLE +-- insert the json as "long string" aka BLOB +-- write a json_each statement +-- use the json_each statement as a subquery with a json_extract to create our column values + +DROP TABLE IF EXISTS temp.country_json; +CREATE TABLE IF NOT EXISTS temp.country_json +( +the_json BLOB -- the column and the column type +); + +INSERT INTO temp.country_json +VALUES( +'[ + { + "country": "Afghanistan", + "city": "Kabul" + }, + { + "country": "Albania", + "city": "Tirana" + } +]' +); + +SELECT key +,JSON_EXTRACT(value,'$.country') as country +,JSON_EXTRACT(value,'$.city') as city + +FROM ( + SELECT * + FROM country_json,JSON_EACH(country_json.the_json, '$') -- for each of the json elements, everything +) \ No newline at end of file diff --git a/04_this_cohort/live_code/module_5/SELF_JOIN.sql b/04_this_cohort/live_code/module_5/SELF_JOIN.sql new file mode 100644 index 000000000..2b7fe7058 --- /dev/null +++ b/04_this_cohort/live_code/module_5/SELF_JOIN.sql @@ -0,0 +1,32 @@ +/* MODULE 5 */ +/* SELF JOIN */ + + +/* 1. Create a self-joining hierarchy */ + +DROP TABLE IF EXISTS TEMP.employees; +CREATE TEMP TABLE TEMP.employees +( +emp_id INT +,emp_name text +,mgr_id INT +); + +INSERT INTO TEMP.employees +VALUES(1,'Thomas',3) +,(2,'Niyaz', 4) +,(3,'Rohan', NULL) +,(4, 'Jennie',3); + +SELECT * FROM TEMP.employees; + +SELECT e.emp_name, m.emp_name as mgr_name -- rename the column so its possible to understand the structure +,coalesce(m.emp_name,e.emp_name) as self_boss -- if you want Rohan to be his own boss + +FROM temp.employees as e +LEFT JOIN temp.employees as m + ON e.mgr_id = m.emp_id -- identifying which id is the parent and which id is the child + + + + diff --git a/04_this_cohort/live_code/module_5/VENDOR_DAILY_SALES_view.sql b/04_this_cohort/live_code/module_5/VENDOR_DAILY_SALES_view.sql new file mode 100644 index 000000000..367d2b9a8 --- /dev/null +++ b/04_this_cohort/live_code/module_5/VENDOR_DAILY_SALES_view.sql @@ -0,0 +1,29 @@ +/* MODULE 5 */ +/* VIEW */ + +/* 1. Create a vendor daily sales view */ +DROP VIEW IF EXISTS vendor_daily_sales; +CREATE VIEW IF NOT EXISTS vendor_daily_sales AS + + SELECT + md.market_date + ,market_day + ,market_week + ,market_year + ,vendor_name + ,SUM(quantity*cost_per_quantity) as sales + + + FROM market_date_info md + INNER JOIN customer_purchases cp + ON md.market_date = cp.market_date + INNER JOIN vendor v + ON cp.vendor_id = v.vendor_id + + WHERE market_date = DATE('now','localtime') + + GROUP BY cp.market_date, v.vendor_id; + + + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_5/dynamic_view.sql b/04_this_cohort/live_code/module_5/dynamic_view.sql new file mode 100644 index 000000000..0f53c0bc1 --- /dev/null +++ b/04_this_cohort/live_code/module_5/dynamic_view.sql @@ -0,0 +1,50 @@ +/* MODULE 5 */ +/* DYNAMIC VIEW */ + +/* 1. Create todays vendor daily sales view */ +DROP VIEW IF EXISTS todays_vendor_daily_sales; +CREATE VIEW IF NOT EXISTS todays_vendor_daily_sales AS + + SELECT + md.market_date + ,market_day + ,market_week + ,market_year + ,vendor_name + ,SUM(quantity*cost_per_quantity) as sales + + FROM market_date_info md + INNER JOIN ( + SELECT * FROM + customer_purchases + UNION + SELECT * FROM + new_customer_purchases) cp + ON md.market_date = cp.market_date + + INNER JOIN vendor v + ON cp.vendor_id = v.vendor_id + + WHERE cp.market_date = DATE('now','localtime') + + GROUP BY cp.market_date, v.vendor_id; + + + +/* spoilers below */ + + + + + +-- THIS ONLY WORKS IF YOU HAVE DONE THE PROPER STEPS FOR IMPORTING +-- 1) update new_customer_purchases to today +-- 2) add the union +-- 3) add the where statement +-- 4) update the market_date_info to include today + + + + + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_5/module_5.sqbpro b/04_this_cohort/live_code/module_5/module_5.sqbpro index 118989da4..6606a2644 100644 --- a/04_this_cohort/live_code/module_5/module_5.sqbpro +++ b/04_this_cohort/live_code/module_5/module_5.sqbpro @@ -1,4 +1,4 @@ -
/* MODULE 5 */ +
/* MODULE 5 */ /* INSERT UPDATE DELETE */ @@ -9,21 +9,32 @@ CREATE TEMP TABLE product_expanded AS --SELECT * FROM product_expanded /* 1. add a product to the temp table */ - +--INSERT +INSERT INTO product_expanded +VALUES(24,'Almonds','1 lb',3,'lbs'); /* 2. change the product_size for THAT product */ - - +--UPDATE +UPDATE product_expanded +SET product_size = '.5 kg', product_qty_type = 'kg' +WHERE product_id = 24; /* 3. delete the newly added product */ +--DELETE + +DELETE FROM product_expanded +--SELECT * FROM product_expanded -- helping you determine which rows you are looking at, are they the right rows before you delete it? +WHERE product_id = 24; -------------------------------------------------------------------------------------------------------------------------------------------- -/* MODULE 5 */ +/* MODULE 5 */ /* VIEW */ /* 1. Create a vendor daily sales view */ +DROP VIEW IF EXISTS vendor_daily_sales; +CREATE VIEW IF NOT EXISTS vendor_daily_sales AS SELECT md.market_date @@ -39,29 +50,62 @@ CREATE TEMP TABLE product_expanded AS ON md.market_date = cp.market_date INNER JOIN vendor v ON cp.vendor_id = v.vendor_id + + WHERE market_date = DATE('now','localtime') GROUP BY cp.market_date, v.vendor_id; -------------------------------------------------------------------------------------------------------------------------------------------- -/* MODULE 5 */ +/* MODULE 5 */ /* VIEW in another query */ /* 1. Transform the daily sales view into a sales by vendor per week result */ +SELECT +market_year +,market_week +,vendor_name +,SUM(sales) as sales +FROM vendor_daily_sales +GROUP BY market_year +,market_week +,vendor_name -------------------------------------------------------------------------------------------------------------------------------------------- -/* MODULE 5 */ +/* MODULE 5 */ /* DYNAMIC VIEW */ +/* 1. Create todays vendor daily sales view */ +DROP VIEW IF EXISTS todays_vendor_daily_sales; +CREATE VIEW IF NOT EXISTS todays_vendor_daily_sales AS - - - - + SELECT + md.market_date + ,market_day + ,market_week + ,market_year + ,vendor_name + ,SUM(quantity*cost_per_quantity) as sales + + FROM market_date_info md + INNER JOIN ( + SELECT * FROM + customer_purchases + UNION + SELECT * FROM + new_customer_purchases) cp + ON md.market_date = cp.market_date + + INNER JOIN vendor v + ON cp.vendor_id = v.vendor_id + + WHERE cp.market_date = DATE('now','localtime') + + GROUP BY cp.market_date, v.vendor_id; @@ -71,18 +115,6 @@ CREATE TEMP TABLE product_expanded AS - - - - - - - - - - - - -- THIS ONLY WORKS IF YOU HAVE DONE THE PROPER STEPS FOR IMPORTING -- 1) update new_customer_purchases to today -- 2) add the union @@ -94,12 +126,14 @@ CREATE TEMP TABLE product_expanded AS -------------------------------------------------------------------------------------------------------------------------------------------- -/* MODULE 5 */ +/* MODULE 5 */ /* UPDATE statements for view */ /* 1. SET market_date equal to today for new_customer_purchases */ +UPDATE new_customer_purchases +SET market_date = DATE('now','localtime') @@ -116,10 +150,11 @@ VALUES('....','....','....','....','8:00 AM','2:00 PM','nothing interesting','Su */ - +INSERT INTO market_date_info +VALUES('2026-08-05','Wednesday','32','2026','8:00 AM','2:00 PM','nothing interesting','Summer','25','28',0,0) -------------------------------------------------------------------------------------------------------------------------------------------- -/* MODULE 5 */ +/* MODULE 5 */ /* CROSS JOIN */ @@ -135,11 +170,30 @@ VALUES('small'), SELECT * FROM TEMP.sizes; +DROP TABLE IF EXISTS TEMP.colours; +CREATE TEMP TABLE IF NOT EXISTS TEMP.colours (colour TEXT); + +INSERT INTO TEMP.colours +VALUES('red'), +('green'), +('orange'), +('purple'); + +SELECT * FROM TEMP.sizes; +SELECT * FROM TEMP.colours; + +SELECT product_name,product_qty_type,size--,colour +FROM product -- 23 rows +CROSS JOIN temp.sizes -- 3 ROWS +-- 3*23 = 69 rows for the cartesian product +--CROSS JOIN temp.colours -- 4 ROWS -- add colour if you would like +-- 69*4 = 276 rows for the cartesian product + -------------------------------------------------------------------------------------------------------------------------------------------- -/* MODULE 5 */ +/* MODULE 5 */ /* SELF JOIN */ @@ -160,4 +214,56 @@ VALUES(1,'Thomas',3) ,(4, 'Jennie',3); SELECT * FROM TEMP.employees; - + +SELECT e.emp_name, m.emp_name as mgr_name -- rename the column so its possible to understand the structure +,coalesce(m.emp_name,e.emp_name) as self_boss -- if you want Rohan to be his own boss + +FROM temp.employees as e +LEFT JOIN temp.employees as m + ON e.mgr_id = m.emp_id -- identifying which id is the parent and which id is the child + + + + +--JSON to TABLE + +-- create a temp TABLE +-- insert the json as "long string" aka BLOB +-- write a json_each statement +-- use the json_each statement as a subquery with a json_extract to create our column values + +DROP TABLE IF EXISTS temp.country_json; +CREATE TABLE IF NOT EXISTS temp.country_json +( +the_json BLOB -- the column and the column type +); + +INSERT INTO temp.country_json +VALUES( +'[ + { + "country": "Afghanistan", + "city": "Kabul" + }, + { + "country": "Albania", + "city": "Tirana" + } +]' +); + +SELECT key +,JSON_EXTRACT(value,'$.country') as country +,JSON_EXTRACT(value,'$.city') as city + +FROM ( + SELECT * + FROM country_json,JSON_EACH(country_json.the_json, '$') -- for each of the json elements, everything +)CREATE TABLE "booth" ( + "booth_number" int(11) NOT NULL, + "booth_price_level" varchar(45) NOT NULL, + "booth_description" varchar(255) NOT NULL, + "booth_type" varchar(45) NOT NULL, + PRIMARY KEY ("booth_number"), + UNIQUE ("booth_number") +) diff --git a/04_this_cohort/live_code/module_5/update_statements_for_view.sql b/04_this_cohort/live_code/module_5/update_statements_for_view.sql new file mode 100644 index 000000000..153212ce7 --- /dev/null +++ b/04_this_cohort/live_code/module_5/update_statements_for_view.sql @@ -0,0 +1,28 @@ +/* MODULE 5 */ +/* UPDATE statements for view */ + + +/* 1. SET market_date equal to today for new_customer_purchases */ + +UPDATE new_customer_purchases +SET market_date = DATE('now','localtime') + + + +/* 2. Add today's info to the market_date_info + +we need to add +1. today's date +2. today's day +3. today's week number +4. today's year + +INSERT INTO market_date_info +VALUES('....','....','....','....','8:00 AM','2:00 PM','nothing interesting','Summer','25','28',0,0); + +*/ + +INSERT INTO market_date_info +VALUES('2026-08-05','Wednesday','32','2026','8:00 AM','2:00 PM','nothing interesting','Summer','25','28',0,0) + +-------------------------------------------------------------------------------------------------------------------------------------------- diff --git a/04_this_cohort/live_code/module_5/view_in_another_query.sql b/04_this_cohort/live_code/module_5/view_in_another_query.sql new file mode 100644 index 000000000..021a1a496 --- /dev/null +++ b/04_this_cohort/live_code/module_5/view_in_another_query.sql @@ -0,0 +1,18 @@ +/* MODULE 5 */ +/* VIEW in another query */ + +/* 1. Transform the daily sales view into a sales by vendor per week result */ +SELECT +market_year +,market_week +,vendor_name +,SUM(sales) as sales + + +FROM vendor_daily_sales + +GROUP BY market_year +,market_week +,vendor_name + +--------------------------------------------------------------------------------------------------------------------------------------------