Skip to content

lab tried for practicing & revising #365

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tomapasta wants to merge 1 commit into
base: main
Choose a base branch
from
Open

lab tried for practicing & revising #365

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 92 additions & 0 deletions .ipynb_checkpoints/README-checkpoint.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
![logo_ironhack_blue 7](https://user-images.githubusercontent.com/23629340/40541063-a07a0a8a-601a-11e8-91b5-2f13e4e6b441.png)

# LAB | Connecting Python to SQL

<details>
<summary>
<h2>Learning Goals</h2>
</summary>

This lab allows you to practice and apply the concepts and techniques taught in class.

Upon completion of this lab, you will be able to:

- Write a Python script to connect to a relational database using the appropriate Python library and query it using SQL commands.

<br>
<hr>

</details>

<details>
<summary>
<h2>Prerequisites</h2>
</summary>

Before this starting this lab, you should have learnt about:

- Basic SQL queries
- Python and Pandas
- Data Wrangling, which involves tasks such as grouping, aggregating, dealing with indexes, renaming columns, merging data and performing calculations.

<br>
<hr>

</details>


## Introduction

Welcome to the Connecting Python to SQL lab!

In this lab, you will be working with the [Sakila](https://dev.mysql.com/doc/sakila/en/) database on movie rentals. Specifically, you will be practicing how to do basic SQL queries using Python. By connecting Python to SQL, you can leverage the power of both languages to efficiently manipulate and analyze large datasets. Throughout this lab, you will practice how to use Python to retrieve and manipulate data stored in the Sakila database using SQL queries. Let's get started!

## Challenge

In this lab, the objective is to identify the customers who were active in both May and June, and how did their activity differ between months. To achieve this, follow these steps:

1. Establish a connection between Python and the Sakila database.

2. Write a Python function called `rentals_month` that retrieves rental data for a given month and year (passed as parameters) from the Sakila database as a Pandas DataFrame. The function should take in three parameters:

- `engine`: an object representing the database connection engine to be used to establish a connection to the Sakila database.
- `month`: an integer representing the month for which rental data is to be retrieved.
- `year`: an integer representing the year for which rental data is to be retrieved.

The function should execute a SQL query to retrieve the rental data for the specified month and year from the rental table in the Sakila database, and return it as a pandas DataFrame.

3. Develop a Python function called `rental_count_month` that takes the DataFrame provided by `rentals_month` as input along with the month and year and returns a new DataFrame containing the number of rentals made by each customer_id during the selected month and year.

The function should also include the month and year as parameters and use them to name the new column according to the month and year, for example, if the input month is 05 and the year is 2005, the column name should be "rentals_05_2005".


*Hint: Consider making use of pandas [groupby()](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.groupby.html)*

4. Create a Python function called `compare_rentals` that takes two DataFrames as input containing the number of rentals made by each customer in different months and years.
The function should return a combined DataFrame with a new 'difference' column, which is the difference between the number of rentals in the two months.

## Requirements

- Fork this repo
- Clone it to your machine



## Getting Started

Complete the challenges. Follow the instructions and add your code and explanations as necessary.

## Submission

- Upon completion, run the following commands:

```bash
git add .
git commit -m "Solved lab"
git push origin master
```

- Paste the link of your lab in Student Portal.



302 changes: 302 additions & 0 deletions .ipynb_checkpoints/lab-sql-python-connection-checkpoint.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,302 @@
{
"cells": [
{
"cell_type": "code",
"execution_count": 1,
"id": "810c130a-4818-4e78-af5a-5b257a7fe455",
"metadata": {},
"outputs": [
{
"name": "stdin",
"output_type": "stream",
"text": [
"Please, input your SQL password: ········\n"
]
}
],
"source": [
"#1.2 In this lab, the objective is to identify the customers who were active \n",
"#in both May and June, and how did their activity differ between months. \n",
"#To achieve this, follow these steps:\n",
"\n",
"#Establish a connection between Python and the Sakila database.\n",
"import pandas as pd\n",
"import numpy as np\n",
"import pymysql\n",
"from sqlalchemy import create_engine\n",
"import getpass # To get the password without showing the input\n",
"password = getpass.getpass(\"Please, input your SQL password: \")\n"
]
},
{
"cell_type": "code",
"execution_count": 13,
"id": "d16152b6-1e76-4138-b570-3715b1967aa6",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"<div>\n",
"<style scoped>\n",
" .dataframe tbody tr th:only-of-type {\n",
" vertical-align: middle;\n",
" }\n",
"\n",
" .dataframe tbody tr th {\n",
" vertical-align: top;\n",
" }\n",
"\n",
" .dataframe thead th {\n",
" text-align: right;\n",
" }\n",
"</style>\n",
"<table border=\"1\" class=\"dataframe\">\n",
" <thead>\n",
" <tr style=\"text-align: right;\">\n",
" <th></th>\n",
" <th>rental_id</th>\n",
" <th>rental_date</th>\n",
" <th>inventory_id</th>\n",
" <th>customer_id</th>\n",
" <th>return_date</th>\n",
" <th>staff_id</th>\n",
" <th>last_update</th>\n",
" </tr>\n",
" </thead>\n",
" <tbody>\n",
" </tbody>\n",
"</table>\n",
"</div>"
],
"text/plain": [
"Empty DataFrame\n",
"Columns: [rental_id, rental_date, inventory_id, customer_id, return_date, staff_id, last_update]\n",
"Index: []"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#2 Write a Python function called rentals_month that retrieves rental data for \n",
"#a given month and year (passed as parameters) from the Sakila database as \n",
"#a Pandas DataFrame. The function should take in three parameters:\n",
"#engine: an object representing the database connection engine to be used \n",
"#to establish a connection to the Sakila database.\n",
"#month: an integer representing the month for which rental data is to be retrieved.\n",
"#year: an integer representing the year for which rental data is to be retrieved.\n",
"\n",
"import pandas as pd\n",
"from sqlalchemy import create_engine, text\n",
"\n",
"bd = \"sakila\"\n",
"connection_string = f'mysql+pymysql://root:{password}@localhost/{bd}'\n",
"engine = create_engine(connection_string)\n",
"\n",
"#first attempt : \n",
"#query = text(\"SELECT * FROM sakila.rental\")\n",
"#df_rental = pd.read_sql(query, engine)\n",
"#print(\"DataFrame loaded successfully!\")\n",
"#df_rental.head()\n",
"\n",
"#The function should execute a SQL query to retrieve the rental data for \n",
"#the specified month and year from the rental table in the Sakila database, \n",
"#and return it as a pandas DataFrame.\n",
"def rentals_month (engine, month, year):\n",
" query = text(\"\"\"\n",
" SELECT \n",
" * FROM \n",
" sakila.rental \n",
" WHERE \n",
" MONTH(rental_date) = :month AND \n",
" YEAR(rental_date) = :year\n",
" \"\"\")\n",
" \n",
" df_rental = pd.read_sql(\n",
" query, \n",
" engine, \n",
" params={'month': month, 'year': year}\n",
" )\n",
" \n",
" return df_rental\n",
"#testing rentals_month (engine, 7,5)"
]
},
{
"cell_type": "code",
"execution_count": 3,
"id": "47f1a3b4-0cf2-4ba1-8e27-077a6d6d9bdb",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"MySQL Connector/Python is installed and accessible!\n"
]
}
],
"source": [
"import mysql.connector\n",
"print(\"MySQL Connector/Python is installed and accessible!\")"
]
},
{
"cell_type": "code",
"execution_count": 24,
"id": "e48d2e6b-c973-4635-b86d-c9e42bddd495",
"metadata": {},
"outputs": [
{
"ename": "NameError",
"evalue": "name 'month' is not defined",
"output_type": "error",
"traceback": [
"\u001b[31m---------------------------------------------------------------------------\u001b[39m",
"\u001b[31mNameError\u001b[39m Traceback (most recent call last)",
"\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[24]\u001b[39m\u001b[32m, line 40\u001b[39m\n\u001b[32m 32\u001b[39m rental_count_month_df = pd.read_sql(\n\u001b[32m 33\u001b[39m query, \n\u001b[32m 34\u001b[39m engine, \n\u001b[32m 35\u001b[39m params={\u001b[33m'\u001b[39m\u001b[33mmonth\u001b[39m\u001b[33m'\u001b[39m: month, \u001b[33m'\u001b[39m\u001b[33myear\u001b[39m\u001b[33m'\u001b[39m: year}\n\u001b[32m 36\u001b[39m )\n\u001b[32m 38\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m rental_count_month_df\n\u001b[32m---> \u001b[39m\u001b[32m40\u001b[39m \u001b[43mrental_count_month\u001b[49m\u001b[43m(\u001b[49m\u001b[43mrentals_month\u001b[49m\u001b[43m)\u001b[49m\n",
"\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[24]\u001b[39m\u001b[32m, line 13\u001b[39m, in \u001b[36mrental_count_month\u001b[39m\u001b[34m(df_rental)\u001b[39m\n\u001b[32m 11\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34mrental_count_month\u001b[39m (df_rental):\n\u001b[32m---> \u001b[39m\u001b[32m13\u001b[39m month_str = \u001b[38;5;28mstr\u001b[39m(\u001b[43mmonth\u001b[49m)\n\u001b[32m 14\u001b[39m year_str = \u001b[38;5;28mstr\u001b[39m(year)\n\u001b[32m 16\u001b[39m new_column_name = \u001b[33mf\u001b[39m\u001b[33m\"\u001b[39m\u001b[33mrentals_\u001b[39m\u001b[38;5;132;01m{\u001b[39;00mmonth_str\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m_\u001b[39m\u001b[38;5;132;01m{\u001b[39;00myear_str\u001b[38;5;132;01m}\u001b[39;00m\u001b[33m\"\u001b[39m\n",
"\u001b[31mNameError\u001b[39m: name 'month' is not defined"
]
}
],
"source": [
"#3. Develop a Python function called rental_count_month that takes the DataFrame \n",
"#provided by rentals_month as input along with the month and year and returns \n",
"#a new DataFrame containing the number of rentals made by each customer_id \n",
"#during the selected month and year.\n",
"#The function should also include the month and year as parameters and \n",
"#use them to name the new column according to the month and year, for example, \n",
"#if the input month is 05 and the year is 2005, \n",
"#the column name should be \"rentals_05_2005\".\n",
"#Hint: Consider making use of pandas groupby()\n",
"\n",
"def rental_count_month (df_rental):\n",
"\n",
" month_str = str(month)\n",
" year_str = str(year)\n",
" \n",
" new_column_name = f\"rentals_{month_str}_{year_str}\"\n",
" \n",
" query = text(f\"\"\"\n",
" SELECT \n",
" customer_id, \n",
" COUNT(rental_id) AS {new_column_name} \n",
" FROM \n",
" rental \n",
" WHERE \n",
" MONTH(rental_date) = :month AND \n",
" YEAR(rental_date) = :year\n",
" GROUP BY \n",
" customer_id\n",
" \"\"\")\n",
" \n",
" # 3. Read data and pass parameters securely\n",
" rental_count_month_df = pd.read_sql(\n",
" query, \n",
" engine, \n",
" params={'month': month, 'year': year}\n",
" )\n",
" \n",
" return rental_count_month_df\n",
" \n",
"rental_count_month(rentals_month)"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "a90917c5-6e61-453d-9c34-4680b9ba6c54",
"metadata": {},
"outputs": [],
"source": [
"#4. Create a Python function called compare_rentals that takes two DataFrames \n",
"#as input.\n",
"#containing the number of rentals made by each customer in different months and \n",
"#yr\n",
"#The function should return a combined DataFrame with a new 'difference' column, \n",
"#which is the difference between the number of rentals in the two months."
]
},
{
"cell_type": "code",
"execution_count": 27,
"id": "8fcb8bef-1b97-41b6-a9f5-3d400656724f",
"metadata": {},
"outputs": [
{
"ename": "NameError",
"evalue": "name 'rental_count_month_df' is not defined",
"output_type": "error",
"traceback": [
"\u001b[31m---------------------------------------------------------------------------\u001b[39m",
"\u001b[31mNameError\u001b[39m Traceback (most recent call last)",
"\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[27]\u001b[39m\u001b[32m, line 27\u001b[39m\n\u001b[32m 23\u001b[39m combined_df = combined_df.fillna(\u001b[32m0\u001b[39m)\n\u001b[32m 25\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m combined_df\n\u001b[32m---> \u001b[39m\u001b[32m27\u001b[39m compare_rentals(\u001b[43mrental_count_month_df\u001b[49m(df_rental))\n",
"\u001b[31mNameError\u001b[39m: name 'rental_count_month_df' is not defined"
]
}
],
"source": [
"import pandas as pd\n",
"\n",
"def compare_rentals(rental_count_month_df, df_rental):\n",
" \n",
" count_col_1 = [col for col in df1.columns if col != 'customer_id'][0]\n",
" count_col_2 = [col for col in df2.columns if col != 'customer_id'][0]\n",
"\n",
" \n",
" combined_df = pd.merge(\n",
" df1, \n",
" df2, \n",
" on='customer_id', \n",
" how='outer'\n",
" )\n",
" \n",
" \n",
" combined_df['difference'] = combined_df[count_col_2].sub(\n",
" combined_df[count_col_1], \n",
" fill_value=0\n",
" )\n",
" \n",
" \n",
" combined_df = combined_df.fillna(0)\n",
" \n",
" return combined_df\n",
"\n",
"compare_rentals(rental_count_month_df(df_rental))"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "396f577e-cf7e-46e2-9eaa-fa1a251ea1d9",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "data_env",
"language": "python",
"name": "data_env"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.13.9"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Loading