{"cells":[{"source":" $\"Kaggle\"$ ","metadata":{},"cell_type":"markdown"},{"cell_type":"markdown","id":"2ecd54de","metadata":{"papermill":{"duration":0.005575,"end_time":"2024-10-31T22:57:48.523142","exception":false,"start_time":"2024-10-31T22:57:48.517567","status":"completed"},"tags":[]},"source":["# Preprocessing Summary\n","\n","This notebook covers essential preprocessing steps for preparing data for machine learning regression models. Each section performs a specific transformation on the data to ensure it's suitable for modeling. Below is a summary of the sections:\n","\n","1. **Dataset Importation**: The dataset is loaded using `pandas` for data manipulation and analysis.\n","2. **Feature Selection**: Independent (X) and dependent (Y) variables are extracted from the dataset, where X contains the features, and Y is the target variable.\n","3. **Handling Missing Values**: Missing values in the dataset are handled using `SimpleImputer`, which replaces `NaN` values with the mean of the respective feature.\n","4. **Encoding Categorical Data**: The dataset contains categorical variables, which are encoded using `LabelEncoder` and `OneHotEncoder` to convert them into a numerical format suitable for machine learning algorithms.\n","5. **Train-Test Split**: The data is split into training and testing sets using `train_test_split` from scikit-learn.\n","6. **Feature Scaling**: Standardization of features is done with `StandardScaler`, which normalizes the feature values to ensure that each has a mean of 0 and a standard deviation of 1.\n","\n","The diagram below illustrates these preprocessing steps:\n","\n","![Data Preprocessing Workflow](https://www.techtarget.com/rms/onlineimages/steps_for_data_preprocessing-f_mobile.png)\n"]},{"cell_type":"code","execution_count":1,"id":"c66ccc55","metadata":{"execution":{"iopub.execute_input":"2024-10-31T22:57:48.534155Z","iopub.status.busy":"2024-10-31T22:57:48.533419Z","iopub.status.idle":"2024-10-31T22:58:01.068425Z","shell.execute_reply":"2024-10-31T22:58:01.067126Z"},"id":"NeTri3Ay1Z1s","outputId":"7aff8ce1-119d-4fbe-d1da-fb0276a0c0ac","papermill":{"duration":12.54338,"end_time":"2024-10-31T22:58:01.071213","exception":false,"start_time":"2024-10-31T22:57:48.527833","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["Requirement already satisfied: scikit-learn in /opt/conda/lib/python3.7/site-packages (1.0.2)\r\n","Requirement already satisfied: joblib>=0.11 in /opt/conda/lib/python3.7/site-packages (from scikit-learn) (1.2.0)\r\n","Requirement already satisfied: numpy>=1.14.6 in /opt/conda/lib/python3.7/site-packages (from scikit-learn) (1.21.6)\r\n","Requirement already satisfied: scipy>=1.1.0 in /opt/conda/lib/python3.7/site-packages (from scikit-learn) (1.7.3)\r\n","Requirement already satisfied: threadpoolctl>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from scikit-learn) (3.1.0)\r\n","\u001b[33mWARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv\u001b[0m\u001b[33m\r\n","\u001b[0m"]}],"source":["! pip install -U scikit-learn"]},{"cell_type":"code","execution_count":2,"id":"64ed89b0","metadata":{"execution":{"iopub.execute_input":"2024-10-31T22:58:01.083443Z","iopub.status.busy":"2024-10-31T22:58:01.082581Z","iopub.status.idle":"2024-10-31T22:58:01.09726Z","shell.execute_reply":"2024-10-31T22:58:01.096469Z"},"id":"xvqJ3ledxD4e","papermill":{"duration":0.023033,"end_time":"2024-10-31T22:58:01.099455","exception":false,"start_time":"2024-10-31T22:58:01.076422","status":"completed"},"tags":[]},"outputs":[],"source":["import pandas as pd\n","\n","dataset = pd.read_csv('/kaggle/input/dataforpreprocessing/Data.csv')"]},{"cell_type":"code","execution_count":3,"id":"e5ae51db","metadata":{"execution":{"iopub.execute_input":"2024-10-31T22:58:01.110341Z","iopub.status.busy":"2024-10-31T22:58:01.110049Z","iopub.status.idle":"2024-10-31T22:58:01.131106Z","shell.execute_reply":"2024-10-31T22:58:01.130074Z"},"id":"P9Yi0Ymby533","outputId":"93a4fa52-868b-4fa8-c3e1-37d0fe760c32","papermill":{"duration":0.02943,"end_time":"2024-10-31T22:58:01.133734","exception":false,"start_time":"2024-10-31T22:58:01.104304","status":"completed"},"tags":[]},"outputs":[{"data":{"text/html":["

\n","\n","\n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n"," \n","

	Country	Age	Salary	Purchased
0	France	44.0	72000.0	No
1	Spain	27.0	48000.0	Yes
2	Germany	30.0	54000.0	No
3	Spain	38.0	61000.0	No
4	Germany	40.0	NaN	Yes
5	France	35.0	58000.0	Yes
6	Spain	NaN	52000.0	No
7	France	48.0	79000.0	Yes
8	Germany	50.0	83000.0	No
9	France	37.0	67000.0	Yes

\n","

"],"text/plain":[" Country Age Salary Purchased\n","0 France 44.0 72000.0 No\n","1 Spain 27.0 48000.0 Yes\n","2 Germany 30.0 54000.0 No\n","3 Spain 38.0 61000.0 No\n","4 Germany 40.0 NaN Yes\n","5 France 35.0 58000.0 Yes\n","6 Spain NaN 52000.0 No\n","7 France 48.0 79000.0 Yes\n","8 Germany 50.0 83000.0 No\n","9 France 37.0 67000.0 Yes"]},"execution_count":3,"metadata":{},"output_type":"execute_result"}],"source":["dataset"]},{"cell_type":"code","execution_count":4,"id":"51f0fcda","metadata":{"execution":{"iopub.execute_input":"2024-10-31T22:58:01.145433Z","iopub.status.busy":"2024-10-31T22:58:01.145141Z","iopub.status.idle":"2024-10-31T22:58:01.153686Z","shell.execute_reply":"2024-10-31T22:58:01.152765Z"},"id":"Wtgpleq5y72k","outputId":"5051ae14-ab05-4ba5-c50d-10eeafbebfa2","papermill":{"duration":0.017111,"end_time":"2024-10-31T22:58:01.156027","exception":false,"start_time":"2024-10-31T22:58:01.138916","status":"completed"},"tags":[]},"outputs":[{"data":{"text/plain":["array([['France', 44.0, 72000.0],\n"," ['Spain', 27.0, 48000.0],\n"," ['Germany', 30.0, 54000.0],\n"," ['Spain', 38.0, 61000.0],\n"," ['Germany', 40.0, nan],\n"," ['France', 35.0, 58000.0],\n"," ['Spain', nan, 52000.0],\n"," ['France', 48.0, 79000.0],\n"," ['Germany', 50.0, 83000.0],\n"," ['France', 37.0, 67000.0]], dtype=object)"]},"execution_count":4,"metadata":{},"output_type":"execute_result"}],"source":["#independent variables\n","X =dataset.iloc[:,:-1].values\n","X"]},{"cell_type":"code","execution_count":5,"id":"d48bed4c","metadata":{"execution":{"iopub.execute_input":"2024-10-31T22:58:01.16747Z","iopub.status.busy":"2024-10-31T22:58:01.167179Z","iopub.status.idle":"2024-10-31T22:58:01.173429Z","shell.execute_reply":"2024-10-31T22:58:01.172483Z"},"id":"0Yxp7uZGzO-1","outputId":"532c6b3b-4ab4-4acf-b50d-3090eaf6f8f8","papermill":{"duration":0.014474,"end_time":"2024-10-31T22:58:01.175672","exception":false,"start_time":"2024-10-31T22:58:01.161198","status":"completed"},"tags":[]},"outputs":[{"data":{"text/plain":["array(['No', 'Yes', 'No', 'No', 'Yes', 'Yes', 'No', 'Yes', 'No', 'Yes'],\n"," dtype=object)"]},"execution_count":5,"metadata":{},"output_type":"execute_result"}],"source":["#dependent variable\n","Y = dataset.iloc[:,3].values\n","Y"]},{"cell_type":"code","execution_count":6,"id":"8b67d93b","metadata":{"execution":{"iopub.execute_input":"2024-10-31T22:58:01.187491Z","iopub.status.busy":"2024-10-31T22:58:01.186999Z","iopub.status.idle":"2024-10-31T22:58:02.24377Z","shell.execute_reply":"2024-10-31T22:58:02.242178Z"},"id":"1ryH9KAs1_gG","outputId":"ea3ec3a0-5a14-4f5a-bfe8-3f7e26a41b10","papermill":{"duration":1.065646,"end_time":"2024-10-31T22:58:02.246527","exception":false,"start_time":"2024-10-31T22:58:01.180881","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["[['France' 44.0 72000.0]\n"," ['Spain' 27.0 48000.0]\n"," ['Germany' 30.0 54000.0]\n"," ['Spain' 38.0 61000.0]\n"," ['Germany' 40.0 63777.77777777778]\n"," ['France' 35.0 58000.0]\n"," ['Spain' 38.77777777777778 52000.0]\n"," ['France' 48.0 79000.0]\n"," ['Germany' 50.0 83000.0]\n"," ['France' 37.0 67000.0]]\n"]}],"source":["#handle missing values\n","from sklearn.impute import SimpleImputer\n","import numpy as np\n","\n","imputer = SimpleImputer(missing_values = np.NaN, strategy = 'mean', )\n","imputer = imputer.fit(X[:, 1:3])\n","X[:,1:3]=imputer.transform(X[:,1:3])\n","\n","print(X)"]},{"cell_type":"code","execution_count":7,"id":"37e2b5c0","metadata":{"execution":{"iopub.execute_input":"2024-10-31T22:58:02.260785Z","iopub.status.busy":"2024-10-31T22:58:02.260392Z","iopub.status.idle":"2024-10-31T22:58:02.266447Z","shell.execute_reply":"2024-10-31T22:58:02.265494Z"},"id":"wxrPQ8zB2uTm","outputId":"3bf09d79-5709-443f-c6d6-0bb850556a01","papermill":{"duration":0.016156,"end_time":"2024-10-31T22:58:02.269172","exception":false,"start_time":"2024-10-31T22:58:02.253016","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["[0 1 0 0 1 1 0 1 0 1]\n"]}],"source":["#handle categorical data\n","from sklearn.preprocessing import OneHotEncoder , LabelEncoder\n","encoder = LabelEncoder()\n","Y = encoder.fit_transform(Y)\n","\n","print(Y)"]},{"cell_type":"code","execution_count":8,"id":"f488214f","metadata":{"execution":{"iopub.execute_input":"2024-10-31T22:58:02.281852Z","iopub.status.busy":"2024-10-31T22:58:02.281497Z","iopub.status.idle":"2024-10-31T22:58:02.289097Z","shell.execute_reply":"2024-10-31T22:58:02.288147Z"},"id":"MWHNkKUM39PF","outputId":"5c8785c4-051d-4a43-825e-42e2912dd8d1","papermill":{"duration":0.016928,"end_time":"2024-10-31T22:58:02.291765","exception":false,"start_time":"2024-10-31T22:58:02.274837","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["[[1.0 0.0 0.0 44.0 72000.0]\n"," [0.0 0.0 1.0 27.0 48000.0]\n"," [0.0 1.0 0.0 30.0 54000.0]\n"," [0.0 0.0 1.0 38.0 61000.0]\n"," [0.0 1.0 0.0 40.0 63777.77777777778]\n"," [1.0 0.0 0.0 35.0 58000.0]\n"," [0.0 0.0 1.0 38.77777777777778 52000.0]\n"," [1.0 0.0 0.0 48.0 79000.0]\n"," [0.0 1.0 0.0 50.0 83000.0]\n"," [1.0 0.0 0.0 37.0 67000.0]]\n"]}],"source":["oneHotEncoder = OneHotEncoder()\n","X_transformed = oneHotEncoder.fit_transform(X[:,[0]]).toarray()\n","# Reshape the remaining columns to ensure they have two dimensions\n","X_remaining = X[:, 1:].reshape(X.shape[0], -1)\n","X_encoded = np.hstack((X_transformed , X_remaining))\n","\n","print(X_encoded)"]},{"cell_type":"code","execution_count":9,"id":"66f91230","metadata":{"execution":{"iopub.execute_input":"2024-10-31T22:58:02.304225Z","iopub.status.busy":"2024-10-31T22:58:02.303932Z","iopub.status.idle":"2024-10-31T22:58:02.309763Z","shell.execute_reply":"2024-10-31T22:58:02.308965Z"},"id":"CW7bK_vO4iy5","papermill":{"duration":0.014405,"end_time":"2024-10-31T22:58:02.311828","exception":false,"start_time":"2024-10-31T22:58:02.297423","status":"completed"},"tags":[]},"outputs":[],"source":["from sklearn.model_selection import train_test_split\n","# Split the data into training and test sets\n","X_train, X_test, y_train, y_test = train_test_split(X_encoded, Y, test_size=0.2, random_state=42)\n"]},{"cell_type":"code","execution_count":10,"id":"e18741be","metadata":{"execution":{"iopub.execute_input":"2024-10-31T22:58:02.324084Z","iopub.status.busy":"2024-10-31T22:58:02.323565Z","iopub.status.idle":"2024-10-31T22:58:02.331432Z","shell.execute_reply":"2024-10-31T22:58:02.330144Z"},"id":"649l1c6Dpy8t","outputId":"ef692038-12de-4b27-b851-2b1112cc774c","papermill":{"duration":0.016515,"end_time":"2024-10-31T22:58:02.333833","exception":false,"start_time":"2024-10-31T22:58:02.317318","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["[[1.0 0.0 0.0 -0.7529426005471072 -0.6260377781240918]\n"," [1.0 0.0 0.0 1.008453807952985 1.0130429500553495]\n"," [1.0 0.0 0.0 1.7912966561752484 1.8325833141450703]\n"," [0.0 1.0 0.0 -1.7314961608249362 -1.0943465576039322]\n"," [1.0 0.0 0.0 -0.3615211764359756 0.42765697570554906]\n"," [0.0 1.0 0.0 0.22561095973072184 0.05040823668012247]\n"," [0.0 0.0 1.0 -0.16581046438040975 -0.27480619351421154]\n"," [0.0 0.0 1.0 -0.013591021670525094 -1.3285009473438525]]\n"]}],"source":["#feature scaler\n","from sklearn.preprocessing import StandardScaler\n","\n","sts = StandardScaler()\n","X_train[:,3:5] = sts.fit_transform(X_train[:,3:5])\n","X_test[:,3:5] = sts.transform(X_test[:,3:5])\n","\n","print(X_train)"]},{"cell_type":"code","execution_count":11,"id":"de15017a","metadata":{"execution":{"iopub.execute_input":"2024-10-31T22:58:02.346457Z","iopub.status.busy":"2024-10-31T22:58:02.345938Z","iopub.status.idle":"2024-10-31T22:58:06.465799Z","shell.execute_reply":"2024-10-31T22:58:06.464743Z"},"id":"kBTEWBkI958m","outputId":"1bd16bb2-1e7a-48bc-f2d6-f7cbf9781752","papermill":{"duration":4.128945,"end_time":"2024-10-31T22:58:06.468365","exception":false,"start_time":"2024-10-31T22:58:02.33942","status":"completed"},"tags":[]},"outputs":[{"name":"stdout","output_type":"stream","text":["Linear Regression: MSE = 0.910 \n"," R^2 Score = -2.6391\n","Ridge Regression: MSE = 0.746 \n"," R^2 Score = -1.9827\n","Lasso Regression: MSE = 0.250 \n"," R^2 Score = 0.0000\n","Decision Tree: MSE = 1.000 \n"," R^2 Score = -3.0000\n","Random Forest: MSE = 0.661 \n"," R^2 Score = -1.6440\n","Support Vector Regression: MSE = 0.557 \n"," R^2 Score = -1.2266\n","XGBoost: MSE = 1.005 \n"," R^2 Score = -3.0184\n","\n","Best Model: Lasso Regression with MSE = 0.250\n"]}],"source":["from sklearn.linear_model import LinearRegression, Ridge, Lasso\n","from sklearn.tree import DecisionTreeRegressor\n","from sklearn.ensemble import RandomForestRegressor\n","from sklearn.svm import SVR\n","from sklearn.neighbors import KNeighborsRegressor\n","from xgboost import XGBRegressor\n","from sklearn.metrics import mean_squared_error, mean_absolute_error, r2_score\n","\n","\n","\n","models = [\n"," ('Linear Regression', LinearRegression()),\n"," ('Ridge Regression', Ridge()),\n"," ('Lasso Regression', Lasso()),\n"," ('Decision Tree', DecisionTreeRegressor()),\n"," ('Random Forest', RandomForestRegressor()),\n"," ('Support Vector Regression', SVR()),\n"," ('XGBoost', XGBRegressor(objective='reg:squarederror', random_state=42))\n","]\n","\n","# Dictionary to store the MSE for each model\n","mse_scores = {}\n","\n","# Evaluate each model\n","for name, model in models:\n"," # Fit the model to the training data\n"," model.fit(X_train, y_train)\n","\n"," # Predict the target variable on the test data\n"," y_pred = model.predict(X_test)\n","\n"," # Calculate the Mean Squared Error (MSE)\n"," mse = mean_squared_error(y_test, y_pred)\n","\n"," # Store the MSE score in the dictionary\n"," mse_scores[name] = mse\n"," r2 = r2_score(y_test, y_pred)\n","\n"," print(f\"{name}: MSE = {mse:.3f} \\n R^2 Score = {r2:.4f}\")\n","\n","\n","\n","# Find the model with the lowest MSE\n","best_model_name = min(mse_scores, key=mse_scores.get)\n","best_mse = mse_scores[best_model_name]\n","\n","print(f\"\\nBest Model: {best_model_name} with MSE = {best_mse:.3f}\")"]},{"cell_type":"code","execution_count":null,"id":"c870ad4b","metadata":{"papermill":{"duration":0.005997,"end_time":"2024-10-31T22:58:06.481053","exception":false,"start_time":"2024-10-31T22:58:06.475056","status":"completed"},"tags":[]},"outputs":[],"source":[]}],"metadata":{"colab":{"provenance":[]},"kaggle":{"accelerator":"gpu","dataSources":[{"datasetId":3156070,"sourceId":5463215,"sourceType":"datasetVersion"},{"datasetId":5968335,"sourceId":9748747,"sourceType":"datasetVersion"},{"datasetId":5988228,"sourceId":9775479,"sourceType":"datasetVersion"}],"dockerImageVersionId":30458,"isGpuEnabled":true,"isInternetEnabled":true,"language":"python","sourceType":"notebook"},"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.7.12"},"papermill":{"default_parameters":{},"duration":29.631789,"end_time":"2024-10-31T22:58:07.608551","environment_variables":{},"exception":null,"input_path":"__notebook__.ipynb","output_path":"__notebook__.ipynb","parameters":{},"start_time":"2024-10-31T22:57:37.976762","version":"2.4.0"}},"nbformat":4,"nbformat_minor":5}

FilesExpand file tree

data-preprocessing-and-model-selection-regression.ipynb

Latest commit

History

data-preprocessing-and-model-selection-regression.ipynb

File metadata and controls