DATE:
EXP NO. PAGE NO :
WEEK-5:
AIM: Demonstrate decision tree algorithm for a classification problem and perform parameter tuning for
better results
DESCRIPTION:
Decision Tree is a supervised learning algorithm that splits data into subsets based on feature values,
creating a tree-like structure of decisions. Each internal node represents a feature, each branch represents a
decision, and each leaf node represents a class label.
To improve the model's performance, we use parameter tuning with GridSearchCV to find the optimal
hyperparameters for the decision tree. These hyperparameters include:
max_depth: The maximum depth of the tree.
min_samples_split: The minimum number of samples required to split an internal node.
min_samples_leaf: The minimum number of samples required to be at a leaf node.
criterion: The measure of the quality of a split (either Gini or Entropy).
CODE:
import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split, GridSearchCV
from [Link] import DecisionTreeClassifier
from [Link] import load_iris
from [Link] import accuracy_score, confusion_matrix, classification_report
data = load_iris()
X = [Link]
y = [Link]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
dt_model = DecisionTreeClassifier(random_state=42)
dt_model.fit(X_train, y_train)
y_pred = dt_model.predict(X_test)
accuracy = accuracy_score(y_test, y_pred)
conf_matrix = confusion_matrix(y_test, y_pred)
class_report = classification_report(y_test, y_pred)
print(f"Initial Accuracy: {accuracy * 100:.2f}%")
print("\nConfusion Matrix:")
print(conf_matrix)
print("\nClassification Report:")
print(class_report)
param_grid = {
'max_depth': [3, 5, 10, None],
'min_samples_split': [2, 5, 10],
'min_samples_leaf': [1, 2, 5],
'criterion': ['gini', 'entropy']
}
ADITYA UNIVERSITY [Link]. 23A91A6105
DATE:
EXP NO. PAGE NO :
grid_search = GridSearchCV(estimator=dt_model, param_grid=param_grid, cv=5, n_jobs=-1)
grid_search.fit(X_train, y_train)
print(f"Best Parameters: {grid_search.best_params_}")
print(f"Best Cross-Validation Score: {grid_search.best_score_ * 100:.2f}%")
best_dt_model = grid_search.best_estimator_
y_pred_tuned = best_dt_model.predict(X_test)
accuracy_tuned = accuracy_score(y_test, y_pred_tuned)
conf_matrix_tuned = confusion_matrix(y_test, y_pred_tuned)
class_report_tuned = classification_report(y_test, y_pred_tuned)
print(f"\nTuned Model Accuracy: {accuracy_tuned * 100:.2f}%")
print("\nTuned Confusion Matrix:")
print(conf_matrix_tuned)
print("\nTuned Classification Report:")
print(class_report_tuned)
OUTPUT:
Initial Accuracy: 96.67%
Confusion Matrix:
[[10 0 0]
[ 0 9 2]
[ 0 0 9]]
Classification Report:
precision recall f1-score support
0 1.00 1.00 1.00 10
1 1.00 0.82 0.90 11
2 0.82 1.00 0.90 9
accuracy 0.97 30
macro avg 0.94 0.94 0.93 30
weighted avg 0.95 0.97 0.95 30
Best Parameters: {'criterion': 'gini', 'max_depth': 5, 'min_samples_leaf': 1, 'min_samples_split': 5}
Best Cross-Validation Score: 96.67%
Tuned Model Accuracy: 100.00%
Tuned Confusion Matrix:
[[10 0 0]
[ 0 11 0]
[ 0 0 9]]
Tuned Classification Report:
precision recall f1-score support
0 1.00 1.00 1.00 10
ADITYA UNIVERSITY [Link]. 23A91A6105
DATE:
EXP NO. PAGE NO :
1 1.00 1.00 1.00 11
2 1.00 1.00 1.00 9
accuracy 1.00 30
macro avg 1.00 1.00 1.00 30
weighted avg 1.00 1.00 1.00 30
ADITYA UNIVERSITY [Link]. 23A91A6105
