Binary Classification in Machine Learning as Suggested by AI-Assistants

An example of using scikit-learn with artificial data

5 min readNov 17, 2023

It is assumed that you have already installed Python, some packages (Matplotlib, NumPy, SciPy), pandas, and scikit-learn.

Artificial data

In order to assure about characteristics of the data, here we will create our own artificial data. A spreadsheet, Microsoft Excel, is used as follow.

The formula for cell C2 is =IF((A2+B2)>=10, 1, 0) which has following appearance as the formula implemented relatively form c2 to c122 cells.

There are two classes of data, where the first is labeled with 0 and the second is labeled with 1.

Read test

To see the data following lines of code

import pandas as pd

data = pd.read_excel("data_xyz.xlsx")
print(data)

will produce

$ py ml_read_test.py
      x   y  z
0     0   0  0
1     1   0  0
2     2   0  0
3     3   0  0
4     4   0  0
..   ..  .. ..
116   6  10  1
117   7  10  1
118   8  10  1
119   9  10  1
120  10  10  1

[121 rows x 7 columns]

as a DataFrame. The spreadsheet data_xyz.xlsx can be found here.

Code from AI-assistants

Let us get help from two AI-assistant, which are Bing Chat and ChatGPT.

A Bing code is as follow

# import modules
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

# load data
data = pd.read_excel("data_xyz.xlsx")

# split data into features and labels
X = data.iloc[:,0:2]
y = data.iloc[:,2]

# split data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)

# scale features
scaler = StandardScaler()
X_train = scaler.fit_transform(X_train)
X_test = scaler.transform(X_test)

# create and fit classifier
clf = LogisticRegression()
clf.fit(X_train, y_train)

# evaluate classifier
accuracy = clf.score(X_test, y_test)
print("Accuracy:", accuracy)

which gives

$ py ml_class_bin_bing.py
Accuracy: 1.0

as output.

But the suggestion from ChatGPT as in here


import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix

# Load the dataset from the XLSX file
xlsx_path = 'data_xyz.xlsx'  # Replace with the actual path to your XLSX file
df = pd.read_excel(xlsx_path, header=None, names=['Feature1', 'Feature2', 'Label'])

# Split the data into features (X) and labels (y)
X = df[['Feature1', 'Feature2']]
y = df['Label']

# Split the data into training and testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Standardize the features (optional but often recommended)
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

# Initialize the Logistic Regression model
model = LogisticRegression()

# Train the model
model.fit(X_train_scaled, y_train)

# Make predictions on the test set
y_pred = model.predict(X_test_scaled)

# Evaluate the model
accuracy = accuracy_score(y_test, y_pred)
conf_matrix = confusion_matrix(y_test, y_pred)
classification_rep = classification_report(y_test, y_pred)

# Print the evaluation metrics
print(f'Accuracy: {accuracy:.2f}')
print(f'Confusion Matrix:\n{conf_matrix}')
print(f'Classification Report:\n{classification_rep}')

does not work, with

$ py ml_class_bin_chatgpt.py
Traceback (most recent call last):
  File "D:\python\src\import\external\sklearn\ml_class_bin_chatgpt.py", line 21, in <module>
    X_test_scaled = scaler.transform(X_test)
                    ^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\62812\AppData\Roaming\Python\Python311\site-packages\sklearn\utils\_set_output.py", line 157, in wrapped
    data_to_wrap = f(self, X, *args, **kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\62812\AppData\Roaming\Python\Python311\site-packages\sklearn\preprocessing\_data.py", line 1006, in transform
    X = self._validate_data(
        ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\62812\AppData\Roaming\Python\Python311\site-packages\sklearn\base.py", line 605, in _validate_data
    out = check_array(X, input_name="X", **check_params)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\62812\AppData\Roaming\Python\Python311\site-packages\sklearn\utils\validation.py", line 915, in check_array
    array = _asarray_with_order(array, order=order, dtype=dtype, xp=xp)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\62812\AppData\Roaming\Python\Python311\site-packages\sklearn\utils\_array_api.py", line 380, in _asarray_with_order
    array = numpy.asarray(array, order=order, dtype=dtype)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Program Files\Python311\Lib\site-packages\pandas\core\generic.py", line 2070, in __array__
    return np.asarray(self._values, dtype=dtype)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: could not convert string to float: 'x'

as output.

Challenges

Do you know what are the differences between the helps from the AI-assistants?
Can you find the mistakes and the fix the not-working example?
Put in the comment of this story if you like to accept the challenges.

So, in this story the artificial data has been generated and can be used in binary classification using scikit-learn with help of two famous AI-assistants, where one help works but the other does not.

Update

Using command line argument supported by Python (nikhilaggarwal3, 2022) and sys.exit() function to stop the program (Olumide, 2023) two previous codes are modified as follow

import sys

# check number of arguments
n = len(sys.argv)
if n < 2:
  print("Usage: py ml_class_bin_mix.py [n]")
  print()
  print("n\t 0 for Bing Chat")
  print("\t 1 for Chat GPT 3.5")
  print()
  sys.exit(0)
else:
  # 0 = Bing Chat, 1 = ChatGPT 3.5
  AI = int(sys.argv[1])


# show choice
if AI == 0:
  print("Using code from Bing Chat")
if AI == 1:
  print("Using code from Chat GPT 3.5")


# import modules
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LogisticRegression

if AI == 1:
  from sklearn.metrics \
  import accuracy_score, classification_report, confusion_matrix


# read data from file
if AI == 0:
  # load data
  data = pd.read_excel("data_xyz.xlsx")

if AI == 1:
  # Load the dataset from the XLSX file
  # Replace with the actual path to your XLSX file
  xlsx_path = 'data_xyz.xlsx'
  df = pd.read_excel(
    xlsx_path,
    #header=None,
    names=['Feature1', 'Feature2', 'Label']
  )


# get features and labels
if AI == 0:
  # split data into features and labels
  X = data.iloc[:,0:2]
  y = data.iloc[:,2]

if AI == 1:
  # Split the data into features (X) and labels (y)
  X = df[['Feature1', 'Feature2']]
  y = df['Label']



# get training and testing sets
if AI == 0:
  # split data into training and testing sets
  pass
  
if AI == 1:
  # Split the data into training and testing sets
  pass
  
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)


# adjust features
if AI == 0:
  # scale features
  scaler = StandardScaler()
  X_train = scaler.fit_transform(X_train)
  X_test = scaler.transform(X_test)

if AI == 1:
  # Standardize the features (optional but often recommended)
  scaler = StandardScaler()
  X_train_scaled = scaler.fit_transform(X_train)
  X_test_scaled = scaler.transform(X_test)


# create model
if AI == 0:
  # create and fit classifier
  clf = LogisticRegression()
  clf.fit(X_train, y_train)

if AI == 1:
  # Initialize the Logistic Regression model
  model = LogisticRegression()

  # Train the model
  model.fit(X_train_scaled, y_train)

  # Make predictions on the test set
  y_pred = model.predict(X_test_scaled)


# evaluate model
if AI == 0:
  # evaluate classifier
  accuracy = clf.score(X_test, y_test)
  print("Accuracy:", accuracy)

if AI == 1:
  # Evaluate the model
  accuracy = accuracy_score(y_test, y_pred)
  conf_matrix = confusion_matrix(y_test, y_pred)
  classification_rep = classification_report(y_test, y_pred)

  # Print the evaluation metrics
  print(f'Accuracy: {accuracy:.2f}')
  print(f'Confusion Matrix:\n{conf_matrix}')
  print(f'Classification Report:\n{classification_rep}')

if AI == 0:
  print()

which gives output for from both AI-assistants

$ py ml_class_bin_mix.py
Usage: py ml_class_bin_mix.py [n]

n        0 for Bing Chat
         1 for Chat GPT 3.5

$ py ml_class_bin_mix.py 0
Using code from Bing Chat
Accuracy: 1.0

$ py ml_class_bin_mix.py 1
Using code from Chat GPT 3.5
Accuracy: 1.00
Confusion Matrix:
[[11  0]
 [ 0 14]]
Classification Report:
              precision    recall  f1-score   support

           0       1.00      1.00      1.00        11
           1       1.00      1.00      1.00        14

    accuracy                           1.00        25
   macro avg       1.00      1.00      1.00        25
weighted avg       1.00      1.00      1.00        25

Now both codes works.

When there is not any argument, it shows the usage,
when the argument is 0 it uses code from Bing Chat, and
when the argument is 1 it uses code from Chat GPT 3.5.

And for other error, it should not use header=None while reading Excel file.