Partial fitting incrementally trains an estimator on one batch at a time. Repeated calls to partial_fit let a model learn from datasets that are too large to fit in memory, while updating coefficients after each batch. section 4.3.5
A practical workflow includes:
- Provide target labels during the initial setup so the algorithm knows the classes. section 4.3.5
- Shuffle and process batches repeatedly across iterations or epochs to consolidate learning. section 4.3.5
- Balance additional training against overfitting, since repeatedly showing the same examples can reduce generalization. section 4.3.5
Listing 4.10 Out-of-core logistic regression with L2 regularization
from sklearn.linear_model import SGDClassifier
from sklearn.model_selection import train_test_split
from sklearn.utils import gen_batches
from sklearn.metrics import accuracy_score
def generate_batches(X,
y,
batch_size,
random_state): #1
"""split dataset into batches """
examples = len(X)
batches = gen_batches(n=examples,
batch_size=batch_size) #2
sequence = np.arange(examples)
if random_state:
np.random.seed(random_state) #3
np.random.shuffle(sequence)
for batch in batches:
items = sequence[batch]
yield(X.iloc[items], y.iloc[items]) #4
model = SGDClassifier(loss="log_loss",
average=True,
penalty='l2',
alpha=0.001) #5
column_transform = ColumnTransformer(
[('categories', categorical_onehot_encoding, low_card_categorical),
('numeric', numeric_standardization, continuous)],
remainder='drop',
verbose_feature_names_out=False,
sparse_threshold=0.0)
X_train, X_test, y_train, y_test = train_test_split(data, target_median,
test_size=0.20,
random_state=0) #6
iterations = 10
for j in range(iterations): #7
generator = generate_batches(X_train, y_train, batch_size=256,
random_state=j)
for k, (Xt, yt) in enumerate(generator):
if k == 0:
column_transform.fit(Xt)
Xt = column_transform.transform(Xt)
if k == 0:
model.partial_fit(Xt, yt, classes=(0, 1)) #8
else:
model.partial_fit(Xt, yt) #9
predictions = model.predict(column_transform.transform(X_test))
score = accuracy_score(y_true=y_test, y_pred=predictions)
print(f"Accuracy on test set: {score:0.3f}") #10
Use the out-of-core logistic-regression listing as a runnable pattern for generating shuffled batches and training an SGDClassifier incrementally. section 4.3.5
