Is it possible to set initial parameter value for logistic regression online updates?

[cliu-sift]

I want to build a online logistic regression, and we know that logistic regression is trying to find the optimal \beta by:
logit(p) = X\beta
I'm asking if it is possible to specify the initial value of \beta in rivers model config below
model = compose.Pipeline( preprocessing.StandardScaler(), linear_model.LogisticRegression(intercept_init = 0.0, initializer = init))

Thank you!

[gbolmier]

Hey @cliu-sift 👋

You can do so by providing an initializer object to your model at instantiation time, see the initializers API reference in the optim module.

from river import linear_model
from river import optim

init = optim.initializers.Normal(mu=0, sigma=1, seed=42)
model = linear_model.LogisticRegression(initializer=init)

Hope it answers your question

[cliu-sift]

But it seems that the initializer linear_model.LogisticRegression(initializer=init) only supports for scaler. I tried to input with a vector, but it throws an error:

[MaxHalford]

@cliu-sift what code did you run? I suspect that you passed a 1D numpy.ndarray to the initializer parameter, which is not what @gbolmier did in his example.

To be clear, what's your goal? Is it to select the starting value of each coefficient? Why would you want to do that?

[cliu-sift]

Yeah, I want to set the starting value of each coefficient. As I have two parts of data. The first part can be ran from traditional LR which gives us the global optimum of the coefficient, and I want to use this coefficient to be the initial coefficient for the send part of data which I will adopt online learning as I guess this will be close to the global optimum of the loss function for the second part two? But I found out that in the initializer of linear_model.LogisticRegression(initializer=init), the init seems to only accept scalar. I don't have way to pass a vector to make it runnable. Below is an example. No matter what type of init I use, it will be a scalar:

from sklearn import datasets, metrics
from river import stream
from river import linear_model
from river import optim
from river import preprocessing

scaler = preprocessing.StandardScaler()
optimizer = optim.SGD(lr=0.01)
init = river.optim.initializers.Constant(2)
init = river.optim.initializers.Normal(2)
log_reg = linear_model.LogisticRegression(optimizer, initializer=init)
y_true = []
y_pred = []
for xi, yi in stream.iter_sklearn_dataset(datasets.load_breast_cancer(), shuffle=True, seed=42):
    # Scale the features
    xi_scaled = scaler.learn_one(xi).transform_one(xi)

    # Test the current model on the new "unobserved" sample
    yi_pred = log_reg.predict_proba_one(xi_scaled)
    # Train the model with the new sample
    log_reg.learn_one(xi_scaled, yi)

    # Store the truth and the prediction
    y_true.append(yi)
    y_pred.append(yi_pred[True])

print(f'ROC AUC: {metrics.roc_auc_score(y_true, y_pred):.3f}')

[MaxHalford]

Ok I understand better now. I don't think @gbolmier understand what you meant when he answered 😅

This should work:

from river import linear_model
from river.utils import VectorDict

lr = linear_model.LogisticRegression()
lr._weights = VectorDict({
    'foo': 1,
    'bar': 2
})

For your usecase, you should be able to do something like this:

from river import linear_model
from river.utils import VectorDict

lr1 = linear_model.LogisticRegression()
# train lr1 ...

lr2 = linear_model.LogisticRegression()
lr2._weights = VectorDict(lr1.weights)

(Meta: use ````py` to highlight your Python code, it's easier to read, I did it for you)

[cliu-sift]

Wow, it works, Thank you!

Really appreciate your help!