enerzyme.models.activation.ShiftedSoftplus#
- class enerzyme.models.activation.ShiftedSoftplus(dim_feature: int = 1, initial_alpha: float = 1.0, initial_beta: float = 1.0, learnable: bool = False)[source]#
Bases:
BaseScaledTemperedActivation- __init__(dim_feature: int = 1, initial_alpha: float = 1.0, initial_beta: float = 1.0, learnable: bool = False) None[source]#
Shifted softplus activation function with learnable feature-wise parameters: f(x) = alpha/beta * (softplus(beta*x) - log(2)) softplus(x) = log(exp(x) + 1) For beta -> 0 : f(x) -> 0.5*alpha*x For beta -> inf: f(x) -> max(0, alpha*x)
- Arguments:
- num_features (int):
Dimensions of feature space.
- initial_alpha (float):
Initial “scale” alpha of the softplus function.
- initial_beta (float):
Initial “temperature” beta of the softplus function.