GAMSPy and Machine Learning

GAMSPy heralds a new era of possibilities, bridging the gap between machine learning and optimization that was previously difficult to cross using GAMS alone. Here’s why GAMSPy is the ultimate choice:

• Easy to understand, easy to write

  • eq1[...] = y == tanh(a @ W + b)

• Versatility in Solver Selection

  • regression.solve(solver="your_favourite_solver")

• It provides a robust algebraic language that allows you to experiment with how a neural network is implemented.

• Built-in flexibility:

  • You are not limited to inference; you can also train your neural network.

  • You can build the architecture from scratch using GAMSPy

• Development speed of Python combined with model generation speed of the GAMS execution engine

  • Equations and variables are generated in GAMS, not in Python, giving GAMSPy a speed advantage.

We are continually developing our ML-related features. If you have specific needs or require additional information, please use our Discourse platform.

• Matrix Operations
• NN Formulations (ReLU, Conv2d…)
• Classic ML Formulations
• Embedding a Trained Neural Network in GAMSPy

Complexity Overview

This section gives an overview of the “friendly” model types used in the formulations. For instance, if you only work with Linear, the problem remains linear and can be solved as an LP. On the other hand, if you combine it with relu_with_binary_var, the formulation becomes a MIP.

Visit NN Formulations for more information.

NN Formulation	Type
Linear
Conv1d
Conv2d
MaxPool2d
MinPool2d
AvgPool2d
relu_with_binary_var
relu_with_complementarity_var	(non-convex)
relu_with_sos1_var
leaky_relu_with_binary_var
softmax
softplus
log_softmax
tanh
sigmoid
gelu
TorchSequential	Depends[*]
RNN	Depends on activation
GRU

[*]

The complexity of a TorchSequential model depends both on the underlying sequential structure and on how it is embedded.

While types are shown with color coding, these colors (apart from the LP case) do not directly reflect runtime. In practice, there are situations where the complementarity formulation for ReLU can be faster than the binary formulation, and vice versa.

Visit ML Formulations for more information.

ML Formulation	Type
RegressionTree
RandomForest
GradientBoosting