Within a few decades, AI/ML had just a few years prior promised to alter our planet. However, there are already complaints about how it is not living up to those promises. "My machine learning model fared so brilliantly in training, but it's a tremendous disappointment when put to production," is a common complaint.
First of all, it's just too soon to dismiss AI and ML. Second, as engineers and developers, we need to change how we work in order to realise its full potential.
Try Angel Sayani's three simple suggestions if your machine learning models are performing well in the lab but not on the assembly line.
1. Keep it loose-fitting
Implementing this strategy is the simplest of the three because are a lot of research papers & studies being carried out on the topic of overfitted models, and there are many methods available. In essence, an overfitted model is one that performs exceptionally well during training but poorly with real data after deployment, according to Angel. Although the causes are unique, the training data is either too small or too straightforward for the model to develop its ability to spot broad patterns.
You can employ one of the following methods to prevent overfitting when using a conventional machine learning algorithm:
- Regularization: which rewards simpler models while penalising complex ones
- Cross-validation: improves output verification by separating training and validation data
- Dropout (only when utilising neural networks): By purposefully turning off some neurons during training, it drives the remaining neurons to absorb more information
- Ensemble learning: Instead of depending on a single machine learning model, ensemble learning uses forecasts that are averages of several different unique prediction models
2. Accurately manage real-time data
Data processing has to deal with training-serving skew. According to Angel, "the training data that you use for your model is often supplied from batch pipelines."
The information is most likely stored in a file system or database somewhere in your company. In essence, you effectively process that data and utilise it to train the model. However, because it is streaming data, prediction data is frequently handled more haphazardly.
When using real-world prediction data, a model may not perform well due to this fundamental processing difference. You can make sure that batch and streaming data take the same path in order to minimise this processing disparity.
The processing of batch and streaming data can be integrated using a number of existing architectures. Lambda and Kappa architecture are a couple of instances of this. By including these design principles into your machine learning pipelines, you can be sure that the model will be taught how to properly handle real-time input.
3. After deployment, take a glimpse at your model
Any ML model seeks to represent relationships that exist in the real world, and these interactions are dynamic. Concept drift is a common problem with models. They evolve with time. Models must be continuously monitored and current data must be put into them or they become stale and lose their usefulness.
To keep learning and remaining relevant, machine learning models require human overseers. These people keep an eye on the model and update it as necessary with fresh data. Essentially, learning must continue to be as dynamic as the world the model is attempting to forecast.
When a model faces organised opposition, such in the frequent AI use-cases of fraud detection, fake news identification, and quantitative trading, human minders become even more crucial. Hackers frequently use these to trick or defraud the model. The output of these models is reviewed by human minders who then look for instances where the data was incorrectly categorised and utilise that information to retrain the model to remain relevant.
Remember, Angel cautions, "deploying your machine learning model is just the first step. There is still a ton of work to be done. Don't just go on to the next intriguing issue."