Machine Learning Reality

The Reality of Machine Learning in Production

Everyone wants to use machine learning until they realize models are just very confident random number generators with training. At 21CT, we built a fraud detection model that was 99% accurate in testing. In production, it flagged every transaction on Tuesdays as fraudulent. Turns out our training data didn’t have any Tuesday transactions because of how we’d split the dataset. The model had learned that Tuesday = fraud, which is technically correct if you’ve never seen a legitimate Tuesday transaction.

The worst part about ML in production is model drift - your model slowly becomes wrong as the world changes, like a map that doesn’t know about new roads. Our fraud detection model was trained on 2012 data and by 2014 was flagging all mobile transactions as suspicious because smartphones barely existed in our training set. It was like having a security guard who’d never seen a cell phone and thought everyone texting was planning a heist.

My favorite ML bug was when our model achieved 100% accuracy by memorizing the training data, including the random IDs we’d assigned to each row. It would look at a transaction, recognize it had never seen that specific ID before, and correctly identify it as new. We’d invented a very expensive way to check if a number exists in a list. The model was perfectly accurate and completely useless, like a weather forecast that’s always right because it only predicts weather that already happened.