AI is used mostly for reducing routine human work

Real-world applications of machine learning models are dynamic. The model is being used in a constantly changing environment where the data and therefore performance of the model is ever changing.

A real-world machine learning model can become a cycle of annotation and retraining through a human-machine loop. Instead of building a model with the best testing performance we should focus on how much human effort can be saved, according to an article from Berkley AI research.

The aim should be in minimising human annotation tasks to therefore improve the efficiency of the model. This is like active learning.

"Low-confidence predictions are sent for human annotation, and high-confidence predictions are trusted for downstream tasks or pseudo-labels for model updates."

I often see this approach in real-world applications. For example, in predictive medicine, a model can triage patients so that clinicians only review those that have a high likelihood of being at risk, saving human effort. Any low confidence patients can be manually evaluated and used to train the model, improving performance in the future.

I think most application of AI will be like this, an augmentation of human intelligence that takes over some of the routine tasks while leaving more complex decisions to humans. The mutual augmentation improves both the quality and the efficiency of the AI model and the result.