Evaluating machine learning
A lot of people are getting used to having to evaluate machine learning. You can even buy a book from O-Reilly dedicated to the subject.[1] Decisions are being made all the time. We are starting to see machine learning get built into applications and are actively starting to appear as use cases. For me it all goes back to evaluating the return on investment being modeled as an outcome of using machine learning. In that case you are evaluating the financial effect of using machine learning on a specific use case. Sometimes you can get stuck on modeling model performance and trying to understand the inside baseball of machine learning. Those elements are a vital part of managing and deploying machine learning in your organization, but they ultimately are simply processes and diagnostic measures and evaluation should focus on the outcome. All of that could very well be inputs within the evaluation of the return on investment being calculated for your machine learning use case.
Getting ready to build up a stack of stuff about evaluating machine learning to write this post was a solid reminder of just how much content exists related to this topic. People are selling grids, white papers, and all sorts of marketing materials. Pretty much every consulting agency has a take on how to evaluate machine learning models and usage within an enterprise. It is surprising how much effort they go to within the process to obfuscate the final and true question of what the return on the investment would be from implementation. Generally in the business world the time where machine learning was a research and development question has passed. Within the industry the technology has really moved from speculative to something that can be used and developed with reasonable confidence.
Expanding your search to really dig into and search for, “evaluating machine learning return on investment,” will change the results you are getting in surprising ways. The number of white papers, friendly guides, and marketing missives on the subject will sort of drop off and you can find a few more thoughts directed squarely at the subject in question. Adding the return on investment part to the search helps dial in results that include people really talking about what happens after the implementation and the modeling to figure out if the use case is having financially positive results for the organization. It really goes to answering questions about the outcomes of having invested in the deployment of machine learning within the organization.
Links and thoughts:
The Vergecast discuss the official reveal of Windows 11 at Microsoft's event on Thursday
https://megaphone.link/VMP9322407846
Decoder with Nilay Patel “Microsoft CEO Satya Nadella on the business of Windows”
https://megaphone.link/VMP3710346080
This was a pretty good video on Pub Sub from Google Cloud Tech “Choosing Pub Sub or Pub Sub Lite? - ep. 11”
Check out “[ML News] CVPR bans social media paper promotion | AI restores Rembrandt | GPU prices down” from Yannic Kilcher
Top 5 Tweets of the week:
Footnotes:
[1] https://www.oreilly.com/data/free/files/evaluating-machine-learning-models.pdf when something becomes mainstream you will see a book from O-Reilly on the subject. They pretty much celebrate the known within the technology world.
What’s next for The Lindahl Letter?
Week 25: Teaching kids ML
Week 26: Machine learning as a service
Week 27: The future of machine learning
Week 28: Machine learning certifications?
Week 29: Machine learning feature selection
I’ll try to keep the what’s next list forward looking with at least five weeks of posts in planning or review. If you enjoyed reading this content, then please take a moment and share it with a friend.