Building out your ML Ops
Two weeks ago you got the first part of this talk from Ai4 delivered to your inbox as part of the Lindahl Letter. You are now getting the other half of my talk from the Ai4 2021 Retail, Supply Chain, & Marketing Summit, which occurred from April 7-8, 2021. My topic was, “The ML scale problem: Thinking about where and when to use ML, ROI models, synthetic data, repeatable frameworks, and teams.” You can get directly to the video of my talk via this link to YouTube:
Some of the major players within the information technology space are trying to break into the machine learning operations (or MLOps) space. Like anything else, picking the right tools to get things done is about matching the right technology and use case to achieve the best possible results.[1] A lot of reports and other building blocks of commentary and suggestion are digging into MLOps frameworks.[2]
We are really starting to see some solid maturity in the MLOps space.[3] The next stage will be either a round of purchasing where established players buy up the upstart players building MLOps or the established players will build out the necessary elements to move past the newer players in the enterprise level market.[4]
Let's dig in here and start to get into the MLOps part of the equation. So, I'll tell you a pro tip real quick. If you've never done this you might want to consider going out to GitHub and just look at GitHub slash trending: https://github.com/trending. Now if you've never done that and you've never looked and to see exactly what people were working on right now. It's a powerful way for you to just think about what's important to people working in the now. When I was writing this the number one thing that was trending was a project related to https://github.com/dogecoin/dogecoin which is not all that surprising thanks to a Tweet from Elon Musk this week.
So anyway, I grabbed this GitHub repository information for a few of the key open source machine learning projects I'm tracking for comparison. Within the next table, I'm going to dig into a little bit deeper on some of the ML projects that I think are very valuable.
So, let's look at the first one here TensorFlow. So TensorFlow has 154,162 stars. People have really used it. It has 2,933 contributors that means that almost 3,000 people are contributing to TensorFlow. From that point you can see that to get to PyTorch, you can see it drops off considerably. It's going to go from around 154k stars to just 47k stars.
And the number of contributors drops off significantly as well. Now, you're down to around 1,785. Now on the PyTorch example, they do have 4,620 branches which honestly I don't know why you would want to look at that many branches. No human wants to manage 4,000 branches of anything. That is unmanageable in terms of iteration.
So I definitely understand more on the TensorFlow side of things, but that does not mean that scikit-learn which has roughly 44,000 stars and has 1,936 contributors is not interesting. So you can kind of see here that the three major projects that are out there for machine learning are definitely adopted. People are using them and they're making forks of it, they're making versions of it, and they're starting to really dig into it out in the wild of software development right now.
So now if we take it to the next level and look a little deeper in terms of what's happening with the MLOps part of it. You're gonna see a major drop-off. Remember TensorFlow had 154,162 stars. Now you're starting to see the number of stars drop off considerably. You're starting to see that number of stars at 10,000 or less. You are starting to see kubeflow, mlflow, and some of these things that you know are complex stuff like metaflow from Netflix and you're only gonna see 4,000 stars and each of these things is gonna have sub 500 contributors.
So based on the order of magnitude difference between TensorFlow and some of these things one of the things that you can directly tell from this information is that none of these tools for MLOps really grabbed significant market share. None of them have become the key product yet.
And so, we haven't seen everyone swarm in and start using these things. One of the reasons that this is the case is on that previous chart when we saw the different things for an API and you start thinking about the different ecosystems that are out there the pipelines and the ecosystem for something in a bucket to where you can be.
You're going to start using the ecosystem stuff that's available. So if you start going into bucket 2 where you're on Azure you're on GCP and you're making those. Decisions have to be made. If you are on AWS you're gonna work within the stack for machine learning that's available for that ecosystem. That's why you chose to go there.
That's one of the reasons why that is gonna be the place that you're gonna be at. So. I think it's really interesting here and you can see that none of these are branching out as many branches as the previous previous look at it So this space here if we take these items and we break them out into a chart that is sort of similar to the previous chart.
You're gonna see that I think there's a wide spectrum here. Now, you may not totally agree with how I place these things versus scale and maturity and that's okay. You know, feel free to let me know if you have a different opinion on one of these technologies. You know depending on where you're at with your ML use case, you're gonna have some very different machine learning operations.
The major players are gonna be predominant in that space. And so that's where I think you're starting to see the mix of other things after it that are really starting to take off and they do have vibrant ecosystems and communities Weights and biases for example, people are really interested in that and it has a vibrant community around it and some of these other items definitely definitely have vibrant communities and they're building out.
And what I'll be looking for here and probably in the next year or so is to see which ones really bubble up to being super successful for that bucket 3 instance where people are building stuff out and they need MLOps tools to be able to successfully operationalize their strategy. Because there's going to be so much stuff being done that's not done out in an ecosystem.
It's going to be the stuff that's done... not the back of the envelope type stuff. It's going to be the stuff that you're doing and building within the machine learning space to do that next level of thing. And that next level thing isn't going to happen from the APIs because it's not going to be a pre-trained model that you're buying to be able to do these things.
People are going to want to do things that are complex and are going to be directly tied to what they're delivering from their business. Those business use cases and those very tailored specific use cases for delivery in your machine learning workflow are going to require so much more than what that API can provide.
And because of that and that delivery you're going to start seeing these MLOps companies and GitHub projects and things really take off and I'm looking in the next year or so to see exactly which ones gain the predominant market share.
What do we cover?
Well, we covered initiating what exactly is scale, right? We talked about bucket one, we talked about bucket two and we talked about bucket three which may have lost meaning because those slides were covered a while ago bucket one is the APIs. It is the thing that you can go out and just get a hold of right now, but you can connect to it and start using it within your workflow.
For bucket two those things in the ecosystems where we are going to build out your machine learning in that ecosystem, use their tools, and bucket three is gonna be a look at exactly what can happen for you when you're dealing with things that are built on premise when you're using those open source tools when you're building in your own environment.
Now, I want to close here with my key takeaways for you. My key takeaways are as always to have an ML strategy. Make sure that you really know that story that you're gonna communicate understand your pipelines and what you're building out to deliver that email strategy because if you don't consider the pipelines, if you don't think about the MLOps, if you don't look at those things that we were pulling up on those slides, you're not going to be ready to deliver.
And if you're not ready to deliver you may miss your opportunity because when people commit funding they're going to expect results. And if you can't communicate back with a story, it's based around your return on investment, you're not gonna be successful in the long term. And that means really figuring out your MLOps.
Links and thoughts:
I really enjoyed the idea and vision of this GCP Cloud Tech video “Reclaim your time with AppSheet Automation”
You can check out Dr. Jure Leskovec in this lecture “CS224W: Machine Learning with Graphs | Winter 2021 | Lecture 1.1 - Why Graphs”
This video from PCWorld’s Alaina was pretty darn good this week, “What Type of SSD Should You Buy?”
Check out Machine Learning Street Talk this week… they had Francois Chollet on the show this week…
Top 5 Tweets of the week:
Footnotes:
[1] This is interesting to read https://www.gartner.com/reviews/market/data-science-machine-learning-platforms
What’s next for The Lindahl Letter?
Week 14: My Ai4 Healthcare NYC 2019 talk revisited
Week 15: What are people really doing with machine learning?
Week 16: Ongoing ML cloud costs
Week 17: Figuring out ML readiness
Week 18: Could ML predict the lottery?
I’ll try to keep the what’s next list forward looking with at least five weeks of posts in planning or review. If you enjoyed reading this content, then please take a moment and share it with a friend.