What is scientific machine learning?
Well you might have guessed by now that scientific machine learning is a thing. Sometimes it is even abbreviated as SciML [1]. Purdue University instructors even have an edX course you can sign up for and take, “Introduction to Scientific Machine Learning”[2]. Imagine blending scientific computing (algorithmic solving) and machine learning together and you get scientific machine learning. Scientific computing tends to involve using algorithmic models for computational optimization or differential equations to work toward solving typically complex problems. You can spend a lot of time reading about the various problems people are using scientific computing to solve. A quick Google Scholar search returned more than a thousand articles with the phrase, “scientific machine learning,” within the text somewhere [3]. One preprint paper on arXiv with more than 200 citations stood out titled, “Universal Differential Equations for Scientific Machine Learning”[4]. It was a major investment of my time and energy to read all 55 pages. Things got really deep pretty quickly as the paper digs into the SciML software ecosystem. This is not an area of machine learning where I have done anything related to this discrete research space.
The really interesting coding element here is that a lot of this type of SciML development seems to be happening using the Julia programming language. That is not a language that I have ever really worked with for any coding project. Not only does it seem like there is a community that is really excited about SciML, but also it appears that the community is excited about coding in the Julia language. They have 7 core components prepared over at https://sciml.ai/ which is written in Julia and noted to be the home of, “SciML Scientific Machine Learning Software”[5]. Those scientific machine learning components on the website include: 1) High Performance and Feature-Filled Differential Equation Solving, 2) Physics-Informed Model Discovery and Learning, 3) A Polyglot Userbase, 4) Compiler-Assisted Model Analysis and Sparsity Acceleration, 5) ML-Assisted Tooling for Model Acceleration, 6) Differentiable Scientific Data Structures and Simulators, and 7) Tools for Accelerated Algorithm Development and Research. My initial observations is that the scientific machine learning community seems to communicate in terms of sharing algorithms more than just sharing trained models and focusing on very specific methods of model development. That is interesting in general and it very quickly drives the reader to having to focus on reading the language of mathematics instead of descriptive prose.
Links and thoughts:
Machine Learning Street Talk was back this week with “061: Interpolation, Extrapolation and Linearisation (Prof. Yann LeCun, Dr. Randall Balestriero)”
Check out this video from The Verge showing the “Best of CES 2022”
“Why so shy Nvidia? - WAN Show January 7, 2022”
“AI Show Live - Episode 46 - Updates to Semantic Search and Speech and more from Ayşegül and Bea”
Top 5 Tweets of the week:
Footnotes:
[1] Scientific Machine Learning (SciML) Projects. julia. (n.d.). Retrieved January 8, 2022, from https://julialang.org/jsoc/gsoc/sciml/
[2] Purdue University. (n.d.). Introduction to Scientific Machine Learning. edX. Retrieved January 8, 2022, from https://www.edx.org/course/introduction-to-machine-learning-2
[3] Google. (n.d.). Scientific Machine Learning. Google Scholar. Retrieved January 8, 2022, from https://scholar.google.com/scholar?q=%22scientific+machine+learning%22&hl=en&as_sdt=0&as_vis=1&oi=scholart
[4] Rackauckas, C., Ma, Y., Martensen, J., Warner, C., Zubov, K.V., Supekar, R.B., Skinner, D.J., & Ramadhan, A. (2020). Universal Differential Equations for Scientific Machine Learning. ArXiv, abs/2001.04385. Retrieved January 8, 2022, from https://arxiv.org/abs/2001.04385
[5] SciML Scientific Machine Learning Software. SciML.ai. (n.d.). Retrieved January 8, 2022, from https://sciml.ai/
What’s next for The Lindahl Letter?
Week 52: That one with a machine learning post
Week 53: Machine learning interview questions
Week 54: What is a Chief AI Officer (CAIO)?
Week 55: Who is acquiring machine learning patents?
Week 56: Comparative analysis of national AI strategies
I’ll try to keep the what’s next list forward looking with at least five weeks of posts in planning or review. If you enjoyed reading this content, then please take a moment and share it with a friend.