Introduction
Machine learning is a subfield of artificial intelligence (AI) that enables computer systems to identify, or learn, patterns in data using statistical methods. These learned patterns are captured in a model that can be used to make predictions or perform tasks without explicit programming.
The training process is central to machine learning (ML). In order to produce a model, we train the system by applying a learning algorithm to a training dataset. Once a model is trained, it can then be used for inference (also known as scoring or prediction.)
For a simple example, consider the problem of predicting home values from available real estate data. We know intuitively that a relationship exists between the characteristics of a home such as its size, number of bedrooms and baths, and location (i.e., our features), and the prices the properties ultimately sell for (i.e., our labels). Yet without using machine learning, it would be very difficult to accurately capture this relationship as a rule or set of rules, especially as we increase the number of features.
This example helps illustrate the importance of machine learning not just as a statistical tool, but as a new way of developing software. This viewpoint has been captured succinctly as “Software 2.0” by Andrej Karpathy, Director of AI at Tesla. Software 2.0 refers to the idea that in traditional programming (i.e., “Software 1.0”), software developers start with some data they want to operate on and a set of explicit rules for processing that data. Those rules are manually encoded as software programs via the software development process. These programs are then used to solve the task at hand, whatever that might be.
NOTE: There are several types of machine learning. While the specific use of labeled training datasets is characteristic of supervised learning, as opposed to unsupervised or reinforcement learning, the first is the most popular type of machine learning in use today. While this Solution Guide occasionally refers to labels and other supervised learning concepts, its key messages apply broadly.
In contrast, machine learning (i.e., Software 2.0) allows us to extract the rules for solving some task automatically from a set of features and labels in a training dataset. This is particularly useful in situations for which it would be very difficult for humans to identify, much less program, the complete set of governing rules.
For such tasks, like identifying faces or objects in photos, distinguishing spam emails, or identifying fraudulent transactions, it has proven much more effective to use machine learning models than manually coded programs. With machine learning, we can essentially teach computers how to program themselves using data!
Modern ML is powered by algorithms that, for the most part, have been available for decades. Nonetheless, the field is experiencing dramatic growth due to a number of shifts in the industry. Recent changes include an explosion in the amount of available training data and affordable computing power, advances in how models are trained, and a boom in the amount and quality of tools for developing machine learning solutions.
have contributed to the modern ML revolution
A notable subfield of machine learning is deep learning. Deep learning (DL) today largely refers to the use of a specific type of machine learning model—one made up of neural networks with many layers. Deep neural networks (DNNs) can identify very complex patterns in their input data, making deep learning a powerful technique. Today, DNNs represent the state of the art in many applications involving voice, video, and pictures, and are increasingly finding use in natural language processing and tabular data. The unfortunate drawback of deep learning is that training these models can require substantially more labeled data, time, and computational horsepower than traditional ML models.
Industries spanning healthcare, publishing, finance, manufacturing, and more are increasingly turning to ML and DL to improve facets of their business. Potential applications include everything from decision-making, customer service, recommendations, and even building entirely new offerings like self-driving vehicles.
Many early ML adopters have found that as they grow their internal machine learning capacity, more (and more profitable) ways to apply the technology become apparent. For enterprises seeking to take full advantage of the transformational opportunities offered by machine learning, improving their ability to bring ML-based solutions to more areas of the business can be a worthwhile investment.
In our next article in this series, we will talk about Machine Learning Platforms and why they’re important.
