A Quick Guide to Deep Learning

Add bookmark

AIIA Editorial Team
06/03/2019

What is Deep Learning?

Deep learning is an approach to machine learning (ML), which falls under artificial intelligence (AI), which is most commonly used to label complex data. Machine learning can use neural networks, a computer system modeled after the human brain, to processes information. It is an algorithm designed to recognize patterns, calculate the probability of a certain outcome occurring, and “learn” through error and successes using a feedback loop.

The algorithms that are utilized in deep learning are best suited for cases where the outputs will always be the same. For example, deep learning algorithms can be trained to classify a cat at a high rate of success, because a cat is a static output. The strength in deep learning lies in its ability to extract features from data to classify. Approaching predictive analytics with artificial neural networks is less successful due in part to the changing seas of prediction. Predictive analytics takes historical data and garners insight from that data to predict future outcomes. This is why deep learning is best deployed in computer vision, speech recognition, medical diagnostics, and other categorization applications.

The concept of deep learning—albeit not the term, which came later—originated in the 1950s when artificial neural networks took shape. These algorithms were simple and populated by hand. As computing power grew, the field of big data began to feed these deep learning algorithms with larger and larger datasets, leading to faster and more accurate results.

Cloud Computing

The immense amount of data, or big data that goes into deep learning has outgrown personal computer or even local server capacities. Cloud computing remotely connects multiple servers. Advancements in artificial intelligence have stalled several times in history. These periods are known as AI Winter. Lack of processing power was one of the main causes of AI Winters. The power of cloud computing helped kickstart the next round of artificial intelligence advancements. Some predict a long AI Spring.

Google’s Yuval Dvir explores the opportunities that machine learning and cloud services are enabling.

Source: AIIA.net: AI and Intelligent Automation

What is an Artificial Neural Network (ANN)?

Deep learning relies on artificial neural networks (ANN) to receive, process, and deliver data. An artificial neural network is a framework modeled after the human brain. This framework consists of inputs, outputs, and hidden layers. Artificial neural networks have artificial neurons, or nodes, that weigh input data and categorize aspects of that data, connect to other nodes, and feed it to the next hidden layer until an output is achieved.

Input Layer - The input node in an artificial neural network model is simply the information provided to the network. If these inputs are labeled, the learning model is considered supervised. If the inputs are unlabeled, the machine learning algorithm must categorize through pattern analysis, which is called unsupervised learning.

Preprocessing input data is a critical step not to be undervalued. When training a deep network, accuracy is sacrificed if the input data isn’t first processed. If an image of a pig is fed into the input layer when training the algorithm to distinguish between cats and dogs, the neural network becomes polluted with unexpected data in the supervised learning model. Similarly, if hairless cats aren’t accounted for in the input data, the cat classification algorithm is missing an input that could help it correctly classify hairless cats in the future.

Hidden Layers - It is in the hidden layers that all the “thinking” happens. Hidden layers can be complex, plentiful, and—thanks to modern-day computing power—are able to process high amounts of data. Each connection between nodes in a hidden layer garners a numbered weight. The heavier the weight, the stronger the connection to the next node. For example, if an ANN is classifying cats, an input node with a curly tail garners a lower weight than an input node with a long, fuzzy tail.

Output Layer - The output layer in an ANN is the conclusion of the data. The algorithm has now taken the input data, weighed and dispersed it to nodes throughout the hidden layers, and disseminated the information into output nodes.

There are several different types of artificial neural networks, each with specific use cases. Additionally, ANNs can work on top of one another and feed into one another, correcting mistakes and reweighing outputs on a large scale and to a granular level.

Feedforward Neural Network

The feedforward neural network is the simplest neural network where data only flows through connected nodes in one direction. There is no looping, dead ends, or backflow. The feedforward neural network is the foundation for other neural network frameworks. Due to their simplistic nature, FNNs work fine for a general classification of datasets. While the science behind FNN is still applicable, with current computing and data analytics power, layering them with other neural networks or adding additional features into the algorithm is now commonplace.

Convolutional Neural Network

Just as neural networks are loosely modeled after the brain, convolutional neural networks are inspired by the visual cortex. A CNN sees images as RGB pixels represented as numbers, and processes width, height, and depth. Hidden layers examine features and create a feature map which is then passed on to the next hidden layer. These hidden layers eventually merge with their newfound knowledge to create an output, or conclusion.

CNNs require large amounts of data in order to avoid overfitting. Overfitting is where the deep learning model produces outputs too similar to the inputs. The goal is to provide CNNs with enough data to allow it to uncover, recognize, and apply general patterns. For example, if just a few cat and dog images are fed into a CNN, overfitting can result in a narrow output of cat versus dog. With enough data, a CNN can discriminate between a cat and a dog based on unique and highly specific features.

Convolutional neural networks are responsible for image recognition, video analysis, and even drug discovery through the creation of 3D molecule and treatment models. Where humans have difficulty with granular classifications, such as dog breeds, CNN excels. However, CNN struggles to process small or thin images in a way humans don’t.

Recurrent Neural Network

As its name implies, RNNs are able to take output data and loops it back into the ANN as input data. Because of its memory, RNNs are able to see the bigger picture and make data generalizations in a more predictable way.

Long short-term memory networks (LSTM) is a type of RNN whose memory is efficiently directed and looped throughout the hidden layers. Simply put, a LSTM opens and closes gates between nodes to allow relevant information from previous nodes in and filter out unnecessary noise, allowing a RNN to extend its memory for longer periods of time.

Language modeling and prediction is a use case of RNN. The input is usually a sequence of words, and the output is a predicted sequence of words that matches the style of the input. For example, this Shakespeare text was actually written by an RNN after being fed large sets of Shakespeare works as its input.

Speech recognition is another use of RNN when audio is used as the input. Just like the human brain learns and memorizes certain rules like the letter q is nearly always followed with the letter u, RNN working with speech recognition can learn syntax and use this knowledge to recognize and predict speech.

Ted Graham, head of open innovation at General Motors, talks about transparency within Deep Learning

Source: The AI Network Podcast

Activation Functions

In order to process information quickly and accurately, the artificial neural network needs to decide if a particular feature of its input is relevant or irrelevant. Activation functions work as on/off switches for a particular node, deciding whether or not to weigh it and connect it to another node or to ignore the node completely. Continuing with the cat/dog example, an activation function turns off the node that categorizes inputs as “four legged.” Since both cats and dogs have four legs, continuing to progress that node through the ANN layers dilutes the processing power of the ANN.

Backprop Algorithm

To account for errors discovered within an ANN, the backward propagation of errors, or backprop algorithm was born. This is a supervised learning model where incorrect outputs are corrected and pushed back into the previous layer. For example, if an ANN weighed a dog tail too heavily when working to classify cats, the model supervisor adjusts the weight accordingly until the correct output is achieved.

The Future of Deep learning

Deep learning is evolving exponentially. Smart healthcare, education, smart cities, IoT and autonomous vehicles are all fields where the application of Deep learning will potentially change these fields as we know it.

The Internet of Things (IoT) connects the world around us to the internet through sensors, communication devices, and the cloud. Autonomous cars, medical innovations like ingestible sensors, and advanced robotics are slowly becoming a reality in part due to Deep learning. If artificial neural networks are modeled after the human neural network, it can be said that sensor technology will become the eyes and ears that feed inputs to the artificial neural networks.

Conclusion

Deep learning is highly complex both conceptually and mathematically. Technological advancements and software as a service (SaaS) has opened up the world of Deep learning to businesses and laypeople alike. As computing power and cloud computing advances, the only limit to the power of Deep learning and artificial intelligence in general will be the imagination of the human mind.