Deep Learning Pipeline

how to use it. 

• TensorFlow Basics for Development 
• XOR Implementation Using TensorFlow 
• Linear Regression in TensorFlow 
• Summary

Part Two:  Data
Covers everything about data. From data collection to understanding intuition to data processing and preparation. 

 

Chapter 4:  Defining Data
This chapter as its name suggests is about defining data. Readers should know the type of data they are dealing with so they can choose the right approach for preparing that data. 

• Defining Data 
• Why should you read this chapter? 
• Structured, semi-structured, and unstructured data 
• Divide and Conquer 
• The types of data you will deal with 

o Tabular (Numerical and Categorical) 

• Quantitative versus Qualitative data 
•  The four levels of data 
• Nominal level 
• Ordinal level 
• Interval level 
• Ratio level 
• Example - Titanic 

o Text 

• Example - Classifying IMDB Movie Reviews o Images
• Type of images (2D, 3D, 4D) 
• Example - CIFAR-10 

·      Quick recap and check 

·      Summary 

 

Chapter 5:  Data Wrangling and Preprocessing
After understanding the data, readers now choose the approaches and methodologies for preparing it.

• The deep learning pipeline revisited 
• Data loading and preprocessing 

o Data Loading with Numpy 

o Data Loading with Pandas 

• Missing and Noisy Data 
• Dealing with big datasets 
• Accessing other data formats 
• Data preprocessing 
• Data augmentation 
• Categorical to Text data 
• Methods can not use in text data 
• Tokenization, Steaming, and Stop-words 
• The Bag-of-Words Model (BoW) 

o CountVectorizer and Term Frequency-Inverse Document Frequency (TF-IDF) 

• Summary

 

Chapter 6:  Data Resampling
After cleaning and preparing the dataset, the reader should know how to sample this dataset in the right way. Choosing wrong samples from your data may influence the result of your models. So in this chapter, we will illustrate all techniques and approaches needed to sample your dataset in the right way. 

• Creating training and test sets 
• Cross-Validation

o Validation set technique
oLeave-One-Out Cross-Validation (LOOCV) o K-Fold Cross-Validation 

• Bootstrap
• Case Study -- Generators (modify) 
• Summary

 

Chapter 7:  Feature Selection and Feature Engineering
In this chapter, we will describe a very important topic in the pipeline, which is feature selection and engineering. The audience should know how to select and choose the most important input feature that contributes most to your output feature. We will cover the process of using domain knowledge of the data to create features that make machine learning algorithms work. And also feature selection and engineering, which are fundamental to the application of machine and deep learning. 

Datasets used in this chapter 

Feature selection and filtering 

Principal component analysis 

o Non-Negative Matrix Factorization (NMF) o Sparse PCA
oKernel PCA 

• ZCA Whitening 
• Topic Model and Latent Dirichlet Allocation (LDA) 
• Atom extraction and dictionary learning 
• Summary

 

Part Three:  TensorFlow
In this part, first we illustrate the fundamental concepts of deep learning. Then we deep dive to the core of neural networks and the types of neural networks, describing each type and showing the important concepts of the equations of deep learning. Finally we present real-life examples of each type. 

 

Chapter 8:  Deep Learning Fundamentals
In this chapter, we will describe the basic functions that deep learning is built on. Then we will try to build layers from these functions, and combine these layers together to get more complex models that will help us solve more complex problems. All that will be described by TensorFlow examples. 

• Perceptron
  oSingle Perceptron 

o Multi Perceptron 

• Difference neural network layers 

o input layer 

o hidden layer

o output layer

sigmoid

Softmax

• Shallow vs. Deep Neural Networks 
• Activation Functions 
• Gradient Descent 
• Loss Functions and Backpropagation 
• Vanishing and Exploding Gradients 
• TensorFlow Basics 

o Placeholder vs. Variable vs. Constant
oGradient-Descent Optimization Methods from a Deep-Learning Perspective 

o Learning Rate in Mini-batch Approach to Stochastic Gradient Descent
oOptimizers in TensorFlow 

• Full-Batch vs. Stochastic vs. Mini-Batch Gradient Descent 
• Summary

 

Chapter 9:  Improving Deep Neural Networks
After building the deep learning model, the improvement starts. This chapter covers optimization, tuning, and choosing hyperparameter techniques. The reader will also learn to evaluate, optimize, and tune their models parameters to reach optimal solutions and accuracy. 

• Gradient-Descent Optimization Methods 
• Choosing the Learning Rate 
• Dropout Layers and Regularization 
• Optimizers in TensorFlow 
• Normalization Techniques 

o Batch Normalization 

o Weight Normalization 

o Group Normalization
oInstance Normalization 

o Layer Normalization 

•  Summary

 

Chapter 10:  Convolutional Neural Networks
One of the important classes of deep learning is the convolutional neural network. In this chapter, we will illustrate everything about CNN from the one-dimensional mask to the advanced stuff like weight sharing and the difference between equivariance and invariance. We will illustrate a case study using the famous dataset CIFAR-10 

 What is ConvNet 

• Convolution Operation 

o 1D Convolution 

o 2D Convolution 

• Two-dimensional Unit Step Function 
• Common Image-Processing Filters 

o Mean and Median Filters
oGaussian Filter
oGradient-based Filters
oSobel Edge-Detection Filter o Identity Transform 

• Convolution Neural Networks 
• Layers of Convolution Neural Networks 

o Input Layer
oConvolution Layer oPooling Layer 

Backpropagation Through the Convolutional and Pooling Layer 
• Weight Sharing Through Convolution and Its Advantages 
• Translation Equivariance and Invariance 
• Case Study -- Digit Recognition on the CIFAR-10 Dataset 
• Summary

 

Chapter 11:  Sequential Models
Another class of deep learning is the sequential models. In this chapter, we will describe the problem of sequential data and the rise of recurrent neural networks as well as the problem and also the evolution of the GRU and LSTM. 

• Recurrent Neural Networks (RNN) 
• Language Modeling 
• Backpropagation Through Time (BPTT) 
• Case Study -- Predicting the Next Word in a Sentence 
• Vanishing and Exploding Gradient Problem in RNN 
• Solution to Vanishing and Exploding Gradients Problem in RNNs 
• Long Short-Term Memory (LSTM) 
• Exploding and Vanishing Gradient Reduction in LSTM 
• Case Study -- Digit Identification on the MNIST Dataset 
• Gated Recurrent Unit (GRU) 
• Bidirectional RNN (Bi-RNN) 
• Summary

 

Part Four:  Applications and Appendix
We will give the readers this part to make sure that they practice by using TensorFlow and build the pipeline. 

 

Chapter 12:  Selected Topics in Computer Vision
After finishing CNN in part three, it's good to add some extra knowledge that makes it easier for readers when they work, like using pre-built architectures and transfer learning. 

•  Different Architectures in Convolutional Neural Networks 

o LeNet 

o AlexNet 

o VGG16 

o ResNet 

• Transfer Learning 
• Guidelines for Using Transfer Learning 

o InceptionV3 

o Pre-trained VGG16 

 Summary 

 

Chapter 13:  Selected Topics in Natural Language Processing
In this chapter, we fill the gaps that the audience needs for working with text. Giving the readers all the advanced approaches and techniques of natural language processing. 

• Vector Space Model 
Vector Representation of Words 
• Word2Vec

o Continuous Bag of Words (CBOW)
oSkip-Gram Model for Word Embedding 



o GloVe 

 Summary 

 

Chapter 14:  Model Deployment
Here we will show the audience how model deployment works. It's one last important step needed to communicate with others such as stakeholders and decision makers. 

• Introduction to Model Deployment. 
• Type and Architecture of deployments. 
• Introduction to Flask. 
• Installing Flask 
• Building a small REST Service 
• Real-life train and test. 
• Case Study -- tiny predictors 
• Summary

 

Chapter 15:  Applications
Here we will show some case studies to make sure that the reader gets the full knowledge and understanding of how to build pipelines with real-life examples. 

• Case Study -- Tabular Datasets 
• Case Study -- IMDB Movie Review Data with Paragraph2Vec 
• Case Study -- Image Segmentation 
• Conclusion