Feature extraction and engineering is the iterative process of creating the features and labels needed to train the model through a series of data transformations. It is often performed in lockstep with model training, because the ability to identify helpful features can have a significant impact on the overall success of the modeling effort. Simple examples of feature engineering include generating derived features (such as calculating an age from a birthdate) or converting categorical variables (such as transaction types) into one-hot encoded, or binary, vectors (e.g. turning “male/female” data into male=o, female=1.)
With deep learning, features are usually straightforward because deep neural networks (DNNs) generate their own internal transformations. With traditional machine learning, feature engineering can be quite challenging and relies heavily on the creativity and experience of the data scientist and their understanding of the business domain or ability to effectively collaborate with domain experts.