Data Science & Machine Learning
ML models, algorithms, tools, and real-world applications
What is machine learning and how does it differ from traditional programming?
Machine learning is a subset of artificial intelligence where systems learn patterns from data to make predictions or decisions without being explicitly programmed for each task. Unlike traditional programming, which uses hand-coded rules, ML models infer rules automatically from training examples, improving performance as more data becomes available. [Source: MIT CSAIL]
What is deep learning and how does it relate to neural networks?
Deep learning is a class of machine learning that uses multi-layered artificial neural networks to model complex patterns in data. Each layer learns progressively abstract representations, enabling breakthroughs in image recognition, speech processing, and natural language understanding that shallower models could not achieve. [Source: National Institute of Standards and Technology]
What is supervised learning in machine learning?
Supervised learning trains a model on labeled input-output pairs so it can predict outputs for new, unseen inputs. Common tasks include classification (assigning categories) and regression (predicting continuous values). Examples include spam detection, medical diagnosis, and house-price prediction, where ground-truth labels guide training. [Source: NIST AI Glossary]
What is unsupervised learning and when is it used?
Unsupervised learning finds hidden structure in unlabeled data without predefined output categories. Clustering, dimensionality reduction, and anomaly detection are core applications. It is used when labeled data is scarce or expensive, such as customer segmentation, gene-expression analysis, and network-intrusion detection. [Source: Stanford CS229 Course Notes]
What is an artificial neural network?
An artificial neural network (ANN) is a computational model inspired by biological neurons, consisting of interconnected layers of nodes that process inputs through weighted connections and activation functions. ANNs underpin modern deep learning and are trained via backpropagation to minimize prediction error across millions of parameters. [Source: IEEE]
What is overfitting in machine learning and how can it be prevented?
Overfitting occurs when a model learns training data too precisely, capturing noise rather than generalizable patterns, causing poor performance on new data. Prevention strategies include cross-validation, regularization (L1/L2), dropout layers, early stopping, and increasing training data volume to improve generalization. [Source: Stanford CS229]
What is cross-validation and why is it important in model evaluation?
Cross-validation is a resampling technique that partitions data into multiple subsets, training and evaluating the model on different folds to obtain a reliable performance estimate. K-fold cross-validation is the most common variant, reducing bias from a single train-test split and helping detect overfitting early. [Source: Stanford CS229]
How do you evaluate the performance of a machine learning model?
Model performance is evaluated using task-specific metrics: accuracy, precision, recall, and F1-score for classification; mean absolute error and RMSE for regression; AUC-ROC for ranking. NIST recommends combining multiple metrics with held-out test sets and fairness audits to avoid misleading single-metric assessments. [Source: NIST AI Risk Management Framework]
What are the most common machine learning algorithms?
Widely used ML algorithms include linear and logistic regression, decision trees, random forests, gradient boosting (XGBoost, LightGBM), support vector machines, k-nearest neighbors, and neural networks. Algorithm choice depends on data size, feature dimensionality, interpretability requirements, and whether the task is classification, regression, or clustering. [Source: NIST]
What is a random forest algorithm and when should you use it?
Random forest is an ensemble method that trains many decision trees on random data subsets and aggregates their predictions by majority vote or averaging. It handles high-dimensional data, is robust to outliers, provides feature importance scores, and performs well without extensive hyperparameter tuning on both classification and regression tasks. [Source: IEEE]
What is gradient boosting and how does XGBoost differ from standard gradient boosting?
Gradient boosting builds an ensemble sequentially, where each new tree corrects residual errors of the prior model by following the negative gradient of a loss function. XGBoost extends this with regularization terms, second-order gradients, column subsampling, and hardware-optimized parallel tree construction, achieving superior speed and accuracy. [Source: ACM SIGKDD]
What is natural language processing (NLP) and what are its main applications?
Natural language processing enables computers to understand, interpret, and generate human language. Key applications include machine translation, sentiment analysis, named-entity recognition, question answering, text summarization, and large language models like GPT. NLP combines linguistics, statistics, and deep learning to bridge human communication and machine understanding. [Source: ACM]
What is a transformer model and why did it revolutionize AI?
The transformer architecture, introduced in the 2017 paper 'Attention Is All You Need,' uses self-attention mechanisms to weigh token relationships across entire sequences simultaneously, replacing recurrent networks. This parallelization enabled training at unprecedented scale, underpinning models such as BERT, GPT, and T5 that dominate modern NLP and vision tasks. [Source: Google Research / ACM]
What is clustering in data science and what algorithms are commonly used?
Clustering groups similar data points together without predefined labels, revealing natural structure in datasets. K-means, DBSCAN, hierarchical agglomerative clustering, and Gaussian mixture models are widely used. Applications span customer segmentation, document grouping, image compression, and biological sequence analysis in genomics research. [Source: Stanford CS229]
What is dimensionality reduction and why does it matter in machine learning?
Dimensionality reduction compresses high-dimensional data into fewer informative features, reducing computational cost, mitigating the curse of dimensionality, and improving model generalization. Principal component analysis (PCA) and t-SNE are dominant techniques; autoencoders offer deep-learning-based alternatives for non-linear reduction in complex datasets. [Source: Stanford CS229]
What is feature engineering and why is it critical for machine learning performance?
Feature engineering transforms raw data into informative input variables that improve model accuracy. Tasks include normalization, encoding categorical variables, creating interaction terms, handling missing values, and deriving domain-specific indicators. Quality features often matter more than algorithm choice, directly determining ceiling performance on structured datasets. [Source: NIST]
What is data preprocessing in data science and what steps does it involve?
Data preprocessing transforms raw, messy data into a clean format suitable for modeling. Key steps include handling missing values, removing duplicates, outlier treatment, feature scaling (normalization/standardization), encoding categorical variables, and train-test splitting. Preprocessing quality directly impacts model reliability and is estimated to consume 60–80% of a data scientist's time. [Source: NIST Data Quality]
What are the most important Python libraries for machine learning and data science?
Core Python ML libraries include NumPy and pandas for data manipulation, scikit-learn for classical ML algorithms, TensorFlow and PyTorch for deep learning, Matplotlib and Seaborn for visualization, and Hugging Face Transformers for NLP. These open-source tools form the standard production stack across academic and industrial ML projects. [Source: IEEE Software]
What is MLOps and why is it important for deploying machine learning models?
MLOps (Machine Learning Operations) applies DevOps principles to ML workflows, encompassing continuous integration and deployment of models, automated retraining pipelines, data and model versioning, monitoring for drift, and governance. It bridges the gap between experimental data science and reliable production systems, reducing model failure rates after deployment. [Source: NIST AI RMF]
What is fairness in machine learning and how is bias detected and mitigated?
ML fairness requires that model predictions do not systematically disadvantage protected groups based on attributes like race, gender, or age. Bias can enter through unrepresentative training data, proxy features, or feedback loops. Detection uses disparate impact analysis and equalized odds metrics; mitigation includes reweighting, adversarial debiasing, and post-processing calibration. [Source: NIST AI RMF]
What is regularization in machine learning and what are L1 vs L2 regularization?
Regularization adds a penalty term to the loss function to discourage overly complex models. L1 regularization (Lasso) adds the sum of absolute weights, promoting sparsity by zeroing unimportant features. L2 regularization (Ridge) penalizes squared weights, shrinking all coefficients smoothly. Both reduce overfitting; Lasso is preferred when feature selection is needed. [Source: Stanford CS229]