Machine Learning

Can computers learn to “hear” the rhythm in a stream of data, much like we hear the rhythm in music? And by using this skill, can they better protect us from equipment failures, financial fraud, or health problems? A new scientific paper titled “CaPulse: Detecting Anomalies by Tuning in to the Causal Rhythms of Time Series” attempts to answer these questions. The Problem with Anomalies We live in a world of data. From our heartbeats and stock market fluctuations to energy consumption in a smart city—all of this is time series data, collected at regular intervals. Often lurking within this data are anomalies: strange, unexpected events that can signal a problem. This could be a sudden cardiac arrhythmia, a suspicious bank transaction, or an impending engine failure in a factory. ...

In the world of artificial intelligence, data is the fuel that powers machine learning models. But what if that fuel is contaminated? Mislabeled data, known as label noise, is a huge problem that can cause even the best algorithms to learn complete nonsense. The paper “ε-Softmax: Approximating One-Hot Vectors for Mitigating Label Noise,” accepted at the prestigious NeurIPS 2024 conference, offers an elegant solution. The Problem: When a Model Blindly Trusts Its Labels Let’s imagine we’re training a model to recognize animals. We show it a picture of a cute cat. In the traditional approach, we give it an absolutely certain piece of information, a so-called one-hot vector: ...

In the field of machine learning, a model’s ability to assess its own confidence is crucial for its reliability, especially in high-stakes applications like medicine or autonomous vehicles. The arXiv paper 2508.00754, titled “A Simple and Effective Method for Uncertainty Quantification and OOD Detection”, by Yaxin Ma, Benjamin Colburn, and Jose C. Principe, introduces an innovative and efficient approach to this problem. The paper focuses on two related concepts: uncertainty quantification and Out-of-Distribution (OOD) detection. ...

Clinical trial enrollment is a critical bottleneck in drug development: nearly 80% of trials fail to meet target enrollment, costing up to $8 million per day if delayed. In this work, we introduce a multimodal deep‐learning framework that not only predicts total participant count but also quantifies uncertainty around those predictions. Challenges in Enrollment Forecasting Traditional approaches fall into two camps: Deterministic models – e.g. tabular ML like XGBoost or LightGBM – which output a point estimate but ignore variability in recruitment rates. Stochastic models – e.g. Poisson or Poisson–Gamma processes – which simulate recruitment and give confidence intervals, but often struggle with high-dimensional, heterogeneous data. Model Architecture Inputs ...

The paper “Consensus-Driven Active Model Selection” introduces CODA, a method that selects the best machine learning model using the predictions of many candidate models and minimal labeled data. CODA builds a probabilistic framework that leverages model agreement and disagreement to guide which examples should be labeled next. 🚀 Key Concepts Active model selection: Instead of labeling a full validation set, CODA selectively chooses which data points to label by estimating which would be most informative. Consensus modeling: CODA uses a Bayesian adaptation of the Dawid-Skene model to evaluate model performance based on agreement among models. PBest distribution: Represents the current belief about which model is best, updated with each newly labeled data point. 🧪 How Does CODA Work? Model predictions are collected over unlabeled data. A consensus label for each data point is calculated using a weighted sum of predictions from all models. Each model is assigned a confusion matrix prior using a Dirichlet distribution: $$ \theta_{k, c, c’} = \frac{\beta_{c, c’} + \alpha \hat{M}_{k, c, c’}}{T} $$ CODA updates a probabilistic estimate over which model is best: $$ PBest(h_k) = \int_0^1 f_k(x) \prod_{l \ne k} F_l(x) dx $$ It selects the next data point to label by maximizing expected information gain: $$ EIG(x_i) = H(PBest) - \sum_c \hat{\pi}(c \mid x_i) H(PBest^c) $$ 📊 Results CODA outperforms previous state-of-the-art methods on 18 out of 26 benchmark tasks. Achieves optimal model selection with up to 70% fewer labels compared to baselines. Especially effective in multi-class tasks (e.g., DomainNet, WILDS). ❗ Limitations In binary classification with high data imbalance, CODA may underperform due to biased early estimates (e.g., CivilComments, CoLA datasets). CODA assumes that consensus is meaningful; highly divergent models may reduce effectiveness. 🔮 Future Work Better priors from human knowledge or unsupervised features. Extension to non-classification tasks and alternative metrics. Integration with active learning and active testing frameworks. Links Based on the publication 📄 arXiv:2507.23771 PDF

Ever wondered whether that expensive jar of “acacia honey” is the real deal? Or if the origin listed on the label truly reflects the soil and flowers it came from? In a new study, researchers used machine learning and mineral analysis to uncover the botanical and geographical roots of honey — all without needing a microscope. The Science Behind It When bees produce honey, they also carry tiny traces of minerals from the plants and soil around them. These mineral fingerprints — elements like calcium, magnesium, or zinc — vary depending on the environment. By measuring them, we can build a kind of chemical signature for each honey. ...

In many real-world tasks—like forecasting the paths of cars at a busy intersection, coordinating fleets of delivery robots, or simulating pedestrian movement—models must reason about not just where things are, but how they face or rotate relative to each other. That’s the SE(2) geometry: 2D position + heading. Traditional Transformer models that account for rotation and translation invariance (SE(2)-invariant) need to compute relative poses between every pair of objects. If you have $n$ objects, this leads to memory cost growing like $O(n^2)$—which becomes prohibitively expensive when $n$ is large. ...

The paper “Understanding the Evolution of the Neural Tangent Kernel at the Edge of Stability” by Kaiqi Jiang, Jeremy Cohen, and Yuanzhi Li explores how the Neural Tangent Kernel (NTK) evolves during deep network training, especially under the Edge of Stability (EoS) regime. What is the NTK? The Neural Tangent Kernel (NTK) is a matrix that captures how tiny weight changes affect network outputs on each training example. It lets us analyze neural networks with tools from kernel methods, offering theoretical insights into learning dynamics. What is the Edge of Stability? When training with a large learning rate $\eta$, the largest eigenvalue of the NTK (or the loss Hessian) exceeds the stability threshold $2/\eta$ and then oscillates around it. This phenomenon, called Edge of Stability, combines elements of instability with phases of rapid learning. Key Findings Alignment Shift Higher $\eta$ leads to stronger final Kernel Target Alignment (KTA) between the NTK and the label vector $y$. ...

In recent years, Low‑Rank Adaptation (LoRA) has become a cornerstone technique for parameter‑efficient fine‑tuning of large language models (LLMs) and diffusion models. By injecting low‑rank matrices into pre-trained weights, LoRA drastically reduces memory and compute requirements, enabling rapid experimentation and deployment. However, practitioners face two persistent challenges: Initialization ambiguity: Different low‑rank factor pairs $$A, B$$ can represent the same adapted weight update $AB^\top$, leading to unstable or suboptimal starts. Redundant parameterization: Without a canonical representation, gradient updates can wander through equivalent parameter configurations. The RiemannLoRA framework, introduced by Bogachev et al., offers a unifying geometric viewpoint that removes these ambiguities and yields faster, more stable fine‑tuning. ...

Imagine you’re analyzing sensor data. Suddenly one sensor shows -999°C. That’s an outlier — a single data point that can completely ruin your analysis. 🧩 What is factorization? Matrix factorization means decomposing data $X$ into two non-negative components: $$ X \approx WH $$ Where $W$ contains “features” and $H$ shows how much of each is needed. 💡 The problem Classical methods like NMF are sensitive to noise and outliers. When data is messy, analysis breaks down. ...