Modeling Affective Distress from Video: A Modality-Aware Multi-Branch Learning Approach

Published in Proceedings of the 28th ACM International Conference on Multimodal Interaction (ICMI '26) • Advised by Dr. Haroon R. Lone

Automated detection of mental health conditions such as stress and Social Anxiety Disorder (SAD) from clinical interview video feeds presents severe challenges because affective distress manifests as both rapid, localized micro-expressions and sustained behavioral trends. Whereas conventional methods process heterogeneous behavioral cues through a single sequential encoder—leading to signal distortion—we propose a Multi-Branch Modality-Aware Multiple Instance Learning (MIL) framework that explicitly disentangles behavioral signals based on their temporal dynamics.

High-variance, sparse facial Action Units (extracted via OpenFace) are routed to a 1D CNN branch to capture localized micro-events, while continuous, autocorrelated postural landmarks and gaze vectors (extracted via MediaPipe) are routed to a Bidirectional LSTM branch to capture gradual behavioral trajectories. Window-level representations are unified via gated attention MIL into interpretable participant-level predictions. Trained with joint Binary Cross-Entropy and Supervised Contrastive losses, the architecture features gradient-based spatial saliency mapping to 2D video heatmaps and achieves 71.04% accuracy on a clinical Trier Social Stress Test (TSST) interview benchmark, with competitive cross-dataset generalization across StressID, UBFC-Phys, and AVEC 2017.

ACM Digital Library | PDF Paper | View Code on GitHub

Teacher-Guided Discriminative Flow Matching for Latent Representation Learning

Accepted at Northern Lights Deep Learning Conference (NLDL 2027) • Research at IIT Guwahati (Advised by Dr. Ayon Borthakur)

This research introduces DFM (Discriminative Flow Matching), investigating whether a teacher-guided latent vector field can serve as an effective feature-space regularizer for discriminative visual representation learning. Rather than transporting features from an arbitrary noise prior or matching generic class prototypes, DFM pairs a strongly augmented student view with a weakly augmented view from an Exponential Moving Average (EMA) teacher network, modeling sample-specific latent displacement along linear probability paths.

We train a conditional neural vector field conditioned on the current state, original student state, and interpolation time to predict the velocity required to traverse the displacement path (stabilized with target velocity norm clipping). A sample-dependent learned scalar gate dynamically scales the residual velocity update, while a joint training objective optimizes classification on both unrefined student and refined representations, flow-matching regression, and stop-gradient teacher endpoint transport. On CIFAR-100, DFM elevates baseline top-1 accuracy from 68.23% to 69.26% on the unrefined backbone and 69.38% with one-step refinement, while transferring effectively to pretrained Tiny-ImageNet (top-1 from 63.04% to 63.50%, top-5 from 84.98% to 85.99%) and demonstrating significantly tighter class-centroid clustering, lower Expected Calibration Error (ECE), and superior corruption robustness (CIFAR-100-C).

Camera-ready / preprint available upon request (proceedings to appear).

CA-AFP: Cluster-Aware Adaptive Federated Pruning for Heterogeneous FL

Under Submission at ACM IKDD CoDS-COMAD 2026 • Advised by Dr. Haroon R. Lone

Federated learning across real-world edge devices must accommodate non-IID client data distributions while minimizing communication and compute bottlenecks. Existing sparse FL methods impose a uniform, shared network topology across all participants, constraining heterogeneous client groups from adapting their layer-wise connectivity to their specific data distributions. We propose CA-AFP (Cluster-Aware Adaptive Federated Pruning), a clustered dynamic sparse FL framework that enables client groups to independently evolve specialized sparse network topologies under a common resource budget.

CA-AFP first clusters participating clients based on the cosine similarity of their accumulated optimization trajectories and subsequently conducts adaptive magnitude-based pruning and gradient-guided parameter regrowth independently within each cluster. We empirically analyze the structural evolution of sparse topologies across training—demonstrating that client clusters develop specialized layer-wise sparse connectivity patterns (particularly in later network layers) rather than merely diverging in weight values. Evaluated across four heterogeneous FL benchmarks (CIFAR-10, CIFAR-100, FEMNIST, and WISDM) under a matched 70% parameter sparsity budget, CA-AFP achieves the highest accuracy among sparse baselines on CIFAR-10, FEMNIST, and CIFAR-100, significantly accelerates early-stage convergence, and provides a Pareto-optimal accuracy-to-communication trade-off.

Predictive Modeling in ODI Cricket: Training Strategy, Feature Engineering, & Explainability

Under Review (Revision Stage) at International Journal of Sports Science & Coaching (SAGE Publications) • Advised by Dr. Kundan Kandhway

Predicting the outcome of One Day International (ODI) cricket matches is exceptionally challenging due to high variance and innings-dependent game dynamics. This paper introduces a comprehensive machine learning framework trained on 1,394 international ODI matches (2002–2024). To eliminate temporal data leakage and ensure realistic real-world deployment, we formulate a strict chronological expanding-window training and evaluation pipeline.

We engineer inning-specific static features (team relative strength metrics, venue characteristics, toss decision) alongside granular dynamic features (cumulative runs, required run-rate gap, wickets in hand, and pressure indices). We systematically compare two distinct modeling paradigms across Logistic Regression, SVM, Random Forest, XGBoost, Naive Bayes, and Neural Networks: a 'Unified Inning Model' trained across all overs, versus an 'Over-Wise Model' trained independently for each over. Our analysis proves that feature effectiveness is maximized by a hybrid strategy: the Unified Inning Model is decisively superior for the 1st innings where match outcome correlates steadily with cumulative run rate and team priors; conversely, the Over-Wise Model Switching strategy is decisively superior for the 2nd innings, capturing non-linear chasing volatility and achieving an outstanding 84.6% prediction accuracy. Game-theoretic SHAP interpretability analysis quantitatively decodes the dynamic transition of feature attribution across match phases.

View Code on GitHub

Vehicle detection and speed tracking

The purpose of this project was to build an automated system for monitoring traffic flow and enforcing speed limits. By training a YOLOv8 model, we achieved real-time vehicle detection and classification from video feeds. The system uses perspective transformation to accurately calculate the speed of each vehicle as it passes through a user-defined zone. This technology is highly valuable for intelligent traffic management systems, enhancing road safety and surveillance capabilities.

View Code on GitHub

Dimensionality Reduction Analysis

High-dimensional data can be computationally expensive and difficult to interpret. This project provides a comparative analysis of key dimensionality reduction techniques like PCA, LDA, and t-SNE. We applied these algorithms to various datasets to evaluate their effectiveness in preserving important data structures while reducing complexity. The work serves as a practical guide for selecting the appropriate technique for feature extraction and data visualization in machine learning workflows.

View Code on GitHub

Crater and Boulder Detection on Planetary Surfaces

This project automates the identification of geological features from planetary images, a critical task for rover navigation and scientific analysis. We implemented a computer vision pipeline using the Canny edge detector and Circular Hough Transform to accurately locate craters and boulders. This automated approach is far more efficient than manual analysis, enabling faster mapping of extraterrestrial surfaces and aiding in the selection of safe landing sites for future missions.

View Code on GitHub

Image Analysis with SIFT, BoW, and HOG

This project delves into classical feature-based image classification methods. We implemented the Scale-Invariant Feature Transform (SIFT) to extract robust keypoints from images, which were then clustered into a "Bag of Visual Words" (BoW). By creating histograms from these visual words, alongside Histogram of Oriented Gradients (HOG) features, we built a powerful model for classifying images. This work is fundamental to understanding the building blocks of many modern computer vision systems.

View Code on GitHub

Data Visualisation: Energy, Occupancy, and Weather

The goal of this project was to uncover hidden patterns and relationships within complex, multi-source datasets. By merging and visualizing data on energy consumption, building occupancy, and corresponding weather conditions, we aimed to identify key factors influencing energy usage. This type of analysis is crucial for developing smarter, more efficient energy management systems and promoting sustainable building practices through data-driven insights.

View Code on GitHub

Optimal Design of Serosurveys during Pandemic

Under the supervision of Dr. Rajesh Sundaresan at IISc Bengaluru, this project focused on the optimal design of serosurveys for disease burden estimation. The work involved determining how to best distribute a budget for various tests to accurately estimate disease prevalence. Key concepts studied include Maximum Likelihood Estimators, the Fisher Information matrix, Convex Optimisation, and Game-Theoretic approaches.

View Project Report