Nilesh Sarkar / Blog

Sparse Autoencoder experiments tied to a KD paper under review at COLM 2026. Toy-model validation of the KD minimum-width theorem plus three real-LM trials sweeping L1 sparsity on Pythia-410M (24 layers × 6 checkpoints each on Hugging Face).

Building an Agentic RAG workflow with LangGraph and Streamlit. The system prioritizes PDF Q&A and falls back to web search when necessary.

Running experiments with Stable Diffusion v1.5 via diffusers. Exploring negative prompts, guidance scales, and prompt engineering basics.

Knowledge distillation, mechanistic interpretability, and world models. KD paper under review at COLM 2026.

Circuit-level studies of transformer internals: attention patterns, probing, and activation patching to understand what models actually compute.

Systematic benchmark of 13 CNNs and 5 ViTs for PCOS ultrasound detection. Hybrid CNN-Transformer models (EfficientFormer-L1, MobileViT-Small) hit 99.81% accuracy with AUC up to 1.0. Includes a novel three-stage MD5 + pHash + cross-class deduplication pipeline that cleaned 70.4% of the public PCOS-XAI dataset.

Structure-prediction studies on small proteins - pLDDT confidence, folding trajectories, and transformer folding stacks vs. classical MD baselines.

Hands-on experimentation with LLM fine-tuning, compression, and deployment across diverse hardware. Testing Qwen models (0.5B → 7B) on Indic languages and deploying Gemma 3 1B on Jetson Nano edge devices.