Articles tagged
LoRA
2 articles
HyDRA: Teaching Your Phone to Understand Images Without Breaking the Bank
HyDRA allocates LoRA rank per layer and inside layers automatically, buying a 4.7% gain on mobile vision-language models with no extra trainable parameters.
Machine LearningLoRAFine-tuning
RiemannLoRA: A Unified Riemannian Framework for Ambiguity-Free LoRA Optimization
RiemannLoRA places LoRA updates on the manifold of rank-r matrices, erasing factorization ambiguity: half the fine-tuning steps and up to 1.2 BLEU more.
Machine LearningRiemannian OptimizationLoRA