Ever since I was a kid, I have been fascinated by math, and it quickly became my favorite subject at school. I loved the challenge of solving problems and the satisfaction of finding elegant solutions. I remember wanting to become a mathematician or a physicist when I grew up, but in my home country of Ecuador, there was very little infrastructure or opportunity to pursue a career in either field. So I continued studying mathematics in a personal capacity. I do not mean to suggest that I am at the level of today’s mathematicians, whom I deeply respect and admire; my own level remains closer to that of a strong university graduate, perhaps a first-year mathematics graduate student. Luckily, I later had the opportunity to pursue a PhD in Computer Science, where my focus has been on AI and multimodality. Today, as AI systems have become more capable than we ever imagined, they have begun to significantly affect mathematics–to the point of making some novel discoveries autonomously.

John Schulman (Thinking Machines) on this blog argues, backed by careful experiments, that LoRA RL fine-tuning can match full-parameter RL fine-tuning. His intuition comes from the fact that in common reinforcement learning from verifiable rewards (RLVR) settings the total number of bits fed to the learner is small, so you don’t need to move all base weights to absorb the signal. In particular, he writes:

Self-supervised learning has been a game-changer for training large-scale Vision Transformers (ViTs), allowing models to learn powerful visual representations without human-provided labels. However, popular methods like DINO and iBOT suffer from a curious problem: as they train longer, their ability to handle dense, local tasks degrades, even as their global performance improves. This phenomenon, known as feature degradation, is characterized by noisy patch similarity maps and an over-alignment of patch features with the global CLS token.

In this post, we will see how to load a video using PyTorch, followed by a rant on how we perfom evaluations of video models. We will try various methods to load a video and convert it to a PyTorch tensor. These include VideoClips (from torchvision), torchvision.IO (using PyAV), and decord, my own implementation using ffmpeg and FFCV. We will use the Kinetics-400 dataset as an example. You can find the dataset here. All the code is available in this repository. I modified FFCV to be able to handle videos, you can find the fork here.