In recent years, video question answering based on multimodal large language models (MLLM) has garnered considerable attention, due to the benefits from the substantial advancements in LLMs. However, these models have a notable deficiency in the domains of video temporal grounding and reasoning, posing challenges to the development of effective real-world video understanding systems. Inspired by how humans use video players to interact with the progress bar for video comprehension, we introduce VTimeCoT, a simple yet effective training-free framework, designed for high-performance video grounding and reasoning. The proposed framework incorporates two novel visual tools of the progress bar: a plug-and-play progress bar integration tool and a high-efficiency highlighting tool. In addition, to address the limitations of conventional text-based chain-of-thought (CoT) approaches, we introduce a visuotemporal CoT process that integrates cross-modality reasoning across both video and text. Our approach demonstrates significant performance improvements on both Qwen2VL-7B and GPT4o baselines in tasks of video temporal grounding and reasoning-based question answering. Finally, we showcase that the proposed framework achieves a compositional and interpretable reasoning process.

We propose \( \textbf{VTimeCoT} \), a \( \textbf{V} \)isual \( \textbf{Time} \) \( \textbf{C} \)hain-\( \textbf{o} \)f-\( \textbf{T} \)hought framework for video temporal grounding and reasoning. VTimeCoT constructs cross-modality reasoning across both video and text, which enables the MLLM to utilize progress bar tools to annotate the time progression and highlight the key relevant segments to answer complex temporal questions.

On the left, we demonstrate how the framework iteratively generates thoughts and actions, which dynamically updates the video memory with an overlaid progress bar for reasoning. On the right, we illustrate two novel tools that integrate the frame-sync visual progress bar and highlight key moments.

Through reasoning that integrates the progress bar and highlights, VTimeCoT accurately answers questions related to temporal counting and order, which GPT-4o falls short.