轻松读懂
长难PDF

In this work, we propose Visual-Predictive Instruction Tuning (VPiT)—a simple and effective extension to visual instruction tuning that enables a pretrained LLM to quickly morph into an unified autoregressive model.

VPiT teaches an LLM to predict discrete text tokens and continuous visual tokens from any input sequence.