Innovation in Multimodal LLMs & AI Agents

At the forefront of computer science, artificial intelligence is enabling machines to simulate, extend, and even surpass human intelligence. Computer vision, at its core, empowers machines to “see and understand” the world, from recognizing objects within images to comprehending complex scenes and dynamics. Large models are now driving a revolution in multimodal cognition. Building upon this, the ultimate form of next-generation AI—AI Agents—has emerged. These are not merely conversational assistants but intelligent systems capable of understanding instructions, perceiving environments, and autonomously making decisions to execute tasks, representing the critical leap from “thinking” to “acting”. This program begins with computer vision, allowing students to understand how AI progresses from “seeing” images to “perceiving” the world. It then explores language understanding powered by large models, constructing a cross-modal cognitive bridge between “vision” and “language.” Finally, it focuses on building AI Agents. Students will engage with cutting-edge scenarios such as facial recognition and obstacle perception for autonomous driving, completing the construction and integration of core agent modules. Simultaneously, the program includes field visits to the CUHK campus, the MMLab, and SenseTime, enabling students to personally experience the complete innovation chain from academic research to industrial application.