Question and answer about clips from Zootopia, a cartoon, which tells the story of a determined police officer rabbit named Judy who pairs up with a cunning fox to uncover a conspiracy about missing animals and develop an unexpected friendship.
Question and answer about clips from Goblin, which tells the story of Kim Shin, an immortal ”goblin” who needs to find a human bride to end his endless life but instead meets Ji Eun-tak, a girl fated to die who claims to be the ”goblin’s bride,” leading to a romantic tale unfolding bet.
First, ceate a conda environment:
``` conda env create -f environment.yml conda activate moviechat
Before using the repository, make sure you have obtained the following checkpoints:
``` python apply_delta.py \ --base ckpt/LLaMA/7B_hf \ --target ckpt/Vicuna/7B \ --delta ckpt/Vicuna/vicuna-7b-delta-v1.1 \
Firstly, set the
ckpt in eval_configs/MovieChat.yaml. Then run the script:
``` python inference.py \ --cfg-path eval_configs/MovieChat.yaml \ --gpu-id 0 \ --num-beams 1 \ --temperature 1.0 \ --text-query "What is he doing?" \ --video-path src/examples/Cooking_cake.mp4 \ --fragment-video-path src/video_fragment/output.mp4 \ --cur-min 1 \ --cur-sec 1 \ --middle-video 1 \
Note that, if you want to use the global mode (understanding and question-answering for the whole video), remember to change middle-video into 0.
Visit Official Website