TVMTrailer: A Text-Video-Music AIGC Framework for Film Trailer Generation
提出TVMTrailer模型，包含文本-视频生成网络和视频-音乐生成网络，利用电影素材和剧情简介自动生成预告片及配乐，实验基于两千多对经典电影视频-音频数据验证效果。
In the ever-evolving landscape of media and entertainment, where trends like short-form video content and new media platforms reign supreme, the need for innovative approaches across various industries becomes increasingly apparent. One such industry deeply impacted by these shifts is the film industry, where the creation and dissemination of film trailers stand as pivotal promotional strategies. Making a film trailer by hand is time consuming. However, the emerging artificial intelligence generated content (AIGC) technique has shown significant potential to enhance the efficiency. This article presents a novel model named TVMTrailer, which consists of a text-video generation network (TVGNet) and a video-music generation network (VMGNet). TVGNet employs an encoder–decoder framework, utilizing movie footage and synopses to generate movie trailers. Besides, VMGNet is proposed to generate sound track of our trailer. It combines video and audio features, and uses a transformer model for associative learning to adaptively generate audio clips with features, such as emotion, rhythm and beat. The effectiveness of TVMTrailer is demonstrated through experiment conducted on the proposed dataset and a comprehensive collection of over two thousand video-audio pairs from classic movies.