Alibaba Cloud launched a large-scale model-based AI assistant for work and study "Tongyi Hearing": intelligent record summary, real-time translation

Today, the Alibaba Cloud Summit Guangdong-Hong Kong-Macao Greater Bay Area was held in Guangzhou. Zhou Jingren, Chief Technology Officer of Alibaba Cloud Intelligence, released the latest product of the Tongyi family - "Tongyi Tingwu", an AI assistant for work and study. Through large-scale model technology, this product can be applied to various scenarios such as conference discussions, teaching and training, research interviews, and video viewing.

Tongyi Tingwu adopts the understanding and summarization technology provided by Tongyi Qianwen large model, which can become a powerful AI assistant for users' work and study. It can help users efficiently transcribe, retrieve, summarize and organize audio and video content anytime, anywhere, such as automatically taking notes, organizing interviews, extracting PPT, etc. During the public beta period, users can receive more than 100 hours of listening and transcribing time for free.

Zhou Jingren pointed out that Tongyi Hearing can provide a brand-new audio-visual product experience. In terms of real-time dialogue, it has functions such as real-time recording, multilingual translation, speech summary, and question review; in terms of video experience, Tongyi Tingwu supports cloud disk import and local upload, chapter quick view, full-text summary, etc. According to the on-site demonstration, Tingwu has integrated more than ten AI functions, which can comprehensively improve the efficiency of converting audio and video into graphic and text forms. Not only can it generate meeting records with high accuracy and distinguish between different speakers, but the "savvy" of common sense listening is extremely high. The large model can divide audio and video chapters and form abstracts, summary texts and the views of each speaker within one second. , Organize focus and to-do items. In the future, the large model will also provide functions such as one-click extraction of PPT, questions to AI for multiple audio and video contents, and generalization of specific passages.

According to reports, Tongyi Tingwu has been officially tested since June 1, and users can experience it through the official website . Officials call it the first large-scale model application product in China that is open to public testing.

By visiting the official website of Tongyi Tingwu, we learned that the app can realize real-time transcription and audio-video conversion in scenarios such as conferences, courses, interviews, and trainings, intelligently generate summaries, and translate in real time, breaking barriers in cross-language communication. At the same time, Tongyi Listening also supports quick marking of key information, and users can easily export recognition results and notes to local or Alibaba cloud disks, so as to manage knowledge more efficiently.

Here we also bring you the official introduction of Tongyi’s listening and comprehension core competencies:

Real-time voice transcription, generate intelligent records, search and locate to help efficient review

• Enable real-time recording, complete precipitation of communication content, accurate and quick generation of records, and corresponding playback of sounds and words
• Supports independent retrieval of keywords, precise positioning of core information, and easy review of session highlights

File transfer, massive file upload at the same time, saving you time and effort

• Quickly upload audio and video files such as conferences, studies, interviews, etc., and can upload 50 local files at the same time, or upload files in Alibaba Cloud disk
• Supports automatic distinction of speakers, and the transcription results are automatically saved in "My Records", which is convenient for viewing and reviewing at any time

Real-time translation, support for Chinese-English translation, smooth cross-language collaboration

• After the real-time transcription is completed, it supports one-click translation between Chinese and English, breaking the language barrier and easily realizing barrier-free communication

Quickly mark, highlight key points, problems, and to-dos, and key information is clear at a glance

• Support highlighting key points, problems, and to-do items, support filtering and batch excerpts, and review and organize more clearly

Easy export, rich content for you to choose, various formats for you to choose

• Support downloading of original text, notes, audio and video, and translation, and export the required content with one click
• Multiple download file formats: support word, pdf and srt subtitle file export

Reference link:

Official website experience link: https://tingwu.aliyun.com/