Isbn: 9783389179833 - generating instrument sounds aligned with video via human body keypoints: a deep learning approach to multimodal audio-visual synthesis (5 résultats)

Etat : New.

Taschenbuch. Etat : Neu. Druck auf Anfrage Neuware - Printed after ordering - Research Paper (undergraduate) from the year 2026 in the subject Computer Science, grade: Good, , language: English, abstract: Historical video archives and recordings from the past often suffer from degraded or completely missing audio tracks due to deterioration of storage media, recording limitations of the era, or loss during archival processes. Similarly, silent films and performance documentation may lack synchronized sound entirely. Emerging generative artificial intelligence techniques have demonstrated the potential to reconstruct missing audio content by analyzing visual information alone-a capability particularly valuable for restoring cultural heritage materials and historical performance recordings. However, when applied to complex activities such as musical instrument performance, existing methods have shown limited accuracy in capturing the nuances of sound production. Prior research has established that SpecVQGAN architectures combined with Transformer-based mechanisms can improve video-to-audio generation. This work introduces an enhanced model that augments SpecVQGAN by incorporating human skeletal pose features, specifically designed to elevate the quality of generated musical instrument sounds. Through comprehensive evaluation using both subjective user studies and objective quantitative metrics, we demonstrate that the proposed framework significantly outperforms existing approaches in reconstructing authentic instrumental audio from archival and silent performance videos.…

Taschenbuch. Etat : Neu. This item is printed on demand - it takes 3-4 days longer - Neuware 40 pp. Englisch.

Taschenbuch. Etat : Neu. This item is printed on demand - Print on Demand Titel. Neuware -Research Paper (undergraduate) from the year 2026 in the subject Computer Science, grade: Good, , language: English, abstract: Historical video archives and recordings from the past often suffer from degraded or completely missing audio tracks due to deterioration of storage media, recording limitations of the era, or loss during archival processes. Similarly, silent films and performance documentation may lack synchronized sound entirely. Emerging generative artificial intelligence techniques have demonstrated the potential to reconstruct missing audio content by analyzing visual information alone-a capability particularly valuable for restoring cultural heritage materials and historical performance recordings. However, when applied to complex activities such as musical instrument performance, existing methods have shown limited accuracy in capturing the nuances of sound production. Prior research has established that SpecVQGAN architectures combined with Transformer-based mechanisms can improve video-to-audio generation. This work introduces an enhanced model that augments SpecVQGAN by incorporating human skeletal pose features, specifically designed to elevate the quality of generated musical instrument sounds. Through comprehensive evaluation using both subjective user studies and objective quantitative metrics, we demonstrate that the proposed framework significantly outperforms existing approaches in reconstructing authentic instrumental audio from archival and silent performance videos. 40 pp. Englisch.…

Taschenbuch. Etat : Neu. Generating Instrument Sounds Aligned with Video via Human Body Keypoints | A Deep Learning Approach to Multimodal Audio-Visual Synthesis | Haruka Okano (u. a.) | Taschenbuch | Englisch | 2026 | GRIN Verlag | EAN 9783389179833 | Verantwortliche Person für die EU: preigu GmbH & Co. KG, Lengericher Landstr. 19, 49078 Osnabrück, mail[at]preigu[dot]de | Anbieter: preigu Print on Demand.…