TY - GEN
T1 - Audio-driven human body motion analysis and synthesis
AU - Ofli, F.
AU - Canton-Ferrer, C.
AU - Tilmanne, J.
AU - Demir, Y.
AU - Bozkurt, E.
AU - Yemez, Y.
AU - Erzin, E.
AU - Tekalp, A. M.
PY - 2008
Y1 - 2008
N2 - This paper presents a framework for audio-driven human body motion analysis and synthesis. We address the problem in the context of a dance performance, where gestures and movements of the dancer are mainly driven by a musical piece and characterized by the repetition of a set of dance figures. The system is trained in a supervised manner using the multiview video recordings of the dancer. The human body posture is extracted from multiview video information without any human intervention using a novel marker-based algorithm based on annealing particle filtering. Audio is analyzed to extract beat and tempo information. The joint analysis of audio and motion features provides a correlation model that is then used to animate a dancing avatar when driven with any musical piece of the same genre. Results are provided showing the effectiveness of the proposed algorithm.
AB - This paper presents a framework for audio-driven human body motion analysis and synthesis. We address the problem in the context of a dance performance, where gestures and movements of the dancer are mainly driven by a musical piece and characterized by the repetition of a set of dance figures. The system is trained in a supervised manner using the multiview video recordings of the dancer. The human body posture is extracted from multiview video information without any human intervention using a novel marker-based algorithm based on annealing particle filtering. Audio is analyzed to extract beat and tempo information. The joint analysis of audio and motion features provides a correlation model that is then used to animate a dancing avatar when driven with any musical piece of the same genre. Results are provided showing the effectiveness of the proposed algorithm.
KW - Audio-driven body motion synthesis
KW - Dancing avatar animation
KW - Multicamera motion capture
UR - https://www.scopus.com/pages/publications/51449089854
U2 - 10.1109/ICASSP.2008.4518089
DO - 10.1109/ICASSP.2008.4518089
M3 - Conference contribution
AN - SCOPUS:51449089854
SN - 1424414849
SN - 9781424414840
T3 - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
SP - 2233
EP - 2236
BT - 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP
T2 - 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP
Y2 - 31 March 2008 through 4 April 2008
ER -