NDLI: Audio-visual feature integration based on piecewise linear transformation for noise robust automatic speech recognition

Content Provider	IEEE Xplore Digital Library
Author	Kashiwagi, Y. Suzuki, M. Minematsu, N. Hirose, K.
Copyright Year	2012
Description	Author affiliation: Graduate School of Engineering, The University of Tokyo, Japan (Suzuki, M.; Minematsu, N.) \|\| Graduate School of Information Science and Technology (Kashiwagi, Y.; Hirose, K.)
Abstract	Multimodal speech recognition is a promising approach to realize noise robust automatic speech recognition (ASR), and is currently gathering the attention of many researchers. Multimodal ASR utilizes not only audio features, which are sensitive to background noises, but also non-audio features such as lip shapes to achieve noise robustness. Although various methods have been proposed to integrate audio-visual features, there are still continuing discussions on how the vest integration of audio and visual features is realized. Weights of audio and visual features should be decided according to the noise features and levels: in general, larger weights to visual features when the noise level is low and vice versa, but how it can be controlled? In this paper, we propose a method based on piecewise linear transformation in feature integration. In contrast to other feature integration methods, our proposed method can appropriately change the weight depending on a state of an observed noisy feature, which has information both on uttered phonemes and environmental noise. Experiments on noisy speech recognition are conducted following to CENSREC-1-AV, and word error reduction rate around 24% is realized in average as compared to a decision fusion method.
Starting Page	149
Ending Page	152
File Size	257578
Page Count	4
File Format	PDF
ISBN	9781467351256
e-ISBN	9781467351263
e-ISBN	9781467351249
DOI	10.1109/SLT.2012.6424213
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2012-12-02
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Visualization Feature enhancement SPLICE Multimodal ASR Error analysis Noise Hidden Markov models Speech recognition Speech Noise measurement noise robustness
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Audio-visual feature integration based on piecewise linear transformation for noise robust automatic speech recognition.

A robust frontend for ASR: Combining denoising, noise masking and feature normalization

Audio-visual deep learning for noise robust speech recognition

Noise robust speech recognition with a switching linear dynamic model

Feature enhancement by speaker-normalized splice for robust speech recognition

Towards improving ASR robustness for PSN & GSM telephone applications

Discriminative piecewise linear transformation based on deep learning for noise robust automatic speech recognition

A novel model characteristics for noise-robust Automatic Speech Recognition based on HMM

Integrating audio and visual information to provide highly robust speech recognition

Audio-visual feature integration based on piecewise linear transformation for noise robust automatic speech recognition

Similar Documents

Audio-visual feature integration based on piecewise linear transformation for noise robust automatic speech recognition.

A robust frontend for ASR: Combining denoising, noise masking and feature normalization

Audio-visual deep learning for noise robust speech recognition

Noise robust speech recognition with a switching linear dynamic model

Feature enhancement by speaker-normalized splice for robust speech recognition

Towards improving ASR robustness for PSN & GSM telephone applications

Discriminative piecewise linear transformation based on deep learning for noise robust automatic speech recognition

A novel model characteristics for noise-robust Automatic Speech Recognition based on HMM

Integrating audio and visual information to provide highly robust speech recognition

Audio-visual feature integration based on piecewise linear transformation for noise robust automatic speech recognition