NDLI: Bounds on the kullback-leibler divergence rate between hidden markov models

Content Provider	IEEE Xplore Digital Library
Author	Vidyasagar, M.
Copyright Year	2007
Description	Author affiliation: Tata Consultancy Services, Hyderabad (Vidyasagar, M.)
Abstract	The Kullback-Leibler (K-L) divergence rate is a natural extension of the familiar K-L divergence between probability vectors, to the situation where one is observing a sequence of dependent samples, such as for example the state sequence of a Markov chain or the output sequence of a hidden Markov model (HMM). In this paper, we study the problem of estimating the K-L divergence rate between two HMMs with a common output space, where the underlying Markov chains can evolve over different state spaces. At present, there is no closed-form formula for the K-L divergence rate for HMMs, though there is a closed-form formula for the K-L divergence rate between Markov chains over a common state space. In this paper, we give an alternate formulation of the K-L divergence rate between two stationary stochastic processes. Using this alternate formulation, we derive an upper bound for the K-L divergence rate between two HMMs. This estimate converges geometrically fast to the correct answer as the length of the observation increases. However, it is not a particularly elegant estimate. It is hoped that future research will lead to tighter estimates if not a closed-form formula for the K-L divergence rate.
Starting Page	6160
Ending Page	6165
File Size	212051
Page Count	6
File Format	PDF
ISBN	9781424414970
ISSN	01912216
DOI	10.1109/CDC.2007.4434365
Language	English
Publisher	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher Date	2007-12-12
Publisher Place	USA
Access Restriction	Subscribed
Rights Holder	Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subject Keyword	Hidden Markov models Stochastic processes State-space methods Upper bound Probability distribution Computational biology Closed-form solution USA Councils State estimation Information theory
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Library of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Variational Kullback-Leibler divergence for Hidden Markov models

Lower and upper bounds for approximation of the Kullback-Leibler divergence between Gaussian Mixture Models

Upper Bound Kullback–Leibler Divergence for Transient Hidden Markov Models

Upper and lower bounds for approximation of the Kullback-Leibler divergence between Hidden Markov models

Upper bound Kullback-Leibler divergence for hidden Markov models with application as discrimination measure for speech recognition (2006)

Upper Bound Kullback-Leibler Divergence for Hidden Markov Models with Application as Discrimination Measure for Speech Recognition

Lower and upper bounds for approximation of the kullback-leibler divergence between gaussian mixture models.

Quickest change detection and Kullback-Leibler divergence for two-state hidden Markov models

The Kullback-Leibler Divergence Rate between Markov Sources (2004)

Bounds on the kullback-leibler divergence rate between hidden markov models

Similar Documents

Variational Kullback-Leibler divergence for Hidden Markov models

Lower and upper bounds for approximation of the Kullback-Leibler divergence between Gaussian Mixture Models

Upper Bound Kullback–Leibler Divergence for Transient Hidden Markov Models

Upper and lower bounds for approximation of the Kullback-Leibler divergence between Hidden Markov models

Upper bound Kullback-Leibler divergence for hidden Markov models with application as discrimination measure for speech recognition (2006)

Upper Bound Kullback-Leibler Divergence for Hidden Markov Models with Application as Discrimination Measure for Speech Recognition

Lower and upper bounds for approximation of the kullback-leibler divergence between gaussian mixture models.

Quickest change detection and Kullback-Leibler divergence for two-state hidden Markov models

The Kullback-Leibler Divergence Rate between Markov Sources (2004)

Bounds on the kullback-leibler divergence rate between hidden markov models