NDLI: Extracting semantic representations from word co-occurrence statistics: stop-lists, stemming, and SVD

Content Provider	SpringerLink
Author	Bullinaria, John A. Levy, Joseph P.
Copyright Year	2012
Abstract	In a previous article, we presented a systematic computational study of the extraction of semantic representations from the word–word co-occurrence statistics of large text corpora. The conclusion was that semantic vectors of pointwise mutual information values from very small co-occurrence windows, together with a cosine distance measure, consistently resulted in the best representations across a range of psychologically relevant semantic tasks. This article extends that study by investigating the use of three further factors—namely, the application of stop-lists, word stemming, and dimensionality reduction using singular value decomposition (SVD)—that have been used to provide improved performance elsewhere. It also introduces an additional semantic task and explores the advantages of using a much larger corpus. This leads to the discovery and analysis of improved SVD-based methods for generating semantic representations (that provide new state-of-the-art performance on a standard TOEFL task) and the identification and discussion of problems and misleading results that can arise without a full systematic study.
Starting Page	890
Ending Page	907
Page Count	18
File Format	PDF
ISSN	07433808
Journal	Behavior Research Methods
Volume Number	44
Issue Number	3
e-ISSN	15543528
Language	English
Publisher	Springer-Verlag
Publisher Date	2012-01-19
Publisher Place	New York
Access Restriction	Subscribed
Subject Keyword	Semantic representation Corpus statistics SVD Cognitive Psychology
Content Type	Text
Resource Type	Article

Sl.	Authority	Responsibilities	Communication Details
1	Ministry of Education (GoI), Department of Higher Education	Sanctioning Authority	https://www.education.gov.in/ict-initiatives
2	Indian Institute of Technology Kharagpur	Host Institute of the Project: The host institute of the project is responsible for providing infrastructure support and hosting the project	https://www.iitkgp.ac.in
3	National Digital Library of India Office, Indian Institute of Technology Kharagpur	The administrative and infrastructural headquarters of the project	Dr. B. Sutradhar bsutra@ndl.gov.in
4	Project PI / Joint PI	Principal Investigator and Joint Principal Investigators of the project	Dr. B. Sutradhar bsutra@ndl.gov.in Prof. Saswat Chakrabarti will be added soon
5	Website/Portal (Helpdesk)	Queries regarding NDLI and its services	support@ndl.gov.in
6	Contents and Copyright Issues	Queries related to content curation and copyright issues	content@ndl.gov.in
7	National Digital Libarray of India Club (NDLI Club)	Queries related to NDLI Club formation, support, user awareness program, seminar/symposium, collaboration, social media, promotion, and outreach	clubsupport@ndl.gov.in
8	Digital Preservation Centre (DPC)	Assistance with digitizing and archiving copyright-free printed books	dpc@ndl.gov.in
9	IDR Setup or Support	Queries related to establishment and support of Institutional Digital Repository (IDR) and IDR workshops	idr@ndl.gov.in

Extracting semantic representations from word co-occurrence statistics: stop-lists, stemming, and SVD

Performance impact of stop lists and morphological decomposition on word–word corpus-based semantic space models

Extracting semantic representations from word co-occurrence statistics: A computational study

Performance impact of stop lists and morphological decomposition on word–word corpus-based semantic space models

Extracting semantic representations from word co-occurrence statistics: A computational study (2007)

Extracting semantic representations from word co-occurrence statistics: A computational study

Extracting Semantic Representations from Large Text Corpora (1997)

Extracting semantic representations from word co-occurrence statistics: a computational study (2006).

Extracting Semantic Representations from Large Text Corpora (1997)

Extracting semantic representations from word co-occurrence statistics: stop-lists, stemming, and SVD

Similar Documents

Extracting semantic representations from word co-occurrence statistics: stop-lists, stemming, and SVD

Performance impact of stop lists and morphological decomposition on word–word corpus-based semantic space models

Extracting semantic representations from word co-occurrence statistics: A computational study

Performance impact of stop lists and morphological decomposition on word–word corpus-based semantic space models

Extracting semantic representations from word co-occurrence statistics: A computational study (2007)

Extracting semantic representations from word co-occurrence statistics: A computational study

Extracting Semantic Representations from Large Text Corpora (1997)

Extracting semantic representations from word co-occurrence statistics: a computational study (2006).

Extracting Semantic Representations from Large Text Corpora (1997)

Extracting semantic representations from word co-occurrence statistics: stop-lists, stemming, and SVD