[go: up one dir, main page]

Lee et al., 2007 - Google Patents

Automatic word spacing using probabilistic models based on character n-grams

Lee et al., 2007

Document ID
17856123246120352462
Author
Lee D
Rim H
Yook D
Publication year
Publication venue
IEEE Intelligent Systems

External Links

Snippet

On the Internet, information is largely in text form, which often includes such errors as spelling mistakes. These errors complicate natural language processing because most NLP applications aren't robust and assume that the input data is noise free. Preprocessing is …
Continue reading at ieeexplore.ieee.org (other versions)

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/27—Automatic analysis, e.g. parsing
    • G06F17/2765—Recognition
    • G06F17/2775—Phrasal analysis, e.g. finite state techniques, chunking
    • G06F17/278—Named entity recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/27—Automatic analysis, e.g. parsing
    • G06F17/2765—Recognition
    • G06F17/277—Lexical analysis, e.g. tokenisation, collocates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/27—Automatic analysis, e.g. parsing
    • G06F17/2705—Parsing
    • G06F17/2715—Statistical methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/27—Automatic analysis, e.g. parsing
    • G06F17/2785—Semantic analysis
    • G06F17/279—Discourse representation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/27—Automatic analysis, e.g. parsing
    • G06F17/2705—Parsing
    • G06F17/271—Syntactic parsing, e.g. based on context-free grammar [CFG], unification grammars
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/28—Processing or translating of natural language
    • G06F17/2809—Data driven translation
    • G06F17/2827—Example based machine translation; Alignment
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/27—Automatic analysis, e.g. parsing
    • G06F17/274—Grammatical analysis; Style critique
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/27—Automatic analysis, e.g. parsing
    • G06F17/273—Orthographic correction, e.g. spelling checkers, vowelisation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/21—Text processing
    • G06F17/22—Manipulating or registering by use of codes, e.g. in sequence of text characters
    • G06F17/2217—Character encodings
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/28—Processing or translating of natural language
    • G06F17/2863—Processing of non-latin text
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/28—Processing or translating of natural language
    • G06F17/2872—Rule based translation
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/18—Speech classification or search using natural language modelling
    • G10L15/183—Speech classification or search using natural language modelling using context dependencies, e.g. language models
    • G10L15/19—Grammatical context, e.g. disambiguation of the recognition hypotheses based on word sequence rules
    • G10L15/197—Probabilistic grammars, e.g. word n-grams
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/18—Speech classification or search using natural language modelling
    • G10L15/183—Speech classification or search using natural language modelling using context dependencies, e.g. language models
    • G10L15/187—Phonemic context, e.g. pronunciation rules, phonotactical constraints or phoneme n-grams
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/3061—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F17/30634—Querying
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/06—Creation of reference templates; Training of speech recognition systems, e.g. adaptation to the characteristics of the speaker's voice
    • G10L15/065—Adaptation
    • G10L15/07—Adaptation to the speaker

Similar Documents

Publication Publication Date Title
JP4833476B2 (en) Language input architecture that converts one text format to the other text format with modeless input
Kasewa et al. Wronging a right: Generating better errors to improve grammatical error detection
JP5535417B2 (en) Language input architecture that converts from one text format to another, resistant to spelling errors, typographical errors, and conversion errors
Azmi et al. A survey of automatic Arabic diacritization techniques
Wilcox-O’Hearn et al. Real-word spelling correction with trigrams: A reconsideration of the Mays, Damerau, and Mercer model
Lee et al. Automatic word spacing using probabilistic models based on character n-grams
Janssen NeoTag: a POS Tagger for Grammatical Neologism Detection.
Alkanhal et al. Automatic stochastic arabic spelling correction with emphasis on space insertions and deletions
Faili et al. Vafa spell-checker for detecting spelling, grammatical, and real-word errors of Persian language
JP2018206262A (en) Word linking identification model learning device, word linking detection device, method and program
Tufiş et al. DIAC+: A professional diacritics recovering system
Go et al. Gramatika: A grammar checker for the low-resourced Filipino language
Wu et al. Integrating dictionary and web n-grams for Chinese spell checking
Elgamal et al. Arabic diacritics in the wild: Exploiting opportunities for improved diacritization
Kaur et al. Hybrid approach for spell checker and grammar checker for Punjabi
Kumar et al. Design and implementation of NLP-based spell checker for the Tamil language
Singha et al. Part of speech tagging in Manipuri with hidden markov model
Hládek et al. Online natural language processing of the Slovak language
Chiu et al. Chinese spell checking based on noisy channel model
Mijlad et al. Arabic text diacritization: Overview and solution
Mekki et al. COTA 2.0: An automatic corrector of Tunisian Arabic social media texts
Sharma et al. Improving existing punjabi grammar checker
Hladek et al. Unsupervised spelling correction for Slovak
Ratnam et al. Phonogram-based Automatic Typo Correction in Malayalam Social Media Comments
Laki Investigating the Possibilities of Using SMT for Text Annotation