[go: up one dir, main page]

Jang et al., 2022 - Google Patents

Beyond distributional hypothesis: Let language models learn meaning-text correspondence

Jang et al., 2022

View PDF
Document ID
14476609102306378018
Author
Jang M
Mtumbuka F
Lukasiewicz T
Publication year
Publication venue
arXiv preprint arXiv:2205.03815

External Links

Snippet

The logical negation property (LNP), which implies generating different predictions for semantically opposite inputs, is an important property that a trustworthy language model must satisfy. However, much recent evidence shows that large-size pre-trained language …
Continue reading at arxiv.org (PDF) (other versions)

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/27—Automatic analysis, e.g. parsing
    • G06F17/2705—Parsing
    • G06F17/271—Syntactic parsing, e.g. based on context-free grammar [CFG], unification grammars
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/28—Processing or translating of natural language
    • G06F17/2872—Rule based translation
    • G06F17/2881—Natural language generation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/27—Automatic analysis, e.g. parsing
    • G06F17/2765—Recognition
    • G06F17/277—Lexical analysis, e.g. tokenisation, collocates
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/30—Information retrieval; Database structures therefor; File system structures therefor
    • G06F17/3061—Information retrieval; Database structures therefor; File system structures therefor of unstructured textual data
    • G06F17/30634—Querying
    • G06F17/30657—Query processing
    • G06F17/30675—Query execution
    • G06F17/30684—Query execution using natural language analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/27—Automatic analysis, e.g. parsing
    • G06F17/2785—Semantic analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/27—Automatic analysis, e.g. parsing
    • G06F17/274—Grammatical analysis; Style critique
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/20—Handling natural language data
    • G06F17/28—Processing or translating of natural language
    • G06F17/2809—Data driven translation
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/18—Speech classification or search using natural language modelling
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06N—COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computer systems utilising knowledge based models
    • G06N5/02—Knowledge representation
    • G06N5/022—Knowledge engineering, knowledge acquisition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06F—ELECTRICAL DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/50—Computer-aided design
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING; COUNTING
    • G06N—COMPUTER SYSTEMS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N99/00—Subject matter not provided for in other groups of this subclass

Similar Documents

Publication Publication Date Title
Janda et al. Syntactic, semantic and sentiment analysis: The joint effect on automated essay evaluation
d'Sa et al. Bert and fasttext embeddings for automatic detection of toxic speech
Haller et al. Survey on automated short answer grading with deep learning: from word embeddings to transformers
Stanovsky et al. Supervised open information extraction
Zhang et al. AMBERT: A pre-trained language model with multi-grained tokenization
Ghavidel et al. Using BERT and XLNET for the Automatic Short Answer Grading Task.
Bokka et al. Deep Learning for Natural Language Processing: Solve your natural language processing problems with smart deep neural networks
Zhang et al. Sentiment classification model for Chinese micro-blog comments based on key sentences extraction
Jang et al. Beyond distributional hypothesis: Let language models learn meaning-text correspondence
Sebastian et al. Bert for natural language processing in bahasa Indonesia
Viswanath et al. Fairpy: A toolkit for evaluation of social biases and their mitigation in large language models
Lin et al. Sentiment analysis on COVID tweets using COVID-Twitter-BERT with auxiliary sentence approach
Wahab et al. Dibert: Dependency injected bidirectional encoder representations from transformers
Tokpo et al. Fairflow: an automated approach to model-based counterfactual data augmentation for nlp
Trinh et al. Combining lexicon-based and learning-based methods for sentiment analysis for product reviews in Vietnamese language
Thabah et al. A deep connection to Khasi language through pre-trained embedding
Bhowmick et al. Character-level inclusive transformer architecture for information gain in low resource code-mixed language
Chobey et al. Can training neural language models on a curriculum with developmentally plausible data improve alignment with human reading behavior?
Ananth et al. Grammatical tagging for the Kannada text documents using hybrid bidirectional long-short term memory model
Huang et al. Discourse-aware graph networks for textual logical reasoning
Hou et al. AUTOMATIC ASSESSMENT OF STUDENTS'FREE-TEXT ANSWERS WITH DIFFERENT LEVELS
Hourali et al. Coreference resolution using neural mcdm and fuzzy weighting technique
Muaidi Levenberg-Marquardt learning neural network for part-of-speech tagging of Arabic sentences
Pei-Chi et al. On learning psycholinguistics tools for english-based creole languages using social media data
Raja et al. The grammatical structure used by a Tamil lyricist: a linear regression model with natural language processing