[go: up one dir, main page]

WO2014070872A3 - System and method for multimodal interaction with reduced distraction in operating vehicles - Google Patents

System and method for multimodal interaction with reduced distraction in operating vehicles Download PDF

Info

Publication number
WO2014070872A3
WO2014070872A3 PCT/US2013/067477 US2013067477W WO2014070872A3 WO 2014070872 A3 WO2014070872 A3 WO 2014070872A3 US 2013067477 W US2013067477 W US 2013067477W WO 2014070872 A3 WO2014070872 A3 WO 2014070872A3
Authority
WO
WIPO (PCT)
Prior art keywords
service request
operating vehicles
multimodal interaction
input
reduced distraction
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2013/067477
Other languages
French (fr)
Other versions
WO2014070872A2 (en
Inventor
Fuliang Weng
Zhe Feng
Zhongnan Shen
Kui Xu
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Robert Bosch GmbH
Original Assignee
Robert Bosch GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Robert Bosch GmbH filed Critical Robert Bosch GmbH
Publication of WO2014070872A2 publication Critical patent/WO2014070872A2/en
Publication of WO2014070872A3 publication Critical patent/WO2014070872A3/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/34—Route searching; Route guidance
    • G01C21/36—Input/output arrangements for on-board computers
    • G01C21/3605—Destination input or retrieval
    • G01C21/362—Destination input or retrieval received from an external device or application, e.g. PDA, mobile phone or calendar application
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223—Execution procedure of a spoken command
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/226—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics

Landscapes

  • Engineering & Computer Science (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Acoustics & Sound (AREA)
  • Computational Linguistics (AREA)
  • Automation & Control Theory (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

A method of interaction with an in-vehicle information system includes receiving first and second inputs from an operator with first and second input devices, respectively. The method further includes identifying a service request corresponding to the first input, and a parameter of the service request with a value that is included in the second input with controller in the in-vehicle information system. The controller executes stored program instructions to perform the identified service request with reference to the identified parameter.
PCT/US2013/067477 2012-10-30 2013-10-30 System and method for multimodal interaction with reduced distraction in operating vehicles Ceased WO2014070872A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201261720180P 2012-10-30 2012-10-30
US61/720,180 2012-10-30

Publications (2)

Publication Number Publication Date
WO2014070872A2 WO2014070872A2 (en) 2014-05-08
WO2014070872A3 true WO2014070872A3 (en) 2014-06-26

Family

ID=49627040

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2013/067477 Ceased WO2014070872A2 (en) 2012-10-30 2013-10-30 System and method for multimodal interaction with reduced distraction in operating vehicles

Country Status (1)

Country Link
WO (1) WO2014070872A2 (en)

Families Citing this family (95)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8677377B2 (en) 2005-09-08 2014-03-18 Apple Inc. Method and apparatus for building an intelligent automated assistant
US9318108B2 (en) 2010-01-18 2016-04-19 Apple Inc. Intelligent automated assistant
US8977255B2 (en) 2007-04-03 2015-03-10 Apple Inc. Method and system for operating a multi-function portable electronic device using voice-activation
US8676904B2 (en) 2008-10-02 2014-03-18 Apple Inc. Electronic devices with voice command and contextual data processing capabilities
US10255566B2 (en) 2011-06-03 2019-04-09 Apple Inc. Generating and processing task items that represent tasks to perform
US10276170B2 (en) 2010-01-18 2019-04-30 Apple Inc. Intelligent automated assistant
US10417037B2 (en) 2012-05-15 2019-09-17 Apple Inc. Systems and methods for integrating third party services with a digital assistant
BR112015018905B1 (en) 2013-02-07 2022-02-22 Apple Inc Voice activation feature operation method, computer readable storage media and electronic device
US10652394B2 (en) 2013-03-14 2020-05-12 Apple Inc. System and method for processing voicemail
US10748529B1 (en) 2013-03-15 2020-08-18 Apple Inc. Voice activated device for use with a voice-based digital assistant
AU2014278592B2 (en) 2013-06-09 2017-09-07 Apple Inc. Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant
US10176167B2 (en) 2013-06-09 2019-01-08 Apple Inc. System and method for inferring user intent from speech inputs
CN105453026A (en) 2013-08-06 2016-03-30 苹果公司 Auto-activating smart responses based on activities from remote devices
US10170123B2 (en) 2014-05-30 2019-01-01 Apple Inc. Intelligent assistant for home automation
US9715875B2 (en) 2014-05-30 2017-07-25 Apple Inc. Reducing the need for manual start/end-pointing and trigger phrases
US9430463B2 (en) 2014-05-30 2016-08-30 Apple Inc. Exemplar-based natural language processing
EP3149728B1 (en) 2014-05-30 2019-01-16 Apple Inc. Multi-command single utterance input method
US9338493B2 (en) 2014-06-30 2016-05-10 Apple Inc. Intelligent automated assistant for TV user interactions
EP3166023A4 (en) * 2014-07-04 2018-01-24 Clarion Co., Ltd. In-vehicle interactive system and in-vehicle information appliance
US10127911B2 (en) 2014-09-30 2018-11-13 Apple Inc. Speaker identification and unsupervised speaker adaptation techniques
US9668121B2 (en) 2014-09-30 2017-05-30 Apple Inc. Social reminders
US10152299B2 (en) 2015-03-06 2018-12-11 Apple Inc. Reducing response latency of intelligent automated assistants
US9721566B2 (en) 2015-03-08 2017-08-01 Apple Inc. Competing devices responding to voice triggers
US9886953B2 (en) 2015-03-08 2018-02-06 Apple Inc. Virtual assistant activation
DE102015005853A1 (en) * 2015-05-06 2016-11-10 Airbus Ds Electronics And Border Security Gmbh Electronic system and method for planning and executing a vehicle mission
US10460227B2 (en) 2015-05-15 2019-10-29 Apple Inc. Virtual assistant in a communication session
US10083688B2 (en) 2015-05-27 2018-09-25 Apple Inc. Device voice control for selecting a displayed affordance
US10200824B2 (en) 2015-05-27 2019-02-05 Apple Inc. Systems and methods for proactively identifying and surfacing relevant content on a touch-sensitive device
US9578173B2 (en) 2015-06-05 2017-02-21 Apple Inc. Virtual assistant aided communication with 3rd party service in a communication session
US20160378747A1 (en) 2015-06-29 2016-12-29 Apple Inc. Virtual assistant for media playback
US10671428B2 (en) 2015-09-08 2020-06-02 Apple Inc. Distributed personal assistant
US10747498B2 (en) 2015-09-08 2020-08-18 Apple Inc. Zero latency digital assistant
US10331312B2 (en) 2015-09-08 2019-06-25 Apple Inc. Intelligent automated assistant in a media environment
US10740384B2 (en) 2015-09-08 2020-08-11 Apple Inc. Intelligent automated assistant for media search and playback
US11587559B2 (en) 2015-09-30 2023-02-21 Apple Inc. Intelligent device identification
US10691473B2 (en) 2015-11-06 2020-06-23 Apple Inc. Intelligent automated assistant in a messaging environment
US10956666B2 (en) 2015-11-09 2021-03-23 Apple Inc. Unconventional virtual assistant interactions
US10223066B2 (en) 2015-12-23 2019-03-05 Apple Inc. Proactive assistance based on dialog communication between devices
US11227589B2 (en) 2016-06-06 2022-01-18 Apple Inc. Intelligent list reading
US12223282B2 (en) 2016-06-09 2025-02-11 Apple Inc. Intelligent automated assistant in a home environment
US10586535B2 (en) 2016-06-10 2020-03-10 Apple Inc. Intelligent digital assistant in a multi-tasking environment
DK201670540A1 (en) 2016-06-11 2018-01-08 Apple Inc Application integration with a digital assistant
US12197817B2 (en) 2016-06-11 2025-01-14 Apple Inc. Intelligent device arbitration and control
DK179415B1 (en) 2016-06-11 2018-06-14 Apple Inc Intelligent device arbitration and control
DE102016221564A1 (en) 2016-10-13 2018-04-19 Bayerische Motoren Werke Aktiengesellschaft Multimodal dialogue in a motor vehicle
US10497346B2 (en) * 2017-01-04 2019-12-03 2236008 Ontario Inc. Three-dimensional simulation system
US11204787B2 (en) 2017-01-09 2021-12-21 Apple Inc. Application integration with a digital assistant
DK201770383A1 (en) 2017-05-09 2018-12-14 Apple Inc. User interface for correcting recognition errors
DK180048B1 (en) 2017-05-11 2020-02-04 Apple Inc. MAINTAINING THE DATA PROTECTION OF PERSONAL INFORMATION
US10726832B2 (en) 2017-05-11 2020-07-28 Apple Inc. Maintaining privacy of personal information
DK179496B1 (en) 2017-05-12 2019-01-15 Apple Inc. USER-SPECIFIC Acoustic Models
DK201770428A1 (en) 2017-05-12 2019-02-18 Apple Inc. Low-latency intelligent automated assistant
DK179745B1 (en) 2017-05-12 2019-05-01 Apple Inc. SYNCHRONIZATION AND TASK DELEGATION OF A DIGITAL ASSISTANT
CN110651324B (en) * 2017-05-15 2023-05-16 苹果公司 Multi-modal interface
DK201770411A1 (en) 2017-05-15 2018-12-20 Apple Inc. MULTI-MODAL INTERFACES
US20180336892A1 (en) 2017-05-16 2018-11-22 Apple Inc. Detecting a trigger of a digital assistant
DK179560B1 (en) 2017-05-16 2019-02-18 Apple Inc. Far-field extension for digital assistant services
US20180336275A1 (en) 2017-05-16 2018-11-22 Apple Inc. Intelligent automated assistant for media exploration
US10818288B2 (en) 2018-03-26 2020-10-27 Apple Inc. Natural assistant interaction
US11145294B2 (en) 2018-05-07 2021-10-12 Apple Inc. Intelligent automated assistant for delivering content from user experiences
US10928918B2 (en) 2018-05-07 2021-02-23 Apple Inc. Raise to speak
DK179822B1 (en) 2018-06-01 2019-07-12 Apple Inc. Voice interaction at a primary device to access call functionality of a companion device
US10892996B2 (en) 2018-06-01 2021-01-12 Apple Inc. Variable latency device coordination
DK201870355A1 (en) 2018-06-01 2019-12-16 Apple Inc. Virtual assistant operation in multi-device environments
DK180639B1 (en) 2018-06-01 2021-11-04 Apple Inc DISABILITY OF ATTENTION-ATTENTIVE VIRTUAL ASSISTANT
US11027822B2 (en) * 2018-06-21 2021-06-08 Rockwell Collins, Inc. Control system for touchless operation of mechanical input devices
US11010561B2 (en) 2018-09-27 2021-05-18 Apple Inc. Sentiment prediction from textual data
US11462215B2 (en) 2018-09-28 2022-10-04 Apple Inc. Multi-modal inputs for voice commands
US10839159B2 (en) 2018-09-28 2020-11-17 Apple Inc. Named entity normalization in a spoken dialog system
US11170166B2 (en) 2018-09-28 2021-11-09 Apple Inc. Neural typographical error modeling via generative adversarial networks
US11475898B2 (en) 2018-10-26 2022-10-18 Apple Inc. Low-latency multi-speaker speech recognition
US11638059B2 (en) 2019-01-04 2023-04-25 Apple Inc. Content playback on multiple devices
US12039215B2 (en) * 2019-01-07 2024-07-16 Cerence Operating Company Multimodal input processing for vehicle computer
US11348573B2 (en) 2019-03-18 2022-05-31 Apple Inc. Multimodality in digital assistant systems
DK201970509A1 (en) 2019-05-06 2021-01-15 Apple Inc Spoken notifications
US11423908B2 (en) 2019-05-06 2022-08-23 Apple Inc. Interpreting spoken requests
US11475884B2 (en) 2019-05-06 2022-10-18 Apple Inc. Reducing digital assistant latency when a language is incorrectly determined
US11307752B2 (en) 2019-05-06 2022-04-19 Apple Inc. User configurable task triggers
US11140099B2 (en) 2019-05-21 2021-10-05 Apple Inc. Providing message response suggestions
US11496600B2 (en) 2019-05-31 2022-11-08 Apple Inc. Remote execution of machine-learned models
DK180129B1 (en) 2019-05-31 2020-06-02 Apple Inc. USER ACTIVITY SHORTCUT SUGGESTIONS
US11289073B2 (en) 2019-05-31 2022-03-29 Apple Inc. Device text to speech
DK201970511A1 (en) 2019-05-31 2021-02-15 Apple Inc Voice identification in digital assistant systems
US11227599B2 (en) 2019-06-01 2022-01-18 Apple Inc. Methods and user interfaces for voice-based control of electronic devices
US11360641B2 (en) 2019-06-01 2022-06-14 Apple Inc. Increasing the relevance of new available information
US11348581B2 (en) * 2019-07-12 2022-05-31 Qualcomm Incorporated Multi-modal user interface
WO2021056255A1 (en) 2019-09-25 2021-04-01 Apple Inc. Text detection using global geometry estimators
CN111008532B (en) * 2019-12-12 2023-09-12 广州小鹏汽车科技有限公司 Voice interaction method, vehicle and computer readable storage medium
US12301635B2 (en) 2020-05-11 2025-05-13 Apple Inc. Digital assistant hardware abstraction
US11061543B1 (en) 2020-05-11 2021-07-13 Apple Inc. Providing relevant data items based on context
US11038934B1 (en) 2020-05-11 2021-06-15 Apple Inc. Digital assistant hardware abstraction
US11755276B2 (en) 2020-05-12 2023-09-12 Apple Inc. Reducing description length based on confidence
US11490204B2 (en) 2020-07-20 2022-11-01 Apple Inc. Multi-device audio adjustment coordination
US11438683B2 (en) 2020-07-21 2022-09-06 Apple Inc. User identification using headphones
US12434134B2 (en) * 2023-07-28 2025-10-07 Cerence Operating Company Multimodal sensor fusion for building virtual gamepad for in-car games

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002063599A1 (en) * 2001-02-05 2002-08-15 International Business Machines Corporation System and method for multi-modal focus detection, referential ambiguity resolution and mood classification using multi-modal input
US20050288934A1 (en) * 2004-06-29 2005-12-29 Canon Kabushiki Kaisha Multimodal input method
US20100241431A1 (en) * 2009-03-18 2010-09-23 Robert Bosch Gmbh System and Method for Multi-Modal Input Synchronization and Disambiguation

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7716056B2 (en) 2004-09-27 2010-05-11 Robert Bosch Corporation Method and system for interactive conversational dialogue for cognitively overloaded device users

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002063599A1 (en) * 2001-02-05 2002-08-15 International Business Machines Corporation System and method for multi-modal focus detection, referential ambiguity resolution and mood classification using multi-modal input
US20050288934A1 (en) * 2004-06-29 2005-12-29 Canon Kabushiki Kaisha Multimodal input method
US20100241431A1 (en) * 2009-03-18 2010-09-23 Robert Bosch Gmbh System and Method for Multi-Modal Input Synchronization and Disambiguation

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
BASTIAN PFLEGING ET AL: "Multimodal interaction in the car", PROCEEDINGS OF THE 4TH INTERNATIONAL CONFERENCE ON AUTOMOTIVE USER INTERFACES AND INTERACTIVE VEHICULAR APPLICATIONS, AUTOMOTIVEUI '12, 19 October 2012 (2012-10-19), New York, New York, USA, pages 155, XP055114585, ISBN: 978-1-45-031751-1, DOI: 10.1145/2390256.2390282 *
MICHAEL FELD ET AL: "The Automotive Ontology: Managing Knowledge Inside the Vehicle and Sharing it Between Cars", PROCEEDINGS OF THE 3RD INTERNATIONAL CONFERENCE ON AUTOMOTIVE USER INTERFACES AND INTERACTIVE VEHICULAR APPLICATIONS, AUTOMOTIVEUI '11, 2 December 2011 (2011-12-02), NY, USA, pages 79 - 86, XP055114619, Retrieved from the Internet <URL:http://dl.acm.org/ft_gateway.cfm?id=2381429&ftid=1295256&dwn=1&CFID=314795502&CFTOKEN=10673073> [retrieved on 20140417], DOI: 10.1145/2381416.2381429 *

Also Published As

Publication number Publication date
WO2014070872A2 (en) 2014-05-08

Similar Documents

Publication Publication Date Title
WO2014070872A3 (en) System and method for multimodal interaction with reduced distraction in operating vehicles
MX2016011207A (en) In-vehicle tutorial.
MX2020013467A (en) System, method, and app for managing waste services.
EP2639752A3 (en) Method and apparatus for providing late return detection of a shared vehicle
EP3083297A4 (en) Air conditioning system, method for controlling the same and hybrid vehicle
CA2975617C (en) Programmatically determining location information in connection with a transport service
WO2014091238A3 (en) Range extender control
GB201310192D0 (en) Powertrain control system and method
EP2899710B8 (en) On-demand vehicle operation management device, on-demand vehicle operation management method, and on-demand vehicle operation management system
EP3967545A3 (en) Charging of a fleet of electric vehicles
PL2928734T3 (en) Method, system and control device for actuating an immobiliser for trailer vehicles
WO2011162554A3 (en) Component for network system and method for controlling same
WO2012135737A3 (en) Rules execution platform system and method
GB2514291A (en) Method and system for prioritizing a plurality of variable speed devices
EP2695127A4 (en) System and method for viewership validation based on cross-device contextual inputs
EP3057076A4 (en) Parked vehicle detection device, vehicle management system, and control method
EP2738478A3 (en) Intelligent comfort management using natural language processing to interface with a comfort system controller
CA2876605C (en) Diagnostic system for a vehicle
WO2014009214A3 (en) Method and device for operating a vehicle
WO2014012717A8 (en) Conditional interaction control for a virtual object
EP3276876A4 (en) Management device, vehicle, management method, and computer program
WO2013092187A3 (en) Method and device for adapting a voltage limit in an electrical system
GB2515659B (en) Self tuning universal steering control system, method, and system for off-road vehicles
WO2012126422A3 (en) Method, device and system for controlling account management operation
WO2015036416A3 (en) Device and method for controlling mobility

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13795032

Country of ref document: EP

Kind code of ref document: A2

122 Ep: pct application non-entry in european phase

Ref document number: 13795032

Country of ref document: EP

Kind code of ref document: A2