JP2026019055A - system - Google Patents

Abstract

A system is provided.
[Solution] A system including a means for acquiring a user's location information, a means for acquiring a user's heart rate information, a means for transmitting the acquired location information and heart rate information to a server, a means for receiving the transmitted location information and heart rate information, a means for acquiring map information around the current location from a map service based on the received location information, a means for selecting appropriate music from a music distribution service based on the received heart rate information, a means for acquiring the latest disaster prevention information corresponding to the area, a means for integrating the acquired map information, music, and disaster prevention information and generating personalized information to provide to the user, a means for transmitting the generated personalized information to a terminal, and terminal means.
[Selected Figure] Figure 1

Description

The technology disclosed herein relates to a system.

Patent document 1 discloses a persona chatbot control method executed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Japanese Patent Application Laid-Open No. 2022-180282

Current music streaming services have difficulty providing real-time information based on the user's situation, and simply playing a playlist does not fully meet user needs. They also lack mechanisms for providing timely disaster prevention information or region-specific information (e.g., store coupons). There is a growing need for highly personalized information systems that users can use with peace of mind while running or walking.

To solve this problem, the present invention configures a system based on the following: First, a GPS sensor is used as a means of acquiring the user's location information. Next, a means is provided for acquiring the user's heart rate information, which is periodically sent to a server. The server receives the transmitted location information and heart rate information, and obtains map information for the area around the current location from a map service based on the location information. It also selects appropriate music from a music distribution service based on the heart rate information, and obtains the latest disaster prevention information for the area. This information is integrated to generate personalized information to provide to the user, and sent to the terminal. This allows the user to receive music, map information, disaster prevention information, and area-specific information tailored to their situation in real time.

"User" refers to an individual who uses this system and provides location and heart rate information.

"Location Information" refers to data that indicates a user's current geographic location.

"Heart rate information" refers to biometric data such as the user's heart rate and heart rhythm.

"Terminal" refers to a device carried or worn by a user that has sensors and communication capabilities.

"Server" refers to a central system capable of receiving, analyzing, and synthesizing information.

"GPS Sensor" refers to a Global Positioning System (GPS) sensor device used to measure a specific location on Earth.

"Heart rate sensor" refers to a biometric sensor device for measuring a user's heart rate.

"Map Service" refers to an online service that provides geographic information.

"Music distribution service" refers to an online service that distributes music via the Internet.

"Disaster prevention information" refers to information regarding emergencies such as natural disasters.

"Personalized information" refers to information that is customized to a user's individual circumstances and preferences.

"Tempo" refers to the speed or rhythmic pace of music.

"Coupon information" refers to information that offers discounts or special offers for specific services or products.

1 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a first embodiment. 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. FIG. 10 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a second embodiment. FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. FIG. 10 is a conceptual diagram illustrating an example of the configuration of a data processing system according to a third embodiment. FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. 1 shows an emotion map onto which multiple emotions are mapped. 1 shows an emotion map onto which multiple emotions are mapped. FIG. 2 is a sequence diagram illustrating a processing flow of the data processing system according to the first embodiment. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in application example 1. FIG. 10 is a sequence diagram showing the flow of processing of the data processing system in the second embodiment when an emotion engine is combined. FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined.

Below, an example of an embodiment of a system relating to the technology disclosed herein will be described with reference to the accompanying drawings.

First, let me explain the terminology used in the following explanation.

In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, a processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose Computing on Graphics Processing Units), an APU (Accelerated Processing Unit), etc.

In the following embodiments, coded random access memory (RAM) is memory in which information is temporarily stored and is used by the processor as work memory.

In the following embodiments, the coded storage refers to one or more non-volatile storage devices that store various programs, parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

In the following embodiments, a communication I/F (Interface) with a symbol is an interface that includes a communication processor, an antenna, etc. The communication I/F controls communication between multiple computers. Examples of communication standards that can be applied to the communication I/F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

In the following embodiments, "A and/or B" is synonymous with "at least one of A and B." In other words, "A and/or B" means that it may be just A, just B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and/or B" also applies when three or more things are expressed connected by "and/or."

[First embodiment]

Figure 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I/F 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

The reception device 38 is equipped with a touch panel 38A, a microphone 38B, etc., and receives user input. The touch panel 38A detects contact with an indicator (e.g., a pen or finger) to receive user input via the indicator. The microphone 38B detects the user's voice to receive user input via voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

The output device 40 is equipped with a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible by the user 20 (e.g., audio and/or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54.

Figure 2 shows an example of the main functions of the data processing device 12 and smart device 14.

As shown in FIG. 2, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Form for implementing the invention

This invention is a system that provides personalized information in real time based on a user's location information and heart rate information. This system basically consists of three elements: a server, a terminal, and a user. The roles and specific functions of each are explained below.

User

Users are individuals who use this system and provide location and heart rate information. Users wear wearable devices (e.g., smart watches, earphones, audio glasses, etc.) through which they acquire, send, and receive information.

Device

The device is responsible for obtaining location and heart rate information from the user and sending it to the server. The main processes performed by the device are described below.

1. Obtaining location information:

The device uses its built-in GPS sensor to obtain the user's current location.

The acquired location information is updated periodically.

2. Obtaining heart rate information:

The device uses a built-in heart rate sensor to measure the user's heart rate.

Measured heart rate information is also updated regularly.

3. Transmission of information:

The periodically acquired location and heart rate information is packaged into packets and sent to the server.

Server

The server receives and analyzes information sent from the device and integrates various data to provide the user with the necessary information. The main processes performed by the server are described below.

1. Receiving information:

The server receives the location information and heart rate information sent from the device.

2. Obtaining map information:

The server calls the map service API based on the received location information and obtains map information for the area around the current location.

3. Music Selection:

The server analyzes the heart rate based on the received heart rate information and retrieves appropriate music from the music distribution service API.

Example: If your heart rate is high, choose up-tempo music, and if your heart rate is low, choose relaxation music.

4. Obtaining disaster prevention information:

The server obtains disaster prevention information corresponding to the area from the disaster prevention information service API based on the location information.

5. Generating personalized information:

The acquired map information, music, disaster prevention information, and other personalized information (e.g., store coupons, etc.) are integrated as needed to generate optimal personalized information.

6. Sending personalized information:

The server then sends the generated personalized information back to the device.

Specific examples

For users running

The user is wearing the smartwatch while running.

The device obtains the user's current location using a GPS sensor and measures the heart rate using a heart rate sensor. For example, let's say the heart rate is 160 BPM.

The device sends the acquired location and heart rate information to the server.

The server calls the map service API based on the location information and obtains map information within the park.

Next, because the heart rate is high, up-tempo music is retrieved from the music distribution service API. In addition, the disaster prevention information service API is called to check the necessary disaster prevention information.

Finally, this information is integrated to generate personalized information. This information includes information about the user's current location, selected music, necessary disaster prevention information, etc.

The server sends the generated personalized information to the user's device, which analyzes it and sends notifications to the smartwatch or plays music through earphones.

This system allows users to listen to appropriate music in real time while running, while also receiving necessary map and disaster prevention information, making everyday activities safer and more comfortable.

The processing flow is explained below.

Step 1:

The user puts on the wearable device and starts the system. The device completes initial setup and enables the GPS sensor and heart rate sensor.

Step 2:

The device uses the GPS sensor to obtain the user's current location. This location information is obtained periodically (e.g., every 5 seconds).

Step 3:

The device uses a heart rate sensor to measure the user's heart rate. Heart rate information is also collected periodically (e.g., every 5 seconds).

Step 4:

The device collects location and heart rate information and compiles it into packets, which it then sends to the server.

Step 5:

The server receives the location and heart rate information sent from the device.

Step 6:

Based on the location information received by the server, the map service API is called to obtain map information for the area around the current location.

Step 7:

The server analyzes the received heart rate information and calls the music distribution service API to select music according to the user's heart rate.

Example: If your heart rate is high (e.g., 160 BPM), choose up-tempo songs.

Step 8:

Based on the location information, the server obtains the latest disaster prevention information for the area from the disaster prevention information service API.

Step 9:

The server integrates map information, selected music, and disaster prevention information to generate optimal personalized information.

Step 10:

The server sends the generated personalized information to the device.

Step 11:

The device analyzes the personalized information received from the server.

Step 12:

The device will provide information to the user in an appropriate manner based on the information analyzed.

Example: Play selected music through earphones and notify the smartwatch of current location and disaster prevention information.

This processing flow allows users to receive the information they need in real time. The system aims to improve the user experience by providing the most appropriate information tailored to the user's current situation.

(Example 1)

Next, we will explain Example 1. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

Previous personalized information provision systems did not fully utilize users' location information or heart rate information, making it difficult to provide personalized information in real time. Furthermore, because music selection and disaster prevention information provision were not centralized, users were unable to receive the information they needed in a timely manner, resulting in issues with safety and convenience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for acquiring user location information, means for acquiring user heart rate information, means for transmitting the acquired location information and heart rate information to a central processing unit, means for receiving the transmitted location information and heart rate information, means for acquiring map information near the current location from a map information providing service based on the received location information, means for selecting appropriate music from a music distribution service based on the received heart rate information, means for acquiring the latest disaster prevention information corresponding to the area, means for integrating the acquired map information, music, and disaster prevention information to generate personalized information to provide to the user, means for transmitting the generated personalized information to the terminal, and means for notifying the user of the personalized information received by the terminal. This makes it possible to utilize the user's location information and heart rate information in real time, centrally manage necessary map information, music, and disaster prevention information, and provide it to the user immediately.

"User" means an individual who uses the system and provides location and heart rate information.

"Location information" is data indicating the user's current geographical latitude and longitude.

"Heart rate information" is data that indicates the user's current heart rate.

A "central processing unit" is a device that has the function of receiving and processing location information and heart rate information sent from a terminal.

A "terminal" is a device worn by the user that has the function of acquiring location information and heart rate information and transmitting this information to a central processing unit.

"Map information service" is an online service that provides map information around your current location based on location information.

A "music distribution service" is an online service that selects and provides appropriate music based on heart rate information.

"Disaster prevention information" refers to information including the latest disaster information and evacuation advisories for a specific area.

"Personalized information" is individually tailored information that is generated by integrating acquired map information, music, and disaster prevention information and providing it to users.

"Notification" is the act of presenting information to the user via a device.

This invention is a system that provides personalized information in real time based on a user's location information and heart rate information. This system basically consists of three elements: a server, a terminal, and a user. The roles and specific functions of each are explained below.

User

Users are individuals who use this system and provide location and heart rate information. Users wear wearable devices (e.g., smart watches, earphones, audio glasses, etc.) through which they acquire, send, and receive information.

Device

The device acquires location and heart rate information from the user and sends it to the server. Specifically, it performs the following processes:

1. Obtaining location information

The device uses the built-in GPS sensor to obtain the user's current location. Using the GPS sensor allows for detailed and accurate location information to be obtained.

The device will periodically update this location information to keep it up to date.

2. Obtaining heart rate information

The device measures the user's heart rate using a built-in heart rate sensor. Heart rate sensors often use optical sensors or electrocardiogram sensors.

Your device will regularly update your heart rate information to keep your health information up to date.

3. Transmission of Information

The acquired location and heart rate information is packaged into packets and sent to the server. This is done using HTTP POST requests or WebSocket communication.

Server

The server receives the information sent from the device, performs the necessary data analysis, and generates personalized information based on the acquired data. The main processes performed by the server are described below.

1. Receiving information

The server receives the location and heart rate information sent from the device. This is done using an HTTP server or WebSocket server.

2. Obtaining map information

Based on the received location information, the server calls a map information service API to obtain map information for the area around the current location. Typically, the Google Maps API is used.

3. Music Selection

The server analyzes the heart rate based on the received heart rate information and retrieves appropriate music from the music distribution service API. If the heart rate is high, an up-tempo song is selected, and if it is low, a relaxing song is selected. Spotify API or Apple Music API are commonly used.

4. Obtaining disaster prevention information

Based on the received location information, the server obtains disaster prevention information corresponding to that area from a disaster prevention information service API. For example, there is the Japan Meteorological Agency API.

5. Generating personalized information

The server integrates the acquired map information, music, disaster prevention information, etc., and generates personalized information to provide to users. This process is implemented using server-side languages such as Python and Node.js.

6. Sending personalized information

The server then sends the generated personalized information back to the device.

Specific examples

For users running

The user is wearing the smartwatch while running.

The device obtains the user's current location using the GPS sensor, records it as latitude 35.658581 and longitude 139.745433, and measures the heart rate as 160 BPM using the heart rate sensor.

The device sends the acquired location and heart rate information to the server.

The server calls the map information service API based on the location information and obtains map information within the park.

Since the heart rate is 160 BPM, the server retrieves up-tempo songs from the music distribution service API. It also calls the disaster prevention information service API to retrieve disaster prevention information for the area around the park.

The server integrates this information to generate personalized information, including park maps, music selections, disaster prevention information, and more.

The server sends the generated personalized information to the device, which receives it and displays map information on the smartwatch, plays music through earphones, and notifies users of disaster prevention information.

Example prompt

Specific examples of prompts are as follows:

"You are currently in a park. Your heart rate is 160 BPM. Please play some upbeat music and get information about nearby disasters."

This system allows users to listen to appropriate music in real time while running, while also receiving necessary map and disaster prevention information, making everyday activities safer and more comfortable.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1:

Input: Wearable device worn by the user.

Action: The user puts on a wearable device such as a smartwatch or earphones.

Output: None.

Step 2:

Input: Wearable device.

How it works: The device periodically obtains the user's current location (latitude and longitude) using the built-in GPS sensor.

Specifically, location information is obtained by calling the navigator.geolocation.getCurrentPosition method.

Output: Obtained location information (e.g., latitude 35.658581, longitude 139.745433).

Step 3:

Input: Wearable device.

How it works: The device periodically measures the user's heart rate using the built-in heart rate sensor.

Specifically, the smartwatch collects data from the heart rate sensor and calculates the heart rate.

Output: Acquired heart rate information (e.g., 160 BPM).

Step 4:

Input: Acquired location and heart rate information.

Operation: The device collects location and heart rate information and sends it to the server using an HTTP POST request.

Specifically, data is constructed in JSON format and sent to the "/data" endpoint via an HTTP POST request.

Output: Location and heart rate information sent to the server.

POST /data HTTP/1.1

Host: example.com

Content-Type: application/json

{

"location": {"latitude": 35.658581, "longitude": 139.745433},

"heartRate": 160

}

Step 5:

Input: Location and heart rate information received by the server.

Operation: The server receives location and heart rate information sent from the device and analyzes the data.

Specifically, the server endpoint /data receives the request and parses the JSON-formatted data.

Output: Analyzed location and heart rate information.

Step 6:

Input: Parsed location information.

Operation: The server calls the map information service API based on the analyzed location information and obtains map information for the area around the current location.

Specifically, send an API request like the one below.

GET https://maps.googleapis.com/maps/api/geocode/json?latlng=35.658581,139.745433&key=YOUR_API_KEY

Output: Obtained map information.

Step 7:

Input: Analyzed heart rate information.

Operation: The server calls the music distribution service API based on the analyzed heart rate information and retrieves appropriate music.

If your heart rate is high, choose up-tempo music, and if it's low, choose relaxation music.

Specifically, send an API request like the one below.

GET https://api.spotify.com/v1/recommendations?seed_genres=upbeat&target_tempo=160&key=YOUR_API_KEY

Output: Retrieved music information.

Step 8:

Input: Parsed location information.

Operation: The server calls the disaster prevention information service API based on the analyzed location information and obtains the latest disaster prevention information for the area.

Specifically, send an API request like the one below.

GET https://www.jma.go.jp/bosai/forecast/data/overview_forecast/130000.json

Output: Obtained disaster prevention information.

Step 9:

Input: Obtained map information, music information, and disaster prevention information.

Operation: The server consolidates these and generates personalized information to provide to the user.

Specifically, data is integrated using server-side languages such as Python and Node.js, and personalized information in JSON format is generated.

Output: Generated personalized information.

{

"mapInfo": { ... },

"musicInfo": { ... },

"disasterInfo": { ... }

}

Step 10:

Input: Generated personalization information.

Operation: The server sends the generated personalized information to the device via an HTTP response or push notification.

Specifically, it is returned as an HTTP response or sent using the push notification service.

Output: Personalized information sent to the device.

Step 11:

Input: Personalization information received by the device.

Operation: The device receives and analyzes the personalized information sent from the server.

Specifically, it parses HTTP responses and push notification data.

Output: Parsed personalization information.

Step 12:

Input: Parsed personalization information.

Operation: The device displays the received map information on the screen of the smartwatch or smartphone.

Specifically, the acquired map data is rendered using Google Maps SDK etc.

Output: Displayed map information.

Step 13:

Input: Parsed personalization information.

Operation: The device plays the received music information through the earphones.

Specifically, the obtained music URL is used for streaming playback.

Output: Played music.

Step 14:

Input: Parsed personalization information.

Operation: The device displays the received disaster prevention information using the smartwatch's notification function.

Specifically, we will utilize the function that displays notifications on smartwatches.

Output: Notified disaster prevention information.

(Application Example 1)

Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

Conventional personalized information provision systems have had difficulty efficiently utilizing a user's location information and heart rate information to provide optimal content to users in real time. Furthermore, because these systems rely on limited information sources, it has been difficult to meet the diverse needs of users. The present invention aims to solve these problems and provide a system that provides appropriate music, podcasts, disaster prevention information, and more in real time based on a user's heart rate and location information.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes means for acquiring user location information, means for acquiring user heart rate information, means for acquiring map information around the current location from a map service based on the received location information, means for selecting appropriate music or podcasts from a content distribution service based on the received heart rate information, and means for acquiring the latest disaster prevention information for the area. This allows the user to receive optimal music, podcasts, disaster prevention information, etc. in real time according to their current location and heart rate.

text

"User" refers to an individual who uses this system and provides location information and heart rate information.

"Location information" is data that indicates a user's current geographical location.

"Heart rate information" is data that indicates the user's heart rate and represents the user's health condition and activity level.

The "server" is a system component that receives and analyzes location and heart rate information, and generates and provides various personalized information.

"Map service" is an external service that provides geographical information and map data based on location information.

"Music distribution service" is an external service that provides music tracks in streaming or download format.

"Content distribution service" is an external service that provides multimedia content such as music and podcasts.

"Disaster prevention information" is data that provides information on local disaster risks and emergency response measures.

"Personalized information" is information generated specifically for the user by integrating acquired map information, music, podcasts, and disaster prevention information.

A "terminal" is a device that acquires a user's location information and heart rate information and transmits this information to a server.

A "satellite positioning system sensor" is a device that receives satellite signals and measures location information.

"Tempo-controlled music" refers to music selected based on heart rate and with a rhythm that is appropriate for the user's current activity level.

"Podcast" refers to audio content provided over the Internet and covers a variety of genres, including education, news, and entertainment.

text

This invention is a system that provides personalized information in real time based on a user's location information and heart rate information. This system consists of three elements: the user, the terminal, and the server.

1. User

Users are individuals who use this system and provide location and heart rate information. Specifically, users wear wearable devices such as smartwatches and fitness trackers, through which they acquire and transmit location and heart rate information.

2. Device

The device is responsible for obtaining location and heart rate information from the user and sending it to the server. Specific processes include the following:

Location acquisition: The device uses the built-in satellite positioning system sensor to acquire the user's current location.

Acquiring heart rate information: The device uses the built-in heart rate sensor to measure the user's heart rate.

Information transmission: The acquired location and heart rate information is periodically sent to the server.

3. Server

The server receives and analyzes the information sent from the device and integrates various data to provide the user with the necessary information.

Information reception: The server receives location and heart rate information sent from the device.

Obtaining map information: Calls the map service API based on the received location information and obtains map information around the current location.

Content selection: Analyzes the heart rate based on the received heart rate information and retrieves appropriate music or podcasts from the content distribution service API. For example, if the heart rate is high, upbeat music is selected, and if it is low, a relaxing podcast is selected.

Obtaining disaster prevention information: Obtain the latest disaster prevention information for your area based on your location information from the disaster prevention information service API.

Generating personalized information: Integrates acquired map information, music, podcasts, and disaster prevention information to generate personalized information to provide to users.

Send information: Send the generated personalized information back to your device.

Specific examples

For example, if a user is running, the device will use a satellite positioning system sensor to obtain the user's location information and a heart rate sensor to measure the user's heart rate. Suppose the user's heart rate is 150 BPM. This information is sent to the server, which then calls a map service API to obtain map information for the current location. Based on the high heart rate, the server will obtain up-tempo music from a content distribution service API. In addition, it will obtain the latest local disaster prevention information and combine this information to generate personalized information. Finally, this information is sent to the user's device, allowing the user to receive appropriate music and disaster prevention information in real time.

Example prompt

"If the user's heart rate is 150 BPM, can you recommend some upbeat music suitable for running? Also, tell me the latest news in the area."

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

text

Step 1:

Acquires the user's location and heart rate information. The device uses the built-in satellite positioning system sensor to acquire the user's current location and the heart rate sensor to measure the heart rate. The input is the user's location and heart rate information, and the output is a data packet sent to the server.

Step 2:

The server receives location and heart rate information sent from the device. This is the initial stage of data processing and analysis. The input is the data packets sent from the device, and the output is processable data stored within the server.

Step 3:

The server calls the map service API based on the received location information and obtains map information for the area around the current location. The input is location information, and the output is the obtained map information. Specifically, it sends an API request and receives map information as a response.

Step 4:

The server analyzes the received heart rate information and selects appropriate content (music or podcast). At this time, it calls the content distribution service API based on the heart rate to obtain the appropriate track. The input is the heart rate information, and the output is the selected music or podcast. Specifically, if the heart rate is high, it queries for up-tempo music, and if the heart rate is low, it queries for relaxation podcasts.

Step 5:

The server obtains the latest disaster prevention information for the region based on the location information from the disaster prevention information service API. The input is location information, and the output is disaster prevention information. Specifically, it sends an API request to obtain disaster prevention information for the relevant region, and receives the disaster prevention information as a response.

Step 6:

The server integrates the acquired map information, music, podcasts, and disaster prevention information to generate personalized information to provide to users. The input is the various acquired data, and the output is the integrated personalized information. Specifically, the various data are integrated to generate personalized information in JSON format.

Step 7:

The server sends the generated personalized information to the terminal. The input is the generated personalized information, and the output is a data packet sent to the terminal. Specifically, the server sends data to the terminal and provides information to the user in real time.

Step 8:

The device receives personalized information sent from the server and notifies the user. The input is the data packets sent from the server, and the output is the presentation of information to the user. Specific operations include displaying a notification on the smartwatch display and playing music or podcasts through the earphones.

Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

Form for implementing the invention

This invention is a system that provides personalized content in real time based on a user's location information, heart rate information, and emotional state. This system consists of three elements: a server, a terminal, and a user. It also includes an emotion engine for recognizing the user's emotions. Below, we will explain the role and specific functions of each element, as well as the operation of the entire system.

User

Users are individuals who use this system and provide location information, heart rate information, and emotional information. Users wear wearable devices (e.g., smart watches, earphones, audio glasses, etc.) through which they acquire, send, and receive information.

Device

The device is responsible for obtaining location information, heart rate information, and emotional information from the user and sending it to the server. Below, we will explain in detail the main functions performed by the device.

1. Obtaining location information:

The device uses its built-in GPS sensor to obtain the user's current location.

Location information is updated periodically (e.g., every 5 seconds).

2. Obtaining heart rate information:

The device uses a built-in heart rate sensor to measure the user's heart rate.

Heart rate information is also updated periodically (e.g., every 5 seconds).

3. Acquiring emotional information:

Using the device's built-in camera and microphone, the user's facial expressions and tone of voice are analyzed, and the emotion engine recognizes the user's emotions.

4. Transmission of Information:

The periodically acquired location information, heart rate information, and emotion information are packaged into packets and sent to the server.

Server

The server receives and analyzes information sent from the device and integrates various data to provide the user with the necessary information. Below, we will explain in detail the main functions and processes performed by the server.

1. Receiving information:

The server receives location information, heart rate information, and emotion information sent from the device.

2. Obtaining map information:

The server calls the map service API based on the received location information and obtains map information for the area around the current location.

3. Music Selection:

The server analyzes the received heart rate information and retrieves music that corresponds to the user's heart rate from the music distribution service API.

Example: If your heart rate is high (e.g., 160 BPM), choose an up-tempo song.

Emotional information is also taken into consideration to select songs that suit the user's emotional state.

Example: If your emotional state is "Relaxed," select relaxation music.

4. Obtaining disaster prevention information:

Based on the location information, the server obtains the latest disaster prevention information for the area from the disaster prevention information service API.

5. Generating personalized information:

The acquired map information, selected music, disaster prevention information, and the user's emotional information are integrated to generate optimal personalized information.

6. Sending personalized information:

The server sends the generated personalized information to the device.

Specific examples

For users running

The user is wearing a smartwatch and earphones while running.

The device acquires the user's current location using a GPS sensor and measures their heart rate using a heart rate sensor. For example, let's say the heart rate is 160 BPM. Furthermore, using the device's built-in camera and microphone, the emotion engine recognizes the user's emotional state as "excited."

The device sends this information to the server.

The server calls the map service API based on the location information and obtains map information within the park.

Next, because the heart rate is high, up-tempo music is retrieved from a music distribution service API. Also, because the emotional state is "excited," songs that will maintain tension are selected.

In addition, it calls the disaster prevention information service API and checks the necessary disaster prevention information.

Finally, this information is integrated to generate personalized information, including information about the user's current location, selected music, necessary disaster prevention information, and information based on their emotional state.

The server sends the generated personalized information to the user's device, which analyzes it, notifies the smartwatch, and plays music through the earphones.

This system allows users to receive appropriate music, map information, and disaster prevention information in real time while running. It also takes into account the user's emotional state, making it possible to provide a more personalized experience.

The processing flow is explained below.

Step 1:

The user puts on the wearable device and launches the application. The device activates the GPS sensor and heart rate sensor.

Step 2:

The device obtains the user's current location using the GPS sensor. This location information is updated periodically (e.g., every 5 seconds).

Step 3:

The device measures the user's heart rate using a heart rate sensor. Heart rate information is also updated periodically (e.g., every 5 seconds).

Step 4:

The device's built-in camera periodically captures the user's facial expressions, and the microphone collects the user's tone of voice. Based on this, the emotion engine analyzes the user's emotional state.

Step 5:

The device collects location information, heart rate information, and emotion information and sends them as packets to the server.

Step 6:

The server receives the location information, heart rate information, and emotion information sent from the device.

Step 7:

Based on the location information received by the server, the map service API is called to obtain map information for the area around the current location.

Step 8:

The server analyzes the received heart rate information and retrieves music corresponding to the user's heart rate from the music distribution service API. For example, if the heart rate is 160 BPM, an up-tempo song is selected.

Step 9:

The server uses an emotion engine to analyze the user's emotional state and selects appropriate music based on the results. For example, if the emotional state is "relaxed," it selects relaxation-oriented music.

Step 10:

Based on the location information, the server obtains the latest disaster prevention information for the area from the disaster prevention information service API.

Step 11:

The server integrates the acquired map information, selected music, disaster prevention information, and emotional information to generate optimal personalized information.

Step 12:

The server sends the generated personalized information to the device.

Step 13:

The device analyzes the personalized information received from the server.

Step 14:

The device will provide information to the user in an appropriate manner based on the analyzed information. For example, selected music will be played through earphones, and the smartwatch will notify the user of their current location, disaster prevention information, and emotional advice.

This detailed processing step allows users to efficiently receive the most appropriate information in real time. The system aims to improve the user experience by providing the most appropriate information tailored to the user's current situation and emotional state.

(Example 2)

Next, we will explain Example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

Currently, there are systems that provide personalized information based on a user's location and heart rate information, but adding emotional information to these systems would enable even more accurate personalization. However, such systems do not currently exist, and the provision of appropriate information based on the user's emotional state is insufficient, making improving user satisfaction a challenge.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for acquiring user location information, means for acquiring user heart rate information, means for acquiring user emotional information, means for transmitting the acquired location information, heart rate information, and emotional information to the server, means for receiving the transmitted location information, heart rate information, and emotional information, means for acquiring map information around the current location from a map service based on the received location information, means for selecting appropriate music from a music distribution service based on the received heart rate information and emotional information, means for acquiring the latest disaster prevention information corresponding to the area, means for integrating the acquired map information, music, disaster prevention information, and emotional information to generate personalized information to provide to the user, means for transmitting the generated personalized information to the terminal, and terminal means. This enables the provision of highly accurate personalized information based on more multifaceted information including the user's emotional state.

"User location information" refers to the coordinate information of the user's current location.

"User's heart rate information" refers to data measuring the user's heart rate.

"User emotional information" refers to data that expresses the user's emotional state, and is obtained by analyzing facial expressions, tone of voice, etc.

A "server" is a computer system that receives and analyzes data sent from a terminal, consolidates the necessary information, and sends it to the terminal.

"Terminal" refers to a device worn or used by a user that acquires location information, heart rate information, and emotional information and transmits it to a server.

A "GPS sensor" is a sensor that receives signals from satellites and determines your current location on Earth.

"Map Service" refers to a service that provides geographic information through a specific API.

"Music distribution service" refers to a service that provides music data through a specific API.

"Disaster prevention information" refers to information regarding emergencies such as natural disasters, and is provided according to the region.

"Personalized information" refers to information specific to each user that is generated by integrating each user's location information, heart rate information, emotional information, etc.

An "emotion engine" refers to software or hardware that analyzes a user's facial expressions and tone of voice to recognize their emotional state.

This invention is a system that provides personalized content in real time based on a user's location information, heart rate information, and emotional state. This system consists of three elements: a server, a terminal, and a user. It also includes an emotion engine for recognizing the user's emotions. Below, we will explain the role and specific functions of each element, as well as the operation of the entire system.

User

Users are individuals who use this system and provide location information, heart rate information, and emotional information. Users wear wearable devices (e.g., smartwatches, earphones, etc.) through which they acquire, send, and receive information. Users begin using the system by wearing the smartwatch on their wrist and the earphones in their ears.

Device

The device is responsible for obtaining location information, heart rate information, and emotional information from the user and sending it to the server. Below, we will explain in detail the main functions performed by the device.

1. Obtaining location information:

The device uses the built-in GPS sensor to obtain the user's current location. Location information is updated periodically (e.g., every 5 seconds).

2. Obtaining heart rate information:

The device uses a built-in heart rate sensor to measure the user's heart rate. Heart rate information is also updated periodically (e.g., every 5 seconds).

3. Acquiring emotional information:

Using the device's built-in camera and microphone, the user's facial expressions and tone of voice are analyzed, and the emotion engine recognizes the user's emotions.

4. Transmission of Information:

The periodically acquired location information, heart rate information, and emotion information are packaged into packets and sent to the server.

Server

The server receives and analyzes information sent from the device and integrates various data to provide the user with the necessary information. Below, we will explain in detail the main functions and processes performed by the server.

1. Receiving information:

The server receives location information, heart rate information, and emotion information sent from the device.

2. Obtaining map information:

The server calls the map service API based on the received location information and obtains map information for the area around the current location.

3. Music Selection:

The server analyzes the received heart rate information and retrieves music from the music distribution service API that corresponds to the user's heart rate and emotional state.

Example: If your heart rate is high (e.g., 160 BPM), choose an up-tempo song. If your emotional state is "excited," choose an energetic song.

4. Obtaining disaster prevention information:

Based on the location information, the server obtains the latest disaster prevention information for the area from the disaster prevention information service API.

5. Generating personalized information:

The acquired map information, selected music, disaster prevention information, and the user's emotional information are integrated to generate optimal personalized information.

6. Sending personalized information:

The server sends the generated personalized information to the device.

Specific examples

For users running

Suppose a user is wearing a smartwatch and earphones while running. The smartwatch periodically obtains the user's current location using a GPS sensor and measures their heart rate using a heart rate sensor. For example, if the user's heart rate is 160 BPM, the device sends this information to the server. At the same time, the emotion engine uses the device's built-in camera and microphone to recognize the user's emotional state as "excited."

The server first calls a map service API based on the location information and obtains map information for the park where the user is running. Next, because the heart rate is high, it obtains up-tempo music from a music distribution service API, and because the emotional state is "excited," it selects energetic music. It also calls a disaster prevention information service API to check disaster prevention information related to the user's current location.

By integrating this data (map information, music information, disaster prevention information), the server generates personalized information optimized for the user and sends it to the device. The device then plays the delivered music through earphones and displays the necessary information on the smartwatch screen.

Example prompt

"Please explain the specific processing steps of a system that provides real-time personalized music and map information when a user's heart rate is high and their emotional state is 'excited' while running."

The flow of the identification process in Example 2 will be explained using Figure 13.

Step 1: The user puts on the wearable device.

The user puts on a smartwatch, earphones, or other wearable device. The device is then ready to acquire the user's location, heart rate, and emotional information. The input for this step is the user themselves, and the output is the wearable device they are wearing.

Step 2: The device acquires location information

The device uses the built-in GPS sensor to obtain the user's current location. For example, if the user is in a park, the device obtains their location coordinates. The input is the signal from the GPS sensor, and the output is location coordinate data (e.g., latitude 35.6895 degrees, longitude 139.6917 degrees). The location information is updated periodically (e.g., every 5 seconds).

Step 3: The device acquires heart rate information.

The device uses a built-in heart rate sensor to measure the user's heart rate. For example, the heart rate is measured at 160 BPM. The input is data from the heart rate sensor, and the output is heart rate data. Heart rate information is also updated periodically (for example, every 5 seconds).

Step 4: The device acquires emotional information.

The device uses a camera and microphone to analyze the user's facial expressions and tone of voice. The emotion engine processes the results of this analysis and recognizes the user's emotional state as "excitement." The input is sensor data from the camera and microphone, and the output is emotional information.

Step 5: The device sends the information to the server.

The device collects location, heart rate, and emotional information and compiles it into packets, which are then sent to the server. The input is location, heart rate, and emotional data, and the output is the data sent to the server. Transmission is carried out periodically.

Step 6: The server receives the information

The server receives location information, heart rate information, and emotional information sent from the device. The input is the data sent from the device, and the output is the data stored on the server.

Step 7: The server retrieves map information.

The server calls the map service API based on the received location information and obtains map information for the area around the current location. The input is location data, and the output is map information data (e.g., obtaining geographic information using the Google Maps API).

Step 8: The server selects music.

The server analyzes the received heart rate and emotional information and retrieves music from a music distribution service API that corresponds to the user's heart rate and emotional state. For example, if a user's heart rate is 160 BPM and they are in an "excited" state, it will select an up-tempo, energetic song. The input is heart rate and emotional information data, and the output is the selected music data (e.g., retrieving songs using the Spotify API).

Step 9: The server obtains disaster prevention information.

Based on the location information, the server obtains the latest disaster prevention information for the region from the disaster prevention information service API. The input is location information data, and the output is disaster prevention information data (e.g., using the disaster prevention information service API to obtain the latest information for the region).

Step 10: The server generates personalized information

The server integrates the acquired map information, selected music, disaster prevention information, and emotional information to generate optimal personalized information. The input is map information, music data, disaster prevention information, and emotional information, and the output is integrated personalized information.

Step 11: The server sends personalized information to the device.

The server sends the generated personalized information to the terminal. The input is the personalized information data, and the output is the data sent to the terminal.

Step 12: The device analyzes the information and notifies the user.

The device analyzes the received personalized information and notifies the user of appropriate information. Specifically, map information and disaster prevention information are displayed on the smartwatch's notification screen, and selected music is played through the earphones. The input is personalized information data, and the output is the form of information provided to the user (screen display and music playback).

(Application Example 2)

Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart device 14 will be referred to as the "terminal."

With the proliferation of information in modern society and the diversification of individual user needs, there is a growing need to provide personalized content tailored to the user's real-time situation. In particular, real-time information provision systems based on biometric data such as location information and heart rate contribute to improving the user experience. However, current systems are not yet able to effectively integrate this data and provide personalized content that also responds to the user's emotional state. Therefore, there is a need for a system that provides personalized information in real time based on the user's location information, heart rate information, and emotional state.

The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring user location information, means for acquiring user heart rate information, means for acquiring the user's emotional state, means for transmitting the received location information, heart rate information, and emotional information to the server, means for acquiring map information near the current location from a map service based on the received location information, means for selecting appropriate music from a music distribution service based on the received heart rate information, means for generating personalized content based on the emotional state, means for acquiring the latest disaster prevention information corresponding to the region, means for integrating the acquired map information, music, disaster prevention information, and emotional information to generate personalized information to provide to the user, means for transmitting the generated personalized information to the terminal, and terminal means. This makes it possible to provide personalized information that integrates appropriate map information, music, and disaster prevention information in real time based on the user's real-time biometric data and emotional state.

"Location information" refers to geographic coordinate information that indicates the user's current location.

"Heart rate information" refers to biometric data such as heart rate and heart rhythm obtained from the user's heartbeat.

"Emotional state" refers to information that indicates the user's emotions, such as psychological states such as stress, relaxation, and excitement.

"Personalized information" refers to content and service information that is individually customized based on the user's location information, heart rate information, and emotional state.

"Map information" is data that shows the terrain, roads, facilities, etc. within a geographic space.

A "music distribution service" is a service that distributes music via the Internet.

"Disaster prevention information" refers to the latest information on disasters such as earthquakes, fires, and typhoons.

A "server" is a central processing unit that receives, analyzes, and processes information sent by users.

"Terminal" refers to a device carried or worn by a user, including smartphones and smartwatches.

This invention is a system that acquires a user's location information, heart rate information, and emotional state in real time and provides personalized content. This system is primarily composed of three elements: a server, a terminal, and a user. Below, we will explain in detail the operation of each element and the system as a whole.

User

The user is the person who uses this system and provides location information, heart rate information, and emotional information. The user wears a wearable device such as a smartwatch, earphones, or smart glasses, and the information is collected and sent to the server.

Device

The terminal is responsible for obtaining information from the user and sending it to the server. Its main functions are as follows:

1. Obtaining location information:

The device uses the built-in GPS sensor to obtain the user's current location. Location information is updated periodically.

2. Obtaining heart rate information:

The device uses a built-in heart rate sensor to measure the user's heart rate, and this information is also updated periodically.

3. Acquiring emotional information:

Using the device's built-in camera and microphone, the emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions.

4. Transmission of Information:

The processed data is sent to the server.

Server

The server receives, analyzes, and processes information sent from the device. The main processes are as follows:

1. Receiving information:

Receives location information, heart rate information, and emotion information sent from the device.

2. Obtaining map information:

Based on the received location information, the map service API is called to obtain map information around the current location.

3. Music Selection:

Based on heart rate information, appropriate music is selected from a music distribution service API. For example, if the heart rate is high, an up-tempo song is selected. Emotional information is also taken into consideration, and songs are selected that match the user's emotional state.

4. Emotion-based content generation:

Based on emotional information, it generates content such as videos and podcasts that are optimal for the user's emotional state.

5. Obtaining disaster prevention information:

The latest disaster prevention information is obtained from the disaster prevention information service API based on location information.

6. Generating personalized information:

The acquired map information, selected music, disaster prevention information, and emotional information are integrated to generate personalized information to provide to the user.

7. Transmission of Information:

The final generated personalized information is sent to the device.

Specific examples

For users running

The user wears a smartwatch and earphones while running. The device obtains current location information obtained from the GPS sensor and heart rate measured by the heart rate sensor (e.g., 160 BPM). The emotion engine also recognizes the user's emotional state as "excitement." This information is then periodically sent to the server.

The server calls a map service API based on the received location information and obtains map information for the park. Next, because the heart rate information is high, it obtains up-tempo songs from a music distribution service API and selects music suitable for maintaining excitement based on the emotional state. It also calls a disaster prevention information service API to check any necessary disaster prevention information.

This information is then integrated to generate personalized information, which is then sent back to the user's device. The user can check this information on their smartwatch, and the selected music will play from the earphones.

Example of a prompt for a generative AI model:

"A user is feeling stressed while walking through a city. Their heart rate is 120 BPM. What content would you recommend?"

This system allows users to enjoy content that is best suited to their situation in real time.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

Users acquire information by wearing a wearable device.

Specific actions:

The user wears a wearable device such as a smartwatch, earphones, or smart glasses. The user's current location is acquired using a GPS sensor, and heart rate information is measured using a heart rate sensor. In addition, the device's built-in camera and microphone are used to analyze the user's facial expressions and tone of voice to obtain emotional information.

input:

User location, heart rate, and emotional state

output:

Acquired location information, heart rate information, and emotional information

Step 2:

The device sends the acquired information to the server.

Specific actions:

The device periodically collects location information, heart rate information, and emotional information, and then compiles them into packets and sends them to a server. Communication is carried out via the Internet.

input:

Acquired location information, heart rate information, and emotional information

output:

Location, heart rate, and emotional information sent to the server

Step 3:

The server analyzes the information received.

Specific actions:

The server receives location information, heart rate information, and emotional information sent from the device. It analyzes the received data and identifies and organizes each piece of information.

input:

Transmitted location information, heart rate information, and emotional information

output:

Organized location, heart rate, and emotional information

Step 4:

The server obtains map information.

Specific actions:

The server calls the map service API based on the received location information and obtains map information for the area around the current location. The obtained map information provides detailed geographic data about the user's current location.

input:

Organized location information

output:

Retrieved map information

Step 5:

The server selects the music.

Specific actions:

The server analyzes the received heart rate information and retrieves music corresponding to the user's heart rate from the music distribution service API. It also takes emotional information into account and selects music appropriate to the user's emotional state.

input:

Organized heart rate and emotional information

output:

Selected music

Step 6:

The server acquires disaster prevention information.

Specific actions:

The server retrieves the latest disaster prevention information for the area based on the location information from the disaster prevention information service API. This provides information such as disaster risks near the user's current location.

input:

Organized location information

output:

Disaster prevention information obtained

Step 7:

The server generates personalized information.

Specific actions:

The server integrates the acquired map information, selected music, disaster prevention information, and emotional information to generate personalized information to provide to the user.

input:

Map information, selected music, disaster prevention information, emotional information

output:

Generated personalized information

Step 8:

The server sends the generated personalized information to the device.

Specific actions:

The server sends the generated personalized information to the device. The sent information is analyzed on the user's device and notified or played back appropriately.

input:

Generated personalized information

output:

Personalized information sent from the server

Step 9:

Devices provide personalized information to users

Specific actions:

The device analyzes the received personalized information and provides it to the user via the smartwatch or earphones. Specifically, notifications are displayed on the smartwatch and the selected music is played through the earphones.

input:

Personalized information received

output:

Personalized information provided to users

The specific processing unit 290 transmits the results of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the results of the specific processing. The microphone 38B acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). Examples of data generation model 58 include generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>) and Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the smart device 14.

[Second embodiment]

Figure 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

As shown in FIG. 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I/F 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user 20's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

Figure 4 shows an example of the main functions of the data processing device 12 and smart glasses 214. As shown in Figure 4, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Form for implementing the invention

This invention is a system that provides personalized information in real time based on a user's location information and heart rate information. This system basically consists of three elements: a server, a terminal, and a user. The roles and specific functions of each are explained below.

User

Users are individuals who use this system and provide location and heart rate information. Users wear wearable devices (e.g., smart watches, earphones, audio glasses, etc.) through which they acquire, send, and receive information.

Device

The device is responsible for obtaining location and heart rate information from the user and sending it to the server. The main processes performed by the device are described below.

1. Obtaining location information:

The device uses its built-in GPS sensor to obtain the user's current location.

The acquired location information is updated periodically.

2. Obtaining heart rate information:

The device uses a built-in heart rate sensor to measure the user's heart rate.

Measured heart rate information is also updated regularly.

3. Transmission of information:

The periodically acquired location and heart rate information is packaged into packets and sent to the server.

Server

The server receives and analyzes information sent from the device and integrates various data to provide the user with the necessary information. The main processes performed by the server are described below.

1. Receiving information:

The server receives the location information and heart rate information sent from the device.

2. Obtaining map information:

The server calls the map service API based on the received location information and obtains map information for the area around the current location.

3. Music Selection:

The server analyzes the heart rate based on the received heart rate information and retrieves appropriate music from the music distribution service API.

Example: If your heart rate is high, choose up-tempo music, and if your heart rate is low, choose relaxation music.

4. Obtaining disaster prevention information:

The server obtains disaster prevention information corresponding to the area from the disaster prevention information service API based on the location information.

5. Generating personalized information:

The acquired map information, music, disaster prevention information, and other personalized information (e.g., store coupons, etc.) are integrated as needed to generate optimal personalized information.

6. Sending personalized information:

The server then sends the generated personalized information back to the device.

Specific examples

For users running

The user is wearing the smartwatch while running.

The device obtains the user's current location using a GPS sensor and measures the heart rate using a heart rate sensor. For example, let's say the heart rate is 160 BPM.

The device sends the acquired location and heart rate information to the server.

The server calls the map service API based on the location information and obtains map information within the park.

Next, because the heart rate is high, up-tempo music is retrieved from the music distribution service API. In addition, the disaster prevention information service API is called to check the necessary disaster prevention information.

Finally, this information is integrated to generate personalized information. This information includes information about the user's current location, selected music, necessary disaster prevention information, etc.

The server sends the generated personalized information to the user's device, which analyzes it and sends notifications to the smartwatch or plays music through earphones.

This system allows users to listen to appropriate music in real time while running, while also receiving necessary map and disaster prevention information, making everyday activities safer and more comfortable.

The processing flow is explained below.

Step 1:

The user puts on the wearable device and starts the system. The device completes initial setup and enables the GPS sensor and heart rate sensor.

Step 2:

The device uses the GPS sensor to obtain the user's current location. This location information is obtained periodically (e.g., every 5 seconds).

Step 3:

The device uses a heart rate sensor to measure the user's heart rate. Heart rate information is also collected periodically (e.g., every 5 seconds).

Step 4:

The device collects location and heart rate information and compiles it into packets, which it then sends to the server.

Step 5:

The server receives the location and heart rate information sent from the device.

Step 6:

Based on the location information received by the server, the map service API is called to obtain map information for the area around the current location.

Step 7:

The server analyzes the received heart rate information and calls the music distribution service API to select music according to the user's heart rate.

Example: If your heart rate is high (e.g., 160 BPM), choose up-tempo songs.

Step 8:

Based on the location information, the server obtains the latest disaster prevention information for the area from the disaster prevention information service API.

Step 9:

The server integrates map information, selected music, and disaster prevention information to generate optimal personalized information.

Step 10:

The server sends the generated personalized information to the device.

Step 11:

The device analyzes the personalized information received from the server.

Step 12:

The device will provide information to the user in an appropriate manner based on the information analyzed.

Example: Play selected music through earphones and notify the smartwatch of current location and disaster prevention information.

This processing flow allows users to receive the information they need in real time. The system aims to improve the user experience by providing the most appropriate information tailored to the user's current situation.

(Example 1)

Next, we will explain Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Previous personalized information provision systems did not fully utilize users' location information or heart rate information, making it difficult to provide personalized information in real time. Furthermore, because music selection and disaster prevention information provision were not centralized, users were unable to receive the information they needed in a timely manner, resulting in issues with safety and convenience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for acquiring user location information, means for acquiring user heart rate information, means for transmitting the acquired location information and heart rate information to a central processing unit, means for receiving the transmitted location information and heart rate information, means for acquiring map information near the current location from a map information providing service based on the received location information, means for selecting appropriate music from a music distribution service based on the received heart rate information, means for acquiring the latest disaster prevention information corresponding to the area, means for integrating the acquired map information, music, and disaster prevention information to generate personalized information to provide to the user, means for transmitting the generated personalized information to the terminal, and means for notifying the user of the personalized information received by the terminal. This makes it possible to utilize the user's location information and heart rate information in real time, centrally manage necessary map information, music, and disaster prevention information, and provide it to the user immediately.

"User" means an individual who uses the system and provides location and heart rate information.

"Location information" is data indicating the user's current geographical latitude and longitude.

"Heart rate information" is data that indicates the user's current heart rate.

A "central processing unit" is a device that has the function of receiving and processing location information and heart rate information sent from a terminal.

A "terminal" is a device worn by the user that has the function of acquiring location information and heart rate information and transmitting this information to a central processing unit.

"Map information service" is an online service that provides map information around your current location based on your location information.

A "music distribution service" is an online service that selects and provides appropriate music based on heart rate information.

"Disaster prevention information" refers to information including the latest disaster information and evacuation advisories for a specific area.

"Personalized information" is individually tailored information that is generated by integrating acquired map information, music, and disaster prevention information and providing it to users.

"Notification" is the act of presenting information to the user via a device.

This invention is a system that provides personalized information in real time based on a user's location information and heart rate information. This system basically consists of three elements: a server, a terminal, and a user. The roles and specific functions of each are explained below.

User

Users are individuals who use this system and provide location and heart rate information. Users wear wearable devices (e.g., smart watches, earphones, audio glasses, etc.) through which they acquire, send, and receive information.

Device

The device acquires location and heart rate information from the user and sends it to the server. Specifically, it performs the following processes:

1. Obtaining location information

The device uses the built-in GPS sensor to obtain the user's current location. Using the GPS sensor allows for detailed and accurate location information to be obtained.

The device will periodically update this location information to keep it up to date.

2. Obtaining heart rate information

The device measures the user's heart rate using a built-in heart rate sensor. Heart rate sensors often use optical sensors or electrocardiogram sensors.

Your device will regularly update your heart rate information to keep your health information up to date.

3. Transmission of Information

The acquired location and heart rate information is packaged into packets and sent to the server. This is done using HTTP POST requests or WebSocket communication.

Server

The server receives the information sent from the device, performs the necessary data analysis, and generates personalized information based on the acquired data. The main processes performed by the server are described below.

1. Receiving information

The server receives the location and heart rate information sent from the device. This is done using an HTTP server or WebSocket server.

2. Obtaining map information

Based on the received location information, the server calls a map information service API to obtain map information for the area around the current location. Typically, the Google Maps API is used.

3. Music Selection

The server analyzes the heart rate based on the received heart rate information and retrieves appropriate music from the music distribution service API. If the heart rate is high, an up-tempo song is selected, and if it is low, a relaxing song is selected. Spotify API or Apple Music API are commonly used.

4. Obtaining disaster prevention information

Based on the received location information, the server obtains disaster prevention information corresponding to that area from a disaster prevention information service API. For example, there is the Japan Meteorological Agency API.

5. Generating personalized information

The server integrates the acquired map information, music, disaster prevention information, etc., and generates personalized information to provide to users. This process is implemented using server-side languages such as Python and Node.js.

6. Sending personalized information

The server then sends the generated personalized information back to the device.

Specific examples

For users running

The user is wearing the smartwatch while running.

The device obtains the user's current location using the GPS sensor, records it as latitude 35.658581 and longitude 139.745433, and measures the heart rate as 160 BPM using the heart rate sensor.

The device sends the acquired location and heart rate information to the server.

The server calls the map information service API based on the location information and obtains map information within the park.

Since the heart rate is 160 BPM, the server retrieves up-tempo songs from the music distribution service API. It also calls the disaster prevention information service API to retrieve disaster prevention information for the area around the park.

The server integrates this information to generate personalized information, including park maps, music selections, disaster prevention information, and more.

The server sends the generated personalized information to the device, which receives it and displays map information on the smartwatch, plays music through earphones, and notifies users of disaster prevention information.

Example prompt

Specific examples of prompts are as follows:

"You are currently in a park. Your heart rate is 160 BPM. Please play some upbeat music and get information about nearby disasters."

This system allows users to listen to appropriate music in real time while running, while also receiving necessary map and disaster prevention information, making everyday activities safer and more comfortable.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1:

Input: Wearable device worn by the user.

Action: The user puts on a wearable device such as a smartwatch or earphones.

Output: None.

Step 2:

Input: Wearable device.

How it works: The device periodically obtains the user's current location (latitude and longitude) using the built-in GPS sensor.

Specifically, location information is obtained by calling the navigator.geolocation.getCurrentPosition method.

Output: Obtained location information (e.g., latitude 35.658581, longitude 139.745433).

Step 3:

Input: Wearable device.

How it works: The device periodically measures the user's heart rate using the built-in heart rate sensor.

Specifically, the smartwatch collects data from the heart rate sensor and calculates the heart rate.

Output: Acquired heart rate information (e.g., 160 BPM).

Step 4:

Input: Acquired location and heart rate information.

Operation: The device collects location and heart rate information and sends it to the server using an HTTP POST request.

Specifically, data is constructed in JSON format and sent to the "/data" endpoint via an HTTP POST request.

Output: Location and heart rate information sent to the server.

POST /data HTTP/1.1

Host: example.com

Content-Type: application/json

{

"location": {"latitude": 35.658581, "longitude": 139.745433},

"heartRate": 160

}

Step 5:

Input: Location and heart rate information received by the server.

Operation: The server receives location and heart rate information sent from the device and analyzes the data.

Specifically, the server endpoint /data receives the request and parses the JSON-formatted data.

Output: Analyzed location and heart rate information.

Step 6:

Input: Parsed location information.

Operation: The server calls the map information service API based on the analyzed location information and obtains map information for the area around the current location.

Specifically, send an API request like the one below.

GET https://maps.googleapis.com/maps/api/geocode/json?latlng=35.658581,139.745433&key=YOUR_API_KEY

Output: Obtained map information.

Step 7:

Input: Analyzed heart rate information.

Operation: The server calls the music distribution service API based on the analyzed heart rate information and retrieves appropriate music.

If your heart rate is high, choose up-tempo music, and if it's low, choose relaxation music.

Specifically, send an API request like the one below.

GET https://api.spotify.com/v1/recommendations?seed_genres=upbeat&target_tempo=160&key=YOUR_API_KEY

Output: Retrieved music information.

Step 8:

Input: Parsed location information.

Operation: The server calls the disaster prevention information service API based on the analyzed location information and obtains the latest disaster prevention information for the area.

Specifically, send an API request like the one below.

GET https://www.jma.go.jp/bosai/forecast/data/overview_forecast/130000.json

Output: Obtained disaster prevention information.

Step 9:

Input: Obtained map information, music information, and disaster prevention information.

Operation: The server consolidates these and generates personalized information to provide to the user.

Specifically, data is integrated using server-side languages such as Python and Node.js, and personalized information in JSON format is generated.

Output: Generated personalized information.

{

"mapInfo": { ... },

"musicInfo": { ... },

"disasterInfo": { ... }

}

Step 10:

Input: Generated personalization information.

Operation: The server sends the generated personalized information to the device via an HTTP response or push notification.

Specifically, it is returned as an HTTP response or sent using the push notification service.

Output: Personalized information sent to the device.

Step 11:

Input: Personalization information received by the device.

Operation: The device receives and analyzes the personalized information sent from the server.

Specifically, it parses HTTP responses and push notification data.

Output: Parsed personalization information.

Step 12:

Input: Parsed personalization information.

Operation: The device displays the received map information on the screen of the smartwatch or smartphone.

Specifically, the acquired map data is rendered using Google Maps SDK etc.

Output: Displayed map information.

Step 13:

Input: Parsed personalization information.

Operation: The device plays the received music information through the earphones.

Specifically, the obtained music URL is used for streaming playback.

Output: Played music.

Step 14:

Input: Parsed personalization information.

Operation: The device displays the received disaster prevention information using the smartwatch's notification function.

Specifically, we will utilize the function that displays notifications on smartwatches.

Output: Notified disaster prevention information.

(Application Example 1)

Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Conventional personalized information provision systems have had difficulty efficiently utilizing a user's location information and heart rate information to provide optimal content to users in real time. Furthermore, because these systems rely on limited information sources, it has been difficult to meet the diverse needs of users. The present invention aims to solve these problems and provide a system that provides appropriate music, podcasts, disaster prevention information, and more in real time based on a user's heart rate and location information.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes means for acquiring user location information, means for acquiring user heart rate information, means for acquiring map information around the current location from a map service based on the received location information, means for selecting appropriate music or podcasts from a content distribution service based on the received heart rate information, and means for acquiring the latest disaster prevention information for the area. This allows the user to receive optimal music, podcasts, disaster prevention information, etc. in real time according to their current location and heart rate.

text

"User" refers to an individual who uses this system and provides location information and heart rate information.

"Location information" is data that indicates a user's current geographical location.

"Heart rate information" is data that indicates the user's heart rate and represents the user's health condition and activity level.

The "server" is a system component that receives and analyzes location and heart rate information, and generates and provides various personalized information.

"Map service" is an external service that provides geographical information and map data based on location information.

"Music distribution service" is an external service that provides music tracks in streaming or download format.

"Content distribution service" is an external service that provides multimedia content such as music and podcasts.

"Disaster prevention information" is data that provides information on local disaster risks and emergency response measures.

"Personalized information" is information generated specifically for the user by integrating acquired map information, music, podcasts, and disaster prevention information.

A "terminal" is a device that acquires a user's location information and heart rate information and transmits this information to a server.

A "satellite positioning system sensor" is a device that receives satellite signals and measures location information.

"Tempo-controlled music" refers to music selected based on heart rate and with a rhythm that is appropriate for the user's current activity level.

"Podcast" refers to audio content provided over the Internet and covers a variety of genres, including education, news, and entertainment.

text

This invention is a system that provides personalized information in real time based on a user's location information and heart rate information. This system consists of three elements: the user, the terminal, and the server.

1. User

Users are individuals who use this system and provide location and heart rate information. Specifically, users wear wearable devices such as smartwatches and fitness trackers, through which they acquire and transmit location and heart rate information.

2. Device

The device is responsible for obtaining location and heart rate information from the user and sending it to the server. Specific processes include the following:

Location acquisition: The device uses the built-in satellite positioning system sensor to acquire the user's current location.

Acquiring heart rate information: The device uses the built-in heart rate sensor to measure the user's heart rate.

Information transmission: The acquired location and heart rate information is periodically sent to the server.

3. Server

The server receives and analyzes the information sent from the device and integrates various data to provide the user with the necessary information.

Information reception: The server receives location and heart rate information sent from the device.

Obtaining map information: Calls the map service API based on the received location information and obtains map information around the current location.

Content selection: Analyzes the heart rate based on the received heart rate information and retrieves appropriate music or podcasts from the content distribution service API. For example, if the heart rate is high, upbeat music is selected, and if it is low, a relaxing podcast is selected.

Obtaining disaster prevention information: Obtain the latest disaster prevention information for your area based on your location information from the disaster prevention information service API.

Generating personalized information: Integrates acquired map information, music, podcasts, and disaster prevention information to generate personalized information to provide to users.

Send information: Send the generated personalized information back to your device.

Specific examples

For example, if a user is running, the device will use a satellite positioning system sensor to obtain the user's location information and a heart rate sensor to measure the user's heart rate. Suppose the user's heart rate is 150 BPM. This information is sent to the server, which then calls a map service API to obtain map information for the current location. Based on the high heart rate, the server will obtain up-tempo music from a content distribution service API. In addition, it will obtain the latest local disaster prevention information and combine this information to generate personalized information. Finally, this information is sent to the user's device, allowing the user to receive appropriate music and disaster prevention information in real time.

Example prompt

"If the user's heart rate is 150 BPM, can you recommend some upbeat music suitable for running? Also, tell me the latest news in the area."

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

text

Step 1:

Acquires the user's location and heart rate information. The device uses the built-in satellite positioning system sensor to acquire the user's current location and the heart rate sensor to measure the heart rate. The input is the user's location and heart rate information, and the output is a data packet sent to the server.

Step 2:

The server receives location and heart rate information sent from the device. This is the initial stage of data processing and analysis. The input is the data packets sent from the device, and the output is processable data stored in the server.

Step 3:

The server calls the map service API based on the received location information and obtains map information for the area around the current location. The input is location information, and the output is the obtained map information. Specifically, it sends an API request and receives map information as a response.

Step 4:

The server analyzes the received heart rate information and selects appropriate content (music or podcast). At this time, it calls the content distribution service API based on the heart rate to obtain the appropriate track. The input is the heart rate information, and the output is the selected music or podcast. Specifically, if the heart rate is high, it queries for up-tempo music, and if the heart rate is low, it queries for relaxation podcasts.

Step 5:

The server obtains the latest disaster prevention information for the region based on the location information from the disaster prevention information service API. The input is location information, and the output is disaster prevention information. Specifically, it sends an API request to obtain disaster prevention information for the relevant region, and receives the disaster prevention information as a response.

Step 6:

The server integrates the acquired map information, music, podcasts, and disaster prevention information to generate personalized information to provide to users. The input is the various acquired data, and the output is the integrated personalized information. Specifically, the various data are integrated to generate personalized information in JSON format.

Step 7:

The server sends the generated personalized information to the terminal. The input is the generated personalized information, and the output is a data packet sent to the terminal. Specifically, the server sends data to the terminal and provides information to the user in real time.

Step 8:

The device receives personalized information sent from the server and notifies the user. The input is the data packets sent from the server, and the output is the presentation of information to the user. Specific operations include displaying a notification on the smartwatch display and playing music or podcasts through the earphones.

It is also possible to further combine an emotion engine that estimates the user's emotion. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

Form for implementing the invention

This invention is a system that provides personalized content in real time based on a user's location information, heart rate information, and emotional state. This system consists of three elements: a server, a terminal, and a user. It also includes an emotion engine for recognizing the user's emotions. Below, we will explain the role and specific functions of each element, as well as the operation of the entire system.

User

Users are individuals who use this system and provide location information, heart rate information, and emotional information. Users wear wearable devices (e.g., smart watches, earphones, audio glasses, etc.) through which they acquire, send, and receive information.

Device

The device is responsible for obtaining location information, heart rate information, and emotional information from the user and sending it to the server. Below, we will explain in detail the main functions performed by the device.

1. Obtaining location information:

The device uses its built-in GPS sensor to obtain the user's current location.

Location information is updated periodically (e.g., every 5 seconds).

2. Obtaining heart rate information:

The device uses a built-in heart rate sensor to measure the user's heart rate.

Heart rate information is also updated periodically (e.g., every 5 seconds).

3. Acquiring emotional information:

Using the device's built-in camera and microphone, the user's facial expressions and tone of voice are analyzed, and the emotion engine recognizes the user's emotions.

4. Transmission of Information:

The periodically acquired location information, heart rate information, and emotion information are packaged into packets and sent to the server.

Server

The server receives and analyzes information sent from the device and integrates various data to provide the user with the necessary information. Below, we will explain in detail the main functions and processes performed by the server.

1. Receiving information:

The server receives location information, heart rate information, and emotion information sent from the device.

2. Obtaining map information:

The server calls the map service API based on the received location information and obtains map information for the area around the current location.

3. Music Selection:

The server analyzes the received heart rate information and retrieves music that corresponds to the user's heart rate from the music distribution service API.

Example: If your heart rate is high (e.g., 160 BPM), choose an up-tempo song.

Emotional information is also taken into consideration to select songs that suit the user's emotional state.

Example: If your emotional state is "Relaxed," select relaxation music.

4. Obtaining disaster prevention information:

Based on the location information, the server obtains the latest disaster prevention information for the area from the disaster prevention information service API.

5. Generating personalized information:

The acquired map information, selected music, disaster prevention information, and the user's emotional information are integrated to generate optimal personalized information.

6. Sending personalized information:

The server sends the generated personalized information to the device.

Specific examples

For users running

The user is wearing a smartwatch and earphones while running.

The device acquires the user's current location using a GPS sensor and measures their heart rate using a heart rate sensor. For example, let's say the heart rate is 160 BPM. Furthermore, using the device's built-in camera and microphone, the emotion engine recognizes the user's emotional state as "excited."

The device sends this information to the server.

The server calls the map service API based on the location information and obtains map information within the park.

Next, because the heart rate is high, up-tempo music is retrieved from a music distribution service API. Also, because the emotional state is "excited," songs that will maintain tension are selected.

In addition, it calls the disaster prevention information service API and checks the necessary disaster prevention information.

Finally, this information is integrated to generate personalized information, including information about the user's current location, selected music, necessary disaster prevention information, and information based on their emotional state.

The server sends the generated personalized information to the user's device, which analyzes it, notifies the smartwatch, and plays music through the earphones.

This system allows users to receive appropriate music, map information, and disaster prevention information in real time while running. It also takes into account the user's emotional state, making it possible to provide a more personalized experience.

The processing flow is explained below.

Step 1:

The user puts on the wearable device and launches the application. The device activates the GPS sensor and heart rate sensor.

Step 2:

The device obtains the user's current location using the GPS sensor. This location information is updated periodically (e.g., every 5 seconds).

Step 3:

The device measures the user's heart rate using a heart rate sensor. Heart rate information is also updated periodically (e.g., every 5 seconds).

Step 4:

The device's built-in camera periodically captures the user's facial expressions, and the microphone collects the user's tone of voice. Based on this, the emotion engine analyzes the user's emotional state.

Step 5:

The device collects location information, heart rate information, and emotion information and sends them as packets to the server.

Step 6:

The server receives the location information, heart rate information, and emotion information sent from the device.

Step 7:

Based on the location information received by the server, the map service API is called to obtain map information for the area around the current location.

Step 8:

The server analyzes the received heart rate information and retrieves music corresponding to the user's heart rate from the music distribution service API. For example, if the heart rate is 160 BPM, an up-tempo song is selected.

Step 9:

The server uses an emotion engine to analyze the user's emotional state and selects appropriate music based on the results. For example, if the emotional state is "relaxed," it selects relaxation-oriented music.

Step 10:

Based on the location information, the server obtains the latest disaster prevention information for the area from the disaster prevention information service API.

Step 11:

The server integrates the acquired map information, selected music, disaster prevention information, and emotional information to generate optimal personalized information.

Step 12:

The server sends the generated personalized information to the device.

Step 13:

The device analyzes the personalized information received from the server.

Step 14:

The device will provide information to the user in an appropriate manner based on the analyzed information. For example, selected music will be played through earphones, and the smartwatch will notify the user of their current location, disaster prevention information, and emotional advice.

This detailed processing step allows users to efficiently receive the most appropriate information in real time. The system aims to improve the user experience by providing the most appropriate information tailored to the user's current situation and emotional state.

(Example 2)

Next, we will explain Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

Currently, there are systems that provide personalized information based on a user's location and heart rate information, but adding emotional information to these systems would enable even more accurate personalization. However, such systems do not currently exist, and the provision of appropriate information based on the user's emotional state is insufficient, making improving user satisfaction a challenge.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for acquiring user location information, means for acquiring user heart rate information, means for acquiring user emotional information, means for transmitting the acquired location information, heart rate information, and emotional information to the server, means for receiving the transmitted location information, heart rate information, and emotional information, means for acquiring map information around the current location from a map service based on the received location information, means for selecting appropriate music from a music distribution service based on the received heart rate information and emotional information, means for acquiring the latest disaster prevention information corresponding to the area, means for integrating the acquired map information, music, disaster prevention information, and emotional information to generate personalized information to provide to the user, means for transmitting the generated personalized information to the terminal, and terminal means. This enables the provision of highly accurate personalized information based on more multifaceted information including the user's emotional state.

"User location information" refers to the coordinate information of the user's current location.

"User's heart rate information" refers to data measuring the user's heart rate.

"User emotional information" refers to data that expresses the user's emotional state, and is obtained by analyzing facial expressions, tone of voice, etc.

A "server" is a computer system that receives and analyzes data sent from a terminal, consolidates the necessary information, and sends it to the terminal.

"Terminal" refers to a device worn or used by a user that acquires location information, heart rate information, and emotional information and transmits it to a server.

A "GPS sensor" is a sensor that receives signals from satellites and determines your current location on Earth.

"Map Service" refers to a service that provides geographic information through a specific API.

"Music distribution service" refers to a service that provides music data through a specific API.

"Disaster prevention information" refers to information regarding emergencies such as natural disasters, and is provided according to the region.

"Personalized information" refers to information specific to each user that is generated by integrating each user's location information, heart rate information, emotional information, etc.

An "emotion engine" refers to software or hardware that analyzes a user's facial expressions and tone of voice to recognize their emotional state.

This invention is a system that provides personalized content in real time based on a user's location information, heart rate information, and emotional state. This system consists of three elements: a server, a terminal, and a user. It also includes an emotion engine for recognizing the user's emotions. Below, we will explain the role and specific functions of each element, as well as the operation of the entire system.

User

Users are individuals who use this system and provide location information, heart rate information, and emotional information. Users wear wearable devices (e.g., smartwatches, earphones, etc.) through which they acquire, send, and receive information. Users begin using the system by wearing the smartwatch on their wrist and the earphones in their ears.

Device

The device is responsible for obtaining location information, heart rate information, and emotional information from the user and sending it to the server. Below, we will explain in detail the main functions performed by the device.

1. Obtaining location information:

The device uses the built-in GPS sensor to obtain the user's current location. Location information is updated periodically (e.g., every 5 seconds).

2. Obtaining heart rate information:

The device uses a built-in heart rate sensor to measure the user's heart rate. Heart rate information is also updated periodically (e.g., every 5 seconds).

3. Acquiring emotional information:

Using the device's built-in camera and microphone, the user's facial expressions and tone of voice are analyzed, and the emotion engine recognizes the user's emotions.

4. Transmission of Information:

The periodically acquired location information, heart rate information, and emotion information are packaged into packets and sent to the server.

Server

The server receives and analyzes information sent from the device and integrates various data to provide the user with the necessary information. Below, we will explain in detail the main functions and processes performed by the server.

1. Receiving information:

The server receives location information, heart rate information, and emotion information sent from the device.

2. Obtaining map information:

The server calls the map service API based on the received location information and obtains map information for the area around the current location.

3. Music Selection:

The server analyzes the received heart rate information and retrieves music from the music distribution service API that corresponds to the user's heart rate and emotional state.

Example: If your heart rate is high (e.g., 160 BPM), choose an up-tempo song. If your emotional state is "excited," choose an energetic song.

4. Obtaining disaster prevention information:

Based on the location information, the server obtains the latest disaster prevention information for the area from the disaster prevention information service API.

5. Generating personalized information:

The acquired map information, selected music, disaster prevention information, and the user's emotional information are integrated to generate optimal personalized information.

6. Sending personalized information:

The server sends the generated personalized information to the device.

Specific examples

For users running

Suppose a user is wearing a smartwatch and earphones while running. The smartwatch periodically obtains the user's current location using a GPS sensor and measures their heart rate using a heart rate sensor. For example, if the user's heart rate is 160 BPM, the device sends this information to the server. At the same time, the emotion engine uses the device's built-in camera and microphone to recognize the user's emotional state as "excited."

The server first calls a map service API based on the location information and obtains map information for the park where the user is running. Next, because the heart rate is high, it obtains up-tempo music from a music distribution service API, and because the emotional state is "excited," it selects energetic music. It also calls a disaster prevention information service API to check disaster prevention information related to the user's current location.

By integrating this data (map information, music information, disaster prevention information), the server generates personalized information optimized for the user and sends it to the device. The device then plays the delivered music through earphones and displays the necessary information on the smartwatch screen.

Example prompt

"Please explain the specific processing steps of a system that provides real-time personalized music and map information when a user's heart rate is high and their emotional state is 'excited' while running."

The flow of the identification process in Example 2 will be explained using Figure 13.

Step 1: The user puts on the wearable device.

The user puts on a smartwatch, earphones, or other wearable device. The device is then ready to acquire the user's location, heart rate, and emotional information. The input for this step is the user themselves, and the output is the wearable device they are wearing.

Step 2: The device acquires location information

The device uses the built-in GPS sensor to obtain the user's current location. For example, if the user is in a park, the device obtains their location coordinates. The input is the signal from the GPS sensor, and the output is location coordinate data (e.g., latitude 35.6895 degrees, longitude 139.6917 degrees). The location information is updated periodically (e.g., every 5 seconds).

Step 3: The device acquires heart rate information.

The device uses a built-in heart rate sensor to measure the user's heart rate. For example, the heart rate is measured at 160 BPM. The input is data from the heart rate sensor, and the output is heart rate data. Heart rate information is also updated periodically (for example, every 5 seconds).

Step 4: The device acquires emotional information.

The device uses a camera and microphone to analyze the user's facial expressions and tone of voice. The emotion engine processes the results of this analysis and recognizes the user's emotional state as "excitement." The input is sensor data from the camera and microphone, and the output is emotional information.

Step 5: The device sends the information to the server.

The device collects location, heart rate, and emotional information and compiles it into packets, which are then sent to the server. The input is location, heart rate, and emotional data, and the output is the data sent to the server. Transmission is carried out periodically.

Step 6: The server receives the information

The server receives location information, heart rate information, and emotional information sent from the device. The input is the data sent from the device, and the output is the data stored on the server.

Step 7: The server retrieves map information.

The server calls the map service API based on the received location information and obtains map information for the area around the current location. The input is location data, and the output is map information data (e.g., obtaining geographic information using the Google Maps API).

Step 8: The server selects music.

The server analyzes the received heart rate and emotional information and retrieves music from a music distribution service API that corresponds to the user's heart rate and emotional state. For example, if a user's heart rate is 160 BPM and they are in an "excited" state, it will select an up-tempo, energetic song. The input is heart rate and emotional information data, and the output is the selected music data (e.g., retrieving songs using the Spotify API).

Step 9: The server obtains disaster prevention information.

Based on the location information, the server obtains the latest disaster prevention information for the region from the disaster prevention information service API. The input is location information data, and the output is disaster prevention information data (e.g., using the disaster prevention information service API to obtain the latest information for the region).

Step 10: The server generates personalized information

The server integrates the acquired map information, selected music, disaster prevention information, and emotional information to generate optimal personalized information. The input is map information, music data, disaster prevention information, and emotional information, and the output is integrated personalized information.

Step 11: The server sends personalized information to the device.

The server sends the generated personalized information to the terminal. The input is the personalized information data, and the output is the data sent to the terminal.

Step 12: The device analyzes the information and notifies the user.

The device analyzes the received personalized information and notifies the user of appropriate information. Specifically, map information and disaster prevention information are displayed on the smartwatch's notification screen, and selected music is played through the earphones. The input is personalized information data, and the output is the form of information provided to the user (screen display and music playback).

(Application Example 2)

Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

With the proliferation of information in modern society and the diversification of individual user needs, there is a growing need to provide personalized content tailored to the user's real-time situation. In particular, real-time information provision systems based on biometric data such as location information and heart rate contribute to improving the user experience. However, current systems are not yet able to effectively integrate this data and provide personalized content that also responds to the user's emotional state. Therefore, there is a need for a system that provides personalized information in real time based on the user's location information, heart rate information, and emotional state.

The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring user location information, means for acquiring user heart rate information, means for acquiring the user's emotional state, means for transmitting the received location information, heart rate information, and emotional information to the server, means for acquiring map information near the current location from a map service based on the received location information, means for selecting appropriate music from a music distribution service based on the received heart rate information, means for generating personalized content based on the emotional state, means for acquiring the latest disaster prevention information corresponding to the region, means for integrating the acquired map information, music, disaster prevention information, and emotional information to generate personalized information to provide to the user, means for transmitting the generated personalized information to the terminal, and terminal means. This makes it possible to provide personalized information that integrates appropriate map information, music, and disaster prevention information in real time based on the user's real-time biometric data and emotional state.

"Location information" refers to geographic coordinate information that indicates the user's current location.

"Heart rate information" refers to biometric data such as heart rate and heart rhythm obtained from the user's heartbeat.

"Emotional state" refers to information that indicates the user's emotions, such as psychological states such as stress, relaxation, and excitement.

"Personalized information" refers to content and service information that is individually customized based on the user's location information, heart rate information, and emotional state.

"Map information" is data that shows the terrain, roads, facilities, etc. within a geographic space.

A "music distribution service" is a service that distributes music via the Internet.

"Disaster prevention information" refers to the latest information on disasters such as earthquakes, fires, and typhoons.

A "server" is a central processing unit that receives, analyzes, and processes information sent by users.

"Terminal" refers to a device carried or worn by a user, including smartphones and smartwatches.

This invention is a system that acquires a user's location information, heart rate information, and emotional state in real time and provides personalized content. This system is primarily composed of three elements: a server, a terminal, and a user. Below, we will explain in detail the operation of each element and the system as a whole.

User

The user is the person who uses this system and provides location information, heart rate information, and emotional information. The user wears a wearable device such as a smartwatch, earphones, or smart glasses, and the information is collected and sent to the server.

Device

The terminal is responsible for obtaining information from the user and sending it to the server. Its main functions are as follows:

1. Obtaining location information:

The device uses the built-in GPS sensor to obtain the user's current location. Location information is updated periodically.

2. Obtaining heart rate information:

The device uses a built-in heart rate sensor to measure the user's heart rate, and this information is also updated periodically.

3. Acquiring emotional information:

Using the device's built-in camera and microphone, the emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions.

4. Transmission of Information:

The processed data is sent to the server.

Server

The server receives, analyzes, and processes information sent from the device. The main processes are as follows:

1. Receiving information:

Receives location information, heart rate information, and emotion information sent from the device.

2. Obtaining map information:

Based on the received location information, the map service API is called to obtain map information around the current location.

3. Music Selection:

Based on heart rate information, appropriate music is selected from a music distribution service API. For example, if the heart rate is high, an up-tempo song is selected. Emotional information is also taken into consideration, and songs are selected that match the user's emotional state.

4. Emotion-based content generation:

Based on emotional information, it generates content such as videos and podcasts that are optimal for the user's emotional state.

5. Obtaining disaster prevention information:

The latest disaster prevention information is obtained from the disaster prevention information service API based on location information.

6. Generating personalized information:

The acquired map information, selected music, disaster prevention information, and emotional information are integrated to generate personalized information to provide to the user.

7. Transmission of Information:

The final generated personalized information is sent to the device.

Specific examples

For users running

The user wears a smartwatch and earphones while running. The device obtains current location information obtained from the GPS sensor and heart rate measured by the heart rate sensor (e.g., 160 BPM). The emotion engine also recognizes the user's emotional state as "excitement." This information is then periodically sent to the server.

The server calls a map service API based on the received location information and obtains map information for the park. Next, because the heart rate information is high, it obtains up-tempo songs from a music distribution service API and selects music suitable for maintaining excitement based on the emotional state. It also calls a disaster prevention information service API to check any necessary disaster prevention information.

This information is then integrated to generate personalized information, which is then sent back to the user's device. The user can check this information on their smartwatch, and the selected music will play from the earphones.

Example of a prompt for a generative AI model:

"A user is feeling stressed while walking through a city. Their heart rate is 120 BPM. What content would you recommend?"

This system allows users to enjoy content that is best suited to their situation in real time.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

Users acquire information by wearing a wearable device.

Specific actions:

The user wears a wearable device such as a smartwatch, earphones, or smart glasses. The user's current location is acquired using a GPS sensor, and heart rate information is measured using a heart rate sensor. In addition, the device's built-in camera and microphone are used to analyze the user's facial expressions and tone of voice to obtain emotional information.

input:

User location, heart rate, and emotional state

output:

Acquired location information, heart rate information, and emotional information

Step 2:

The device sends the acquired information to the server.

Specific actions:

The device periodically collects location information, heart rate information, and emotional information, and then compiles them into packets and sends them to a server. Communication is carried out via the Internet.

input:

Acquired location information, heart rate information, and emotional information

output:

Location, heart rate, and emotional information sent to the server

Step 3:

The server analyzes the information received.

Specific actions:

The server receives location information, heart rate information, and emotional information sent from the device. It analyzes the received data and identifies and organizes each piece of information.

input:

Transmitted location information, heart rate information, and emotional information

output:

Organized location, heart rate, and emotional information

Step 4:

The server obtains map information.

Specific actions:

The server calls the map service API based on the received location information and obtains map information for the area around the current location. The obtained map information provides detailed geographic data about the user's current location.

input:

Organized location information

output:

Retrieved map information

Step 5:

The server selects the music.

Specific actions:

The server analyzes the received heart rate information and retrieves music corresponding to the user's heart rate from the music distribution service API. It also takes emotional information into account and selects music appropriate to the user's emotional state.

input:

Organized heart rate and emotional information

output:

Selected music

Step 6:

The server acquires disaster prevention information.

Specific actions:

The server retrieves the latest disaster prevention information for the area based on the location information from the disaster prevention information service API. This provides information such as disaster risks near the user's current location.

input:

Organized location information

output:

Disaster prevention information obtained

Step 7:

The server generates personalized information.

Specific actions:

The server integrates the acquired map information, selected music, disaster prevention information, and emotional information to generate personalized information to provide to the user.

input:

Map information, selected music, disaster prevention information, emotional information

output:

Generated personalized information

Step 8:

The server sends the generated personalized information to the device.

Specific actions:

The server sends the generated personalized information to the device. The sent information is analyzed on the user's device and notified or played back appropriately.

input:

Generated personalized information

output:

Personalized information sent from the server

Step 9:

Devices provide personalized information to users

Specific actions:

The device analyzes the received personalized information and provides it to the user via the smartwatch or earphones. Specifically, notifications are displayed on the smartwatch and the selected music is played through the earphones.

input:

Personalized information received

output:

Personalized information provided to users

The specific processing unit 290 transmits the results of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). Examples of data generation model 58 include generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>) and Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the smart glasses 214.

[Third embodiment]

Figure 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

As shown in FIG. 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I/F 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user 20's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

Figure 6 shows an example of the main functions of the data processing device 12 and headset terminal 314. As shown in Figure 6, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the headset terminal 314, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the identification process performed by the identification processing unit 290 of the data processing device 12. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Form for implementing the invention

This invention is a system that provides personalized information in real time based on a user's location information and heart rate information. This system basically consists of three elements: a server, a terminal, and a user. The roles and specific functions of each are explained below.

User

Users are individuals who use this system and provide location and heart rate information. Users wear wearable devices (e.g., smart watches, earphones, audio glasses, etc.) through which they acquire, send, and receive information.

Device

The device is responsible for obtaining location and heart rate information from the user and sending it to the server. The main processes performed by the device are described below.

1. Obtaining location information:

The device uses its built-in GPS sensor to obtain the user's current location.

The acquired location information is updated periodically.

2. Obtaining heart rate information:

The device uses a built-in heart rate sensor to measure the user's heart rate.

Measured heart rate information is also updated regularly.

3. Transmission of information:

The periodically acquired location and heart rate information is packaged into packets and sent to the server.

Server

The server receives and analyzes information sent from the device and integrates various data to provide the user with the necessary information. The main processes performed by the server are described below.

1. Receiving information:

The server receives the location information and heart rate information sent from the device.

2. Obtaining map information:

The server calls the map service API based on the received location information and obtains map information for the area around the current location.

3. Music Selection:

The server analyzes the heart rate based on the received heart rate information and retrieves appropriate music from the music distribution service API.

Example: If your heart rate is high, choose up-tempo music, and if your heart rate is low, choose relaxation music.

4. Obtaining disaster prevention information:

The server obtains disaster prevention information corresponding to the area from the disaster prevention information service API based on the location information.

5. Generating personalized information:

The acquired map information, music, disaster prevention information, and other personalized information (e.g., store coupons, etc.) are integrated as needed to generate optimal personalized information.

6. Sending personalized information:

The server then sends the generated personalized information back to the device.

Specific examples

For users running

The user is wearing the smartwatch while running.

The device obtains the user's current location using a GPS sensor and measures the heart rate using a heart rate sensor. For example, let's say the heart rate is 160 BPM.

The device sends the acquired location and heart rate information to the server.

The server calls the map service API based on the location information and obtains map information within the park.

Next, because the heart rate is high, up-tempo music is retrieved from the music distribution service API. In addition, the disaster prevention information service API is called to check the necessary disaster prevention information.

Finally, this information is integrated to generate personalized information. This information includes information about the user's current location, selected music, necessary disaster prevention information, etc.

The server sends the generated personalized information to the user's device, which analyzes it and sends notifications to the smartwatch or plays music through earphones.

This system allows users to listen to appropriate music in real time while running, while also receiving necessary map and disaster prevention information, making everyday activities safer and more comfortable.

The processing flow is explained below.

Step 1:

The user puts on the wearable device and starts the system. The device completes initial setup and enables the GPS sensor and heart rate sensor.

Step 2:

The device uses the GPS sensor to obtain the user's current location. This location information is obtained periodically (e.g., every 5 seconds).

Step 3:

The device uses a heart rate sensor to measure the user's heart rate. Heart rate information is also collected periodically (e.g., every 5 seconds).

Step 4:

The device collects location and heart rate information and compiles it into packets, which it then sends to the server.

Step 5:

The server receives the location and heart rate information sent from the device.

Step 6:

Based on the location information received by the server, the map service API is called to obtain map information for the area around the current location.

Step 7:

The server analyzes the received heart rate information and calls the music distribution service API to select music according to the user's heart rate.

Example: If your heart rate is high (e.g., 160 BPM), choose up-tempo songs.

Step 8:

Based on the location information, the server obtains the latest disaster prevention information for the area from the disaster prevention information service API.

Step 9:

The server integrates map information, selected music, and disaster prevention information to generate optimal personalized information.

Step 10:

The server sends the generated personalized information to the device.

Step 11:

The device analyzes the personalized information received from the server.

Step 12:

The device will provide information to the user in an appropriate manner based on the information analyzed.

Example: Play selected music through earphones and notify the smartwatch of current location and disaster prevention information.

This processing flow allows users to receive the information they need in real time. The system aims to improve the user experience by providing the most appropriate information tailored to the user's current situation.

(Example 1)

Next, Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Previous personalized information provision systems did not fully utilize users' location information or heart rate information, making it difficult to provide personalized information in real time. Furthermore, because music selection and disaster prevention information provision were not centralized, users were unable to receive the information they needed in a timely manner, resulting in issues with safety and convenience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for acquiring user location information, means for acquiring user heart rate information, means for transmitting the acquired location information and heart rate information to a central processing unit, means for receiving the transmitted location information and heart rate information, means for acquiring map information near the current location from a map information providing service based on the received location information, means for selecting appropriate music from a music distribution service based on the received heart rate information, means for acquiring the latest disaster prevention information corresponding to the area, means for integrating the acquired map information, music, and disaster prevention information to generate personalized information to provide to the user, means for transmitting the generated personalized information to the terminal, and means for notifying the user of the personalized information received by the terminal. This makes it possible to utilize the user's location information and heart rate information in real time, centrally manage necessary map information, music, and disaster prevention information, and provide it to the user immediately.

"User" means an individual who uses the system and provides location and heart rate information.

"Location information" is data indicating the user's current geographical latitude and longitude.

"Heart rate information" is data that indicates the user's current heart rate.

A "central processing unit" is a device that has the function of receiving and processing location information and heart rate information sent from a terminal.

A "terminal" is a device worn by the user that has the function of acquiring location information and heart rate information and transmitting this information to a central processing unit.

"Map information service" is an online service that provides map information around your current location based on your location information.

A "music distribution service" is an online service that selects and provides appropriate music based on heart rate information.

"Disaster prevention information" refers to information including the latest disaster information and evacuation advisories for a specific area.

"Personalized information" is individually tailored information that is generated by integrating acquired map information, music, and disaster prevention information and providing it to users.

"Notification" is the act of presenting information to the user via a device.

This invention is a system that provides personalized information in real time based on a user's location information and heart rate information. This system basically consists of three elements: a server, a terminal, and a user. The roles and specific functions of each are explained below.

User

Users are individuals who use this system and provide location and heart rate information. Users wear wearable devices (e.g., smart watches, earphones, audio glasses, etc.) through which they acquire, send, and receive information.

Device

The device acquires location and heart rate information from the user and sends it to the server. Specifically, it performs the following processes:

1. Obtaining location information

The device uses the built-in GPS sensor to obtain the user's current location. Using the GPS sensor allows for detailed and accurate location information to be obtained.

The device will periodically update this location information to keep it up to date.

2. Obtaining heart rate information

The device measures the user's heart rate using a built-in heart rate sensor. Heart rate sensors often use optical sensors or electrocardiogram sensors.

Your device will regularly update your heart rate information to keep your health information up to date.

3. Transmission of Information

The acquired location and heart rate information is packaged into packets and sent to the server. This is done using HTTP POST requests or WebSocket communication.

Server

The server receives the information sent from the device, performs the necessary data analysis, and generates personalized information based on the acquired data. The main processes performed by the server are described below.

1. Receiving information

The server receives the location and heart rate information sent from the device. This is done using an HTTP server or WebSocket server.

2. Obtaining map information

Based on the received location information, the server calls a map information service API to obtain map information for the area around the current location. Typically, the Google Maps API is used.

3. Music Selection

The server analyzes the heart rate based on the received heart rate information and retrieves appropriate music from the music distribution service API. If the heart rate is high, an up-tempo song is selected, and if it is low, a relaxing song is selected. Spotify API or Apple Music API are commonly used.

4. Obtaining disaster prevention information

Based on the received location information, the server obtains disaster prevention information corresponding to that area from a disaster prevention information service API. For example, there is the Japan Meteorological Agency API.

5. Generating personalized information

The server integrates the acquired map information, music, disaster prevention information, etc., and generates personalized information to provide to users. This process is implemented using server-side languages such as Python and Node.js.

6. Sending personalized information

The server then sends the generated personalized information back to the device.

Specific examples

For users running

The user is wearing the smartwatch while running.

The device obtains the user's current location using the GPS sensor, records it as latitude 35.658581 and longitude 139.745433, and measures the heart rate as 160 BPM using the heart rate sensor.

The device sends the acquired location and heart rate information to the server.

The server calls the map information service API based on the location information and obtains map information within the park.

Since the heart rate is 160 BPM, the server retrieves up-tempo songs from the music distribution service API. It also calls the disaster prevention information service API to retrieve disaster prevention information for the area around the park.

The server integrates this information to generate personalized information, including park maps, music selections, disaster prevention information, and more.

The server sends the generated personalized information to the device, which receives it and displays map information on the smartwatch, plays music through earphones, and notifies users of disaster prevention information.

Example prompt

Specific examples of prompts are as follows:

"You are currently in a park. Your heart rate is 160 BPM. Please play some upbeat music and get information about nearby disasters."

This system allows users to listen to appropriate music in real time while running, while also receiving necessary map and disaster prevention information, making everyday activities safer and more comfortable.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1:

Input: Wearable device worn by the user.

Action: The user puts on a wearable device such as a smartwatch or earphones.

Output: None.

Step 2:

Input: Wearable device.

How it works: The device periodically obtains the user's current location (latitude and longitude) using the built-in GPS sensor.

Specifically, location information is obtained by calling the navigator.geolocation.getCurrentPosition method.

Output: Obtained location information (e.g., latitude 35.658581, longitude 139.745433).

Step 3:

Input: Wearable device.

How it works: The device periodically measures the user's heart rate using the built-in heart rate sensor.

Specifically, the smartwatch collects data from the heart rate sensor and calculates the heart rate.

Output: Acquired heart rate information (e.g., 160 BPM).

Step 4:

Input: Acquired location and heart rate information.

Operation: The device collects location and heart rate information and sends it to the server using an HTTP POST request.

Specifically, data is constructed in JSON format and sent to the "/data" endpoint via an HTTP POST request.

Output: Location and heart rate information sent to the server.

POST /data HTTP/1.1

Host: example.com

Content-Type: application/json

{

"location": {"latitude": 35.658581, "longitude": 139.745433},

"heartRate": 160

}

Step 5:

Input: Location and heart rate information received by the server.

Operation: The server receives location and heart rate information sent from the device and analyzes the data.

Specifically, the server endpoint /data receives the request and parses the JSON-formatted data.

Output: Analyzed location and heart rate information.

Step 6:

Input: Parsed location information.

Operation: The server calls the map information service API based on the analyzed location information and obtains map information for the area around the current location.

Specifically, send an API request like the one below.

GET https://maps.googleapis.com/maps/api/geocode/json?latlng=35.658581,139.745433&key=YOUR_API_KEY

Output: Obtained map information.

Step 7:

Input: Analyzed heart rate information.

Operation: The server calls the music distribution service API based on the analyzed heart rate information and retrieves appropriate music.

If your heart rate is high, choose up-tempo music, and if it's low, choose relaxation music.

Specifically, send an API request like the one below.

GET https://api.spotify.com/v1/recommendations?seed_genres=upbeat&target_tempo=160&key=YOUR_API_KEY

Output: Retrieved music information.

Step 8:

Input: Parsed location information.

Operation: The server calls the disaster prevention information service API based on the analyzed location information and obtains the latest disaster prevention information for the area.

Specifically, send an API request like the one below.

GET https://www.jma.go.jp/bosai/forecast/data/overview_forecast/130000.json

Output: Obtained disaster prevention information.

Step 9:

Input: Obtained map information, music information, and disaster prevention information.

Operation: The server consolidates these and generates personalized information to provide to the user.

Specifically, data is integrated using server-side languages such as Python and Node.js, and personalized information in JSON format is generated.

Output: Generated personalized information.

{

"mapInfo": { ... },

"musicInfo": { ... },

"disasterInfo": { ... }

}

Step 10:

Input: Generated personalization information.

How it works: The server sends the generated personalized information to the device via an HTTP response or push notification.

Specifically, it is returned as an HTTP response or sent using the push notification service.

Output: Personalized information sent to the device.

Step 11:

Input: Personalization information received by the device.

Operation: The device receives and analyzes the personalized information sent from the server.

Specifically, it parses HTTP responses and push notification data.

Output: Parsed personalization information.

Step 12:

Input: Parsed personalization information.

Operation: The device displays the received map information on the screen of the smartwatch or smartphone.

Specifically, the acquired map data is rendered using Google Maps SDK etc.

Output: Displayed map information.

Step 13:

Input: Parsed personalization information.

Operation: The device plays the received music information through the earphones.

Specifically, the obtained music URL is used for streaming playback.

Output: Played music.

Step 14:

Input: Parsed personalization information.

Operation: The device displays the received disaster prevention information using the smartwatch's notification function.

Specifically, we will utilize the function that displays notifications on smartwatches.

Output: Notified disaster prevention information.

(Application Example 1)

Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Conventional personalized information provision systems have had difficulty efficiently utilizing a user's location information and heart rate information to provide optimal content to users in real time. Furthermore, because these systems rely on limited information sources, it has been difficult to meet the diverse needs of users. The present invention aims to solve these problems and provide a system that provides appropriate music, podcasts, disaster prevention information, and more in real time based on a user's heart rate and location information.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes means for acquiring user location information, means for acquiring user heart rate information, means for acquiring map information around the current location from a map service based on the received location information, means for selecting appropriate music or podcasts from a content distribution service based on the received heart rate information, and means for acquiring the latest disaster prevention information for the area. This allows the user to receive optimal music, podcasts, disaster prevention information, etc. in real time according to their current location and heart rate.

text

"User" refers to an individual who uses this system and provides location information and heart rate information.

"Location information" is data that indicates a user's current geographical location.

"Heart rate information" is data that indicates the user's heart rate and represents the user's health condition and activity level.

The "server" is a system component that receives and analyzes location and heart rate information, and generates and provides various personalized information.

"Map service" is an external service that provides geographical information and map data based on location information.

"Music distribution service" is an external service that provides music tracks in streaming or download format.

"Content distribution service" is an external service that provides multimedia content such as music and podcasts.

"Disaster prevention information" is data that provides information on local disaster risks and emergency response measures.

"Personalized information" is information generated specifically for the user by integrating acquired map information, music, podcasts, and disaster prevention information.

A "terminal" is a device that acquires a user's location information and heart rate information and transmits this information to a server.

A "satellite positioning system sensor" is a device that receives satellite signals and measures location information.

"Tempo-controlled music" refers to music selected based on heart rate and with a rhythm that is appropriate for the user's current activity level.

"Podcast" refers to audio content provided over the Internet and covers a variety of genres, including education, news, and entertainment.

text

This invention is a system that provides personalized information in real time based on a user's location information and heart rate information. This system consists of three elements: the user, the terminal, and the server.

1. User

Users are individuals who use this system and provide location and heart rate information. Specifically, users wear wearable devices such as smartwatches and fitness trackers, through which they acquire and transmit location and heart rate information.

2. Device

The device is responsible for obtaining location and heart rate information from the user and sending it to the server. Specific processes include the following:

Location acquisition: The device uses the built-in satellite positioning system sensor to acquire the user's current location.

Acquiring heart rate information: The device uses the built-in heart rate sensor to measure the user's heart rate.

Information transmission: The acquired location and heart rate information is periodically sent to the server.

3. Server

The server receives and analyzes the information sent from the device and integrates various data to provide the user with the necessary information.

Information reception: The server receives location and heart rate information sent from the device.

Obtaining map information: Calls the map service API based on the received location information and obtains map information around the current location.

Content selection: Analyzes the heart rate based on the received heart rate information and retrieves appropriate music or podcasts from the content distribution service API. For example, if the heart rate is high, upbeat music is selected, and if it is low, a relaxing podcast is selected.

Obtaining disaster prevention information: Obtain the latest disaster prevention information for your area based on your location information from the disaster prevention information service API.

Generating personalized information: Integrates acquired map information, music, podcasts, and disaster prevention information to generate personalized information to provide to users.

Send information: Send the generated personalized information back to your device.

Specific examples

For example, if a user is running, the device will use a satellite positioning system sensor to obtain the user's location information and a heart rate sensor to measure the user's heart rate. Suppose the user's heart rate is 150 BPM. This information is sent to the server, which then calls a map service API to obtain map information for the current location. Based on the high heart rate, the server will obtain up-tempo music from a content distribution service API. In addition, it will obtain the latest local disaster prevention information and combine this information to generate personalized information. Finally, this information is sent to the user's device, allowing the user to receive appropriate music and disaster prevention information in real time.

Example prompt

"If the user's heart rate is 150 BPM, can you recommend some upbeat music suitable for running? Also, tell me the latest news in the area."

The flow of the identification process in Application Example 1 will be explained using Figure 12.

text

Step 1:

Acquires the user's location and heart rate information. The device uses the built-in satellite positioning system sensor to acquire the user's current location and the heart rate sensor to measure the heart rate. The input is the user's location and heart rate information, and the output is a data packet sent to the server.

Step 2:

The server receives location and heart rate information sent from the device. This is the initial stage of data processing and analysis. The input is the data packets sent from the device, and the output is processable data stored in the server.

Step 3:

The server calls the map service API based on the received location information and obtains map information for the area around the current location. The input is location information, and the output is the obtained map information. Specifically, it sends an API request and receives map information as a response.

Step 4:

The server analyzes the received heart rate information and selects appropriate content (music or podcast). At this time, it calls the content distribution service API based on the heart rate to obtain the appropriate track. The input is the heart rate information, and the output is the selected music or podcast. Specifically, if the heart rate is high, it queries for up-tempo music, and if the heart rate is low, it queries for relaxation podcasts.

Step 5:

The server obtains the latest disaster prevention information for the region based on the location information from the disaster prevention information service API. The input is location information, and the output is disaster prevention information. Specifically, it sends an API request to obtain disaster prevention information for the relevant region, and receives the disaster prevention information as a response.

Step 6:

The server integrates the acquired map information, music, podcasts, and disaster prevention information to generate personalized information to provide to users. The input is the various acquired data, and the output is the integrated personalized information. Specifically, the various data are integrated to generate personalized information in JSON format.

Step 7:

The server sends the generated personalized information to the terminal. The input is the generated personalized information, and the output is a data packet sent to the terminal. Specifically, the server sends data to the terminal and provides information to the user in real time.

Step 8:

The device receives personalized information sent from the server and notifies the user. The input is the data packets sent from the server, and the output is the presentation of information to the user. Specific operations include displaying a notification on the smartwatch display and playing music or podcasts through the earphones.

It is also possible to further combine an emotion engine that estimates the user's emotion. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

Form for implementing the invention

This invention is a system that provides personalized content in real time based on a user's location information, heart rate information, and emotional state. This system consists of three elements: a server, a terminal, and a user. It also includes an emotion engine for recognizing the user's emotions. Below, we will explain the role and specific functions of each element, as well as the operation of the entire system.

User

Users are individuals who use this system and provide location information, heart rate information, and emotional information. Users wear wearable devices (e.g., smart watches, earphones, audio glasses, etc.) through which they acquire, send, and receive information.

Device

The device is responsible for obtaining location information, heart rate information, and emotional information from the user and sending it to the server. Below, we will explain in detail the main functions performed by the device.

1. Obtaining location information:

The device uses its built-in GPS sensor to obtain the user's current location.

Location information is updated periodically (e.g., every 5 seconds).

2. Obtaining heart rate information:

The device uses a built-in heart rate sensor to measure the user's heart rate.

Heart rate information is also updated periodically (e.g., every 5 seconds).

3. Acquiring emotional information:

Using the device's built-in camera and microphone, the user's facial expressions and tone of voice are analyzed, and the emotion engine recognizes the user's emotions.

4. Transmission of Information:

The periodically acquired location information, heart rate information, and emotion information are packaged into packets and sent to the server.

Server

The server receives and analyzes information sent from the device and integrates various data to provide the user with the necessary information. Below, we will explain in detail the main functions and processes performed by the server.

1. Receiving information:

The server receives location information, heart rate information, and emotion information sent from the device.

2. Obtaining map information:

The server calls the map service API based on the received location information and obtains map information for the area around the current location.

3. Music Selection:

The server analyzes the received heart rate information and retrieves music that corresponds to the user's heart rate from the music distribution service API.

Example: If your heart rate is high (e.g., 160 BPM), choose an up-tempo song.

Emotional information is also taken into consideration to select songs that suit the user's emotional state.

Example: If your emotional state is "Relaxed," select relaxation music.

4. Obtaining disaster prevention information:

Based on the location information, the server obtains the latest disaster prevention information for the area from the disaster prevention information service API.

5. Generating personalized information:

The acquired map information, selected music, disaster prevention information, and the user's emotional information are integrated to generate optimal personalized information.

6. Sending personalized information:

The server sends the generated personalized information to the device.

Specific examples

For users running

The user is wearing a smartwatch and earphones while running.

The device acquires the user's current location using a GPS sensor and measures their heart rate using a heart rate sensor. For example, let's say the heart rate is 160 BPM. Furthermore, using the device's built-in camera and microphone, the emotion engine recognizes the user's emotional state as "excited."

The device sends this information to the server.

The server calls the map service API based on the location information and obtains map information within the park.

Next, because the heart rate is high, up-tempo music is retrieved from a music distribution service API. Also, because the emotional state is "excited," songs that will maintain tension are selected.

In addition, it calls the disaster prevention information service API and checks the necessary disaster prevention information.

Finally, this information is integrated to generate personalized information, including information about the user's current location, selected music, necessary disaster prevention information, and information based on their emotional state.

The server sends the generated personalized information to the user's device, which analyzes it, notifies the smartwatch, and plays music through the earphones.

This system allows users to receive appropriate music, map information, and disaster prevention information in real time while running. It also takes into account the user's emotional state, making it possible to provide a more personalized experience.

The processing flow is explained below.

Step 1:

The user puts on the wearable device and launches the application. The device activates the GPS sensor and heart rate sensor.

Step 2:

The device obtains the user's current location using the GPS sensor. This location information is updated periodically (e.g., every 5 seconds).

Step 3:

The device measures the user's heart rate using a heart rate sensor. Heart rate information is also updated periodically (e.g., every 5 seconds).

Step 4:

The device's built-in camera periodically captures the user's facial expressions, and the microphone collects the user's tone of voice. Based on this, the emotion engine analyzes the user's emotional state.

Step 5:

The device collects location information, heart rate information, and emotion information and sends it to the server in packets.

Step 6:

The server receives the location information, heart rate information, and emotion information sent from the device.

Step 7:

Based on the location information received by the server, the map service API is called to obtain map information for the area around the current location.

Step 8:

The server analyzes the received heart rate information and retrieves music corresponding to the user's heart rate from the music distribution service API. For example, if the heart rate is 160 BPM, an up-tempo song is selected.

Step 9:

The server uses an emotion engine to analyze the user's emotional state and selects appropriate music based on the results. For example, if the emotional state is "relaxed," it selects relaxation-oriented music.

Step 10:

Based on the location information, the server obtains the latest disaster prevention information for the area from the disaster prevention information service API.

Step 11:

The server integrates the acquired map information, selected music, disaster prevention information, and emotional information to generate optimal personalized information.

Step 12:

The server sends the generated personalized information to the device.

Step 13:

The device analyzes the personalized information received from the server.

Step 14:

The device will provide information to the user in an appropriate manner based on the analyzed information. For example, selected music will be played through earphones, and the smartwatch will notify the user of their current location, disaster prevention information, and emotional advice.

This detailed processing step allows users to efficiently receive the most appropriate information in real time. The system aims to improve the user experience by providing the most appropriate information tailored to the user's current situation and emotional state.

(Example 2)

Next, Example 2 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

Currently, there are systems that provide personalized information based on a user's location and heart rate information, but adding emotional information to these systems would enable even more accurate personalization. However, such systems do not currently exist, and the provision of appropriate information based on the user's emotional state is insufficient, making improving user satisfaction a challenge.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for acquiring user location information, means for acquiring user heart rate information, means for acquiring user emotional information, means for transmitting the acquired location information, heart rate information, and emotional information to the server, means for receiving the transmitted location information, heart rate information, and emotional information, means for acquiring map information around the current location from a map service based on the received location information, means for selecting appropriate music from a music distribution service based on the received heart rate information and emotional information, means for acquiring the latest disaster prevention information corresponding to the area, means for integrating the acquired map information, music, disaster prevention information, and emotional information to generate personalized information to provide to the user, means for transmitting the generated personalized information to the terminal, and terminal means. This enables the provision of highly accurate personalized information based on more multifaceted information including the user's emotional state.

"User location information" refers to the coordinate information of the user's current location.

"User's heart rate information" refers to data measuring the user's heart rate.

"User emotional information" refers to data that expresses the user's emotional state, and is obtained by analyzing facial expressions, tone of voice, etc.

"Server" refers to a computer system that receives and analyzes data sent from a terminal, consolidates the necessary information, and sends it to the terminal.

"Terminal" refers to a device worn or used by a user that acquires location information, heart rate information, and emotional information and transmits it to a server.

A "GPS sensor" is a sensor that receives signals from satellites and determines your current location on Earth.

"Map Service" refers to a service that provides geographic information through a specific API.

"Music distribution service" refers to a service that provides music data through a specific API.

"Disaster prevention information" refers to information regarding emergencies such as natural disasters, and is provided according to the region.

"Personalized information" refers to information specific to each user that is generated by integrating each user's location information, heart rate information, emotional information, etc.

An "emotion engine" refers to software or hardware that analyzes a user's facial expressions and tone of voice to recognize their emotional state.

This invention is a system that provides personalized content in real time based on a user's location information, heart rate information, and emotional state. This system consists of three elements: a server, a terminal, and a user. It also includes an emotion engine for recognizing the user's emotions. Below, we will explain the role and specific functions of each element, as well as the operation of the entire system.

User

Users are individuals who use this system and provide location information, heart rate information, and emotional information. Users wear wearable devices (e.g., smartwatches, earphones, etc.) through which they acquire, send, and receive information. Users begin using the system by wearing the smartwatch on their wrist and the earphones in their ears.

Device

The device is responsible for obtaining location information, heart rate information, and emotional information from the user and sending it to the server. Below, we will explain in detail the main functions performed by the device.

1. Obtaining location information:

The device uses the built-in GPS sensor to obtain the user's current location. Location information is updated periodically (e.g., every 5 seconds).

2. Obtaining heart rate information:

The device uses a built-in heart rate sensor to measure the user's heart rate. Heart rate information is also updated periodically (e.g., every 5 seconds).

3. Acquiring emotional information:

Using the device's built-in camera and microphone, the user's facial expressions and tone of voice are analyzed, and the emotion engine recognizes the user's emotions.

4. Transmission of Information:

The periodically acquired location information, heart rate information, and emotion information are packaged into packets and sent to the server.

Server

The server receives and analyzes information sent from the device and integrates various data to provide the user with the necessary information. Below, we will explain in detail the main functions and processes performed by the server.

1. Receiving information:

The server receives location information, heart rate information, and emotion information sent from the device.

2. Obtaining map information:

The server calls the map service API based on the received location information and obtains map information for the area around the current location.

3. Music Selection:

The server analyzes the received heart rate information and retrieves music from the music distribution service API that corresponds to the user's heart rate and emotional state.

Example: If your heart rate is high (e.g., 160 BPM), choose an up-tempo song. If your emotional state is "excited," choose an energetic song.

4. Obtaining disaster prevention information:

Based on the location information, the server obtains the latest disaster prevention information for the area from the disaster prevention information service API.

5. Generating personalized information:

The acquired map information, selected music, disaster prevention information, and the user's emotional information are integrated to generate optimal personalized information.

6. Sending personalized information:

The server sends the generated personalized information to the device.

Specific examples

For users running

Suppose a user is wearing a smartwatch and earphones while running. The smartwatch periodically obtains the user's current location using a GPS sensor and measures their heart rate using a heart rate sensor. For example, if the user's heart rate is 160 BPM, the device sends this information to the server. At the same time, the emotion engine uses the device's built-in camera and microphone to recognize the user's emotional state as "excited."

The server first calls a map service API based on the location information and obtains map information for the park where the user is running. Next, because the heart rate is high, it obtains up-tempo music from a music distribution service API, and because the emotional state is "excited," it selects energetic music. It also calls a disaster prevention information service API to check disaster prevention information related to the user's current location.

By integrating this data (map information, music information, disaster prevention information), the server generates personalized information optimized for the user and sends it to the device. The device then plays the delivered music through earphones and displays the necessary information on the smartwatch screen.

Example prompt

"Please explain the specific processing steps of a system that provides real-time personalized music and map information when a user's heart rate is high and their emotional state is 'excited' while running."

The flow of the identification process in Example 2 will be explained using Figure 13.

Step 1: The user puts on the wearable device.

The user puts on a smartwatch, earphones, or other wearable device. The device is then ready to acquire the user's location, heart rate, and emotional information. The input for this step is the user themselves, and the output is the wearable device they are wearing.

Step 2: The device acquires location information

The device uses the built-in GPS sensor to obtain the user's current location. For example, if the user is in a park, the device obtains their location coordinates. The input is the signal from the GPS sensor, and the output is location coordinate data (e.g., latitude 35.6895 degrees, longitude 139.6917 degrees). The location information is updated periodically (e.g., every 5 seconds).

Step 3: The device acquires heart rate information.

The device uses a built-in heart rate sensor to measure the user's heart rate. For example, the heart rate is measured at 160 BPM. The input is data from the heart rate sensor, and the output is heart rate data. Heart rate information is also updated periodically (for example, every 5 seconds).

Step 4: The device acquires emotional information.

The device uses a camera and microphone to analyze the user's facial expressions and tone of voice. The emotion engine processes the results of this analysis and recognizes the user's emotional state as "excitement." The input is sensor data from the camera and microphone, and the output is emotional information.

Step 5: The device sends the information to the server.

The device collects location, heart rate, and emotional information and compiles it into packets, which are then sent to the server. The input is location, heart rate, and emotional data, and the output is the data sent to the server. Transmission is carried out periodically.

Step 6: The server receives the information

The server receives location information, heart rate information, and emotional information sent from the device. The input is the data sent from the device, and the output is the data stored on the server.

Step 7: The server retrieves map information.

The server calls the map service API based on the received location information and obtains map information for the area around the current location. The input is location data, and the output is map information data (e.g., obtaining geographic information using the Google Maps API).

Step 8: The server selects music.

The server analyzes the received heart rate and emotional information and retrieves music from a music distribution service API that corresponds to the user's heart rate and emotional state. For example, if a user's heart rate is 160 BPM and they are in an "excited" state, it will select an up-tempo, energetic song. The input is heart rate and emotional information data, and the output is the selected music data (e.g., retrieving songs using the Spotify API).

Step 9: The server obtains disaster prevention information.

Based on the location information, the server obtains the latest disaster prevention information for the region from the disaster prevention information service API. The input is location information data, and the output is disaster prevention information data (e.g., using the disaster prevention information service API to obtain the latest information for the region).

Step 10: The server generates personalized information

The server integrates the acquired map information, selected music, disaster prevention information, and emotional information to generate optimal personalized information. The input is map information, music data, disaster prevention information, and emotional information, and the output is integrated personalized information.

Step 11: The server sends personalized information to the device.

The server sends the generated personalized information to the terminal. The input is the personalized information data, and the output is the data sent to the terminal.

Step 12: The device analyzes the information and notifies the user.

The device analyzes the received personalized information and notifies the user of appropriate information. Specifically, map information and disaster prevention information are displayed on the smartwatch's notification screen, and selected music is played through the earphones. The input is personalized information data, and the output is the form of information provided to the user (screen display and music playback).

(Application Example 2)

Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the headset-type terminal 314 will be referred to as the "terminal."

With the proliferation of information in modern society and the diversification of individual user needs, there is a growing need to provide personalized content tailored to the user's real-time situation. In particular, real-time information provision systems based on biometric data such as location information and heart rate contribute to improving the user experience. However, current systems are not yet able to effectively integrate this data and provide personalized content that also responds to the user's emotional state. Therefore, there is a need for a system that provides personalized information in real time based on the user's location information, heart rate information, and emotional state.

The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring user location information, means for acquiring user heart rate information, means for acquiring the user's emotional state, means for transmitting the received location information, heart rate information, and emotional information to the server, means for acquiring map information near the current location from a map service based on the received location information, means for selecting appropriate music from a music distribution service based on the received heart rate information, means for generating personalized content based on the emotional state, means for acquiring the latest disaster prevention information corresponding to the region, means for integrating the acquired map information, music, disaster prevention information, and emotional information to generate personalized information to provide to the user, means for transmitting the generated personalized information to the terminal, and terminal means. This makes it possible to provide personalized information that integrates appropriate map information, music, and disaster prevention information in real time based on the user's real-time biometric data and emotional state.

"Location information" refers to geographic coordinate information that indicates the user's current location.

"Heart rate information" refers to biometric data such as heart rate and heart rhythm obtained from the user's heartbeat.

"Emotional state" refers to information that indicates the user's emotions, such as psychological states such as stress, relaxation, and excitement.

"Personalized information" refers to content and service information that is individually customized based on the user's location information, heart rate information, and emotional state.

"Map information" is data that shows the terrain, roads, facilities, etc. within a geographic space.

A "music distribution service" is a service that distributes music via the Internet.

"Disaster prevention information" refers to the latest information on disasters such as earthquakes, fires, and typhoons.

A "server" is a central processing unit that receives, analyzes, and processes information sent by users.

"Terminal" refers to a device carried or worn by a user, including smartphones and smartwatches.

This invention is a system that acquires a user's location information, heart rate information, and emotional state in real time and provides personalized content. This system is primarily composed of three elements: a server, a terminal, and a user. Below, we will explain in detail the operation of each element and the system as a whole.

User

The user is the person who uses this system and provides location information, heart rate information, and emotional information. The user wears a wearable device such as a smartwatch, earphones, or smart glasses, and the information is collected and sent to the server.

Device

The terminal is responsible for obtaining information from the user and sending it to the server. Its main functions are as follows:

1. Obtaining location information:

The device uses the built-in GPS sensor to obtain the user's current location. Location information is updated periodically.

2. Obtaining heart rate information:

The device uses a built-in heart rate sensor to measure the user's heart rate, and this information is also updated periodically.

3. Acquiring emotional information:

Using the device's built-in camera and microphone, the emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions.

4. Transmission of Information:

The processed data is sent to the server.

Server

The server receives the information sent from the device and analyzes and processes it. The main processes are as follows:

1. Receiving information:

Receives location information, heart rate information, and emotion information sent from the device.

2. Obtaining map information:

Based on the received location information, the map service API is called to obtain map information around the current location.

3. Music Selection:

Based on heart rate information, appropriate music is selected from a music distribution service API. For example, if the heart rate is high, an up-tempo song is selected. Emotional information is also taken into consideration to select songs that match the user's emotional state.

4. Emotion-based content generation:

Based on emotional information, it generates content such as videos and podcasts that are optimal for the user's emotional state.

5. Obtaining disaster prevention information:

The latest disaster prevention information is obtained from the disaster prevention information service API based on location information.

6. Generating personalized information:

The acquired map information, selected music, disaster prevention information, and emotional information are integrated to generate personalized information to provide to the user.

7. Transmission of Information:

The final generated personalized information is sent to the device.

Specific examples

For users running

The user wears a smartwatch and earphones while running. The device obtains current location information obtained from the GPS sensor and heart rate measured by the heart rate sensor (e.g., 160 BPM). The emotion engine also recognizes the user's emotional state as "excitement." This information is then periodically sent to the server.

The server calls a map service API based on the received location information and obtains map information for the park. Next, because the heart rate information is high, it obtains up-tempo songs from a music distribution service API and selects music suitable for maintaining excitement based on the emotional state. It also calls a disaster prevention information service API to check any necessary disaster prevention information.

This information is then integrated to generate personalized information, which is then sent back to the user's device. The user can check this information on their smartwatch, and the selected music will play from the earphones.

Example of a prompt for a generative AI model:

"A user is feeling stressed while walking through a city. Their heart rate is 120 BPM. What content would you recommend?"

This system allows users to enjoy content that is best suited to their situation in real time.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

Users acquire information by wearing a wearable device.

Specific actions:

The user wears a wearable device such as a smartwatch, earphones, or smart glasses. The user's current location is acquired using a GPS sensor, and heart rate information is measured using a heart rate sensor. In addition, the device's built-in camera and microphone are used to analyze the user's facial expressions and tone of voice to obtain emotional information.

input:

User location, heart rate, and emotional state

output:

Acquired location information, heart rate information, and emotional information

Step 2:

The device sends the acquired information to the server.

Specific actions:

The device periodically collects location information, heart rate information, and emotional information, and then compiles them into packets and sends them to a server. Communication is carried out via the Internet.

input:

Acquired location information, heart rate information, and emotional information

output:

Location, heart rate, and emotional information sent to the server

Step 3:

The server analyzes the information received.

Specific actions:

The server receives location information, heart rate information, and emotional information sent from the device. It analyzes the received data and identifies and organizes each piece of information.

input:

Transmitted location information, heart rate information, and emotional information

output:

Organized location, heart rate, and emotional information

Step 4:

The server obtains map information.

Specific actions:

The server calls the map service API based on the received location information and obtains map information for the area around the current location. The obtained map information provides detailed geographic data about the user's current location.

input:

Organized location information

output:

Retrieved map information

Step 5:

The server selects the music.

Specific actions:

The server analyzes the received heart rate information and retrieves music corresponding to the user's heart rate from the music distribution service API. It also takes emotional information into account and selects music appropriate to the user's emotional state.

input:

Organized heart rate and emotional information

output:

Selected music

Step 6:

The server acquires disaster prevention information.

Specific actions:

The server retrieves the latest disaster prevention information for the area based on the location information from the disaster prevention information service API. This provides information such as disaster risks near the user's current location.

input:

Organized location information

output:

Disaster prevention information obtained

Step 7:

The server generates personalized information.

Specific actions:

The server integrates the acquired map information, selected music, disaster prevention information, and emotional information to generate personalized information to provide to the user.

input:

Map information, music selection, disaster prevention information, emotional information

output:

Generated personalized information

Step 8:

The server sends the generated personalized information to the device.

Specific actions:

The server sends the generated personalized information to the device. The sent information is analyzed on the user's device and notified or played back appropriately.

input:

Generated personalized information

output:

Personalized information sent from the server

Step 9:

Devices provide personalized information to users

Specific actions:

The device analyzes the received personalized information and provides it to the user via the smartwatch or earphones. Specifically, notifications are displayed on the smartwatch and the selected music is played through the earphones.

input:

Personalized information received

output:

Personalized information provided to users

The specific processing unit 290 transmits the results of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). Examples of data generation model 58 include generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>) and Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the headset-type terminal 314.

[Fourth embodiment]

Figure 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

As shown in FIG. 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

The data processing device 12 includes a computer 22, a database 24, and a communication I/F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I/F 26 are also connected to the bus 34. The communication I/F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and/or a LAN (Local Area Network).

The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I/F 44, and a control target 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and control target 443 are also connected to the bus 52.

The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user 20's surroundings (e.g., an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

The communication I/F 44 is connected to the network 54. The communication I/Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I/Fs 44 and 26 is carried out in a secure manner.

The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, in the data processing device 12, specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32.

The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as the specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

Storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

In the robot 414, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

Next, we will explain the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Form for implementing the invention

This invention is a system that provides personalized information in real time based on a user's location information and heart rate information. This system basically consists of three elements: a server, a terminal, and a user. The roles and specific functions of each are explained below.

User

Users are individuals who use this system and provide location and heart rate information. Users wear wearable devices (e.g., smart watches, earphones, audio glasses, etc.) through which they acquire, send, and receive information.

Device

The device is responsible for obtaining location and heart rate information from the user and sending it to the server. The main processes performed by the device are described below.

1. Obtaining location information:

The device uses its built-in GPS sensor to obtain the user's current location.

The acquired location information is updated periodically.

2. Obtaining heart rate information:

The device uses a built-in heart rate sensor to measure the user's heart rate.

Measured heart rate information is also updated regularly.

3. Transmission of information:

The periodically acquired location and heart rate information is packaged into packets and sent to the server.

Server

The server receives and analyzes information sent from the device and integrates various data to provide the user with the necessary information. The main processes performed by the server are described below.

1. Receiving information:

The server receives the location information and heart rate information sent from the device.

2. Obtaining map information:

The server calls the map service API based on the received location information and obtains map information for the area around the current location.

3. Music Selection:

The server analyzes the heart rate based on the received heart rate information and retrieves appropriate music from the music distribution service API.

Example: If your heart rate is high, choose up-tempo music, and if your heart rate is low, choose relaxation music.

4. Obtaining disaster prevention information:

The server obtains disaster prevention information corresponding to the area from the disaster prevention information service API based on the location information.

5. Generating personalized information:

The acquired map information, music, disaster prevention information, and other personalized information (e.g., store coupons, etc.) are integrated as needed to generate optimal personalized information.

6. Sending personalized information:

The server then sends the generated personalized information back to the device.

Specific examples

For users running

The user is wearing the smartwatch while running.

The device obtains the user's current location using a GPS sensor and measures the heart rate using a heart rate sensor. For example, let's say the heart rate is 160 BPM.

The device sends the acquired location and heart rate information to the server.

The server calls the map service API based on the location information and obtains map information within the park.

Next, because the heart rate is high, up-tempo music is retrieved from the music distribution service API. In addition, the disaster prevention information service API is called to check the necessary disaster prevention information.

Finally, this information is integrated to generate personalized information. This information includes information about the user's current location, selected music, necessary disaster prevention information, etc.

The server sends the generated personalized information to the user's device, which analyzes it and sends notifications to the smartwatch or plays music through earphones.

This system allows users to listen to appropriate music in real time while running, while also receiving necessary map and disaster prevention information, making everyday activities safer and more comfortable.

The processing flow is explained below.

Step 1:

The user puts on the wearable device and starts the system. The device completes initial setup and enables the GPS sensor and heart rate sensor.

Step 2:

The device uses the GPS sensor to obtain the user's current location. This location information is obtained periodically (e.g., every 5 seconds).

Step 3:

The device uses a heart rate sensor to measure the user's heart rate. Heart rate information is also collected periodically (e.g., every 5 seconds).

Step 4:

The device collects location and heart rate information and compiles it into packets, which it then sends to the server.

Step 5:

The server receives the location and heart rate information sent from the device.

Step 6:

Based on the location information received by the server, the map service API is called to obtain map information for the area around the current location.

Step 7:

The server analyzes the received heart rate information and calls the music distribution service API to select music according to the user's heart rate.

Example: If your heart rate is high (e.g., 160 BPM), choose up-tempo songs.

Step 8:

Based on the location information, the server obtains the latest disaster prevention information for the area from the disaster prevention information service API.

Step 9:

The server integrates map information, selected music, and disaster prevention information to generate optimal personalized information.

Step 10:

The server sends the generated personalized information to the device.

Step 11:

The device analyzes the personalized information received from the server.

Step 12:

The device will provide information to the user in an appropriate manner based on the information analyzed.

Example: Play selected music through earphones and notify the smartwatch of current location and disaster prevention information.

This processing flow allows users to receive the information they need in real time. The system aims to improve the user experience by providing the most appropriate information tailored to the user's current situation.

(Example 1)

Next, Example 1 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Previous personalized information provision systems did not fully utilize users' location information or heart rate information, making it difficult to provide personalized information in real time. Furthermore, because music selection and disaster prevention information provision were not centralized, users were unable to receive the information they needed in a timely manner, resulting in issues with safety and convenience.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

In this invention, the server includes means for acquiring user location information, means for acquiring user heart rate information, means for transmitting the acquired location information and heart rate information to a central processing unit, means for receiving the transmitted location information and heart rate information, means for acquiring map information near the current location from a map information providing service based on the received location information, means for selecting appropriate music from a music distribution service based on the received heart rate information, means for acquiring the latest disaster prevention information corresponding to the area, means for integrating the acquired map information, music, and disaster prevention information to generate personalized information to provide to the user, means for transmitting the generated personalized information to the terminal, and means for notifying the user of the personalized information received by the terminal. This makes it possible to utilize the user's location information and heart rate information in real time, centrally manage necessary map information, music, and disaster prevention information, and provide it to the user immediately.

"User" means an individual who uses the system and provides location and heart rate information.

"Location information" is data indicating the user's current geographical latitude and longitude.

"Heart rate information" is data that indicates the user's current heart rate.

A "central processing unit" is a device that has the function of receiving and processing location information and heart rate information sent from a terminal.

A "terminal" is a device worn by the user that has the function of acquiring location information and heart rate information and transmitting this information to a central processing unit.

"Map information service" is an online service that provides map information around your current location based on your location information.

A "music distribution service" is an online service that selects and provides appropriate music based on heart rate information.

"Disaster prevention information" refers to information including the latest disaster information and evacuation advisories for a specific area.

"Personalized information" is individually tailored information that is generated by integrating acquired map information, music, and disaster prevention information and providing it to users.

"Notification" is the act of presenting information to the user via a device.

This invention is a system that provides personalized information in real time based on a user's location information and heart rate information. This system basically consists of three elements: a server, a terminal, and a user. The roles and specific functions of each are explained below.

User

Users are individuals who use this system and provide location and heart rate information. Users wear wearable devices (e.g., smart watches, earphones, audio glasses, etc.) through which they acquire, send, and receive information.

Device

The device acquires location and heart rate information from the user and sends it to the server. Specifically, it performs the following processes:

1. Obtaining location information

The device uses the built-in GPS sensor to obtain the user's current location. Using the GPS sensor allows for detailed and accurate location information to be obtained.

The device will periodically update this location information to keep it up to date.

2. Obtaining heart rate information

The device measures the user's heart rate using a built-in heart rate sensor. Heart rate sensors often use optical sensors or electrocardiogram sensors.

Your device will regularly update your heart rate information to keep your health information up to date.

3. Transmission of Information

The acquired location and heart rate information is packaged into packets and sent to the server. This is done using HTTP POST requests or WebSocket communication.

Server

The server receives the information sent from the device, performs the necessary data analysis, and generates personalized information based on the acquired data. The main processes performed by the server are described below.

1. Receiving information

The server receives the location and heart rate information sent from the device. This is done using an HTTP server or WebSocket server.

2. Obtaining map information

Based on the received location information, the server calls a map information service API to obtain map information for the area around the current location. Typically, the Google Maps API is used.

3. Music Selection

The server analyzes the heart rate based on the received heart rate information and retrieves appropriate music from the music distribution service API. If the heart rate is high, an up-tempo song is selected, and if it is low, a relaxing song is selected. Spotify API and Apple Music API are commonly used.

4. Obtaining disaster prevention information

Based on the received location information, the server obtains disaster prevention information corresponding to that area from a disaster prevention information service API. For example, there is the Japan Meteorological Agency API.

5. Generating personalized information

The server integrates the acquired map information, music, disaster prevention information, etc., and generates personalized information to provide to users. This process is implemented using server-side languages such as Python and Node.js.

6. Sending personalized information

The server then sends the generated personalized information back to the device.

Specific examples

For users running

The user is wearing the smartwatch while running.

The device obtains the user's current location using the GPS sensor, records it as latitude 35.658581 and longitude 139.745433, and measures the heart rate as 160 BPM using the heart rate sensor.

The device sends the acquired location and heart rate information to the server.

The server calls the map information service API based on the location information and obtains map information within the park.

Since the heart rate is 160 BPM, the server retrieves up-tempo songs from the music distribution service API. It also calls the disaster prevention information service API to retrieve disaster prevention information for the area around the park.

The server integrates this information to generate personalized information, including park maps, music selections, disaster prevention information, and more.

The server sends the generated personalized information to the device, which receives it and displays map information on the smartwatch, plays music through earphones, and notifies users of disaster prevention information.

Example prompt

Specific examples of prompts are as follows:

"You are currently in a park. Your heart rate is 160 BPM. Please play some upbeat music and get information about nearby disasters."

This system allows users to listen to appropriate music in real time while running, while also receiving necessary map and disaster prevention information, making everyday activities safer and more comfortable.

The flow of the identification process in Example 1 will be explained using Figure 11.

Step 1:

Input: Wearable device worn by the user.

Action: The user puts on a wearable device such as a smartwatch or earphones.

Output: None.

Step 2:

Input: Wearable device.

How it works: The device periodically obtains the user's current location (latitude and longitude) using the built-in GPS sensor.

Specifically, location information is obtained by calling the navigator.geolocation.getCurrentPosition method.

Output: Obtained location information (e.g., latitude 35.658581, longitude 139.745433).

Step 3:

Input: Wearable device.

How it works: The device periodically measures the user's heart rate using the built-in heart rate sensor.

Specifically, the smartwatch collects data from the heart rate sensor and calculates the heart rate.

Output: Acquired heart rate information (e.g., 160 BPM).

Step 4:

Input: Acquired location and heart rate information.

Operation: The device collects location and heart rate information and sends it to the server using an HTTP POST request.

Specifically, data is constructed in JSON format and sent to the "/data" endpoint via an HTTP POST request.

Output: Location and heart rate information sent to the server.

POST /data HTTP/1.1

Host: example.com

Content-Type: application/json

{

"location": {"latitude": 35.658581, "longitude": 139.745433},

"heartRate": 160

}

Step 5:

Input: Location and heart rate information received by the server.

Operation: The server receives location and heart rate information sent from the device and analyzes the data.

Specifically, the server endpoint /data receives the request and parses the JSON-formatted data.

Output: Analyzed location and heart rate information.

Step 6:

Input: Parsed location information.

Operation: The server calls the map information service API based on the analyzed location information and obtains map information for the area around the current location.

Specifically, send an API request like the one below.

GET https://maps.googleapis.com/maps/api/geocode/json?latlng=35.658581,139.745433&key=YOUR_API_KEY

Output: Obtained map information.

Step 7:

Input: Analyzed heart rate information.

Operation: The server calls the music distribution service API based on the analyzed heart rate information and retrieves appropriate music.

If your heart rate is high, choose up-tempo music, and if it's low, choose relaxation music.

Specifically, send an API request like the one below.

GET https://api.spotify.com/v1/recommendations?seed_genres=upbeat&target_tempo=160&key=YOUR_API_KEY

Output: Retrieved music information.

Step 8:

Input: Parsed location information.

Operation: The server calls the disaster prevention information service API based on the analyzed location information and obtains the latest disaster prevention information for the area.

Specifically, send an API request like the one below.

GET https://www.jma.go.jp/bosai/forecast/data/overview_forecast/130000.json

Output: Obtained disaster prevention information.

Step 9:

Input: Obtained map information, music information, and disaster prevention information.

Operation: The server consolidates these and generates personalized information to provide to the user.

Specifically, data is integrated using server-side languages such as Python and Node.js, and personalized information in JSON format is generated.

Output: Generated personalized information.

{

"mapInfo": { ... },

"musicInfo": { ... },

"disasterInfo": { ... }

}

Step 10:

Input: Generated personalization information.

Operation: The server sends the generated personalized information to the device via an HTTP response or push notification.

Specifically, it is returned as an HTTP response or sent using the push notification service.

Output: Personalized information sent to the device.

Step 11:

Input: Personalization information received by the device.

Operation: The device receives and analyzes the personalized information sent from the server.

Specifically, it parses HTTP responses and push notification data.

Output: Parsed personalization information.

Step 12:

Input: Parsed personalization information.

Operation: The device displays the received map information on the screen of the smartwatch or smartphone.

Specifically, the acquired map data is rendered using Google Maps SDK etc.

Output: Displayed map information.

Step 13:

Input: Parsed personalization information.

Operation: The device plays the received music information through the earphones.

Specifically, the obtained music URL is used for streaming playback.

Output: Played music.

Step 14:

Input: Parsed personalization information.

Operation: The device displays the received disaster prevention information using the smartwatch's notification function.

Specifically, we will utilize the function that displays notifications on smartwatches.

Output: Notified disaster prevention information.

(Application Example 1)

Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

Conventional personalized information provision systems have had difficulty efficiently utilizing a user's location information and heart rate information to provide optimal content to users in real time. Furthermore, because these systems rely on limited information sources, it has been difficult to meet the diverse needs of users. The present invention aims to solve these problems and provide a system that provides appropriate music, podcasts, disaster prevention information, and more in real time based on a user's heart rate and location information.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

In this invention, the server includes means for acquiring user location information, means for acquiring user heart rate information, means for acquiring map information around the current location from a map service based on the received location information, means for selecting appropriate music or podcasts from a content distribution service based on the received heart rate information, and means for acquiring the latest disaster prevention information for the area. This allows the user to receive optimal music, podcasts, disaster prevention information, etc. in real time according to their current location and heart rate.

text

"User" refers to an individual who uses this system and provides location information and heart rate information.

"Location information" is data that indicates a user's current geographical location.

"Heart rate information" is data that indicates the user's heart rate and represents the user's health condition and activity level.

The "server" is a system component that receives and analyzes location and heart rate information, and generates and provides various personalized information.

"Map service" is an external service that provides geographical information and map data based on location information.

"Music distribution service" is an external service that provides music tracks in streaming or download format.

"Content distribution service" is an external service that provides multimedia content such as music and podcasts.

"Disaster prevention information" is data that provides information on local disaster risks and emergency response measures.

"Personalized information" is information generated specifically for the user by integrating acquired map information, music, podcasts, and disaster prevention information.

A "terminal" is a device that acquires a user's location information and heart rate information and transmits this information to a server.

A "satellite positioning system sensor" is a device that receives satellite signals and measures location information.

"Tempo-controlled music" refers to music selected based on heart rate and with a rhythm that is appropriate for the user's current activity level.

"Podcast" refers to audio content provided over the Internet and covers a variety of genres, including education, news, and entertainment.

text

This invention is a system that provides personalized information in real time based on a user's location information and heart rate information. This system consists of three elements: the user, the terminal, and the server.

1. User

Users are individuals who use this system and provide location and heart rate information. Specifically, users wear wearable devices such as smartwatches and fitness trackers, through which they acquire and transmit location and heart rate information.

2. Device

The device is responsible for obtaining location and heart rate information from the user and sending it to the server. Specific processes include the following:

Location acquisition: The device uses the built-in satellite positioning system sensor to acquire the user's current location.

Acquiring heart rate information: The device uses the built-in heart rate sensor to measure the user's heart rate.

Information transmission: The acquired location and heart rate information is periodically sent to the server.

3. Server

The server receives and analyzes the information sent from the device and integrates various data to provide the user with the necessary information.

Information reception: The server receives location and heart rate information sent from the device.

Obtaining map information: Calls the map service API based on the received location information and obtains map information around the current location.

Content selection: Analyzes the heart rate based on the received heart rate information and retrieves appropriate music or podcasts from the content distribution service API. For example, if the heart rate is high, upbeat music is selected, and if it is low, a relaxing podcast is selected.

Obtaining disaster prevention information: Obtain the latest disaster prevention information for your area based on your location information from the disaster prevention information service API.

Generating personalized information: Integrates acquired map information, music, podcasts, and disaster prevention information to generate personalized information to provide to users.

Send information: Send the generated personalized information back to your device.

Specific examples

For example, if a user is running, the device will use a satellite positioning system sensor to obtain the user's location information and a heart rate sensor to measure the user's heart rate. Suppose the user's heart rate is 150 BPM. This information is sent to the server, which then calls a map service API to obtain map information for the current location. Based on the high heart rate, the server will obtain up-tempo music from a content distribution service API. In addition, it will obtain the latest local disaster prevention information and combine this information to generate personalized information. Finally, this information is sent to the user's device, allowing the user to receive appropriate music and disaster prevention information in real time.

Example prompt

"If the user's heart rate is 150 BPM, can you recommend some upbeat music suitable for running? Also, tell me the latest news in the area."

The flow of the specific processing in Application Example 1 will be explained using Figure 12.

text

Step 1:

Acquires the user's location and heart rate information. The device uses the built-in satellite positioning system sensor to acquire the user's current location and the heart rate sensor to measure the heart rate. The input is the user's location and heart rate information, and the output is a data packet sent to the server.

Step 2:

The server receives location and heart rate information sent from the device. This is the initial stage of data processing and analysis. The input is the data packets sent from the device, and the output is processable data stored in the server.

Step 3:

The server calls the map service API based on the received location information and obtains map information for the area around the current location. The input is location information, and the output is the obtained map information. Specifically, it sends an API request and receives map information as a response.

Step 4:

The server analyzes the received heart rate information and selects appropriate content (music or podcast). At this time, it calls the content distribution service API based on the heart rate to obtain the appropriate track. The input is the heart rate information, and the output is the selected music or podcast. Specifically, if the heart rate is high, it queries for up-tempo music, and if the heart rate is low, it queries for relaxation podcasts.

Step 5:

The server obtains the latest disaster prevention information for the region based on the location information from the disaster prevention information service API. The input is location information, and the output is disaster prevention information. Specifically, it sends an API request to obtain disaster prevention information for the relevant region, and receives the disaster prevention information as a response.

Step 6:

The server integrates the acquired map information, music, podcasts, and disaster prevention information to generate personalized information to provide to users. The input is the various acquired data, and the output is the integrated personalized information. Specifically, the various data are integrated to generate personalized information in JSON format.

Step 7:

The server sends the generated personalized information to the terminal. The input is the generated personalized information, and the output is a data packet sent to the terminal. Specifically, the server sends data to the terminal and provides information to the user in real time.

Step 8:

The device receives personalized information sent from the server and notifies the user. The input is the data packets sent from the server, and the output is the presentation of information to the user. Specific operations include displaying a notification on the smartwatch display and playing music or podcasts through the earphones.

It is also possible to further combine an emotion engine that estimates the user's emotion. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

Form for implementing the invention

This invention is a system that provides personalized content in real time based on a user's location information, heart rate information, and emotional state. This system consists of three elements: a server, a terminal, and a user. It also includes an emotion engine for recognizing the user's emotions. Below, we will explain the role and specific functions of each element, as well as the operation of the entire system.

User

Users are individuals who use this system and provide location information, heart rate information, and emotional information. Users wear wearable devices (e.g., smart watches, earphones, audio glasses, etc.) through which they acquire, send, and receive information.

Device

The device is responsible for obtaining location information, heart rate information, and emotional information from the user and sending it to the server. Below, we will explain in detail the main functions performed by the device.

1. Obtaining location information:

The device uses its built-in GPS sensor to obtain the user's current location.

Location information is updated periodically (e.g., every 5 seconds).

2. Obtaining heart rate information:

The device uses a built-in heart rate sensor to measure the user's heart rate.

Heart rate information is also updated periodically (e.g., every 5 seconds).

3. Acquiring emotional information:

Using the device's built-in camera and microphone, the user's facial expressions and tone of voice are analyzed, and the emotion engine recognizes the user's emotions.

4. Transmission of Information:

The periodically acquired location information, heart rate information, and emotion information are packaged into packets and sent to the server.

Server

The server receives and analyzes information sent from the device and integrates various data to provide the user with the necessary information. Below, we will explain in detail the main functions and processes performed by the server.

1. Receiving information:

The server receives location information, heart rate information, and emotion information sent from the device.

2. Obtaining map information:

The server calls the map service API based on the received location information and obtains map information for the area around the current location.

3. Music Selection:

The server analyzes the received heart rate information and retrieves music that corresponds to the user's heart rate from the music distribution service API.

Example: If your heart rate is high (e.g., 160 BPM), choose an up-tempo song.

Emotional information is also taken into consideration to select songs that suit the user's emotional state.

Example: If your emotional state is "Relaxed," select relaxation music.

4. Obtaining disaster prevention information:

Based on the location information, the server obtains the latest disaster prevention information for the area from the disaster prevention information service API.

5. Generating personalized information:

The acquired map information, selected music, disaster prevention information, and the user's emotional information are integrated to generate optimal personalized information.

6. Sending personalized information:

The server sends the generated personalized information to the device.

Specific examples

For users running

The user is wearing a smartwatch and earphones while running.

The device acquires the user's current location using a GPS sensor and measures their heart rate using a heart rate sensor. For example, let's say the heart rate is 160 BPM. Furthermore, using the device's built-in camera and microphone, the emotion engine recognizes the user's emotional state as "excited."

The device sends this information to the server.

The server calls the map service API based on the location information and obtains map information within the park.

Next, because the heart rate is high, up-tempo music is retrieved from a music distribution service API. Also, because the emotional state is "excited," songs that will maintain tension are selected.

In addition, it calls the disaster prevention information service API and checks the necessary disaster prevention information.

Finally, this information is integrated to generate personalized information, including information about the user's current location, selected music, necessary disaster prevention information, and information based on their emotional state.

The server sends the generated personalized information to the user's device, which analyzes it, notifies the smartwatch, and plays music through the earphones.

This system allows users to receive appropriate music, map information, and disaster prevention information in real time while running. It also takes into account the user's emotional state, making it possible to provide a more personalized experience.

The processing flow is explained below.

Step 1:

The user puts on the wearable device and launches the application. The device activates the GPS sensor and heart rate sensor.

Step 2:

The device obtains the user's current location using the GPS sensor. This location information is updated periodically (e.g., every 5 seconds).

Step 3:

The device measures the user's heart rate using a heart rate sensor. Heart rate information is also updated periodically (e.g., every 5 seconds).

Step 4:

The device's built-in camera periodically captures the user's facial expressions, and the microphone collects the user's tone of voice. Based on this, the emotion engine analyzes the user's emotional state.

Step 5:

The device collects location information, heart rate information, and emotion information and sends it to the server in packets.

Step 6:

The server receives the location information, heart rate information, and emotion information sent from the device.

Step 7:

Based on the location information received by the server, the map service API is called to obtain map information for the area around the current location.

Step 8:

The server analyzes the received heart rate information and retrieves music corresponding to the user's heart rate from the music distribution service API. For example, if the heart rate is 160 BPM, an up-tempo song is selected.

Step 9:

The server uses an emotion engine to analyze the user's emotional state and selects appropriate music based on the results. For example, if the emotional state is "relaxed," it selects relaxation-oriented music.

Step 10:

Based on the location information, the server obtains the latest disaster prevention information for the area from the disaster prevention information service API.

Step 11:

The server integrates the acquired map information, selected music, disaster prevention information, and emotional information to generate optimal personalized information.

Step 12:

The server sends the generated personalized information to the device.

Step 13:

The device analyzes the personalized information received from the server.

Step 14:

The device will provide information to the user in an appropriate manner based on the analyzed information. For example, selected music will be played through earphones, and the smartwatch will notify the user of their current location, disaster prevention information, and emotional advice.

This detailed processing step allows users to efficiently receive the most appropriate information in real time. The system aims to improve the user experience by providing the most appropriate information tailored to the user's current situation and emotional state.

(Example 2)

Next, Example 2 will be described. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

Currently, there are systems that provide personalized information based on a user's location and heart rate information, but adding emotional information to these systems would enable even more accurate personalization. However, such systems do not currently exist, and the provision of appropriate information based on the user's emotional state is insufficient, making improving user satisfaction a challenge.

The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

In this invention, the server includes means for acquiring user location information, means for acquiring user heart rate information, means for acquiring user emotional information, means for transmitting the acquired location information, heart rate information, and emotional information to the server, means for receiving the transmitted location information, heart rate information, and emotional information, means for acquiring map information around the current location from a map service based on the received location information, means for selecting appropriate music from a music distribution service based on the received heart rate information and emotional information, means for acquiring the latest disaster prevention information corresponding to the area, means for integrating the acquired map information, music, disaster prevention information, and emotional information to generate personalized information to provide to the user, means for transmitting the generated personalized information to the terminal, and terminal means. This enables the provision of highly accurate personalized information based on more multifaceted information including the user's emotional state.

"User location information" refers to the coordinate information of the user's current location.

"User's heart rate information" refers to data measuring the user's heart rate.

"User emotional information" refers to data that expresses the user's emotional state, and is obtained by analyzing facial expressions, tone of voice, etc.

A "server" is a computer system that receives and analyzes data sent from a terminal, consolidates the necessary information, and sends it to the terminal.

"Terminal" refers to a device worn or used by a user that acquires location information, heart rate information, and emotional information and transmits it to a server.

A "GPS sensor" is a sensor that receives signals from satellites and determines your current location on Earth.

"Map Service" refers to a service that provides geographic information through a specific API.

"Music distribution service" refers to a service that provides music data through a specific API.

"Disaster prevention information" refers to information regarding emergencies such as natural disasters, and is provided according to the region.

"Personalized information" refers to information specific to each user that is generated by integrating each user's location information, heart rate information, emotional information, etc.

An "emotion engine" refers to software or hardware that analyzes a user's facial expressions and tone of voice to recognize their emotional state.

This invention is a system that provides personalized content in real time based on a user's location information, heart rate information, and emotional state. This system consists of three elements: a server, a terminal, and a user. It also includes an emotion engine for recognizing the user's emotions. Below, we will explain the role and specific functions of each element, as well as the operation of the entire system.

User

Users are individuals who use this system and provide location information, heart rate information, and emotional information. Users wear wearable devices (e.g., smartwatches, earphones, etc.) through which they acquire, send, and receive information. Users begin using the system by wearing the smartwatch on their wrist and the earphones in their ears.

Device

The device is responsible for obtaining location information, heart rate information, and emotional information from the user and sending it to the server. Below, we will explain in detail the main functions performed by the device.

1. Obtaining location information:

The device uses the built-in GPS sensor to obtain the user's current location. Location information is updated periodically (e.g., every 5 seconds).

2. Obtaining heart rate information:

The device uses a built-in heart rate sensor to measure the user's heart rate. Heart rate information is also updated periodically (e.g., every 5 seconds).

3. Acquiring emotional information:

Using the device's built-in camera and microphone, the user's facial expressions and tone of voice are analyzed, and the emotion engine recognizes the user's emotions.

4. Transmission of Information:

The periodically acquired location information, heart rate information, and emotion information are packaged into packets and sent to the server.

Server

The server receives and analyzes information sent from the device and integrates various data to provide the user with the necessary information. Below, we will explain in detail the main functions and processes performed by the server.

1. Receiving information:

The server receives location information, heart rate information, and emotion information sent from the device.

2. Obtaining map information:

The server calls the map service API based on the received location information and obtains map information for the area around the current location.

3. Music Selection:

The server analyzes the received heart rate information and retrieves music from the music distribution service API that corresponds to the user's heart rate and emotional state.

Example: If your heart rate is high (e.g., 160 BPM), choose an up-tempo song. If your emotional state is "excited," choose an energetic song.

4. Obtaining disaster prevention information:

Based on the location information, the server obtains the latest disaster prevention information for the area from the disaster prevention information service API.

5. Generating personalized information:

The acquired map information, selected music, disaster prevention information, and the user's emotional information are integrated to generate optimal personalized information.

6. Sending personalized information:

The server sends the generated personalized information to the device.

Specific examples

For users running

Suppose a user is wearing a smartwatch and earphones while running. The smartwatch periodically obtains the user's current location using a GPS sensor and measures their heart rate using a heart rate sensor. For example, if the user's heart rate is 160 BPM, the device sends this information to the server. At the same time, the emotion engine uses the device's built-in camera and microphone to recognize the user's emotional state as "excited."

The server first calls a map service API based on the location information and obtains map information for the park where the user is running. Next, because the heart rate is high, it obtains up-tempo music from a music distribution service API, and because the emotional state is "excited," it selects energetic music. It also calls a disaster prevention information service API to check disaster prevention information related to the user's current location.

By integrating this data (map information, music information, disaster prevention information), the server generates personalized information optimized for the user and sends it to the device. The device then plays the delivered music through earphones and displays the necessary information on the smartwatch screen.

Example prompt

"Please explain the specific processing steps of a system that provides real-time personalized music and map information when a user's heart rate is high and their emotional state is 'excited' while running."

The flow of the identification process in Example 2 will be explained using Figure 13.

Step 1: The user puts on the wearable device.

The user puts on a smartwatch, earphones, or other wearable device. The device is then ready to acquire the user's location, heart rate, and emotional information. The input for this step is the user themselves, and the output is the wearable device they are wearing.

Step 2: The device acquires location information

The device uses the built-in GPS sensor to obtain the user's current location. For example, if the user is in a park, the device obtains their location coordinates. The input is the signal from the GPS sensor, and the output is location coordinate data (e.g., latitude 35.6895 degrees, longitude 139.6917 degrees). The location information is updated periodically (e.g., every 5 seconds).

Step 3: The device acquires heart rate information.

The device uses a built-in heart rate sensor to measure the user's heart rate. For example, the heart rate is measured at 160 BPM. The input is data from the heart rate sensor, and the output is heart rate data. Heart rate information is also updated periodically (for example, every 5 seconds).

Step 4: The device acquires emotional information.

The device uses a camera and microphone to analyze the user's facial expressions and tone of voice. The emotion engine processes the results of this analysis and recognizes the user's emotional state as "excitement." The input is sensor data from the camera and microphone, and the output is emotional information.

Step 5: The device sends the information to the server.

The device collects location, heart rate, and emotional information and compiles it into packets, which are then sent to the server. The input is location, heart rate, and emotional data, and the output is the data sent to the server. Transmission is carried out periodically.

Step 6: The server receives the information

The server receives location information, heart rate information, and emotional information sent from the device. The input is the data sent from the device, and the output is the data stored on the server.

Step 7: The server retrieves map information.

The server calls the map service API based on the received location information and obtains map information for the area around the current location. The input is location data, and the output is map information data (e.g., obtaining geographic information using the Google Maps API).

Step 8: The server selects music.

The server analyzes the received heart rate and emotional information and retrieves music from a music distribution service API that corresponds to the user's heart rate and emotional state. For example, if a user's heart rate is 160 BPM and they are in an "excited" state, it will select an up-tempo, energetic song. The input is heart rate and emotional information data, and the output is the selected music data (e.g., retrieving songs using the Spotify API).

Step 9: The server obtains disaster prevention information.

Based on the location information, the server obtains the latest disaster prevention information for the region from the disaster prevention information service API. The input is location information data, and the output is disaster prevention information data (e.g., using the disaster prevention information service API to obtain the latest information for the region).

Step 10: The server generates personalized information

The server integrates the acquired map information, selected music, disaster prevention information, and emotional information to generate optimal personalized information. The input is map information, music data, disaster prevention information, and emotional information, and the output is integrated personalized information.

Step 11: The server sends personalized information to the device.

The server sends the generated personalized information to the terminal. The input is the personalized information data, and the output is the data sent to the terminal.

Step 12: The device analyzes the information and notifies the user.

The device analyzes the received personalized information and notifies the user of appropriate information. Specifically, map information and disaster prevention information are displayed on the smartwatch's notification screen, and selected music is played through the earphones. The input is personalized information data, and the output is the form of information provided to the user (screen display and music playback).

(Application Example 2)

Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 will be referred to as the "terminal."

With the proliferation of information in modern society and the diversification of individual user needs, there is a growing need to provide personalized content tailored to the user's real-time situation. In particular, real-time information provision systems based on biometric data such as location information and heart rate contribute to improving the user experience. However, current systems are not yet able to effectively integrate this data and provide personalized content that also responds to the user's emotional state. Therefore, there is a need for a system that provides personalized information in real time based on the user's location information, heart rate information, and emotional state.

The identification processing by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring user location information, means for acquiring user heart rate information, means for acquiring the user's emotional state, means for transmitting the received location information, heart rate information, and emotional information to the server, means for acquiring map information near the current location from a map service based on the received location information, means for selecting appropriate music from a music distribution service based on the received heart rate information, means for generating personalized content based on the emotional state, means for acquiring the latest disaster prevention information corresponding to the region, means for integrating the acquired map information, music, disaster prevention information, and emotional information to generate personalized information to provide to the user, means for transmitting the generated personalized information to the terminal, and terminal means. This makes it possible to provide personalized information that integrates appropriate map information, music, and disaster prevention information in real time based on the user's real-time biometric data and emotional state.

"Location information" refers to geographic coordinate information that indicates the user's current location.

"Heart rate information" refers to biometric data such as heart rate and heart rhythm obtained from the user's heartbeat.

"Emotional state" refers to information that indicates the user's emotions, such as psychological states such as stress, relaxation, and excitement.

"Personalized information" refers to content and service information that is individually customized based on the user's location information, heart rate information, and emotional state.

"Map information" is data that shows the terrain, roads, facilities, etc. within a geographic space.

A "music distribution service" is a service that distributes music via the Internet.

"Disaster prevention information" refers to the latest information on disasters such as earthquakes, fires, and typhoons.

A "server" is a central processing unit that receives, analyzes, and processes information sent by users.

"Terminal" refers to a device carried or worn by a user, including smartphones and smartwatches.

This invention is a system that acquires a user's location information, heart rate information, and emotional state in real time and provides personalized content. This system is primarily composed of three elements: a server, a terminal, and a user. Below, we will explain in detail the operation of each element and the system as a whole.

User

The user is the person who uses this system and provides location information, heart rate information, and emotional information. The user wears a wearable device such as a smartwatch, earphones, or smart glasses, and the information is collected and sent to the server.

Device

The terminal is responsible for obtaining information from the user and sending it to the server. Its main functions are as follows:

1. Obtaining location information:

The device uses the built-in GPS sensor to obtain the user's current location. Location information is updated periodically.

2. Obtaining heart rate information:

The device uses a built-in heart rate sensor to measure the user's heart rate, and this information is also updated periodically.

3. Acquiring emotional information:

Using the device's built-in camera and microphone, the emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotions.

4. Transmission of Information:

The processed data is sent to the server.

Server

The server receives, analyzes, and processes information sent from the device. The main processes are as follows:

1. Receiving information:

Receives location information, heart rate information, and emotion information sent from the device.

2. Obtaining map information:

Based on the received location information, the map service API is called to obtain map information around the current location.

3. Music Selection:

Based on heart rate information, appropriate music is selected from a music distribution service API. For example, if the heart rate is high, an up-tempo song is selected. Emotional information is also taken into consideration to select songs that match the user's emotional state.

4. Emotion-based content generation:

Based on emotional information, it generates content such as videos and podcasts that are optimal for the user's emotional state.

5. Obtaining disaster prevention information:

The latest disaster prevention information is obtained from the disaster prevention information service API based on location information.

6. Generating personalized information:

The acquired map information, selected music, disaster prevention information, and emotional information are integrated to generate personalized information to provide to the user.

7. Transmission of Information:

The final generated personalized information is sent to the device.

Specific examples

For users running

The user wears a smartwatch and earphones while running. The device obtains current location information obtained from the GPS sensor and heart rate measured by the heart rate sensor (e.g., 160 BPM). The emotion engine also recognizes the user's emotional state as "excitement." This information is then periodically sent to the server.

The server calls a map service API based on the received location information and obtains map information for the park. Next, because the heart rate information is high, it obtains up-tempo songs from a music distribution service API and selects music suitable for maintaining excitement based on the emotional state. It also calls a disaster prevention information service API to check any necessary disaster prevention information.

This information is then integrated to generate personalized information, which is then sent back to the user's device. The user can check this information on their smartwatch, and the selected music will play from the earphones.

Example of a prompt for a generative AI model:

"A user is feeling stressed while walking through a city. Their heart rate is 120 BPM. What content would you recommend?"

This system allows users to enjoy content that is best suited to their situation in real time.

The flow of the specific processing in Application Example 2 will be explained using Figure 14.

Step 1:

Users acquire information by wearing a wearable device.

Specific actions:

The user wears a wearable device such as a smartwatch, earphones, or smart glasses. The user's current location is acquired using a GPS sensor, and heart rate information is measured using a heart rate sensor. In addition, the device's built-in camera and microphone are used to analyze the user's facial expressions and tone of voice to obtain emotional information.

input:

User location, heart rate, and emotional state

output:

Acquired location information, heart rate information, and emotional information

Step 2:

The device sends the acquired information to the server.

Specific actions:

The device periodically collects location information, heart rate information, and emotional information, and then compiles them into packets and sends them to a server. Communication is carried out via the Internet.

input:

Acquired location information, heart rate information, and emotional information

output:

Location, heart rate, and emotional information sent to the server

Step 3:

The server analyzes the information received.

Specific actions:

The server receives location information, heart rate information, and emotional information sent from the device. It analyzes the received data and identifies and organizes each piece of information.

input:

Transmitted location information, heart rate information, and emotional information

output:

Organized location, heart rate, and emotional information

Step 4:

The server obtains map information.

Specific actions:

The server calls the map service API based on the received location information and obtains map information for the area around the current location. The obtained map information provides detailed geographic data about the user's current location.

input:

Organized location information

output:

Retrieved map information

Step 5:

The server selects the music.

Specific actions:

The server analyzes the received heart rate information and retrieves music corresponding to the user's heart rate from the music distribution service API. It also takes emotional information into account and selects music appropriate to the user's emotional state.

input:

Organized heart rate and emotional information

output:

Selected music

Step 6:

The server acquires disaster prevention information.

Specific actions:

The server retrieves the latest disaster prevention information for the area based on the location information from the disaster prevention information service API. This provides information such as disaster risks near the user's current location.

input:

Organized location information

output:

Disaster prevention information obtained

Step 7:

The server generates personalized information.

Specific actions:

The server integrates the acquired map information, selected music, disaster prevention information, and emotional information to generate personalized information to provide to the user.

input:

Map information, selected music, disaster prevention information, emotional information

output:

Generated personalized information

Step 8:

The server sends the generated personalized information to the device.

Specific actions:

The server sends the generated personalized information to the device. The sent information is analyzed on the user's device and notified or played back appropriately.

input:

Generated personalized information

output:

Personalized information sent from the server

Step 9:

Devices provide personalized information to users

Specific actions:

The device analyzes the received personalized information and provides it to the user via the smartwatch or earphones. Specifically, notifications are displayed on the smartwatch and the selected music is played through the earphones.

input:

Personalized information received

output:

Personalized information provided to users

The specific processing unit 290 transmits the results of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the results of the specific processing. The microphone 238 acquires audio indicating the user input regarding the results of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

Data generation model 58 is what is known as generative AI (artificial intelligence). Examples of data generation model 58 include generative AI such as ChatGPT (Internet search <URL: https://openai.com/blog/chatgpt>) and Gemini (Internet search <URL: https://gemini.google.com/?hl=ja>). Data generation model 58 is obtained by performing deep learning on a neural network. A prompt containing an instruction is input to data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input. Data generation model 58 performs inference on the input inference data in accordance with the instructions indicated by the prompt, and outputs the inference results in the form of data such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and/or summarization.

In the above embodiment, an example was given in which the specific processing was performed by the data processing device 12, but the technology disclosed herein is not limited to this, and the specific processing may also be performed by the robot 414.

The emotion identification model 59, which serves as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see Figure 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

Figure 9 shows an emotion map 400 on which multiple emotions are mapped. In emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both emotions and mental states. Emotions that are generally generated from reactions that occur in the brain are arranged on the left side of the concentric circles. Emotions that are generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions that are generally generated from reactions that occur in the brain and are induced by situational judgment are arranged above and below the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the top side of the concentric circles, and the emotion of "discomfort" is arranged on the bottom side. In this way, emotion map 400 maps multiple emotions based on the structure by which emotions are generated, with emotions that tend to occur simultaneously being mapped close together.

These emotions are distributed in the 3 o'clock direction on emotion map 400, and usually fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness takes precedence over internal sensations, resulting in a calm impression.

The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further out you go on emotion map 400, the more visible (expressed in behavior) the emotion becomes.

Here, human emotions are based on various balances such as posture and blood sugar levels, and when these balances deviate from the ideal, it indicates discomfort, and when they approach the ideal, it indicates pleasure. Emotions can also be created for robots, cars, motorcycles, etc., based on various balances such as posture and remaining battery life, so that when these balances deviate from the ideal, it indicates discomfort, and when they approach the ideal, it indicates pleasure. Emotion maps may be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and emotional brain physiological signal analysis systems, Tokushima University, doctoral dissertation: https://ci.nii.ac.jp/naid/500000375379). The left half of the emotion map is lined with emotions belonging to an area called "reaction," where sensation is dominant. The right half of the emotion map is lined with emotions belonging to an area called "situation," where situational awareness is dominant.

The emotion map defines two emotions that encourage learning. One is the negative emotion around the middle of "repentance" or "reflection" on the situation side. In other words, this is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the positive emotion around "desire" on the response side. In other words, this is when the robot experiences positive feelings such as "I want more" or "I want to know more."

The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are close to each other have similar values, as in the emotion map 900 shown in Figure 10. Figure 10 shows an example in which multiple emotions, such as "relieved," "calm," and "reassuring," have similar emotion values.

The system according to the present disclosure has been described above primarily in terms of the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may also be implemented as a general information processing system. The present disclosure may also be implemented, for example, as a software program that runs on a personal computer or an application that runs on a smartphone, etc. The method according to the present disclosure may also be provided to users in the form of SaaS (Software as a Service).

In the above embodiment, an example was given in which a specific process was performed by a single computer 22, but the technology of the present disclosure is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including the computer 22. For example, the data generation model 58 may be provided in a device external to the data processing device 12, and data may be generated in that external device in response to input data.

In the above embodiment, an example was described in which the specific processing program 56 is stored in the storage 32, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

It is not necessary to store the entire specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entire specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

The following types of processors can be used as hardware resources for executing specific processes. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource for executing specific processes by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which are processors with a circuit configuration designed specifically for executing specific processes. All processors have built-in or connected memory, and all use the memory to execute specific processes.

The hardware resource that executes the specific processing may be composed of one of these various processors, or may be composed of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). The hardware resource that executes the specific processing may also be a single processor.

As an example of a configuration using a single processor, first, there is a configuration in which one processor is configured using a combination of one or more CPUs and software, and this processor functions as a hardware resource that executes specific processing. Second, there is a configuration in which a processor is used to realize the functions of an entire system, including multiple hardware resources that execute specific processing, on a single IC chip, as typified by SoC (System-on-a-chip). In this way, specific processing is realized using one or more of the various processors listed above as hardware resources.

More specifically, the hardware structure of these various processors can be an electrical circuit that combines circuit elements such as semiconductor devices. Furthermore, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps can be added, or the processing order can be rearranged, all within the scope of the spirit of the invention.

The above-described written content and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above explanation of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or substitutions may be made to the above-described written content and illustrations, as long as they do not deviate from the spirit of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described written content and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

All publications, patent applications, and technical standards mentioned in this specification are incorporated by reference herein to the same extent as if each individual publication, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

The following is further disclosed regarding the above embodiments.

(Claim 1)

Means for obtaining user location information,

Means for acquiring the user's heart rate information;

Means for transmitting the acquired location information and heart rate information to a server;

Means for receiving the transmitted location information and heart rate information;

Means for obtaining map information around the current location from a map service based on the received location information;

A means for selecting appropriate music from a music streaming service based on the received heart rate information;

A means of obtaining the latest disaster prevention information for your area,

Means for integrating the acquired map information, music, and disaster prevention information to generate personalized information to provide to the user;

Means for transmitting the generated personalized information to the terminal;

A system including terminal means.

(Claim 2)

The system described in claim 1, characterized in that the means for acquiring user location information is performed using a GPS sensor.

(Claim 3)

The system described in claim 1 further comprises means for selecting tempo-controlled music based on the user's heart rate.

"Example 1"

(Claim 1)

Means for obtaining user location information,

Means for acquiring the user's heart rate information;

Means for transmitting the acquired location information and heart rate information to a central processing unit;

Means for receiving the transmitted location information and heart rate information;

Means for obtaining map information around the current location from a map information service based on the received location information;

A means for selecting appropriate music from a music streaming service based on the received heart rate information;

A means of obtaining the latest disaster prevention information for your area,

Means for integrating the acquired map information, music, and disaster prevention information to generate personalized information to provide to the user;

Means for transmitting the generated personalized information to the terminal;

Means for notifying the user of the personalized information received by the device;

A system including

(Claim 2)

The system described in claim 1, characterized in that the means for acquiring user location information is performed using a satellite positioning system sensor.

(Claim 3)

The system described in claim 1 further comprises means for selecting tempo-controlled music based on the user's heart rate.

"Application Example 1"

text

(Claim 1)

Means for obtaining user location information,

Means for acquiring the user's heart rate information;

Means for transmitting the acquired location information and heart rate information to a server;

Means for receiving the transmitted location information and heart rate information;

Means for obtaining map information around the current location from a map service based on the received location information;

A means for selecting appropriate music or podcasts from a content distribution service based on the received heart rate information;

A means of obtaining the latest disaster prevention information for your area,

Means for integrating the acquired map information, music, podcasts, and disaster prevention information to generate personalized information to provide to the user;

Means for transmitting the generated personalized information to the terminal;

A system including terminal means.

(Claim 2)

The system described in claim 1, characterized in that the means for acquiring user location information is performed using a satellite positioning system sensor.

(Claim 3)

The system of claim 1, further comprising means for selecting tempo-controlled music and engaging podcasts based on the user's heart rate.

"Example 2: Combining Emotion Engines"

(Claim 1)

Means for obtaining user location information,

Means for acquiring the user's heart rate information;

Means for acquiring user emotional information,

Means for transmitting the acquired location information, heart rate information, and emotion information to a server;

Means for receiving the transmitted location information, heart rate information, and emotion information;

Means for obtaining map information around the current location from a map service based on the received location information;

Means for selecting appropriate music from a music distribution service based on the received heart rate information and emotional information;

A means of obtaining the latest disaster prevention information for your area,

Means for integrating the acquired map information, music, disaster prevention information, and emotional information to generate personalized information to provide to the user;

Means for transmitting the generated personalized information to the terminal;

A system including terminal means.

(Claim 2)

The system described in claim 1, characterized in that the means for obtaining user location information is performed using a positioning system sensor.

(Claim 3)

The system of claim 1 further comprises means for selecting music with controlled tempo and mood based on the user's heart rate and emotional state.

"Application Example 2: Combining Emotion Engines"

(Claim 1)

Means for obtaining user location information,

Means for acquiring the user's heart rate information;

Means for transmitting the acquired location information and heart rate information to a server;

Means for receiving the transmitted location information and heart rate information;

Means for obtaining map information around the current location from a map service based on the received location information;

A means for selecting appropriate music from a music streaming service based on the received heart rate information;

Means for acquiring the user's emotional state,

Means for generating personalized content based on emotional state;

A means of obtaining the latest disaster prevention information for your area,

Means for integrating the acquired map information, music, disaster prevention information, and emotional information to generate personalized information to provide to the user;

Means for transmitting the generated personalized information to the terminal;

A system including terminal means.

(Claim 2)

The system described in claim 1, characterized in that the means for acquiring user location information is performed using a GPS sensor.

(Claim 3)

The system described in claim 1 further comprises means for selecting tempo-controlled music based on the user's heart rate and emotional state.

10, 210, 310, 410 Data processing system 12 Data processing device 14 Smart device 214 Smart glasses 314 Headset type terminal 414 Robot

Claims (3)

A means for acquiring user location information;
means for acquiring heart rate information of a user;
means for transmitting the acquired location information and heart rate information to a server;
means for receiving the transmitted location information and heart rate information;
means for acquiring map information of the vicinity of the current location from a map service based on the received location information;
a means for selecting appropriate music from a music distribution service based on the received heart rate information;
A means of obtaining the latest disaster prevention information for the area;
means for integrating the acquired map information, music, and disaster prevention information to generate personalized information to be provided to a user;
means for transmitting the generated personalized information to the terminal;
A system including a terminal means. The system described in claim 1, characterized in that the means for acquiring user location information is performed using a GPS sensor. The system described in claim 1 further comprises means for selecting tempo-controlled music based on the user's heart rate.