WO2019004851A1 - Method and system of managing playback of media files at commercial facility - Google Patents

Abstract

This solution is a software and hardware complex for online offline business, in particular, designed to centrally manage a set of media files, such as audio files, video clips, advertising information, etc., at various retail sites, such as retail chains, banks , cafes and restaurants, hotels and business centers, car dealerships, gas stations, shopping centers, etc. In a preferred embodiment of the claimed solution, a computer-implemented way to control media playback is presented. The files on the trade object, containing the stages at which: - the data of the visitors of the trade object are obtained, the data being images of the faces of the visitors; - determine the sex and age of each visitor based on the above images; - determine the groups of visitors that are currently on the shopping facility; - form a list of media files based on certain groups of visitors for playback on the audio / video playback of the shopping facility; - play mentioned media files.

Description

 METHOD AND CONTROL SYSTEM MEDIA PLAYBACK

 FILES ON TRADE OBJECT

TECHNICAL FIELD This solution is a software and hardware complex for offline business network, in particular, is intended for centralized management of a set of media files, such as audio files, video clips, advertising information, etc., on various retail sites, for example, retail networks, banks, cafes and restaurants, hotels and business centers, car dealerships, gas stations, shopping centers, etc. LEVEL OF TECHNOLOGY

To date, there are several solutions that allow you to dynamically control the playback of audio and video on shopping sites.

Patent application WO 2007092053 (Musicstrands INC., August 16, 2007) discloses a system for dynamically targeting interactive advertising by creating media networks formed from files based on playlists. A playlist is created for each of the system users based on their preferences.

DISCLOSURE OF INVENTION

The existing technical problem is the creation of an easy-to-manage system that would have broad functionality that allows for dynamic management of media files depending on the target audience that is at a given point in time at shopping sites.

The claimed solution is designed to quickly and easily create, maintain and update the background music and video to effectively influence customers, taking into account the laws of audio marketing and consumer portrait, as well as analyze the consumer at points of sale.

The technical effect of the implementation of the claimed solution is to provide a centralized management of a set of media files by creating media playlists for different age and gender groups of visitors based on information obtained using video analytics, and playing audio, video and advertising messages intended for a specific target audience. at a given time in a shopping facility. In a preferred embodiment of the claimed solution, a computer-implemented method is presented for controlling the playback of media files on a trading facility, comprising:

- get the data of visitors of the shopping facility, and the data are images of the visitors;

- determine the sex and age of each visitor based on the above images;

- determine the groups of visitors that are currently on the shopping facility;

- form a list of media files based on certain groups of visitors for playback on the audio / video playback of the shopping facility;

- play mentioned media files.

In the private embodiment, the image of the faces of visitors is obtained from the video stream obtained using a video camera.

In another particular embodiment, the video stream is processed using a face detection and recognition algorithm.

In another particular embodiment, the determination of the gender and age of visitors is performed by analyzing facial images.

In another particular embodiment, in determining groups of visitors, gender and age groups are determined by counting the number of visitors.

In another particular embodiment, the list of media files is generated as a playlist based on the rules for playing media files.

In another particular embodiment, the rules are formed based on data on age and sex groups of visitors. In another particular embodiment, the playlist contains a schedule for playing audio and / or video files.

In another particular embodiment, the update of the playlist is carried out dynamically at specified intervals or on the basis of information about the number of visitors of a particular age and gender group. In another preferred embodiment, a media file playback control system on a shopping facility is presented, comprising network-connected audio / video transmissions, media file playback, at least one video fixation device and a computing device, the video recording tool being configured to receive a video stream of visitors'images;

- the computing device is configured to process the incoming video stream, determine images of visitors' faces, then process data to determine the gender and age of visitors, count the number of visitors that form the age and sex group, determine the playlist of media files for the corresponding age and age group at a given point in time;

- audio / video playback means are adapted to play audio and video files in response to receiving commands from a computing device based on the received playlist of media files. In a particular embodiment, the system further comprises a server containing an intelligent processing system that performs the processing of data received from the computing device, on the basis of which the update of the playlist of media files is performed.

In another private implementation, the data network is wired and / or wireless.

In another private implementation, the audio / video playback tools are selected from the group: audio speakers, TV, monitor, soundbar, F-panel, projector, touch panel, monoblock, augmented reality devices, or combinations thereof. In another private implementation, the system additionally contains the lighting control subsystem of the shopping facility.

In another private implementation, the video capture means is an IP camera or IP ΡΤΖ-camera, or a combination of both.

In another private implementation, the computing device contains a client module that initiates data transfer to the server and activates the playback of media files on audio / video playback devices. In another private implementation, the computing device comprises a video analytics module configured to process the incoming video stream.

In another private implementation, the client module contains a set of media files that are played in accordance with the playlist received from the server.

In another private implementation, the intelligent processing system generates rules for playing media files, on the basis of which a playlist is created.

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 illustrates a general view of a media file management system.

FIG. 2A illustrates an example layout of a system in a sales area.

FIG. Figure 2B illustrates an example of how video fixers work.

FIG. 3 illustrates the general steps of the method.

FIG. 4A illustrates the steps of a method of operating video analytics. FIG. 4B illustrates the steps of a method for forming a play list.

FIG. 5 illustrates the general scheme of a computing device.

FIG. 6 illustrates the operation of the logic of the software modules of a computing device.

IMPLEMENTATION OF THE INVENTION FIG. 1 shows a general view of the system (100), which includes a set of software and hardware solutions necessary for the implementation of the task.

The information necessary for the formation of a playlist (or playlist) of media files on a trade facility comes from video recording media (153) located on the site. The tools (153) can be selected from a wide range of cameras (camcorders), in particular, IP cameras, PTZ cameras, etc. At the trade facility, one or several devices can be installed that can communicate with the device (140) via wired and / or wireless information transfer channel (for example, wired connection via USB interface, Wi-Fi data transfer, etc.). P.). Information obtained from video fixation devices (153) is transmitted to a computing device (140), which may be a standard computer. Information about the visitors (10) of the shopping facility is processed by the client module (130), which may be part of the computing device (140) or located on the device connected to the computing device (140) via a data network.

The server (120) contains the necessary list of media files on the basis of which playlists are formed, which are transmitted along with the necessary media files to the computing device (140). Files intended for reproduction on audio playback devices (152) and display devices (151) installed on the territory of a trade object are contained in the memory of the computing device (1 0).

The system of intellectual processing (SW), located on the server (120), is designed to collect statistics on visitors to the shopping facility, update playlists based on the data received and calculate the formulas on the basis of which the further age groups of visitors are determined.

As a display device (1 1), a TV, a monitor, an LCD panel, a projector, a touch panel, a monoblock, augmented reality devices, or combinations of these can be used.

As audio playback devices (152), various types of stereo systems, sound bars, speakers (speakers), and any other type of equipment that can play audio files can be used.

Devices included in the system (100) are interconnected via a data network, which can be a wired or wireless type, or combinations thereof.

The computing device (140) can be connected to the server (120) via a wired network, for example, a LAN, or communicate directly using wireless communication protocols, such as: Wi-Fi, Bluetooth, GSM modem, etc.

The server (120) can run as cloud storage and provide reception / transmission of information via the Internet.

According to FIG. 2A - 2B means of video recording, made in the form of a camera (153) receives an image of visitors (10) of the shopping facility. The camera (153) receives an image of each visitor (10) who is in its visibility zone. A further description of the method of operation of the claimed solution will be presented with reference to FIG. 3, FIG. 4A-4B.

FIG. 3 shows the general scheme for implementing the claimed method. At step (201), the visitor image data (10) obtained by the camera (153) is transmitted to the computing device (140). In the next step (202), the resulting images are processed to identify images of visitors' faces (10). Based on the information obtained at the stage (301), the images of the faces are processed by the recognition algorithm to identify the key features of the person. In particular, based on the information obtained in step (301), In step (203), based on the image data of the visitors' faces (10) of the shopping facility, the computing device (140) determines the sex and age (302) for each received face image using a specialized software. After receiving the information in step (302), it calculates the number of visitors (303) belonging to one of the given gender and age groups. Next, at the stage (204), a list of media files (playlist) is formed based on the information received about the number of visitors (10) of the shopping facility and age and age information, which is then transmitted to devices in the sales area for playback (205).

Information on the number of visitors obtained at step (303) can be transmitted for further statistical processing to the server (120). On the server (120), this information is processed by the intellectual processing system (SW) for its possible use when updating the playlist of media files.

As shown in FIG. 4B, at the stage (304), a direct determination of the gender and age group, which prevails at a given point in time in the territory of the shopping facility, is performed in order to directly generate the list of media files for playback corresponding to this group of visitors.

At the stage (305), on the basis of information about the gender and age group prevailing in the trading floor, the formation of rules for playing media files takes place, on the basis of which a playlist is formed (306). These rules include a queue of playing audio and / or video files and commercials. The playlist can be dynamically updated, as a result of which the duration of playback of certain files, the display of commercials, etc. can change. An example of the distribution of data by gender and age groups is presented in Table 1.

Table 1. Example of the distribution of groups of visitors

Based on the received data, rules are created for playing media files, in particular, advertising materials or playing video and / or audio data. The most massive group of visitors of the shopping facility (10), currently in its territory, is determined.

Example: M (24-27) - 45%. F (36-42) - 20%. Ratio = 2.25 (rounded). For a computing device (140), this would mean that for every 2 promotional material for client group "M (24-27)", 1 material for client group "F (36-42)" should be reproduced. Based on the received data, a further playlist will be formed.

At step (307), the transfer of the corresponding information to the audio playback devices (152) and the display device (151) is performed. Audio data and video data can be synchronized and represent the overall composition of the commercial, or the video sequence for display devices (151) and audio sequence (152) can be different to create an overall atmosphere of a commercial object. Taking into account the definition of the prevailing age and gender group in the sales area, targeted advertising can be performed.

FIG. 5 shows an example of software distribution of the modules included in the client module (130) of the computing device (140). Client Module (130) contains a set of media files (131) for the formation of a playlist playback based on information received from the video analytics module (135).

The video analytics module (135) processes information received from one or more cameras (153), in particular, visitor images for detecting images of visitors on a video stream using the face detection algorithm and applying the gender and age prediction algorithms.

A set of media files (131) contains the following information: commercials (132), background video (133), background audio (134). Background audio and video materials can be independently played on relevant devices in the sales area (151, 152). FIG. 6 shows an example of the operation of the software and hardware logic of the video analytics module (135). The module (135) consists of two main elements: the visitor recognition module (136) and the vertical counter module (137).

Data containing images of visitors received from the camera (153) is transmitted to the module for detecting the object of interest, in particular, the faces of visitors (1361). Module (1361) determines the presence of a face in an image, which, as a rule, can be a frame extracted from a video stream. When a face image is detected, its location and boundaries are determined. For each found part of the image (frame) with the image of a person, its own identifier (ID) is set. After that, the data processed by the module (1361) are transmitted to the processing module of the detection results (1362). In the module (1362), data is prepared for the modules (1363, 1364, 1356) - areas with objects of interest are cut and normalized. After normalization, data is transferred to three modules (1363, 1364, 1356).

The data processed by the module (1362) are transmitted to the following modules:

• Sex determination module (1363);

 · Age determination module (1364);

 • Object tracking module (1365).

The sex determination module (1363) works on the basis of known solutions for predicting and determining gender in a graphic image, in particular, an image of a person’s face (see, for example, Antipov G. Et al. Minimalistic) . The age determination module (1364) determines its age based on the image of the visitor’s face; one or more existing algorithms can also be used, for example, described in patent application US20150332087.

The obtained data on age and gender from the above-mentioned modules (1363, 1364) together with data from the module (1362) are transmitted to the object tracking module (1365). This module (1365) is designed to link objects on different frames and build a history of their movement. The result of this module is a list of trajectories of unique objects and information about their field and age. The received data is transmitted to the counter module (1366). The counter module (1366) is designed to determine the passage of a person through a certain area in an image.

The vertical counter module (137) consists in determining, identifying and counting objects in a given area. This module (137) does not define the faces and gender and age characteristics of objects. The module (1371) is a background handler for identifying and counting visitors. The object recognition module (1372) performs object recognition in the background and identifies it. The module (1373) determines the contour of each object in the background. The object tracking module (1374) is similar in its functionality to the module (1365). The module (1375) is designed to determine the passage of a person through a certain line. When crossing the line, the module (1375) considers that the person entered or left the observed area of the trade room.

The media files on the playlist can also be ranked by the speed of beats per minute (BPM), each of which is matched to a specific age and gender group.

Additionally, the video analytics module (135) can recognize the part of the sales area that has a specific age and gender group of visitors (10) for displaying advertising information on display media (151).

Modules (136), (137) are part of a software and hardware complex that can run on a computing device (140), or on a remote device, connected via a computer network to a device (140), for example, the Internet, Intranet or WAN. FIG. 7 shows the general scheme of the computing device (140).

The device (140) can be executed on the basis of a wide range of electronic devices, for example, a personal computer, laptop, server, server cluster, mobile phone, tablet, mainframe, etc. The device (140) contains one or more processors (141) that perform the main computational work when implementing the steps of the above method (200).

Random access memory (RAM) (142), designed for the operational storage of instructions executed by one or more processors (141).

The storage medium (143) can be a hard disk (HDD), solid state drive (SSD), flash memory (AND-flash, EEPROM, Secure Digital, etc.), optical disk (CD, DVD, Blue Ray) , mini disk or their combination, or any other means, made with the ability to store data.

The input / output (I / O) interfaces (144) are standard ports and interface devices for devices and data transmission, selected on the basis of the required configuration of the device (140), in particular: USB (2.0, 3.0, USB-C, micro, mini), Ethernet, PCI, AGP, COM, LPT, PS / 2, SATA, FireWire, Lightning, etc.

I / O tools (145) are also selected from a known spectrum of various devices, for example, a keyboard, touchpad, touchscreen display, monitor, projector, mouse, joystick, trackball, light pen, stylus, audio output devices (speakers, headphones, built-in speakers , buzzer) etc.

Means of data transmission (146) are selected from devices designed to implement the process of communication between various devices via wired and / or wireless communication, in particular, such devices can be: GSM modem, Wi-Fi transceiver, Bluetooth or BLE module, GPS module, Glonass module, NFC, Ethernet module, etc.

All components of the device (140) are interconnected by means of a common data transfer bus (147).

The embodiments presented in the present materials of the claimed technical solution reveal the preferred aspects of its implementation and should not be used as limiting other, particular embodiments, which are obvious to a person skilled in the art, without departing from the scope of legal protection presented.

Claims

CLAIM 1. A computer-implemented way to control the playback of media files on a trading facility, containing the steps in which:  - get the data of visitors of the shopping facility, and the data are images of the visitors;  - determine the sex and age of each visitor based on the above images;  - determine the groups of visitors that are currently on the shopping facility;  - form a list of media files based on certain groups of visitors for playback on the audio / video playback of the shopping facility;  - play mentioned media files.  2. The method according to claim 1, characterized in that the images of the faces of visitors are obtained from the video stream obtained using a video camera.  3. The method according to claim 2, characterized in that the video stream is processed using an algorithm for detecting and recognizing faces.  4. The method according to claim 1, characterized in that the determination of the sex and age of visitors is performed using the analysis of facial images. 5. The method according to claim 1, characterized in that in determining the groups of visitors carry out the definition of gender and age groups by counting the number of visitors.  6. The method according to claim 1, characterized in that the list of media files is formed as a playlist based on the rules for playing media files. 7. The method according to claim 6, characterized in that the rules are formed on the basis of data on age and sex groups of visitors.  8. The method according to claim 7, characterized in that the playlist contains a schedule for playing audio and / or video files.  9. The method according to claim 1, characterized in that the update of the playlist is carried out dynamically at specified intervals or on the basis of information about the number of visitors of a particular gender and age group. 10. The playback control system of media files on the trade object, containing network-connected audio / video playback media media files, at least one means of video fixation and a computing device, and  - means of video fixation made with the possibility of obtaining a video stream of images of visitors;  - the computing device is configured to process the incoming video stream, determine images of visitors' faces, then process data to determine the gender and age of visitors, count the number of visitors that form the age and sex group, determine the playlist of media files for the corresponding age and age group at a given point in time;  audio / video playback means are adapted to play audio files and video files in response to receiving commands from a computing device based on the received playlist of media files. 1 1. The system of claim 10, characterized in that it further comprises a server containing an intelligent processing system that performs processing of data received from the computing device, on the basis of which the playlist of the media files is updated.  12. The system of claim 10, characterized in that the data network is wired and / or wireless.  13. The system of claim 10, characterized in that the means of audio / video playback are selected from the group: audio speakers, TV, monitor, soundbar, F-panel, projector, touch panel, monoblock, augmented reality device, or combinations thereof. 14. The system of claim 10, characterized in that it further comprises a lighting control subsystem of a commercial facility.  15. The system of claim 10, characterized in that the video recording means are an IP camera or an IP камеру-camera, or combinations thereof.  16. The system of claim 10, characterized in that the computing device contains a client module that initiates the transfer of data to the server and the activation of playback of media files on audio / video playback devices. 17. The system of claim 10, characterized in that the computing device includes a video analytics module, configured to process the incoming video stream. 18. The system of clause 15, characterized in that the client module contains a set of media files that are played in accordance with the playlist received from the server.  19. The system of claim 10, characterized in that the intelligent processing system generates rules for playing media files on the basis of which a playlist is created.