CN111814531A - Method, apparatus, electronic device and storage medium for determining driver behavior - Google Patents

Abstract

The present disclosure relates to methods, apparatus, electronic devices, and storage media for determining driver behavior. In one method, a sequence of images recording the state of the vehicle is acquired, the sequence of images being acquired by an image acquisition device deployed inside the vehicle. Changes in the state of the driver of the vehicle and changes in state of predetermined parts of the vehicle are detected based on the sequence of images. Based on the state change of the driver and the state change of the predetermined portion, the behavior of the driver to provide service to the occupants of the vehicle is determined. Further, corresponding apparatuses, electronic devices and storage media are provided. Using the above implementation manner, the acquired image sequence can be automatically analyzed without manual intervention, and then it can be determined whether the driver has performed the desired service behavior.

Description

Method, apparatus, electronic device and storage medium for determining driver behavior

technical field

Various implementations of the present disclosure relate to image processing, and more particularly, to a method, apparatus, electronic device, and storage medium for determining driver behavior based on a sequence of processed images.

Background technique

With the development of computer technology and network technology, applications for providing online vehicle services have been developed. Passengers can utilize these applications to submit vehicle service requests in order to find a suitable vehicle and travel to a desired destination. In order to provide more comfortable service to passengers, the driver may perform certain types of service behaviors. For example, the driver can help with loading and unloading luggage, helping passengers open and close doors, etc.

Different drivers may behave differently, and it is now possible to collect image sequences to record driver service behaviors. However, the captured image sequences involve a large amount of data, and a lot of manual labor is required to find video sequence clips involving driver service behaviors for human review. At this time, how to process the image sequence in a more effective way, so as to determine whether the driver's service behavior is appropriate and then measure the driver's service level, has become a research hotspot.

SUMMARY OF THE INVENTION

It is desirable to develop and implement a technical solution for determining the service behavior of drivers in a more efficient manner. It is expected that the technical solution can be compatible with existing applications, so that service quality monitoring can be performed in a more effective manner.

According to a first aspect of the present disclosure, there is provided a method for determining driver behavior. In the method, a sequence of images recording the state of the vehicle is acquired, the sequence of images being captured by an image capturing device deployed inside the vehicle. Changes in the state of the driver of the vehicle and changes in state of predetermined parts of the vehicle are detected based on the sequence of images. Based on the state change of the driver and the state change of the predetermined portion, the behavior of the driver to provide service to the occupants of the vehicle is determined.

According to a second aspect of the present disclosure, there is provided an apparatus for determining driver behavior. The device includes: an acquisition module configured to acquire an image sequence recording the state of the vehicle, the image sequence being acquired by an image acquisition device deployed inside the vehicle; a detection module configured to detect the state of the driver of the vehicle based on the image sequence a change and a change in state of a predetermined portion of the vehicle; and a determination module configured to determine an act of the driver providing a service to an occupant of the vehicle based on the change in state of the driver and the change in state of the predetermined portion.

According to a third aspect of the present disclosure, there is provided an electronic device, comprising: a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement according to the present disclosure method of the first aspect.

According to a fourth aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon one or more computer instructions, wherein the one or more computer instructions are executed by a processor to implement the method according to the first aspect of the present disclosure .

Description of drawings

Features, advantages, and other aspects of various implementations of the present disclosure will become more apparent in conjunction with the accompanying drawings and with reference to the following detailed description, several implementations of which are shown here by way of example and not limitation. In the attached image:

FIG. 1 schematically illustrates a block diagram of a vehicle environment in which exemplary implementations according to the present disclosure may be used;

2 schematically illustrates a block diagram of a process for determining driver behavior in accordance with an exemplary implementation of the present disclosure;

FIG. 3 schematically shows a flowchart of a method for determining driver behavior according to an exemplary implementation of the present disclosure;

Figure 4 schematically illustrates a block diagram for acquiring a sequence of images according to an exemplary implementation of the present disclosure;

FIG. 5 schematically illustrates a block diagram for performing sampling on a sequence of raw images according to an exemplary implementation of the present disclosure;

6A and 6B respectively schematically illustrate block diagrams of driver regions in different images in a sequence of images according to an exemplary implementation of the present disclosure;

Figures 7A, 7B and 7C respectively schematically illustrate block diagrams of rear window regions in different images in a sequence of images according to an exemplary implementation of the present disclosure;

FIG. 8 schematically illustrates a block diagram of a vehicle door area in an image in a sequence of images in accordance with an exemplary implementation of the present disclosure; and

9 schematically illustrates a block diagram of an apparatus for determining driver behavior according to an exemplary implementation of the present disclosure.

Detailed ways

Preferred implementations of the present disclosure will be described in more detail below with reference to the accompanying drawings. While preferred implementations of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited by the implementations set forth herein. Rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

As used herein, the term "including" and variations thereof mean open-ended inclusion, ie, "including but not limited to". The term "or" means "and/or" unless specifically stated otherwise. The term "based on" means "based at least in part on". The terms "one example implementation" and "one implementation" mean "at least one example implementation." The term "another implementation" means "at least one additional implementation." The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

The application environment of an exemplary implementation of the present disclosure will be described first with reference to FIG. 1 . Applications such as online vehicle services have been developed. Vehicle service companies can specify service standards that drivers should adhere to. For example, it can be specified that the driver should open and close the doors for the passenger, help the passenger load and unload luggage into the trunk when the passenger is carrying luggage, and so on. Vehicles are usually distributed in various locations on urban roads, and the driver's service behavior usually occurs outside the vehicle, so it is difficult to monitor the driver's service level based on a centralized way.

For reasons of safety or other factors, a technical solution has been proposed to deploy an image acquisition device inside the vehicle to record the state of the vehicle. Although the acquired image sequence can record the driver's behavior, since the acquired image sequence usually spans a large time period (for example, 8 hours a day), how to analyze the service behavior provided by the driver to each passenger one by one becomes a technical problem at this time. problem. In particular, when a customer service agent for a vehicle service application receives a complaint from a passenger (for example, complaining that the driver is not helping with loading and unloading luggage and/or opening and closing doors), the customer service agent has to manually look through a large sequence of images in order to confirm the driver's service behavior .

To at least partially address one or more of the above-mentioned problems and other potential problems, exemplary implementations of the present disclosure propose a technical solution for determining driver behavior. FIG. 1 schematically illustrates a block diagram 100 of a vehicle environment in which exemplary implementations according to the present disclosure may be used. Image acquisition equipment can be deployed inside the vehicle in order to acquire a sequence of images of the state of the vehicle as shown in FIG. 1 . Image 110 schematically shows one image in a sequence of images. According to the exemplary implementation of the present disclosure, the acquired image sequence can be analyzed to determine whether the driver has performed the prescribed service behavior.

An overview of an exemplary implementation of the present disclosure is described first with reference to FIG. 2 . FIG. 2 schematically illustrates a block diagram 200 of a process for determining driver behavior in accordance with an exemplary implementation of the present disclosure. As shown in FIG. 2 , a sequence of images 210 may be acquired, and based on the sequence of images 210 , a state change 220 associated with the vehicle may be determined. The state change 220 may include two aspects: a state change 222 of the driver of the vehicle and a state change 224 of a predetermined portion of the vehicle (eg, the trunk and/or doors). Then, the driver's service behavior 230 may be determined based on the collected state changes 220 described above. With exemplary implementations of the present disclosure, the acquired image sequence 210 can be automatically analyzed without manual intervention to determine whether the driver has performed the desired service behavior.

Hereinafter, more details of an exemplary implementation of the present disclosure will be described with reference to FIG. 3 . FIG. 3 schematically illustrates a flowchart of a method 300 for determining driver behavior according to an exemplary implementation of the present disclosure. At block 310, a sequence of images 210 recording the state of the vehicle is acquired, the sequence of images 210 being captured by an image capture device deployed inside the vehicle. It will be appreciated that video capture devices may be deployed inside the vehicle, for example, capture devices such as cameras may be deployed in vehicle rear view mirrors or other locations. The acquired image sequence 210 may be transmitted to the data center in real-time, and/or may be transmitted periodically.

The driver's service of opening and closing doors and loading and unloading luggage for passengers always occurs during the time period before and after passengers get on and off. The location of the image sequence in the original image sequence related to the above-mentioned service can be found based on the time period during which the driver provided the vehicle service. Specifically, the start time when the driver accepts the vehicle service request from the passenger and the end time when the driver completes the vehicle service request can be obtained. Based on the above two time points, the position of the image sequence to be analyzed can be found more accurately, thereby avoiding the situation of analyzing the entire original image sequence. In this way, processing efficiency can be improved and time overhead can be reduced.

Figure 4 schematically illustrates a block diagram 400 for acquiring a sequence of images according to an exemplary implementation of the present disclosure. As shown in FIG. 4, the raw image sequence 410 represents raw data collected from the image capture device, and the time axis represents the time span during which the driver provides vehicle service. The start time 420 at which the driver accepts the vehicle service request may be obtained from the vehicle service application, and a sequence of images 422 within a predetermined time range after the start time 420 may be selected.

Typically, after accepting a vehicle service request, it will take a while for the driver to travel to the passenger's designated departure point. At this time, the predetermined time range may be designated as, for example, 2 minutes to 5 minutes after the start time 420 . According to an exemplary implementation of the present disclosure, the predetermined time range may be specified based on the expected travel time between the location where the driver accepts the request and the point of departure. The longer the expected time, the greater the difference between the predetermined time range and the start time 420. With exemplary implementations of the present disclosure, only the image sequence 422 needs to be analyzed in order to determine whether a driver assists a passenger with loading luggage and/or opening and closing doors when commencing service, thereby avoiding the various overheads of analyzing the entire raw image sequence 410 .

According to an exemplary implementation of the present disclosure, as shown in FIG. 4 , an end time 430 at which the driver completes the vehicle service request may be obtained, and an image sequence 432 within a predetermined time range after the end time 430 may be selected. For example, the predetermined time range may be specified as a few minutes from the end time 430 . In this way, only the sequence of images 432 needs to be analyzed to determine whether the driver assists the passenger with unloading luggage and/or opening and closing doors at the end of service, thereby avoiding the various overheads of analyzing the entire sequence of raw images 410 .

With exemplary implementations of the present disclosure, it is possible to accurately locate a sequence of images that may include service behaviors in the original sequence of images. In this way, the data volume of the image sequence to be processed can be reduced and the processing efficiency improved.

According to exemplary implementations of the present disclosure, the raw image sequence 410 acquired from the image acquisition device may span a long period of time. Processing for each image in the raw image sequence 410 would take a significant amount of time and processing resources. In order to reduce overhead in terms of processing time and processing resources, sampling may be performed from the original image sequence 410 at predetermined time intervals to form the image sequence to be processed.

FIG. 5 schematically illustrates a block diagram 500 for performing sampling on an original image sequence 410 according to an exemplary implementation of the present disclosure. As shown in FIG. 5 , a predetermined time interval can be specified, for example, it is specified to sample one frame of images from consecutive N (N is an integer, for example, N=4) frame images. The 1 st frame, the N+1 th frame, the 2N+1 th frame, etc. may be selected from the original image sequence 510, and the image sequence 510 may be generated based on the selected respective image frames. With the exemplary implementation of the present disclosure, the amount of data of the image sequence to be processed can be reduced by sampling techniques. In this way, processing time and processing resource overhead can be reduced without affecting the detection effect, thereby improving processing efficiency.

According to an exemplary implementation of the present disclosure, the methods shown in FIGS. 4 and 5 may be combined. For specific addresses, the image sequence 422 and the image sequence 432 associated with the service start and service end may be determined first. Further, sampling may be performed in the image sequences 422 and 432 in order to further reduce the data volume of the image sequences to be analyzed.

How the sequence of images is acquired has been described above with reference to FIGS. 4 and 5 , in the following, how the state change 220 is determined will be described. Returning to FIG. 3 , at block 320 , a change in state 222 of the driver of the vehicle and a change in state 224 of a predetermined portion of the vehicle are detected based on the sequence of images 210 . According to an exemplary implementation of the present disclosure, each image in the sequence of images 210 may be divided into different regions in order to detect state changes in the various regions. Referring first to Figures 6A and 6B, it will be described how to detect the state change 222 of the driver.

FIG. 6A schematically illustrates a block diagram 600A of the driver area 612 in one image 610A of the sequence of images 210 in accordance with an exemplary implementation of the present disclosure. As shown in FIG. 6A, in the sequence of images 210, a driver area 612 associated with the driver's seat of the vehicle may be identified. The driver's state change 222 may be detected by monitoring changes in the image in the driver area 612 . The driver can wait for the passenger at the departure point, and when the passenger arrives, the driver can get off the bus and help the passenger load and unload luggage and/or open and close the doors. At this time, whether the driver's image is included in the driver area 612 may be detected to determine whether the driver gets off the vehicle. According to an exemplary implementation of the present disclosure, whether the driver gets off the vehicle can be detected based on a face recognition method. A first subsequence of images including driver images may be detected from the sequence of images 210 . In this case, the driver is located in the driver area in the images of the first image subsequence.

According to an exemplary implementation of the present disclosure, the driver's face may be recognized based on machine learning techniques. For example, image subsequences that include the driver's face and image subsequences that do not include the driver's face can be detected in the image sequence based on various face recognition models that have been developed or will be developed in the future.

Analysis of the image sequence following the first image subsequence can continue, if a second image subsequence that does not include the driver image is detected after the first image subsequence (at this point, the driver is not in the driver area in the images of the second image subsequence ), it can be determined that the status of the driver changes to "get off". 6B schematically illustrates a block diagram 600B of a driver area in another image in a sequence of images in accordance with an exemplary implementation of the present disclosure. If it is determined that the subsequent image sequence does not include the driver's face image, it can be determined that the driver has gotten off the vehicle.

According to an exemplary implementation of the present disclosure, in order to avoid misjudgment caused by actions such as the driver turning his head or bending over, a time length threshold may be specified. For example, if it is determined that the images in the second sub-sequence of images span a time length above a time length threshold, it may be determined that the driver's state change is "getting off." For specific addresses, the time length threshold can be specified as 0.5 seconds or other values. Alternatively and/or additionally, the time length threshold may also be specified in units of image frames.

With the exemplary implementation of the present disclosure, it is not necessary to collect information outside the vehicle, and it is possible to detect whether the driver gets off the vehicle based only on the state inside the vehicle. Further, using image sub-sequences instead of a single image frame to determine whether a driver gets off the bus can reduce the possibility of misjudgment, thereby providing an accurate basis for determining the driver's service behavior.

According to an exemplary implementation of the present disclosure, a predetermined area may be designated for a predetermined portion of the vehicle. Here, the predetermined portion may include a trunk and a door of the vehicle. The state change of the trunk can be used to detect whether the driver helps the passenger load and unload luggage, and the state change of the door can be used to detect whether the driver helps the passenger to open and close the door. Specifically, in the sequence of images 210, a predetermined area associated with a predetermined portion of the vehicle may be identified. Hereinafter, how to determine the state change of the trunk is first described with reference to FIGS. 7A to 7C .

7A schematically illustrates a block diagram 700A of a rear window area in image 710A in an image sequence in accordance with an exemplary implementation of the present disclosure. It will be appreciated that, based on the sequence of images 210, it is only possible to see from the rear window whether the trunk is opened. In order to determine the state change of the trunk, a rear window area 712 may be specified in the image sequence 210 . In the sequence of images after the driver has exited the vehicle, a change in the state of the trunk may be detected based on the rear window area 712 .

Specifically, a third image subsequence in which the trunk is in a closed state can be detected from the image sequence (in the rear window area 712 in the images of the third image subsequence, the trunk is in a closed state). As shown in FIG. 7A , the rear window area 712 shows that the trunk is in a closed state at this time.

According to an exemplary implementation of the present disclosure, the state of the trunk may be determined based on machine learning techniques. For example, training samples including images of trunks of various models in closed, in the process of opening, and open states may be collected in advance, and a judgment model describing the state of the trunk may be generated based on the training samples. An image from a sequence of images can be fed into the model in order to determine the state of the trunk in that image.

Then, in the image sequence after the third image subsequence, it is possible to continue to search for whether there is an image in which the trunk is opened. If a fourth image subsequence with the trunk open (in other words, the trunk in the rear window area 712 is open in the images of the fourth image subsequence) is detected after the third image subsequence, it may be determined that the trunk The status changes to "Open".

According to an exemplary implementation of the present disclosure, in order to further improve the detection accuracy, a state in which the trunk is in the process of being opened may also be detected in the image sequence. Specifically, the image subsequence including the trunk being opened may be detected between the third image subsequence and the fourth image subsequence. 7B schematically illustrates a block diagram 700B of a rear window area 712 in image 710B in an image sequence in accordance with an exemplary implementation of the present disclosure. As shown in Figure 7B, the rear window area 712 shows that the trunk is in the open vehicle and has been partially opened.

According to an exemplary implementation of the present disclosure, if a fifth image subsequence (the trunk in the rear window area 712 in the image of the fifth image subsequence is detected between the third image subsequence and the fourth image subsequence is in the process of opening), it can be determined that the status of the trunk changes to "open".

7C schematically illustrates a block diagram 700C of a rear window area 712 in image 710C in an image sequence in accordance with an exemplary implementation of the present disclosure. As shown in Figure 7C, the rear window area 712 shows that the trunk has been fully opened. At this time, based on the third image subsequence indicating that the trunk is closed, the fifth image subsequence indicating that the trunk is in the process of opening, and the fourth image subsequence indicating that the trunk is open, the " Open" state changes.

With the exemplary implementation of the present disclosure, it is only necessary to determine the state of the trunk in the rear window area 712 in the image sequence, and compare multiple states with a time sequence relationship to accurately determine the state change of the trunk.

According to an exemplary implementation of the present disclosure, in order to improve the detection accuracy, it may be further determined whether the driver appears in the rear window area 712 during the trunk opening process. For example, it may be further determined whether the driver assists the passenger in loading and unloading luggage by comparing the clothing of the driver collected in the vehicle with the clothing of the person appearing in the rear window area 712 . For another example, the driver's service behavior can be determined by performing face recognition of the driver in the rear window area 712 . It will be understood that because the scene outside the rear window will constantly change, such as the shadow of clouds, the occlusion of the following car and other special situations, detecting the driver's actions through the timing of state changes can provide higher accuracy.

How to determine the state change of the trunk has been described above with reference to FIGS. 7A to 7C . A change in state of a vehicle door may be determined in a similar manner to determine whether the driver opens or closes the door for a passenger. FIG. 8 schematically shows a block diagram 800 of a vehicle door area in an image 800 in an image sequence according to an exemplary implementation of the present disclosure. As shown, a door area 812 may be designated for the door.

According to example implementations of the present disclosure, the state of the doors in the door area 812 may be determined based on machine learning techniques. For example, training samples including images of doors of various vehicle types being closed, in the process of opening, and open may be collected in advance, and a judgment model describing the state of the doors may be generated based on the training samples. An image in the sequence of images can be fed into the model to determine the state of the door in the image. With the exemplary implementation of the present disclosure, it is only necessary to determine the state of the vehicle door in the vehicle door area 812 in the image sequence, and to compare multiple states having a time sequence relationship, the state change of the vehicle door can be accurately determined.

How to determine the state change 222 of the driver and the state change 224 of the predetermined portion has been described above with reference to FIGS. 6A , 6B, 7A, 7B, 7C and 8 . The following will return to FIG. 3 to describe how the driver's behavior is determined. At block 330 of FIG. 3, based on the state change 222 of the driver and the state change 224 of the predetermined portion, the behavior of the driver to provide service to the occupants of the vehicle is determined. It will be understood that the service standard to which the driver should adhere can be specified. For example, it can be specified that the driver should open and close the doors for the passenger, and should help the passenger load and unload luggage into the trunk when the passenger is carrying luggage. Therefore, the behavior at this time may include at least any one of the behavior of loading and unloading luggage for the passenger and the behavior of opening and closing the vehicle door for the passenger. With exemplary implementations of the present disclosure, driver service behavior can be automatically determined and reviewed for compliance with various types of service specifications by analyzing image sequences. In this way, the labor and time overhead of manual review can be reduced.

According to an exemplary implementation of the present disclosure, if state changes related to 'getting off' and 'opening' are detected in sequence, it may be determined that the driver performed a prescribed action. In one example, if it is detected in the image sequence 210 that the driver gets out of the vehicle and the door is opened, it may be determined that the driver opens and closes the door for the passenger. In another example, if the driver is detected in the sequence of images getting out of the vehicle with the trunk open, it may be determined that the driver is loading and unloading luggage for the passenger. According to an exemplary implementation of the present disclosure, if only a state change related to "on" is detected, but "getting out" of the driver is not detected, it may be determined that the driver has not performed the prescribed service behavior.

With the exemplary implementation of the present disclosure, it is not necessary to manually check the sequence of images related to each vehicle service one by one, but the service level of the driver can be evaluated based on automatic image processing. Further, when there is a dispute such as a passenger complaint, the method 300 described above can be invoked to preliminarily determine whether the driver has performed the prescribed service behavior. In this way, the workload of the customer service personnel can be reduced and the work efficiency can be improved.

An example of a method according to the present disclosure has been described in detail above with reference to FIGS. 2 to 8 , and the implementation of the corresponding apparatus will be described below. According to an exemplary implementation of the present disclosure, there is provided an apparatus for determining a driver's behavior, comprising: an acquisition module configured to acquire an image sequence recording the state of the vehicle, the image sequence being obtained by an image acquisition device deployed inside the vehicle collected; a detection module configured to detect a state change of the driver of the vehicle and a state change of a predetermined portion of the vehicle based on the sequence of images; and a determination module configured to determine the driver based on the state change of the driver and the state change of the predetermined portion The act of providing service to the occupants of a vehicle.

According to an exemplary implementation of the present disclosure, the detection module includes: a driver area identification module configured to identify, in a sequence of images, a driver area associated with a driver seat of a vehicle; and a driver change detection module configured to identify a driver area based on the driver area Detect the status change of the driver.

According to an exemplary implementation of the present disclosure, the change detection module includes: a first subsequence detection module configured to detect a first subsequence of images from a sequence of images in which a driver is located in a driver area; The second subsequence detection module is configured to, in response to detecting the second image subsequence after the first image subsequence, in which the driver is not in the driver area in the images of the second image subsequence, determine that the state of the driver is changed to "down" car".

According to an exemplary implementation of the present disclosure, the change detection module further includes a comparison module configured to, in response to determining that the time length spanned by the images in the second subsequence of images satisfies the time length threshold, determine that the driver's state change is "down" car".

According to an exemplary implementation of the present disclosure, the predetermined portion includes a trunk and a door of the vehicle, wherein the detection module includes: a predetermined area identification module configured to identify, in the sequence of images, a predetermined area associated with the predetermined portion of the vehicle; and A vehicle change detection module configured to detect a state change of a predetermined portion based on a predetermined area.

According to an exemplary implementation of the present disclosure, the vehicle change detection module includes: a third subsequence detection module configured to detect a third subsequence of images from the sequence of images, a predetermined subsequence in a predetermined region in an image of the third subsequence of images the portion is in an off state; the fourth subsequence detection module is configured to, in response to detecting the fourth image subsequence after the third image subsequence, a predetermined portion in a predetermined area in the image of the fourth image subsequence being in an open state , it is determined that the state of the predetermined part changes to "open".

According to an exemplary implementation of the present disclosure, the vehicle change detection module further includes a fifth subsequence detection module configured in response to detecting the fifth image subsequence between the third image subsequence and the fourth image subsequence, In the image of the fifth sub-sequence of images, the predetermined portion in the predetermined region is in the state of being opened, and it is determined that the state of the predetermined portion is changed to "open".

According to an exemplary implementation of the present disclosure, the behavior of the driver includes at least any one of the behavior of loading and unloading luggage for the passenger and the behavior of opening and closing the vehicle door for the passenger, and the determining module includes at least any one of the following: a first behavior determining module , configured to determine that the driver performed an action in response to detecting "getting off" and "opening" in sequence; and a second behavior determination module configured to respond to detecting "opening" but not detecting "getting off" , to determine that the driver did not perform the action.

According to an exemplary implementation of the present disclosure, the acquisition module includes at least any one of the following: a first image sequence acquisition module configured to acquire an image sequence within a predetermined time range after the start time when the driver accepts the vehicle service request from the passenger and a second image sequence acquisition module configured to acquire an image sequence within a predetermined time range after the end time point when the driver completes the vehicle service request.

According to an exemplary implementation of the present disclosure, the acquisition module includes: an original image sequence acquisition module configured to acquire an original image sequence from an image acquisition device; and a sampling module configured to execute from the original image sequence at predetermined time intervals Sampling to form a sequence of images.

According to an exemplary implementation of the present disclosure, there is provided an electronic device comprising: a memory and a processor; wherein the memory is used to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to achieve the above method described.

9 shows a block diagram of a computing device/server 900 in which one or more embodiments of the present disclosure may be implemented. It should be understood that the computing device/server 900 shown in FIG. 9 is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein.

As shown in FIG. 9, computing device/server 900 is in the form of a general purpose computing device. Components of computing device/server 900 may include, but are not limited to, one or more processors or processing units 910, memory 920, storage devices 930, one or more communication units 940, one or more input devices 950, and one or more Output device 960. The processing unit 910 may be an actual or virtual processor and can perform various processes according to programs stored in the memory 920 . In a multi-processor system, multiple processing units execute computer-executable instructions in parallel to increase the parallel processing capabilities of the computing device/server 900 .

Computing device/server 900 typically includes multiple computer storage media. Such media can be any available media accessible by computing device/server 900, including but not limited to volatile and nonvolatile media, removable and non-removable media. Memory 920 may be volatile memory (eg, registers, cache, random access memory (RAM)), non-volatile memory (eg, read only memory (ROM), electrically erasable programmable read only memory (EEPROM) , Flash) or some combination of them. Storage device 930 may be removable or non-removable media, and may include machine-readable media, such as flash drives, magnetic disks, or any other media that may be capable of storing information and/or data (eg, training data for training). ) and can be accessed within computing device/server 900.

Computing device/server 900 may further include additional removable/non-removable, volatile/non-volatile storage media. Although not shown in Figure 9, disk drives for reading or writing from removable, non-volatile magnetic disks (eg, "floppy disks") and for reading or writing from removable, non-volatile optical disks may be provided CD-ROM drive for reading or writing. In these cases, each drive may be connected to a bus (not shown) by one or more data media interfaces. Memory 920 may include a computer program product 925 having one or more program modules configured to perform various methods or actions of various embodiments of the present disclosure.

The communication unit 940 enables communication with other computing devices through a communication medium. Additionally, the functions of the components of computing device/server 900 may be implemented in a single computing cluster or multiple computing machines capable of communicating over a communication connection. Thus, computing device/server 900 may operate in a networked environment using logical connections to one or more other servers, network personal computers (PCs), or another network node.

Input device 950 may be one or more input devices, such as a mouse, keyboard, trackball, and the like. Output device 960 may be one or more output devices, such as a display, speakers, printer, and the like. The computing device/server 900 may also communicate with one or more external devices (not shown), such as storage devices, display devices, etc., through the communication unit 940, as needed, with one or more external devices that enable the user to communicate with the computing device/server. 900 interacts with any device (eg, network card, modem, etc.) that enables computing device/server 900 to communicate with one or more other computing devices. Such communication may be performed via an input/output (I/O) interface (not shown).

According to an exemplary implementation of the present disclosure, there is provided a computer-readable storage medium having stored thereon one or more computer instructions, wherein the one or more computer instructions are executed by a processor to implement the method described above.

Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products implemented in accordance with the present disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.

These computer readable program instructions may be provided to the processing unit of a general purpose computer, special purpose computer or other programmable data processing apparatus to produce a machine that causes the instructions when executed by the processing unit of the computer or other programmable data processing apparatus , resulting in means for implementing the functions/acts specified in one or more blocks of the flowchart and/or block diagrams. These computer readable program instructions can also be stored in a computer readable storage medium, these instructions cause a computer, programmable data processing apparatus and/or other equipment to operate in a specific manner, so that the computer readable medium on which the instructions are stored includes An article of manufacture comprising instructions for implementing various aspects of the functions/acts specified in one or more blocks of the flowchart and/or block diagrams.

Computer-readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other equipment to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other equipment to produce a computer-implemented process , thereby causing instructions executing on a computer, other programmable data processing apparatus, or other device to implement the functions/acts specified in one or more blocks of the flowcharts and/or block diagrams.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various implementations of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executables for implementing the specified logical function(s) instruction. In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented in dedicated hardware-based systems that perform the specified functions or actions , or can be implemented in a combination of dedicated hardware and computer instructions.

While various implementations of the present disclosure have been described above, the foregoing description is exemplary, not exhaustive, and not limiting of the disclosed implementations. Numerous modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The terminology used herein was chosen to best explain the principles of the implementations, the practical application or improvement over the technology in the marketplace, or to enable others of ordinary skill in the art to understand the implementations disclosed herein.

Claims (13)

1. A method for determining driver behavior, comprising:

acquiring a sequence of images recording a state of a vehicle, the sequence of images being acquired by an image acquisition device deployed inside the vehicle;

detecting a change in state of a driver of the vehicle and a change in state of a predetermined portion of the vehicle based on the sequence of images; and

determining an action of the driver to provide a service to a passenger of the vehicle based on the state change of the driver and the state change of the predetermined portion.

2. The method of claim 1, wherein detecting the change in state of the driver comprises:

identifying, in the sequence of images, a driver zone associated with a driver seat of the vehicle; and

detecting the state change of the driver based on the driver region.

3. The method of claim 2, wherein detecting the change in state of the driver based on the driver zone comprises:

detecting a first image sub-sequence from the image sequence, in the images of which the driver is located in the driver region;

in response to detecting a second image sub-sequence after the first image sub-sequence, the driver is not in the driver zone in images of the second image sub-sequence, determining that the state change of the driver is "alighting".

4. The method of claim 3, wherein detecting the change in state of the driver based on the driver zone further comprises:

in response to determining that a length of time spanned by images in the second image subsequence satisfies a length of time threshold, determining that the state change of the driver is "alighting".

5. The method of claim 3, wherein the predetermined portion comprises a trunk and a door of the vehicle, wherein detecting the change in state of the predetermined portion based on the sequence of images comprises:

identifying, in the sequence of images, a predetermined region associated with the predetermined portion of the vehicle; and

detecting the change in state of the predetermined portion based on the predetermined area.

6. The method of claim 5, wherein detecting the change in state of the predetermined portion based on the predetermined area comprises:

detecting a third image sub-sequence from the sequence of images, the predetermined portion in the predetermined area being in an off state in the images of the third image sub-sequence;

in response to detecting a fourth image sub-sequence subsequent to the third image sub-sequence, the predetermined portion in the predetermined area being in an on state in images of the fourth image sub-sequence, determining that the state change of the predetermined portion is "on".

7. The method of claim 6, wherein detecting the change in state of the predetermined portion based on the predetermined area further comprises:

in response to detecting a fifth image sub-sequence between the third image sub-sequence and the fourth image sub-sequence, the predetermined portion in the predetermined area being in an in-process state in images of the fifth image sub-sequence, determining that the state change of the predetermined portion is "on".

8. The method of claim 6, wherein the behavior of the driver includes at least any one of loading and unloading luggage for the passenger and opening and closing a door for the passenger, and determining the behavior of the driver includes at least any one of:

determining that the driver performed the action in response to detecting "alighting" and "opening" in order; and

in response to detecting "open" but not detecting "alighting," determining that the driver is not performing the action.

9. The method of claim 1, wherein acquiring the sequence of images comprises at least any one of:

obtaining the sequence of images within a predetermined time range after a start time of acceptance of a vehicle service request by the driver from the passenger; and

acquiring the sequence of images within a predetermined time range after an end time point at which the driver completes the vehicle service request.

10. The method of claim 1, wherein acquiring the sequence of images comprises:

acquiring an original image sequence from the image acquisition device; and

at predetermined time intervals, sampling is performed from the original image sequence to form the image sequence.

11. An apparatus for determining driver behavior, comprising:

an acquisition module configured to acquire a sequence of images recording a state of a vehicle, the sequence of images being acquired by an image acquisition device deployed inside the vehicle;

a detection module configured to detect a change of state of a driver of the vehicle and a change of state of a predetermined portion of the vehicle based on the sequence of images; and

a determination module configured to determine an action of the driver to provide a service to a passenger of the vehicle based on the state change of the driver and the state change of the predetermined portion.

12. An electronic device, comprising:

a memory and a processor;

wherein the memory is to store one or more computer instructions, wherein the one or more computer instructions are to be executed by the processor to implement the method of any one of claims 1 to 10.

13. A computer readable storage medium having one or more computer instructions stored thereon, wherein the one or more computer instructions are executed by a processor to implement the method of any one of claims 1 to 10.

Images