[go: up one dir, main page]

WO2009138037A1 - Système de service vidéo, appareil de service vidéo et procédé correspondant d’extraction d’images clés de service vidéo - Google Patents

Système de service vidéo, appareil de service vidéo et procédé correspondant d’extraction d’images clés de service vidéo Download PDF

Info

Publication number
WO2009138037A1
WO2009138037A1 PCT/CN2009/071783 CN2009071783W WO2009138037A1 WO 2009138037 A1 WO2009138037 A1 WO 2009138037A1 CN 2009071783 W CN2009071783 W CN 2009071783W WO 2009138037 A1 WO2009138037 A1 WO 2009138037A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
vector
motion vector
feature vector
motion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2009/071783
Other languages
English (en)
Chinese (zh)
Inventor
邸佩云
胡昌启
元辉
马彦卓
常义林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Publication of WO2009138037A1 publication Critical patent/WO2009138037A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/14—Picture signal circuitry for video frequency region
    • H04N5/147—Scene change detection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/107—Selection of coding mode or of prediction mode between spatial and temporal predictive coding, e.g. picture refresh
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/136—Incoming video signal characteristics or properties
    • H04N19/137—Motion inside a coding unit, e.g. average field, frame or block difference
    • H04N19/139—Analysis of motion vectors, e.g. their magnitude, direction, variance or reliability
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field

Definitions

  • the embodiments of the present invention relate to the field of communications technologies, and in particular, to a video service system, a video service device, and a method for extracting key frames thereof.
  • the extraordinary state means that the motion state of the object changes significantly, for example, from rest to motion, from motion to rest, the direction of motion changes, or the speed of motion changes significantly.
  • the change of the video scene is a reflection of the significant change of the motion state of the object in the scene.
  • the scene change also includes the switching of the scene. It can be considered that the switching of the scene is the sudden movement of the object in the original scene to the infinity, and the object in the new scene is moved from infinity, and the motion state of the object is intense. Variety.
  • frames are used to describe video information, and key frames are frames that best represent video information.
  • the so-called key frame refers to the frame in which the object in the scene has abnormal motion.
  • the other frame scenes between the abnormal frames remain normal.
  • the scene refers to a set of several shots containing content related.
  • the interframe difference method is applied to the kth frame and the k-1th frame to obtain a rough outline of the moving object (referred to as a first contour), and then Using the multi-level edge detection algorithm to obtain the contour of all objects in the kth frame (referred to as the second contour), and the second contour is ANDed with the first contour to obtain a clearer contour than the first contour (referred to as the third Contour), then add a rectangular frame to the moving object based on the third contour, that is, the moving object is framed by a rectangle, and the motion is obtained by the Geodesic Active Contour Model in the Level Set Method.
  • the edge contour of the object finally by judging the appearance of the edge contour of the moving object , disappear, displacement changes, and shape changes to select keyframes.
  • the contour information of all moving objects needs to be extracted, and the contour information is calculated. Because the algorithm process of extracting the contour information is complicated, the calculation amount of the prior art 1 is large;
  • the first technique is to extract the key frames from stationary to moving or from moving to stationary, but the key frames that are suddenly shifted by constant speed cannot be extracted.
  • the prior art 2 does not consider the directional problem, that is, the prior art 2 cannot reflect the uniform speed but the moving direction occurs.
  • the motion vector information of the individual moving objects is large, and the motion vectors of the individual moving objects are very Xiaoyan, after the average value calculation, may cause the perceived motion energy value of the kth frame to be small, which does not reflect the change of the kth frame well, which may lead to the misjudgment of the key frame, that is, the kth frame cannot be selected as Keyframe.
  • An embodiment of the present invention provides a method for extracting a key frame, so as to solve the problem that the prior art solution cannot accurately extract a key frame that changes in a uniform speed but changes in the direction of motion.
  • a key frame extraction method including:
  • a video service device comprising:
  • a video key frame extraction module configured to obtain, according to a motion vector of each frame in the acquired video data stream; a feature vector set of motion vectors of each frame; and a motion vector corresponding to the feature vector set of the adjacent two frames before and after Whether the direction and the amplitude change; extract the key frame from the video data stream by using the judgment result of whether the change occurs.
  • a video service system comprising: the video service device and the user terminal device, the video service device obtaining, according to a motion vector of each frame in the acquired video data stream; a feature vector set of motion vectors of each frame; Determining whether the direction and the amplitude of the motion vector corresponding to the feature vector set of the two adjacent frames before and after the change; extracting the key frame from the video data stream by using the determination result of whether the change occurs, and then, for the user The terminal device provides the key frame.
  • the video service device, the video service system and the key frame extraction method thereof provide the feature vector set of the motion vector by using the motion vector of the frame, and determine the feature vector set corresponding to the two adjacent frames. Whether the direction and amplitude of the motion vector change to extract the key frame can effectively extract the frame with sudden change in speed and uniform velocity but change direction, which reduces the error rate and complexity of extracting key frames.
  • FIG. 1 is a system diagram of a video service system according to Embodiment 1 of the present invention.
  • FIG. 2 is a block diagram of a video service device according to Embodiment 1 of the present invention.
  • FIG. 3 is a block diagram of a video service apparatus according to Embodiment 2 of the present invention.
  • FIG. 4 is a block diagram of a video service device according to Embodiment 3 of the present invention.
  • FIG. 5 is a flowchart of a method for extracting a key frame according to Embodiment 4 of the present invention.
  • FIG. 6 is a histogram of an X-segment vector in a key frame extraction method according to Embodiment 4 of the present invention.
  • FIG. 7 is a histogram of a y-divided vector in a key frame extraction method according to a fourth embodiment of the present invention.
  • FIG. 1 is a system diagram of a video service system 10 according to Embodiment 1 of the present invention.
  • the video service system 10 includes: a video service device 20 and a user terminal device 30.
  • the video service device 20 and the user terminal device 30 are connected via a network (not shown) or the video service device 20 and the user terminal device 30.
  • the peers are placed in the same video terminal device.
  • the video service apparatus 20 is configured to extract a key frame by determining whether a direction and an amplitude of a motion vector corresponding to a feature vector set of two adjacent frames in the video stream change, and extract the extracted key frame.
  • the ranking is divided and provided to the user terminal device 30.
  • the video service device 20 can be a video retrieval service device or a video transmission service device or a video encoding service device.
  • FIG. 2 is a block diagram of a video service device 20 according to Embodiment 1 of the present invention.
  • the video service device 20 is a video retrieval service device and is used to provide a video retrieval information service for the user terminal device 30.
  • the video service device 20 includes a video storage module 200 and a video key frame extraction module 210.
  • the user terminal device 30 includes a key frame copy memory unit 300 and a user search and play interface 310.
  • the video storage module 200, the key frame copy memory unit 300, and the user search and play interface 310 are all well-known technologies, and their functions are not described in detail.
  • the key frame extraction module 210 is configured to acquire a motion vector of each frame in the video data stream, and acquire a feature vector set of the motion vector.
  • the key frame extraction module 210 composes a motion vector set by combining motion vectors having the same motion vector value, and a motion vector set having the largest number of motion vectors as a feature vector set.
  • the video key frame extraction module 210 decomposes the motion vector into a sub-vector in the X-axis direction and a sub-vector in the y-axis direction.
  • the feature vector set may also be acquired by using a combination of amplitude and angle, and the motion vector set of the background and the foreground may be separately extracted by a clustering method, and the motion vector set of the foreground is a feature.
  • Vector collection may also be acquired by using a combination of amplitude and angle, and the motion vector set of the background and the foreground may be separately extracted by a clustering method, and the motion vector set of the foreground is a feature.
  • the key frame extraction module 210 is further configured to extract a key frame by determining whether the direction and the amplitude of the motion vector corresponding to the feature vector set of the two adjacent frames are changed. In this embodiment, the key frame extraction module 210 extracts the key by determining whether the direction and amplitude of the motion vector of the feature vector set of the kth frame are different from the direction and magnitude of the motion vector of the feature vector set of the k-1th frame. frame.
  • the key frame extraction module 210 sets the direction of the motion vector corresponding to the feature vector set. It can be decomposed into the x-axis direction and the y-axis direction, and the magnitude of its motion vector can be expressed by the sum of the X-divided vector size and the y-divided vector size. In this embodiment, the key frame extraction module 210 determines the direction of the X-segment vector of the motion vector of the feature vector set of the k-th frame by determining the direction of the X-segment vector of the motion vector of the feature vector set of the k-th frame.
  • the direction of the y-divided vector of the motion vector of the feature vector set of the k-frame is changed with respect to the direction of the y-divided vector of the motion vector of the feature vector set of the k-th frame, or by judging the motion of the feature vector set of the k-th frame
  • the amplitude of the vector differs from the amplitude of the motion vector of the feature vector set of the k-1 frame by more than a predetermined threshold value, and the kth frame is used as the key frame.
  • the key frame extraction module 210 is further configured to determine the category of the extracted key frame after extracting the key frame.
  • the key frame categories are classified into a first type of key frame, a second type of key frame, and a third type of key frame.
  • the first type of key frames are excellent level key frames
  • the second type of key frames are good level key frames
  • the third type of key frames are general level key frames.
  • the key frame extraction module 2 10 determines the extracted key frame category by determining whether the direction and the amplitude of the motion vector corresponding to the feature vector set of the two adjacent frames are changed, that is, the key extracted. The level of the frame.
  • the key frame extraction module 210 determines the direction of the X-segment vector of the motion vector of the feature vector set of the k-th frame relative to the X-segment vector of the motion vector of the feature vector set of the k-1th frame.
  • the direction has changed, and the direction of the y-divided vector of the motion vector of the feature vector set of the k-th frame is changed from the direction of the y-divided vector of the motion vector of the feature vector set of the k-th frame, ⁇
  • the class of the frame is the first type of key frame, that is, the level of the k-th frame is divided into excellent levels.
  • the key frame extraction module 210 determines the direction of the X-segment vector of the motion vector of the feature vector set of the k-th frame relative to the X-segment vector of the motion vector of the feature vector set of the k-1th frame.
  • the direction is changed, and the direction of the y-divided vector of the motion vector of the feature vector set of the k-th frame is changed by determining the direction of the y-divided vector of the motion vector of the feature vector set of the k-th frame, and By judging that the magnitude of the motion vector of the feature vector set of the kth frame is different from the magnitude of the motion vector of the feature vector set of the k-1 frame by no more than a predetermined threshold, then k
  • the category of the frame is the second type of key frame, that is, the level of the k-th frame is divided into good levels.
  • the key frame extraction module 210 determines the feature vector set of the kth frame.
  • the direction of the x-segment vector of the combined motion vector is changed with respect to the direction of the X-segment vector of the motion vector of the feature vector set of the k-1th frame, or by determining the motion vector of the feature vector set of the k-th frame
  • the direction of the divided vector is changed with respect to the direction of the y-divided vector of the motion vector of the feature vector set of the k1th frame, and by determining the magnitude of the motion vector of the feature vector set of the k-th frame and the k-1th frame
  • the magnitude of the motion vector of the feature vector set differs by more than a predetermined threshold, then the kth
  • the category of the frame is the second type of key frame, that is, the level of the k-th frame is divided into good levels.
  • the key frame extraction module 210 determines the X-score of the motion vector of the motion vector of the k-th frame by determining the direction of the X-segment vector of the motion vector of the feature vector set of the k-th frame.
  • the direction of the vector changes, and by determining that the direction of the y-divided vector of the motion vector of the feature vector set of the k-th frame changes with respect to the direction of the y-divided vector of the motion vector of the feature vector set of the k1st frame.
  • the category of the kth frame is the second type of key frame, that is, the level of the kth frame is divided into good levels.
  • the key frame extraction module 210 determines the direction of the X-segment vector of the motion vector of the feature vector set of the k-th frame relative to the X-segment vector of the motion vector of the feature vector set of the k-1th frame.
  • the direction has changed, or the direction of the y-divided vector of the motion vector of the feature vector set of the k-th frame is changed by the direction of the y-divided vector of the motion vector set of the k-th frame, and the judgment is made.
  • the magnitude of the motion vector of the feature vector set of the kth frame is different from the amplitude of the motion vector of the feature vector set of the k-1th frame by no more than a predetermined threshold, then the kth
  • the class of the frame is the third type of key frame, that is, the level of the k-th frame is divided into a general level.
  • the key frame extraction module 210 determines the X-score of the motion vector of the motion vector of the k-th frame by determining the direction of the X-segment vector of the motion vector of the feature vector set of the k-th frame.
  • the direction of the vector changes, or by determining that the direction of the y-divided vector of the motion vector of the feature vector set of the k-th frame changes with respect to the direction of the y-divided vector of the motion vector of the feature vector set of the k1st frame.
  • the category of the kth frame is the third type of key frame, that is, the level of the kth frame is divided into general levels.
  • the video key frame extraction module 210 extracts a key frame from the video data stream of the video storage module 210, and transmits the classified key frame to the key frame of the user terminal device 30 for temporary storage.
  • the unit 300 plays the key frame information for the user to search and play the interface 310.
  • the video key frame extraction module 210 classifies the key frame by determining whether the direction and the amplitude of the motion vector corresponding to the feature vector set of the two adjacent frames are changed. In the case of poor communication quality of the network, the non-key frames are discarded first. If the communication quality is further deteriorated, the key frames with lower levels are discarded, so that the information of interest to the user can be better protected.
  • FIG. 3 is a block diagram of a video service device 20 according to Embodiment 2 of the present invention.
  • the video service device 20 is a video transmission service device, and further includes a video collection module 220, a video encoding module 230, and a scalable network transmission module 240.
  • the video collection module 220 is connected to the video key frame extraction module 210 and the video encoding module 230
  • the video encoding module 230 is connected to the video key frame extraction module 210, the scalable network transmission module 240, and the video encoding module 230.
  • the scalable network transmission module 240 is coupled to the video encoding module 230, the video keyframe extraction module 210, and the video storage module 200.
  • the video key frame extraction module 210 directly extracts the key frame from the compressed data stream transmitted by the video storage module 200, and then the key frame.
  • the location and level information is sent to the scalable network transmission module 240 along with the compressed data stream.
  • the scalable network transmission module 240 selects a corresponding protection policy according to the key frame information or a frame loss policy in the case of a limited rate to transmit the data stream.
  • the key frame extraction module 210 extracts the key frame information from the original video data stream transmitted by the video collection module 220, and the video encoding module 230 works in the same manner.
  • the original video data stream is encoded as a compressed video data stream and then passed to the scalable network transmission module 240 along with the key frame information.
  • the video service device 20 is a video encoding service device, further including a variable image (Group of
  • the video collection module 220 is connected to the video key frame extraction module 210 and the variable GOP video encoding module 250, and the variable GOP video encoding module 250 and the video key frame extraction module 210, the video storage module 200, and the video.
  • the collection module 220 and the scalable network transmission module 240 are connected.
  • the variable GOP video encoding module 250 encodes the key frame as an I frame, thereby implementing unequal length GOP encoding, which can improve encoding efficiency. Since the key frames are graded, when the two high-level key frames are far away, one or several low-level key frames can be inserted between them, so that the video of the random access engraving is not played. As for losing too many frames.
  • variable image group layer view The frequency encoding module 250 acts as an image group between every two key frames (Group of
  • GOP The division of Picture, GOP
  • the division of Picture, GOP will make the code stream have robust code stream transmission characteristics, facilitate the unequal protection transmission in transmission, and convenient frame dropping strategy; and high compression efficiency and access characteristics, GOP internal
  • the correlation is strong and the correlation between the two is easy to remove.
  • the access point is a key frame and conforms to the characteristics of the human eye.
  • FIG. 5 is a flowchart of a method for extracting a key frame according to Embodiment 4 of the present invention.
  • step S300 a video data stream is received.
  • step S302 a motion vector of each frame is acquired from the video data stream.
  • the motion vectors of each frame are separately decomposed, and the decomposition can be selected by the coordinate axes, and each motion vector is decomposed into a sub-vector of the X direction and a sub-vector of the y direction, that is, each motion vector is available ( Xi, yi) to express.
  • step S304 a feature vector set of motion vectors for each frame is acquired.
  • motion vectors having the same motion vector value are grouped into a motion vector set, and a motion vector set having the largest number of motion vectors is used as a feature vector set.
  • the following is: first extracting an X-segment vector having the same value and the largest number of ⁇ , or a y-divided vector having the same extracted value and the largest number ⁇ ; y-score corresponding to the X-segment vector value one-to-one Under the condition of the vector value, the value of the y-segment vector with the largest number of ⁇ is extracted or the value of the X-segment vector with the largest number of ⁇ is extracted under the condition that the y-divided vector value has a one-to-one corresponding X-segment vector value.
  • the method of establishing a one-dimensional histogram is used for explanation.
  • the X-segment vector value of the motion vector is analyzed and the y-divided vector corresponding to xi_mo S t is analyzed.
  • the value yi_mo S t of the y-divided vector may also be extracted first, and then the xi_mo S t of the x-divided vector may be extracted.
  • the feature vector set may also be acquired by using a combination of amplitude and angle, and the motion vector set of the background and the foreground may be extracted by a clustering method, and the motion vector set of the foreground is a feature vector set.
  • step S306 it is determined whether each frame is a key frame. In this embodiment, by judging the two adjacent Whether the direction and the amplitude of the motion vector corresponding to the feature vector set of the frame change determines whether each frame is a key frame. In this embodiment, it is determined whether the kth frame is the key by determining whether the direction and the amplitude of the motion vector of the feature vector set of the kth frame are different from the direction and the amplitude of the motion vector of the feature vector set of the k-1th frame. frame.
  • Step S306 If it is determined that the kth frame is not a key frame, continue to determine whether the k+1th frame is a key frame, that is, proceed to step S306; if it is determined that the kth frame is a key frame, extract the kth frame as a key frame, and Step S308 is performed.
  • the direction of the motion vector can be divided into an X-axis direction and a y-axis direction, and the magnitude of the motion vector is represented by a sum of the X-divided vector size and the y-divided vector size.
  • the X-segment vector if the X value is positive, its direction is represented by ten, if it is 0, it is represented by 0, and if it is negative, it is represented by one. The same is true for the direction of the y-divided vector.
  • the direction of the X-segment vector of the motion vector of the feature vector set of the k-th frame is determined to be different from the direction of the X-segment vector of the motion vector of the feature vector set of the k-th frame, , that is, the direction of the X-divided vector changes from ten to one or from one to ten or from 0 to non-zero or from non-zero to 0, then the k-th frame is a key frame, and the k-th frame is extracted. As a keyframe.
  • the direction of the y-divided vector of the motion vector of the feature vector set of the k-th frame is changed, that is, The direction of the y-divided vector is changed from ten to one or from one to ten or from 0 to non-zero or from non-zero to 0, and the k-th frame is extracted as a key frame.
  • the kth frame is the key Frame, and extract the kth frame as a key frame.
  • the threshold value is 60. In other embodiments of the invention, the threshold value may also be other values.
  • step S308 it is determined whether the category of the key frame is the first type of key frame.
  • the categories of the key frames are divided into a first type of key frame, a second type of key frame, and a third type of key frame.
  • the first type of key frames are excellent level key frames
  • the second type of key frames are good class key frames
  • the third type of key frames are general class key frames.
  • the category of the key frame is judged by judging whether the direction and the amplitude of the motion vector corresponding to the feature vector set of the two adjacent frames are changed, that is, the key frame is classified.
  • step S316 is performed; otherwise, step S310 is performed.
  • step S310 is performed.
  • step S310 is performed.
  • step S316 is performed; otherwise, step S310 is performed.
  • step S310 is performed.
  • step S310 it is determined whether the category of the key frame is the second type of key frame. If it is determined that the category of the key frame is the second type of key frame, step S316 is performed; otherwise, step S312 is performed.
  • the magnitude of the amplitude is different from the magnitude of the motion vector magnitude of the feature vector set of the k-1 frame by no more than a predetermined threshold, and the category of the kth frame is the second type of key frame, that is, the kth frame is divided into good key frames. .
  • the direction of the X-segment vector of the motion vector of the feature vector set of the k-th frame is changed, the direction of the X-segment vector of the motion vector of the feature vector set of the k-th frame is changed. , that is, the direction of the X-segment vector is changed from ten to one or from one to ten, and the magnitude of the motion vector magnitude of the feature vector set of the k-th frame and the motion of the feature vector set of the k-1 frame are determined. If the magnitude of the vector amplitude differs by more than a predetermined threshold, then the category of the kth frame is the second type of key frame, that is, the kth frame is divided into good key frames.
  • the direction of the y-divided vector of the motion vector of the feature vector set of the k-th frame is changed, the direction of the y-divided vector of the motion vector of the feature vector set of the k-th frame is changed. , that is, the direction of the y-divided vector is changed from ten to one or from one to ten, and the magnitude of the motion vector magnitude of the feature vector set of the k-th frame and the motion of the feature vector set of the k-1 frame are determined.
  • the magnitude of the vector magnitude is different If the threshold is greater than the predetermined threshold, the category of the kth frame is the second type of key frame, that is, the kth frame is divided into good key frames.
  • the direction of the X-segment vector of the motion vector of the feature vector set of the k-th frame is changed, the direction of the X-segment vector of the motion vector of the feature vector set of the k-th frame is changed. , that is, the direction of the X-segment vector changes from 0 to non-zero or from non-zero to 0, and the direction of the y-divided vector of the motion vector of the feature vector set of the k-th frame is determined relative to the feature vector set of the k-1th frame.
  • the direction of the y-divided vector of the motion vector changes, that is, the direction of the y-divided vector changes from 0 to non-zero or from non-zero to 0, then the category of the k-th frame is the second type of key frame, that is, the k-th frame is divided.
  • a good grade keyframe is the second type of key frame, that is, the k-th frame is divided.
  • step S312 it is determined whether the category of the key frame is a third type of key frame. If it is determined that the category of the key frame is the third type of key frame, step S316 is performed; otherwise, step S314 is performed.
  • the direction of the y-divided vector of the motion vector of the feature vector set of the k-th frame is changed, the direction of the y-divided vector of the motion vector set of the k-th frame is changed. , that is, the direction of the y-divided vector is changed from ten to one or from one to ten, and the magnitude of the motion vector magnitude of the feature vector set of the k-th frame and the motion of the feature vector set of the k-1 frame are determined. If the magnitude of the vector amplitude differs by no more than a predetermined threshold, then the category of the kth frame is a third type of key frame, that is, the kth frame is divided into general level key frames.
  • step S314 the undivided key frame is transmitted to the user terminal device 30.
  • step S316 the key frame of the category is transmitted to the user terminal device 30.
  • the "predetermined threshold value" in the above embodiment of the present invention may be a constant value or a value that varies depending on the scene.
  • the video service device 20, the video service system 10, and the key frame extraction method thereof are provided by using the motion vector of the frame to obtain a feature vector set of the motion vector, and determining the feature vector set of the adjacent two frames before and after. Whether the direction and amplitude of the corresponding motion vector change to extract key frames, thereby effectively extracting frames with sudden changes in speed and uniform velocity but changing direction, reducing the error rate and complexity of extracting key frames, and reducing the amount of calculation; Similarly, by classifying the key frames by using the direction and magnitude of the motion vector, those non-key frames can be discarded first in the network communication quality difference. If the communication quality is further deteriorated, the key frames with lower levels are discarded. Better protect user interest information.
  • the present invention can be implemented by hardware, or can be realized by means of software plus necessary general hardware platform, the present invention.
  • the technical solution can be embodied in the form of a software product, which can be stored in a computer readable storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a plurality of instructions for making a computer device (may be a personal computer, server, or network device, etc.) Perform the methods described in various embodiments of the present invention.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Image Analysis (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

La présente invention concerne un procédé d’extraction de l’image clé, un appareil de service vidéo et un système de service vidéo. Le procédé est utilisé pour extraire l’image clé du flux de données vidéo dans le système de service vidéo. Le procédé comprend : l’obtention du vecteur de mouvement de chaque image dans le flux de données vidéo, et l’obtention des caractéristiques de l’agrégat de vecteurs selon le vecteur de mouvement; la détermination de la correspondance ou non de la modification ou non de la direction et de l’amplitude du vecteur de mouvement correspondant à l’agrégat de vecteur de caractéristiques des deux images adjacentes vers l’avant et vers l’arrière; l’extraction de l’image clé au moyen du résultat déterminé. Ainsi l’image dont la vitesse change brusquement peut être extraite efficacement.
PCT/CN2009/071783 2008-05-13 2009-05-13 Système de service vidéo, appareil de service vidéo et procédé correspondant d’extraction d’images clés de service vidéo Ceased WO2009138037A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN 200810067177 CN101582063A (zh) 2008-05-13 2008-05-13 视频服务系统、视频服务装置及其关键帧的提取方法
CN200810067177.8 2008-05-13

Publications (1)

Publication Number Publication Date
WO2009138037A1 true WO2009138037A1 (fr) 2009-11-19

Family

ID=41318372

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2009/071783 Ceased WO2009138037A1 (fr) 2008-05-13 2009-05-13 Système de service vidéo, appareil de service vidéo et procédé correspondant d’extraction d’images clés de service vidéo

Country Status (2)

Country Link
CN (1) CN101582063A (fr)
WO (1) WO2009138037A1 (fr)

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9158974B1 (en) 2014-07-07 2015-10-13 Google Inc. Method and system for motion vector-based video monitoring and event categorization
US9170707B1 (en) 2014-09-30 2015-10-27 Google Inc. Method and system for generating a smart time-lapse video clip
US9449229B1 (en) 2014-07-07 2016-09-20 Google Inc. Systems and methods for categorizing motion event candidates
US9501915B1 (en) 2014-07-07 2016-11-22 Google Inc. Systems and methods for analyzing a video stream
USD782495S1 (en) 2014-10-07 2017-03-28 Google Inc. Display screen or portion thereof with graphical user interface
US10127783B2 (en) 2014-07-07 2018-11-13 Google Llc Method and device for processing motion events
US10140827B2 (en) 2014-07-07 2018-11-27 Google Llc Method and system for processing motion event notifications
US10192415B2 (en) 2016-07-11 2019-01-29 Google Llc Methods and systems for providing intelligent alerts for events
US10380429B2 (en) 2016-07-11 2019-08-13 Google Llc Methods and systems for person detection in a video feed
US10664688B2 (en) 2017-09-20 2020-05-26 Google Llc Systems and methods of detecting and responding to a visitor to a smart home environment
US10685257B2 (en) 2017-05-30 2020-06-16 Google Llc Systems and methods of person recognition in video streams
US10957171B2 (en) 2016-07-11 2021-03-23 Google Llc Methods and systems for providing event alerts
US10972685B2 (en) 2017-05-25 2021-04-06 Google Llc Video camera assembly having an IR reflector
US11035517B2 (en) 2017-05-25 2021-06-15 Google Llc Compact electronic device with thermal management
US11082701B2 (en) 2016-05-27 2021-08-03 Google Llc Methods and devices for dynamic adaptation of encoding bitrate for video streaming
CN113542868A (zh) * 2021-05-26 2021-10-22 浙江大华技术股份有限公司 视频关键帧选取方法、装置、电子设备、存储介质
US11356643B2 (en) 2017-09-20 2022-06-07 Google Llc Systems and methods of presenting appropriate actions for responding to a visitor to a smart home environment
US11599259B2 (en) 2015-06-14 2023-03-07 Google Llc Methods and systems for presenting alert event indicators
US11689784B2 (en) 2017-05-25 2023-06-27 Google Llc Camera assembly having a single-piece cover element
US11783010B2 (en) 2017-05-30 2023-10-10 Google Llc Systems and methods of person recognition in video streams
US11893795B2 (en) 2019-12-09 2024-02-06 Google Llc Interacting with visitors of a connected home environment

Families Citing this family (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2010119410A1 (fr) * 2009-04-14 2010-10-21 Koninklijke Philips Electronics N.V. Extraction d'images clés pour une analyse de contenu vidéo
JP5746766B2 (ja) 2010-08-02 2015-07-08 ペキン ユニバーシティ 効果的なビデオの分類および検索のための典型的モーションフローの抽出
CN104125430B (zh) * 2013-04-28 2017-09-12 华为技术有限公司 视频运动目标检测方法、装置以及视频监控系统
CN103413322B (zh) * 2013-07-16 2015-11-18 南京师范大学 序列视频关键帧提取方法
CN104754322B (zh) * 2013-12-27 2018-01-23 华为技术有限公司 一种立体视频舒适度评价方法及装置
CN107748569B (zh) * 2017-09-04 2021-02-19 中国兵器工业计算机应用技术研究所 用于无人机的运动控制方法、装置及无人机系统
CN109976646B (zh) * 2019-03-22 2020-11-10 上海沈德医疗器械科技有限公司 磁共振扫描控制和影像传输方法、服务端和程序
CN111553302B (zh) * 2020-05-08 2022-01-04 深圳前海微众银行股份有限公司 关键帧选取方法、装置、设备及计算机可读存储介质
CN111836072B (zh) * 2020-05-21 2022-09-13 北京嘀嘀无限科技发展有限公司 视频处理方法、装置、设备和存储介质
CN115037936A (zh) * 2021-03-04 2022-09-09 华为技术有限公司 视频编解码方法及装置
CN114697764B (zh) * 2022-06-01 2022-09-02 深圳比特微电子科技有限公司 一种生成视频摘要的方法、装置和可读存储介质
CN116723335B (zh) * 2023-06-29 2024-06-18 西安邮电大学 用视频压缩编码信息提取视频关键帧的方法
CN116681715B (zh) * 2023-08-04 2023-10-10 杭州脉流科技有限公司 基于像素值变化的血管分割方法、装置、设备和存储介质

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1842165A (zh) * 2005-03-31 2006-10-04 株式会社东芝 产生内插帧的方法和装置
CN1898948A (zh) * 2003-12-23 2007-01-17 皇家飞利浦电子股份有限公司 用于稳定化视频数据的方法和系统

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1898948A (zh) * 2003-12-23 2007-01-17 皇家飞利浦电子股份有限公司 用于稳定化视频数据的方法和系统
CN1842165A (zh) * 2005-03-31 2006-10-04 株式会社东芝 产生内插帧的方法和装置

Cited By (57)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9779307B2 (en) 2014-07-07 2017-10-03 Google Inc. Method and system for non-causal zone search in video monitoring
US9886161B2 (en) 2014-07-07 2018-02-06 Google Llc Method and system for motion vector-based video monitoring and event categorization
US9213903B1 (en) 2014-07-07 2015-12-15 Google Inc. Method and system for cluster-based video monitoring and event categorization
US9224044B1 (en) * 2014-07-07 2015-12-29 Google Inc. Method and system for video zone monitoring
US9354794B2 (en) 2014-07-07 2016-05-31 Google Inc. Method and system for performing client-side zooming of a remote video feed
US9420331B2 (en) 2014-07-07 2016-08-16 Google Inc. Method and system for categorizing detected motion events
US9449229B1 (en) 2014-07-07 2016-09-20 Google Inc. Systems and methods for categorizing motion event candidates
US9479822B2 (en) 2014-07-07 2016-10-25 Google Inc. Method and system for categorizing detected motion events
US9489580B2 (en) 2014-07-07 2016-11-08 Google Inc. Method and system for cluster-based video monitoring and event categorization
US9501915B1 (en) 2014-07-07 2016-11-22 Google Inc. Systems and methods for analyzing a video stream
US9544636B2 (en) 2014-07-07 2017-01-10 Google Inc. Method and system for editing event categories
US9602860B2 (en) 2014-07-07 2017-03-21 Google Inc. Method and system for displaying recorded and live video feeds
US9609380B2 (en) 2014-07-07 2017-03-28 Google Inc. Method and system for detecting and presenting a new event in a video feed
US11011035B2 (en) 2014-07-07 2021-05-18 Google Llc Methods and systems for detecting persons in a smart home environment
US9674570B2 (en) 2014-07-07 2017-06-06 Google Inc. Method and system for detecting and presenting video feed
US9672427B2 (en) 2014-07-07 2017-06-06 Google Inc. Systems and methods for categorizing motion events
US10867496B2 (en) 2014-07-07 2020-12-15 Google Llc Methods and systems for presenting video feeds
US10977918B2 (en) 2014-07-07 2021-04-13 Google Llc Method and system for generating a smart time-lapse video clip
US9940523B2 (en) 2014-07-07 2018-04-10 Google Llc Video monitoring user interface for displaying motion events feed
US10108862B2 (en) 2014-07-07 2018-10-23 Google Llc Methods and systems for displaying live video and recorded video
US10127783B2 (en) 2014-07-07 2018-11-13 Google Llc Method and device for processing motion events
US10140827B2 (en) 2014-07-07 2018-11-27 Google Llc Method and system for processing motion event notifications
US10180775B2 (en) 2014-07-07 2019-01-15 Google Llc Method and system for displaying recorded and live video feeds
US10789821B2 (en) 2014-07-07 2020-09-29 Google Llc Methods and systems for camera-side cropping of a video feed
US10192120B2 (en) 2014-07-07 2019-01-29 Google Llc Method and system for generating a smart time-lapse video clip
US11250679B2 (en) 2014-07-07 2022-02-15 Google Llc Systems and methods for categorizing motion events
US10452921B2 (en) 2014-07-07 2019-10-22 Google Llc Methods and systems for displaying video streams
US10467872B2 (en) 2014-07-07 2019-11-05 Google Llc Methods and systems for updating an event timeline with event indicators
US9158974B1 (en) 2014-07-07 2015-10-13 Google Inc. Method and system for motion vector-based video monitoring and event categorization
US11062580B2 (en) 2014-07-07 2021-07-13 Google Llc Methods and systems for updating an event timeline with event indicators
US9170707B1 (en) 2014-09-30 2015-10-27 Google Inc. Method and system for generating a smart time-lapse video clip
USD893508S1 (en) 2014-10-07 2020-08-18 Google Llc Display screen or portion thereof with graphical user interface
USD782495S1 (en) 2014-10-07 2017-03-28 Google Inc. Display screen or portion thereof with graphical user interface
US11599259B2 (en) 2015-06-14 2023-03-07 Google Llc Methods and systems for presenting alert event indicators
US11082701B2 (en) 2016-05-27 2021-08-03 Google Llc Methods and devices for dynamic adaptation of encoding bitrate for video streaming
US10657382B2 (en) 2016-07-11 2020-05-19 Google Llc Methods and systems for person detection in a video feed
US10380429B2 (en) 2016-07-11 2019-08-13 Google Llc Methods and systems for person detection in a video feed
US10957171B2 (en) 2016-07-11 2021-03-23 Google Llc Methods and systems for providing event alerts
US10192415B2 (en) 2016-07-11 2019-01-29 Google Llc Methods and systems for providing intelligent alerts for events
US11587320B2 (en) 2016-07-11 2023-02-21 Google Llc Methods and systems for person detection in a video feed
US11689784B2 (en) 2017-05-25 2023-06-27 Google Llc Camera assembly having a single-piece cover element
US11353158B2 (en) 2017-05-25 2022-06-07 Google Llc Compact electronic device with thermal management
US11156325B2 (en) 2017-05-25 2021-10-26 Google Llc Stand assembly for an electronic device providing multiple degrees of freedom and built-in cables
US10972685B2 (en) 2017-05-25 2021-04-06 Google Llc Video camera assembly having an IR reflector
US11680677B2 (en) 2017-05-25 2023-06-20 Google Llc Compact electronic device with thermal management
US11035517B2 (en) 2017-05-25 2021-06-15 Google Llc Compact electronic device with thermal management
US11783010B2 (en) 2017-05-30 2023-10-10 Google Llc Systems and methods of person recognition in video streams
US11386285B2 (en) 2017-05-30 2022-07-12 Google Llc Systems and methods of person recognition in video streams
US10685257B2 (en) 2017-05-30 2020-06-16 Google Llc Systems and methods of person recognition in video streams
US11356643B2 (en) 2017-09-20 2022-06-07 Google Llc Systems and methods of presenting appropriate actions for responding to a visitor to a smart home environment
US11256908B2 (en) 2017-09-20 2022-02-22 Google Llc Systems and methods of detecting and responding to a visitor to a smart home environment
US10664688B2 (en) 2017-09-20 2020-05-26 Google Llc Systems and methods of detecting and responding to a visitor to a smart home environment
US11710387B2 (en) 2017-09-20 2023-07-25 Google Llc Systems and methods of detecting and responding to a visitor to a smart home environment
US12125369B2 (en) 2017-09-20 2024-10-22 Google Llc Systems and methods of detecting and responding to a visitor to a smart home environment
US11893795B2 (en) 2019-12-09 2024-02-06 Google Llc Interacting with visitors of a connected home environment
US12347201B2 (en) 2019-12-09 2025-07-01 Google Llc Interacting with visitors of a connected home environment
CN113542868A (zh) * 2021-05-26 2021-10-22 浙江大华技术股份有限公司 视频关键帧选取方法、装置、电子设备、存储介质

Also Published As

Publication number Publication date
CN101582063A (zh) 2009-11-18

Similar Documents

Publication Publication Date Title
WO2009138037A1 (fr) Système de service vidéo, appareil de service vidéo et procédé correspondant d’extraction d’images clés de service vidéo
CN104618803B (zh) 信息推送方法、装置、终端及服务器
CN106162177B (zh) 视频编码方法和装置
US11023618B2 (en) Systems and methods for detecting modifications in a video clip
CN104427291B (zh) 一种图像处理方法及设备
CN104754413B (zh) 基于图像搜索识别电视信号并推荐信息的方法和设备
KR102045575B1 (ko) 스마트 미러 디스플레이 장치
WO2013069605A1 (fr) Système de recherche d'images similaires
EP1938208A1 (fr) Annotation de visages dans une video en continu
EP3513326B1 (fr) Procédés, systèmes et supports pour détecter des vidéos stéréoscopiques par génération d'empreintes digitales pour de multiples parties d'une trame vidéo
CN113395583B (zh) 水印检测方法、装置、计算机设备及存储介质
CN116189251A (zh) 实时人脸图像驱动方法、装置、电子设备及存储介质
CN106851332B (zh) 视频流处理方法、装置和系统
JP6340675B1 (ja) オブジェクト抽出装置、オブジェクト認識システム及びメタデータ作成システム
JP2018206292A (ja) 要約映像生成装置及びプログラム
CN110177332B (zh) 数据传输方法及装置
CN117857808B (zh) 一种基于数据分类压缩的高效视频传输方法及系统
JP5880558B2 (ja) 映像処理システム、視聴者の嗜好判定方法、映像処理装置およびその制御方法と制御プログラム
WO2021129444A1 (fr) Procédé et appareil de regroupement de fichiers, et support de stockage et dispositif électronique
CN107733874A (zh) 信息处理方法、装置、计算机设备和存储介质
CN114387440B (zh) 一种视频裁剪方法、装置及存储介质
WO2023241485A1 (fr) Procédé de traitement vidéo, extrémité d'envoi de vidéo, extrémité de réception de vidéo, support de stockage et produit programme
CN116055762A (zh) 视频合成方法及装置、电子设备和存储介质
KR102826938B1 (ko) 비디오 콘텐츠 아이템들의 지각적 품질 표시자들을 결정하기 위한 방법들, 시스템들, 및 매체들
CN110769258A (zh) 一种用于特定场景多语义区域的图像压缩方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 09745421

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 09745421

Country of ref document: EP

Kind code of ref document: A1