[go: up one dir, main page]

WO2009052618A1 - System, method and computer program for capturing, sharing, and annotating content - Google Patents

System, method and computer program for capturing, sharing, and annotating content Download PDF

Info

Publication number
WO2009052618A1
WO2009052618A1 PCT/CA2008/001866 CA2008001866W WO2009052618A1 WO 2009052618 A1 WO2009052618 A1 WO 2009052618A1 CA 2008001866 W CA2008001866 W CA 2008001866W WO 2009052618 A1 WO2009052618 A1 WO 2009052618A1
Authority
WO
WIPO (PCT)
Prior art keywords
content
operable
user
capture device
capture
Prior art date
Application number
PCT/CA2008/001866
Other languages
French (fr)
Other versions
WO2009052618A8 (en
Inventor
Steven Mann
James Fung
Daniel Chen
Raymond Chung Hing Lo
Original Assignee
Steven Mann
James Fung
Daniel Chen
Raymond Chung Hing Lo
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Steven Mann, James Fung, Daniel Chen, Raymond Chung Hing Lo filed Critical Steven Mann
Publication of WO2009052618A1 publication Critical patent/WO2009052618A1/en
Publication of WO2009052618A8 publication Critical patent/WO2009052618A8/en

Links

Classifications

    • GPHYSICS
    • G11INFORMATION STORAGE
    • G11BINFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/19Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier
    • G11B27/28Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording
    • G11B27/32Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording on separate auxiliary tracks of the same or an auxiliary record carrier
    • G11B27/322Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information detectable on the record carrier by using information signals recorded by the same method as the main recording on separate auxiliary tracks of the same or an auxiliary record carrier used signal is digitally coded
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00Handling natural language data
    • G06F40/10Text processing
    • G06F40/166Editing, e.g. inserting or deleting
    • G06F40/169Annotation, e.g. comment data or footnotes
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/32Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
    • H04L9/3236Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials using cryptographic hash functions
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L9/00Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols
    • H04L9/32Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials
    • H04L9/3297Cryptographic mechanisms or cryptographic arrangements for secret or secure communications; Network security protocols including means for verifying the identity or authority of a user of the system or for message authentication, e.g. authorization, entity authentication, data integrity or data verification, non-repudiation, key authentication or verification of credentials involving time stamps, e.g. generation of time stamps
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/765Interface circuits between an apparatus for recording and another apparatus
    • H04N5/77Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/18Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00Television systems
    • H04N7/18Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/183Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
    • H04N7/185Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source from a mobile camera, e.g. for remote control
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L2209/00Additional information or applications relating to cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00
    • H04L2209/60Digital content management, e.g. content distribution
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L2209/00Additional information or applications relating to cryptographic mechanisms or cryptographic arrangements for secret or secure communication H04L9/00
    • H04L2209/80Wireless
    • H04L2209/805Lightweight hardware, e.g. radio-frequency identification [RFID] or sensor
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/765Interface circuits between an apparatus for recording and another apparatus
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/76Television signal recording
    • H04N5/765Interface circuits between an apparatus for recording and another apparatus
    • H04N5/77Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera
    • H04N5/772Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera the recording apparatus and the television camera being placed in the same enclosure
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/79Processing of colour television signals in connection with recording
    • H04N9/7921Processing of colour television signals in connection with recording for more than one processing mode
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/79Processing of colour television signals in connection with recording
    • H04N9/80Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/804Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components
    • H04N9/8042Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components involving data reduction
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/79Processing of colour television signals in connection with recording
    • H04N9/80Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/804Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components
    • H04N9/8042Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components involving data reduction
    • H04N9/8047Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback involving pulse code modulation of the colour picture signal components involving data reduction using transform coding
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00Details of colour television systems
    • H04N9/79Processing of colour television signals in connection with recording
    • H04N9/80Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/82Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only
    • H04N9/8205Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only involving the multiplexing of an additional signal and the colour video signal

Definitions

  • This invention relates to a system, method and computer program for capturing, sharing and annotating content.
  • Inverse surveillance might, for example, include citizens photographing police misconduct, shoppers photographing shopkeepers, and passengers photographing reckless cab drivers from within the very cab that might, for example, be involved in an automobile accident.
  • An important aspect of inverse surveillance is that it emanates from individuals recording their personal experience and their immediate vicinity, rather than the recording/monitoring of individuals by an outside party.
  • siveillance has been used to describe this recording of an activity by a participant in the activity. Sousveillance includes both "inverse surveillance” (i.e. citizens photographing police, shoppers photographing shopkeepers, etc.), as well as passive capture of everything in the environment (e.g. a person simply recording their entire life for no particular specified reason). Whereas “surveillance” means “to watch from above”, the term “sousveillance” shares French roots in that it is the composite of the French word “sous”, meaning below and the French word “veiller”, meaning to watch.
  • Sousveillance involves the creating recordings, including of audio or video content, as well as other captured signals such as can be measured from a person's body and their environment.
  • “recordings” refers to audio and video recordings, as well as other signals that can be recorded
  • “content” refers to audio or video or other measurable or captureable data.
  • “wireless medium” refers to any form of wireless connection which allows exchanges of data between devices or systems.
  • Some example wireless networks and protocol used in this invention include Global System for Mobile communications (GSM), Universal Mobile
  • UMTS Telecommunications System
  • Wi-Fi IEEE 802.11
  • Bluetooth for short-range data transmission.
  • Cyborglogging is a concept that involves the capture of one's own life on a continuous basis.
  • Applications may be personal safety, health monitoring, or intelligent wearable computer systems that create interactive personal assistants for persons with special needs, or the like. See for example, “Wearable Computing, A first step toward personal imaging", IEEE Computer, 30(2): 25-32, February 1997 (feature article), as well as “Intelligent Signal Processing", Proceedings of the IEEE, 86(11), cover+pp2123-2151, 1998.
  • the cyborglog or "glog” (CyborGLOG) is the successor of the weblog or “blog” (WeBLOG).
  • Blogging has evolved more quickly than glogging, but the technology necessary for glogging is becoming more prevalent, and thus glogging is entering into the mainstream.
  • Personal experiences have been shared making use of an online diary weblog or blog.
  • individuals have been updating web logs to reflect mobile terminal activity (see US Patent 7,069,003).
  • individuals have been making use of wireless cameras or camera phones for the taking of digital pictures and then sending those pictures or "messages" to a remote system via a wireless network (see US Patent 7, 173,651 ).
  • Glogging works similarly but often in a more automated fashion using varied signal processing techniques.
  • Terrorism originated in France, during the "Reign of Terror" (1793 September 5 to 1794 July 28). Originally the term “Terrorism” was used to denote a government's actions against its own people in order to create fear (Terror) as a way of keeping the people docile or obedient. [http://en.wikipedia.org/wiki/Reign_of_Terror]
  • the present invention provides a system for capturing sharing and annotating content characterized in that the system comprises at least one content capture device operable to capture content on a continuous basis and transmit the content and the system comprises at least one remote computer operable to receive the transmitted content and the at least one remote computer includes or is linked to a remotely accessible database to assemble and store the content.
  • a further aspect of the present invention provides a method for capturing sharing and annotating content characterized by the steps of capturing content using a client application residing on a content capturing device; transmitting the content from the client application to a server application residing on one or more remote computers; assembling and storing the content on a database associated with the server application and enabling one or more users or observers to access the content stored within the database and annotate the content.
  • FIG. 1 depicts an individual wearing a head mounted embodiment of the present invention transmitting recordings to one or more servers.
  • FIG. 2 depicts annotations being associated with recordings.
  • FIG. 3 depicts a Time-scalable viewer.
  • FIG. 4 depicts a Composity vs. Time graph.
  • FIG. 5 depicts an interface for the Glogger Intelligent Navigation System.
  • FIG. 6 depicts Sight License aspects of the invention.
  • FIG. 1 depicts a user 100 affixed with a portable device 150 which is mounted or implanted in the body of user 100.
  • the portable device 150 is capable of capturing images, videos or audio and transmitting its recordings to one or more server computers 130.
  • the portable device 150 may be in the form of a wearable cameraphone, a wearable computer, or any handheld device with capturing capability.
  • the sensors unit 180 is a device with a collection of sensors that collects signals from user 100 and transmits the post-processed signals back to the portable device 150.
  • Remote observers 140 may observe and annotate on the recordings user 100 or user 101 transmitted to one or more server computers 130.
  • the wireless mediums 110, 120, 160, 161, 191 can allow the portable device 150 and 151 to exchange content with server computers 130 and remote observers 140.
  • the high bandwidth wireless medium (e.g. 3 G or Wi- Fi network) 120 may be used to transmit both high and low resolution content.
  • a low bandwidth version of one or more images are sent initially and a higher bandwidth version is sent later when bandwidth becomes available.
  • the low resolution version can be broadcast or certified or escrowed by an escrow-stamper, or escrow-broadcast to help insure the integrity of the corresponding higher resolution version that comes later.
  • the higher resolution images can be assigned a similar or identical or related identifier or meta tags, or enter into a database to replace or update the originals, but with a way to verify integrity by going back to escrow- stamped originals.
  • the portable devices 150 and 151 may have different form factor and may run on different operating systems, but both devices may connect and exchange data with each other using the wireless medium 160.
  • a system, method, and computer program is provided that enables a portable device 150 or 151 to operate in "sousveillance mode" to create a recording of the experiences of an individual or one or more objects or other individuals in the individual's vicinity.
  • the portable device is linked to one or more servers 130 that are operable to enable a plurality of users (also referred to as community members) to access a web hosted community such that the shared content is the object of interaction between community members or remote observers 140.
  • the portable device 150 consists of at least one body-borne (hand-held, portable, wearable, implantable, or the like) image capture device, a body borne computational device, means for transmission of one or more images, video, audio, or other data from the capture device, and means for creation of a database records on one or more server computers 130.
  • the portable device in order to enable the community function described below, may include a display or any form of visual unit to enable users to view images or other data stored locally on the portable device.
  • the portable device 150 may be worn (e.g. a WearTelTM wearable camera phone) or hand-held.
  • the portable device 150 may be connected through a wireless medium 160 to at least one other portable device.
  • the wireless medium may transcend time or space, i.e. the goal may be storage and retrieval later (sending to another time) or it may be sending to another place.
  • Each portable device includes or may be linked to a client application in accordance with the present invention that enables sharing of content, including in accordance with the peer-to-peer implementation described below.
  • Each may denote an object in four-dimensional (x, y, z, time) space, i.e. one party of the communication may be a person at one point in time, and the other party may be the same person at a later point in time.
  • the communication is across time, and may not necessarily need to be across space (i.e. personal visual memory prosthetic to help a person in wayf ⁇ nding, by communicating with themselves at a later point in time).
  • the portable device 150 is connected through a wireless medium 110, 120, or 190 to at least one server computer 130.
  • the portable device 150 includes or is linked to a client application that is operable to enable the portable device 150 to capture content, whether a series of files or a data stream, and transmit this to the server computer via the Internet, by means for example of a wireless gateway between the wireless medium and the Internet.
  • the server computers 130 include at least one database and a server application that enables the functions of the server computer described below.
  • the portable device 150 is operable to transmit the series of files or data stream to one or more server computers 130, the server computers 130 being operable to assemble the files or the data stream to provide a "cyborglog" (as further described below), the server computer 130 being further operable to enable one or more users and remote observers 140 to interact with the cyborglog, thereby providing the sousveillance functionality described herein.
  • the server application in 130 includes a database management utility linked to the database that enables content associated with a plurality of community members to be stored to the database, and accessed or shared, in accordance with parameters further described below.
  • the database management utility it provides hierarchical access, security controls, and/or privacy controls related to access or sharing of the content.
  • the server computers 130 are best understood as including a series of hardware and software utilities generally associated with online environments for storing, accessing and sharing content, modified to enable the functions described herein.
  • the hardware and software utilities are those generally associated with enabling community based online environment, again modified to enable the specific community functions described herein such as annotating the content described.
  • the system, method, and computer program provides continuous capture of content for sousveillance purposes.
  • the system, method, and computer program enables the creation of one or more sousveillance recordings that include all or part of a person's life either without the need for a user's thought or effort or with minimum user intervention through intelligence signal processing on the signal collected by the sensors unit 180.
  • the system allows the user to optionally intervene in this recording, including by selectively determining content to be captured and/or transmitted. Content may also be tagged based on user intervention as described below.
  • the portable device 150 may be a body borne device, i.e. a personal device that may be perceived as clothing or personal technology that may be unobtrusive when capturing and transmitting video or audio signals.
  • a body borne device may pass without notice whereas a device that is not body borne such as a bag or a camcorder may not be allowed into certain areas, thereby limiting or interrupting continuous capture of content.
  • a body borne device in the first person perspective may provide a casual capture perspective that ties in to the overall notion of "sousveillance" as opposed to surveillance (personal vs. archicentric). Sousveillance resituates the locus of control of the collected data to the person, as opposed to the organization recording.
  • the sousveillance may take the form of a single log that captures a person's lifelong personal experience.
  • the architecture that implements the invention is a client/server application.
  • the client application running on the one or more image capture devices, and the server application running on the one or more server computers 130.
  • a computer program is provided that may be implemented on a J2ME and Symbian platform and may run on the portable device 130.
  • the "cyborglogger” application has been successfully employed on Nokia Series 60v2 , Sony Ericsson , and Motorola Linux phones. Further, J2ME cellular phones that support the MIDP 2.0 and CLDC 1.0 profiles as well as JSR-135 (Mobile Media API) should be able to run the "cyborglogger” application. Any other portable wireless device with resources suitable for content capture as described may be used.
  • the client "cyborglogger” application is first launched on the portable device 130 such as a cameraphone, the application automatically prompts the registration of a new glogger account.
  • the application form has three fields: username, password and email address. Once registered, the user will receive confirmation information at the client provided email address together with the URL of the glogger space created for the user in the form of http://serverurl/username where username is the field provided by the user previously in the registration process, and serverurl is the domain to the main server.
  • the main menu may provide numerous community functions such as a community page, an album, a friends listing, an inbox, an image recorder, a video recorder, a live events folder, a remote controller on/off option, an application settings option, and exit.
  • the user 100 or 101 running a "cyborglogger" equipped portable device 150 or 151 may upload their recordings to one or more servers 130.
  • the uploading may be accomplished using multimedia message service (MMS), email or a standard HTTP POST through the client application or through the built-in browser on the phone.
  • MMS multimedia message service
  • email or a standard HTTP POST through the client application or through the built-in browser on the phone.
  • the present invention contemplates continuous capture, i.e. the application continuously uploads the image streams to one or more server computers 130 without user intervention, but also provides for selective content capture based on user defined parameters.
  • the user 100 may first capture an image by using a capture button or equivalent.
  • the capture button may take a JPEG/PNG/GIF/TIFF format snapshot or a MPEG1/MPEG2/MP4/3GP/3GPP/WMV/AVI/MOV format video and store it in a local memory buffer.
  • the device may generate a corresponding thumbnail of the recording for display on the display of the device. If user 100 is not satisfied with the recording, user 100 may remove it from local memory by selecting a delete option or equivalent.
  • user 100 may select upload which uploads the recording from the local memory buffer through the use of a HTTP POST request to the server computer via the wireless medium on the device.
  • the user 100 may upload the recording using MMS, email or other equivalent protocol, by submitting the recording as an attachment to the unique glogger email account in this form: i.username- ⁇ assword@serverdomain.
  • users 100 or 101 may directly upload recording using a web browser, and access to the content directly via HTTP GET or POST requests.
  • the server application implements HTTP protocols, and may be implemented using PHP and MySQL, or any languages which are capable of performing similar functions.
  • the server application includes or is linked to a utility that enables one or more users or observers to annotate the content.
  • the annotation of the content provides context therefore, such context for example enabling interaction between community members in connection with the content.
  • This annotation functionality is an aspect of the server application that enables community members authorized for this to associate additional content with particular content.
  • the annotation functionality enables the user originating the content to associate XML formatted descriptions with selective content, and further enables authorized other community members to similarly associate XML formatted descriptions with selected content, thereby providing the framework for interaction between community members in relation to content.
  • the present invention enables a forum wherein users may upload recordings, share recordings and associate comments to the recordings.
  • the server computer 130 enables one or more users to transmit content to the server computer 130 for creation and maintenance of a personal archive, or a group archive.
  • the present invention contemplates including in the server application or linking to the server application a series of tools for supporting the functionality described, including for example improving the performance or efficacy of the system.
  • the server application may include a compression utility that is operable to compress content prior to storage to the database.
  • a security utility may also be included in or linked to the server application so as to provide privacy or security of information.
  • the security utility may provide field level encryption of the content in the database.
  • a resource management utility for optimizing utilization of bandwidth and storage, as further particularized below.
  • the utility is linked to or part of the client application, and enables the functions described below.
  • the resource management utility may be operable to transmit low resolution preview recordings to one or more remote sites.
  • the present invention may be operable to store higher resolution recordings locally in the non- volatile memory of the device.
  • a recording may consist of audiovisual information such as pictures or sound files, or other data recorded from the environment or from the user of the apparatus.
  • the resource management utility is operable adaptively to enable the portable device 150 to transmit the content at a lower resolution; and then to detect when the portable device 150 is operating with improved bandwidth, thereby initiating the transmission of the content previously transmitted in low resolution format via the low bandwidth wireless medium 120, in high resolution format.
  • the system of the present invention includes means for associating the content in the low resolution format with corresponding content in the high resolution format.
  • the server application is operable to replace content in the low resolution format with corresponding content in the high resolution format, once this has been transmitted.
  • content may be stored and accessed in a low resolution format until a request is received from at least one community member or observers 140 to obtain the content in a high resolution format, in which case the high resolution version may be obtained from the portable device 150 on demand as bandwidth is available.
  • any annotation associated with the low resolution version of the content is reproduced in association with the high resolution version of the content, once obtained as depicted in FIG. 2.
  • the cyborglogging community is operable to allow individuals to share their glogging experiences. While glogging records are uploaded to the community servers, observers or the specific individual who posted the recording may attach annotation to the recording. This annotation may take the form of evaluations, comments, or tags on the recording may then be indexed in various ways, as one might do for example in a social networking context, by adding descriptive data to the recording.
  • FIG. 2 depicts a scenario wherein annotation 240 is tagged to a specific record 220 and remains affixed to the specific recording despite the fact that the recording's resolution 210 may increase over time 200 (i.e. when a higher resolution version is uploaded to the community).
  • the time 200 may be indexed using the UNIX Time stamp with the granularity of microseconds. Since the time of recordings can be significant as the record increases in resolution later the time of capture remains associated, while the time of upload is only used for future reference.
  • increased resolution 210 changes the record, each stage of resolution is authenticated as related to each other (i.e. the high resolution record is authenticated as the full resolution version of the transmitted low resolution vice versa).
  • the portable device 150 may store the high resolution records and produce a number of lower intermediate level resolution records for transmission.
  • the associations between the records of varying resolutions may be maintained by transmitting a cryptographic hash function (such as an MD5 hash for example) of each resolution of the records 220 (the low resolution and the higher resolution) at the time of transmission. This allows for the higher resolution version to have their calculated cryptographic hash checked against those transmitted with the initial resolution.
  • a cryptographic hash function such as an MD5 hash for example
  • a message authentication code scheme or digital signatures with public/private keys in place of a cryptographic hash may be used to accomplish the same purpose.
  • an increase in recording resolution can be made evident in a hierarchical evidence history, so that it can be varied that the increase in resolution did not constitute a falsification of the recording as evidence.
  • Such a hierarchical evidence history might be useful in relation to prosecution of criminal activity or in support of a legal or factual argument or finding.
  • a history log file proves the chain of events, so anyone can verify and compare the low quality and high quality records to prove they both depict the same subject matter or event(s). Playback and Display of Recordings
  • the present invention provides several functional categories that are applicable to the notion of playback and the display of recordings such as: Time-scalable viewer, ranking utility, and the storyboard functionality.
  • FIG. 3 depicts a Time-scalable viewer wherein at a high-level overview 300, sparse sets of images in the sequence are viewed, yet at a low-level overview 320 a more temporally dense set of images are viewed centered around the image of interest 330.
  • a time-scalable viewer is associated with the portable device 150 and/or the server computer 130, which viewer 300 and 320 is operable to display the image frames in a multi-frame recording and to provide an adjustable skip factor.
  • this may be a logarithmic skip factor, e.g. the time-axis can adjusted so that every second image frame is shown, or every 4 th , 8 th , 16 th , or the like, image frame is shown.
  • images are skipped by time codes such as day, month, or year rather than frame numbers. It should be further appreciated that images with higher composity 310 may be selected over automated time interval images.
  • the database linked to the server computer 130 may enumerate the user 100's recordings in an ordered list.
  • the "center" image or key-image or reference-image or “now-image” or “cursor-image” or “now-cursor position image” is chosen and its neighbors at a particular skip are determined.
  • forward and back buttons are provided permitting the user 100 to scroll forward or backwards in a temporal fashion.
  • the user 100 may select a recording, while the viewer may be operable to provide zoom buttons (forward/backward) as well.
  • the number of rows and columns may be adjusted to fit the viewer's screen. A single row may be used to represent a timeline.
  • the viewer may be operable to pre-load images of different skip levels to speed up browsing so that when a user is pausing and is viewing an image, the remaining images may be pre-loaded and cached locally in the background.
  • a ranking utility is provided that is operable to enable selected content to be ranked based on significance.
  • the function of the ranking utility is to filter the content based on at least two categories of significance as described below.
  • composition is defined as the degree of compositional effort and intent exerted or exercised by the user in creating the content.
  • the client applications includes a function that enables the detection of composity within defined parameters, thereby enabling categorization of content based on relative significance.
  • FIG. 4 depicts an approximate composity 400 valuation of actions over the time interval 410 as illustrated in a Composity vs. Time graph.
  • automated recordings may be accorded a low composity rating 430, however user initiated recordings may have a high composity rating 440.
  • the signals collected from the sensors unit 180 in FIG. 1 may be used to determine if the user 100 is active or non-active, and thus automatically increase or decrease the composity accordingly as depicted by the composity rating 420.
  • a cyborglog may contain a lifetime's worth of recordings, while many of which are taken automatically without user thought and efforts. Therefore identifying significant records allows a user to later find/reference a particular record more readily (as further expounded upon below).
  • the notion of composity results in the inference that intent at the time of capture, may lead directly to the significance of the recording.
  • a set of sensory data may also be used to determine the composity of the recording. For example, the time interval 420 where the user 100 gives the high heart rate and high acceleration values may be ranked higher because it is more likely that an important or exciting event was happening at the time.
  • Composity itself does not require any more user interaction beyond the taking of the image intentionally.
  • a higher degree of composity is determined by inference from the usage (i.e. users need not direct the invention, while rather the invention infers users' intent) or from the biofeedback of the user.
  • the inference is made using several cues: user initiated capture results in a higher composity than automated capture; user recording at a higher resolution results in a higher composity; automatically captured records, some may capture the scene better than others and these better images (as defined by an image analysis) may have higher composity; numerous records intentionally taken in rapid succession give the record sequence a higher composity as it implies the user was trying to record something of particular importance.
  • the present invention provides a search and/or navigation utility that enables selective searching for or navigation of content having a desired significance factor.
  • the ranking utility may also rely on other criteria for defining relative significance of content such as number of views by others or amount of viewer activity such as comments and taggings. Thus significance may be affected over time by both the sender's measures like composity, and the community's interaction with the content (e.g. by annotation or dialogue in relation to specific content).
  • Another criterion is image rank, which a rating is given to an image by viewers, expressed as an integer - the higher number the greater the rank. Viewers can increase this number by viewing the recording more times.
  • a view may increase an image rank by sharing the image (i.e. send to a friend). Rating an image may increase an image rank.
  • annotating a recording may increase its rank.
  • a record for an event may include a sparse set of records of high composity organized in time allowing for quickly searching for any specific record (e.g. a user knows the event they're searching for and sees that it happened before certain records and after certain records - allowing a person's natural sense of time to guide search).
  • the system prefers high composity recordings when possible, meaning that the set of records presented at each level are of significance to the user and more likely to be recalled in the "memory based" search , or even be those which the user is searching for.
  • first-person (sousveillance) recordings are brought together to describe an event or procedure, which may be storyboarded during the event or procedure.
  • the first person perspective and automated capture of the records means that at the end of performing any task, the record can easily be used as a "How-to" manual.
  • One aspect of the narrative creation is that multiple users can contribute to a single narrative and share a common experience between all gloggers.
  • FIG. 5 depicts an interface for the Glogger Intelligent Navigation System (GIRMS).
  • GIRMS may be implemented on various platform such as Windows, Linux, Mac OSX, or mobile platform such as J2ME and Symbian.
  • the records 51, 53, 54, 55, 58 are the visualization of the recordings captured by the user and stored on the server computer 130 of FIG. 1.
  • a key frame is extracted from each record according to the heuristic index that is computed based on the sensory data streams, i.e. the most representative sequence may be extracted only.
  • the locally cached records may automatically synchronize with the remote server to guarantee that GIRMS always display up-to-date records, i.e. no missing frames or records.
  • the GIRMS may automatically select the most recent records from the database in the server computers 130, or a user may initialize the GIRMS with a filter to get the most relevant information.
  • the processing of the sensory data streams, computation of heuristics index, sorting of the records, and other computational intense operations may be executed locally on the client or user's device or pre-processed on the remote server.
  • the neighborhood records 51, 53, 54 are rendered in 3D space compactly by rotating the frames in the x, y, or z-axis.
  • the perspective projection may be used and therefore the frames may be perceived as a parallelogram in 2D space, i.e. the further the object the smaller it is on screen.
  • the dimension of the frames is also compressed spatially according to the amplitude of the signals from the sensory data such as from an accelerometer, EEG and ECG sensors, etc..
  • the key record 54 is the frame of interest.
  • the navigation buttons 56 and 57 may be used to switch the frame of interest between records that are most relevant to the key record 54.
  • the stack 55 is the visualization of the most relevant records with the key records. If there are no relevant records, the stack 55 will not be shown.
  • the relevancy of records can be determined using factors such as time, space, and correlation of the sensory date, i.e. the closest in time would be the next record captured.
  • the heuristic and reasoning can be expressed as an index by computing the weighted sum or product of one or more data streams, or with any arithmetic operations that is permitted.
  • the button 58 may allow a user to insert sensory data streams to be considered in the heuristic, and may also allow user to remove the stream.
  • the graph 514 is a visualization of the heuristics index generated by the interface.
  • the absolute amplitude of the sinusoid wave in the graph 513 determines the dimension of the frames rendered, and the compression factor of the frames in time.
  • One example scenario may result in a heuristic index is when the user is inactive, sleeping, or resting.
  • the buttons 512 and 515 may be used to skip records where the user is inactive or has relatively low heuristics on the given data stream.
  • the graph 59 illustrates the compression of the frame in the time axis.
  • the frames are skipped more excessively or sampled less frequently over the interval with low heuristic index.
  • the record 58 is compressed spatially and shown in a smaller size compare to other records such as 53.
  • the button 517 may be used to enable or disable to affect of the data stream to the heuristic.
  • the region 513 is a dedicated region for the key record 54 and the one or more sensory data streams of the key record 54.
  • the bar 514 is to represent the exact location of the frame of the record corresponds to the sensory data stream.
  • the playback frame rate of the recordings may be adjusted according to the heuristic index, and the video frames maybe further compressed on the server computer to reduce the bandwidth usages as needed.
  • a user may select any recording as the key record 54 by selecting it, and all behavior of all components in FIG.5 maybe reprogrammed to customize for the needs of user. All records can be annotated and tagged by entering keywords to the field 52.
  • One example of an embodiment of the present invention is a sousveillance system, method, and computer program for assisting the visually impaired by way of a seeing-eye-people collective, personal safety device, and the like.
  • the visually impaired individual may obtain information regarding their personal surroundings via a collective or a community of individuals all sharing their video surveillance logs. This method should be contrasted with computer vision rather it is a manner to allow individuals from the sharing community to view the surveillance logs/glog and then guide the visually impaired individual.
  • the glogger may be equipped with a "remote control" feature wherein a remote user may take control of an individual's glogger device.
  • a visually impaired individual wears a glogger imaging system and receives remote assistance from a collective of one or more helpers.
  • This remote assistance may be audible, for example, by way of a wearable cameraphone with eyeglass-based camera and earphones, so the blind person can hear remote commentary from at least one individual from the collective or the cyborglogging community.
  • this assistance may be tactile by way of a handheld Networked Embedded Sensory System for Intelligence Extender (NESSIE), which consists of a vessel having compressed fluid, such as air, spray out through a plurality of holes. Obstruction of the holes can be sensed with, for example, one or more microphones, so that the NESSIE can feel the touch of the user.
  • the NESSIE can also squirt bursts of air at the user's fingers so the user can feel a message as standard Braille code or as any other desired code, including, possibly, various musical codes.
  • the remote assistance need not always be real-time.
  • a visually impaired individual can record their surroundings and play this information back later or send it later to a seeing-eye-assistant.
  • a visually impaired individual can capture this experience and play it to another individual later in order to receive some knowledge of the art that was experienced earlier that day.
  • a partially sighted person who has enough eyesight to aim a camera, but not enough eyesight to really see clearly what the images are, can aim a wearable cameraphone at subject matter and request remote assistance in identifying more detail in what is seen.
  • many visually impaired individuals can still make out the outlines of people but have trouble recognizing who it is.
  • Such persons can certainly aim a camera at a person and capture a good picture of the person's face, where the face is properly centered in the camera's viewfinder.
  • the visually impaired person then sends the image and asks for remote assistance.
  • the request for remote seeing-assistance can be specifically requested, whereas in other embodiments, the request can be serendipitous.
  • a serendipitous request can work in the context of a social networking site, where a visually impaired person might, for example, share their visual world in the hope that others will happen to participate serendipitously.
  • a remote assistant might signal back with some commentary that either directly helps them see or understand visually their present surroundings.
  • the present invention may also be used to function as a personal safety device.
  • a safe watch program in which a person walking to or from their home or place of work late at night can request one or more remote escorts, or can simply make their online presence known to their community so as to attract a high likelihood of serendipitously participating remote escorts, who may participate as if in a safe-walk program, to report any unusual or suspicious occurrences.
  • the cameraphone device is contained in a shock- resistant/water-resistant enclosure. Further the cameraphone may be brightly colored in appearance as its enhanced visibility may serve as a deterrent. Further, the device may not be readily opened or stopped so that the user cannot turn it off despite an assailant demanding it.
  • the safe watch request for one or more remote escorts this may enacted in a deliberate mode or it may be serendipitous, i.e. a person who is a glogger may ask a friend to help them get safely to a particular destination, or, alternatively, a glogger may put out a "softer" request to multiple observers, with a likelihood of remotely being involved in their life.
  • the system does not necessarily require participation of a specific escort, especially if it can be made known (e.g. by a way of a flashing red light, or the like) to potential assailants that a glogger's audiovisual field of view is being transmitted.
  • the cameraphone device operable to function within the ambit of a cyborglogging community may serve as a deterrent against both individual terrorism and state terrorism, i.e. against both unaffiliated street thugs, as well as police brutality.
  • another useful feature of the present invention is a "personal duress delete" capability.
  • one or more images can be made to appear to be deleted but this personal duress delete function will flag the transmitted images for archival, and also notify one or more remote entities that the user is under duress.
  • an evidence file that provides a true and accurate representation of what an individual experiences, is transmitted live in real-time to a server where the archive can be immediately viewed by large numbers of people, sucti that it would be difficult to falsify the record.
  • this evidence file or sousveillance recording may be distributed in a unicast or a multicast manner (i.e. to be stored in multiple disparate locations) so that it would be more difficult for a small number of organizations to conspire to falsify it.
  • Peer-to-Peer The present invention provides for two levels of record sharing:
  • the client uses their body borne device determines its preferred viewing resolution (compression quality, encoding type, and size for example).
  • the client who is making a request to view a record, transmits their desired viewing resolution to the server along with a request for the record itself.
  • the record may be identified by a unique time prefix and associated user who transmitted the original version.
  • the server transcodes the record to the desired viewing resolution adaptively based on the bandwidth, screen size and other criteria on the device.
  • the server fulfills the client's request and sends the requested record. Any associated annotation and/or comments may additionally be sent with the reply or in another query.
  • a separate HTTP query may be used to determine the available records for browsing for a time period, for a user (for example, records form a particular user taken in the last hour).
  • the second mode of sharing permits records to be shared between user devices on a peer-to-peer basis.
  • a peer-to-peer architecture provides a client application at each participating device, and a wireless network (by means of an associated network management server) connected to each device that is operable to enable a connection between each participating device, and thereafter the client application at each participating device is operable to enable peer-to-peer content exchange, as well as interaction with the content as previously described.
  • BLUETOOTHTM technology e.g. by means of associated service discovery protocol (SDP) and OBEX (Object Exchange) protocols.
  • SDP service discovery protocol
  • OBEX Object Exchange
  • the actual sharing may be achieved via the client application running on the cyborglogging device (i.e. cameraphone).
  • the unique 48-bit BLUETOOTH address of client devices is registered with the server and used to identify peers clients can request user information about peers by detecting nearby BLUETOOTH addresses and querying the server (i.e. determine which users are nearby by detecting and looking up the addresses on the server). Once a nearby device has been detected, the user's device may consult a cached list of addresses to determine if the detected device is in the "sousveillance" community.
  • If not user's device may consult with the server if possible sending the 48-bit address to the server and receiving a community username of the corresponding user if present. If the BLUETOOTH address is a recognized user, one may initiate a connection, requesting a list of records, permission to send/share a record, request a particular record.
  • Observers of the sousveillance recordings may flag certain images as inappropriate.
  • each observer preferably develops an observer reputation index depending on how long they've been a reliable observer, as well as on past consensus with their observer ratings and any final determinations. If inappropriateness flags come from long-time contributing observers, they are weighted higher than inappropriateness flags from new observers. Inappropriateness flags and various other flags can also be weighted by reputation capital, etc., of each observer, total weightings computed accordingly.
  • Cyborglogging lets users both sending records and observe rating increases with community participation. Each record that is uploaded increases a participant's rating. Each page that is viewed increases the sender's rating. Each comment increases a participant's rating. Uploaded records which subsequently create commentary in the community increase the sender's rating. Records deleted due to inappropriateness decrease the sender's rating.
  • a voting system a number of people may be required to vote for deletion, wherein the first vote is a nomination of the image for deletion and commencement of the vote. If the vote fails to delete the image, the rating of the nominating observer may be decreased.
  • an administrator of the system can grant or remove a user's moderator access. User may be suspended by the administrator or multiple moderators if excessive number of inappropriate records is uploaded or sender's rating is below a threshold. Only users with high rating are allowed for displaying the recording on the front page.
  • One embodiment of the present invention includes a continuous scale for granting "powers" from new user to moderator including the ability to appear on the front community page, ability to flag images for deletion, ability to comment on images, to delete comments, to share (via emails or social networking) from site to friends, for granting/revoking these abilities to others as well.
  • a method of doing insurance business includes the steps of supplying each at-risk party with a glogging device, and providing incentives in the way of reduced insurance premiums for each party to send live video of their lives to a central evidence log.
  • an insurance company may provide the use of their call center to assist persons in personal safety.
  • a person may wear a camera and transmitter and also wear an output device or display.
  • the display may be a video display or an audio display.
  • the method involves one or more parties wearing a PSD (Personal Safety Device) comprising at least a sousveillance camera, and possibly other sensors.
  • PSD Personal Safety Device
  • the party may also have a panic button or other panic actuator such as speech recognition or EEG distress activator that automatically or manually activates on brainwave control.
  • the panic activator may comprise heart rate sensor, or more generally ECG waveform sensor.
  • the PSD when activated, notifies a call center to pay particular attention to a person who might be in distress.
  • the call center can take the form of a Safety Management Organization (SMO) similar to a Health Management Organization (HMO), but broader in context to include not just health care but also preventative safety such as perpetrator management.
  • SMO Safety Management Organization
  • HMO Health Management Organization
  • the method of doing business can include perpetrator management. For example, it has been noted that persons with special needs are often abused or threatened. There have even been cases of deaf people being shot dead by police because they didn't respond to a policeman's spoken order.
  • a person might receive remote advice during a tough situation, e.g. a victim of a school bully could stream live video and ask for remote advice on how to handle the situation.
  • the party may have one or more displays, such as a first-person display (for the party), and a second-person display (for the perpetrator to see or hear, or otherwise perceive).
  • the party in actuating a distress call to the call center or SMO gets someone to look through their eyes or perceive their world, and that someone can then manage the perpetrator remotely, for example, by making the perpetrator aware that the situation is being remotely monitored and recorded in such a way that the perpetrator cannot destroy the evidence even if the perpetrator steals the device itself.
  • Perpetrator management aspects of the invention are operated as follows: victim wearing a PSD actuates distress signal; SMO assesses situation and makes decision on action; SMO can communicate with perpetrator to assist in perpetrator management.
  • the display may consist of, for example, a small neckworn or chestworn video display with a red flashing LED or other light source and this may display the face and voice of an SMO employee speaking to the perpetrator.
  • the display need not be limited to audiovisuals such as flashing lights or body- worn sirens or speech or video display.
  • the display may be, or include, a taser or pepper spray discharge device remotely controlled by staff at the SMO.
  • the display may also be, or include, an obscurant such as a release of dense smoke.
  • the system includes a peer-to-peer replacement or addendum to the SMO, in which individuals can rely on their own network of friends to help. For example, if a person is being assaulted by an abusive teacher or security guard, he or she can summon help from friends. Additionally, if desired, those friends can make their remote-link presence known to the perpetrator.
  • virtual cyberfriends or avatars can join in.
  • a "talking head" or recording or pre-recorded perpetrator management script can start at the push of a panic button. In this way the perpetrator might not know whether the depicted conference screen face is real or not. Simply having a recorded perpetrator management script may suffice for purposes of providing a personal safety program.
  • a chestworn video capture device is combined with recording of electrocardiogram (ECG), along with a system to automatically recognize anomalities in ECG waveform and tag, timestamp, etc., corresponding video.
  • ECG electrocardiogram
  • a PTPsend Physical Transfer Protocol, as defined in Chapter 3 Comparametric Transforms for Transmitting Eye Tap Video with Picture Transfer Protocol (PTP) W. Steve G. Mann University of Toronto, in the book: The Transform and Data Compression Handbook * ).
  • the invention can be used to capture evidence that can be viewed by a court of law, while at the same time protecting people's privacy. This can be done with a sight license system that uses file formats, protocols, and software systems to ensure that nobody can see the image data unless authorized to do so (e.g. by court order), or that only certain groups of people can see the data.
  • a sight license system that uses file formats, protocols, and software systems to ensure that nobody can see the image data unless authorized to do so (e.g. by court order), or that only certain groups of people can see the data.
  • the video corresponds to review of Confidential Information like this patent application before it becomes public, the video may be viewable only by a group of people consisting of the inventors, patent lawyer, and the like.
  • there may be certain semi-private spaces such as spaces that are separate for men and women.
  • FIG. 6 depicts a user 601 who is wearing a capturing device 602 and transmitting content via a wireless medium 609 to user 603 who is in proximity or line-of-sight to each other. This can be done by line-of-sight infrared transmission or the like. This may be done only in the field-of-view of the camera using an aremac that has the same field of broadcast as the camera, so that only people who are captured by the camera can receive the image broadcast from the camera initially.
  • FIG. 6 depicts a user 601 who is wearing a capturing device 602 and transmitting content via a wireless medium 609 to user 603 who is in proximity or line-of-sight to each other. This can be done by line-of-sight infrared transmission or the like. This may be done only in the field-of-view of the camera using an aremac that has the same field of broadcast as the camera, so that only people who are captured by the camera can receive the image broadcast from the camera initially.
  • FIG. 6 depicts a user 601
  • the bar 608 illustrates that user 605 is either not in line-of-sight due to obstruction or out-of-sight of user 601, and therefore does not receive the content directly from user 601.
  • this method of content sharing is called subject transmitience or subject-mirroring, or subject-equiveillance.
  • the aremac is well known in the art, as described in "Intelligent Image Processing” by S. Mann, publisher John Wiley and Sons, 2001. Equiveillance is also well known in the literature. Subject mirroring allows people to see themselves in the "mirror" of the camera, i.e. as if the camera were a mirror.
  • the image may be posted immediately to a public website via the Sight Licensing System 613.
  • User 603 can also Privatize the image, i.e. select that the image be saved in a private repository.
  • User 603 can also Accept an image in a single use or global sense, including the setting of profile messages and default picture-meta-messaging.
  • Picture meta messaging is picture messaging of or for meta data, tags, qualifiers, and the like.
  • Sight Licenses can be autonomous agents, such as a wearable system that receives images and automatically assigns a license category.
  • a necklace-worn license indicator can sense whether or not the wearer is clothed, and, when unclothed, automatically select a different Sight License preference profile.
  • a user can program the system with such simple commands as "Allow pictures of me when clothed, and dis-allow pictures of me when I'm naked, unless it is detected that I am unconscious (slipped and fell in the shower for example) or drowning (in the bath, for example)".
  • a user can delegate the decision to a third party, i.e. "Send pictures of me to my agent (such as another human at a remote location) to decide on their disposition”.
  • a person capturing a continuous unbroken record for personal safety and evidence may want integrity of the evidence, but may also want to choose to abide by preferences of those in the environment regarding dissemination of image data.
  • the Sight License is implemented by the steps of: (l)capture of image data; (2)sending image data over a field-of-broadcast that is similar to the field-of-view of a camera that captured the image; (3)enabling those in the f ⁇ eld-of-brpadcast to view one or more images and comment or annotate the images.
  • Sight License may allow capture of images that can't be looked at until a License to Look is obtained.
  • Sight License and Look License can be separately treated by the invention, and different rules for each can be created.
  • Sight License refers to the camera itself, what it can capture, and how that data is disseminated, etc., which may be in encrypted form.
  • the Look License allows a separate decryption key.
  • the decryption key may also be partial, such as a decryption up to a certain resolution depending on the Look License. For example, low-resolution images may show that someone has slipped and fallen in a shower room or is drowning in a bath, but not sufficient resolution as to identify the person or private aspects of the person's body.
  • the invention can capture images in varying resolutions over time and synchronize later. For example, low resolutions are captured immediately and sent, timestamped and encrypted for forensics, broadcast widely so they can't be falsified.
  • a video record of a person's entire life may be captured and low resolution versions of it published while keeping higher resolution private. In this way the person who has the recording can't falsify it and get away undetected because low resolution versions have already been broadcast.
  • a steady low-resolution cyborglog may be sent over low bandwidth cellular phone channels, followed by high resolution sync-up when later a WiFi node is found, or automatically synchronized when later the system is docked to fiber optic link.
  • a rating system can be used to allocate Sight Licensure through ranking. If enough people vote to disclose full or greater resolution of a crime against humanity, it may over-ride national security or privacy interests. In this way the Sight License system gives the possibility of balancing the needs of safety and evidence with the alleged needs of privacy or secrecy.
  • the Sight License also supports subjectrights, such as the right of a subject of veillance to have access to the data of the veillance whether it is surveillance or sousveillance. This is done by way of the aremac broadcast or similar radio directional antenna broadcast that functions like a radio-frequency aremac.
  • Sight License applies to sousveillance as well as to surveillance and may be implemented in surveillance (architecturally mounted cameras) or sousveillance (wearable cameras) or both in conjunction with one another.
  • the camera-mirror or aremac is not the only way to have a Sight License. It may also happen by voting, ratings, or the like, online, with arbitration.
  • an online Sight License uses the steps as follows:
  • this system can operate in a multi-multi-scale fashion, i.e. each of various time- resolution hierarchies can be created for different audiences. For example, jury members of a court may see a medium-resolution video of a slip-and-fall incident or a fight in a shower room, whereas the general public might see only a low-resolution version.
  • the low resolution version might be shown as a news item, to help solicit anonymous crime tips from people who might have been present at the crime scene or might know something about it.
  • the resolution might be increased to a medium level for jury members and possibly to full resolution for a judge of the same geniter as the persons in the shower room.
  • the invention enables people to augment their bodies and mediate their perception, to create one or more time-series of data, plugging into their own senses, etc., to interact and communicate, with capability to implement cyborgspace that is present in their environment all the time, if desired.
  • Much of social networking has the effect of reversing the Hegelian dialectic in which people lose their civilization through, for, example, the historian who tries (and fails at) being objective. All users of the cyborglogging system can become objective. For example, a lifelong capture of situations and responses can be used to help reduce terrorism.
  • the invention can also be used by a person wishing to capture as much as can be known about their own lives as well as the way in which they respond to their own lives. For example, by capturing physiological response to image data (brainwaves, etc.), future generations can learn a great deal about a person and their life (perhaps even enough to re-construct a computerized intelligent agent version of that person after they die).

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Computer Security & Cryptography (AREA)
  • Multimedia (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Theoretical Computer Science (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The present invention relates to a system, method and computer program for capturing, sharing, and annotating content. Specifically the invention relates to a system for capturing sharing and annotating content characterized in that the system comprises at least one content capture device operable to capture content and transmit the content and the system comprises at least one remote computer operable to receive the transmitted content and the at least one remote computer includes or is linked to a remotely accessible database to assemble and store the content. In addition the invention relates to a method for capturing sharing and annotating content characterized by the steps of capturing content using a content capturing device; transmitting the content to a server application residing on one or more remote computers; assembling and storing the content on a database and enabling one or more users to access the content and annotate the content.

Description

SYSTEM. METHOD AND COMPUTER PROGRAM FOR CAPTURING, SHARING,
AND ANNOTATING CONTENT
FIELD OF THE INVENTION
This invention relates to a system, method and computer program for capturing, sharing and annotating content.
BACKGROUND OF THE INVENTION
Surveillance as taken from its French roots literally means "to watch from above". Research expounding on this topic is well established and well known. The practice of surveillance has become widespread. Originally confined to high-crime areas, its scope has become ubiquitous. For example, installing surveillance cameras in the rough and crime- filled East side of town was found to "push crime Westward". Thus cameras were installed in both the East and West sides of town. This made it more difficult to perpetrate street-level crime without getting caught.
Installing surveillance cameras everywhere means that police are watching citizens, shopkeepers are watching shoppers, taxicab drivers are watching passengers, etc. Thus, those on the bottommost rung of the hierarchy are under close scrutiny, whereas those on the next rung, e.g. police, shopkeepers, and taxicab drivers, may escape scrutiny.
Similar to the aforementioned example, when cameras were only installed in the East end, (crime was driven Westward), with surveillance everywhere, crime may be driven Upward. If it is difficult to commit petty crimes such as pick pocketing and shoplifting, the opportunities shift to police officers, shopkeepers, and others at the next-level. Oversight committees may minimize this situation, i.e. where employees such as officers are watched by someone further up the ladder. But oversight can also shift crime even further up the ladder, i.e. toward higher-level corruption.
This is not to say that surveillance is the cause of corruption, anymore than saying that installing cameras in the East end causes crime in the West end, but simply to say that surveillance provides an incomplete picture that leaves room for corruption, and that this incompleteness cannot be fixed with oversight alone.
In summary, we need some way to guard against the possibility that surveillance may push crime up into the middle rungs of the "ladder of life", and oversight may push crime up to the top rungs.
More recently, the notions of "inverse surveillance" have been proposed as a way of balancing an otherwise one-sided "surveillance-only" society. Inverse surveillance might, for example, include citizens photographing police misconduct, shoppers photographing shopkeepers, and passengers photographing reckless cab drivers from within the very cab that might, for example, be involved in an automobile accident.
An important aspect of inverse surveillance is that it emanates from individuals recording their personal experience and their immediate vicinity, rather than the recording/monitoring of individuals by an outside party.
The term "sousveillance" has been used to describe this recording of an activity by a participant in the activity. Sousveillance includes both "inverse surveillance" (i.e. citizens photographing police, shoppers photographing shopkeepers, etc.), as well as passive capture of everything in the environment (e.g. a person simply recording their entire life for no particular specified reason). Whereas "surveillance" means "to watch from above", the term "sousveillance" shares French roots in that it is the composite of the French word "sous", meaning below and the French word "veiller", meaning to watch.
Sousveillance involves the creating recordings, including of audio or video content, as well as other captured signals such as can be measured from a person's body and their environment. In this disclosure "recordings" refers to audio and video recordings, as well as other signals that can be recorded, and "content" refers to audio or video or other measurable or captureable data. In addition, "wireless medium" refers to any form of wireless connection which allows exchanges of data between devices or systems. Some example wireless networks and protocol used in this invention include Global System for Mobile communications (GSM), Universal Mobile
Telecommunications System (UMTS), Wi-Fi (IEEE 802.11), and Bluetooth for short-range data transmission.
With the development of wearable personal electronics, such as smart phones or personal digital assistants (PDAs), it has become increasingly easier to capture and share our personal experiences with others.
Cyborglogging (glogging) is a concept that involves the capture of one's own life on a continuous basis. Applications may be personal safety, health monitoring, or intelligent wearable computer systems that create interactive personal assistants for persons with special needs, or the like. See for example, "Wearable Computing, A first step toward personal imaging", IEEE Computer, 30(2): 25-32, February 1997 (feature article), as well as "Intelligent Signal Processing", Proceedings of the IEEE, 86(11), cover+pp2123-2151, 1998.
The cyborglog or "glog" (CyborGLOG) is the successor of the weblog or "blog" (WeBLOG).
Blogging has evolved more quickly than glogging, but the technology necessary for glogging is becoming more prevalent, and thus glogging is entering into the mainstream. Personal experiences have been shared making use of an online diary weblog or blog. Specifically in the context of the mobile phone, individuals have been updating web logs to reflect mobile terminal activity (see US Patent 7,069,003). In addition, individuals have been making use of wireless cameras or camera phones for the taking of digital pictures and then sending those pictures or "messages" to a remote system via a wireless network (see US Patent 7, 173,651 ). Glogging works similarly but often in a more automated fashion using varied signal processing techniques.
Sousveillance, as a form of glogging, may also be helpful in reducing terrorism. Terrorism originated in France, during the "Reign of Terror" (1793 September 5 to 1794 July 28). Originally the term "Terrorism" was used to denote a government's actions against its own people in order to create fear (Terror) as a way of keeping the people docile or obedient. [http://en.wikipedia.org/wiki/Reign_of_Terror]
The world's first terrorist organization was the French government's Committee Of Public Safety (C.O.P.S.). [http://en.wikipedia.org/wiki/Committee of Public Safety]
One problem with surveillance is its inherent conflict-of-interest, especially with regards to terrorism in its original form.
SUMMARY OF INVENTION
The present invention provides a system for capturing sharing and annotating content characterized in that the system comprises at least one content capture device operable to capture content on a continuous basis and transmit the content and the system comprises at least one remote computer operable to receive the transmitted content and the at least one remote computer includes or is linked to a remotely accessible database to assemble and store the content.
A further aspect of the present invention provides a method for capturing sharing and annotating content characterized by the steps of capturing content using a client application residing on a content capturing device; transmitting the content from the client application to a server application residing on one or more remote computers; assembling and storing the content on a database associated with the server application and enabling one or more users or observers to access the content stored within the database and annotate the content.
In addition to capture, and sharing, there are also new forms of information recall, navigation, and interaction, disclosed by way of the drawings and description to follow.
BRIEF DESCRIPTION OF FIGURES
FIG. 1 depicts an individual wearing a head mounted embodiment of the present invention transmitting recordings to one or more servers.
FIG. 2 depicts annotations being associated with recordings.
FIG. 3 depicts a Time-scalable viewer.
FIG. 4 depicts a Composity vs. Time graph. FIG. 5 depicts an interface for the Glogger Intelligent Navigation System.
FIG. 6 depicts Sight License aspects of the invention.
DETAILED DESCRIPTION
FIG. 1 depicts a user 100 affixed with a portable device 150 which is mounted or implanted in the body of user 100. The portable device 150 is capable of capturing images, videos or audio and transmitting its recordings to one or more server computers 130. The portable device 150 may be in the form of a wearable cameraphone, a wearable computer, or any handheld device with capturing capability. In addition, the sensors unit 180 is a device with a collection of sensors that collects signals from user 100 and transmits the post-processed signals back to the portable device 150. Remote observers 140 may observe and annotate on the recordings user 100 or user 101 transmitted to one or more server computers 130. The wireless mediums 110, 120, 160, 161, 191 can allow the portable device 150 and 151 to exchange content with server computers 130 and remote observers 140. The high bandwidth wireless medium (e.g. 3 G or Wi- Fi network) 120 may be used to transmit both high and low resolution content. In one aspect of the invention, a low bandwidth version of one or more images are sent initially and a higher bandwidth version is sent later when bandwidth becomes available. The low resolution version can be broadcast or certified or escrowed by an escrow-stamper, or escrow-broadcast to help insure the integrity of the corresponding higher resolution version that comes later. The higher resolution images can be assigned a similar or identical or related identifier or meta tags, or enter into a database to replace or update the originals, but with a way to verify integrity by going back to escrow- stamped originals. The portable devices 150 and 151 may have different form factor and may run on different operating systems, but both devices may connect and exchange data with each other using the wireless medium 160.
In one aspect of the present invention, a system, method, and computer program is provided that enables a portable device 150 or 151 to operate in "sousveillance mode" to create a recording of the experiences of an individual or one or more objects or other individuals in the individual's vicinity. In another aspect of the invention, the portable device is linked to one or more servers 130 that are operable to enable a plurality of users (also referred to as community members) to access a web hosted community such that the shared content is the object of interaction between community members or remote observers 140.
In another aspect of the invention, the portable device 150 consists of at least one body-borne (hand-held, portable, wearable, implantable, or the like) image capture device, a body borne computational device, means for transmission of one or more images, video, audio, or other data from the capture device, and means for creation of a database records on one or more server computers 130. The portable device, in order to enable the community function described below, may include a display or any form of visual unit to enable users to view images or other data stored locally on the portable device.
The portable device 150 may be worn (e.g. a WearTel™ wearable camera phone) or hand-held.
The portable device 150 may be connected through a wireless medium 160 to at least one other portable device. The wireless medium may transcend time or space, i.e. the goal may be storage and retrieval later (sending to another time) or it may be sending to another place. Each portable device includes or may be linked to a client application in accordance with the present invention that enables sharing of content, including in accordance with the peer-to-peer implementation described below.
"Each" may denote an object in four-dimensional (x, y, z, time) space, i.e. one party of the communication may be a person at one point in time, and the other party may be the same person at a later point in time. In this case, the communication is across time, and may not necessarily need to be across space (i.e. personal visual memory prosthetic to help a person in wayfϊnding, by communicating with themselves at a later point in time).
In another aspect of the invention, the portable device 150 is connected through a wireless medium 110, 120, or 190 to at least one server computer 130. The portable device 150 includes or is linked to a client application that is operable to enable the portable device 150 to capture content, whether a series of files or a data stream, and transmit this to the server computer via the Internet, by means for example of a wireless gateway between the wireless medium and the Internet. The server computers 130 include at least one database and a server application that enables the functions of the server computer described below. In accordance with the present invention, the portable device 150 is operable to transmit the series of files or data stream to one or more server computers 130, the server computers 130 being operable to assemble the files or the data stream to provide a "cyborglog" (as further described below), the server computer 130 being further operable to enable one or more users and remote observers 140 to interact with the cyborglog, thereby providing the sousveillance functionality described herein.
In one aspect of the invention, the server application in 130 includes a database management utility linked to the database that enables content associated with a plurality of community members to be stored to the database, and accessed or shared, in accordance with parameters further described below. In one aspect of the database management utility, it provides hierarchical access, security controls, and/or privacy controls related to access or sharing of the content.
The server computers 130 are best understood as including a series of hardware and software utilities generally associated with online environments for storing, accessing and sharing content, modified to enable the functions described herein. In another aspect of the invention, the hardware and software utilities are those generally associated with enabling community based online environment, again modified to enable the specific community functions described herein such as annotating the content described.
In one aspect of the present invention, the system, method, and computer program provides continuous capture of content for sousveillance purposes. In a further aspect of the invention, the system, method, and computer program enables the creation of one or more sousveillance recordings that include all or part of a person's life either without the need for a user's thought or effort or with minimum user intervention through intelligence signal processing on the signal collected by the sensors unit 180. At the same time, the system allows the user to optionally intervene in this recording, including by selectively determining content to be captured and/or transmitted. Content may also be tagged based on user intervention as described below.
The portable device 150 may be a body borne device, i.e. a personal device that may be perceived as clothing or personal technology that may be unobtrusive when capturing and transmitting video or audio signals.
One implication of use of a body borne device is that it may pass without notice whereas a device that is not body borne such as a bag or a camcorder may not be allowed into certain areas, thereby limiting or interrupting continuous capture of content.
In addition, a body borne device in the first person perspective may provide a casual capture perspective that ties in to the overall notion of "sousveillance" as opposed to surveillance (personal vs. archicentric). Sousveillance resituates the locus of control of the collected data to the person, as opposed to the organization recording. The sousveillance may take the form of a single log that captures a person's lifelong personal experience.
In one embodiment of the present invention, the architecture that implements the invention is a client/server application. The client application running on the one or more image capture devices, and the server application running on the one or more server computers 130.
Building on the concept of the cyborglog, in one aspect of the invention a computer program is provided that may be implemented on a J2ME and Symbian platform and may run on the portable device 130. The "cyborglogger" application has been successfully employed on Nokia Series 60v2 , Sony Ericsson , and Motorola Linux phones. Further, J2ME cellular phones that support the MIDP 2.0 and CLDC 1.0 profiles as well as JSR-135 (Mobile Media API) should be able to run the "cyborglogger" application. Any other portable wireless device with resources suitable for content capture as described may be used. When the client "cyborglogger" application is first launched on the portable device 130 such as a cameraphone, the application automatically prompts the registration of a new glogger account. In one embodiment, the application form has three fields: username, password and email address. Once registered, the user will receive confirmation information at the client provided email address together with the URL of the glogger space created for the user in the form of http://serverurl/username where username is the field provided by the user previously in the registration process, and serverurl is the domain to the main server.
Once registered, the user 100 will be brought to the main menu of the glogger application. The main menu may provide numerous community functions such as a community page, an album, a friends listing, an inbox, an image recorder, a video recorder, a live events folder, a remote controller on/off option, an application settings option, and exit.
As indicated, the user 100 or 101 running a "cyborglogger" equipped portable device 150 or 151 may upload their recordings to one or more servers 130. The uploading may be accomplished using multimedia message service (MMS), email or a standard HTTP POST through the client application or through the built-in browser on the phone.
It should be understood that the present invention contemplates continuous capture, i.e. the application continuously uploads the image streams to one or more server computers 130 without user intervention, but also provides for selective content capture based on user defined parameters. For example, to upload an image, the user 100 may first capture an image by using a capture button or equivalent. The capture button may take a JPEG/PNG/GIF/TIFF format snapshot or a MPEG1/MPEG2/MP4/3GP/3GPP/WMV/AVI/MOV format video and store it in a local memory buffer. In addition, the device may generate a corresponding thumbnail of the recording for display on the display of the device. If user 100 is not satisfied with the recording, user 100 may remove it from local memory by selecting a delete option or equivalent. However, if the user 100 satisfied with the recording, user 100 may select upload which uploads the recording from the local memory buffer through the use of a HTTP POST request to the server computer via the wireless medium on the device. As discussed, the user 100 may upload the recording using MMS, email or other equivalent protocol, by submitting the recording as an attachment to the unique glogger email account in this form: i.username-ρassword@serverdomain. In addition, users 100 or 101 may directly upload recording using a web browser, and access to the content directly via HTTP GET or POST requests.
In one embodiment of the server computer 130, the server application implements HTTP protocols, and may be implemented using PHP and MySQL, or any languages which are capable of performing similar functions.
In one aspect of the invention, the server application includes or is linked to a utility that enables one or more users or observers to annotate the content. The annotation of the content provides context therefore, such context for example enabling interaction between community members in connection with the content. One implementation of this annotation functionality is an aspect of the server application that enables community members authorized for this to associate additional content with particular content. For example the annotation functionality enables the user originating the content to associate XML formatted descriptions with selective content, and further enables authorized other community members to similarly associate XML formatted descriptions with selected content, thereby providing the framework for interaction between community members in relation to content.
It should be understood that the present invention enables a forum wherein users may upload recordings, share recordings and associate comments to the recordings.
In addition, the server computer 130 enables one or more users to transmit content to the server computer 130 for creation and maintenance of a personal archive, or a group archive. The present invention contemplates including in the server application or linking to the server application a series of tools for supporting the functionality described, including for example improving the performance or efficacy of the system. For example, the server application may include a compression utility that is operable to compress content prior to storage to the database. A security utility may also be included in or linked to the server application so as to provide privacy or security of information. For example, the security utility may provide field level encryption of the content in the database.
Recording and Transmitting
In one aspect of the present invention, a resource management utility is provided for optimizing utilization of bandwidth and storage, as further particularized below. In one aspect of the resource management utility, the utility is linked to or part of the client application, and enables the functions described below. In terms of transmitting captured video content, the resource management utility may be operable to transmit low resolution preview recordings to one or more remote sites. In addition, the present invention may be operable to store higher resolution recordings locally in the non- volatile memory of the device. A recording may consist of audiovisual information such as pictures or sound files, or other data recorded from the environment or from the user of the apparatus.
In one aspect of the invention, it is desirable to capture content at the server computer in a high resolution format. In some situations, however, the glogging individual's cameraphone device may be operating with limited bandwidth (for example because of network conditions), thereby preventing real time capture of the content in a high resolution format as shown in 110 and 120. In such a situation, the resource management utility is operable adaptively to enable the portable device 150 to transmit the content at a lower resolution; and then to detect when the portable device 150 is operating with improved bandwidth, thereby initiating the transmission of the content previously transmitted in low resolution format via the low bandwidth wireless medium 120, in high resolution format. The system of the present invention includes means for associating the content in the low resolution format with corresponding content in the high resolution format. In one aspect the invention, the server application is operable to replace content in the low resolution format with corresponding content in the high resolution format, once this has been transmitted.
In another aspect of the invention, for the purposes of resource management at the server computers 130, content may be stored and accessed in a low resolution format until a request is received from at least one community member or observers 140 to obtain the content in a high resolution format, in which case the high resolution version may be obtained from the portable device 150 on demand as bandwidth is available.
In one aspect of the present invention, any annotation associated with the low resolution version of the content is reproduced in association with the high resolution version of the content, once obtained as depicted in FIG. 2.
Annotating a Recording
The cyborglogging community is operable to allow individuals to share their glogging experiences. While glogging records are uploaded to the community servers, observers or the specific individual who posted the recording may attach annotation to the recording. This annotation may take the form of evaluations, comments, or tags on the recording may then be indexed in various ways, as one might do for example in a social networking context, by adding descriptive data to the recording.
FIG. 2 depicts a scenario wherein annotation 240 is tagged to a specific record 220 and remains affixed to the specific recording despite the fact that the recording's resolution 210 may increase over time 200 (i.e. when a higher resolution version is uploaded to the community). The time 200 may be indexed using the UNIX Time stamp with the granularity of microseconds. Since the time of recordings can be significant as the record increases in resolution later the time of capture remains associated, while the time of upload is only used for future reference. In addition, increased resolution 210 changes the record, each stage of resolution is authenticated as related to each other (i.e. the high resolution record is authenticated as the full resolution version of the transmitted low resolution vice versa).
Further, the portable device 150 may store the high resolution records and produce a number of lower intermediate level resolution records for transmission. The associations between the records of varying resolutions may be maintained by transmitting a cryptographic hash function (such as an MD5 hash for example) of each resolution of the records 220 (the low resolution and the higher resolution) at the time of transmission. This allows for the higher resolution version to have their calculated cryptographic hash checked against those transmitted with the initial resolution.
In subsequent implementations a message authentication code scheme or digital signatures with public/private keys in place of a cryptographic hash may be used to accomplish the same purpose.
Resolution Archival
If desired, an increase in recording resolution can be made evident in a hierarchical evidence history, so that it can be varied that the increase in resolution did not constitute a falsification of the recording as evidence. Such a hierarchical evidence history might be useful in relation to prosecution of criminal activity or in support of a legal or factual argument or finding. A history log file proves the chain of events, so anyone can verify and compare the low quality and high quality records to prove they both depict the same subject matter or event(s). Playback and Display of Recordings
The present invention provides several functional categories that are applicable to the notion of playback and the display of recordings such as: Time-scalable viewer, ranking utility, and the storyboard functionality.
1) Time-scalable viewer
FIG. 3 depicts a Time-scalable viewer wherein at a high-level overview 300, sparse sets of images in the sequence are viewed, yet at a low-level overview 320 a more temporally dense set of images are viewed centered around the image of interest 330.
In another aspect of the present invention, a time-scalable viewer is associated with the portable device 150 and/or the server computer 130, which viewer 300 and 320 is operable to display the image frames in a multi-frame recording and to provide an adjustable skip factor. In some embodiments of the invention, this may be a logarithmic skip factor, e.g. the time-axis can adjusted so that every second image frame is shown, or every 4th, 8th, 16th, or the like, image frame is shown. In another variation of this theme, images are skipped by time codes such as day, month, or year rather than frame numbers. It should be further appreciated that images with higher composity 310 may be selected over automated time interval images.
Further the database linked to the server computer 130 may enumerate the user 100's recordings in an ordered list. The "center" image or key-image or reference-image or "now-image" or "cursor-image" or "now-cursor position image" is chosen and its neighbors at a particular skip are determined. In addition, forward and back buttons are provided permitting the user 100 to scroll forward or backwards in a temporal fashion. In another embodiment, the user 100 may select a recording, while the viewer may be operable to provide zoom buttons (forward/backward) as well. The number of rows and columns may be adjusted to fit the viewer's screen. A single row may be used to represent a timeline. The viewer may be operable to pre-load images of different skip levels to speed up browsing so that when a user is pausing and is viewing an image, the remaining images may be pre-loaded and cached locally in the background.
2) Ranking Utility
In one aspect of the present invention, a ranking utility is provided that is operable to enable selected content to be ranked based on significance. The function of the ranking utility is to filter the content based on at least two categories of significance as described below.
One criterion for defining relative significance of content is "composity". Composity is defined as the degree of compositional effort and intent exerted or exercised by the user in creating the content. In one aspect of the invention, the client applications includes a function that enables the detection of composity within defined parameters, thereby enabling categorization of content based on relative significance.
FIG. 4 depicts an approximate composity 400 valuation of actions over the time interval 410 as illustrated in a Composity vs. Time graph. As indicated by FIG. 4, automated recordings may be accorded a low composity rating 430, however user initiated recordings may have a high composity rating 440. In addition, the signals collected from the sensors unit 180 in FIG. 1 may be used to determine if the user 100 is active or non-active, and thus automatically increase or decrease the composity accordingly as depicted by the composity rating 420.
It is important to identify the relative significance of individual records to a user because a cyborglog may contain a lifetime's worth of recordings, while many of which are taken automatically without user thought and efforts. Therefore identifying significant records allows a user to later find/reference a particular record more readily (as further expounded upon below). The notion of composity results in the inference that intent at the time of capture, may lead directly to the significance of the recording. In addition, a set of sensory data may also be used to determine the composity of the recording. For example, the time interval 420 where the user 100 gives the high heart rate and high acceleration values may be ranked higher because it is more likely that an important or exciting event was happening at the time. Composity itself does not require any more user interaction beyond the taking of the image intentionally. A higher degree of composity is determined by inference from the usage (i.e. users need not direct the invention, while rather the invention infers users' intent) or from the biofeedback of the user.
In one embodiment of the present invention, wherein composity is inferred, the inference is made using several cues: user initiated capture results in a higher composity than automated capture; user recording at a higher resolution results in a higher composity; automatically captured records, some may capture the scene better than others and these better images (as defined by an image analysis) may have higher composity; numerous records intentionally taken in rapid succession give the record sequence a higher composity as it implies the user was trying to record something of particular importance.
In another aspect of the invention, the present invention provides a search and/or navigation utility that enables selective searching for or navigation of content having a desired significance factor.
The ranking utility may also rely on other criteria for defining relative significance of content such as number of views by others or amount of viewer activity such as comments and taggings. Thus significance may be affected over time by both the sender's measures like composity, and the community's interaction with the content (e.g. by annotation or dialogue in relation to specific content). Another criterion is image rank, which a rating is given to an image by viewers, expressed as an integer - the higher number the greater the rank. Viewers can increase this number by viewing the recording more times. In addition, a view may increase an image rank by sharing the image (i.e. send to a friend). Rating an image may increase an image rank. Lastly annotating a recording may increase its rank.
In terms of organization, a record for an event may include a sparse set of records of high composity organized in time allowing for quickly searching for any specific record (e.g. a user knows the event they're searching for and sees that it happened before certain records and after certain records - allowing a person's natural sense of time to guide search).
In addition, rather than presenting records captured automatically (which may be illegible) the system prefers high composity recordings when possible, meaning that the set of records presented at each level are of significance to the user and more likely to be recalled in the "memory based" search , or even be those which the user is searching for.
3) Storvboard Functionality
In another aspect of the present invention, first-person (sousveillance) recordings are brought together to describe an event or procedure, which may be storyboarded during the event or procedure. The first person perspective and automated capture of the records means that at the end of performing any task, the record can easily be used as a "How-to" manual. One aspect of the narrative creation is that multiple users can contribute to a single narrative and share a common experience between all gloggers.
Glogger Intelligent Record Management System FIG. 5 depicts an interface for the Glogger Intelligent Navigation System (GIRMS). GIRMS may be implemented on various platform such as Windows, Linux, Mac OSX, or mobile platform such as J2ME and Symbian. The records 51, 53, 54, 55, 58 are the visualization of the recordings captured by the user and stored on the server computer 130 of FIG. 1. To visualize a video recording, a key frame is extracted from each record according to the heuristic index that is computed based on the sensory data streams, i.e. the most representative sequence may be extracted only. The locally cached records may automatically synchronize with the remote server to guarantee that GIRMS always display up-to-date records, i.e. no missing frames or records. The GIRMS may automatically select the most recent records from the database in the server computers 130, or a user may initialize the GIRMS with a filter to get the most relevant information. The processing of the sensory data streams, computation of heuristics index, sorting of the records, and other computational intense operations may be executed locally on the client or user's device or pre-processed on the remote server. The neighborhood records 51, 53, 54 are rendered in 3D space compactly by rotating the frames in the x, y, or z-axis. The perspective projection may be used and therefore the frames may be perceived as a parallelogram in 2D space, i.e. the further the object the smaller it is on screen. The dimension of the frames is also compressed spatially according to the amplitude of the signals from the sensory data such as from an accelerometer, EEG and ECG sensors, etc.. The key record 54 is the frame of interest. The navigation buttons 56 and 57 may be used to switch the frame of interest between records that are most relevant to the key record 54. The stack 55 is the visualization of the most relevant records with the key records. If there are no relevant records, the stack 55 will not be shown. The relevancy of records can be determined using factors such as time, space, and correlation of the sensory date, i.e. the closest in time would be the next record captured. The heuristic and reasoning can be expressed as an index by computing the weighted sum or product of one or more data streams, or with any arithmetic operations that is permitted. The button 58 may allow a user to insert sensory data streams to be considered in the heuristic, and may also allow user to remove the stream. The graph 514 is a visualization of the heuristics index generated by the interface. The absolute amplitude of the sinusoid wave in the graph 513 determines the dimension of the frames rendered, and the compression factor of the frames in time. One example scenario may result in a heuristic index is when the user is inactive, sleeping, or resting. The buttons 512 and 515 may be used to skip records where the user is inactive or has relatively low heuristics on the given data stream. The graph 59 illustrates the compression of the frame in the time axis. The frames are skipped more excessively or sampled less frequently over the interval with low heuristic index. The record 58 is compressed spatially and shown in a smaller size compare to other records such as 53. The button 517 may be used to enable or disable to affect of the data stream to the heuristic. The region 513 is a dedicated region for the key record 54 and the one or more sensory data streams of the key record 54. The bar 514 is to represent the exact location of the frame of the record corresponds to the sensory data stream. The playback frame rate of the recordings may be adjusted according to the heuristic index, and the video frames maybe further compressed on the server computer to reduce the bandwidth usages as needed. A user may select any recording as the key record 54 by selecting it, and all behavior of all components in FIG.5 maybe reprogrammed to customize for the needs of user. All records can be annotated and tagged by entering keywords to the field 52.
Assisting the Visually Impaired
One example of an embodiment of the present invention is a sousveillance system, method, and computer program for assisting the visually impaired by way of a seeing-eye-people collective, personal safety device, and the like.
The visually impaired individual may obtain information regarding their personal surroundings via a collective or a community of individuals all sharing their video surveillance logs. This method should be contrasted with computer vision rather it is a manner to allow individuals from the sharing community to view the surveillance logs/glog and then guide the visually impaired individual. In addition, the glogger may be equipped with a "remote control" feature wherein a remote user may take control of an individual's glogger device. In one embodiment of the present invention, a visually impaired individual wears a glogger imaging system and receives remote assistance from a collective of one or more helpers. This remote assistance may be audible, for example, by way of a wearable cameraphone with eyeglass-based camera and earphones, so the blind person can hear remote commentary from at least one individual from the collective or the cyborglogging community. In addition, this assistance may be tactile by way of a handheld Networked Embedded Sensory System for Intelligence Extender (NESSIE), which consists of a vessel having compressed fluid, such as air, spray out through a plurality of holes. Obstruction of the holes can be sensed with, for example, one or more microphones, so that the NESSIE can feel the touch of the user. The NESSIE can also squirt bursts of air at the user's fingers so the user can feel a message as standard Braille code or as any other desired code, including, possibly, various musical codes. The remote assistance need not always be real-time.
For example, a visually impaired individual can record their surroundings and play this information back later or send it later to a seeing-eye-assistant.
In a further example, during a trip to an art gallery, a visually impaired individual can capture this experience and play it to another individual later in order to receive some knowledge of the art that was experienced earlier that day.
In another embodiment of the present invention, a partially sighted person, who has enough eyesight to aim a camera, but not enough eyesight to really see clearly what the images are, can aim a wearable cameraphone at subject matter and request remote assistance in identifying more detail in what is seen. For example, many visually impaired individuals can still make out the outlines of people but have trouble recognizing who it is. Such persons can certainly aim a camera at a person and capture a good picture of the person's face, where the face is properly centered in the camera's viewfinder. The visually impaired person then sends the image and asks for remote assistance. In another embodiment of the present invention, the request for remote seeing-assistance can be specifically requested, whereas in other embodiments, the request can be serendipitous. A serendipitous request can work in the context of a social networking site, where a visually impaired person might, for example, share their visual world in the hope that others will happen to participate serendipitously. For example, a remote assistant might signal back with some commentary that either directly helps them see or understand visually their present surroundings.
Personal Safety Implementation
The present invention may also be used to function as a personal safety device. In one embodiment of the present invention, a safe watch program, in which a person walking to or from their home or place of work late at night can request one or more remote escorts, or can simply make their online presence known to their community so as to attract a high likelihood of serendipitously participating remote escorts, who may participate as if in a safe-walk program, to report any unusual or suspicious occurrences.
In one embodiment of the present invention, the cameraphone device is contained in a shock- resistant/water-resistant enclosure. Further the cameraphone may be brightly colored in appearance as its enhanced visibility may serve as a deterrent. Further, the device may not be readily opened or stopped so that the user cannot turn it off despite an assailant demanding it.
As for the safe watch request for one or more remote escorts, this may enacted in a deliberate mode or it may be serendipitous, i.e. a person who is a glogger may ask a friend to help them get safely to a particular destination, or, alternatively, a glogger may put out a "softer" request to multiple observers, with a likelihood of remotely being involved in their life. As can be appreciated by those skilled in the art, the system does not necessarily require participation of a specific escort, especially if it can be made known (e.g. by a way of a flashing red light, or the like) to potential assailants that a glogger's audiovisual field of view is being transmitted.
In a broader context, the cameraphone device operable to function within the ambit of a cyborglogging community may serve as a deterrent against both individual terrorism and state terrorism, i.e. against both unaffiliated street thugs, as well as police brutality.
Personal Duress Delete
In this context, another useful feature of the present invention is a "personal duress delete" capability. In this embodiment, one or more images can be made to appear to be deleted but this personal duress delete function will flag the transmitted images for archival, and also notify one or more remote entities that the user is under duress.
Evidence File
In another aspect of the present invention, an evidence file, that provides a true and accurate representation of what an individual experiences, is transmitted live in real-time to a server where the archive can be immediately viewed by large numbers of people, sucti that it would be difficult to falsify the record.
Further, this evidence file or sousveillance recording may be distributed in a unicast or a multicast manner (i.e. to be stored in multiple disparate locations) so that it would be more difficult for a small number of organizations to conspire to falsify it.
Peer-to-Peer The present invention provides for two levels of record sharing:
1) Via communication with the server and the community.
2) Via communication directly between devices.
When the sharing occurs via the server communication mechanism, the client using their body borne device determines its preferred viewing resolution (compression quality, encoding type, and size for example). The client, who is making a request to view a record, transmits their desired viewing resolution to the server along with a request for the record itself. The record may be identified by a unique time prefix and associated user who transmitted the original version. The server transcodes the record to the desired viewing resolution adaptively based on the bandwidth, screen size and other criteria on the device. The server fulfills the client's request and sends the requested record. Any associated annotation and/or comments may additionally be sent with the reply or in another query. A separate HTTP query may be used to determine the available records for browsing for a time period, for a user (for example, records form a particular user taken in the last hour).
The second mode of sharing permits records to be shared between user devices on a peer-to-peer basis. One example of a peer-to-peer architecture provides a client application at each participating device, and a wireless network (by means of an associated network management server) connected to each device that is operable to enable a connection between each participating device, and thereafter the client application at each participating device is operable to enable peer-to-peer content exchange, as well as interaction with the content as previously described.
Another example uses BLUETOOTH™ technology (e.g. by means of associated service discovery protocol (SDP) and OBEX (Object Exchange) protocols). The actual sharing may be achieved via the client application running on the cyborglogging device (i.e. cameraphone). The unique 48-bit BLUETOOTH address of client devices is registered with the server and used to identify peers clients can request user information about peers by detecting nearby BLUETOOTH addresses and querying the server (i.e. determine which users are nearby by detecting and looking up the addresses on the server). Once a nearby device has been detected, the user's device may consult a cached list of addresses to determine if the detected device is in the "sousveillance" community. If not user's device may consult with the server if possible sending the 48-bit address to the server and receiving a community username of the corresponding user if present. If the BLUETOOTH address is a recognized user, one may initiate a connection, requesting a list of records, permission to send/share a record, request a particular record.
Observer Rating and Flagging
Observers of the sousveillance recordings may flag certain images as inappropriate. In this context each observer preferably develops an observer reputation index depending on how long they've been a reliable observer, as well as on past consensus with their observer ratings and any final determinations. If inappropriateness flags come from long-time contributing observers, they are weighted higher than inappropriateness flags from new observers. Inappropriateness flags and various other flags can also be weighted by reputation capital, etc., of each observer, total weightings computed accordingly.
It should be noted that "Observer Rating" differs from promoting "moderators" in that status is earned through use of system this prevents users from creating "alias" users to harm the system since only long term productive participation is significant.
Cyborglogging lets users both sending records and observe rating increases with community participation. Each record that is uploaded increases a participant's rating. Each page that is viewed increases the sender's rating. Each comment increases a participant's rating. Uploaded records which subsequently create commentary in the community increase the sender's rating. Records deleted due to inappropriateness decrease the sender's rating. In a voting system, a number of people may be required to vote for deletion, wherein the first vote is a nomination of the image for deletion and commencement of the vote. If the vote fails to delete the image, the rating of the nominating observer may be decreased.
Further an administrator of the system can grant or remove a user's moderator access. User may be suspended by the administrator or multiple moderators if excessive number of inappropriate records is uploaded or sender's rating is below a threshold. Only users with high rating are allowed for displaying the recording on the front page.
Users of a high reputation index may become eligible or be automatically promoted to moderator status. One embodiment of the present invention includes a continuous scale for granting "powers" from new user to moderator including the ability to appear on the front community page, ability to flag images for deletion, ability to comment on images, to delete comments, to share (via emails or social networking) from site to friends, for granting/revoking these abilities to others as well.
Method of doing insurance business or personal safety business or providing a Safety Management Organization (SMO).
In another aspect of the invention, a method of doing insurance business includes the steps of supplying each at-risk party with a glogging device, and providing incentives in the way of reduced insurance premiums for each party to send live video of their lives to a central evidence log.
Additionally, an insurance company may provide the use of their call center to assist persons in personal safety. For example, a person may wear a camera and transmitter and also wear an output device or display. The display may be a video display or an audio display. The method involves one or more parties wearing a PSD (Personal Safety Device) comprising at least a sousveillance camera, and possibly other sensors. The party may also have a panic button or other panic actuator such as speech recognition or EEG distress activator that automatically or manually activates on brainwave control. Alternatively the panic activator may comprise heart rate sensor, or more generally ECG waveform sensor.
The PSD, when activated, notifies a call center to pay particular attention to a person who might be in distress. The call center can take the form of a Safety Management Organization (SMO) similar to a Health Management Organization (HMO), but broader in context to include not just health care but also preventative safety such as perpetrator management.
The method of doing business can include perpetrator management. For example, it has been noted that persons with special needs are often abused or threatened. There have even been cases of deaf people being shot dead by police because they didn't respond to a policeman's spoken order.
With the present invention a person might receive remote advice during a tough situation, e.g. a victim of a school bully could stream live video and ask for remote advice on how to handle the situation.
The party may have one or more displays, such as a first-person display (for the party), and a second-person display (for the perpetrator to see or hear, or otherwise perceive). The party in actuating a distress call to the call center or SMO gets someone to look through their eyes or perceive their world, and that someone can then manage the perpetrator remotely, for example, by making the perpetrator aware that the situation is being remotely monitored and recorded in such a way that the perpetrator cannot destroy the evidence even if the perpetrator steals the device itself. Perpetrator management aspects of the invention are operated as follows: victim wearing a PSD actuates distress signal; SMO assesses situation and makes decision on action; SMO can communicate with perpetrator to assist in perpetrator management. The display may consist of, for example, a small neckworn or chestworn video display with a red flashing LED or other light source and this may display the face and voice of an SMO employee speaking to the perpetrator. The display need not be limited to audiovisuals such as flashing lights or body- worn sirens or speech or video display. For example, the display may be, or include, a taser or pepper spray discharge device remotely controlled by staff at the SMO. The display may also be, or include, an obscurant such as a release of dense smoke.
In another aspect of the invention, the system includes a peer-to-peer replacement or addendum to the SMO, in which individuals can rely on their own network of friends to help. For example, if a person is being assaulted by an abusive teacher or security guard, he or she can summon help from friends. Additionally, if desired, those friends can make their remote-link presence known to the perpetrator.
Additionally, virtual cyberfriends or avatars can join in. For example, a "talking head" or recording or pre-recorded perpetrator management script can start at the push of a panic button. In this way the perpetrator might not know whether the depicted conference screen face is real or not. Simply having a recorded perpetrator management script may suffice for purposes of providing a personal safety program.
Video Halter Monitor and related inventions
In another aspect of the invention, a chestworn video capture device is combined with recording of electrocardiogram (ECG), along with a system to automatically recognize anomalities in ECG waveform and tag, timestamp, etc., corresponding video. In one embodiment, a PTPsend (Picture Transfer Protocol, as defined in Chapter 3 Comparametric Transforms for Transmitting Eye Tap Video with Picture Transfer Protocol (PTP) W. Steve G. Mann University of Toronto, in the book: The Transform and Data Compression Handbook*).
Sight License
In the capture of video for personal safety devices, the invention can be used to capture evidence that can be viewed by a court of law, while at the same time protecting people's privacy. This can be done with a sight license system that uses file formats, protocols, and software systems to ensure that nobody can see the image data unless authorized to do so (e.g. by court order), or that only certain groups of people can see the data. For example, if the video corresponds to review of Confidential Information like this patent application before it becomes public, the video may be viewable only by a group of people consisting of the inventors, patent lawyer, and the like. In another example, there may be certain semi-private spaces such as spaces that are separate for men and women. This may occur in a church or synagogue where men and women are separated, or on beaches or other places where there are separate bathing areas for men and women. Existing surveillance of such spaces, by systems such as Poseidon and Swimguard which provide recordings of bathing areas for claims management (drowning incidents, fights, unsafe or anti-social behaviour or the like) do not currently provide a simple way for users of the space to indicate their consent or lack thereof, regarding image capture. The invention allows persons of one geniter (i.e. persons with one kind of genitalia) to indicate a lack of consent to being viewed by persons of the other geniter, or the like. A capture system that complies with the sight-license protocol of the invention receives tags from subjects of image data. For example, when images are captured of a particular space, they may be broadcast in such a way that only people in the space can receive the broadcast. FIG. 6 depicts a user 601 who is wearing a capturing device 602 and transmitting content via a wireless medium 609 to user 603 who is in proximity or line-of-sight to each other. This can be done by line-of-sight infrared transmission or the like. This may be done only in the field-of-view of the camera using an aremac that has the same field of broadcast as the camera, so that only people who are captured by the camera can receive the image broadcast from the camera initially. In FIG. 6, the bar 608 illustrates that user 605 is either not in line-of-sight due to obstruction or out-of-sight of user 601, and therefore does not receive the content directly from user 601. Additionally, this method of content sharing is called subject transmitience or subject-mirroring, or subject-equiveillance. The aremac is well known in the art, as described in "Intelligent Image Processing" by S. Mann, publisher John Wiley and Sons, 2001. Equiveillance is also well known in the literature. Subject mirroring allows people to see themselves in the "mirror" of the camera, i.e. as if the camera were a mirror. For example, if person "A" captures an image, using a Sight License system, it would have an aremac with same field of view that sends the image to the people that A captured from. Subjects of A would then be able to see the image if they wanted to, and they are given a chance to indicate consent or lack thereof. Subjects have a choice of Accept, Reject, Broadcast, or Evidencialize. If a subject Rejects an image, but the person A still needs it for evidence, it might fall in minor conflict but not broadcast. A contention-resolver intervenes if there's contention or disagreement between Person "A" and a subject.
For the moment, consider the case when the two (or more) are in agreement as shown FIG. 6. If user 601 captures an image of user 603, and user 603 consents to broadcast of the image, the image may be posted immediately to a public website via the Sight Licensing System 613. User 603 can also Privatize the image, i.e. select that the image be saved in a private repository. User 603 can also Accept an image in a single use or global sense, including the setting of profile messages and default picture-meta-messaging. Picture meta messaging is picture messaging of or for meta data, tags, qualifiers, and the like. Sight Licenses can be autonomous agents, such as a wearable system that receives images and automatically assigns a license category. For example, a necklace-worn license indicator can sense whether or not the wearer is clothed, and, when unclothed, automatically select a different Sight License preference profile. A user, for example, can program the system with such simple commands as "Allow pictures of me when clothed, and dis-allow pictures of me when I'm naked, unless it is detected that I am unconscious (slipped and fell in the shower for example) or drowning (in the bath, for example)". Similarly a user can delegate the decision to a third party, i.e. "Send pictures of me to my agent (such as another human at a remote location) to decide on their disposition".
This it will become apparent, to those skilled in the art, that a simple wearable technology can be used to either capture images, or to authorize the capture of images.
Obviously if the perpetrator of crime or corruption (whether he or she be a lone miscreant or a person in a position of power such as a police officer) does not authorize image capture, it may well be that the person who captured the image could disregard this request.
Thus the invention allows for arbitration of differences in opinion or preference.
Also a person capturing a continuous unbroken record for personal safety and evidence may want integrity of the evidence, but may also want to choose to abide by preferences of those in the environment regarding dissemination of image data.
The Sight License is implemented by the steps of: (l)capture of image data; (2)sending image data over a field-of-broadcast that is similar to the field-of-view of a camera that captured the image; (3)enabling those in the fϊeld-of-brpadcast to view one or more images and comment or annotate the images.
Additional features may be added to the Sight License system, such as separation of License-to- Look from the capture portion of the Sight License. For example, the Sight License may allow capture of images that can't be looked at until a License to Look is obtained. In this way, Sight License and Look License can be separately treated by the invention, and different rules for each can be created.
Sight License refers to the camera itself, what it can capture, and how that data is disseminated, etc., which may be in encrypted form. The Look License allows a separate decryption key. The decryption key may also be partial, such as a decryption up to a certain resolution depending on the Look License. For example, low-resolution images may show that someone has slipped and fallen in a shower room or is drowning in a bath, but not sufficient resolution as to identify the person or private aspects of the person's body.
The invention can capture images in varying resolutions over time and synchronize later. For example, low resolutions are captured immediately and sent, timestamped and encrypted for forensics, broadcast widely so they can't be falsified.
For example, a video record of a person's entire life may be captured and low resolution versions of it published while keeping higher resolution private. In this way the person who has the recording can't falsify it and get away undetected because low resolution versions have already been broadcast.
Higher resolution can follow when Sight License is obtained, and/or when sufficient bandwidth becomes available. Thus a steady low-resolution cyborglog may be sent over low bandwidth cellular phone channels, followed by high resolution sync-up when later a WiFi node is found, or automatically synchronized when later the system is docked to fiber optic link.
Videos may be annotated in low resolution form, and the annotations and discussions can come later. Also, a rating system can be used to allocate Sight Licensure through ranking. If enough people vote to disclose full or greater resolution of a crime against humanity, it may over-ride national security or privacy interests. In this way the Sight License system gives the possibility of balancing the needs of safety and evidence with the alleged needs of privacy or secrecy.
The Sight License also supports subjectrights, such as the right of a subject of veillance to have access to the data of the veillance whether it is surveillance or sousveillance. This is done by way of the aremac broadcast or similar radio directional antenna broadcast that functions like a radio-frequency aremac.
The concept of Sight License applies to sousveillance as well as to surveillance and may be implemented in surveillance (architecturally mounted cameras) or sousveillance (wearable cameras) or both in conjunction with one another.
The camera-mirror or aremac is not the only way to have a Sight License. It may also happen by voting, ratings, or the like, online, with arbitration.
For example, an online Sight License uses the steps as follows:
1. capture image;
2. post low resolution version to server;
3. run a Sight License Manager that considers both opposition and favor of discolsure; 4. arbitrate based on supposed merits at low resolution;
5. increase resolution until merits in favor are met (e.g. capture and arrest of suspect, evidence requirements met, or the like);
6. reach equilibrium resolution at which the needs for the video's existence are met, without excessive loss of privacy.
Additionally this system can operate in a multi-multi-scale fashion, i.e. each of various time- resolution hierarchies can be created for different audiences. For example, jury members of a court may see a medium-resolution video of a slip-and-fall incident or a fight in a shower room, whereas the general public might see only a low-resolution version. The low resolution version might be shown as a news item, to help solicit anonymous crime tips from people who might have been present at the crime scene or might know something about it.
Within the courtroom environment the resolution might be increased to a medium level for jury members and possibly to full resolution for a judge of the same geniter as the persons in the shower room.
The invention enables people to augment their bodies and mediate their perception, to create one or more time-series of data, plugging into their own senses, etc., to interact and communicate, with capability to implement cyborgspace that is present in their environment all the time, if desired.
It can also fundamentally affect the nature of time and place and who and what we are.
This is done through various kinds of social networking, picture messaging, and third-generation web technologies ("Web 3.0" or existential cyborgspace, i.e. wearable Personal Imaging systems, and the like).
Much of social networking has the effect of reversing the Hegelian dialectic in which people lose their humanity through, for, example, the historian who tries (and fails at) being objective. All users of the cyborglogging system can become objective. For example, a lifelong capture of situations and responses can be used to help reduce terrorism.
The invention can also be used by a person wishing to capture as much as can be known about their own lives as well as the way in which they respond to their own lives. For example, by capturing physiological response to image data (brainwaves, etc.), future generations can learn a great deal about a person and their life (perhaps even enough to re-construct a computerized intelligent agent version of that person after they die).
Variations or modifications to the design and construction of this invention, within the scope of the invention, may occur to those skilled in the art upon reviewing the disclosure herein. Such variations or modifications, if within the spirit of this invention, are intended to be encompassed within the scope of any claims to patent protection issuing upon this invention.

Claims

CLAIMSWhat is claimed is:
1. A system for capturing sharing and annotating content characterized in that the system comprises:
(a) at least one content capture device operable to capture content on a continuous basis and transmit the content; and
(b) at least one remote computer; wherein the at least one remote computer is operable to receive the content, and wherein said at least one remote computer further includes or is linked to a database to assemble and store the content and said system provides means for remotely accessing said database.
2. The system as claimed in claim 1 characterized in that at least one user or observer may remotely access said database.
3. The system as claimed in claim 1 characterized in that the content is transmitted via a wireless medium.
4. The system as claimed in any one of claim 1 characterized in that the content capture device is operable to capture content based on user defined parameters.
5. The system as claimed in any one of claim 1 characterized in that the content capture device is a portable device which is mounted, body borne or implanted in the body of a user.
6. The system as claimed in any one of claim 1 characterized in that the content capture device includes or is linked to a biometric sensor unit operable to capture and provide to the content capture device biometric information for transmittal to the at least one remote computer.
7. The system as claimed in claim 6 characterized in that the content capture device is operable to capture content based on user defined thresholds measured by the biometric sensor unit.
8. The system as claimed in claim 2 characterized in that content capture device is operable to capture content based on user selection.
9. The system as claimed in any claim 2 characterized in that the system enables one or more users or observers to annotate content.
10. The system as claimed in claim 9 characterized in that the system if operable to enable the user and/or an observer to associate descriptions with selected content.
11. The system as claimed in claim 2 characterized in that the content is transmittable between content capture devices.
12. The system as claimed in claim 1 characterized in that the content capture device consists of a portable device such as a camera phone.
13. The system as claimed in any one of claim 1 characterized in that the content capture device is operable to upload the captured content to the at least one remote computer using a multimedia message service, email or HTTP POST.
14. The system as claimed in claim 1 characterized in that the at least one content capture device further includes a client resource management utility operable to transmit low bandwidth content when bandwidth is limited.
15. The system as claimed in claim 14 characterized in that the client resource management utility may detect when bandwidth is not limited and transmit high resolution content.
16. The system as claimed in claim 14 characterized in that the server resource management utility is operable to associate content received in low resolution format from the content capture device with corresponding content received in high resolution from the content capture device.
17. The system as claimed in claim 16 characterized in that the server resource management utility is operable to replace low resolution content sent from the content capture device with high resolution content once it is received from the content capture device.
18. The system as claimed in claim 17 characterized in that the server resource management utility is operable to store low resolution format content until a request is received from at least one user or observer in which case a higher resolution format content is requested from the content capture device.
19. The system as claimed in claim 18 characterized in that the low resolution format content and the high resolution format content may be associated through the use of a cryptographic hash function.
20. The system as claimed in claim 19 characterized in that the lower resolution format content and the high resolution format content may be associated by a public/private key encryption method.
21. A method for capturing sharing and annotating content characterized by the steps of: (a) capturing content using a client application residing on a content capturing device; (b) transmitting the content from the client application to a server application residing on one or more remote computers;
(c) assembling and storing the content on a database associated with the server application; and
(d) enabling one or more users or observers to access the content stored within the database and annotate the content.
22. The method as claimed in claim 21 characterized in that the content is transmitted via a wireless medium.
PCT/CA2008/001866 2007-10-23 2008-10-23 System, method and computer program for capturing, sharing, and annotating content WO2009052618A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US98213507P 2007-10-23 2007-10-23
US60/982,135 2007-10-23

Publications (2)

Publication Number Publication Date
WO2009052618A1 true WO2009052618A1 (en) 2009-04-30
WO2009052618A8 WO2009052618A8 (en) 2009-12-10

Family

ID=40579003

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CA2008/001866 WO2009052618A1 (en) 2007-10-23 2008-10-23 System, method and computer program for capturing, sharing, and annotating content

Country Status (1)

Country Link
WO (1) WO2009052618A1 (en)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101833115A (en) * 2010-05-18 2010-09-15 山东师范大学 Life detection and rescue system based on augment reality technology and realization method thereof
US20100283609A1 (en) * 2009-05-07 2010-11-11 Perpcast, Inc. Personal safety system, method, and apparatus
ITCS20090014A1 (en) * 2009-11-25 2011-05-26 Donato Roberto TELEPHONE RECEIVING A COMMAND FROM ANOTHER TELEPHONE PLACED IN A PLACE AT ITS OWN PLEASURE, SENDING IT IN REAL TIME AND WITHOUT ANY HUMAN INTERVENTION, TO THE PHONE THAT MAKES IT REQUIRED, PHOTO OR RECORDING, VIDEO OR REGISTRATION, REGISTRATION
US8832233B1 (en) 2011-07-20 2014-09-09 Google Inc. Experience sharing for conveying communication status
WO2015127383A1 (en) * 2014-02-23 2015-08-27 Catch Motion Inc. Person wearable photo experience aggregator apparatuses, methods and systems
WO2015184299A1 (en) * 2014-05-30 2015-12-03 Frank Wilczek Systems and methods for expanding human perception
US9251013B1 (en) 2014-09-30 2016-02-02 Bertram Capital Management, Llc Social log file collaboration and annotation
GB2533122A (en) * 2014-12-10 2016-06-15 Edmund Todd Robert An adaptive access-control and surveillance system and method thereof
US9699281B2 (en) 2009-02-27 2017-07-04 Eyecam, Inc. Headset-based telecommunications platform
WO2020006189A1 (en) * 2018-06-29 2020-01-02 The Regents Of The University Of Michigan A wearable camera system for crime deterrence

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB611993A (en) * 1946-06-06 1948-11-05 British United Shoe Machinery Improvements in or relating to the filling of shoe-bottom cavities
US7069003B2 (en) * 2003-10-06 2006-06-27 Nokia Corporation Method and apparatus for automatically updating a mobile web log (blog) to reflect mobile terminal activity
US20060235926A1 (en) * 2005-04-19 2006-10-19 Sony Ericsson Mobile Communications Japan, Inc. Portable communication apparatus
US20060259293A1 (en) * 2005-05-11 2006-11-16 France Telecom Computerized method and apparatus for automatically generating a natural language description of a person's activities
US7173651B1 (en) * 1998-06-02 2007-02-06 Knowles Andrew T Apparatus and system for prompt digital photo delivery and archival
US20070136302A1 (en) * 2005-12-12 2007-06-14 Microsoft Corporation Automated device blog creation
JP2007144072A (en) * 2005-11-25 2007-06-14 Tomoyuki Hisanaga Automatic device for generating diary/blog in online game, generating method, program, and system
JP2008027042A (en) * 2006-07-19 2008-02-07 Nec Corp Automatic blog generation system, automatic blog generation method, and program
JP2008242734A (en) * 2007-03-27 2008-10-09 Nec Corp Home page automatic updating method and system

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
GB611993A (en) * 1946-06-06 1948-11-05 British United Shoe Machinery Improvements in or relating to the filling of shoe-bottom cavities
US7173651B1 (en) * 1998-06-02 2007-02-06 Knowles Andrew T Apparatus and system for prompt digital photo delivery and archival
US7069003B2 (en) * 2003-10-06 2006-06-27 Nokia Corporation Method and apparatus for automatically updating a mobile web log (blog) to reflect mobile terminal activity
US20060235926A1 (en) * 2005-04-19 2006-10-19 Sony Ericsson Mobile Communications Japan, Inc. Portable communication apparatus
US20060259293A1 (en) * 2005-05-11 2006-11-16 France Telecom Computerized method and apparatus for automatically generating a natural language description of a person's activities
JP2007144072A (en) * 2005-11-25 2007-06-14 Tomoyuki Hisanaga Automatic device for generating diary/blog in online game, generating method, program, and system
US20070136302A1 (en) * 2005-12-12 2007-06-14 Microsoft Corporation Automated device blog creation
JP2008027042A (en) * 2006-07-19 2008-02-07 Nec Corp Automatic blog generation system, automatic blog generation method, and program
JP2008242734A (en) * 2007-03-27 2008-10-09 Nec Corp Home page automatic updating method and system

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
STEVE MANN: "Wearable Computing: A First Step Toward Personal Imaging", IEEE COMPUTER, vol. 30, no. 2, February 1997 (1997-02-01), pages 25 - 32 *

Cited By (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9860352B2 (en) 2009-02-27 2018-01-02 Eyecam, Inc. Headset-based telecommunications platform
US9699281B2 (en) 2009-02-27 2017-07-04 Eyecam, Inc. Headset-based telecommunications platform
US9177455B2 (en) * 2009-05-07 2015-11-03 Perpcast, Inc. Personal safety system, method, and apparatus
US20100283609A1 (en) * 2009-05-07 2010-11-11 Perpcast, Inc. Personal safety system, method, and apparatus
US9589447B2 (en) * 2009-05-07 2017-03-07 Perpcast, Inc. Personal safety system, method, and apparatus
ITCS20090014A1 (en) * 2009-11-25 2011-05-26 Donato Roberto TELEPHONE RECEIVING A COMMAND FROM ANOTHER TELEPHONE PLACED IN A PLACE AT ITS OWN PLEASURE, SENDING IT IN REAL TIME AND WITHOUT ANY HUMAN INTERVENTION, TO THE PHONE THAT MAKES IT REQUIRED, PHOTO OR RECORDING, VIDEO OR REGISTRATION, REGISTRATION
CN101833115A (en) * 2010-05-18 2010-09-15 山东师范大学 Life detection and rescue system based on augment reality technology and realization method thereof
US8893010B1 (en) 2011-07-20 2014-11-18 Google Inc. Experience sharing in location-based social networking
US9367864B2 (en) 2011-07-20 2016-06-14 Google Inc. Experience sharing with commenting
US9015245B1 (en) 2011-07-20 2015-04-21 Google Inc. Experience sharing with commenting
US8934015B1 (en) 2011-07-20 2015-01-13 Google Inc. Experience sharing
US8832233B1 (en) 2011-07-20 2014-09-09 Google Inc. Experience sharing for conveying communication status
WO2015127383A1 (en) * 2014-02-23 2015-08-27 Catch Motion Inc. Person wearable photo experience aggregator apparatuses, methods and systems
US20160360160A1 (en) * 2014-02-23 2016-12-08 Catch Motion Inc. Person wearable photo experience aggregator apparatuses, methods and systems
WO2015184299A1 (en) * 2014-05-30 2015-12-03 Frank Wilczek Systems and methods for expanding human perception
US10089900B2 (en) 2014-05-30 2018-10-02 Wolfcub Vision, Inc. Systems and methods for expanding human perception
US9251013B1 (en) 2014-09-30 2016-02-02 Bertram Capital Management, Llc Social log file collaboration and annotation
GB2533122A (en) * 2014-12-10 2016-06-15 Edmund Todd Robert An adaptive access-control and surveillance system and method thereof
WO2020006189A1 (en) * 2018-06-29 2020-01-02 The Regents Of The University Of Michigan A wearable camera system for crime deterrence

Also Published As

Publication number Publication date
WO2009052618A8 (en) 2009-12-10

Similar Documents

Publication Publication Date Title
WO2009052618A1 (en) System, method and computer program for capturing, sharing, and annotating content
US8918133B2 (en) Mobile based neighborhood watch system and client application
US10979636B2 (en) Triggering actions based on shared video footage from audio/video recording and communication devices
US7840203B2 (en) Process and system for automatically transmitting audio/video content from an electronic device to desired recipient(s)
US11423764B2 (en) Emergency communications using client devices that are associated with audio/video recording and communication devices
US9740940B2 (en) Event triggered location based participatory surveillance
US20170124834A1 (en) Systems and methods for secure collection of surveillance data
US20190373219A1 (en) Methods, systems, apparatuses and devices for facilitating management of emergency situations
US20070282623A1 (en) Process for protecting children from online predators
US20180218201A1 (en) Sharing Positive Information Captured Using Audio/Video Recording and Communication Devices
US10841542B2 (en) Locating a person of interest using shared video footage from audio/video recording and communication devices
US12401966B2 (en) Method and network storage device for providing security
US20210264541A1 (en) Community watch with bot based unified social network groups
US8775816B2 (en) Method and apparatus to enhance security and/or surveillance information in a communication network
Mann et al. Cyborglogging with camera phones: Steps toward equiveillance
US20190392692A1 (en) Electronic fingerprint device for identifying perpetrators and witnesses of a crime and method thereof
Michael Sousveillance: Implications for privacy, security, trust, and the law
JP2005215996A (en) Surveillance video management device and surveillance video management method
Duncan et al. The portal monitor: a privacy-enhanced event-driven system for elder care
JP2004158950A (en) Automatic recording video generation system, automatic recording video generation method, automatic recording video generation program, and recording medium for automatic recording video generation program
JP6556391B1 (en) Monitoring system
Michael Redefining surveillance: Implications for privacy, security, trust and the law
CN114520795B (en) Group creation method, group creation device, computer device and storage medium
CN114726816B (en) Method and device for establishing association relationship, electronic equipment and storage medium
Michael et al. Societal implications of wearable technology: Interpreting “trialability on the run”

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 08842719

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 08842719

Country of ref document: EP

Kind code of ref document: A1