WO2013126382A1 - System and method for linking media expressions for purchasing a product or other actionable events - Google Patents

Description

Title:

[001] SYSTEM AND METHOD FOR LINKING MEDIA EXPRESSIONS FOR

PURCHASING A PRODUCT OR OTHER ACTIONABLE EVENTS

Background of the Invention

[002] The present invention relates generally to systems and method for media expressions and purchasing products and conducting actionable events within those media expressions.

[003] In recent years, there has been a dramatic growth in the number of consumers that order goods and/or services, and then pay for them, using electronic devices. For instance, it is fairly common for a purchaser to watch a home shopping television program, choose a product, and order the product over the telephone. The product is shipped and the purchaser is billed at a later time. However, with the advent of mobile digital devices, the requirement to order a product over the telephone becomes unnecessary and burdensome. And often, viewers of media expressions often see or like products or services associated with a media expression or television broadcast, but do not have ability to identify the product or services for means of purchasing the associated product or service. The present invention attempts to solve these problems as well as others.

Summary of the Invention

[004] Methods and systems are provided herein for linking a media expression to an actionable event for a user. The user engages the media expression by scanning a multi-modal artifact or code from the media expression with a digital device. The media expression can be a 3-D environment, a 3-D virtual reality (3-D VR) environment, or a 2-D environment, such as a television show, movie, online media expression, internet infomercials, video games, and the like. The digital device may be a smartphone, tablet, laptop, smart remote, e-reader, and the like, which is capable of capturing multi-modal artifacts. In one embodiment, the actionable event may include, but is not limited to: the user purchasing a product or service associated with the scanned multi-modal artifact data; offering the user a discount code associated with the scanned multimodal artifact data; offering the user a promotional subscription associated with the scanned multi-modal artifact data; or any other consumer inducement. The actionable event may be an embedded in the digital device or otherwise an on-board application on a media display leading to a direct purchase by the user of a product or service associated with the captured artifact, or the actionable event exchanges communications with a server system or 3^rd party provider for purposes of making purchases of a product or service associated with the captured multi-modal artifact. In association with the purchase, the server could couple a URL associated with the captured multi-modal artifact, receive a coupon, and receive associated discounts for the product or related products, or other consumer inducements such as reward points, airline miles, and the like.

[005] The actionable event may be communicated from a data repository or server, which acts upon the multi-modal artifact and returns an actionable event to the user's digital device. A user's digital device allows users to engage ("scan") artifacts from the media expression. In some embodiments, a multi-modal image recognition application allows for the capture of data that can be representative of a sensory input, or be digitally encoded information. The digitally encoded data could represent a barcode, a 2D code, a 3D code, a 3d-VR code, a matrix code, a data matrix, video, audio, a QR code, or other such symbologies. Captured multi-modal artifact data may alternatively be representative of a pure image, the selection of an internet link by the user, detection of an RFID tag, an NFC tag, a software input, or the like. The Actionable Event may be enabled by a mobile payment system. The mobile payment system may include, but is not limited to: Premium SMS based transactional payments, Direct Mobile Billing, Mobile web payments (WAP), or Contactless NFC (Near Field Communication). Alternatively, the Actionable Event may exchange information with a data repository or server for purchasing the product or service associated with the multi-modal artifact.

Brief Description of the Drawings

[0(56] The accompanying appendices and/or drawings illustrate various non-limiting, example, inventive aspects in accordance with the present disclosure:

FIGS, la-b show combined logic and data flow block diagram illustrating a general overview of embodiments of a Code Triggered Information Server (CTIS) for delivering

Actionable Events;

[008] FIG. 2A shows a particular, exemplary embodiment of the GCSI/CTIS, wherein the actionable event is served to the user related to a business commercial in a media expression; FIG. 2B shows a particular, exemplary embodiment of the GCSI/CTIS, wherein the actionabl e event is served to the user related to a Food Network broadcast; FIG. 2C shows a particular, exemplary embodiment of the GCSI/CTIS, wherein the actionable event is served to the user engaging a 3D- VR video game;

[1199] FIG. 3 shows user trajectories through a network of 3^ra party provider zones; [010] FIG. 4 shows an embodiment of the web interface for user registration;

[911] FIG, 5a shows three types of 2D barcodes that may be employed within the CTIS; FIGS. 5b-c show logic flow for code enhancement in one embodiment of CTIS operation; FIG. 5d shows an implementation of logic flow for customized repair and/or decoding of obscure or ambiguous scanned codes;

[012] FIG. 6 shows an illustration of one embodiment of the Scan Message data structure;

[913] FIGS. 7a-b shows an illustration of one embodiment of a user profile data structure and an implementation of a user profile user interface in one embodiment of GCSI/CTIS operation

[014] FIG. 8 shows an illustration of one embodiment of the information Base (Actionable Event) data structure;

[015] FIG. 9 shows a logic flow diagram of one embodiment of the user profile query heuristic;

[916] FIG. J O shows an exemplary histogram of 3^id party provider zone breaches;

[0Ϊ7] FIGS. lla~d show some embodiments of systems for selecting non-triggered information to serve to users;

[018] FIGS. 12a-b show examples of actionabie events provided by the CTIS;

[919] FIG. 13 shows a combined logic and data flow block diagram for an embodiment of the CTIS wherein contextual Actionable Events are replaced by Actionable Events based on ambient conditions;

[02Θ] FIG. 14 shows a combined logic and data flow block diagram for a particular exemplary embodiment of the CTIS in which allergy medication Actionable Events are served in response to a high pollen count;

[021] FIG. 15 shows a combined logic and data flow block diagram for a particular exemplary embodiment of the CTIS in which sunscreen Actionable Events are served in response to heat and high UV levels;

[922] FIG. 1 shows a combined logic and data flow block diagram for an embodiment of the CTIS wherein the selection of contextual Actionable Events is influenced or enhanced by ambient conditions;

[023] FIG, 17 shows a combined logic and data flow block diagram for a particular exemplar}' embodiment of the CTIS in which an Actionable Event to purchase a luxury convertible is served based on warm weather conditions, a scanned code, and the user profi le;

[024[ FIG. 18 shows a combined logic and data flow block diagram for a particular exemplary embodiment of the CTIS in which an Actionable Event to purchase Florida travel is served based on cold/rainy weather conditions, a scanned code, and the user profile;

{025] FIG. J 9 shows a combined logic and data flow block diagram for a particular exemplary embodiment of the CTIS in which an Actio able Event to purchase ice cream is served based on warm weather conditions, a scanned code, and the user profile;

[026] FIG, 20 shows a combined logic and data flow block diagram for a particular exemplary embodiment of the CTIS in which an Actionable Event to purchase ponchos and umbrellas is served based on rainy weather conditions, a scanned code, and the user profile;

[027] FIG, 21 shows a combined logic and data flow block diagram for an embodiment of the CTIS wherein Ads or preferences are served to a User Agent device;

[028] FIG, 22 shows a combined logic and data flow block diagram for a particular exemplary embodiment of the CTIS in which an Actionable Event to purchase a sports team's season tickets is served along with a pay-per-view event supplied directly to a user's cable box;

[029] FIG, 23 shows a combined logic and data flow block diagram for a particular exemplar}' embodiment of the CTIS in which an Actionabl e Event to purchase virus scan subscription software is served;

{030] FIG. 24 shows an implementation of a user interface manifesting GCSI/CTIS functionality on a portable electronic device in one embodiment of GCSI/CTIS operation;

[031] FIG. 25 shows an implementation of a user interface manifesting GCSI/CTIS functionality on digital paper in one embodiment of GCSI/CTIS operation;

[032] FIG. 26 shows an implementation of a user interface manifesting GCSi functionality on a digital billboard in one embodiment of GCSI/CTIS operation;

[033] FIG, 27 shows an implementation of a user interface manifesting GCSI/CTIS functionality for serving a short messaging service text message prompt in one embodiment of GCSI/CTIS operation;

{0343 FIG. 28 shows an implementation of a user interface manifesting GCSI/CTIS functionality on a mobile phone in one embodiment of GCSI/CTI S operation;

[035] FIG, 29 shows an implementation of a user interface manifesting GCSI/CTIS functionality on a portable communication device in one embodiment of GCSI/CTIS operation;

{0363 FIG. 30 shows an implementation of logic flow for media acquisition in one embodiment of GCSI/CTIS operation; [037] FIG. 31 shows an illustration of media sampling and full media acquisition in one embodiment of GCSI/CTIS operation;

[038] FIGS, 32a~b show an im lementation of logic flow for code scan nionetization in one embodiment of GCSI/CTIS operation;

[039] FIG S. 33a-e sho w aspects of implementations of JagTag encoding and codes in some embodiments of GCSI/CTIS operation;

[040] FIG, 34 shows a computer sy sterilization of the GCSI/CTIS; and

[Ml] FIG. 35a is a block diagram illustrating a CT S configured to assign users having similar interests and situation information into dynamic groups according to an embodiment; and FIG, 35b is a block diagram illustrating a CTIS configured to identify profile elements that are related to specific conte t according to an embodiment,

DETAILED DESCRIPTION OF THE INVENTION DEFINITIONS & TERMINOLOGY

Servers, computers, clients, networks, routers

[042] Typically, users, which may be people and/or other systems, engage information technology systems (e.g., commonly computers) to facilitate information processing. In turn, computers employ processors to process information; such processors are often referred to as central processing units (CPU). A common form of processor is referred to as a microprocessor. CPUs use communicative signals to enable various operations. Such communicative signals may be stored and/or transmitted in batches as program and/or data components facilitate desired operations. These stored instruction code signals may engage the CPU circuit components to perform desired operations. A common type of program is a computer operating system, which, commonly, is executed by CPU on a computer; the operating system enables and facilitates users to access and operate computer information technology and resources. Common resources employed in information technology systems include: input and output mechanisms through which data may pass into and out of a computer; memory storage into which data may be saved; and processors by which information may be processed. Often information technology systems are used to collect data for later retrieval, analysis, and manipulation, commonly, which is facilitated through a database program. Information technology systems provide interfaces that allow users to access and operate various system components.

[043] In one embodiment, the code triggered information server controller 3401 may be connected to and/or communicate with entities such as, but not limited to: one or more users from user input devices 3411; peripheral devices 3412; a cryptographic processor device 3428; and/or a communications network 3413, as shown in FIG. 34.

[044] Networks are commonly thought to comprise the interconnection and interoperation of clients, servers, and intermediary nodes in a graph topology. It should be noted that the term

"server" as used throughout this disclosure refers generally to a computer, other device, program, or combination thereof that processes and responds to the requests of remote users across a communications network. Servers serve their information to requesting "clients." The term "client" as used herein refers generally to a computer, other device, program, or combination thereof that is capable of processing and making requests and obtaining and processing any responses from servers across a communications network. A computer, other device, program, or combination thereof that facilitates, processes information and requests, and/or furthers the passage of information from a source user to a destination user is commonly referred to as a "node." Networks are generally thought to facilitate the transfer of information from source points to destinations. A node specifically tasked with furthering the passage of information from a source to a destination is commonly called a "router." There are many forms of networks such as Local Area Networks (LANs), Pico networks, Wide Area Networks (WANs), Wireless Networks (WLANs), etc. For example, the Internet is generally accepted as being an interconnection of a multitude of networks whereby remote clients and servers may access and interoperate with one another.

[045] The code triggered information server controller 3401 may be based on common computer systems that may comprise, but are not limited to, components such as: a computer systemization 3402 connected to memory 3429.

Computer Systemization

[046] A computer systemization 3402 may comprise a clock 3430, central processing unit (CPU) 3403, a read only memory (ROM) 3406, a random access memory (RAM) 3405, and/or an interface bus 3407, and most frequently, although not necessarily, are all interconnected and/or communicating through a system bus 3404, as shown in FIG. 34. Optionally, the computer systemization may be connected to an internal power source 3486. Optionally, a cryptographic processor 3426 may be connected to the system bus. The system clock typically has a crystal oscillator and provides a base signal. The clock is typically coupled to the system bus and various clock multipliers that will increase or decrease the base operating frequency for other components interconnected in the computer systemization. The clock and various components in a computer systemization drive signals embodying information throughout the system. Such transmission and reception of signals embodying information throughout a computer systemization may be commonly referred to as communications. These communicative signals may further be transmitted, received, and the cause of return and/or reply signal communications beyond the instant computer systemization to: communications networks, input devices, other computer systemizations, peripheral devices, and/or the like. Of course, any of the above components may be connected directly to one another, connected to the CPU, and/or organized in numerous variations employed as exemplified by various computer systems.

[047] The CPU comprises at least one high-speed data processor adequate to execute program components for executing user and/or system-generated requests. The CPU may be a

microprocessor such as AMD's Athlon, Duron and/or Opteron; IBM and/or Motorola's PowerPC; IBM's and Sony's Cell processor; Intel's Celeron, Itanium, Pentium, Xeon, and/or XScale; and/or the like processor(s). The CPU interacts with memory through signal passing through conductive conduits to execute stored signal program code according to conventional data processing techniques. Such signal passing facilitates communication within the code triggered information server controller and beyond through various interfaces. Should processing requirements dictate a greater amount speed, parallel, mainframe and/or super-computer architectures may similarly be employed. Alternatively, should deployment requirements dictate greater portability, smaller Personal Digital Assistants (PDAs) may be employed.

Power Source

[048] The power source 3486 may be of any standard form for powering small electronic circuit board devices such as the following power cells: alkaline, lithium hydride, lithium ion, lithium polymer, nickel cadmium, solar cells, and/or the like, as shown in FIG. 34. Other types of AC or DC power sources may be used as well. In the case of solar cells, in one embodiment, the case provides an aperture through which the solar cell may capture photonic energy. The power cell 3486 is connected to at least one of the interconnected subsequent components of the code triggered information server thereby providing an electric current to all subsequent components. In one example, the power source 3486 is connected to the system bus component 3404. In an alternative embodiment, an outside power source 3486 is provided through a connection across the I/O interface 3408. For example, a USB and/or IEEE 1394 connection carries both data and power across the connection and is therefore a suitable source of power.

Interface Adapters

[049] Interface bus(ses) 3407 may accept, connect, and/or communicate to a number of interface adapters, conventionally although not necessarily in the form of adapter cards, such as but not limited to: input output interfaces (I/O) 3408, storage interfaces 3409, network interfaces 3410, and/or the like, as shown in FIG. 34. Optionally, cryptographic processor interfaces 3427 similarly may be connected to the interface bus. The interface bus provides for the

communications of interface adapters with one another as well as with other components of the computer systemization. Interface adapters are adapted for a compatible interface bus. Interface adapters conventionally connect to the interface bus via a slot architecture. Conventional slot architectures may be employed, such as, but not limited to: Accelerated Graphics Port (AGP), Card Bus, (Extended) Industry Standard Architecture ((E)ISA), Micro Channel Architecture (MCA), NuBus, Peripheral Component Interconnect (Extended) (PCI(X)), PCI Express, Personal Computer Memory Card International Association (PCMCIA), and/or the like.

[050] Storage interfaces 3409 may accept, communicate, and/or connect to a number of storage devices such as, but not limited to: storage devices 3414, removable disc devices, and/or the like. Storage interfaces may employ connection protocols such as, but not limited to: (Ultra) (Serial) Advanced Technology Attachment (Packet Interface) ((Ultra) (Serial) ATA(PI)), (Enhanced) Integrated Drive Electronics ((E)IDE), Institute of Electrical and Electronics Engineers (IEEE) 1394, fiber channel, Small Computer Systems Interface (SCSI), Universal Serial Bus (USB), and/or the like.

[051] Network interfaces 3410 may accept, communicate, and/or connect to a communications network 3413. Through a communications network 3413, the code triggered information server controller is accessible through remote clients 3433b (e.g., computers with web browsers) by users 3433a. Network interfaces may employ connection protocols such as, but not limited to: direct connect, Ethernet (thick, thin, twisted pair 10/100/1000 Base T, and/or the like), Token Ring, wireless connection such as IEEE 802.1 la-x, and/or the like. A communications network may be any one and/or the combination of the following: a direct interconnection; the Internet; a Local Area Network (LAN); a Metropolitan Area Network (MAN); an Operating Missions as Nodes on the Internet (OMNI); a secured custom connection; a Wide Area Network (WAN); a wireless network (e.g., employing protocols such as, but not limited to a Wireless Application Protocol (WAP), I-mode, and/or the like); and/or the like. A network interface may be regarded as a specialized form of an input output interface. Further, multiple network interfaces 3410 may be used to engage with various communications network types 3413. For example, multiple network interfaces may be employed to allow for the communication over broadcast, multicast, and/or unicast networks.

[052] Input Output interfaces (I/O) 3408 may accept, communicate, and/or connect to user input devices 3411, peripheral devices 3412, cryptographic processor devices 3428, and/or the like. I/O may employ connection protocols such as, but not limited to: Apple Desktop Bus (ADB); Apple Desktop Connector (ADC); audio: analog, digital, monaural, RCA, stereo, and/or the like; IEEE 1394a-b; infrared; joystick; keyboard; midi; optical; PC AT; PS/2; parallel; radio; serial; USB; video interface: BNC, coaxial, composite, digital, Digital Visual Interface (DVI), RCA, RF antennae, S-Video, VGA, and/or the like; wireless; and/or the like. A common output device is a television set, which accepts signals from a video interface. Also, a video display, which typically comprises a Cathode Ray Tube (CRT) or Liquid Crystal Display (LCD) based monitor with an interface (e.g., DVI circuitry and cable) that accepts signals from a video interface, may be used. The video interface composites information generated by a computer systemization and generates video signals based on the composited information in a video memory frame. Typically, the video interface provides the composited video information through a video connection interface that accepts a video display interface (e.g., an RCA composite video connector accepting an RCA composite video cable; a DVI connector accepting a DVI display cable, etc.).

[053] User input devices 3411 may be card readers, dongles, finger print readers, gloves, graphics tablets, joysticks, keyboards, mouse (mice), remote controls, retina readers, trackballs, trackpads, and/or the like.

[054] Peripheral devices 3412 may be connected and/or communicate to I/O and/or other facilities of the like such as network interfaces, storage interfaces, and/or the like. Peripheral devices may be audio devices, cameras, dongles (e.g., for copy protection, ensuring secure transactions with a digital signature, and/or the like), external processors (for added functionality), goggles, microphones, monitors, network interfaces, printers, scanners, storage devices, video devices, video sources, visors, and/or the like.

[055] It should be noted that although user input devices and peripheral devices may be employed, the code triggered information server controller may be embodied as an embedded, dedicated, and/or monitor-less (i.e., headless) device, wherein access would be provided over a network interface connection.

[056] Cryptographic units such as, but not limited to, microcontrollers, processors 3426, interfaces 3427, and/or devices 3428 may be attached, and/or communicate with the code triggered information server controller. A MC68HC16 microcontroller, commonly manufactured by Motorola Inc., may be used for and/or within cryptographic units. Equivalent microcontrollers and/or processors may also be used. The MC68HC16 microcontroller utilizes a 16-bit multiply- and-accumulate instruction in the 16 MHz configuration and requires less than one second to perform a 512-bit RSA private key operation. Cryptographic units support the authentication of communications from interacting agents, as well as allowing for anonymous transactions.

Cryptographic units may also be configured as part of CPU. Other commercially available specialized cryptographic processors include VLSI Technology's 33 MHz 6868 or Semaphore Communications' 40 MHz Roadrunner 184.

Memory

[057] Generally, any mechanization and/or embodiment allowing a processor to affect the storage and/or retrieval of information is regarded as memory 3429, as shown in FIG. 34.

However, memory is a fungible technology and resource, thus, any number of memory

embodiments may be employed in lieu of or in concert with one another. It is to be understood that the code triggered information server controller and/or a computer systemization may employ various forms of memory 3429. For example, a computer systemization may be configured wherein the functionality of on-chip CPU memory (e.g., registers), RAM, ROM, and any other storage devices are provided by a paper punch tape or paper punch card mechanism; of course such an embodiment would result in an extremely slow rate of operation. In a typical

configuration, memory 3429 will include ROM 3406, RAM 3405, and a storage device 3414. A storage device 3414 may be any conventional computer system storage. Storage devices may include a drum; a (fixed and/or removable) magnetic disk drive; a magneto-optical drive; an optical drive (i.e., CD ROM/RAM/Recordable (R), Rewritable (RW), DVD R/RW, etc.); an array of devices (e.g., Redundant Array of Independent Disks (RAID)); and/or other devices of the like. Thus, a computer systemization generally requires and makes use of memory.

Component Collection [058] The memory 3429 may contain a collection of program and/or database components and/or data such as, but not limited to: operating system component(s) 3415 (operating system);

information server component(s) 3416 (information server); user interface component(s) 3417 (user interface); Web browser component(s) 3418 (Web browser); database(s) 3419; mail server component(s) 3421 ; mail client component(s) 3422; cryptographic server component(s) 3420 (cryptographic server); the code triggered information server component(s) 3435; and/or the like (i.e., collectively a component collection) , as shown in FIG. 34. These components may be stored and accessed from the storage devices and/or from storage devices accessible through an interface bus. Although non-conventional program components such as those in the component collection, typically, are stored in a local storage device 3414, they may also be loaded and/or stored in memory such as: peripheral devices, RAM, remote storage facilities through a communications network, ROM, various forms of memory, and/or the like.

Operating System

[059] The operating system component 3415 is an executable program component facilitating the operation of the code triggered information server controller, as shown in FIG. 34. Typically, the operating system facilitates access of I/O, network interfaces, peripheral devices, storage devices, and/or the like. The operating system may be a highly fault tolerant, scalable, and secure system such as Apple Macintosh OS X (Server), AT&T Plan 9, Be OS, Linux, Unix, and/or the like operating systems. However, more limited and/or less secure operating systems also may be employed such as Apple Macintosh OS, Microsoft DOS, Microsoft Windows

2000/2003/3.1/95/98/CE/Millenium/NT/Vista/XP (Server), Palm OS, and/or the like. An operating system may communicate to and/or with other components in a component collection, including itself, and/or the like. Most frequently, the operating system communicates with other program components, user interfaces, and/or the like. For example, the operating system may contain, communicate, generate, obtain, and/or provide program component, system, user, and/or data communications, requests, and/or responses. The operating system, once executed by the CPU, may enable the interaction with communications networks, data, I/O, peripheral devices, program components, memory, user input devices, and/or the like. The operating system may provide communications protocols that allow the code triggered information server controller to communicate with other entities through a communications network 3413. Various

communication protocols may be used by the code triggered information server system controller as a subcarrier transport mechanism for interaction, such as, but not limited to: multicast, TCP/IP, UDP, unicast, and/or the like.

[060] The operating system component 3415 may also be a mobile operating system, mobile OS, such as iOS (Apple), ANDROID, WINDOWS PHONE (Microsoft), WINDOWS PHONE 7 (Microsoft), BLACKBERRY OS (RIM), WebOS (Hewlett-Packard) , MEEGO or SYMBIAN, for example. The mobile OS may be a manufacture-built proprietary operating system, a third party proprietary, or an open source operating system. Mobile OS's are employed on a wide variety of mobile devices useful with the present invention, such as smartphones and tablet computers.

Information Server

[061] An information server component 3416 is a stored program component that is executed by a CPU, as shown in FIG. 34. The information server may be a conventional Internet information server such as, but not limited to Apache Software Foundation's Apache, Microsoft's Internet Information Server, and/or the. The information server may allow for the execution of program components through facilities such as Active Server Page (ASP), ActiveX, (ANSI) (Objective-) C (++), C#, Common Gateway Interface (CGI) scripts, Java, JavaScript, Practical Extraction Report Language (PERL), Python, WebObjects, and/or the like. The information server may support secure communications protocols such as, but not limited to, File Transfer Protocol (FTP);

HyperText Transfer Protocol (HTTP); Secure Hypertext Transfer Protocol (HTTPS), Secure Socket Layer (SSL), and/or the like. The information server provides results in the form of Web pages to Web browsers, and allows for the manipulated generation of the Web pages through interaction with other program components. After a Domain Name System (DNS) resolution portion of an HTTP request is resolved to a particular information server, the information server resolves requests for information at specified locations on the code triggered information server controller based on the remainder of the HTTP request. For example, a request such as

http://123.124.125.126/myInformation.html might have the IP portion of the request

"123.124.125.126" resolved by a DNS server to an information server at that IP address; that information server might in turn further parse the http request for the "/mylnformation.html" portion of the request and resolve it to a location in memory containing the information

"mylnformation.html." Additionally, other information serving protocols may be employed across various ports, e.g., FTP communications across port 21 , and/or the like. An information server may communicate to and/or with other components in a component collection, including itself, and/or facilities of the like. Most frequently, the information server communicates with the code triggered information server database 3419, operating systems, other program components, user interfaces, Web browsers, and/or the like.

[062] Access to the code triggered information server database may be achieved through a number of database bridge mechanisms such as through scripting languages as enumerated below (e.g., CGI) and through inter-application communication channels as enumerated below (e.g., CORBA, WebObjects, etc.). Any data requests through a Web browser are parsed through the bridge mechanism into appropriate grammars as required by the code triggered information server. In one embodiment, the information server would provide a Web form accessible by a Web browser. Entries made into supplied fields in the Web form are tagged as having been entered into the particular fields, and parsed as such. The entered terms are then passed along with the field tags, which act to instruct the parser to generate queries directed to appropriate tables and/or fields. In one embodiment, the parser may generate queries in standard SQL by instantiating a search string with the proper join/select commands based on the tagged text entries, wherein the resulting command is provided over the bridge mechanism to the code triggered information server as a query. Upon generating query results from the query, the results are passed over the bridge mechanism, and may be parsed for formatting and generation of a new results Web page by the bridge mechanism. Such a new results Web page is then provided to the information server, which may supply it to the requesting Web browser.

[063] Also, an information server may contain, communicate, generate, obtain, and/or provide program component, system, user, and/or data communications, requests, and/or responses.

User Interface

[064] The function of computer interfaces in some respects is similar to automobile operation interfaces. Automobile operation interface elements such as steering wheels, gearshifts, and speedometers facilitate the access, operation, and display of automobile resources, functionality, and status. Computer interaction interface elements such as check boxes, cursors, menus, scrollers, and windows (collectively and commonly referred to as widgets) similarly facilitate the access, operation, and display of data and computer hardware and operating system resources,

functionality, and status. Operation interfaces are commonly called user interfaces. Graphical user interfaces (GUIs) such as the Apple Macintosh Operating System's Aqua or Lion, Microsoft's Windows XP, Windows 7 or Unix's X- Windows provide a baseline and means of accessing and displaying information graphically to users. Mobile OS's, discussed above, also typically employ GUIs, but may also be text-based, and provide user interfaces to mobile devices.

[065] A user interface component 3417, whether on a desktop or laptop computing device or a mobile device, is a stored program component that is executed by a CPU, as shown in FIG. 34. The user interface may be a conventional graphic user interface as provided by, with, and/or atop operating systems and/or operating environments such as those operating systems discussed above, and/or the like. The user interface may allow for the display, execution, interaction, manipulation, and/or operation of program components and/or system facilities through textual and/or graphical facilities. The user interface provides a facility through which users may affect, interact, and/or operate a computer system. A user interface may communicate to and/or with other components in a component collection, including itself, and/or facilities of the like. Most frequently, the user interface communicates with operating systems, other program components, and/or the like. The user interface may contain, communicate, generate, obtain, and/or provide program component, system, user, and/or data communications, requests, and/or responses.

Web Browser

[066] A Web browser component 3418 is a stored program component that is executed by a CPU, as shown in FIG. 34. The Web browser may be a conventional hypertext viewing application such as Microsoft Internet Explorer or Netscape Navigator. Secure Web browsing may be supplied with 128 bit (or greater) encryption by way of HTTPS, SSL, and/or the like. Some Web browsers allow for the execution of program components through facilities such as Java, JavaScript, ActiveX, and/or the like. Web browsers and like information access tools may be integrated into PDAs, cellular telephones, and/or other mobile devices. A Web browser may communicate to and/or with other components in a component collection, including itself, and/or facilities of the like. Most frequently, the Web browser communicates with information servers, operating systems, integrated program components (e.g., plug-ins), and/or the like; e.g., it may contain, communicate, generate, obtain, and/or provide program component, system, user, and/or data communications, requests, and/or responses. Of course, in place of a Web browser and information server, a combined application may be developed to perform similar functions of both. The combined application would similarly affect the obtaining and the provision of information to users, user agents, and/or the like from the code triggered information server enabled nodes. The combined application may be nugatory on systems employing standard Web browsers.

Mail Server

[067] A mail server component 3421 is a stored program component that is executed by a CPU 3403, as shown in FIG. 34. The mail server may be a conventional Internet mail server such as, but not limited to sendmail, Microsoft Exchange, and/or the like. The mail server may allow for the execution of program components through facilities such as ASP, ActiveX, (ANSI)

(Objective-) C (++), CGI scripts, Java, JavaScript, PERL, pipes, Python, WebObjects, and/or the like. The mail server may support communications protocols such as, but not limited to: Internet message access protocol (IMAP), Microsoft Exchange, post office protocol (POP3), simple mail transfer protocol (SMTP), and/or the like. The mail server can route, forward, and process incoming and outgoing mail messages that have been sent, relayed and/or otherwise traversing through and/or to the code triggered information server.

[068] Access to the code triggered information server mail may be achieved through a number of APIs offered by the individual Web server components and/or the operating system.

[069] Also, a mail server may contain, communicate, generate, obtain, and/or provide program component, system, user, and/or data communications, requests, information, and/or responses.

Mail Client

[070] A mail client component 3422 is a stored program component that is executed by a CPU 3403, as shown in FIG. 34. The mail client may be a conventional mail viewing application such as Apple Mail, Microsoft Entourage, Microsoft Outlook, Microsoft Outlook Express, Mozilla Thunderbird, and/or the like. Mail clients may support a number of transfer protocols, such as: IMAP, Microsoft Exchange, POP3, SMTP, and/or the like. A mail client may communicate to and/or with other components in a component collection, including itself, and/or facilities of the like. Most frequently, the mail client communicates with mail servers, operating systems, other mail clients, and/or the like; e.g., it may contain, communicate, generate, obtain, and/or provide program component, system, user, and/or data communications, requests, information, and/or responses. Generally, the mail client provides a facility to compose and transmit electronic mail messages.

Cryptographic Server

[071] A cryptographic server component 3420 is a stored program component that is executed by a CPU 3403, cryptographic processor 3426, cryptographic processor interface 3427, cryptographic processor device 3428, and/or the like, as shown in FIG. 34. Cryptographic processor interfaces will allow for expedition of encryption and/or decryption requests by the cryptographic component; however, the cryptographic component, alternatively, may run on a conventional CPU. The cryptographic component allows for the encryption and/or decryption of provided data. The cryptographic component allows for both symmetric and asymmetric (e.g., Pretty Good Protection (PGP)) encryption and/or decryption. The cryptographic component may employ cryptographic techniques such as, but not limited to: digital certificates (e.g., X.509 authentication framework), digital signatures, dual signatures, enveloping, password access protection, public key management, and/or the like. The cryptographic component will facilitate numerous

(encryption and/or decryption) security protocols such as, but not limited to: checksum, Data Encryption Standard (DES), Elliptical Curve Encryption (ECC), International Data Encryption Algorithm (IDEA), Message Digest 5 (MD5, which is a one way hash function), passwords, Rivest Cipher (RC5), Rijndael, RSA (which is an Internet encryption and authentication system that uses an algorithm developed in 1977 by Ron Rivest, Adi Shamir, and Leonard Adleman), Secure Hash Algorithm (SHA), Secure Socket Layer (SSL), Secure Hypertext Transfer Protocol (HTTPS), and/or the like. Employing such encryption security protocols, the code triggered information server may encrypt all incoming and/or outgoing communications and may serve as node within a virtual private network (VPN) with a wider communications network. The cryptographic component facilitates the process of "security authorization" whereby access to a resource is inhibited by a security protocol wherein the cryptographic component effects authorized access to the secured resource. In addition, the cryptographic component may provide unique identifiers of content, e.g., employing an MD5 hash to obtain a unique signature for a digital audio file. A cryptographic component may communicate to and/or with other components in a component collection, including itself, and/or facilities of the like. The cryptographic component supports encryption schemes allowing for the secure transmission of information across a communications network to enable the code triggered information server component to engage in secure transactions if so desired. The cryptographic component facilitates the secure accessing of resources on the code triggered information server and facilitates the access of secured resources on remote systems; i.e., it may act as a client and/or server of secured resources. Most frequently, the cryptographic component communicates with information servers, operating systems, other program components, and/or the like. The cryptographic component may contain, communicate, generate, obtain, and/or provide program component, system, user, and/or data communications, requests, and/or responses.

The Code triggered information server Database

[072] The code triggered information server database component 3419 may be embodied in a database and its stored data, as shown in FIG. 34. The database is a stored program component, which is executed by the CPU; the stored program component portion configuring the CPU to process the stored data. The database may be a conventional, fault tolerant, relational, scalable, secure database such as Oracle or Sybase. Relational databases are an extension of a flat file.

Relational databases consist of a series of related tables. The tables are interconnected via a key field. Use of the key field allows the combination of the tables by indexing against the key field; i.e., the key fields act as dimensional pivot points for combining information from various tables.

Relationships generally identify links maintained between tables by matching primary keys.

Primary keys represent fields that uniquely identify the rows of a table in a relational database.

More precisely, they uniquely identify rows of a table on the "one" side of a one -to-many relationship.

[073] Alternatively, the code triggered information server database may be implemented using various standard data-structures, such as an array, hash, (linked) list, struct, structured text file (e.g., XML), table, and/or the like. Such data-structures may be stored in memory and/or in (structured) files. In another alternative, an object-oriented database may be used, such as Frontier, ObjectStore, Poet, Zope, and/or the like. Object databases can include a number of object collections that are grouped and/or linked together by common attributes; they may be related to other object collections by some common attributes. Object-oriented databases perform similarly to relational databases with the exception that objects are not just pieces of data but may have other types of functionality encapsulated within a given object. If the code triggered information server database is implemented as a data-structure, the use of the code triggered information server database 3419 may be integrated into another component such as the code triggered information server component 3435. Also, the database may be implemented as a mix of data structures, objects, and relational structures. Databases may be consolidated and/or distributed in countless variations through standard data processing techniques. Portions of databases, e.g., tables, may be exported and/or imported and thus decentralized and/or integrated.

[074] Alternatively, the CTIS database 3419 may be decentralized and distributed across multiple server sites or reside entirely within a Cloud Computing Environment (the "Cloud"), as shown in FIG. 34. Further, the CTIS database 3419 may consist of multiple databases distributed across the Internet or in the Cloud, and may, for example, include various search commercial engines such as GOOGLE, YAHOO SEARCH, BF G or the like as data look up resources and pointers to URLs of putative interest to a user.

[075] In one embodiment, the database component 3419 includes several tables 3419a,b. A user profile table 3419a includes fields such as, but not limited to: a user ID, name, email address, address, demographic profile, hardware ID, scan history record, scan statistics, and/or the like. The user profile table may support and/or track multiple entity accounts on a code triggered

information server. An Information Base table 3419b includes fields such as, but not limited to: 3^rd party provider ID, 3^rd party ID, 3^rd party preference content, preference labels, geographic tags, temporal tags, subject tags, preference ratings, preference statistics, and/or the like.

[076] In one embodiment, the code triggered information server database may interact with other database systems. For example, employing a distributed database system, queries and data access by OLBS modules may treat the combination of the code triggered information server database, an integrated data security layer database as a single database entity.

[077] In one embodiment, user programs may contain various user interface primitives, which may serve to update the code triggered information server. Also, various accounts may require custom database tables depending upon the environments and the types of clients the code triggered information server may need to serve. It should be noted that any unique fields may be designated as a key field throughout. In an alternative embodiment, these tables have been decentralized into their own databases and their respective database controllers (i.e., individual database controllers for each of the above tables). Employing standard data processing techniques, one may further distribute the databases over several computer systemizations and/or storage devices. Similarly, configurations of the decentralized database controllers may be varied by consolidating and/or distributing the various database components 3419a,b. The code triggered information server may be configured to keep track of various settings, inputs, and parameters via database controllers.

[078] The code triggered information server database may communicate to and/or with other components in a component collection, including itself, and/or facilities of the like. Most frequently, the code triggered information server database communicates with the code triggered information server component, other program components, and/or the like. The database may contain, retain, and provide information regarding other nodes and data.

The Code triggered information server

[079] The code triggered information server component 3435 is a stored program component that is executed by a CPU, as shown in FIG. 34. The code triggered information server affects accessing, obtaining and the provision of information, preferences, services, transactions, and/or the like across various communications networks.

[080] The code triggered information server component enables the identification, generation, and aggregation of Scan Messages and Actionable Events.

[081] The code triggered information server component enabling access of information between nodes may be developed by employing standard development tools such as, but not limited to: (ANSI) (Objective-) C (++), Apache components, binary executables, database adapters, Java, JavaScript, mapping tools, procedural and object oriented development tools, PERL, Python, shell scripts, SQL commands, web application server extensions, WebObjects, and/or the like. In one embodiment, the code triggered information server employs a cryptographic server to encrypt and decrypt communications. The code triggered information server component may communicate to and/or with other components in a component collection, including itself, and/or facilities of the like. Most frequently, the code triggered information server component communicates with the code triggered information server database, operating systems, other program components, and/or the like. The code triggered information server may contain, communicate, generate, obtain, and/or provide program component, system, user, and/or data communications, requests, preferences, and/or responses.

[082] In a further embodiment, the code triggered information server may interact with one or more third-party database systems to retrieve requested information. For example, the CTIS may receive a request for information regarding a product. In turn, the CTIS accesses a third-party database containing the information requests, e.g., a database maintained by the manufacturer of the product, a database maintained by a retailer, or a database maintained by a product information aggregator (e.g., Google Shopping). The database accessed is also determined by the type of information requested. Types of information requested includes brand information, products by brand name, product images, URLs for branded products, point of purchase URLs, customer reviews of branded products, pricing, product information, product descriptions, product registrations, product compositions, product configurations, product interactions, styles, medical reference, drug information; drug interactions, drug compatibility, supplement compatibility, and drug recalls.

Distributed Code triggered information server

[083] The structure and/or operation of any of the code triggered information server node controller components may be combined, consolidated, and/or distributed in any number of ways to facilitate development and/or deployment. Similarly, the component collection may be combined in any number of ways to facilitate deployment and/or development. To accomplish this, one may integrate the components into a common code base or in a facility that can dynamically load the components on demand in an integrated fashion.

[084] The component collection may be consolidated and/or distributed in countless variations through standard data processing and/or development techniques. Multiple instances of any one of the program components in the program component collection may be instantiated on a single node, and/or across numerous nodes to improve performance through load-balancing and/or data- processing techniques. Furthermore, single instances may also be distributed across multiple controllers and/or storage devices; e.g., databases. All program component instances and controllers working in concert may do so through standard data processing communication techniques.

[085] The configuration of the code triggered information server controller will depend on the context of system deployment. Factors such as, but not limited to, the budget, capacity, location, and/or use of the underlying hardware resources may affect deployment requirements and configuration. Regardless of if the configuration results in more consolidated and/or integrated program components, results in a more distributed series of program components, and/or results in some combination between a consolidated and distributed configuration, data may be

communicated, obtained, and/or provided. Instances of components consolidated into a common code base from the program component collection may communicate, obtain, and/or provide data. This may be accomplished through intra-application data processing communication techniques such as, but not limited to: data referencing (e.g., pointers), internal messaging, object instance variable communication, shared memory space, variable passing, and/or the like.

[086] If component collection components are discrete, separate, and/or external to one another, then communicating, obtaining, and/or providing data with and/or to other component components may be accomplished through inter-application data processing communication techniques such as, but not limited to: Application Program Interfaces (API) information passage; (distributed) Component Object Model ((D)COM), (Distributed) Object Linking and Embedding ((D)OLE), and/or the like), Common Object Request Broker Architecture (CORBA), process pipes, shared files, and/or the like. Messages sent between discrete component components for inter-application communication or within memory spaces of a singular component for intra-application communication may be facilitated through the creation and parsing of a grammar. A grammar may be developed by using standard development tools such as lex, yacc, XML, and/or the like, which allow for grammar generation and parsing functionality, which in turn may form the basis of communication messages within and between components. Again, the configuration will depend upon the context of system deployment.

Code triggered information server Controller

[087] FIG. 34 of the present disclosure illustrates inventive aspects of a code triggered information server controller 3401 in a block diagram. In this embodiment, the code triggered information server controller 3401 may serve to aggregate, process, store, search, serve, identify, instruct, generate, match, and/or update Scan Messages, Actionable Events, Ads, user profiles, and/or other related data.

SCAN AND ACTIONABLE EVENT MESSAGES

[088] In one embodiment, a code triggered information server (CTIS) employs two basic actors: (i) a user (100a) capable of registering interest in an activity, transmitting and receiving data, scanning or capturing a multi-modal artifact or code suitable that may be captured by means of a digital device external to or embedded in a media expression (102), television, or display system, and displaying information; and (ii) a server (101) capable of communicating with the user, storing/updating user profiles, rendering an actionable event, and selecting information to serve to the user, as shown in FIG. la. In addition to the two basic actors, the CTIS enables the creation of two types of transactional articles: i) Scan Messages (115) and ii) Actionable Events (145). The CTIS provides context-specific Actionable Events in response to Scan Messages, which are prepared in response to the registration of activity interest by the user, in order to supply information and/or advertising to the user, or allow the user to purchase a product or service (170) related to the Scan Message. Actionable events may include, but are not limited to: the user purchasing a product or service associated with the scanned multi-modal artifact data; offering a discount code associated with the scanned multi-modal artifact data; offering a promotional subscription associate with the scanned multi-modal artifact data; or any consumer inducement. The CTIS generates Actionable Events by processing Scan Message content with user profile information, containing basic profiling data as well as a history of Scan Messages supplied by a given user. As such, Actionable Events are transactional articles that heighten a provider's opportunity to supply relevant and targeted information, allow the user to purchase products or services from media expressions (102) and/or advertising to the user that is both context and user specific. Without loss of generality, we will heretofore refer to the information and/or advertising content as "Actionable Events" or alternatively as "Ads." The CTIS may also elect to send context and user-specific Actionable Events based solely on the user profile, without the user having recently registered activity interest, if it is determined from the user's profile, particularly the history of Scan Messages, that a particular Actionable Event is appropriate.

"REGISTERING" INTEREST (SCANNING)

[089] In one embodiment, users of the CTIS scan codes or multi-modal artifacts that may be captured by means of a digital device external to or embedded in a media expression (102), television, or display system to register interest, and the CTIS acts as a portal for consumers to access information supplemental to the context in which the codes or multi-modal artifacts are found to send an Actionable Event for product or service purchase. Users may scan codes or multi-modal artifacts found in media publications, on billboards or signs, on consumer products or packages, on websites or television screens, and/or the like. In doing so, the users may identify themselves via an identification code specific to the mobile device, digital device, and service account with which the scan is made. At the same time, the scanned code or multi-modal artifact can be recorded, as well as the time of day and user location; such information may form the basis of consumer profiles that may be saved by the CTIS. As these scans are indicative of interest in various subjects, goods, and/or services, they may serve as a basis for an individual customer tracking system. As such, advertisers or 3^rd party providers can finely tune and efficiently tailor their efforts towards achieving maximum return on the advertising expenditure. The media expression can be a 3-D environment, a 3-D virtual reality (3-D VR) environment, or a 2-D environment, such as a television show, movie, online media expression, internet infomercials, and the like.

Artifact Capture [090] As used herein the term "multi-modal artifact" is intended to include, without limitation, products, activities, services, print, visual, electronic or audible media, barcodes, brand names, product configurations, including, for example, packaging or container configurations, shapes or color combinations for products (e.g., pharmaceutical capsule color coding or pill shapes), video, body movements or gestures, olfactory scents, haptic or tactile stimuli, sound stimuli, and gustatory or taste stimuli. Multi-modal artifact and code may be used interchangeably throughout this application.

[091] Artifacts capable of being capture include, among other things, 3D barcodes that are have regions of differing heights, with the regions, their patterns and their relative heights being representative of data. In accordance with one embodiment of the present invention, the three dimensional nature of artifacts, including barcodes, may be used advantageously for allowing sight or hearing impaired individuals to capture data from artifacts with 3D codes, whether 3D barcodes per se, or other 3D surfaces capable of acting as data sources, and have that data captured, processed and provided to such individuals in a form that makes the information encoded by the 3D data more available to the user. In one particular embodiment, sight impaired individuals often employ their haptic senses to, for example, read using braille, to scan products to identify container shapes or features or the like. These skills are, of course, learned and require considerable effort on the part of the individual. The present invention may be used with sight impaired individuals to capture sensory data otherwise unavailable or difficult to access to such individuals and convert the sensory data into a usable form, e.g., sounds. Similarly, in accordance with another embodiment of the invention, a hearing impaired individual may use the present invention to capture audible sensory data, e.g., music, street sounds, warning sounds or the like, and covert such unavailable or difficult to access sensory data in a form, e.g., visual, that renders such audible sensory data more available and meaningful to a hearing impaired individual. In this manner, the present invention may be employed to simply capture and convert data existent in one sensory form to another sensory form that is more useful or available to a user having a particular disability or impairment.

[092] The data captured may be single mode or multi-modal data, and may consist, for example of any sensory-type input, e.g., visual, auditory, olfactory, haptics or gustatory. Visual inputs may consist of image capture, pattern recognition, text recognition, bar codes, such as UPC or QR codes, hand gestures, or the like. Auditory inputs may consist of music, sound recognition, sound pattern recognition, spoken word recognition, or the like. Olfactory inputs may consist of inputs from artificial noses, such as that described by Stitzel, S.E., et al. in Artificial Noses, Ann. Rev. Biomed. Eng. Vol. 13, 1-25 (Aug. 2011) or in U.S. Patent No. 7,261,857, gas chromatographs or other instruments for detecting the presence of airborne elements. Tactile inputs may consist of data acquired by virtue of surface roughness measurements, such as by atomic force microscopy, by a tactile-sensing system such as that described by Decherchi, S., et al, IEEE Transactions on Robotics, 27(3), 635-639, June, 2011, or by tactile sensors as described in U.S. Patent No.

7,823,467, each of which is expressly incorporated by reference. Finally, gustatory, or the taste inputs, may be obtained by employing a taste sensor, such as that described in U.S. Patent No. 7,858,036 or the taste sensor data system described in U.S. Patent Application Publication No. 20040107053, each of which is expressly incorporated by reference.

[093] In one embodiment, the data capture device 100b may be in the proximity contact with the product of interest, such as Near Field Communications (NFC), Radio Frequency Identification (RFID), Bluetooth, Infrared, Wireless communication, Automatic Identification and Data Capture (AIDC), and other short range communications and the like. AIDC refers to the methods of automatically identifying objects, collecting data about them, and entering that data directly into computer systems (i.e. without human involvement). Technologies typically considered as part of AIDC include bar codes, Radio Frequency Identification (RFID), biometrics, magnetic stripes, Optical Character Recognition (OCR), smart cards, and voice recognition. AIDC is the process or means of obtaining external data, particularly through analysis of images, sounds or videos. To capture data, a transducer is employed which converts the actual image or a sound into a digital file. The file is then stored and at a later time it can be analyzed by a computer, or compared with other files in a database to verify identity or to provide authorization to enter a secured system. Capturing of data can be done in various ways; the best method depends on application. AIDC also refers to the methods of recognizing objects, getting information about them and entering that data or feeding it directly into computer systems without any human involvement. Automatic identification and data capture technologies include barcodes, RFID, bokodes, OCR, magnetic stripes, smart cards and biometrics (like iris and facial recognition system).

[094] As discussed above, artifacts capable of being capture include, among other things, 3D barcodes that are have regions of differing heights, with the regions, their patterns and their relative heights being representative of data. In accordance with one embodiment of the present invention, the three dimensional nature of artifacts, including barcodes, may be used advantageously for allowing sight or hearing impaired individuals to capture data from artifacts with 3D codes, whether 3D barcodes per se, or other 3D surfaces capable of acting as data sources, and have that data captured, processed and provided to such individuals in a form that makes the information encoded by the 3D data more available to the user. In one particular embodiment, sight impaired individuals often employ their haptic senses to, for example, read using braille, to scan products to identify container shapes or features or the like. These skills are, of course, learned and require considerable effort on the part of the individual. The present invention may be used with sight impaired individuals to capture sensory data otherwise unavailable or difficult to access to such individuals and convert the sensory data into a usable form, e.g., sounds. Similarly, in accordance with another embodiment of the invention, a hearing impaired individual may use the present invention to capture audible sensory data, e.g., music, street sounds, warning sounds or the like, and covert such unavailable or difficult to access sensory data in a form, e.g., visual, that renders such audible sensory data more available and meaningful to a hearing impaired individual. In this manner, the present invention may be employed to simply capture and convert data existent in one sensory form to another sensory form that is more useful or available to a user having a particular disability or impairment.

Code Triggered Information Server Basic Overview

[095] FIG. la shows a combined logic and data flow block diagram illustrating a general overview of the CTIS in one embodiment. The CTIS may be configured so that a user (100a) may engage with/in a media expression (102) to capture a digital code or artifact (105) with a scanning digital device (100b), as for example by using a cell phone camera to scan and/or take a picture of a barcode, 2D code, 3D code, 3D-VR codes, 2-D or 3-D artifacts, matrix code, data matrix, QR code, or other such symbology. An example of a mobile device and software capable of such code capture is the Nokia 6600 cellular telephone equipped with ConnexTo software. In other embodiments, the capture device (100b) may be capable of scanning codes associated with multimodal codes or artifacts, such as RFID tags, NFC tags, and/or the like. Alternatively, the digital device may be a tablet, e-reader, smart remote, computer, and/or the like. Alternatively, the CTIS may be triggered by the user selecting an internet link (such as on an internet enabled mobile device, digital device, television, and the like), by making decisions within a virtual world (such as a massively multiplayer online game), and/or the like. Without loss of generality, all such forms of interest-registering CTIS triggering will be referred to as "code scanning" or "artifact scanning" or "multi-modal artifact capturing." The time and location (as determined by GPS, GPRS, or other such geo-positioning technologies) of the mobile device are determined (110), and a Scan

Message (115) is generated containing a user identification (ID), a hardware ID that identifies the type of mobile device being used, a geocode (location of the device), a timestamp, and the scanned code (see FIG. 6 and discussion of Scan Message below). Additional user-input information may also be supplied at this point. The Scan Message is packaged and sent (120) to a server (101) that receives and recognizes it (125). The user's profile, specific to the supplied user ID, is updated with the new Scan Message (i.e., it is added as a transaction to their profile transaction history) and queried (130) for information to use in the preparation of an Actionable Event (140) that is also based on the content of the current Scan Message. The server selects (135) the information content (see FIG. 9) of the Actionable Event (145) from an Information Base and sends it to the user's mobile display device, which displays the pertinent Actionable Event content to the user (150).

[096] The Actionable Event (145) may be an embedded or otherwise on-board application leading to a direct purchase by the user of a product or service associated with the captured artifact or exchanges communications with a server system or 3^rd party provider for purposes of making purchases of a product or service associated with the captured artifact. In association with the purchase, the server could couple a URL associated with the captured artifact, receive a coupon, and receive associated discounts for the product or related products, or other consumer

inducements such as reward points, airline miles, and the like.

[097] The Actionable Event may be enabled by a mobile payment system. The mobile payment system may be Premium SMS based transactional payments, Direct Mobile Billing, Mobile web payments (WAP), or Contactless NFC (Near Field Communication). SMS based transactional payments are when the user sends a payment request via an SMS text message or an USSD to a short code and a premium charge is applied to their phone bill or their online wallet. The merchant involved is informed of the payment success and can then release the paid for goods. A

Multimedia Messaging Service can also deliver barcodes which can then be scanned for confirmation of payment by a merchant. This is used as an electronic ticket for access to cinemas and events or to collect hard goods. Mobile web payments may use direct operator billing, credit card, eWallets, SmartWallet, or online wallets. eWallet or "digital wallet" is also increasingly being used to describe mobile phones, especially smartphones, which store an individual's credentials and utilize wireless technologies such as near field communication (NFC) to carry out financial transactions. Online wallets may include PayPal, Amazon Payments, eBay, and Google Checkout. NFC employs a special mobile phone equipped with a smartcard that is waved near a reader module. Most transactions do not require authentication, but some require authentication using PF , before transaction is completed. The payment could be deducted from a pre-paid account or charged to a digital wallet or bank account directly. NFC may be employed by Google Wallet, Visa DPS, and the like.

MMS

[098] In one implementation, the Scan Message and/or scanned code may be sent to the server (101) by a Multimedia Messaging Service (MMS) protocol. In other implementations, a wide variety of other communication protocols may be employed, including but not limited to: Short Messaging Service (SMS), instant messaging, web browser based messaging, email, Enhanced Messaging System (EMS), TCP/IP, WAP, and/or the like.

SHORT CODE

[099] In one implementation, the server (101) may be addressed by a short code, short numbers, Common Short Code (CSC), and/or the like for sending the Scan Message and/or scanned code. For example, a short code may appear alongside a code in a print publication, and a user may be informed that the scanned code should be sent to the indicated short code number in order to retrieve an Actionable Event, advertisement, coupon, and/or the like. In other implementations, other addressing means may be employed, such as an ordinary telephone number, an email address, a universal resource locater (URL), and/or the like. In another implementation, the short code and/or other server address may be encoded within part of the code itself. When the user scans the code with his or her mobile device, the code may be partially decoded by the client device to extract the address and automatically send the rest of the code thereto.

SCANNING

[0100] FIG. lb shows an implementation of combined logic and data flow in another embodiment of CTIS operation. A code (155) is scanned by a camera phone (156), and the resulting image content is converted to an MMS format to be sent via short code (157) through a gateway provider (158) to the CTIS (159) server. At the server, the image information may be enhanced, manipulated, and/or otherwise processed (160) so as to aid in determining the appropriate content to serve in response. After enhancement, the image content is read (161) and the content and format for the appropriate response is determined (162) (see, e.g., FIGS. 9-11). A response message is formed in MMS format (163) and is sent (164) back to a gateway provider (158) to be served to the user's camera phone (156).

[0101] In an alternative implementation, the scanned code may be directly decoded on the client mobile device instead of on the server. The decoded content may then be converted to MMS format and sent via short code to the server in order to retrieve the appropriate content associated with the code. That content may be returned from the server to the client mobile device by means of an MMS protocol as well.

CTIS SERVER

[0102] For illustration, the CTIS is shown in FIG. 2A for a particular, exemplary embodiment: providing targeted and contextual information and Actionable Events to the user related to a Business commercial on a TV. Here, a user employs a cell phone to scan a code (205a) or other multi-modal artifact suitable to capture by means of a digital device external to or embedded in the television or display system associated with Business commercial (207), such as by using the phone's built in camera to photograph a 2D matrix code, 3D code, or 3d-VR code (e.g., QR code). The location of the scan (San Francisco, Calif.) is determined by the phone's built in GPS receiver, and this location is incorporated as a geocode, along with the user ID, the hardware ID of the particular type of device (e.g., cell phone) used, the time, and the code content, into a Scan Message. This is sent to a server (201) via the cell phone's data connection (e.g., wireless internet systems such as lxEV-DO and the like), and the server queries a database of user profiles based on the supplied user ID in order to update the appropriate user profile with the content of the current Scan Message and to determine what, if any, information in the user profile may be relevant to the preparation of the current Actionable Event. Here, the server (201) recognizes that the user is 55 years of age and has recently been scanning artifacts and/or ads related to

investments and retirement planning in relation to the queried user profile (230a). In one embodiment, the user can supply a list of interests to be considered in addition to those culled from their scan history. The server selects and retrieves (235 a) the appropriate content to include in the Actionable Event, comprised of purchasing events, coupon codes, discount codes, special offers (237a), and an Actionable Event to purchase a brokerage or retirement savings account (238a), including information on branch locations in San Francisco near the user. This reply is sent by the server to the user's mobile device which displays the Actionable Event message content (250a). The user may then purchase a brokerage or retirement savings account or service (25 la), which may be employed with a mobile payment system, such as Google Wallet and the like.

[0103] Alternatively, CTIS is shown in FIG. 2B for a particular embodiment providing targeted and contextual information and Actionable Events to the user related to a Food Network television broadcast. Here, a user employs a cell phone or other digital device (200b) to scan and capture a QR code or other multi-modal artifact associated with Food Network television broadcast (208), such as by using the phone's built in camera to photograph a 2D matrix code, 3D code, or 3D-VR code (e.g., QR code). The location of the scan is determined by the phone's built in GPS receiver, and this location is incorporated as a geocode, along with the user ID, the hardware ID of the particular type of device (e.g., cell phone) used, the time, and the code content, into a Scan Message. This is sent to a server (201) via the cell phone's data connection (e.g., wireless internet systems such as lxEV-DO and the like), and the server queries a database of user profiles based on the supplied user ID in order to update the appropriate user profile with the content of the current Scan Message and to determine what, if any, information in the user profile may be relevant to the preparation of the current Actionable Event (230b). Here, the server updates the User profile. In one embodiment, the user can supply a list of interests for the Food Network channel or related products and services to be considered in addition to those culled from their scan history. The server (201) selects and retrieves (235b) the appropriate content to include in the Actionable Event, comprised of purchasing events, coupon codes, discount codes, special offers (237b), and an Actionable Event to provide a discount promotion associated with the Food Network TV broadcast (238b), including providing the opportunity to make the purchase by displaying the Actionable Event on the digital device (250b) and the user purchases the product or service (251b) utilizing the digital device and associated purchase applications enabled by the digital device. The purchase applications may be a mobile payment system, such as Google Wallet and the like.

[0104] Alternatively, CTIS is shown in FIG. 2C for a particular embodiment providing targeted and contextual information and Actionable Events to the user related to a 3D-VR video game on an Xbox 360 and the like. The user (200a) is playing a road race game featuring the latest cars and concept cars. One of the cars is the newest Ford subcompact model targeted at the 18-25 age group. Within the game scenario, an image is presented of a special limited edition version of Ford's new car with a time sensitive promotion for purchase. Here, the user (200a) employs a cell phone or other digital device (200b) to scan and capture (205c) a 3D image presented on the display screen or other multi-modal artifact associated with 3D-VR game (209), which is associated to the Video game console for purposes of showing the game in progress.

[0105] The 3D image on the digital device reflects a special promotion tied to Ford's new car. The user engages the 3D image to launch the URL for Ford's website, set-up specifically for the promotion.

[0106] The Scan Message, including the 3D image capture, is sent to a server 201 via the cell phone's data connection (e.g., wireless internet systems such as lxEV-DO and the like), and the server queries a database of car profiles and promotions tied to the Scan Message (230c) to determine what, if any, information in the 3D image may be relevant to the preparation of the current Actionable Event. Here, the server selects and retrieves (235 c) the appropriate content to include in the Actionable Event, comprised of purchasing events, promotions, discount codes, special offers (237c), and an Actionable Event to provide a URL to Ford's website for the specific promotion tied to Ford's car (238c), including a Dealer Network option to allow the user to select the nearest dealer associated with the special promotion (250c), and the user selects the dealer for contact, follow up, and purchasing inquiries (251c) utilizing the digital device.

SPACE-TIME TRAJECTORIES - NO SCAN

[0107] In one embodiment, the CTIS can serve information to the user based on the collected knowledge of user space-time trajectories contained in the user profile, without the need for any initiating code or multi-modal artifact scan. This embodiment is best illustrated by example, as drawn in FIG. 3. Here, a user rides a bus from home (300) to work (310) on four of the last five days, and takes a different bus to his doctor's office (330) on the remaining day. The space-time trajectories for the four work trips (305) and the one doctor trip (325) are determined by codes that the user scans with his cell phone from a newspaper during the bus rides. In this example, the user is assumed to scan a code every ten minutes starting at 7:30 AM until the destination is reached, and the geographic location of each scan is indicated by the position of the timestamp along each trajectory. In one embodiment, tracking occurs without scanning, simply by having the device send geocodes and timestamps by itself. Along the way to his destinations, the user passes a number of Actionable Event or 3^rd party providers (marked with X's) with Actionable Event or 3^rd party provider content in the CTIS Information Base, with dotted circles indicating the locus of points that are a distance of two miles from each provider. One Actionable Event or 3^r party provider in particular (315), a gourmet coffee shop, is within two miles (320) of the user's space- time trajectory at 7:50 AM in four of the last five days. The user's profile also lists "coffee" as an interest, both based on user specifications during registration and on the user's advertisement and article scan record. Consequently, on the sixth day, the server supplies the user with an Actionable Event for the coffee shop (315) at 7:45 AM to purchase a coffee or food item for delivery or pickup.

Code Triggered Information Server (CTIS) Detailed Overview

User and 3^rd Party Provider Participation

[0108] The CTIS may be enabled by the participation of 3^rd party/ Actionable Event providers and users. In one embodiment, 3^rd party providers supply Actionable Events for storage on the CTIS server. In an alternative embodiment, the Actionable Events are stored on third party servers that are referenced by the CTIS server. The user experience may be enhanced by the incorporation of personal preferences and demographic information to optimize the targeting of Actionable Event content. Consequently, one embodiment would incorporate a user registration procedure. Users may register for the CTIS via the internet, telephone, postal mail, and/or the like. If a particular embodiment of the CTIS requires client software for the user to install, this could be supplied upon successful user registration. FIG. 4 shows an exemplary embodiment of a user registration interface 400, wherein a user can input Basic Info 405 (e.g., member ID, password, secret question/answer, e-mail address, etc.), Device Info 410 (e.g., mobile phone carrier, phone manufacturer & model, operating system, etc.), Demographics 415 (e.g., gender, year of birth, zip code, country of residence, household income, job title, industry, etc.), and Content Preferences 420 (e.g., interests, subscriptions, content delivery options, personal history options, cache options, geo-tracking options, actionable event delivery options, etc.) in a plurality of fields 425. In an alternative embodiment, a basic user identification packet (e.g., a "cookie") is sent with a user's first code scan, thus establishing a basic user profile that can be filled in with greater detail at a later time.

Artifact and Code Capture (105)

[0109] Scannable codes and multi-modal artifacts may be found in a variety of media expressions, including but not limited to newspaper and magazine articles, signs and billboards, flyers, store locations and kiosks, consumer products, packages, clothing, stickers, websites, software displays, television broadcasts, virtual worlds, and/or the like. The nature of the code and multi-modal artifact and associated method with which the code or multi-modal artifact can be scanned will depend to some extent on where the code or multi-modal artifact is found, but among the possible types of codes are standard barcodes (e.g., UPC, EAN, Code 128, etc.), 2D matrix codes (e.g., QR code, Semacode, Data Glyphs, mCodes, etc.), user-input codes (e.g., text messages), RFID tags, NFC tags, pure images (e.g., an image of a sign to be analyzed by optical character recognition), website links, software inputs, and/or the like. These codes may be located on any medium (e.g., on newspapers, magazines, books, video content, computer screens, embedded in objects (e.g., RFID tags on clothes, etc.), and/or the like). One embodiment employs 2D matrix codes, examples of which are provided in FIG. 5a, due to their large data capacity, ease of scanning, and resilience to damage or error. These examples include QR codes 500, Mcodes 505, Semacodes 510, and/or the like.

Artifact or Code Enhancement

[0110] In scanning a code or artifact, the code image may not always be recorded faithfully and/or in an ideal and/or suitable condition for decoding and/or matching to associated content.

Consequently, the CTIS may be configured to enhance and/or otherwise process a code image in order to better recognize the code. FIGS. 5b-c show implementations of logic flow for processing and/or managing non-ideal code scans in embodiments of CTIS operation. In FIG. 5b, a scanned code is received at the CTIS server (513) and a first attempt is made to decode a received code scan (515). In one implementation, a received code scan may take the form of a bitmap image and/or a compressed bitmap image. If the decoding is successful (520), then the code is matched to associated content (525) that is ultimately destined to be served to a user. Otherwise, a

determination is made as to whether enhancement should be applied to the scanned code (530). If not, then an error message and/or suggestions on how to improve the quality of the scanned code may be returned to the user (535). The CTIS may decide not to apply enhancement, for example, if enhancement has already been applied unsuccessfully to the same image before or if the quality of the image is so poor as to be clearly beyond improvement. If enhancement is to be applied, then the CTIS determines the appropriate enhancement type (540) based, for example, on the quality and character of the scanned code image and subsequently applies the enhancement (545), after which a new attempt is made to decode the scanned code.

[0111] In some implementations, multiple enhancement processes may be applied against a scanned code and the results compared in order to determine the encoded content. Prior to enhancement, the image may be downsized if necessary for speed of processing in applying multiple enhancements against the image. Image quality may also be increased if necessary in order to more successfully apply enhancements. For code images having extraneous information and/or noise, an enhancement process may be applied to remove all extraneous information, despeckle, and/or the like. For code images having shades of gray and/or shadows obscuring the encoded content, an enhancement process may be applied to manipulate the contrast and/or brightness of the image in order to create a pure black and white image for cleaner decoding. For code images that are tilted and/or not taken flat in a plane, an enhancement process may be applied to tilt the plane of the code to create a flat plane to aid in successful decoding. For code images that are rotated or skewed, an enhancement process may be applied to de-rotate, deskew, and/or otherwise recreate proper rectangular pixel groupings that can be decoded. For code images that are taken out of focus, an enhancement process may be applied to sharpen the image into recognizable pixel groupings. Additionally, the CTIS may apply edge recognition processes to the code image to create a set of potential code images that can be processed and compared to records in a code database.

Partial Decoding

[0112] In some cases in which full decoding of a scanned code image is impossible, a partial decoding may be accomplished and serving of content achieved on that basis. A partial decoding may, for example, result in a probabilistic code match without ever successfully decoding the complete code. FIG. 5c shows an implementation of logic flow for partial decoding in one embodiment of CTIS operation. A scanned code image is received (548), partially decoded (550), and the partially decoded content used to query a code database to find matching codes (555). A determination is made if any matches exist (560) and, if not, then the CTIS may return an error message to the user and/or make suggestions about how to improve the image of the scanned code (565). If a match is found, however, then a determination is made as to whether there are multiple matches (570). If so, then the CTIS may consider ancillary factors (575) in attempting to isolate a single code as the most likely match to the received code. Such ancillary factors may include, but are not limited to: the incoming phone number, the carrier, the time of day, current code traffic, the source of the code, the context of the code, the user ID, the hardware ID, the user behavioral history, the user location, and/or the like. In considering ancillary factors, the CTIS is thus capable of predicting the true code based on knowledge of the user scanning the code, the context of the code, ambient factors, and/or the like. Once a unique code has been identified, this is designated as the matching code (580) and the CTIS proceeds with determining the appropriate associated content to serve to the user.

[0113] In an illustrative example of partial decoding, a given matrix code may encode the string

ABC123. In multiple processes, it may be determined that the code contains A 1 3 (where the blanks represent unknown values), _C1_, and AB_. Through these multiple processes, the CTIS system now knows that the code contains ABC 1 3 and may check this against codes in a code database to determine how many matches exist. If it finds two matching codes (e.g., ABC 123 and ABC 143), it may consider ancillary factors. For example, ABC 123 may correspond to an advertisement for a new action movie while ABC 143 corresponds to a coupon for women's cosmetics. If the user supplying the code is determined, based on a supplied user ID, to be a man with a history of scanning movie-related codes, then the CTIS system may infer that ABC 123 is the matching code and serve the corresponding content. In one implementation, the CTIS server may further supply a confirmation request for partially matched codes, requesting the user to affirm or deny that the supplied content is in fact the content that he or she was seeking.

Scan Message Data Structure (115)

[0114] The Scan Message generated by the CTIS is, itself, a novel data structure. In one embodiment that is illustrated in FIG. 6, it contains an identification field (605) comprising a unique user ID (605a) and a hardware ID (605b), both automatically supplied by the device used to scan and transmit the scanned code or multi-modal artifact; a coordinate field (610) comprising the location geocode (610a) (e.g., GPS coordinate, latitude and longitude, city and state, etc.) and timestamp (610b) of the scan; and a code or multi-modal artifact field (615) supplying the specific content of the scanned code or multi-modal artifact, which may include a subject code or multi- modal artifact (615 a) for the code or multi-modal artifact context, a source code or multi-modal artifact (615b) indicating where the code or multi-modal artifact was located, and a content code or multi-modal artifact (615c) indicating the specific information, if any, to be supplied in response to a scan of that code or multi-modal artifact. The source 615b of the code or multimodal artifact may be from any media expression, including television, radio, and the like.

[0115] In one embodiment, the XML for the Scan Message takes the following form:

<Scan_Message> <ID>

<User ID> 012345 </User ID>

<Hardware ID> Nokia 660 </Hardware ID>

</ID>

<Coordinates>

<Geocode> GPS 40 46.516 -73 57.98 </Geocode>

<Timestamp> 08/21/2006 13 :45 :28 </Timestamp>

</Coordinates>

<Code or multi-modal artifact >

<Subject> entertainment; movies; comedy </Subject>

<Source> New York Times Media Expression</Source>

<Content> Show times for latest popular comedy movie </Content>

</Code or multi-modal artifact >

</Scan_Message>

Data Transmission (120, 125, 145, 150)

[0116] In one embodiment, sending and receiving the Scan Message and Actionable Event will employ standard data transmission techniques to ensure successful transmission and to preserve data integrity (e.g., TCP/IP, lxEV-DO, etc.). This is relevant for the Scan Message, which will contain the coded information scanned by the user. Such techniques may include but are not limited to the use of standard transmission protocols, "handshaking," data compression, data encryption, and error correction.

User Profile Structure (130)

[0117] The content of the user profile generated by the CTIS is itself a novel data structure. In one embodiment illustrated in FIG. 7a, the profile (700), which is uniquely specified by the user ID, will contain a category of quasi-static user information (705) that is generally fixed over time and one of dynamic user information (730) that is updated with each successive Scan Message that the user submits. The quasi-static info (the "quasi" qualifier indicates that this information may be updated, for example if a user changes their mobile device, however it does not change with each successive code scan) may include a hardware ID (710); "census" info (715) such as name, address, phone number, e-mail address, age, sex, race, marital status, number and age of children, job title, annual income, etc.; subjects of interest specified by the user (720); and information regarding allowed methods to contact the user (725). In one embodiment, this information could be supplied by the user when registering for the CTIS service as, for example, on a website. The dynamic info in the profile may include a scan record (735), comprising the time (740), location (745), and scan code or multi-modal artifact (750) (including subject (750a), source (750b) and content (750c)); statistics related to the scan history (755); and the type of media expression 760 including TV 760a, radio 760b, or any 2-D, 3D, or 3D-VR source 760c, and/or the like. Such statistics may include but are not limited to the most popular scan subjects, most popular scan sources, user space-time trajectories, etc. In one embodiment, the profile contains a series of identifying codes distilled from the profile content (e.g., demographic category code based on census info, subject codes, source codes, etc.) that can be compared with similar codes in the Scan Message to determine the appropriate content to include in the Actionable Event. In an alternative embodiment, the profile contains identifying keywords or "tags" based on the profile content that can be compared with similar tags in the Scan Message to determine the appropriate content to include in the Actionable Event. In both cases, the identifying labels for the user profiles may be input by CTIS administrators or automatically generated by an appropriate computer algorithm such as stripping header labels from actionable event descriptions (e.g., for example, stripping the header tags from an HTML actionable event).

[0118] In one embodiment, the XML for the User Profile may take the following form:

<User>

<Quasi-static info>

<User_ID>123-45-6789</User_ID>

<Hardware ID> Nokia 660 </Hardware ID>

<Census info> John Smith; 123 Maple Dr., Smalltown, CA 92676; (123)456-7890;

jsmith@email.com; 55 years; male; white; married; 2 children; etc.

</Census info>

<Interests> camping; fishing; classic cars; movies; etc. </Interests>

<Contact restrictions> Weekdays 8 AM - 7 PM only </Contact restrictions>

</Quasi-static info>

<Dynamic info>

<Scan record>

<Scan #1> <Time> 08/21/2006 13:45:28 </Time>

<Geocode> GPS 40 46.516 -73 57.98 </Geocode>

<Code>

<Subject> entertainment; movies; comedy </Subject>

<Source> New York Times </Source>

<Content> Showtimes for latest popular comedy movie </Content>

</Code>

</Scan_#l>

<Scan_#2>

etc...

</Scan_#2>

etc...

</Scan record>

<Scan statistics>

<Popular subjects> entertainment; cars; travel; etc. </Popular subjects>

<Popular sources> New York Times; Car and Driver; CNN.com; etc. </Popular sources> <Trajectories> Data files listing time and geocode coordinates, computed average trajectories, likely locations, etc. </Trajectories>

etc...

</Scan statistics>

</Dynamic info>

<Code>

<Subject> entertainment; movies; comedy </Subject>

<Source> New York Times </Source>

<Content> Showtimes for latest popular comedy movie </Content>

</Code>

</User>

[0119] In one embodiment, data accumulated in a plurality of user profiles may be analyzed to extract information about codes that are scanned. For example, the frequency with which a particular code is scanned may be extracted from user profiles and parsed by geographic, temporal, and/or demographic criteria to yield code profiling information. This information may be stored in a code and/or Actionable Event profile. In an alternative embodiment, the code/Actionable Event profile may be constructed and/or updated by the CTIS immediately upon the receipt of a Scan Message rather than being extracted from user profiles.

Information Base (135)

Actionable Event and Labels - searching

[0120] The Information Base contains the content that may be included in the Actionable Event sent to the user and may be stored on the CTIS central server or on third party servers accessed by the CTIS. In one embodiment, each Actionable Event is associated with searchable labels. The Actionable Event content together with its labels, collectively referred to here as an Actionable Event, forms a novel data structure. In one embodiment illustrated in FIG. 8, the Actionable Event (800) is comprised of the specific Actionable Event content (805) (e.g., text, images, video, etc.) and a set of identifiers (810) including subject tags (815), information characterizing the temporal character of the Actionable Event based on which the Actionable Event may be triggered (820) (e.g., 6 to 10 AM for a breakfast Actionable Event), information characterizing the geographic specificity of the Actionable Event based on which the Actionable Event may be triggered (825), demographic specificity of the Actionable Event (830) (e.g., an Actionable Event for family vacations), and the hardware requirements of the Actionable Event (835) (e.g., hardware IDs of devices capable of displaying the content of a particular Actionable Event). In one embodiment, the Actionable Event would also contain information identifying the source in which the

Actionable Event code is to appear. In another embodiment, the Actionable Event would also contain code/ Actionable Event profile data, describing the history and/or statistics of scans related to the Actionable Event.

[0121] In one embodiment, the XML for an Actionable Event may be in the following form:

< Actionable Event >

< Actionable Event_ID> 123</ Actionable Event_ID>

< Actionable Event content>

textual movie show time listings (or images, audio, video, URLs, etc.)

</ Actionable Event content>

<Identifiers>

<Subject> entertainment; movies; comedy </Subject>

<Temporal> 08/21/2006; 12:00:00 to 14:00:00 </Temporal> <Geographic> Smalltown, CA 92676 </Geographic>

<Demographic> 17 years and older </Demographic>

<Hardware> any (text only) </Hardware>

</Identifiers>

</ Actionable Event >

STATIC CONTENT AND DYNAMIC CONTENT IN REPLY

[0122] The CTIS queries the user profile to determine which Actionable Event to include from the Information Base in the Actionable Event message in order to tailor those Actionable Events to the user's profile characteristics, interests, and trajectories. In one embodiment, a scanned code will yield two components in the Actionable Event message content: i) static content that is the same for every user who scans the code, and ii) dynamic content that depends on the context of the user and the user profile. The static content (i) is determined from the scan code, while the dynamic content (ii) is determined by a combination of the scan code and the user profile, requiring a dynamic content generation heuristic to combine their respective influences. While a variety of such heuristic are possible and contemplated as being within the scope of this disclosure, and the specific heuristic employed in a given realization will likely depend on the precise application and intention, we discuss one embodiment of this heuristic here for illustrative purposes with reference to FIG. 9. In one embodiment, the scan code and user profile is labeled with keywords or "tags" that identify the subject content of each. In an alternative embodiment, the scan code, user profile, and Actionable Events are labeled with identifying codes. In either case, the heuristic exhibited in FIG. 9 first filters all Actionable Events in the Information Base by the subject code or tag of the scanned code (900). The Actionable Events from 900 are filtered by the hardware ID (905) to ensure that the Actionable Event content can be successfully acquired and displayed by the user. The Actionable Events from 905 are filtered into separate categories by scan location (910al), user address (910a2), user specified subject interests (910bl), user scan subject history (910b2), scan time (910c), or user demographic category (910d). Actionable Events in each of these filtered categories are compared to determine which occur most commonly across 910a, b, c, & d (915). The server determines if 915 has yielded a single Actionable Event (920), and if so, it incorporates the Actionable Event's content into the Actionable Event (930). Otherwise, it chooses one

Actionable Event at random from the remaining Actionable Events (925) for incorporation. It must be emphasized again that this is but a single, specific embodiment of the user profile query heuristic intended for illustrative purposes. The heuristic is highly flexible and can be finely tuned for the specific application at hand. Some other possible methods may include alternative ordering of filters, addition or removal of filters, weighted filtering, complex conditional trees, and/or the like.

Heuristics - Filtering - Weighting

[0123] As an exemplary implementation of this embodiment of the Actionable Event filtering heuristic, we consider the situation discussed above with reference to FIG. 2A. Here, the user scans a code from an article on finance and investment in the business section of a media publication. Keywords associated with this code may include Business, Finance, Investment, etc. Consequently, only those Actionable Events possessing these identifying keywords are passed at 900. In an alternative embodiment, the keywords may be ranked in order or relevance and the Actionable Events filtered accordingly. The Actionable Events from 900 are filtered by Hardware ID; we presently assume that the user's mobile device is equipped with advanced capabilities (text, image, audio, video, internet, etc.), so that all Actionable Events are passed at 905. In an alternative embodiment, the Actionable Event filter may preferentially choose those Actionable Events that take fullest advantage of the user device's display capabilities. Of the remaining Actionable Events, there is one for a brokerage firm that is tagged with a location code

corresponding to the user's address (San Francisco, Calif), the user's specified subject interests (investing, retirement), and a portion of the user's demographic profile (age 55 or older). The three tag matches (location, subject, and demographic) distinguish this Actionable Event from the others in the Information Base, which, in this example, all have two or fewer tag matches; so this unique Actionable Event, a brokerage firm located in San Francisco, Calif, with the option to purchase retirement services, is chosen to be served to the user's mobile device.

RELEVANCE RATING

[0124] In one embodiment, the user may respond to the Actionable Event with a relevance rating. This rating may then be stored in the user profile and/or employed to refine the criteria for tailoring future Actionable Events to that user.

[0125] In one embodiment, the Actionable Events would not be filtered by hardware ID at 905. Rather, when a final Actionable Event has been selected for inclusion in the Actionable Event following filtration under the other relevant criteria, the server will determine whether that

Actionable Event is properly formatted for the device specified by the hardware ID. If so, then the Actionable Event will be included in the Actionable Event as is. Otherwise, it will be converted into the required format appropriate to the user's mobile device. For example, if the selected Actionable Event contains color images but the server determines that the user's hardware device is an early model BlackBerry that only accommodates monochromatic text; the server will implement a peripheral application to strip the text content from the Actionable Event and incorporate only that in the Actionable Event. In an alternative embodiment, if the user's digital device is not capable of interacting via NFC, NFC-based Actionable Events will be filtered out and not served to the user's device.

Non-Triggered Actionable Event Serving

[0126] Though Actionable Events served by the CTIS are determined by codes scanned at some time in the past, it is possible for the CTIS to serve Actionable Events without any immediately contiguous, causal code scan. A general illustration of this capability was given above in FIG. 3, where each Actionable Event provider has specific geographic locations of operation. These are assumed to be stores/restaurants for the present purposes of illustration in this disclosure, but they may have different character for different embodiments and applications, including but not limited to kiosks, signs and billboards, offices, hospitals, real estate, shelters, emergency service providers, traffic monitors, individuals, websites, virtual world locations, and/or the like. A chosen distance surrounding each Actionable Events provider location defines a "provider zone," where the metric by which "distance" is gauged may be generalized in the case of cyberspace/virtual world locations. A provider zone breach occurs when a user scans a code within a provider zone. This may be regardless of the content of the scanned code, for in this example the goal is to record the user's space-time position. In an alternative embodiment, a provider zone breach can be registered even if a user does not scan a code within the provider zone, if the CTIS can infer a breach from two or more code scans outside the provider zone through interpolation. When such a provider zone breach occurs, a special note may be incorporated into the user profile, such as in the Scan statistics module (755), indicating the time and the particular Actionable Event provider.

[0127] In an alternative implementation, the CTIS may be configured to automatically query a user position without scanned codes. For example, a user's mobile device may be configured to periodically register its GPS location with the CTIS system. In another implementation, the CTIS may be able to continuously monitor the position of a client mobile device based on the cell site or tower to which the device is communicatively coupled. Cell site based user location may also be registered when a user scans a code.

[0128] From an accumulated record of these breaches, data of the form of FIG. 10 can be determined. FIG. 10 shows a histogram (1000) of provider zone breaches in 20 minute intervals for a particular user over a chosen time scale, which we will take to be two months in this example. In one embodiment, such histograms will correspond to breaches of specific Actionable Events provider locations; while in an alternative embodiment, they will correspond to breaches of any of a plurality of locations corresponding to a specific Actionable Events provider; while in still another embodiment, they will correspond to breaches of different locations of related but different Actionable Events providers, chosen to target a specific collective user demographic. In this exemplary histogram, the significant provider zone breaches (those exceeding the threshold) are clustered around two times: one approximately centered at 9 AM (1005), and the other at 6 PM (1010). This may represent, for example, a user scanning codes as he passes the same Actionable Event provider location while riding a bus to and from work. In one embodiment, a threshold breach level (1015), indicated in FIG. 10 by a horizontal dotted line, is specified such that all breaches below this threshold number are ignored. This threshold level should be specified with respect to the time scale of the histogram; in the present example, the threshold is 3 breaches per two month period. The average peak times (1020, 1025) of the breach clusters that exceed this threshold are computed (marked by X's in FIG. 10), and a time is chosen prior to those averages (1030, 1035) to serve an appropriate Actionable Event to the user. In one embodiment, provider zone breaches could be sorted by scan subject, so that a histogram of the form of FIG. 10 could be constructed for a specific category of scans in order to further tailor the supplied Actionable Event content.

TRAJECTORIES IN CYBERSPACE-TIME

[0129] In another embodiment, the system may be generalized to analyze trajectories in

"cyberspace-time." In this embodiment, a record of trajectories is kept of the user's activity on the world wide web as browsed, for example, on the user's internet enabled mobile device. A generalized concept of "distance" may be incorporated, for example reflecting an average number of links needed to get from one website to another. Provider locations become websites in this embodiment, and a zone breach may be registered when the user is a specified number of links away from the provider's website. Then, just as before, the CTIS can anticipate user cyberspace- time trajectories and supply Actionable Events, coupons, pop-ups, etc. related to websites that the user is approaching. For example, the CTIS may track that a user visits web site A at 9:00 AM and web site B at 9:20 AM every day. The CTIS may then provide ads to website A at 8:50 AM and to website B at 9: 10 that are targeted to that user. In an alternative embodiment, the CTIS may provide this targeting information to existing ad serving systems, which in turn may use the targeting information for timely placement of Actionable Events. The aggregation of such targeting information, in particular, may be useful to advertisers and website owners. For example, reports may be run on user profiles sorted by location (e.g., web sites) and time of visit. This information may be supplied, along with user profile IDs, which may then be used by the advertising servers and/or websites to pre-cache Actionable Events that are relevant to the users' interest just prior to their anticipated visits to the website. As such, the CTIS works in conjunction with certain websites to supply user- specific Actionable Events on those sites. As a result, the content of the user profile within the CTIS may be used to select Actionable Events that the user browses using their mobile device. In an alternative embodiment, web-based Actionable Events may be specific to the trajectories of users in geographic space-time, since that information is included in the user profile. For example, if a user is known to be passing by a coffee shop at a specific location and time every day (e.g., a coffee shop on 123 Main Street, NY, N.Y. at 9:00 AM), and is also know to visit a specific website at that time (e.g., at 9:00 AM the user visits website C reading the daily news on their walk to work), then the system may provide the user with Actionable Events directly to their PDA or embedded in website C (because it was pre- loaded as already discussed) for the coffee shop. As such, the CTIS provides a mechanism to provide geographically relevant Actionable Events to traditional websites. In yet another embodiment, the total content of the websites browsed by a user on their mobile device may be modified by the content of the user profile within the CTIS. For example, a news website may select articles for a user that are specific to the user profile, including their interests,

demographics, current time and location, etc.

VIRTUAL SPACE

[0130] In another embodiment, the CTIS may analyze the user trajectories in "virtual worlds" or "synthetic realities" and serve Actionable Events accordingly. An example of a virtual world is a massively multiplayer online game, such as The Sims Online, EverQuest, World of Warcraft, Second Life, and/or the like. In such a virtual environment, a code scan may be comprised of a particular user action or decision such as initiating or completing a game mission, clicking on an icon, saving a game, etc. In such virtual worlds, geographies and virtual locations may be used in employing trajectories. The history of user activity within the virtual world may then be incorporated into the user profile and used by the CTIS to serve Actionable Events within the context of the virtual world or elsewhere. For example, in games like Second Life, virtual televisions, billboards, etc. may be used as Actionable Event delivery mechanisms.

[0131] Selecting the proper Actionable Event to serve to the user in the non-triggered Actionable Event serving embodiment of the CTIS may be accomplished using a variety of heuristics that are best tailored to suit the specific use or application. Nevertheless, we present an example of a specific embodiment in FIG. 11a for illustrative purposes. In this embodiment, a weighting system may be employed to select among the Actionable Events of a specific Actionable Event provider once it is established that an Actionable Event from this provider is to be served to the user based on the analysis of FIG. 10. This weighting system begins by assigning a score of 1 (1100) to all Actionable Events in the Information Base corresponding to the provider. This score is multiplied by a subject coefficient (1105) for every match between the Actionable Event subject and the user specified subject interest or scan subject history. That coefficient is 2 in the present example, so N matches would yield a weighting factor of 2^N. The Actionable Event score may also be multiplied by a time coefficient (1110) if the serving time (1030 or 1035) falls within a time range specified for a particular Actionable Event. In the present example, the time coefficient is 4, expressing the greater importance of this factor compared to a single subject match. Finally, the Actionable Event score may be multiplied by a demographic coefficient (1115) for every match between the Actionable Event demographic category and that of the user. That coefficient is 1.5 in the present example, reflecting the diminished importance of this factor compared to the time and subject considerations. M demographic category matches will yield a weighting factor of 1.5^M. Once all of the Actionable Event provider's Actionable Events are scored in this fashion, those Actionable Events with the highest scores for that provider may be selected over the rest (1120). Similarly, the highest scoring Actionable Events from other providers are selected and the highest scoring Actionable Events over all providers can be selected to serve for a particular time interval (1125). Finally, the proper time to serve the Actionable Event (e.g., 30 minutes prior to the expected average provider zone breach time) is established (1130) and, if there is no further ambiguity (1135), the proper Actionable Event is served at that time (1145). Any residual ambiguity may be resolved by selecting randomly from the remaining Actionable Events (1140). SELECTING ACTIONABLE EVENTS

[0132] As a specific example of how this might work, we assume that the histogram (1000) of FIG. 10 represents provider zone breaches for a fast-food restaurant that a user passes on his way to and from work. An Actionable Event in the restaurant's base of provided Actionable Events matches the user's stated interest in gourmet coffee, and this Actionable Event is further labeled by a time identifier indicating that it is best served between 8 AM and 10 AM. Consequently, at 30 minutes before the user's average breach time of the provider's zone, the server supplies the Actionable Event to the user's mobile device, touting this restaurant's new line of gourmet coffee drinks. Alternatively, the user's profile may specify that he is married with two children. This matches the demographic category of an Actionable Event in the provider's base which is also labeled by a time identifier indicating that it is best served between 5 PM and 8 PM. As a result, 30 minutes before the user breaches the provider's zone on his way home from work, the server supplies the Actionable Event to the user's mobile device, touting their family-pack of

hamburgers.

MARKETING RULES

[0133] In an alternative embodiment, the CTIS may be configured to receive and implement marketer rules for serving Actionable Events. FIG. lib shows a schematic illustration of data structures related to establishment of a marketer rule for Actionable Event serving in one embodiment of CTIS operation. User profile data sets (1147) may include a variety of data fields related to user activities, interests, characteristics, and/or the like, such as but not limited to:

geographic visits (1149), as may be recorded by code scans; time (1151) associated with code scans; interest contacts (1153), such as may be reflect the user's interests and may be based on the context and/or associated content of scanned codes or multi-modal artifacts; gender, age, demographic, and/or other personal information (1155); peer groups and user preferences (1157); and/or the like. Server data pulls (1158) may reference one or more data feeds to which the CTIS server has access in order to draw information that may be pertinent to marketer rules and/or Actionable Event serving procedures. A wide variety of different types of feeds may be accessible, including but not limited to: global newsfeeds (1159), such as associated press wire services, weather information, sports scores, political and social events, financial market data, and/or the like; category sector feeds (1161), such as more specific and/or special interest news feeds;

marketer specified feeds (1163); past Reply Messages (1164); and/or the like. A marketer rule (1165) may be established by selecting one or more fields (1167) from the user profile data sets (1147) and/or the server data pulls (1158) and supplying associated parameters (1165) and/or parameter values associated with those fields. In the illustrated implementation, the rule associates a Petroleum Gazette field (1169) with a parameter specifying more than three oil articles read per day (1175). In addition, the rule associates a gas station visitation field (1171) with a parameter specifying more than three visits per week (1177). If both of these conditions are deemed true for a given CTIS user, then the CTIS may serve one or more Actionable Events selected for and/or associated with the marketer rule. A price-per- impression (1179) may be charged for each time that the Actionable Event is served to a user under the circumstances specified by the rule. In one implementation, a marketer may propose a price-per-impression when he or she creates the rule, which is then submitted to the CTIS and/or a CTIS administrator for approval. In another implementation, the CTIS may automatically generate a price-per-impression based on the rule and/or the marketer. For example, a fixed charge may be applied for each additional field and associated parameter added to the rule, as this refines the specificity with which the associated Actionable Events will be served and increases the likelihood of transactional consummation. Once a rule is complete, it may be entered for storage and subsequent use in a marketer rules database (1181). It should be understood that the functionality described for this embodiment of the CTIS may be integrated or employed within any of the other CTIS embodiments described herein.

[0134] In one embodiment, the XML for the above marketer rule may take a form similar to the following:

<Rule>

<Rule_ID> 123</Rule_ID>

<Rule_name> Oil Rule </Rule_name>

<Rule_owner> Ads R Us </Rule_owner>

<Conditionl>

<Fieldl> Petroleum Gazette </Fieldl>

<Parameterl> 4 or more oil articles/day </Parameterl>

</Conditionl>

<Condition2>

<Field2> Gas Station </Field2> <Parameter2> 4 or more visits per week </Parameter2>

</Condition2>

< Actionable Event _database> Hybrid car Ads </ Actionable Event _database>

<Price>

<Amount> $1.00 </Amount>

<Contingency> Impression </Contingency>

</Price>

</Rule>

[0135] In one implementation, the elements of FIG. lib may form the basis of a user interface for marketer rule generation. A marketer and/or other CTIS user may be presented with lists of user profile data sets (1147) and server data pulls (1158) that may be dragged and dropped to the marketer rule box (1 165) to generate a marketer rule. When the parameter values have been specified and the rule is complete, the user may click OK to submit the rule and/or drag and drop the rule to a marketer rules database icon (1181).

[0136] FIG. 11c shows an implementation of logic flow for generating a rule queue in one embodiment of CTIS operation. A rule is instantiated (1182), and the CTIS server checks parameter quanta associated with the rule (1183). A runtime quantum is discerned for the rule (1184) to determine the timing for applying the rule. The rule is subsequently parsed to generate the appropriate query structure (1185), and the cron schedule for the rule is established based on the runtime quantum (1186). The rule may then be pushed to the cron schedule queue (1187), and a determination made as to whether there are any additional new rules to process (1188).

[0137] FIG. lid shows an implementation of logic flow for cron job queue management in one embodiment of CTIS operation. The CTIS may monitor the cron job queue and, for each queued cron job associated with a rule that is next in the queue (1189), the CTIS may determine whether or not it is time for that particular cron job to run (1190). If not, then the CTIS passes to the next cron job in the queue (1191). On the other hand, if it is time to run, then the appropriate databases specified in the rule are queried, and the particular cron job is dequeued (1192). If the results of the database query match the rule criteria so as to require serving an Actionable Event impression to a user, then that match indication may be passed to Actionable Event selection modules in order to choose an Actionable Event to supply to the user. In one implementation, the marketer rule itself may uniquely determine an Actionable Event to serve. In another implementation, the marketer rule may determine a set or class of Actionable Events to serve, from which a unique Actionable Event may then be randomly selected. In yet another implementation, as Actionable Event may be selected from the marketer rule determined set or class of Ads based on a user profile.

[0138] It must be emphasized again that this is but a single, specific embodiment of the non- triggered Actionable Event serving procedure intended for illustrative purposes. The procedure is highly flexible and can be finely tuned for the specific application at hand. Some other possible methods may include different relative weighting factors, alternative weighting considerations, categorical filtering, consideration of the number of provider zone breaches above the threshold level, graduated consideration of the distance from the provider location, modified provider zone shapes, pattern recognition algorithms, artificial intelligence facilitation, and/or the like.

COUPON AND CONTENT SHARING

[0139] In another embodiment, the CTIS provides targeted coupons to consumers. By scanning a code or multi-modal artifact in an article or advertisement of interest, a user may be provided with an Actionable Event providing a coupon code for a store or establishment near the location where the scan took place, thereby increasing the likelihood that the coupon would be used. Choosing the coupon based on the contents of the user profile may further increase the likelihood of a consummated transaction.

[0140] FIGS. 12a-b show two examples wherein the CTIS supplies coupons and/or discount info to a user. FIG. 12a shows a Media Expression or commercial 1200 for hay fever medication that contains a scan code or multi-modal artifact. Upon scanning the code or multi-modal artifact, the user's cell phone is supplied with a purchase option for Drug X 1205 for 25% off the advertised medication, along with a nearby store at which to pickup the drug after purchase. FIG. 12b shows a media expression related to events and nightlife in New York City 1215 that contains a scan code or multi-modal artifact. Upon scanning the code or multi-modal artifact to obtain additional content or actionable events related to NYC events and nightlife, the user may also be provided with a contextual actionable event for a local bar 1220, including an announcement of happy hour discount prices based on the time of the scan. In an alternative embodiment, the user may purchase a drink for nearby friend or a coupon could be provided to the user based on an anticipated provider zone breach rather than a contiguous, causal code scan.

PRODUCT INFORMATION [0141] In another embodiment, the CTIS can be used to provide information regarding consumer products that cannot be included on the product packaging. By scanning a code printed on a consumer product package, the user could be provided with nutritional information, ingredients, website links, coupons, etc. Additional advertisements could also be served based on the nature of the product scanned and the user profile. For example, a user who scans a container of mustard and has "grilling" listed among his/her interests may be served an advertisement and/or coupon for hot dogs in addition to the mustard information.

[0142] The CTIS provides a targeted information serving system that may be applied to a wide variety of marketing and information dispensation applications. In one embodiment, the CTIS creates enhanced Actionable Events with content that cannot be included in a print ad. For example, a user who scans a code related to an automobile ad is provided additional content pertaining to the automobile such as detailed images, video, audio, reviews, and links to additional information. Based on the geocode provided in the Scan Message, the CTIS may provide a list of local dealerships and pricing. Based on the user profile, the CTIS may suggest other automobiles by the same manufacturer that are popular with customers that share the same income bracket, geographic address, number of children, etc. as the user.

COORDINATING PERSONNEL BASED ON PROFILES

[0143] In an embodiment, personnel directives may be served that are targeted based on characteristic profiles, behavioral patterns, and/or contextual surroundings and situations. A project manager may deliver personnel directives based on user activities, such as may be registered by codes or multi-modal artifacts scanned by mobile devices, as well as user characteristics and an accumulated history of user activities (e.g., code scans). The scans, together with geographic, temporal, and user-specific information, are obtained by the server that receives, processes, and records the message. Based on these messages and a user profile—which may include continuously updated user-specific behavior information, situational and ambient information, an accumulated history of scanned code messages, and integration with outside database information—the server selects information to serve to the users' mobile devices from an information base. For example, a user who scans a barcode for a house-building project with his/her cell phone and whose profile lists his/her role as "mason" may be directed to begin mixing mortar. When he/she is finished, he/she may scan a completion barcode to receive the next directive (e.g., to begin laying bricks). In one embodiment, the code triggered information server allows managers to supply product information and/or personnel directives to users by selecting content based on a combination of the context of the code that the user scanned, the user's status and/or employment position as recorded in his or her user profile, the user's demographic profile, situational information, and a record of scanned codes. These factors can be analyzed and employed by the CTIS for both user tracking and behavioral profiling/targeting. In another embodiment, the code triggered information server may supply personnel directives solely based on user profiles with no immediately contiguous code scans. For example, the system may predict user trajectories based on the collected history of code scans, including the locations where the scans occurred. Then, based on these predicted trajectories, it may direct all users with

"emergency service personnel" in their profiles who are projected to be in the vicinity of an emergency situation to report for duty. It may even provide more detailed directives to each specific type of emergency service personnel based on the title and experience level listed in each user's profile, and on the specific location of the user with respect to the emergency situation.

[0144] In an embodiment, the CTIS allows managers to obtain better operational success rates by supplying personnel directives to users that are specific to the users' current geographic locations at specific times of the day. In another embodiment, the CTIS provides rapid and evolving geographic and historical statistical profiling of user progress and behavior. In another

embodiment, the CTIS provides an expanded platform to supply large quantities of information and/or personnel directives with a minimal amount of publication space.

[0145] From the user point of view, the CTIS provides a mechanism by which, in exchange for scanning a code and providing basic information, he/she can receive the most targeted information and/or personnel directives relevant to his/her role in the collective operation. At the same time the system grants a manager a powerful tracking tool, to monitor roles, accomplishments, failures, and behavioral patterns of users on individual, demographic, temporal, and geographic scales. This allows, among other things, the deployment of anticipatory information/personnel directives that are served to users in anticipation of their locations and/or operational progress based on an accumulated history of user tracking data.

[0146] Managers participate by contributing content to an Information Base that categorizes and labels that content under a number of considerations, such as subject matter, geographic and temporal identity, status/role specificity, demographic specificity, and/or the like. Elements of the Information Base can then be compared via these labels with comparable labels found in a database of user profiles in order to match content with users via subject, location, time, demographic category, and/or the like. As such, the CTIS efficiently facilitates project management by providing personnel directives to users with highly tailored specificity.

CODES TO EMPLOYEES

[0147] In another embodiment, the CTIS provides a mechanism of instituting plans for large-scale coordination of personnel using a single coded broadcast. High-level officials could determine which of a number of coordination plans to institute, each being associated with a unique 2D matrix code. When a plan is decided upon, the code is broadcast (such as over television or the internet) and lower-level employees scan the code to obtain instructions, diagrams, maps, etc. that are tailored to the employees' specific profile contents (i.e., job, skills, location, etc.). There may also be authorization information included in the employee profiles that determines whether they are allowed to decode the broadcast. As each employee progresses in their role within the plan, additional codes could be provided to the employee to scan, both to update their tasks and to track their progress, space-time trajectories, etc.

EMPLOYEE TRACKING

[0148] In another embodiment, the CTIS provides a means of employee tracking. For example, tasks are supplied to employees in the form of 2D matrix codes. When a code is scanned by a given employee, the scan is recorded in the employee's profile and the employee is provided with instructions specific to that task and their position within the organization. When the task is completed, the employee is provided with a completion code that, when scanned, records the employee's successful completion of that task and provides instructions for the next task and/or reward information. In this manner, a record of employee accomplishments is kept as well as information regarding the time taken by the employee to accomplish various tasks. A statistical record of an employee's experience by subject matter may also be generated and included in his/her profile using the CTIS.

VIRTUAL WORLD EMBODIMENTS

[0149] In embodiments, a user profile is used to provide information querying and serving in a virtual world. Such virtual worlds may include, for example, massively multiplayer online games like The Sims Online, Everquest, World of Warcraft, Second Life, and/or the like. Information and/or advertisement providers may use a code triggered information server to serve context, demographic, and behavior targeted information to users in a virtual world. Users, in turn, trigger the provision of information by scanning or observing codes or information, or by making decisions within a virtual world such as attempting a mission within a game. The triggers, together with virtual world geographic, temporal, and user-specific information, are obtained by the server that receives, processes, and records the message. Based on these messages and a user profile— which may include continuously updated user-specific behavior information, situational and ambient information, an accumulated history of triggers, and integration with outside database information—the server selects information to serve to the user in a virtual world from an information base. For example, a user in Second Life who likes modern clothing for his/her avatar may be presented with ads on virtual billboards from advertisers of virtual in-game clothing stores that are near his/her virtual position. In another example, an advertisement tailored to a user's particular interests or behavioral patterns may be placed at a location within a virtual world that the user is known to routinely pass.

[0150] In one embodiment, the CTIS allows advertisers to supply product information in a virtual world to the consumers who would be most interested in such ads; this may be achieved by the CTIS selecting advertising content based on a combination of the context of the code that the consumer scanned, the consumer's stated interests as recorded in his or her user profile, the consumer's demographic profile, a decision made within a virtual world, situational information, and a record of scanned codes and/or virtual world decisions; these factors can be analyzed and employed by the CTIS for both user tracking and behavioral profiling/targeting. For example, a player in The Sims Online who frequently acquires new furniture for his/her virtual apartment and has a stated interest in modern art may be served advertisements for an actual furniture store that sells modern furniture and is near the user's geographic location following one such virtual furniture acquisition. In another example, a user in Second Life that likes modern clothing for his/her character may be presented with ads on virtual billboards from advertisers of virtual in- game clothing stores that are nearby the user's virtual position (or offer a virtual transportation link that will instantly move the user's avatar to the virtual in-game store).

[0151] The CTIS delivers targeted Actionable Events to users in a virtual world while

simultaneously providing detailed customer tracking information to Actionable Event providers. The CTIS supplies Actionable Event providers with greater precision and focus to provide consumers with the information that is most relevant to their interests, behavioral patterns, and space-time and/or virtual world trajectories to maximize the possibility of consummating a transaction. In one embodiment, the CTIS is designed to allow advertisers to place ads at virtual world locations at specific times of the day such as to increase the likelihood of garnering the attention of prospective customers. In another embodiment, the CTIS yields time and virtual world location resolved records of scans for the determination of coded advertisement visibility. In another embodiment, the CTIS provides rapid and evolving virtual world geographic and historical statistical profiling of user interest and coded advertisement quality. In another embodiment, the CTIS provides an expanded platform to supply large quantities of information and content with a minimal amount of publication space.

[0152] From the user point of view, the CTIS provides a mechanism by which, in exchange for officially registering interest and providing basic information, he/she can receive the most targeted solicitations, enhanced content, sales offers/coupons, and/or the like within a virtual world. User benefits include: relevant and targeted advertising; mobile context specific information;

contiguous location and/or immediate event information; coupons and/or offers for many types of goods and services; and/or the like.

[0153] At the same time, the system grants the advertiser a powerful tracking tool, to monitor behavioral patterns of consumers on individual, demographic, temporal, and virtual world geographic scales. This allows, among other things, the deployment of anticipatory advertising: serving advertisements to users in anticipation of their locations and/or potential interests based on an accumulated history of user tracking data. Other Actionable Event provider benefits include: compilation of highly specific customer tracking information, including customer virtual world trajectories; virtual world location and/or time specific advertising; highly flexible coded advertising placement; quick and updated determination of coded advertisement visibility; and/or the like.

[0154] Of benefit to both users and Actionable Event providers is the fact that users improve the specificity of their Actionable Events with each successive scan, since each scan further refines the contents of the user profile. Users thus have a motivation to scan as many codes or multimodal artifacts as possible related to subjects of interest and will thereby be exposed to more advertisements than in more traditional, passive advertising and marketing schemes.

[0155] Actionable Event providers participate by contributing content to an Information Base that categorizes and labels that content under a number of considerations, such as subject matter, virtual world geographic and temporal identity, demographic specificity, and/or the like. Elements of the Information Base can then be compared via these labels with comparable labels found in a database of user profiles in order to match content with users via subject, virtual world location, time, demographic category, and/or the like. As such, the CTIS efficiently facilitates commerce by providing Actionable Events to consumers with highly tailored specificity.

QUERYING AND SERVING INFORMATION ON THE INTERNET

[0156] In an embodiment, the CITS delivers information on the internet based on user activities, such as may be registered by codes scanned by mobile devices or website links selected on a computer, as well as user characteristics and an accumulated history of user activities. For example, information and/or Actionable Event providers may use CTIS to serve context, demographic, and behavior targeted information or Actionable Events to users on the internet. In particular, information and/or Actionable Events are served on the internet that are targeted to the individual based on the individual's characteristic profile, behavioral patterns, and present contextual surroundings, either in real space of cyberspace. For example, a web-based Actionable Event may be provided on an internet enabled mobile device for goods and services located near a mobile user, which Actionable Event is selected based on the user's behavioral patterns and stated interests.

[0157] In one embodiment, the CTIS allows Actionable Event providers to supply product information on websites to the consumers who would be most interested in such Actionable Events; this may be achieved by the CTIS selecting Actionable Event content based on a combination of the context of the code or multi-modal artifact that the consumer scanned or the web link that the user selected, the consumer's stated interests as recorded in his or her user profile, the consumer's demographic profile, situational information, and a record of scanned codes/artifacts or web links; these factors can be analyzed and employed by the CTIS for both user tracking and behavioral profiling/targeting. For example, a user with a stated interest in running and a recent pattern of scanning codes related to shoes may be served a web-based Actionable Event on his/her internet-enabled mobile device for a new running shoe being sold at a nearby shoe store. In another example, a user profiled to be a teenager and having a pattern of browsing websites related to comic books may be served a web-based Actionable Event for the newest comic book themed movie when he/she visits a movie-listings website

[0158] Users, in turn, trigger the provision of information by scanning or observing codes or information, or by selecting web links. The triggers, together with geographic, temporal, and user- specific information, are obtained by the server that receives, processes, and records the message. Based on these messages and a user profile—which may include continuously updated user- specific behavior information, situational and ambient information, an accumulated history of trigger messages, and integration with outside database information— the server selects information to serve to a user on the internet from an information base.

[0159] For example, a user with a recorded history of interest in coffee products may be served an Actionable Event for a nearby coffee shop while browsing the web on his/her PDA. In one embodiment, information may also be served to users on the internet based solely on the user profiles, and without any initiating trigger. This is based on user trajectories or web-surfing habits deduced from the accumulated history of triggers. For example, a user known to routinely visit a music vendor website near the same time each day may be served an Actionable Event for the latest top-selling CD shortly before that time.

AMBIENT CONDITIONS AND IMPACTS

[0160] In embodiments, ambient and/or other situational conditions that may be considered in selecting and targeting information include temperature, weather, light levels, UV levels, pollen count, humidity, air quality, prices of commodities and/or consumer goods, stock and/or financial index prices, global and/or local news events, internet activities, sports scores, entertainment news, security warnings, and/or the like. As such, this disclosure details a code triggered information server (CTIS) that delivers information based on user activities, such as may be registered by codes scanned by mobile devices, as well as user characteristics, an accumulated history of user activities (e.g., code scans), and ambient conditions. For example, a user in the midst of high ambient temperatures who is profiled as a teenager who enjoys sweets and has a recorded history of scanning codes pertaining to fast food restaurants may be served an Actionable Event for the newest ice cream sundae product at a fast food restaurant near the user.

[0161] The user's code scans, together with geographic, temporal, and user-specific information, are obtained by the server that receives, processes, and records the message. Based on these messages and a user profile— which may include continuously updated user-specific behavior information, situational and ambient information, an accumulated history of scanned code/artifact messages, and integration with outside database information— the server selects information to serve to the users' mobile devices from an information base. In one embodiment, the code triggered information server allows Actionable Event providers to supply product information to the consumers who would be most interested in such Actionable Events; this may be achieved by the CTIS selecting Actionable Event content based on a combination of the context of the code/artifact that the consumer scanned, the consumer's stated interests as recorded in his or her user profile, the consumer's demographic profile, situational information, and a record of scanned codes/artifacts; these factors can be analyzed and employed by the CTIS for both user tracking and behavioral profiling/targeting.

[0162] In another embodiment, the CTIS may serve Actionable Events based on prevailing ambient conditions or climate at users' space-time positions. For example, this may occur when the relevance score for contextual Actionable Events is low. An illustration of the logic and data flow in this embodiment is provided in FIG. 13, wherein the contextual Actionable Events are selected from the Information Base (as illustrated in FIG. la) on the basis of code or multi-modal artifact scan and user profile are supplanted in favor of Actionable Events chosen on the basis of prevailing ambient conditions at 1340. Exemplary embodiments are provided in FIGS. 14-15. In FIG. 14, an Actionable Event to purchase allergy medication is provided 1447 in lieu of weak contextual actionable event options 1442 in response to a code scan from the weather commercial or broadcast in a media expression 1407 in conjunction with ambient weather information, including pollen count, queried at the server 1412. In FIG. 15, an Actionable Event for sunscreen is provided 1547 in lieu of weak contextual Actionable Event options 1542 in response to a code scan from the front page section of a media publication or current newspaper 1507 in conjunction with ambient weather information, including humidity and UV index, queried directly by the user's mobile device 1512. In various embodiments, certain ambient conditions such as UV levels, humidity, download/upload speeds, and/or the like may be detected by the user's mobile device itself, while in other embodiments, ambient conditions are separately supplied to a database used in conjunction with the CTIS.

[0163] In another embodiment, the CTIS may serve contextual Actionable Events with content enhanced by reference to prevailing ambient conditions. An illustration of the logic and data flow in this embodiment is provided in FIG. 16 wherein the contextual Actionable Events selected from the information base at 1635 are further refined by the ambient condition information at 1640. Exemplary embodiments are provided in FIGS. 17-20. In FIG. 17, a code is scanned from the Business News broadcast of a media expression 1707 and the CTIS selects luxury car

Actionable Events for display 1739 based on the scanned code, user location, and user profile 1732, and these Actionable Events are further refined based on the ambient weather conditions provided to the server at 1712. In this case, warm and sunny weather provides further context to cause the CTIS to serve an Actionable Event for a purchase or lease promotion for a luxury convertible 1747. In FIG. 18, a code or artifact is scanned from the Travel section of a media expression 1807 and the CTIS selects travel Actionable Events for display 1839 based on the scanned code/artifact and user profile 1832, which includes recent scan code/artifact transactions pertaining to travel and Miami sports scores. The CTIS consequently queries the prevailing ambient weather conditions, comprised of cold and rainy weather, and supplies an Actionable Event for purchasing Miami travel opportunities, or registering for discounts or promotions for Miami travel. In FIG. 19, a code or artifact is scanned from the Weekend section of a media expression 1907 and the CTIS selects a contextual Actionable Event for purchasing a meal or receiving a discount code for eating out 1939 from the Information base 1935 to serve based on the scanned code and the prevailing ambient weather conditions, which are hot and sunny 1912. This Actionable Event is further refined by the user profile contents 1932 indicating a college student interested in comics and entertainment, and an Actionable Event to purchase ice cream or receive a discount code from an ice cream shop is finally supplied 1947. In FIG. 20, a code or artifact is scanned from a baseball team schedule within a media expression 2007. Based on the prevailing ambient weather conditions, including scattered showers 2012, the scanned

code/artifact, and the recent activity in the user profile 2032, an Actionable Event is supplied for purchasing ponchos and golf umbrellas 2042 from a nearby store or vendor located at the ball park.

[0164] In various embodiments, the CTIS may serve Actionable Events based on a wide variety of different ambient and/or situational conditions that may be detected by the client mobile device, provided by internal or third party databases or information feeds, and/or the like. Some non- limiting examples of potential ambient and/or situational conditions that may be considered and/or employed in selecting information to serve are temperature, weather, light levels, UV levels, pollen count, humidity, air quality, prices of commodities and/or consumer goods, stock and/or financial index prices, global and/or local news events, internet activities, sports scores, entertainment news, security warnings, and/or the like. Consideration of such factors, by themselves or in conjunction with knowledge of a user such as may be gleaned from a user profile, may form the basis of information serving rules that have great specificity of targeting. For example, a marketer rule within the CTIS may specify that an Actionable Event for participating bars that includes a coupon for a particular brand of beer be served to all male users between the ages of 21 and 40 with a history of scanning baseball-related codes within three hours of any time that the local baseball team wins a game. In another example, a marketer rule within the CTIS may specify that an Actionable Event for hybrid vehicles be served to all mothers with young children who have a history of scanning codes in Time magazine whenever the local price of gasoline is greater than $2.50/gallon and they scan a code in an issue of Time that contains articles on both global warming and the Iraq War. Such hyper-targeted information serving, enabled by the CTIS, may prove very valuable to marketers and, in fact, to a wide variety of information dissemination applications. Actionable Event providers may be able to purchase Actionable Event time and/or impressions that are conditional on events, people, time, place, ambient conditions, and/or combinations thereof, and/or the like. Consequently, a premium may be charged to Actionable Event providers for each Actionable Event served, and that premium may increase with the number of factors considered in serving the Actionable Event (i.e., the degree to which the Actionable Event serving is targeted). In another implementation, a different price may be charged per impression depending on the characteristics of the individual to which the Actionable Event is served. For example, the hybrid car Actionable Event described above may cost an advertiser $0.75 for an impression to a mom with adult children but $1.00 for an impression to a mom with young children. Such graded pricing schemes may be specified within a marketing rule data structure.

INTERNET CLIMATE

[0165] In another embodiment, Actionable Events may be served based on internet "climate," such as serving Actionable Events for virus detection software during an outbreak or serving Actionable Events for high-speed broadband internet service on days of high internet traffic.

Conditions for specific websites (i.e., cyberspace-time positions) could even be taken into account. For example, an Actionable Event for high-speed broadband internet service could be supplied to visitors of a website experiencing particularly high traffic.

[0166] In another embodiment, the CTIS may interface with various customer information databases, customer relationship management databases, and/or any other third party data sources in order to extract information that may be used to further target and/or refine Actionable Event selection for a given user. In one implementation, a marketer may supply the CTIS with access to a given marketer-specific database and/or data feed, in order to allow the CTIS to access the information therein during Actionable Event selection processes. It should be understood that the functionality described for this embodiment of the CTIS may be integrated or employed within any of the other CTIS embodiments described herein.

INFORMATION QUERYING AND SERVING ON MOBILE DEVICES BASED ON PROFILES

[0167] In an embodiment, information querying and serving are triggered by codes that are scanned by a user. The information is selected based on the profile of the user. In an illustrative example, a user who is profiled as a teenager with a stated interest in comic books and a record of scanning codes/artifacts related to movies may be served an advertisement for the latest comic book themed movie playing in the next hour near his/her present geographic position the next time he/she scans a code/artifact from the entertainment section of the newspaper.

DEVICE OPTIMIZED ADS

[0168] In another embodiment, the CTIS can interface with and optimize Actionable Events for various User Agent devices, which may include PCs, laptops, PDAs, DVRs, cable boxes, internet- coupled media devices, and generally with programmable electronic devices possessing external data transfer capabilities and/or the like. FIG. 21 provides an illustration of the logic and data flow in one embodiment. At 2115, the Scan Message content further incorporates an optional Send to Device info, which specifies the destination User Agent device for a particular Actionable Event. Then, at 2150, the Actionable Event content selected by the CTIS is configured and/or optimized for the User Agent device. Exemplary embodiments are provided in FIGS. 22-23. In FIG. 22, a code or multi-modal artifact from a sports media expression is scanned by a cell phone 2207, and a set-top-box/cable box/DVR User Agent device is specified in the optional Send to Device information of the Scan Message 2217. Consequently, in addition to contextual Actionable Events supplied to the user's cell phone containing links purchase tickets, discount ticket packages, promotional game nights 2237 and season ticket offers 2239, additional content is configured and sent to the User Agent device, such as authorization and payment to descramble a pay-per-view sporting event, purchase a television or cable sports package, and the like. In one embodiment, the user's mobile device would provide a signal to the User Agent device to cause it to send a Request for Info to the CTIS (2252). In another embodiment, the CTIS could send an appropriately configured signal to the User Agent device without the need for a Request for Info signal. In FIG. 23, a code or multi-modal artifact is scanned from the Technology media expression using a cell phone 2307, and the optional Send to Device info in the Scan Message specifies to send to a PC laptop 2317. In accordance with the ambient conditions embodiment discussed previously, the CTIS gauges the ambient conditions associated with the device (2320) and determines that there has been a recent outbreak of a particular computer virus over the internet 2322. Consequently, the CTIS supplies an offer for purchasing Virus Scan software subscriptions 2344, and may send Virus Scan software directly to the specified PC laptop based on user response for purchasing the subscription.

ANTICIPATORY INFORMATION QUERYING AND SERVING ON MOBILE DEVICES BASED ON PROFILES

[0169] In one embodiment, information may also be served to users based solely on the user profiles, and without any initiating code scan. This may be based on predicted space-time trajectories derived from the accumulated history of scanned codes.

[0170] The CTIS delivers information based on user activities, such as may be registered by codes/artifacts scanned by mobile devices, as well as user characteristics and an accumulated history of user activities (e.g., code/artifact scans). The scans, together with geographic, temporal, and user-specific information, are obtained by the server that receives, processes, and records the message. Based on these messages and a user profile—which may include continuously updated user-specific behavior information, situational and ambient information, an accumulated history of scanned code/artifact messages, and integration with outside database information—the server selects information to serve to the users' mobile devices from an information base. In one embodiment, the code triggered information server allows advertisers to supply product information to the consumers who would be most interested in such Actionable Events; this may be achieved by the CTIS selecting Actionable Event content based on a combination of the context of the code that the consumer scanned, the consumer's stated interests as recorded in his or her user profile, the consumer's demographic profile, situational information, and a record of scanned codes. These factors may be analyzed and employed by the CTIS for both user tracking and behavioral profiling/targeting. This information can then be supplied to a user based on anticipated user trajectories and/or interests, as may be determined from the record of scanned codes/artifacts.

[0171] For example, the space-time trajectories of a user who has scanned barcodes in magazines using his/her cell phone over a period of time may be determined, and future likely trajectories determined. Then, Actionable Events could be served to the user based on where the user is expected to be at a given time, his/her demographic profile, his/her stated interests, and the types of media expressions from which he/she has scanned codes/artifacts in the past. As an illustrative example, a user may have a history of scanning codes related to fast food, a stated interest in sweets, and a project path passing near a particular fast food restaurant at a particular time. This user may be served an Actionable Event for the latest dessert product at that restaurant on his/her cell phone shortly before he/she is expected to pass the restaurant.

DETERMINING AND ANNOUNCING PROXIMITY BETWEEN TRAJECTORIES

[0172] The present disclosure is directed to apparatuses, methods and systems for determining and announcing proximity between trajectories. Existing schemes fail to take full advantage of the tracking and context targeting possibilities enabled by the present disclosure. In particular, they do not allow tracking and proximity notification that is targeted to the individual consumer's characteristic profile, behavioral patterns, and present contextual surroundings. As such, this disclosure details a code triggered information server (CTIS) that tracks users and notifies them of proximity with other users based on user activities, such as may be registered by codes/artifacts scanned by mobile devices, web links selected on a computer, or decisions made in a virtual world. For example, a recorded history of scanned barcodes, as well as the time and location of each scan, may be analyzed to determine where a user has been and what they were interested in scanning at the time. Based on such a record, a prediction of a user's likely location may be made. Then, intersections or near-misses for such predicted trajectories between different users may be determined and reported. The CTIS may also incorporate user characteristics and interests.

[0173] As the first proximity notification system with geographic, context, and behavioral specificity based on user profiles, the CTIS is designed to analyze user behavioral patterns in order to statistically derive user trajectories and tracking information. In one embodiment, the CTIS is designed to compare expected user space-time trajectories to determine when two or more users are likely to be within a specified distance of each other at a given time. In another embodiment, the CTIS is designed to compare expected user web-surfing habits to determine when two or more users are likely to be on the same website, or within a specified number of links from each other, at a given time. In another embodiment, the CTIS is designed to compare expected user trajectories within a virtual world to determine when two or more users are likely to be within a specified virtual distance from each other within the virtual world. In another embodiment, the CTIS provides rapid and evolving geographic and historical statistical profiling of user activity. A GRAPHICAL CODE-SERVING INTERFACE

[0174] In an embodiment, information is displayed in an electronic system by displaying a first visual token and animating the first visual token to reveal a second visual token comprising a coded symbol. The animation of the first visual token may be in response to a user action. In an embodiment, the user action is a selection response that uses a user pointer. The animation may include, for example rotating the first visual token to reveal a coding symbol on a proximate side of the first visual token.

[0175] The encoding symbologies for example, matrix codes, may be placed proximate to graphical elements in a wide variety of digital display contexts and media, including portable display devices, mp3 players, cell phones, digital paper, animated billboards, websites, internet or television broadcasts, software interfaces, and/or the like. Users may capture images of information encoding symbologies from such displays, for example using a cell phone camera, and decode them with software applets to receive additional information, multimedia content, offers, coupons, notifications, and/or the like.

[0176] Serving codes to users may be accomplished by a variety of different means and in a variety of different contexts. FIGS. 24-26 illustrate some implementations of graphical code serving interfaces (GCSIs) in embodiments of the GCSI/CTIS system. In these implementations, a graphical display and/or interface initially presents a first visual token for display. This first visual token may comprise virtually any image, picture, drawing, emblem, icon, logo, animation, and/or the like. This first visual token may then be animated to reveal a second visual token comprising encoded information, such as a QR code, matrix code, bar code, and/or the like. Animation of the first visual token may, for example, comprise rotation of the first visual token to reveal the second visual token on a proximate side of the first visual token. For example, a two-dimensional picture in the graphical display may rotate in the third dimension to reveal a matrix code on the back of the picture. Other forms of animation may also be employed within different implementations. For example, the first visual token may dissolve, morph, disassemble and reassemble, and/or the like to reveal the second visual token. In some implementations, the animation of the first visual token to reveal the second visual token may occur only in response to some user interaction with the first visual token. For example, a computer screen may display a first visual token persistently until a user clicks on the first visual token with a mouse pointer, at which time the first visual token may rotate to reveal the second visual token on its back. In another example, an electronic billboard may persistently show an advertisement until a motion sensor detects someone walking by, at which time the advertisement dissolves to reveal a matrix code underneath. In still another example, an electronic sign may display a particular image until a detector coupled to the sign detects a signal and/or an authorization code from a compatible, portable electronic device near the detector, at which time the image may morph into a barcode. In still another example, the first visual token displayed on a touch-sensitive display may flip over to reveal a code when a user touches it with a finger or runs a finger over it to simulate a flipping motion.

[0177] FIG. 24 shows an implementation of a user interface manifesting GCSI/CTIS functionality on a portable electronic device in one embodiment of GCSI/CTIS operation. A wide variety of different portable devices may execute and/or coordinate with GCSI/CTIS functionality, such as but not limited to portable media players, cellular telephones, blackberries, PDAs, and/or the like. In the illustrated implementation, a portable media player displays album information, including an image of the album cover, on a graphical display, where the displayed information corresponds to music being played on the player 2401. A user may click on or otherwise select the album cover image, causing it to increase in size 2405 and rotate 2410-2415 to reveal a matrix code on the image 2420.

[0178] FIG. 25 shows an implementation of a user interface manifesting GCSI/CTIS functionality on digital paper in one embodiment of GCSI/CTIS operation. Here, an image is included in one area of an article displayed on the paper 2501. Either by itself or in response to some user action, the image flips 2505 to reveal the code on its other side 2510.

[0179] FIG. 26 shows an implementation of a user interface manifesting GCSI functionality on a digital billboard in one embodiment of GCSI/CTIS operation. A public, electronic billboard display shows an advertisement comprising a product logo 2601. The product logo rotates 2605 to reveal a matrix code on its opposite side 2610. Flipping of the logo may occur periodically or aperiodically, upon detection of a nearby viewer, upon detection of a portable device capable of decoding the matrix code, and/or the like.

CODE-MEDIATED CONTENT DELIVERY PLATFORM

[0180] Embodiments are directed to apparatuses, methods, and systems for a code-mediated content delivery platform comprising a Graphical Code-Serving Interface and a Code Triggered Information Server ("GCSI/CTIS"). The GCSI/CTIS connects information encoding symbologies, and by proxy the underlying information content, to graphical display elements, and facilitates generation of encoding symbologies, error-correction, media sampling and purchasing, social networking, and sales promotion programs.

[0181] In an embodiment, active display elements provide access to information, particularly information that is targeted to an individual's profile, interests, and/or contextual surroundings and that is portable and/or made available in unexpected physical locations. The active display elements facilitate the serving of such information via placement of information encoding symbologies, such as matrix codes, proximate to graphical elements in a wide variety of digital display contexts and media, including portable display devices, mp3 players, cell phones, digital paper, animated billboards, websites, internet or television broadcasts, software interfaces, and/or the like. Users may capture images of information encoding symbologies from such displays, for example using a cell phone camera, and decode them with software applets to receive additional information, multimedia content, offers, coupons, notifications, and/or the like.

[0182] In one embodiment, a code/artifact sent via a communications network from a mobile communications device from a user is received. The code/artifact is based on a scan of an optically-readable coding symbol. A code database is queried to discern a multimedia content data identifier corresponding to the received code. Multimedia content data is retrieved from a multimedia database based on the multimedia content data identifier. A sample of the multimedia content data is sent to the mobile communications device via a communications network. A sample acceptability message is received from the user. The multimedia content data is sent to a data repository associated with the user provided that the sample acceptability message indicates that the user wishes to receive the multimedia content data.

[0183] In another embodiment, an alphanumeric string comprising a plurality of characters and representing a location of multimedia content data is received from a first data source. Each of the plurality of characters of the alphanumeric string is converted into a corresponding binary number to yield a plurality of binary numbers based on a character correspondence table. The plurality of binary numbers are concatenated to yield a bitwise array. An optically-readable encoding symbol comprising a plurality of disconnected solid shapes is drawn in which the presence of a solid shape at a position in the symbol indicates presence of a 1 in a corresponding position of the bitwise array.

[0184] In another embodiment, an incomplete code associated with an optically-readable encoding symbol is received. A code database is queried based on the incomplete code to determine a plurality of possible matching codes. The code records associated with the plurality of possible matching codes are queried to retrieve code presentation conditions associated with each of the plurality of possible matching codes. A plurality of code repair schemes are retrieved from a code repair database based on code presentation conditions. Each of the plurality of code repair schemes are applied to the incomplete code to yield a plurality of repaired codes. A repaired code is selected from the plurality of repaired codes based on at least one pre-designated code fidelity rule. The repaired code is designated as a correct code corresponding to the optically-readable encoding symbol.

[0185] In another embodiment, a scan indicator comprising an indication that a user has scanned an optically-readable encoding symbol with a mobile scanning device is received. The optically- readable encoding symbol is associated with a good or service. A retailer at which the user scanned the optically-readable encoding symbol is identified. A retailer record is queried to determine whether the retailer is subscribed to a sales promotion program. The retailer is charged a first retailer fee based on the received scan indicator if the retailer is subscribed.

[0186] In another embodiment, an online social network is provided. A user identifier is received and stored. User profile information is received and stored in association with the user identifier. A plurality of user scan indicators indicating that a user has scanned a plurality of optically- readable encoding symbols are received. The plurality of user scan indicators are stored in association with the user identifier. At least one friend designation indicating an association with at least one other user is received and stored in association with the user profile.

[0187] Users may scan codes found in media publications, on billboards or signs, on consumer products or packages, on websites or television screens, on movie screens, on clothing or accessories, on mobile device displays, other media expressions, and/or the like.

[0188] In an alternative implementation, content associated with a captured code/artifact or other user interest registration may be sent to a data repository other than a user's mobile device. For example, a user may scan a code/artifact corresponding to media content. Rather than triggering the GCSI/CTIS to send the media content to the mobile device with which the code was scanned, the code scan may trigger the GCSI/CTIS to send the media content to a user's computer, an online data repository, and/or the like for storage and later retrieval by the user. For example, the GCSI/CTIS may discern a user identity from the Scan Message (115), query user access and/or authorization information, and use that access and/or authorization information to access an online data repository corresponding to the user to store requested media content thereon for later retrieval and/or use by the user.

[0189] FIG. 5d shows an implementation of logic flow for customized repair and/or decoding of obscure or ambiguous scanned codes. An ambiguous code may be received at a GCSI/CTIS system at 581 , such as may be sent via a communications network from a user's mobile scanning device. In one implementation, a scanned code may be qualified as an ambiguous code if an attempt to decode the code does not yield a single result. In one implementation, the system may also receive code scanning conditions 582 describing any of a variety of circumstances of the code scan which may affect the quality and/or fidelity of the scanned code. For example, some relevant circumstances may include the time of day, light levels, use of flash, mobile scanning device type and/or characteristics, code size, contrast, brightness, sharpness, skew, rotation, and/or the like. The ambiguous code and scan conditions may then be passed to a repair queue 583, whereby the system may analyze the ambiguous code and attempt to discern the correct decoded content. A plurality of possible matching codes are determined at 584, and a code database is queried at 585 to extract code presentation conditions corresponding to each of the plurality of matching codes. Code presentation conditions may describe how and where the code was displayed and may comprise a wide variety of factors, such as but not limited to medium (e.g., newspaper, glossy paper, product packaging, television display, website, billboard, and/or the like), likely obscuring factors (e.g., glass covering, tears, distance, and/or the like), and/or the like. The GCSI/CTIS system may then generate a code repair database query based on the presentation conditions and/or scan conditions 586 and query the repair database to extract repair schemes corresponding to the particular presentation and/or scan conditions. For example, a particular code repair scheme may be tailored for and/or correspond to codes displayed on billboards that are scanned after dark by Nokia mobile phones. Extracted repair schemes corresponding to each of the possible matching codes are applied to the ambiguous code at 588, and a determination is made at 589 as to whether any one of the possible matches is now more likely to be correct than the others. This

determination may be made, for example, by measuring the fidelity of codes with various repair schemes applied and selecting as the correct code that which has the highest resulting fidelity. If no remaining ambiguity exists as to the correct code, then the favored code is registered with the GCSI/CTIS system at 592. Otherwise, the GCSI/CTIS may generate a custom recommendation message 590 and send the message to a user and/or a user's mobile device 591. The custom message may, for example, provide tailored recommendations on how the user may improve the fidelity of the code scan based on the possible matching codes and the associated presentation conditions and/or scan conditions. Tailored recommendations, as well as the repair schemes themselves, may be adjusted based on feedback received from users about which codes they were actually trying to scan. In one implementation, the GCSI/CTIS may provide a query message to a user presenting a summary of content associated with alternative matching codes in order to determine which content the user was actually trying to access. It should be noted that code analysis and/or repair may be undertaken on a code image level, alphanumeric string level, binary string level, and/or the like basis. Combinations of such bases may also be undertaken, such as by first analyzing the ambiguous code in its alphanumeric string form and then, if ambiguity remains, analyzing the code in its binary form. Further discussion on such an analysis is provided below.

[0190] In an illustrative example of partial decoding, a given matrix code may encode the string ABC123. In multiple processes, it may be determined that the code contains A 1 3 (where the blanks represent unknown values), _C1_, and AB_.

[0191] Through these multiple processes, the GCSI/CTIS system now knows that the code contains ABC1 3 and may check this against codes in a code database to determine how many matches exist. If it finds two matching codes (e.g., ABC 123 and ABC 143), it may consider ancillary factors, including code presentation and/or code scan conditions, user profile

information, and/or the like. For example, ABC 123 may correspond to an advertisement for a new action movie while ABC 143 corresponds to a coupon for women's cosmetics. If the user supplying the code is determined, based on a supplied user ID, to be a man with a history of scanning movie- related codes, then the GCSI/CTIS system may infer that ABC123 is the matching code and serve the corresponding content. In one implementation, the GCSI/CTIS server may further supply a confirmation request for partially matched codes, requesting the user to affirm or deny that the supplied content is in fact the content that he or she was seeking.

[0192] In one implementation, analysis of ambiguous codes may comprise a two-stage process. In a first stage, an encoded string comprising a number of alphanumeric characters, one or more of which may be unclear, may be compared to a database of alphanumeric strings to determine which, if any, match. In one implementation, a Levenshtein distance metric may be employed to determine the similarity of an input string to strings existing in the database. In one implementation, the input string may only be compared against "live" strings in the database (i.e., strings that are currently engaged in active use and/or tied to actual content). If ambiguity remains about which code has been scanned after this character-based analysis, a GCSI/CTIS system may apply a second stage of analysis, whereby the input alphanumeric string is converted into a corresponding binary bitset array (see, e.g., FIGS. 33a-b and associated discussion below) and a comparison is performed between binary values of the input string and the strings stored in the code database. The most significant bits come first in the bitset array while the least significant bits come last, so priority weighting may be applied against potential value matches based on which bits match between the input and stored strings.

User Interface

[0193] FIG. 7b shows an implementation of a user profile user interface in one embodiment of GCSI/CTIS operation. The profile page 761 may include a display of basic user information 763 and/or a user picture 766, and may provide a wide variety of other user information, including demographic information, interests, media libraries, friend lists, code scanning activities, and/or the like. The displayed implementation further includes a plurality of tabs 769 allowing the user to view profile information, edit his or her profile, view photos, groups to which the user may belong, and/or edit settings, including sharing and/or privacy settings 772. Privacy settings may, for example, control access of other individuals to a user's personal information, code scanning activity, and/or the like. In some implementations, certain information about a user, in particular regarding codes that a user has scanned and/or the content associated with those scans, may be shared with other users. For example, a user may be able to inquire, via a GCSI/CTIS affiliated system, as to what codes have been recently scanned by other users that are within a particular geographic radius of the user. In another example, a user scanning a code to receive a song may also be provided with information about the code scanning activities of other users whose codes also corresponded to songs.

[0194] In one implementation, a user may be allowed to restrict which information of theirs is sent to other users and/or to restrict which information of other users is sent to them. For example, a user may have a "friends" list that has a different level of access than the general public.

[0195] In one implementation, a user may have a personal code that is associated with the user and/or a user profile. In one implementation, other users can scan the personal code to

automatically join the first user's friend list. In another implementation, a personal code my comprise a short- link to the user's profile 761, and/or to other profile pages or webpages associated with the user, such as a Facebook page, Myspace page, and/or the like. In one implementation, the user can set a forwarding instruction on his or her GCSI/CTIS page to forward inquiries corresponding to scans of the user's personal code to another page. In another implementation, the user may provide the personal code on other pages as a link back to the user's GCSI/CTIS profile page.

[0196] In one implementation, a user may be provided recommendations in response to a code scan. For example, the user may scan a code corresponding to a particular song by a particular musical artist. The user may then be provided a listing of other songs, artists, and/or the like having codes recently scanned by the members of the user's friends list and/or that may be further related to the song and/or artist whose code the user scanned.

[0197] In one implementation, a user's code scans may be integrated with an instant messaging, blogging, micro-blogging, and/or the like service, whereby the user's code/artifact scans are automatically rendered as descriptive summaries and displayed to other users, such as on a webpage, via SMS text messages, emails, and/or the like. For example, each code/artifact scan may have a pre-set text description associated therewith that, upon scanning of the code, is displayed on a user's Twitter.com page. In one implementation, a user may receive an incentive payment for particular types of content that are provided to the user's friends, associates, blog subscribers, and/or the like as a result of the user's code/artifact scans. For example, a reward may be provided to the user every time the user scans a code or multi-modal artifact corresponding to a particular brand of products and a message associated with those products is supplied to subscribers to the user's profile and/or blog. A further reward may be provided to the user if it can be determined that one of the user's associates made a purchase based on the provided message.

[0198] The privacy settings in the displayed implementation include permissions for displaying "tweets," songs, videos, and purchases 775 associated with the user's code scans to various security levels for other people 778, including the general public, private/friends, and "snaps" (i.e., those who may have scanned a user's personal code and/or a code corresponding to the user's profile). Selection of a particular privacy setting may generate a sharing rule that is stored in association with the user and consulted each time a user scans a code corresponding to one of the categories shown at 775 to determine whether or not code-associated information should be provided and/or sent to any of the categories of people shown at 778. [0199] The profile page 761 further includes a field 779 wherein a user may specify the location of a data storage facility to which some or all media and/or other content that the user acquires via code scans are to be sent and/or stored. For example, the user may specify an online storage facility (e.g., Apple's iDisk service) in which the user's code scan-acquired content is to be stored.

[0200] The profile page 761 further includes interface components configured to generate a personal code, in this case a JagTag 780. Selection of the button at 780 causes the GCSI/CTIS system to associate a code with the user and/or the user's profile and generate a corresponding code, which is displayed for the user. The user may then elect to save an image of the code to his or her computer desktop, send the code in an email, send the code in an MMS message, and/or the like via the interface elements at 781.

[0201] In one implementation, coupons may be supplied to a user's mobile device with digital rights management (DRM) software included to prevent or discourage coupon copying, sharing, and/or the like. In another implementation, no DRM software is included with the coupon.

[0202] In one implementation, a GCSI/CTIS server may query, store, and/or consider data pertaining to media demographics associated with a particular medium in which a code/artifact is published; the subject content associated with a code/artifact; the type of information requested by scanning a code/artifact; and/or the like. For example, a user scanning a code/artifact associated with a product review for a hybrid sports utility vehicle (SUV) in a men's magazine may trigger a GCSI/CTIS system to draw and/or analyze data pertaining to media demographics associated with the men's magazine, subject content associated with hybrid vehicles (i.e., environmentally conscious consumers), subject content associated with SUVs (i.e., outdoor activity or families), and product reviews. Thus, the GCSI/CTIS may be configured to consider both the subject of a user's inquiry, the source of the inquiry, the type of inquiry, and/or the like.

[0203] In another embodiment, the GCSI/CTIS may allow users to instantly subscribe to and/or enroll in services, loyalty programs, discount programs, and/or the like by scanning codes or multi-modal artifacts associated therewith. For example, a user may subscribe to receive weather updates by scanning a weather-linked code/artifact. In another example, a user may enroll in a brand-affiliated coupon program by scanning a code/artifact associated with that program.

Selection of a subscription and/or enrollment code may cause the GCSI/CTIS to submit a user's personal information to an integrated subsystem and/or third-party system for subscription registration. In some implementations, the GCSI/CTIS may require verification from the user, separate from the code scan itself that the user wishes to enroll and/or subscribe to the associated service. In some implementations, a user may submit trigger specifications and/or conditions for enrolled service notifications. For example, a user may specify that he or she only wishes to receive weather updates associated with heavy rain or thunderstorms. In another example, a user may specify that he or she only wishes to receive coupons for products from a particular grocery store and not from other grocery stores. Scanning of a code associated with a subscription program wherein specification of triggers and/or conditions is permitted may cause a GCSI/CTIS system to retrieve and display a user interface screen to the user that is tailored to the particular subscription and/or enrollment service and configured to receive trigger and/or condition specifications.

[0204] In another embodiment, the GCSI/CTIS may be configured to track and/or implement a rewards program. For example, in a single-tiered rewards program implementation, a user may scan a code every time he or she purchases a particular item (e.g., a sandwich from the local sandwich shop) in order to register each purchase and store a record thereof in association with the user. The user may then receive a discount, free item, and/or the like after a pre-designated number of purchases and associated scans are registered, after which the record of scans is reset to start anew. In another example, in a multi-tiered rewards program implementation, a user may continue to register additional purchases and associated code scans to receive larger and/or more significant benefits, discounts, and/or the like.

[0205] In another embodiment, the GCSI/CTIS may be configured to implement contests and/or prize awards. For example, a user may enroll in a contest by scanning a code associated with the contest, causing personal information, user identifiers, and/or the like to be submitted to a contest subsystem. Such contests may comprise instant- win and/or other types of lotteries, auctions, raffles, and/or the like and/or may enroll the user in a contest program through which they may receive further instructions, possibly including additional codes for scanning, in the future.

[0206] In another embodiment, the GCSI/CTIS may be configured as a voting facility. A user may register a vote for a particular candidate, ballot initiative, viewpoint, survey result, and/or anything else for which a vote may be registered by scanning a code associated with the user's choice. In one implementation, a user identifier may be discerned and submitted with the vote in order to prevent repeat voting, to correlate with voter registration, and/or the like. In one implementation, voter identifying information may be stripped from a scan message prior to the message being provided to particular agencies or individuals, in order to preserve a secret-ballot style of voting. [0207] In another embodiment, the GCSI/CTIS may be configured as a facility to effectuate purchases of goods and/or services. By scanning a code associated with a particular good or service, a message may be sent to a GCSI/CTIS system identifying the good or service associated with the code as well as identifying the user, such as based on a mobile device identifier. The GCSI/CTIS system may then automatically generate a bill for the user, charge a specified credit account, deduct a payment from a debit account, and/or the like to effectuate payment for the good or service. In one implementation, the GCSI/CTIS may return a payment confirmation, such as to the user's mobile device and/or a retailer's point-of-sale device to confirm that the user has provided adequate payment.

[0208] In another embodiment, the GCSI/CTIS may be configured to facilitate communications. For example, a code may correspond to a particular telephone number, IP address, and/or the like. Scanning of the code by a communications-enabled mobile device may cause the mobile device to automatically establish a connection with the address to which the code corresponds. In one implementation, the connection may be established directly by an on-board applet loaded on the mobile device. In another implementation, the connection may be established by an instruction sent to the mobile device by a GCSI/CTIS subsystem in response to a received code scan.

[0209] FIG. 27 shows an implementation of a user interface manifesting GCSI/CTIS functionality for serving an SMS text message prompt in one embodiment of GCSI/CTIS operation. A first visual token 2701 shows an image and text, such as may correspond to an advertisement, logo, and/or the like. The token may appear in a wide variety of contexts or media expressions, such as on a website, in a virtual world, on a cell phone or other portable communications and/or media device (e.g., Blackberry, iPhone, iPod, and/or the like), on an electronic billboard, on broadcast television or recorded video content, and/or the like. The first visual token may be animated, such as by flipping, rotating, revolving, and/or the like (2705, 2710), to reveal encoded content on a proximate side thereof. In the implementation illustrated in FIG. 27, a message appears along with an SMS code that a user may key in to an SMS enabled communication device to purchase the associated product or receive associated content. In alternative implementations, other types of codes and/or code communication protocols may be displayed and/or employed for

communication with GCSI/CTIS systems, such as but not limited to: MMS, instant messaging, web browser based messaging, email, Enhanced Messaging System (EMS), TCP/IP, WAP, mobile payment systems, and/or the like. For example, a first visual token may be animated to reveal an email address, website address, instant message nickname, and/or the like on a proximate side to enable a user to connect to and/or communicate with a GCSI/CTIS system and/or affiliated entity for the receipt of information, coupons, offers, advertisements, media, and/or the like.

[0210] FIG. 28 shows an implementation of a user interface manifesting GCSI/CTIS functionality on a mobile phone 2801 in one embodiment of GCSI/CTIS operation. A first visual token comprising an image, actionable event, advertisement, logo, and/or the like 2805 is displayed on a mobile phone display screen. The first visual token may be animated 2810, such as in response to a user interaction (e.g., pressing a button on the mobile telephone, clicking on the first visual token with a pointer widget, and/or the like). In an alternative implementation, the first visual token may be automatically animated and/or animate without the need for any user interaction. In the illustrated implementation, the animated first visual token flips to reveal a second visual token comprising a message and an SMS text message prompt on a proximate side 2815. In one implementation, the user may manually key in the text message and/or destination address to purchase the associate product or receive the associated content. In another implementation, the GCSI/CTIS interface may supply a selectable link that, when selected, will automatically generate an SMS text message to purchase the associate product or retrieve the associated content. In still another implementation, such as the example illustrated at 2815, the user may be allowed to automatically purchase the product or generate and/or send the SMS text message from the second visual token by pressing a "Send" button on the mobile phone, clicking and/or otherwise interacting with the second visual token, and/or the like.

[0211] In one implementation, a user may receive a coupon and/or coupon code in response to an interaction with the second visual token, sending of an SMS text and/or other message in response to the second visual token, and/or the like. For example, in one implementation, a user may receive a numerical coupon code in response to a sent SMS text message, wherein the numerical coupon code may be presented to a retailer, entered into a website, and/or the like to receive discounts, free items, special offers, and/or the like. In another implementation, the user may receive a scannable code, such as a QR code, matrix code, and/or the like in response to a sent SMS text message, wherein the scannable code may be scanned by a retailer in order for the user to receive the associated benefits, discounts, and/or the like. For example, the scannable code may appear on a user's mobile phone display screen and may be scannable therefrom to provide the user with the benefits associated thereto. In another implementation, the user may receive an e- mail message in response to interaction with a graphical code-serving interface, the message containing requested content and/or links thereto, scannable codes, coupon codes, and/or the like.

[0212] In another implementation, a first visual token may comprise a "poster" image

corresponding to a full-length movie, movie preview, video clip, television show, and/or the like. The second visual token revealed on a proximate side of the first visual token may then enable a user to retrieve the full video content associated with the poster image. For example, a user may scroll through a series of poster images corresponding to a series of full-length movies. When the user finds a movie he or she desires to retrieve, watch, download, and/or the like, he or she may select the corresponding poster image, which may then be animated to reveal a second visual token, such as an SMS instruction, message, delivery address, phone number, and/or the like, whereby the user may interact with a GCSI/CTIS system and/or affiliated entity to retrieve the desired movie content.

[0213] In one implementation, a series of first visual tokens may be displayed as scrollable thumbnails, within a cinematic presentation, as an immersive slideshow, and/or the like.

[0214] FIG. 29 shows an implementation of a user interface manifesting GCSI/CTIS functionality on a portable communication device in one embodiment of GCSI/CTIS operation. The portable communication device 2901 shown in the illustrated implementation may represent a BlackBerry, iPhone, PDA, and/or the like. A first visual token 2905 is animated 2910 to reveal a message and a selectable hyperlink on a proximate side 2915, in this case a URL connecting to content, information, offers, coupons, and/or the like. In one implementation, a user may be permitted to directly select the hyperlink from the displayed image using interface elements of the device in order to retrieve content, view a webpage, interact with a GCSI/CTIS and/or affiliated system, and/or the like.

[0215] A variety of operational models may be employed to provide code-serving interface capabilities to users and content providers alike in accordance with the embodiments and/or implementations described herein. In one implementation, a central GCSI/CTIS server may enable content providers to supply first and/or second visual tokens, associated SMS and/or MMS codes, and/or any other information necessary to connect users to supplied content. The central

GCSI/CTIS server may then outfit the supplied tokens and/or other information with HTML markup information allowing the code-serving interface to be embedded in any webpage or other display media. In one implementation, the HTML markup information may further include a link to one or more downloadable applets that may be required to enable a user to view and/or interact with the code-serving interface. Having provided the data corresponding to a first and/or second visual token, SMS and/or MMS code, and/or other information, a content provider may be provided with an embeddable link, URL, and/or the like that may be cut and pasted into a web page HTML to embed the code-serving interface therein.

[0216] In another implementation, a content provider may submit just a first visual token and response content and/or a means of addressing said content, and a GCSI/CTIS server may generate a corresponding code-serving interface, a dial-in code and/or SMS and/or MMS address, hyperlink, and/or the like to allow users to interact with the first visual token to receive the response content. In one implementation, the generated code-serving interface may include HTML markup information enabling it to be embeddable on any webpage. A fee may be charged to the content provider, such as on a pay-per-click basis, for user interactions with the generated code- serving interface.

[0217] In still another implementation, a webpage or other display area may include a generic placeholder for code-serving interfaces that may be filled by code-serving interfaces and/or associated content supplied from one or more advertisement servers and/or GCSI/CTIS servers. The supplied code-serving interfaces may be selected based on webpage content, contextual cues, and/or the like and/or based on the goals of content providers and/or code-serving interface creators. For example, an advertiser may wish to place a particular code-serving interface associated with a new action movie on any webpage having a generic placeholder that mentions extreme sports. A fee may be charged to an advertiser for such targeted and/or contextual placement, such as on a bid-per-keyword basis. Fees may also be charged on a pay-per-click basis (i.e., levying a charge whenever a user clicks a code-serving interface to reveal the code, communication address, and/or the like on a proximate side).

GCSI/CTIS Media Acquisition Example

[0218] In some embodiments, the GCSI/CTIS may be configured to supply media content, music, images, video, and/or the like to users based on the scanning of codes/artifacts as described above. Acquisition of full media content may, in some implementations, be preceded by acquisition of media samples that may be reviewed by an acquiring user to determine whether or not to proceed with acquisition of the full content. For example, a billboard may display the Billboard Top 20 songs with a separate code or multi-modal artifact for each song on the list. Consumers, in this example, may scan codes corresponding to each song in the list for which they have interest using a mobile scanning device and receive a sample of each song. In one implementation, a sampling user may further be provided with an opportunity to purchase and/or otherwise acquire the full media content once the sample has been received and/or reviewed. In one implementation, media may be acquired by a user via MMS messaging, while in another implementation, media may be acquired by means of an integrated media service (IMS). The IMS may, in one implementation, comprise an on-demand media service wherein users may maintain accounts, sample and/or purchase media, download and/or upload media, and/or the like. An example of an IMS is Apple's iTunes Store. FIG. 30 shows an implementation of logic flow for media sampling and full media acquisition in one embodiment of GCSI/CTIS operation. A code may be scanned by a user 3001 , such as via a mobile device. The scanned code information, as well as hardware information identifying the device, the user, and/or the like may then be received by GCSI/CTIS from the mobile device, such as via a communications network. The GCSI/CTIS may then make a determination, such as may be based on the hardware information, user information, a user profile, and/or the like, as to whether the mobile device is configured with an IMS 3005. If the mobile device is configured with an IMS, the GCSI/CTIS may issue an instruction to the mobile device to connect to the IMS 3010, allowing the user to engage IMS functionality such as the sampling, purchasing, downloading, viewing/playing, and/or the like of media files 3015.

[0219] If the user's mobile device is not configured with an IMS, the GCSI/CTIS may send a message to the device, such as an SMS text message 3020, to determine if the user desires to receive a sample of the requested media 3025. In one implementation, the GCSI/CTIS may first interact with a media service to determine if a sample is available before offering the sample to the user. If a sample is desired, the GCSI/CTIS may send a media sample to the user's mobile device, such as in the form of an MMS message. The sample may, in various implementations, comprise a clipped and/or low-quality version of an audio file, image file, video file, publication, and/or the like. In one implementation, the GCSI/CTIS may first acquire the sample from a music service in order to provide it to the user's mobile device. In an alternative implementation, the GCSI/CTIS may itself generate a sample from an original copy of a media file, such as may be acquired from a media service.

[0220] The GCSI/CTIS may then generate and send a query, such as via SMS text message 3035, to determine if the user would like to purchase the media associated with the scanned code 3040. If not, the GCSI/CTIS may exit the routine associated with the acquisition of the current media file, present alternative and/or related media files for user consideration, and/or the like 3045. If, on the other hand, the user indicates a desire to purchase the media file in question, the

GCSI/CTIS may query the user, such as via an SMS text message, as to which of a selection of pricing and/or download options the user would like to pursue 3050. For example, in various implementations, the user may be provided with options as to the quality and/or format of media files acquired, the method of acquisition, accessibility and/or rental options, and/or the like. In the illustrated implementation, the user may be presented with an option as to the method of media acquisition. Specifically, a determination may be made as to whether or not the media is to be acquired directly by the user's mobile device 3055. In the illustrated implementation, the determination is based on a user preference. However in alternative implementations, the determination may be made, in whole or in part, based on a variety of other factors, such as detected capabilities of the user's mobile device, available network bandwidth, pre-set profile settings, and/or the like. If the user selects for the media to be directly provided to his or her mobile device, then a corresponding fee may be charged and the media may be streamed, downloaded, and/or the like directly to the user's mobile device 3060. Alternatively, if the user does not desire to acquire the media directly to his or her mobile device, then the GCSI/CTIS may charge a price corresponding to an indirect delivery mode and conduct the media file to an intermediary storage facility for later retrieval 3065. For example, the GCSI/CTIS may instruct a media service to allocate the media file to a user account for later retrieval by the user. In another example, the GCSI/CTIS may acquire the media file and/or email the file to a user email account for later retrieval. When the user is ready, he or she may obtain the media file from the intermediary storage facility, transfer the file to a mobile device, and/or the like.

[0221] The charging of a fee for acquired media may proceed in a variety of different manners within various embodiments of the GCSI/CTIS. For example, in one implementation, a user may be directly requested to enter payment information (e.g., credit card number, checking account number, etc.) at the time of purchase. The entered information may be verified before the media is made available to the user and/or the user's mobile device. In another implementation, a user may enter payment information into a user profile and have a corresponding account automatically charged when the user directs the GCSI/CTIS to acquire media.

[0222] FIG. 31 shows an illustration of media acquisition in one embodiment of GCSI/CTIS operation. In the illustrated implementation, a printed publication 3101 displays an advertisement for a new release by a recording artist 3105, the advertisement including a scannable matrix code that facilitates the acquisition of media associated with the new release by a user. In an alternative implementation, an animated user interface may present a scannable code on a proximate side of a first image, such as on a video billboard, website, television program, and/or the like. The media may, for example, comprise an entire album, a single, a ringtone, a video, an image, and/or the like. In one implementation, the media provided may depend on the scanned code in conjunction with a user profile containing user preferences, a history of user behavior and/or prior scanned codes, and/or the like. The user may engage the media content by scanning the code with his or her mobile device 3110, after which the GCSI/CTIS may query the user, such as via an SMS text message, as to whether or not he or she wishes to sample and/or purchase the requested media 3115. The user may respond, in one implementation, by sending a reply text message to be relayed to the GCSI/CTIS. In one implementation, the GCSI/CTIS may subsequently provide, stream, upload, and/or the like the requested media directly to the user's mobile device for storage and/or playback 3120. In another implementation, the GCSI/CTIS may provide the requested media to an intermediary storage facility, such as a user's media service account, an email server, an ftp server, and/or the like. The user may then subsequently procure the media from the intermediary storage facility, transfer the media to a mobile device, and/or the like 3125 at a later time.

[0223] FIGS. 32a-b show an implementation of logic flow for code scan monetization in one embodiment of GCSI/CTIS operation. In FIG. 32a, a code is scanned at 3201. In the illustrated implementation, the code may be found on a physical item that is subject to purchase and may be found in a retail establishment. For example, a compact disc (CD) in a music store may be embellished with a scannable code that, when scanned, may provide a sample of the music stored on the CD, deliver a ringtone to the user's mobile scanning device, provide artist information, images, concert dates, a coupon, and/or the like. In another implementation, the item for purchase may not be physical but rather may be provided online, such as on a website, and an associated code may be provided on the page on which the item is offered for purchase. Once the code is scanned 3201, a determination may be made as to whether the retailer providing the item is subscribed to a GCSI/CTIS service 3205. In one implementation, the retailer may be discerned based on a geographic position provided to the GCSI/CTIS by a code message sent from the user's mobile device that may, for example, be compared to records of retailer locations. The geographic position may be determined, for example, by an on-board GPS unit on the user's mobile device or based on geographic information contained in the scanned code. Subscription to the service may, for example, entail licensing to embellish items with GCSI/CTIS affiliated codes in exchange for a fee, such as a one-time or periodic fee and/or a per-scan fee, such as illustrated in the

implementation shown in FIG. 32. If the retailer is determined at 3205 to be a subscriber, then the GCSI/CTIS may register a retailer payment 3210. For example, in one implementation, registering payment may comprise noting the fee in a fee record. In an alternative implementation, the GCSI/CTIS may automatically deduct the fee from a pre-designated account. A determination may also be made as to whether the manufacturer, publisher, distributor, and/or the like of the item in question is a subscriber 3215. For example, for the case of the CD discussed above, the

"manufacturer" may comprise a record label. If the manufacturer is determined to be a subscriber, then a manufacturer payment is registered at 3220.

[0224] Discernment of retailer, competitor, and/or manufacturer identities may be accomplished by a variety of different schemes in various implementations and/or embodiments of GCSI/CTIS operation. For example, a unique subset of codes may be reserved for a particular manufacturer, retailer, and/or the like, such that a scan message corresponding to any code of the unique subset may be associated by a GCSI/CTIS system with the particular manufacturer, retailer, and/or the like. In another implementation, retailers associated with a given scanned code may be discerned based on a geoposition of the mobile scanning device at the time of code scanning. FIG. 32b provides an illustration of an implementation of logic flow for discerning retailers in one embodiment of GCSI/CTIS operation. A determination is made at 3267 as to whether the mobile scanning device used to scan a code is position sensitive. Position sensitivity of a mobile device may be facilitated, for example, by an on-board GPS unit, mobile device and/or cellular tower triangulation, trilateration, multilateration, and/or the like. If the device includes one or more position sensitive facilities, the GCSI/CTIS may discern the position of the mobile device 3270. Position discernment by the mobile device and/or the GCSI/CTIS may include drawing raw geographic position data from one of the facilities described above using, for example, mobile device software development kit (SDK) tools, an example of which are those associated with the iPhone SDK toolkit. The GCSI/CTIS system may then query position records based on the position discerned at 3270 to seek retailers matching that position 3273. A determination is made at 3276 as to whether any matching retailers are found. If no retailers match the discerned mobile device position, or if the mobile device lacks position sensitivity at 3267, the GCSI/CTIS system may query code records based on the scanned code received from the mobile device to seek retailers matching the scanned code 3279. A determination is made at 3282 if any retailers are code discernible and, if not, then the GCSI/CTIS may return an error message and/or record that no matching retailer was found 3285. If any matches are found either by position or code discernment, then a determination may be made as to whether there exist multiple retailer matches 3288. If not, then the unique matching retailer may be recorded, provided to facilitate subsequent code scan monetization, and/or the like 3294. If multiple retailer matches exist, the GCSI/CTIS system may apply one or more retailer ambiguity resolution schemes 3291 in order to narrow down the results to a unique retailer match. A wide variety of ambiguity resolution schemes may be employed within different implementations and/or embodiments of GCSI/CTIS operation. For example, in one implementation, a GCSI/CTIS system may consult a user profile associated with the user who's mobile device issued the code scan to determine whether the user's code scan history indicates a preference for a particular retailer, retailer type, and/or the like which may assist in further discerning the unique retailer associated with the current code scan. In another implementation, the GCSI/CTIS system may provide a plurality of possible retailer matches in a message (e.g., an SMS message) to a user's mobile device and request feedback as to which retailer is the correct one in whose establishment the code was scanned.

[0225] A determination may then be made as to whether a conversion has taken place whereby the user has actually purchased the item associated with the scanned code. In one implementation, the GCSI/CTIS may monitor activity on a credit card associated with a user account once that user has registered a code scan. A subsequent purchase using the associated credit card may then register a conversion with the GCSI/CTIS. If no conversion is detected, such as within a pre-designated period of time, then the code scan monetization is complete for the particular code scan 3230. If, on the other hand, a conversion is registered, a determination may be made as to whether the manufacturer is subscribed to pay further fees upon item conversion 3235. If so, then the additional manufacturer payment for conversion is registered at 3240.

[0226] A determination may be made as to whether the conversion, i.e., the purchase of the code- associated item, was made at the retailer at which the code was scanned 3245. If so, then a determination may be made as to whether the retailer is subscribed to pay an additional fee associated with item conversion 3250. If so, that payment is registered at 3255. If the conversion occurs at a competitor retailer different than the original retailer at which the item-associated code was scanned (e.g., another brick-and-mortar retailer, another website, etc.), a determination may be made as to whether the competitor retailer is subscribed to pay a fee for item conversion 3260. If so, then the competitor payment may be registered at 3265. In one implementation, a competitor may be permitted to supply a message (e.g., an SMS text message, and/or the like) to a user's mobile device when the user scans codes associated with particular items at particular retail locations in order to entice the user to purchase the item from the competitor instead of at the particular retail location in which the code was scanned. The competitor message may, for example, include a coupon and/or other discount on the sale price of the code-associated item. In one implementation, the competitor may be charged a separate fee for being allowed to supply such messages, such as a per-message fee, a one-time or periodic fee, and/or the like.

[0227] In some implementations, fees charged to manufacturers, retailers, and/or competitors may be substantially the same. In alternative implementations, different fees may be charged to manufacturers, retailers, and/or competitors.

[0228] FIGS. 33a-e show aspects of implementations of JagTag encoding and codes in some embodiments of GCSI/CTIS operation. FIG. 33a shows an implementation of logic flow for JagTag encoding in one embodiment of GCSI/CTIS operation. An alphanumeric string may be associated with content to which a JagTag code is ultimately to be associated 3301. In one implementation, a five character string is employed. Each character of the alphanumeric string may then be converted into a corresponding integer, such as may be based on a correspondence table 3302. FIG. 33b shows an implementation of such a corresponding table relating decoded characters, including lowercase letters 3313, numbers 3314, and uppercase letters 3315, to encoded values 3316 comprising integers between 0 and 62. To convert the alphanumeric string into a JagTag code, each character in the string is converted into a corresponding integer between 0 and 62 using the table in FIG. 33b. A determination may then be made as to whether any of the characters comprise punctuation marks or other unrecognizable or uncovertable characters 3303. If so, those characters are set to blanks or zeros in the resulting string of integers 3304. A

determination may also be made as to whether the number of characters in the alphanumeric string exceeds a pre-designated maximum number 3305 (e.g., 5 characters). If so, additional characters are ignored 3306.

[0229] Each resulting integer may then be converted to a corresponding binary value 3307, and a determination may be made as to whether there are a sufficient number of "on" bits (i.e., bits equal to 1) 3308. For example, the GCSI/CTIS may require the existence of at least some minimum number (e.g., 9) of on bits. If not, then the system may return an error message and/or recommend that the user try a different initial alphanumeric string 3309. Otherwise, the GCSI/CTIS system may concatenate the string of binary numbers to yield a single 32-bit long bitset array 3310. The system may then generate a JagTag bit representation 331 1 , as described below, and output the resulting JagTag to a physical display and/or storage (e.g., storing an image formatted JagTag in a database, printing the JagTag, displaying the JagTag on a display screen, and/or the like) 3312.

[0230] The first bit of the bitset array may be considered a most-significant bit and the last bit of the bitset array may be considered a least-significant bit. Once complete, the 32-bit binary bitset array may be converted into a JagTag code representation. FIG. 33c further illustrates JagTag encoding in one implementation. The JagTag may include an L-shaped component 3318 that may, in one implementation, promote orientational and/or size discrimination of the JagTag, such as by a scanning or analyzing device. The JagTag may further include four circular elements 3319 that may also serve to promote orientational and/or size discrimination. The most significant bit of the bitset array may be encoded as a circle at the position indicated by the crossed circle at 3320, with a white circle (owing to the dark background of the L-shaped component 3318) indicating a 1 and no circle indicating a 0. The next most significant bit of the bitset array, then, may be encoded as a circle at the position indicated by the crossed circle at 3325. The numbering scheme indicated at 3330 shows how the circles (or lack of circles) at positions numbered from 0 to 31 correspond to the bits in the 32-bit bitset array, with 0 the most significant bit and 31 the least significant bit. If the bit falls on the L-shaped component 3318, a white circle is drawn for a bit value of 1 , and otherwise a dark circle is drawn for a bit value of 1.

[0231] In one implementation, a shell script may be configured to accept an alphanumeric string comprising characters to be encoded and to return a JagTag in an image format, such as a PNG, JPG, GIF, BMP, and/or the like file format. In one implementation, the shell script may be further configured to receive a color specification (e.g., an HTML-style RGB triplet).

[0232] Configuring a JagTag for optimal recognition may be accomplished in a variety of different ways within different implementations. For example, a color for the JagTag code may be selected to as to have high contrast with the background color. A high number of "on" bits (i.e., bits equal to 1) may also improve code recognition, as discussed above. Accordingly, in one implementation, resulting JagTag codes having less than a minimum number of on bits may be rejected. Other measures that may be taken in various implementations to improve JagTag recognition may include maximizing code dimensions, ensuring that the code is surrounded by a substantial margin (e.g., at least 25% of the code's width and height), and ensuring fidelity of the code by eliminating any obscuring interferences. JagTag codes may be configured for a wide variety of physical manifestations, including printing on magazine pages, billboards, newspapers, and/or the like, displaying on video display screens, television broadcasts, websites, and/or the like. JagTags may also be configured into different shapes or display configurations in various imp lementations .

[0233] FIG. 33d shows one non-limiting embodiment of data loss resilient codes 3331, with characteristics that include: reducing errors based on "bleeding" of encoded data points by using circles/dots rather than squares that touch each other (this combats fuzziness of image as well as reduced light conditions); using a stronger identifier to discover processing directionality (rotation, skew, slant)—this is the black line with white dots shown in the figure; including data points within the identifier in order to maximize encoded data as well as allow for reduction in size of code; using equidistant lengths on identifier to process for skew and slant; using constant data points within the directional identifier to process for skew and slant—this may be top left, bottom right and bottom left white dots (not indicated in FIG. 33d); reducing the amount of data encoded in the code—most likely 5-8 characters— in order to allow for size reduction of the code;

developing code characteristics so partial decoding allows for matching code to correct data on the server— for example, if only 70% of the characters from a code are discernible, the GCSI/CTIS can use server logic to extrapolate what the code is as well as the campaign/client it's associated with, i.e., information about the exact publication the code is in may be lost, but the GCSI/CTIS system can figure out the campaign and return an appropriate message; with regard to partial decode matching, server side and database logic focuses both on creating codes for clients and campaigns that are unique sets in order to enable partial decode matching as well as providing a set of rules around time, mobile number, and likelihood in order to determine the best potential match for partial decodes.

[0234] In the embodiment of the basic codes 3331 shown in FIG. 33d, design wrapped around the code does not impact the effectiveness of the decoding. High tone color differentials are allowed, so rather than black, dark blue or red could be used. The white dots in the identifiers are less likely to allow for color changes although a very light yellow or gray may be allowed.

[0235] In a further embodiment, the dots most likely to be lost in an image may be assessed and the least useful information encoded appropriately based on that assessment. An interesting opportunity for this code is that it allows for some logos to potentially replace the dots. For example, an Alltel logo can be used nearly as well as the dots, or a Nike logo is decodable with some server side code modifications.

[0236] The codes have an additional benefit in that they are in and of themselves, aesthetically pleasing, and as such offer significant advantages over previous codes. The ability in one embodiment, to vary the colors of the code, allows it to better integrate and look better in marketing contexts.

[0237] FIG. 33e shows some implementations of alternative JagTag display configurations, including circular 3335, triangular 3340, modified squares (3345, 3350, 3365), rectangular 3370, letter shaped (J shapes shown at 3355 and 3360), and/or the like. JagTags may also be displayed in proximity to and/or integrated with other codes, logos, insignias, and/or the like, such as shown at 3375.

AMBIGUOUS CODES

[0238] In an embodiment, a processor-implemented method is disclosed for processing and responding to ambiguous content requests. A scanned code image and a user ID are received and analyzed to determine if the information encoded therein is fully decodable. A scanned code disambiguation process is selected and applied to the ambiguous scanned code images to determine a most likely code. Accumulated user information in a user profile associated with the user ID is queried. A content database is queried to extract requested content based on the most likely code and the accumulated user information. The requested content is sent to the user.

CONNECTING USERS

Forming Groups

[0239] In another embodiment, the CTIS is configured to connect users based on user profiles. For example, in analogy to the "provider zone" breaches discussed above, the CTIS may also track the intersection of user space-time trajectories. In particular, if one user is within a specified distance of the projected space-time trajectory of another user with whom the first user is associated, a message may be sent to one or both users notifying them of their potential proximity. Further consideration of user profiles may also be utilized in such notifications. [0240] As previously described and illustrated in FIG. 7b, a user program interface provides a user to access semi-static profile information and add new information or edit existing

information. The user may also establish a friends list and regulate which elements of the semi- static and dynamic portions of the user profile may be shared with "friends."

[0241] FIG. 35A is a block diagram illustrating a CTIS configured to assign users having similar interests and situational information into dynamic groups according to an embodiment.

[0242] In an embodiment, the semi-static elements (3604) and the dynamic elements (3606) in a profile datastore (3602) are accessed by a dynamic group algorithm (3610) operating on a CTIS. The dynamic group algorithm (3610) identifies groups appropriate for a particular user of the CTIS. An invitation to join a group is issued to the particular user (3612). A determination is made whether the particular user accepts the invitation to join a particular group (3614). If the user declines the invitation, that is, the result of 3614 is "NO," the process ends (3616). If the user accepts the invitation, that is, the result of 3614 is "YES," the user is assigned to a group (3620a, 3620b, ... , 3620n) and a record is made in a group directory (3618). By way of illustration and not by way of limitation, a user may accept and invitation to join Group A but not Group B, may join both Group A and Group B, or reject the invitation to join any of the Groups identified by the dynamic group algorithm (3610).

[0243] In an embodiment, the logic of 3614 is reversed. That is, the user must affirmatively opt- out of the group assignment made by the CTIS.

[0244] As illustrated in FIG. 35A, the dynamic group algorithm (3610) evaluates both semi-static elements (3604) and dynamic elements (3606) of a user profile to determine what groups are appropriate for a particular user. The dynamic profile elements (3606) may include situational information (3608). Situation information may include a location of a user and a trajectory of a user. Using this information, the dynamic group algorithm may group users of the CTIS with similar profiles who are in geographic proximity to each. The geographic proximity may be as large as a city and as small as a bar, a trade show, and a concert venue.

[0245] Group members typically share core information, such as a user name. In an embodiment, a group member may specify the personal information he or she is will to share with other group members. A group member may also specify which group members it prefers to be "invisible" to. In an embodiment, the sharing of the user name allows the group members to communicate via the CTIS but does not provide direct communication to between group members. Thus, the CTIS operates a communication proxy to allow group member to be restrict communications from particular members of the group or all members of the group.

[0246] In an embodiment, the core information for a particular group may be determined by the group's purpose. The group purpose may also be associated with a particular rule set that determines how the core personal information is shared. For example, in group whose purpose is facilitate social interactions, age, religious preferences, music preferences and other personal interest details may be part of the core information. In a group whose purpose is to facilitate business interactions, the core information may be limited to professional information and information relating to products and services.

[0247] In another embodiment, the situational information (3608) may identify users the CTIS with similar profiles who are participating in a common virtual world. Users with these attributes may be assigned to a group for that particular virtual world. In this example, the core information may be specific to the virtual world and the particular user's historical interaction with that world.

[0248] Users of the CTIS with similar profiles, who are skiing, albeit in different locations, may be assigned to yet another group that utilizes a core information rule set that is specific to skiing. Users at the same ski resort may be assigned to a group specific to that resort. The shared information rule set for this group may include an option that allows members to provide indentifying information to facilitate in-person meetings.

[0249] The membership of a group may be constantly in flux as the situational information (3608) of the group members changes. In an embodiment, members of a dynamic group may make the group permanent by adding some or all of the members to a friends list associated with a group name.

[0250] FIG. 35B is a block diagram illustrating a CTIS configured to identify profile elements that are related to specific content according to an embodiment.

[0251] As previously described, users with substantially similar semi-static profile elements (3604) who scan a particular code may receive different content based on differences in their dynamic profile elements (3606).

[0252] In an embodiment, a second user who has not received particular content may request the CTIS to compare the profile of the first user with the profile of the second user to determine what factors in the dynamic elements of the first user's profile caused the content to be served to the first user but not the second user. A content analyzer (3622) receives content or a content identifier that was received by a first user of the CTIS. The content analyzer also receives the semi-static profile elements (3604) the dynamic profile elements (3606) of a second user of the CTIS from the profile datastore (3602). The content analyzer (3622) evaluates the served content and the profile data to determine what profile elements could be added to the second user's profile for the second user to receive the served content. The content analyzer generates a profile report (3626).

[0253] For example, a first user may receive a coupon for a free beverage from a vendor in response to the user passing the vendor's physical location or browsing to the vendor's website. In order to obtain the coupon, the second user may be required to travel to the physical site or navigate to the vendor's website. Note that this process differs from embodiments in which a coupon may be shared, in that the second user is compelled to engage the vendor at some level to obtain the coupon. From the vendor's perspective, the value of the coupon is used to incentivize the second user to engage the vendor.

[0254] In many cases, the reasons for serving the content to the first user may be more complex than suggested in the example. For example, a coupon may have been served to the first user based on the first user's history of interest in beverages and purchases made from the vendor or establishments similar to the vendor. Even if the second user were to visit the physical or virtual location of the vendor, the second user may not meet the requirements for receiving the coupon.

[0255] In an embodiment, the CTIS may optionally offer to supplement the second user's profile to include elements that would qualify the second user for the served content (3628). If the CTIS does not make an offer to supplement the second user's profile, that is, if the result of 3628 is "NO," the process ends (3630). If the CTIS makes an offer to supplement the second user's profile, that is, the result of 3630 is "YES," then a determination is made whether the second user accepts the offer (3632). If the second user rejects the offer, that is, the result of 3632 is "NO," the process ends (3630). If the second user accepts the offer, that is, the result of 3632 is "YES," the second user's profile is assigned supplemental profile elements (3634). The "supplemental" profile elements may be stored separately from other elements and may be temporary with a defined "time-to-live" or may be permanently stored in association with the user's identifier. The served content is acquired using the supplemental profile elements (3636).

[0256] In an embodiment, a code or token may be provided to the second user that could be presented by the second user to the physical or virtual location of the vendor that would cause the coupon to be served to the second user without regard to the second user's profile. In an embodiment, the token or code may be provided only with the vendor's consent.

EMERGENCY

[0257] In another embodiment, the CTIS provides a means of disseminating targeted information or Actionable Events over a large scale. For example, during a widespread emergency, a single 2D matrix code could be broadcast on television. When users scan the code, they are supplied with regional information/instructions, shelter locations, maps, etc. Certain users may be provided even more specific instructions if, for example, their profile lists their occupation as some form of emergency personnel. In an alternative embodiment, the CTIS can provide targeted information to any or all users expected to be in the vicinity of a particular area or event based on their history of space-time trajectories. For example, if a chemical spill occurs and users are projected to be in the vicinity of it, a prior warning to avoid the area can be broadcast to the users' mobile devices. In an alternative embodiment, traffic alerts can be broadcast to the mobile devices of users projected to be in the vicinity of a high-traffic area before they are stuck there. The CTIS may supply targeted Actionable Events, including coupons, for nearby businesses, restaurants, coffee shops, etc. to these users for them to solicit while they wait for traffic to disperse. In general, just as the CTIS can provide information to users about where they should go (e.g., Actionable Event provider locations), it can also warn them to stay out of areas that they should not go, all based on the projected user trajectories.

REAL ESTATE LISTINGS

[0258] In another embodiment, the CTIS provides an efficient mechanism of serving real estate listings. For example, when a user scans a code related to real estate listings, the CTIS supplies him/her with advertisements and/or listings that are close to their geographic location and appropriate to their demographic profile (i.e., income, marital status, number of children, etc.). In addition, the CTIS may provide additional listings to the user based on projected breaches of zones corresponding to available homes or apartments that suit the user's profile.

[0259] In another embodiment, the CTIS expedites software downloads to a user's mobile device. For example, by scanning a 2D matrix code in an ad for a cell phone ringtone version of a popular song, the user can automatically download the ringtone to the cell phone with which they scan the code. The transaction is recorded in the user profile and may result in the display of an Actionable Event related to the latest album by the artist who recorded the ringtone song, or an album by a related artist, at some future time. REDUCED PRINTING

[0260] In another embodiment, the CTIS allows for reduced use of printed space in media publications. For example, an ordinary printed article or advertisement may be supplanted by a 2D matrix code that supplies the content directly to the user's mobile display device. Furthermore, the supplied content may be dressed with additional content that is specifically chosen for the user based on his/her profile or current location, thus increasing the value and utility of the content while still utilizing significantly reduced print space. This increased content value, coupled with the reduction in print space, may translate to significantly increased advertising revenue for media publications. In another embodiment, scan codes may be placed in web-based ads so that users may scan those ads with their mobile device. In another embodiment, scan codes may be placed in television-based ads.

NEWS

[0261] In another embodiment, the CTIS provides a means to supply urgent or breaking information. For example, a breaking news story that arrives too late to be included as a printed article can still be incorporated into a publication via a 2D matrix code. Publications could even have a permanent code in each issue that users can scan in order to access timely breaking and/or supplemental news for which there was a lack of space or time to include in the printed publication.

[0262] A system for communicating an actionable event in response to scanning a multi-modal artifact within a media expression, comprises: a processor; a memory in communication with the processor and containing program instructions; an input and output in communication with the processor and memory; wherein the processor executes program instructions contained in the memory and the program instructions comprise: receiving multi-modal artifact data scanned from the media expression; communicating at least some of the multi-modal artifact data to a data repository; comparing the communicated data with the data present in the data repository; and sending at least one actionable event from the data repository based upon the communicated data. The actionable event may be the user purchasing a product or service associated with the scanned multi-modal artifact data. The multi-modal artifact data is representative of at least one sensory input of visual, auditory, haptics, olfactory, or gustatory. The multi-modal artifact data is representative of data that is digitally encoded. The scanned multi-modal artifact data may comprise data that is representative of a barcode, 2D code, matrix code, data matrix, and/or QR code. The scanned multi-modal artifact data may comprise data that is representative of a pure image, the selection of an internet link by a user, RFID tags, NFC tags, and/or software inputs. The scanned multi-modal artifact data is communicated to the data repository by SMS protocol, MMS protocol, instant messaging, web browser based messaging, email, Enhanced Messaging System, TCP/IP, and/or WAP. The data repository is a component of a code triggered information server database. The code triggered information server database communicates with a third party provider. The at least one actionable event received from the data repository comprises a SMS or MMS message. The at least one actionable event received from the data repository comprises product or service information, a URL link, a coupon, a multimedia clip, and/or Reply/SMS Message or Actionable Event content.

[0263] A method of linking a media expression to an actionable event, comprises: receiving an instruction from a user; identifying the type of product or service information requested; receiving multi-modal artifact data associated with the media expression; identifying at least one product or service associated with the multi-modal artifact data; retrieving product or service information associated with the at least one identified product or service and the type of product information requested; retrieving user information associated with the type of product or service information requested; composing an actionable event including the retrieved product information and the retrieved user information; and communicating the actionable event to the user.

Claims

Claims What is claimed is:

1. A method of linking a media expression to an actionable event, comprising:

a. Receiving multi-modal artifact data scanned from the media expression;

b. Communicating at least some of the scanned multi-modal artifact data to a data repository;

c. Comparing the communicated data with the data present in the data repository; and d. Sending at least one actionable event from the data repository based upon the

communicated data.

2. The method of Claim 1, wherein the actionable event is selected from the group consisting essentially of: the user purchasing a product or service associated with the scanned multi-modal artifact data; offering a discount code associated with the scanned multi-modal artifact data; offering a promotional subscription associate with the scanned multi-modal artifact data; or a consumer inducement.

3. The method of Claim 1, wherein the at least one actionable event is coupled to a mobile payment system.

4. The method of Claim 1, wherein the scanned multi-modal artifact data comprises data being representative of at least one sensory input of visual, auditory, haptics, olfactory, or gustatory.

5. The method of Claim 1, wherein the scanned multi-modal artifact data comprises data that is present in a media expression, wherein the media expression is one of a television, radio, internet, video games, or 3-D environments.

6. The method of Claim 1, wherein the scanned multi-modal artifact comprises data that is representative of a barcode, 2D code, 3D code, 3D-VR code, matrix code, data matrix, QR code, Near Field Communications (NFC) tag, Radio Frequency Identification (RFID) tag, Bluetooth, Infrared, and/or Wireless communication.

7. The method of Claim 1, further comprising levying a fee from a third party associated with the scanned multi-modal artifact in response to the sending of the at least one actionable event.

8. The method of Claim 1, wherein the at least some scanned multi-modal artifact data is communicated to the data repository by SMS protocol, MMS protocol, instant messaging, web browser based messaging, email, Enhanced Messaging System, TCP/IP, and/or WAP.

9. The method of Claim 1, wherein the data repository is a component of a code triggered information server database.

10. The method of Claim 9, wherein the code triggered information server database

communicates with a third party provider.

11. The method of Claim 1 , wherein the communicated data further comprises a user ID, a hardware ID, a geocode, a timestamp, a subject code, a source code, a content code, and/or a media expression code.

12. The method of Claim 1, wherein the at least one actionable event from the data repository comprises product or service information, a URL link, a coupon, a multimedia clip, and/or Reply/SMS Message or Actionable Event content.

13. A method of mobile purchasing of a product or service associated with a media expression, comprising:

a. Capturing multi-modal artifact data present within the media expression;

b. Processing the captured multi-modal artifact data and communicating at least some of the captured multi-modal artifact data to a data repository; and

c. Receiving an actionable event from the data repository, the actionable event containing at least some data corresponding to the captured multi-modal artifact data.

14. The method of Claim 13, wherein the captured multi-modal artifact data comprises data being representative of at least one sensory input of visual, auditory, haptics, olfactory, or gustatory.

15. The method of Claim 13, wherein the actionable event comprises a prompt to purchase a product or service associated with the captured multi-modal artifact data.

16. The method of Claim 13, wherein the purchase of the product is coupled to a mobile payment system.

17. A method of linking a media expression to an actionable event, comprising:

a. Capturing multi-modal artifact data present within the media expression;

b. Communicating at least some of the captured multi-modal artifact data to a data repository;

c. Comparing the communicated multi-modal artifact data with data present in the data repository; and

d. Returning at least one actionable event from the data repository based upon the communicated data.

18. The method of Claim 17, wherein the captured multi-modal artifact data comprises data being representative of at least one sensory input of visual, auditory, haptics, olfactory, or gustatory.

19. The method of Claim 17, wherein the at least one actionable event comprises a prompt to purchase a product or service associated with the scanned multi-modal artifact data.

20. The method of Claim 19, wherein the purchase of the product is coupled to a mobile payment system.