[go: up one dir, main page]

WO2015116270A2 - Appareil et procédés de commande robotique à degré de liberté réduit - Google Patents

Appareil et procédés de commande robotique à degré de liberté réduit Download PDF

Info

Publication number
WO2015116270A2
WO2015116270A2 PCT/US2014/063540 US2014063540W WO2015116270A2 WO 2015116270 A2 WO2015116270 A2 WO 2015116270A2 US 2014063540 W US2014063540 W US 2014063540W WO 2015116270 A2 WO2015116270 A2 WO 2015116270A2
Authority
WO
WIPO (PCT)
Prior art keywords
freedom
input
training
control signal
degree
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2014/063540
Other languages
English (en)
Other versions
WO2015116270A3 (fr
Inventor
Jean-Baptiste Passot
Oleg SINYAVSKIY
Eugene Izhikevich
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Brain Corp
Original Assignee
Brain Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US14/070,269 external-priority patent/US9566710B2/en
Priority claimed from US14/070,114 external-priority patent/US9463571B2/en
Priority claimed from US14/070,239 external-priority patent/US20150127154A1/en
Application filed by Brain Corp filed Critical Brain Corp
Publication of WO2015116270A2 publication Critical patent/WO2015116270A2/fr
Publication of WO2015116270A3 publication Critical patent/WO2015116270A3/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/004—Artificial life, i.e. computing arrangements simulating life
    • G06N3/008—Artificial life, i.e. computing arrangements simulating life based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs

Definitions

  • the present disclosure relates to adaptive control and training, such as control and training of robotic devices.
  • Robotic devices are used in a variety of industries, such as manufacturing, medical, safety, military, exploration, etc.
  • Robotic "autonomy'; i.e., the degree of human control, varies significantly according to application.
  • Some existing robotic devices e.g., manufacturing assembly and/or packaging
  • Some robotic devices may be programmed in order to perform desired functionality without further supervision.
  • Some robotic devices e.g., surgical robots
  • Robotic devices may comprise hardware components that enable the robot to perform actions in 1 -dimension ⁇ e.g., a single range of movement), 2-dimensions ⁇ e.g., a plane of movement), and/or 3-dimensions ⁇ e.g., a space of movement).
  • movement is characterized according to so-called '"degrees of freedom.
  • a degree of freedom is an independent range of movement; a mechanism with a number of possible independent relative movements (N) is said to have N degrees of freedom.
  • Some robotic devices may operate with multiple degrees of freedom (e.g., a turret and/or a crane arm configured to rotate around vertical and/or horizontal axes). It is further appreciated that some robotic devices may simultaneously control multiple actuators (degrees of freedom) resulting in very complex movements.
  • One aspect of the disclosure relates to a non-transitory computer readable medium having instructions embodied thereon.
  • the instructions when executed, are configured to control a robotic platform.
  • a user interface apparatus is disclosed.
  • the apparatus is for use with a robotic apparatus having a controller, and includes: user input apparatus; and first computerized logic in data communication with the input apparatus and configured to, as part of a first operational iteration, provide an input to the controller.
  • the input is associated with operation of one of first and second degrees of freedom of the robotic apparatus, and is configured to enable the controller to produce: a first control signal to control the first degree of freedom; and a second control signal to control the second degree of freedom, the second control signal being determined based at least on another input being provided via the input apparatus at another iteration, the another input being associated with the second degree of freedom.
  • the given iteration and the another iteration may be characterized by one or more features, such as e.g., an empty intersection.
  • the apparatus further includes switching logic), wherein the switching logic comprises one or more of: (i) a button, (ii) a switch, (iii) a timer, (iv) condition determination logic, (v) logic configured to determine a state of the robotic apparatus, (vi) performance measurement determination logic; and/or (vii) an audio indication detection logic.
  • the switching logic comprises one or more of: (i) a button, (ii) a switch, (iii) a timer, (iv) condition determination logic, (v) logic configured to determine a state of the robotic apparatus, (vi) performance measurement determination logic; and/or (vii) an audio indication detection logic.
  • a method of training an adaptive controller apparatus is disclosed.
  • the controller apparatus is associated with a robot comprising first and second degrees of freedom
  • the method includes: during a first operational iteration: causing the apparatus to produce a first control signal based at least on a context and a first training input, the first control signal being configured to operate the first degree of freedom to execute a first action; and; causing the apparatus to adjust a learning parameter based at least on a first performance relating to the first action and a target action.
  • the method further includes, during a subsequent iteration: causing the apparatus to produce a second control signal based at least on the context and a second training input, the second control signal being configured to operate the second degree of freedom; and causing the apparatus to produce a third control signal based at least on the context and the adjusted learning parameter, the third control signal being configured to operate the first degree of freedom.
  • the second and the third control signals cooperate to cause execution of the target action.
  • a robotic apparatus in another aspect, includes: a platform characterized by at least first and second degrees of freedom; and an adaptive controller apparatus configured to generate first and a second control signals adapted to operate the first and the second degrees of freedom, respectively.
  • the first and the second control signals are configured to cause the platform to perform a target action; and the operation of a given one of the first or the second degrees of freedom is configured based at least on a training input associated with the given degree of freedom.
  • the adaptive controller apparatus is operable in accordance with a supervised learning process configured to produce one of the first or the second control signals at a given iteration based on at least the training input.
  • a robotic system in a further aspect, includes a controlled (robotic) apparatus having multiple degrees of freedom, and an adaptive controller configured to receive input from a supervisory entity (e.g., user, algorithm, etc.) and utilize the input to train the robotic apparatus for subsequent iterations of movement within the degrees of freedom.
  • a supervisory entity e.g., user, algorithm, etc.
  • FIG. 1 is a graphical illustration depicting a robotic manipulator apparatus operable in two degrees of freedom, according to one or more implementations.
  • FIG. 2 is a graphical illustration depicting a robotic control apparatus configured to activate a single robotic actuator at a given time, according to one or more implementations.
  • FIG. 3 is a graphical illustration depicting a robotic rover platform operable in two degrees of freedom, according to one or more implementations.
  • F!G. 4 is a graphical illustration depicting a multilayer neuron network configured to operate multiple degrees of freedom of, e.g., a robotic apparatus of FIG. 1 , according to one or more implementations.
  • FIG. 5 is a graphical illustration depicting a single layer neuron network configured to operate multiple degrees of freedom of, e.g., a robotic apparatus of FIG. 1 , according to one or more implementations.
  • FIG. 6 is a logical flow diagram illustrating a method of operating an adaptive robotic device, in accordance with one or more implementations.
  • FIG. 7 is a logical flow diagram illustrating a method of training an adaptive controller of a robot using a reduced degree of freedom methodology, in accordance with one or more implementations.
  • FIG. 8 is a logical flow diagram illustrating a method of training an adaptive controller apparatus to control a robot using a reduced degree of freedom methodology, in accordance with one or more implementations.
  • FIG. 9A is a block diagram illustrating a computerized system useful for, inter alia, operating a parallel network configured using reduced degree of freedom training methodology, in accordance with one or more implementations.
  • FIG. 9B is a block diagram illustrating a cell-type neuromorphic computerized system useful with, inter alia, reduced degree of freedom methodology of the disclosure, in accordance with one or more implementations.
  • FIG. 9C is a block diagram illustrating a hierarchical neuromorphic computerized system architecture useful with, inter alia, reduced degree of freedom training methodology, in accordance with one or more implementations.
  • FIG. 9D is a block diagram illustrating cell-type neuromorphic computerized system architecture useful with, inter alia, reduced degree of freedom training methodology, in accordance with one or more implementations.
  • bus is meant generally to denote all types of interconnection or communication architecture that are used to access the synaptic and neuron memory.
  • the "bus” may be electrical, optical, wireless, infrared, and/or any type of communication medium.
  • the exact topology of the bus could be, for example: a standard "bus", a hierarchical bus, a network-on-chip, an address-event-representation (AER) connection, and/or any other type of communication topology configured to access e.g., different memories in a pulse-based system.
  • AER address-event-representation
  • “computerized device” may include one or more of personal computers (PCs) and/or minicomputers (e.g., desktop, laptop, and/or other PCs), mainframe computers, workstations, servers, personal digital assistants (PDAs), handheld computers, embedded computers, programmable logic devices, personal communicators, tablet computers, portable navigation aids, J2ME equipped devices, cellular telephones, smart phones, personal integrated communication and/or entertainment devices, and/or any other device capable of executing a set of instructions and processing an incoming data signal.
  • PCs personal computers
  • minicomputers e.g., desktop, laptop, and/or other PCs
  • mainframe computers workstations
  • servers personal digital assistants
  • handheld computers handheld computers
  • embedded computers embedded computers
  • programmable logic devices personal communicators
  • tablet computers tablet computers
  • portable navigation aids J2ME equipped devices
  • J2ME equipped devices J2ME equipped devices
  • cellular telephones cellular telephones
  • smart phones personal integrated communication and/or entertainment devices
  • any other device capable of
  • the term "computer program” or "software” may include any sequence of human and/or machine cognizable steps which perform a function. Such program may be rendered in a programming language and/or environment including one or more of C/C++. C#, Fortran, COBOL, MATLABTM, PASCAL, Python, assembly language, markup languages (e.g., HTML, SGML, XML, VoXML), object-oriented environments (e.g., Common Object Request Broker Architecture (CORBA)), JavaTM (e.g., J2ME, Java Beans), Binary Runtime Environment (e.g., BREW), and/or other programming languages and/or environments.
  • CORBA Common Object Request Broker Architecture
  • JavaTM e.g., J2ME, Java Beans
  • Binary Runtime Environment e.g., BREW
  • transmission channel includes a link between any two or more entities (whether physical (wired or wireless), or logical/virtual) which enables information exchange between the entities, and may be characterized by a one or more variables affecting the information exchange.
  • memory may include an integrated circuit and/or other storage device adapted for storing digital data.
  • memory may include one or more of ROM, PROM, EEPROM, DRAM, Mobile DRAM, SDRAM, DDR 2 SDRAM, EDO/FPMS, RLDRAM, SRAM, "flash” memory (e.g., NAND/NOR), memristor memory, PSRAM, and/or other types of memory.
  • integrated circuit (1C) and “chip” are meant to refer without limitation to an electronic circuit manufactured by the patterned diffusion of elements in or on to the surface of a thin substrate.
  • integrated circuits may include field programmable gate arrays (e.g., FPGAs), programmable logic devices (PLD), reconfigurable computer fabrics (RCFs), application-specific integrated circuits (ASICs), printed circuits, organic circuits, and/or other types of computational circuits.
  • FPGAs field programmable gate arrays
  • PLD programmable logic devices
  • RCFs reconfigurable computer fabrics
  • ASICs application-specific integrated circuits
  • printed circuits organic circuits, and/or other types of computational circuits.
  • microprocessor and “digital processor” are meant generally to include digital processing devices.
  • digital processing devices may include one or more of digital signal processors (DSPs), reduced instruction set computers (RISC), general-purpose (CISC) processors, microprocessors, gate arrays (e.g., field programmable gate arrays (FPGAs)), PLDs, reconfigurable computer fabrics (RCFs), array processors, secure microprocessors, application-specific integrated circuits (ASICs), and/or other digital processing devices.
  • DSPs digital signal processors
  • RISC reduced instruction set computers
  • CISC general-purpose processors
  • microprocessors gate arrays (e.g., field programmable gate arrays (FPGAs)), PLDs, reconfigurable computer fabrics (RCFs), array processors, secure microprocessors, application-specific integrated circuits (ASICs), and/or other digital processing devices.
  • FPGAs field programmable gate arrays
  • RCFs reconfigurable computer fabrics
  • ASICs application-
  • a network interface refers to any signal, data, and/or software interface with a component, network, and/or process.
  • a network interface may include one or more of FireWire (e.g., FW400, FW800, etc.), USB (e.g., USB2), Ethernet (e.g., 10/100, 10/100/1000 (Gigabit Ethernet), 10-Gig-E, etc.).
  • FireWire e.g., FW400, FW800, etc.
  • USB e.g., USB2
  • Ethernet e.g., 10/100, 10/100/1000 (Gigabit Ethernet), 10-Gig-E, etc.
  • MoCA MoCA
  • Coaxsys e.g., TVnetTM
  • radio frequency tuner e.g., in-band or OOB, cable modem, etc.
  • Wi-Fi 802.1 1
  • WiMAX 802.16
  • PAN e.g., 802.15
  • cellular e.g., 3G, LTE/LTE-A/TD-LTE, GSM, etc.
  • IrDA families and/or other network interfaces.
  • Wi-Fi includes one or more of IEEE-Std. 802.1 1, variants of IEEE-Std. 802.1 1 , standards related to IEEE-Std. 802.1 1 (e.g., 802.1 1 a/b/g/n/s/v), and/or other wireless standards.
  • wireless means any wireless signal, data, communication, and/or other wireless interface.
  • a wireless interface may include one or more of Wi-Fi, Bluetooth, 3G (3GPP/3GPP2), HSDPA/HSUPA, TDMA, CDMA (e.g., IS-9 A, WCDMA, etc.), FHSS, DSSS, GSM, PAN/802.15, WiMAX (802.16), 802.20, narrowband/FDMA, OFDM, PCS/DCS, LTE/LTE-A/TD-LTE, analog cellular, CDPD, satellite systems, millimeter wave or microwave systems, acoustic, infrared (i.e., IrDA), and/or other wireless interfaces.
  • a robot may be trained by a user using a supervised learning approach.
  • the user may interface to the robot via a control apparatus.
  • the user may be unable to control a target number of controllable degrees of freedom (CDOF) of the robot simultaneously.
  • CDOF controllable degrees of freedom
  • the user's inability to control the target number of the robot's CDOF may arise from a mismatch between configurations of the remote control (e.g., a single axis joystick and or slider) and the robot ' s capabilities (e.g., a rover with multi-axes actuators).
  • the user's inability to simultaneously operate the target number of robot's CDOF may be due to a physical and/or mental complexity of the task arising from e.g., controlling multiple CDOF simultaneously in multiple spatial dimensions (e.g., a user attempting to train a robotic manipulator arm with six or more CDOF), user dexterity, and/or other considerations.
  • the robot may implement an adaptive controller implemented using e.g., a neuron network. Training the adaptive controller may comprise partial set training during so- called "trials". The user may train the adaptive controller to separately train a first actuator subset and a second actuator subset of the robot. During a first set of trials, the control apparatus may be configured to select and operate a first subset of the robot's complement of actuators e.g., operate a shoulder joint of a manipulator arm. The adaptive controller network may be configured to generate control commands for the shoulder joint actuator based on the user input and output of the learning process.
  • Training the adaptive controller may comprise partial set training during so- called "trials”.
  • the user may train the adaptive controller to separately train a first actuator subset and a second actuator subset of the robot.
  • the control apparatus may be configured to select and operate a first subset of the robot's complement of actuators e.g., operate a shoulder joint of a manipulator arm.
  • the adaptive controller network may be configured to generate control commands for the
  • the adaptive controller may be trained to operate the second subset (e.g., an elbow joint) during a second set of trials.
  • the user may provide control input for the second actuator, while the previously trained network may provide control signaling for the first actuator (the shoulder).
  • the adaptive controller may be capable of controlling the first and the second actuators in absence of user input by e.g., combining the training of the first and second trials.
  • FIG. 1 illustrates one implementation of a robotic apparatus for use with the robot training methodology set forth herein.
  • the apparatus 100 of FIG. 1 may comprise a manipulator arm comprised of limbs 1 10, 1 12.
  • the limb 1 10 orientation may be controlled by a motorized joint 102
  • the limb 1 12 orientation may be controlled by a motorized joint 106.
  • the joints 102, 106 may enable control of the arm 100 in two degrees of freedom, shown by arrows 108, 1 18 in FIG. 1.
  • the robotic arm apparatus 100 may be controlled in order to perform one or more target actions, e.g., reach a target 120.
  • the arm 100 may be controlled using an adaptive controller (e.g., comprising a neuron network described below with respect to FIGS. 4-5).
  • the controller may be operable in accordance with a supervised learning process described in e.g., commonly owned, and co-pending U.S.
  • the supervised learning process may receive supervisory input (training) from a trainer.
  • the trainer may comprise a computerized agent and/or a human user.
  • the training input may be provided by the user via a remote control apparatus e.g., such as illustrated in FIG. 2.
  • the control apparatus 200 may be configured to provide teaching input to the adaptive controller and/or operate the robotic arm 100 via control element 214.
  • control element 214 comprises a slider with a single direction 218 representing one degree of freedom (DOF).
  • DOF degree of freedom
  • a lateral or “translation” degree of freedom refers to a displacement with respect to a point of reference.
  • a rotational degree of freedom refers to a rotation about an axis.
  • Other common examples of control elements include e.g., joysticks, touch pads, mice, track pads, dials, etc. More complex control elements may offer even more CDOF; for example, so called 6DOF controllers may offer translation in 3 directions (forward, backward, up/down), and rotation in 3 axis (pitch, yaw, roll).
  • the control apparatus 200 provides one or more control signals (e.g., teaching input).
  • the one or more control signals represent a fewer number of DOF than the robot can support.
  • the control apparatus 200 provides control signals for a single (1) CDOF, whereas the robotic arm 100 supports two (2) CDOFs.
  • the control apparatus 200 may further comprise a switch element 210 configured to select the joint 102 or joint 106 the control signals should be associated with.
  • Other common input apparatus which may be useful to specify the appropriate CDOF include, without limitation: buttons, keyboards, mice, and/or other devices.
  • the control apparatus 200 may be utilized to provide supervisory input to train a mobile robotic platfonn 300 characterized by two degrees of freedom (indicated by arrows 314, 310).
  • the platfonn 300 may comprise a motorized set of wheels 312 configured to move the platfonn (as shown, along the direction 314).
  • the platform 300 may also comprise a motorized turret 304 (adapted to support an antenna and/or a camera) that is configured to be rotated about the axis 310.
  • the supervisory signal comprises: (i) an actuator displacement value (selected by the slider 218), (ii) a selection as to the appropriate actuator mechanism (selected by the switch element 210), torque values for individual joints, and/or other.
  • the actuators control the angular displacement for the robotic limbs.
  • the actuators control the linear displacement (via a motorized wheel drive), and a rotational displacement about the axis 310.
  • supervisory signals that include e.g., multiple actuator displacement values (e.g., for multi-CDOF controller elements), multiple actuator selections, and/or other.
  • the illustrated examples are readily understood to translate the value from the actuator displacement value to a linear displacement, angular displacement, rotational displacement, etc. Translation may be proportional, non- proportional, linear, non-linear, etc.
  • the actuator displacement value may be 'Tine" over some ranges (e.g., allowing small precision manipulations), and much more "coarse ' ' over other ranges (e.g., enabling large movements).
  • velocity values may also be used.
  • an actuator velocity value may indicate the velocity of movement which may be useful for movement which is not bounded within a range per se.
  • the motorized wheel drive and the turret rotation mechanisms may not have a limited range.
  • actuator mechanisms vary widely based on application. Actuators may use hydraulic, pneumatic, electrical, mechanical, etc. mechanisms to generate e.g., linear force, rotational force, linear displacement, angular displacement, etc. Common examples include: pistons, comb drives, worm drives, motors, rack and pinion, chain drives, etc.
  • the training signal may comprise a supervisory signal (e.g., a spike) that triggers neuron response.
  • a supervisory signal e.g., a spike
  • FIGS. 4-5 adaptive controllers of robotic apparatus (e.g., 100, 300 of FIGS. 1 , 3) comprising a neuron network is graphically depicted.
  • FIG. 4 a multilayer neuron network configured to control multiple degrees of freedom (e.g., the robotic arm apparatus 100 of FIG. 1 ), according to one or more implementations is presented.
  • the multilayer network 500 of neurons is depicted within FIG. 4.
  • the 500 comprises: an input neuron layer (neurons 502, 504, 506), a hidden neuron layer (neurons 522, 524, 526), and an output neuron layer (neurons 542, 544).
  • the neurons 502, 504, 506 of the input layer may receive sensory input 508 and communicate their output to the neurons 522, 524, 526 via one or more connections (512, 514, 516 in FIG. 4).
  • the input layer of neurons may be referred to as non-adaptive feature extraction layer that is configured to respond to occurrence of one or more features/objects (e.g., edges, shapes, color, and or other) represented by the input 508.
  • the neurons 522, 524, 526 of the hidden layer may communicate output (generated based on one or more inputs 512, 514, 516 and feedback signal 530) to one or more output layer neurons 542, 544 via one or more connections (532, 534, 536 in FIG. 5).
  • the network 500 of FIG. 4 may be referred to as the two-layer network comprising two learning layers: layer of connections between the input and the hidden neuron layers (e.g., 512, 514, characterized by efficacies 518, 528), and layer of connections between the hidden and the output neuron layers (e.g., 532, 534 characterized by efficacies 548, 538).
  • network connectivity may be e.g., one-to-one, one-to-all, all-to-one, some to some, and/or other methods.
  • a network layer may provide an error feedback signal to a preceding layer.
  • the neurons (542, 544) of the output layer provide error feedback to the neurons (522, 524, 526) of the hidden layer.
  • T neurons (522, 524, 526) of the hidden layer provide feedback to the input layer neurons (502, 504, 506).
  • the error propagation may be implemented using any applicable methodologies including those described in , e.g. U.S. Patent Application Serial No. 13/465.903 entitled "APPARATUS AND METHODS FOR BACKWARD PROPAGATION OF ERRORS IN A SPIKING NEURON NETWORK' * , filed October 15, 2013, incorporated herein by reference in its entirety.
  • the exemplary network 500 may comprise a network of spiking neurons configured to communicate with one another by means of "spikes” or electrical pulses.
  • pre-synaptic and ''postsynaptic '* are used to describe a neuron ' s relation to a connection.
  • the units 502 and 522 are referred to as the pre-synaptic and the post-synaptic unit, respectively. It is noteworthy, that the same unit is referred to differently with respect to different connections.
  • unit 522 is referred to as the pre-synaptic unit with respect to the connection 532, and the post-synaptic unit with respect to the connection 512.
  • the error signal 520, 530 may be propagated using spikes, e.g., as described in U.S. Patent Application Serial No. 14/054,366, entitled "APPARATUS AND METHODS FOR BACKWARD PROPAGATION OF ERRORS IN A SPIKING NEURON NETWORK", filed October 15, 2013, the foregoing being incorporated herein by reference in its entirety.[
  • the input 508 may comprise data used for solving a particular control task.
  • the signal 508 may comprise a stream of raw sensor data and/or preprocessed data.
  • Raw sensor data may include data conveying information associated with one or more of proximity, inertia!, terrain imaging, and/or other information.
  • Preprocessed data may include data conveying information associated with one or more of velocity, information extracted from accelerometers, distance to obstacle, positions, and/or other information.
  • the signal 508 may comprise an array of pixel values in the input image, or preprocessed data.
  • Preprocessed data may include data conveying information associated with one or more of levels of activations of Gabor filters for face recognition, contours, and/or other information.
  • the input signal 508 may comprise a target motion trajectory.
  • the motion trajectory may be used to predict a future state of the robot on the basis of a current state and the target state.
  • the signal 508 in FIG. 4 may be encoded as spikes, as described in detail in commonly owned, and co-pending U.S. Patent Application Serial No. 13/842,530 entitled “ADAPTIVE PREDICTOR APPARATUS AND METHODS", filed March 15, 2013, incorporated supra.
  • the input 508 may comprise a stream of pixel values associated with one or more digital images.
  • the input may comprise electromagnetic waves (e.g., visible light, IR, UV, and/or other types of electromagnetic waves) entering an imaging sensor array.
  • the imaging sensor array may comprise one or more of RGCs, a charge coupled device (CCD), an active-pixel sensor (APS), and/or other sensors.
  • the input signal may comprise a sequence of images and/or image frames.
  • the sequence of images and/or image frame may be received from a CCD camera via a receiver apparatus and/or downloaded from a file.
  • the image may comprise a two-dimensional matrix of RGB values refreshed at a 25 Hz frame rate.
  • image parameters are merely exemplary, and many other image representations (e.g., bitmap, CMYK, HSV, HSL, grayscale, and/or other representations) and/or frame rates are equally useful with the present technology.
  • Pixels and/or groups of pixels associated with objects and/or features in the input frames may be encoded using, for example, latency encoding described in commonly owned and co-pending U.S. Patent Application No.
  • encoding may comprise adaptive adjustment of neuron parameters, such neuron excitability described in commonly owned and co-pending U.S. Patent Application No. 13/623,820 entitled “APPARATUS AND METHODS FOR ENCODING OF SENSORY DATA USING ARTIFICIAL SPIKING NEURONS", filed September 20, 2012, the foregoing being incorporated herein by reference in its entirety.
  • connection efficacy which in general may refer to a magnitude and/or probability of input into a neuron affecting neuron output.
  • the efficacy may comprise, for example a parameter (e.g., synaptic weight) used for adaptation of one or more state variables of post-synaptic units (e.g., 530).
  • the efficacy may comprise a latency parameter by characterizing propagation delay from a pre-synaptic unit to a post-synaptic unit. In some implementations, greater efficacy may correspond to a shorter latency.
  • the efficacy may comprise probability parameter by characterizing propagation probability from pre-synaptic unit to a post-synaptic unit; and/or a parameter characterizing an impact of a pre- synaptic spike on the state of the post-synaptic unit.
  • Individual neurons of the network 500 may be characterized by a neuron state.
  • the neuron state may, for example, comprise a membrane voltage of the neuron, conductance of the membrane, and/or other parameters.
  • the learning process of the network 500 may be characterized by one or more learning parameters, which may comprise input connection efficacy, output connection efficacy, training input connection efficacy, response generating (firing) threshold, resting potential of the neuron, and/or other parameters.
  • some learning parameters may comprise probabilities of signal transmission between the units (e.g., neurons) of the network 500.
  • the training input 540 is differentiated from sensory inputs (e.g., inputs 508) as follows.
  • input data e.g., spike events
  • changes in the neuron state e.g., increase neuron membrane potential and/or other parameters
  • Changes in the neuron state may cause the neuron to generate a response (e.g., output a spike).
  • the training input 540 (also "teaching data") causes (i) changes in the neuron dynamic model (e.g., modification of parameters a,b,c,d of Izhikevich neuron model, described for example in commonly owned and co-pending U.S. Patent Application Serial No.
  • the teaching data may trigger neuron output in order to facilitate learning.
  • the teaching data may be communicated to other components of the control system.
  • data 508 arriving to neurons of the network may cause changes in the neuron state (e.g., increase neuron membrane potential and/or other parameters). Changes in the neuron state may cause the neuron to generate a response (e.g., output a spike).
  • the training input 540 is absent; the input data 508 is required for the neuron to generate output.
  • one of the outputs may be configured to actuate the first CDOF of the robotic arm 100 (e.g., joint 102); another output (e.g., generated by neuron 542) may be configured to actuate the second CDOF of the robotic arm 100 (e.g., the joint 106).
  • FIG. 4 illustrates a multilayer neuron network having three layers of neurons
  • Complex systems may require more neuron layers whereas simpler systems may utilize fewer layers.
  • implementation may be driven by other cost benefit analysis. For example, power consumption, system complexity, number of inputs, number of outputs, the presence (or lack of) existing technologies, etc. may affect the multilayer neuron network implementation.
  • FIG. 5 depicts an exemplary neuron network 550 for controlling multiple degrees of freedom (e.g., the robotic ann apparatus 100 of FIG. 1 ), according to one or more implementations is presented.
  • the network 550 of FIG. 5 may comprise two layers of neurons.
  • the first layer also referred to as the input layer
  • the second layer also referred to as the output layer
  • the input layer neurons receive sensory input 558 and communicate their output to the output layer neurons (572, 574) via one or more connections (e.g., 562, 564, 566 in FIG. 5).
  • the network 550 of FIG. 5 may be referred to as the single-layer network comprising one learning layer of connections (e.g., 562, 566 characterized by efficacies e.g., 578, 568).
  • the first neuron layer (e.g., 552, 554, 556) may be referred to as non-adaptive feature extraction layer configured to respond to occurrence of one or more features/objects (e.g., edges, shapes, color, and or other) in the input 558.
  • the second layer neurons (572, 574) generate control output 576, 570 based on one or more inputs received from the first neuron layer (e.g., 562, 564, 566) to a respective actuator (e.g., the joints 102, 106 in FIG. 1).
  • network connectivity may be e.g., one- to-one, one-to-all, all-to-one, some to some, and/or other methods.
  • the network 500 and/or 550 of FIGS. 4-5 may be operable in accordance with a supervised learning process configured based on teaching signal 540, 560, respectively.
  • the network 500, 550 may be configured to optimize performance (e.g., performance of the robotic apparatus 100 of FIG. 1 ) by minimizing the average value of a performance function e.g., as described in detail in commonly owned and co-pending U.S. Patent Application Serial No. 13/487,499, entitled "STOCHASTIC APPARATUS AND METHODS FOR IMPLEMENTING GENERALIZED LEARNING RULES", incorporated herein by reference in its entirety.
  • supervised learning methodologies may be used for training artificial neural networks, including but not limited to, an error back propagation, described in, e.g. U.S. Patent Application Serial No. 13/465,903 entitled "APPARATUS AND METHODS FOR BACKWARD PROPAGATION OF ERRORS IN A SPIKING NEURON NETWORK ** , filed October 15, 2013, incorporated supra, naive and semi -naive Bayes classifier, described in, e.g. U.S. Patent Application Serial No.
  • the synaptic plastic rules may comprise one or more spike-timing dependent plasticity rules, such as rules comprising feedback described in commonly owned and co-pending U.S. Patent Application Serial No. 13/465,903 entitled “SENSORY INPUT PROCESSING APPARATUS ⁇ A SPIKING NEURAL NETWORK *” , filed May 7, 2012; rules configured to modify of feed forward plasticity due to activity of neighboring neurons, described in co-owned U.S. Patent Application Serial No. 13/488,106, entitled “SPIKING NEURON NETWORK APPARATUS AND METHODS", filed June 4, 2012; conditional plasticity rules described in U.S. Patent Application Serial No.
  • neuron operation may be configured based on one or more inhibitory connections providing input configured to delay and/or depress response generation by the neuron, as described in commonly owned and co-pending U.S. Patent Application Serial No. 13/660,923, entitled “ADAPTIVE PLASTICITY APPARATUS AND METHODS FOR SPIKING NEURON NETWORK", filed October 25, 2012, the foregoing being incorporated herein by reference in its entirety.
  • Connection efficacy updated may be effectuated using a variety of applicable methodologies such as, for example, event-based updates described in detail in commonly owned and copending U.S. Patent Application Nos. 13/239,255 filed September 21 , 201 1 , entitled "APPARATUS AND METHODS FOR SYNAPTIC UPDATE IN A PULSE-CODED NETWORK'-, 13/588.774.
  • a neuron process may comprise one or more learning rules configured to adjust neuron state and/or generate neuron output in accordance with neuron inputs.
  • the one or more learning rules may comprise state dependent learning rules described, for example, in commonly owned and co-pending U.S. Patent Application Serial No. 13/560,902, entitled “APPARATUS AND METHODS FOR STATE-DEPENDENT LEARNING IN SPIKING NEURON NETWORKS", filed July 27, 2012 and/or U.S. Patent Application Serial No. 13/722,769 filed December 20, 2012, and entitled “APPARATUS AND METHODS FOR STATE-DEPENDENT LEARNING IN SPIKING NEURON NETWORKS", each of the foregoing being incorporated herein by reference in its entirety.
  • the single-layer network 550 of FIG. 5 may be embodied in an adaptive controller configured to operate a robotic platform characterized by multiple degrees of freedom (e.g., the robotic arm 100 of FIG. 1 with two CDOF).
  • the network 550 outputs 570, 576 of FIG. 5 may, be configured to operate the joints 102, 106, respectively, of the robotic arm in FIG 1.
  • the network 550 may trained to operate a first subset of the robot's available CDOF (e.g., the joint 102 in FIG 1 ).
  • Efficacy of the connections communicating signals from the first layer of the network 550 (e.g., the neurons 552, 554, 556) to the second layer neurons (e.g., efficacy 568 of the connection 566 communicating data to the neuron 574 in FIG. 5) may be adapted in accordance with a learning method.
  • the network 550 may trained to operate a second subset of the robot ' s available CDOF (e.g., the joint 106 in FIG 1 ). Efficacy of the connections communicating signal from the first layer of the network 550 (e.g., the neurons 552, 554, 556) to the second layer neurons (e.g., efficacy 578 of the connection 562 communicating data to the neuron 572 in FIG. 5) may be adapted in accordance with the learning method.
  • the first layer of the network 550 e.g., the neurons 552, 554, 556
  • the second layer neurons e.g., efficacy 578 of the connection 562 communicating data to the neuron 572 in FIG. 5
  • the two-layer network architecture may enable separate adaptation of efficacy for individual network outputs. That is, efficacy of connections into the neuron 572 (obtained when training the neuron 572 to operate the joint 102) may be left unchanged when training the neuron 574 to operate the joint 106.
  • the multi-layer network 500 of FIG. 4 may be embodied in an adaptive controller configured to operate a robotic platform characterized by multiple degrees of freedom (e.g., the robotic ann 100 of FIG. 1 with two CDOF).
  • the network 500 outputs 546, 547 of FIG. 4 may be configured to operate the joints 102, 106, respectively, of the arm in FIG 1.
  • the network 500 may trained to operate a first subset of the robot's available CDOF (e.g., the joint 102 in FIG 1 ).
  • Efficacy of connections communicating signal from the first layer of the network 500 e.g., the neurons 502, 504, 506 to the second layer neurons (e.g., efficacy 518, 528 of connections 514, 512 communicating data to neurons 526, 522 in FIG. 4) may be adapted in accordance with a learning method.
  • Efficacy of connections communicating signal from the second layer of the network 500 e.g., the neurons 522, 524, 526) to the second layer output neuron (e.g., efficacy 548 of connections 532 communicating data to the neuron 542 in FIG. 4) may be adapted in accordance with the learning method.
  • the network 500 may trained to operate a second subset of the robot's available CDOF (e.g., the joint 106 in FIG 1 ).
  • efficacy of connections communicating signal from the second layer of the network 500 e.g., the neurons 522, 524, 526) to the second layer output neuron (e.g., efficacy 538 of connections 534 communicating data to the neuron 544 in FIG. 4) may be adapted in accordance with the learning method.
  • the efficacy of connections communicating signal from the first layer of the network to the second layer neurons determined during the first plurality of trials may be further adapted or refined during the second plurality of trials in accordance with the learning method., using, e.g., optimization methods based on a cost/reward function.
  • the cost/reward function may be configured the user and/or determined by the adaptive system during the first learning stage.
  • FIGS. 6-8 illustrate methods of training an adaptive apparatus of the disclosure in accordance with one or more implementations.
  • methods 600, 700, 800 may be accomplished with one or more additional operations not described, and/or without one or more of the operations discussed.
  • the order in which the operations of methods 600, 700, 800 are illustrated in FIGS. 6-8 described below is not limiting, the various steps may be performed in other orders, etc.
  • various steps of the methods 600, 700, 800 may be substituted for equivalent or substantially equivalent steps.
  • the methods 600, 700, 800 presented below are illustrative, any and all of the modifications described herein are readily performed by those of ordinary skill in the related arts.
  • methods 600, 700, 800 may be implemented in one or more processing devices (e.g., a digital processor, an analog processor, a digital circuit designed to process information, an analog circuit designed to process information, a state machine, and/or other mechanisms for electronically processing information).
  • the one or more processing devices may include one or more devices executing some or all of the operations of methods 600, 700, 800 in response to instructions stored electronically on an electronic storage medium.
  • the one or more processing devices may include one or more devices configured through hardware, firmware, and/or software to be specifically designed for execution of one or more of the operations of methods 600, 700, 800.
  • Operations of methods 600, 700, 800 may be utilized with a robotic apparatus ⁇ see e.g., the robotic arm 100 of FIG. l and the mobile robotic platform 300 of FIG. 3) using a remote control robotic apparatus (such as is illustrated in FIG. 2).
  • FIG. 6 is a logical flow diagram illustrating a generalized method for operating an adaptive robotic device, in accordance with one or more implementations.
  • a first actuator associated with a first CDOF operation of a robotic device is selected.
  • the CDOF selection may be effectuated by issuing an instruction to the robotic control apparatus (e.g., pressing a button, issuing a voice command, an audible signal (e.g., a click), an initialization after power- on/reset sequence, a pre-defined programming sequence, etc.).
  • the CDOF selection may be effectuated based on a timer event, and/or training performance reaching a target level, e.g., determined based on ability of the trainer to position of one of the joints within a range from a target position.
  • the first CDOF selection comprises selecting joint 102 of the robotic arm 100.
  • the adaptive controller is trained to actuate movement in the first CDOF of the robot to accomplish a target action.
  • the nature of the task is too complex to be handled with a single CDOF and thus require multiple CDOFs.
  • Operation 604 may comprise training a neuron network (such as e.g., 500, 550 of FIGS. 4-5) in accordance with a supervised learning method.
  • the adaptive controller may comprise one or more predictors, training may be based on a cooperation between the trainer and the controller, e.g., as described in commonly owned and co-pending U.S. Patent Application Serial No. 13/953,595 entitled “APPARATUS AND METHODS FOR TRAINING AND CONTROL OF ROBOTIC DEVICES", filed July 29, 2013, each of the foregoing being incorporated herein by reference in its entirety and/or U.S. Patent Application Serial No.
  • a second actuator associated with a second CDOF operation of the robotic device is selected.
  • the CDOF selection may be effectuated by issuing an instruction to the robotic control apparatus (e.g., pressing the button 210, issuing a voice command, and/or using another communication method).
  • the second CDOF selection may comprise selecting the other joint 106 of the robotic arm.
  • the adaptive controller may be trained to operate the second
  • the operation 608 may comprise training a neuron network (such as e.g., 500, 550 of FIGS. 4-5) in accordance with a supervised learning method.
  • the adaptive controller may be configured to operate the first CDOF of the robot based on outcome of the training during operation 608.
  • the trainer may initially operate the second CDOF of the robot. Training based on cooperation between the trainer and the controller, e.g., as described above with respect to operation 608, may enable knowledge transfer from the trainer to the controller so as to enable the controller to operate the robot using the first and the second CDOFs.
  • the trainer may utilize a remote interface (e.g., the control apparatus 200 of FIG. 2) in order to provide teaching input for the first and the second CDOF training trials.
  • the method 600 may be used with any number of degrees of freedom, additional degrees being iteratively implemented. For example, for a device with six (6) degrees of freedom, training may be performed with six independent iterations, where individual iteration may be configured to train one ( 1 ) degree of freedom. Moreover, more complex controllers may further reduce iterations by training multiple simultaneous degrees of freedom; e.g., three (3) iterations of a controller with two (2) degrees of freedom, two (2) iterations of a controller with three (3) degrees of freedom, etc.
  • the robotic apparatus may support a number of degrees of freedom which is not evenly divisible by the degrees of freedom of the controller.
  • a robotic mechanism that supports five (5) degrees of freedom can be trained in two (2) iterations with a controller that supports three (3) degrees of freedom.
  • FIG. 7 illustrates a method of training an adaptive controller of a robotic apparatus using the reduced degree of freedom methodology described herein, in accordance with one or more implementations.
  • the adaptive controller may comprise a neuron network operable in accordance with a supervised learning process (e.g., the network 500, 550 of FIGS. 4-5, described supra.).
  • a context is determined.
  • the context may be determined based on one or more sensory input and/or feedback that may be provided by the robotic apparatus to the controller.
  • the sensory aspects may include an object being detected in the input, a location of the object, an object characteristic (color/shape), a sequence of movements (e.g., a turn), a characteristic of an environment (e.g., an apparent motion of a wall and/or other surroundings turning and/or approaching) responsive to the movement.
  • the sensory input may be received during one or more training trials of the robotic apparatus,
  • the CDOF of the robotic apparatus is selected for operation.
  • the first and the second CDOF may correspond to operation of the motorized joints 102, 106, respectively, of the manipulator arm 100 in FIG. 1.
  • the method may proceed to operation 706, wherein the neuron network of the adaptive controller may be operated in accordance with the learning process to generate the first CDOF control output based on the context (e.g., learn a behavior associated with the context).
  • the teaching signal for the first CDOF may comprise (i) a signal provided by the user via a remote controller, (ii) a signal provided by the adaptive system for the controlled CDOF, and/or (iii) a weighted combination of the above (e.g., using constant and/or adjustable weights).
  • the method may proceed to operation 710 wherein the neuron network of the adaptive controller is operated in accordance with the learning process configured to generate the second CDOF control output based on the context (e.g., learn a behavior associated with the context).
  • the context e.g., learn a behavior associated with the context.
  • network configuration associated with the learned behavior at operation 704 and/or 710 may be stored.
  • the network configuration may comprise efficacy of one or more connections of the network (e.g., weights) that may have been adapted during training.
  • FIG. 8 illustrates a method of training an adaptive apparatus to control a robot using a reduced degree of freedom methodology, in accordance with one or more implementations.
  • the robot may be characterized by two or more degrees of freedom; the adaptive controller apparatus may be configured to control a selectable subset of the CDOF of the robot during a trial.
  • an actuator associated with a CDOF is selected for training.
  • the CDOF selection may be effectuated by issuing an instruction to the robotic control apparatus (e.g., pressing a button, issuing an audible signal (e.g., a click, and/or a voice command), and/or using another communication method).
  • the CDOF selection may be effectuated based on a timer event, and/or training performance reaching a target level. For example, upon learning to position/move one joint to a target location, the controller may automatically switch to training of another joint.
  • the method proceeds to operation 824, where training input for the first CDOF (CDOF1 ) is provided.
  • the first CDOF training comprises training the joint 106.
  • the training input may include one or more motor commands and/or action indications communicated using the remote control apparatus 200 of FIG. 2.
  • control output may be determined in accordance with the learning process and context.
  • the context may comprise the input into the adaptive controller e.g., as described above with respect to operation 702 of method 700.
  • the control output determined at operation 828 may comprise the first CDOF control instructions 830 and/or the second CDOF control instructions 844.
  • the learning process may be implemented using an iterative approach wherein control of one CDOF may be learned partly before switching to learning another CDOF. Such back and forth switching may be employed until the target performance is attained.
  • control CDOF 1 output 830 may be combined with the first CDOF training input provided at operation 824.
  • the combination of operation 826 may be configured based on a transfer function.
  • the transfer function may comprise addition, union, a logical 'AND ? operation, and/or other operations e.g., as described in commonly owned and co-pending U.S. Patent Application Serial No. 13/842,530 entitled "ADAPTIVE PREDICTOR APPARATUS AND METHODS", filed March 15, 2013, incorporated supra.
  • the first actuator associated with the first CDOF (CDOF1 ) of the robotic device is operated in accordance with the control output determined at operation 826.
  • the actuator for joint 102 is operated based on a combination of the teaching input provided by a trainer and a predicted control signal determined by the adaptive controller during learning and in accordance with the context.
  • the method proceeds to operation 840, where training input for the second CDOF (CDOF2) is provided.
  • CDOF2 training input for the second CDOF
  • the second CDOF training comprises training the joint 102.
  • the training input includes one or more motor commands and/or action indications communicated using the remote control apparatus 200 of FIG. 2.
  • control CDOF 2 output 844 may be combined with the second CDOF training input provided at operation 840.
  • the combination of operation 842 may be configured based on a transfer function.
  • the transfer function may comprise addition, union, a logical 'AND ' operation, and/or other operations e.g., as described in commonly owned and co-pending U.S. Patent Application Serial No. 13/842,530 entitled “ADAPTIVE PREDICTOR APPARATUS AND METHODS", filed March 15, 2013, incorporated supra.
  • the second actuator associated with the second CDOF (CDOF2) of the robotic device is operated in accordance with the control output determined at operation 842.
  • the actuator for joint 106 is operated based on a combination of the teaching input provided by a trainer and a predicted control signal determined by the adaptive controller during learning and in accordance with the context.
  • the CDOF 1 may be operated contemporaneously with the operation of the CDOF 2 based on the output 830 determined during prior training trials.
  • FIGS. 9A-9D Various exemplary computerized apparatus configured to implement learning methodology set forth herein are now described with respect to FIGS. 9A-9D.
  • FIG. 9A A computerized neuromorphic processing system, consistent with one or more implementations, for use with an adaptive robotic controller described, supra, is illustrated in FIG. 9A.
  • the computerized system 1 100 of FIG. 9A may comprise an input device 1 1 10, such as, for example, an image sensor and/or digital image interface.
  • the input interface 1 1 10 may be coupled to the processing block (e.g., a single or multi-processor block) via the input communication interface 1 1 14.
  • the interface 1 1 14 may comprise a wireless interface (cellular wireless, Wi-Fi, Bluetooth, etc.) that enables data transfer to the processor 1 102 from remote I/O interface 1 100, e.g.
  • One such implementation may comprise a central processing apparatus coupled to one or more remote camera devices providing sensory input to the pre-processing block.
  • the system 1 100 further may comprise a random access memory (RAM) 1 108, configured to store neuronal states and connection parameters and to facilitate synaptic updates.
  • RAM random access memory
  • synaptic updates may be performed according to the description provided in, for example, in commonly owned and co-pending U.S. Patent Application No. 13/239,255 filed September 21 , 201 1 , entitled "APPARATUS AND METHODS FOR SYNAPTIC UPDATE IN A PULSE-CODED NETWORK", incorporated by reference, supra.
  • the memory 1 108 may be coupled to the processor 1 102 via a direct connection 1 1 16 (e.g., memory bus).
  • the memory 1 108 may also be coupled to the processor 1 102 via a high-speed processor bus 1 1 12.
  • the system 3 100 may comprise a nonvolatile storage device 1 106.
  • the nonvolatile storage device 1 106 may comprise, inter alia, computer readable instructions configured to implement various aspects of spiking neuronal network operation. Examples of various aspects of spiking neuronal network operation may include one or more of sensory input encoding, connection plasticity, operation model of neurons, learning rule evaluation, other operations, and/or other aspects.
  • the nonvolatile storage 1 106 may be used to store state information of the neurons and connections for later use and loading previously stored network configuration.
  • the nonvolatile storage 1 106 may be used to store state information of the neurons and connections when, for example, saving and/or loading network state snapshot, implementing context switching, saving current network configuration, and/or performing other operations.
  • the current network configuration may include one or more of connection weights, update rules, neuronal states, learning rules, and/or other parameters.
  • the computerized apparatus 1 100 may be coupled to one or more of an external processing device, a storage device, an input device, and/or other devices via an I/O interface 1 120.
  • the I/O interface 1 120 may include one or more of a computer I/O bus (PCI-E), wired (e.g., Ethernet) or wireless (e.g., Wi-Fi) network connection, and/or other I/O interfaces.
  • PCI-E computer I/O bus
  • wired e.g., Ethernet
  • wireless e.g., Wi-Fi
  • the input/output (I/O) interface may comprise a speech input (e.g., a microphone) and a speech recognition module configured to receive and recognize user commands.
  • a speech input e.g., a microphone
  • a speech recognition module configured to receive and recognize user commands.
  • processing devices may be used with computerized system 1 100, including but not limited to, a single core/multicore CPU, DSP, FPGA, GPU, ASIC, combinations thereof, and/or other processing entities (e.g., computing clusters and/or cloud computing services).
  • processing entities e.g., computing clusters and/or cloud computing services.
  • user input/output interfaces may be similarly applicable to implementations of the disclosure including, for example, an LCD/LED monitor, touch-screen input and display device, speech input device, stylus, light pen, trackball, and/or other devices.
  • the neuromorphic processing system 1 130 of FIG. 9B may comprise a plurality of processing blocks (micro-blocks) 1 140.
  • Individual micro cores may comprise a computing logic core 1 132 and a memory block 1 134.
  • the logic core 1 132 may be configured to implement various aspects of neuronal node operation, such as the node model, and synaptic update rules and/or other tasks relevant to network operation.
  • the memory block may be configured to store, inter alia, neuronal state variables and connection parameters (e.g., weights, delays, I/O mapping) of connections 1 138.
  • the micro-blocks 1 140 may be interconnected with one another using connections 1 138 and routers 1 136.
  • connections 1 138 and routers 1 136 may be interconnected with connections 1 138 and routers 1 136.
  • connection layout in FIG. 9B is exemplary, and many other connection implementations (e.g., one to all, all to all, and/or other maps) are compatible with the disclosure.
  • the neuromorphic apparatus 1 130 may be configured to receive input (e.g., visual input) via the interface 1 142. In one or more implementations, applicable for example to interfacing with computerized spiking retina, or image array, the apparatus 1 130 may provide feedback information via the interface 1 142 to facilitate encoding of the input signal.
  • input e.g., visual input
  • the apparatus 1 130 may provide feedback information via the interface 1 142 to facilitate encoding of the input signal.
  • the neuromorphic apparatus 1 130 may be configured to provide output via the interface 1 144. Examples of such output may include one or more of an indication of recognized object or a feature, a motor command (e.g., to zoom/pan the image array), and/or other outputs.
  • the apparatus 1 130 may interface to external fast response memory (e.g., RAM) via high bandwidth memory interface 1 148, thereby enabling storage of intermediate network operational parameters.
  • intermediate network operational parameters may include one or more of spike timing, neuron state, and/or other parameters.
  • the apparatus 1 130 may interface to external memory via lower bandwidth memory interface 1 146 to facilitate one or more of program loading, operational mode changes, retargeting, and/or other operations.
  • Network node and connection information for a current task may be saved for future use and flushed.
  • Previously stored network configuration may be loaded in place of the network node and connection information for the current task, as described for example in commonly owned and co-pending U.S. Patent Application Serial No.
  • External memory may include one or more of a Flash drive, a magnetic drive, and/or other external memory.
  • FIG. 9C illustrates one or more implementations of shared bus neuromorphic computerized system 1 145 comprising micro-blocks 1 140, described with respect to FIG. 9B, supra.
  • the system 1 ] 45 of FIG. 9C may utilize shared bus 1 1 7, 1 149 to interconnect micro- blocks 1 140 with one another.
  • FIG. 9D illustrates one implementation of cell-based neuromorphic computerized system architecture configured to optical flow encoding mechanism in a spiking network is described in detail.
  • the neuromorphic system 1 150 may comprise a hierarchy of processing blocks (cells blocks).
  • the lowest level LI cell 1 152 of the apparatus 1 150 may comprise logic and memory blocks.
  • the lowest level LI cell 1 152 of the apparatus 1 150 may be configured similar to the micro block 1 140 of the apparatus shown in FIG. 9B.
  • a number of cell blocks may be arranged in a cluster and may communicate with one another via local interconnects 1 162, 1 164. Individual clusters may form higher level cell, e.g., cell L2, denoted as 1 154 in FIG. 9d.
  • L2 clusters may communicate with one another via a second level interconnect 1 166 and form a super-cluster L3, denoted as 1 156 in FIG. 9D.
  • the super-clusters 1 154 may communicate via a third level interconnect 3 168 and may form a next level cluster.
  • the hierarchical structure of the apparatus 1 150 comprising four cells-per-level, is merely one exemplary implementation, and other implementations may comprise more or fewer cells per level, and/or fewer or more levels.
  • Different cell levels e.g., LI , L2, L3 of the apparatus 1 150 may be configured to perform functionality various levels of complexity.
  • individual LI cells may process in parallel different portions of the visual input (e.g., encode individual pixel blocks, and/or encode motion signal), with the L2, L3 cells performing progressively higher level functionality (e.g., object detection).
  • Individual ones of L2, L3, cells may perform different aspects of operating a robot with one or more L2/L3 cells processing visual data from a camera, and other L2/L3 cells operating motor control block for implementing lens motion what tracking an object or performing lens stabilization functions.
  • the neuromorphic apparatus 1 150 may receive input (e.g., visual input) via the interface 1 160.
  • input e.g., visual input
  • the apparatus 1 150 may provide feedback information via the interface 1 160 to facilitate encoding of the input signal.
  • the neuromorphic apparatus 1 150 may provide output via the interface 1 170.
  • the output may include one or more of an indication of recognized object or a feature, a motor command, a command to zoom/pan the image array, and/or other outputs.
  • the apparatus 1 150 may perform all of the I/O functionality using single I/O block (not shown).
  • the apparatus 1 150 may interface to external fast response memory (e.g., RAM) via a high bandwidth memory interface (not shown), thereby enabling storage of intermediate network operational parameters (e.g., spike timing, neuron state, and/or other parameters).
  • the apparatus 1 150 may interface to external memory via a lower bandwidth memory interface (not shown) to facilitate program loading, operational mode changes, retargeting, and/or other operations.
  • Network node and connection information for a current task may be saved for future use and flushed.
  • Previously stored network configuration may be loaded in place of the network node and connection information for the current task, as described for example in commonly owned and co-pending U.S. Patent Application Serial No. 13/487,576, entitled ⁇ 'DYNAMICALLY RECONFIGURABLE STOCHASTIC LEARNING APPARATUS AND METHODS", incorporated, supra.
  • one or more portions of the apparatus 1 150 may be configured to operate one or more learning rules, as described for example in commonly owned and co-pending U.S. Patent Application Serial No. 13/487,576 entitled “DYNAMICALLY RECONFIGURABLE STOCHASTIC LEARNING APPARATUS AND METHODS", filed June 4, 2012, incorporated herein by reference in its entirety.
  • one block e.g., the L3 block 1 156
  • another block e.g., the L2 block 1 156) via interval interconnects 1 166, 1 168.
  • One or more of the methodologies comprising partial degree of freedom learning and/or use of reduced CDOF robotic controller described herein may facilitate training and/or operation of robotic devices.
  • a user interface may be configured to operate a subset of robot's CDOF (e.g., one joint of a two joint robotic manipulator arm).
  • the methodologies of the present disclosure may enable a user to train complex robotic devices (e.g., comprising multiple CDOFs) using the reduced CDOF control interface.
  • the user may focus on achieving target performance (e.g., placing the manipulator joint at a target orientation) without being burdened by control of the whole robotic device.
  • operation of the robot by the user may be augmented by the controller output for the already trained CDOF (e.g., the joint 102 in FIG. 1 ), Such cooperation between the controller and the user may enable the latter to focus on training the second CDOF subset without being distracted by the necessity of controlling the first CDOF subset.
  • the methodology described herein may enable use of simpler remote control devices (e.g., single joystick) to train multiple CDOF robots, more complex tasks, and/or more robust learning results (e.g., in a shorter time and/or with a lower error compared to the prior art).
  • the robot training methodologies described herein may reduce cognitive load on a user of a robot, e.g., by allowing the user to focus on a given CDOF and obviating the need to coordinate multiple control signals for multiple CDOF.
  • Dexterity requirements placed on a trainer and/or trainer may be simplified as the user may utilize, e.g., a single to train and/or control a complex (e.g., with multiple CDOF) robotic body.
  • a complex e.g., with multiple CDOF
  • Using the partial degree of freedom (cascade) training methodology of the disclosure may enable use of a simpler (e.g., a single DOF) control interface configured, e.g., to control a single CDOF to control a complex robotic apparatus comprising multiple CDOF.
  • Partial degree of freedom training may enable the trainer to focus on a subset of DOF that may be more difficult to train, compared to other DOF. Such approach may reduce training time for the adaptive control system as addition as additional training time may be dedicated to the difficult to train DOF portion without retraining (and potentially confusing) a better behaving DOF portion.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Robotics (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Manipulator (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

Appareil et procédés destinés à entraîner et à commander, par exemple, des dispositifs robotiques. Selon une première mise en œuvre, un robot peut être entraîné par un utilisateur à l'aide d'un apprentissage supervisé. L'utilisateur peut être incapable de commander tous les degrés de liberté du robot simultanément. L'utilisateur peut faire l'interface avec le robot par l'intermédiaire d'un appareil de commande conçu pour sélectionner et faire fonctionner un sous-ensemble du complément d'actionneurs du robot. Le robot peut comprendre un dispositif de commande adaptatif comprenant un réseau neuronal. Le dispositif de commande adaptatif peut être conçu pour produire des instructions de commande d'actionneur sur la base de l'entrée et de la sortie utilisateur du processus d'apprentissage. L'entraînement du dispositif de commande adaptatif peuvent comprendre l'entraînement d'un ensemble partiel. L'utilisateur peut entraîner le dispositif de commande adaptatif pour faire fonctionner un premier sous-ensemble d'actionneurs. Après l'apprentissage pour faire fonctionner le premier sous-ensemble, le dispositif de commande adaptatif peut être entraîné pour faire fonctionner un autre sous-ensemble de degrés de liberté sur la base d'une entrée utilisateur par l'intermédiaire de l'appareil de commande.
PCT/US2014/063540 2013-11-01 2014-10-31 Appareil et procédés de commande robotique à degré de liberté réduit Ceased WO2015116270A2 (fr)

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
US14/070,269 US9566710B2 (en) 2011-06-02 2013-11-01 Apparatus and methods for operating robotic devices using selective state space training
US14/070,239 2013-11-01
US14/070,114 US9463571B2 (en) 2013-11-01 2013-11-01 Apparatus and methods for online training of robots
US14/070,239 US20150127154A1 (en) 2011-06-02 2013-11-01 Reduced degree of freedom robotic controller apparatus and methods
US201361901235P 2013-11-07 2013-11-07

Publications (2)

Publication Number Publication Date
WO2015116270A2 true WO2015116270A2 (fr) 2015-08-06
WO2015116270A3 WO2015116270A3 (fr) 2015-11-12

Family

ID=53759690

Family Applications (2)

Application Number Title Priority Date Filing Date
PCT/US2014/063540 Ceased WO2015116270A2 (fr) 2013-11-01 2014-10-31 Appareil et procédés de commande robotique à degré de liberté réduit
PCT/US2014/063543 Ceased WO2015116271A2 (fr) 2013-11-01 2014-10-31 Appareil et procédés permettant de faire fonctionner des robots à l'aide d'un apprentissage d'espace d'états sélectif

Family Applications After (1)

Application Number Title Priority Date Filing Date
PCT/US2014/063543 Ceased WO2015116271A2 (fr) 2013-11-01 2014-10-31 Appareil et procédés permettant de faire fonctionner des robots à l'aide d'un apprentissage d'espace d'états sélectif

Country Status (1)

Country Link
WO (2) WO2015116270A2 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11893474B2 (en) 2015-10-23 2024-02-06 Semiconductor Energy Laboratory Co., Ltd. Semiconductor device and electronic device

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9314924B1 (en) * 2013-06-14 2016-04-19 Brain Corporation Predictive robotic controller apparatus and methods
EP3935574A1 (fr) * 2019-03-05 2022-01-12 HRL Laboratories, LLC Composition de réseau module pour un compilateur neuromorphique bayésien

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4617502A (en) * 1983-06-30 1986-10-14 Hitachi, Ltd. Method and apparatus for controlling a robot hand along a predetermined path
JP3269005B2 (ja) * 1997-05-12 2002-03-25 川崎重工業株式会社 ロボット制御装置
US7783383B2 (en) * 2004-12-22 2010-08-24 Intelligent Hospital Systems Ltd. Automated pharmacy admixture system (APAS)
WO2009119379A1 (fr) * 2008-03-28 2009-10-01 株式会社 ダイヘン Système de commande de robot
WO2009145969A2 (fr) * 2008-04-02 2009-12-03 University Of Pittsburgh-Of The Commonwealth System Of Higher Education Commande corticale d'un dispositif prothétique
KR20110016521A (ko) * 2009-08-12 2011-02-18 삼성전자주식회사 인간형 로봇의 전신 동작 제어 장치 및 그 방법
US8260460B2 (en) * 2009-09-22 2012-09-04 GM Global Technology Operations LLC Interactive robot control system and method of use
RO125970B1 (ro) * 2010-01-21 2019-03-29 Institutul De Mecanica Solidelor Al Academiei Române Metodă şi dispozitiv pentru controlul dinamic al unui robot păşitor
US9888973B2 (en) * 2010-03-31 2018-02-13 St. Jude Medical, Atrial Fibrillation Division, Inc. Intuitive user interface control for remote catheter navigation and 3D mapping and visualization systems
US8886359B2 (en) * 2011-05-17 2014-11-11 Fanuc Corporation Robot and spot welding robot with learning control function
US9070083B2 (en) * 2011-12-13 2015-06-30 Iucf-Hyu Industry-University Cooperation Foundation Hanyang University Method for learning task skill and robot using thereof

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11893474B2 (en) 2015-10-23 2024-02-06 Semiconductor Energy Laboratory Co., Ltd. Semiconductor device and electronic device
US12387093B2 (en) 2015-10-23 2025-08-12 Semiconductor Energy Laboratory Co., Ltd. Semiconductor device and electronic device

Also Published As

Publication number Publication date
WO2015116270A3 (fr) 2015-11-12
WO2015116271A2 (fr) 2015-08-06
WO2015116271A3 (fr) 2015-11-12

Similar Documents

Publication Publication Date Title
US11279026B2 (en) Reduced degree of freedom robotic controller apparatus and methods
US9566710B2 (en) Apparatus and methods for operating robotic devices using selective state space training
US11161241B2 (en) Apparatus and methods for online training of robots
US10717191B2 (en) Apparatus and methods for haptic training of robots
US11331800B2 (en) Adaptive predictor apparatus and methods
US11279025B2 (en) Apparatus and methods for operating robotic devices using selective state space training
US9008840B1 (en) Apparatus and methods for reinforcement-guided supervised learning
US9082079B1 (en) Proportional-integral-derivative controller effecting expansion kernels comprising a plurality of spiking neurons associated with a plurality of receptive fields
US9189730B1 (en) Modulated stochasticity spiking neuron network controller apparatus and methods
US9367798B2 (en) Spiking neuron network adaptive control apparatus and methods
US9792546B2 (en) Hierarchical robotic controller apparatus and methods
US9242372B2 (en) Adaptive robotic interface apparatus and methods
US8996177B2 (en) Robotic training apparatus and methods
US9296101B2 (en) Robotic control arbitration apparatus and methods
US20150005937A1 (en) Action selection apparatus and methods
US20220203524A1 (en) Apparatus and methods for operating robotic devices using selective state space training
WO2014201422A2 (fr) Appareil et procédés pour une commande robotique et un entrainement robotique hiérarchiques
WO2015116270A2 (fr) Appareil et procédés de commande robotique à degré de liberté réduit

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14880790

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14880790

Country of ref document: EP

Kind code of ref document: A2