Show Casing Autonomous blank feeding of packaging machines TIMAIRIS Univ. de Aveiro/IMA S.p.A. UAVR/IMA

Transcription

1 Show Casing Autonomous blank feeding of packaging machines TIMAIRIS Univ. de Aveiro/IMA S.p.A. UAVR/IMA

2 TIMAIRIS Use Case Task: Autonomous Blank feeding of packaging machines Check that the blank is the right one for the format being produced Take a pile of blanks from the pallet Orient the pile versus the loading position Feed the pile into the magazine Human, machine and robot interaction has to take place in conditions of safety (From EuRoC Playbook, 2014/12/08) 2

3 TIMAIRIS Use Case Industrial state of the art Blank feeding is performed manually by a human operator little value added; errors in feeding; strenuous task; high cost Depalletization in safety cage very structured environment; complex gripper; high dependency on blank type Human blank feeding Depalletization in a safety cage 3

4 TIMAIRIS Showcase Target: Simplified version of complete blank feeding task Main simplifications: Single packaging machine Prototype of blank magazine No blank identification Blanks always the same shape but various prints No prediction of the need for blank feeding (multi feed optimization) Feeding starts when stack is at predefined level Navigation environment The environment is shared with single human user Simpler communication with user Initial version of multimodal interface (from TIMAIRIS DoW, 2015/12/06) 4

5 Showcase Setup Open area of ~20m 2 (4m x 5m) Prototype of blank magazine 2 Pallets with real blanks KUKA platform with blank gripper and 2 laser range finders Showcase simulation environment used during development Showcase realistic environment 5

6 Showcase Plan Round 1 Single pallet with blank piles; no humans within the cell. identify pallet and blank magazine; recognize need for blanks; pick a pile of blanks; go to the blank magazine and place pile in the feeding mechanism. Round 2 (in addition to Round 1) Two pallets with blank piles; still no humans in the cell. select pallet with the smaller number of piles; recognize empty pallet; switch to the other one adapting navigation routes appropriately. Round 3 (in addition to Round 2) Two pallets plus humans within the cell. perform safe and compliant navigation and manipulation; perform simple multimodal interaction with humans using gestures. 6

7 Showcase main Challenges Manipulation of non-rigid stacked objects Single arm manipulation Smooth placing of blank pile Shared workspace with humans Robust detection of blanks Handling global localization errors Fluent human-robot interaction 7

8 Showcase Metrics Organized to ensure a comprehensive and robust evaluation Objective 1 focussed on Perception M1 - position of piles, pallets, magazine; M2 - need for blanks; M3 -pallets state; M4 - safety situations; Objective 2 focussed on Manipulation and Navigation M1 - pick-up piles; M2 - transport pile; M3 - place pile into magazine; M4- safe manipulation; Objective 3 focussed on Planning M1 - plan order for correctly picking piles from pallet; M2 - adapt navigation paths; Objective 4 focussed on Interaction M1 - recognizing gestures; M2 - track humans for interaction; 8

9 Perception Results Use RGB and depth information for perception ARToolkit marker to localize blank magazine Robust blank pile detection. Handles different stamped patterns Pallet and blank piles detection Blank outline and picking point detection 9

10 Manipulation/Navigation Results New gripper proved to be very robust for picking, transporting and placing blank piles Improved motion planning provided optimized paths gripper using compliance motion planning smooth placing 10

11 Planning Results Plan is derived from blank shape and existing piles on pallets Picking order ensures no collisions between piles Pallet with fewer piles is emptied first No collision with any obstacle Derived picking plan taking into consideration pallet with fewer piles 11

12 Interaction/Safety Results Deep Learning and Heuristics features are used for gesture recognition Safety based on tracking humans near robot with LIDAR Human tracking for safety and HRI Overview of defined safety regions 12

13 Results Summary TIMAIRIS completed the showcase with every objective and metric being accomplished Considering extra demo on Interaction for O4M1 Robustness was very high but not perfect on interaction and safety Perception Manip./Navig. Planning Interaction M1 M2 M3 M4 M1 M2 M3 M4 M1 M2 M1 M2 Target Achieved % 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 13

14 Innovations Skill-based anytime agent architecture Separate task dependent representations and generic agent algorithm New blank pile gripper Single arm manipulation for depalletization, transport and blank feeding Detection of blank pile position Detection is robust to different stamped patterns 14

15 Innovations Improving motion planner / motion planner benchmarking Results from automatic motion planning are filtered Tested several planners; use motion planner most adapted for each skill Multimodal interaction Using gestures commands and GUI Realistic simulator Simulator allows the execution of the complete showcase task Long term goals: Localization and SLAM; effective world representations; black box optimization; multi-robot sensor fusion 15

16 Cooperation Strong cooperation since preparation of stage II and III Proposal Several meetings both in Italy and Portugal IMA and UAVR participated in all showcase code camps Tasks IMA Gripper design and construction (2 versions); Blank magazine design and construction; providing pallets, blanks; use case knowledge Tasks UAVR Perception; Manipulation; Multimodal interface; Safe navigation; Planning Intellectual Property Agreement IMA-UAVR signed Scientific publication with authors from both institutions Master student co-supervised by IMA and UAVR people Approved with summa cum laude (Bologna Univ.) on March

17 Dissemination 10 publications in international conferences - E Pedrosa et al., A skill-based architecture for pick and place manipulation tasks, in Progress in Artificial Intelligence. LNCS 9273, Springer International Publishing, E Pedrosa et al., A Scan Matching Approach to SLAM with a Dynamic Likelihood Field, IEEE Int Conference on Autonomous Robot Systems and Competitions (ICARSC), Bragança, May A Abdolmaleki et al., Contextual policy search for linear and nonlinear generalization of a humanoid walking controller. Journal of Intelligent & Robotic Systems, Vol.83, n.3, , Sep GH Lim, Two-step Learning about Normal and Exceptional Human Behaviors Incorporating Patterns and Knowledge Multisensor Fusion and Integration for Intelligent Systems (MFI), 2016 IEEE International Conference on. IEEE, R Dias et al. "Multi-object tracking with distributed sensing," Multisensor Fusion and Integration for Intelligent Systems (MFI), 2016 IEEE International Conference on. IEEE, A Abdolmaleki et al., Non-parametric contextual stochastic search, IEEE Int. Conference on Intelligent Robots and Systems (IROS), South Korea, p , November R Dias et al., "Real-Time Multi-Object Tracking on Highly Dynamic Environments", in IEEE Int. Conf. on Aut. Robot Systems and Compet. (ICARSC), E Pedrosa et al., "Efficient Localization based on Scan Matching with a Continuous Likelihood Field", in IEEE Int. Conf. on Autonomous Robot Systems and Competitions (ICARSC), F Amaral et al., Skill-based anytime agent architecture for logistics and manipulation tasks EuRoC Challenge 2, Stage II realistic labs: Benchmarking, in IEEE International Conf. on Autonomous Robot Systems and Competitions (ICARSC), GH Lim et al., Rich and robust human-robot interaction on gesture recognition for assembly tasks, in IEEE International Conf. on Autonomous Robot Systems and Competitions (ICARSC), EuRoC related videos at IRIS Lab YouTube page News about TIMAIRIS work in Portuguese press Visits to our lab (schools, national and international industry) EuRoC is one of the projects that is referred 35 visits (last 8 months), more than 1100 visitors Keynote talk at ICAART 2017 Learning Tasks in Robotics: Problems and Solutions referred TIMAIRIS research on Gesture Recognition 17

18 Impact and use of results IMA is actively investing in human aware robotics solutions In terms of machinery, know-how and human resources IMA just acquired a KUKA platform similar to those used in the EuRoC project Showcase workspace mimic conditions found in a real scenario Results obtained in the showcase provide an excellent base for the development of the pilot experiment. Future IMA s products will be designed around the notion that an increasing number of repetitive and low-skilled actions will be carried out by cooperative robotised platforms. 18

19 Conclusion Showcase clearly showed that it is possible to evolve to autonomous blank feeding of packaging machines using robots All quantifiable metrics have been accomplished IMA and UAVR are highly committed to the development of this use case Sustainability and impact of the results is fostered by IMA s leading position in this market Showcase solution would already be able to feed two industrial packaging machines Results obtained from Stages I and II provide an excellent base for the development of the Pilot Experiment 19