Automated Adaptive Packaging in Food Processing An IoT Approach Professor John Gray Professor Zhipeng Wu University of Manchester This project has received funding from the European Union
KBBE.2012.2.3-03: Call: Automation in food packaging systems Food industry is facing: high numbers of food products and varieties and packaging types small batches and small runs need for operational flexibility
KBBE.2012.2.3-03: Automation in food packaging systems Fresh and processed food Convenience and hygiene is important Integration of advanced technologies within robotics In-line quality control Intelligent management, ensuring traceability
1. PickNPack Project: Overall Functionality Variable mould for packaging fabrication. Robotic bin picking and product replacement. Programmable laser sealing and cutting. Automated sealing checking. Multi modal sensor module. Wireless data transfer.
1. PickNPack Project: Overall Functionality Variable mould for packaging fabrication. Robotic bin picking and product replacement. Programmable laser sealing and cutting. Automated sealing checking. Multi modal sensor module. Wireless data transfer.
1. PickNPack Project: Overall Functionality Automated label printing and application with barcode or alternatives. Automated RFID tag application for product identification and retrieval. Overall software architecture for inter-module communication, process control, data storage management and retrieval. Comprehensive user friendly interface.
The Sensor Module New sensor technologies 3D colour camera 3D X-ray Hyperspectral image camera Microwave scanning Automated fusion of sensor data Learning to new quality features and products by training
2. Legal Requirements of Food Product Traceability The EU s General Food Law entered into force in 2002 and makes traceability compulsory for all food and feed businesses. The traceability needs to cover one step backward and one step forward in the food chain, which is also called one-upone-down system.
3. RFID for Traceability Why RFID? Pros and Cons Non line-of-sight Large operating range Read and write capability High data capacity (user memory) Data encryption/automation capability Multiple tag read capability Higher cost Complex in development
4. Traceability System Design for the Packaging Line Products Supplier Super Market Large Container RFID Tag (Type 1) Warehouse Warehouse RFID Type2 Packaging Line Scan&Record Production Line -packaging RFID Tag (Type 1)
Packaging Traceability System Design Summary Product Name Type Arrived Time Temperature Req. Humidity Req. Processed Before Product Name Batch No. RFID ID Processed Time Temperature Req. Humidity Req. Packaging Before Operator Warehouse Name Location Capacity Add: Enter time Leave time Products Large Container Warehouse RFID Tag (Type 1) Supplier Name Location Type Contract Available Period Empty Containers Re-Use Name Location Type Product ID RFID Delivery ID Enter Time Leave Time: T/H Req... Delivery ID Vehicle ID RFID ID2s Operator Departure Time Arrive Time Source Location: Destination: T/H Req... Warehouse Super Market RFID ID2 Container Type Scan & Record RFID Tag (Type 1) Production Line -packaging Content ID/ Container ID/Time Start / Time complete(empty)/contained RFID ID1s/ processed Before Time/Content Type/..
RFID Hardware for Smart Shelf Customised Hardware Design
RF Solutions for Smart Shelf 100% Detection Rate with Product in any Orientation
PnP Traceability System : Operation Functional modules Database Module 1 Database Module 2 Database... Module n LAN RFID Module Antenna User Operation Traceability Application Database Database Operation Tag Info RFID Reader 1 RFID Reader 2 Antenna Antenna Antenna Antenna Antenna Antenna Production Line User Operation Database Operation Handheld Reader Application Tag Info Barcode & QR Code Antenna Camera Antenna
Inter-module Communication Router
Traceability System Integration with the Packaging Line: Functions Required Able to receive/reply message from/to line controller Able to receive request and reply with data Able to request data from other modules Able to broadcast unique IDs and heartbeat to the packaging line
The IoT Architecture Technical challenges in Industrial IoT (IIoT) Heterogeneity of devices and technologies Complexity of systems and tasks Bigger and bigger data Communication and understanding between machines Key issues General system architecture M2M messaging mechanism
Enabling Technologies Data, service, and knowledge layer KDD Domain Knowledge Service Database Semantic Big data M2M Communication Network layer Software & protocol Bluetooth Ad Hoc Mesh Infrastructure-based ZMQ MQTT CoAP Machines Zigbee 6LowPAN 3G 4G LTE WiFi WiMAX Ubuntu TCP UDP HTTP WinCE DDS XMPP AMQP TinyOS LiteOS Android ios Information Message Embedded electronics Sensor Module Controller Robot RFID Reader Barcode Reader MCU DSP FPGA ARM PLC SoC Raw data Physical world Industrial processes Sensors Actuators Physical variables
M2M technologies Existing M2M Messaging Techniques Message Queue Telemetry Transport (MQTT) Constrained Application Protocol (CoAP) Data Distribution Service (DDS) Extensible Messaging and Presence Protocol (XMPP) Advanced Message Queuing Protocol (AMQP) ZeroMQ (ZMQ)Existing M2M Messaging Techniques Since each solution has its strengths and particularly suited fields of applications, the selection of the alternative techniques should be based on the requirements of the practical situations of the industrial applications.
The Scope of this Investigations The Specificities of M2M in Industrial Applications Heterogeneity on hardware and software platforms Frequent notification and peer machine observation Collaborative automation between machines Real-time event handling and data processing Data of various formats and sizes to transmit Requirements Cross-platform interoperability with basic standards Machine discovery and presence capability Flexible data interaction and event notification Reliable, efficient, and fast in speed Failure handling and recovery capability
ZMQ-based M2M Messaging Technology Features It handles I/O asynchronous, in background thread; Components can come and go dynamically and ZMQ will automatically reconnect; It queues messages automatically when needed; It has ways of dealing with over-full queues; Applications talk to each other over arbitrary transports: TCP, multicast, inprocess, inter-process; It lets route messages using a variety of patterns; It lets create proxies to queue, forward, or capture messages with a single call; It handles network errors intelligently, by retrying automatically in case where it makes sense; It does more with less power consumption; Supported more than 40 programming languages and multiple operating systems.
Heartbeating Presence to peer machines in the network Discovery peer machines in the network Notify state change or observe the state changes of peer machines 1 1 4 2 4 2 3 3 1 1 4 2 4 2 3 3
PickNPack QAS Module Module controller (Queue) + Sensors (Worker) + remote controller (client) Notify/observe states with heartbeats Communicate using pre-defined ZMQ messages Microwave sensor Data Raw data Controller- Controller- Microwave Controller- RGB Controller- 3D Hyperspectral Controller- X-Ray Command & Data ZMQ message QAS module controller Command & Data ZMQ message Central controller Worker Queue Client Connect Front: bind(tcp://*5555) Back: bind(tcp://*5556) Connect
Benefits Auto discovery of software modules No networks addresses have to be provided Discovery via symbolic name Line controller checks every module before running Production specific configuration of modules Handled at line-controller Stored in the line-database (Vendor neutral model) Software composition based on ontology Processes, products and machines in a formal way Modules, devices and components Back-tracing to all production steps (Vendor independend)
Future Challenges To extend the concept to a complete supply chain linking primary production to supermarket shelves and point of sale.
Possible Commercial Benefits - Highly responsive supply chain - Comprehensive data acquisition and processing for line efficiency and product quality and security - Optimise throughput, minimise waste and logistics costs - Compatible with emerging trends in additive manufacturing and plug and play scenarios
Environmental / Social Benefits - Minimise production waste and water / energy usage - Increased product security and safety - Minimise factory footprint - Upskill sector workforce with subsequent impact on the economy
1. PickNPack Project: A European Initiative A large scale collaborative project involving 14 international partners and 13million funding.
Recent references 1) Zhaozong Meng, Zhipeng Wu, Cahyo Muvianto and John Gray, A Data-Oriented M2M Messaging Mechanism for Industrial IoT Applications, IEEE Internet of Things Journal, 29 December 2016, 0.1109/JIOT.2016.2646375. (IF=7.596) 2) Zhipeng Wu, Zhaozong Meng and John Gray, IoT-based Techniques for Online M2M-Interactive Itemised Data Registration and Offline Information Traceability in a Digital Manufacturing System, This paper appears in: IEEE Transactions on Industrial Informatics - Print ISSN: 1551-3203, Online ISSN: 1941-0050, Digital Object Identifier: 10.1109/TII.2017.2704613 (IF=6.764)
Thank you