Pilot sites for demonstration, evaluation and assessment activities
Action will assess and verify its results in three pilots, representing: (i) port automation, (ii) smart safety of workers and (iii) cohesive vehicle monitoring and diagnostics. Each pilot will include different scenarios, in which different technological pillars and enablers will be executed and validated, some preliminary KPIs are provided from the perspective of the stakeholders involved. Moreover, experiences from pilots will be used to improve action outcomes (feedback-loop), guaranteeing quality and broad range applicability of results.
Pilot 1: Port automation Pilot
Pilot will be driven by the industrial partner and port terminal operator Terminal Link Group (TL) and will be deployed on its premises – in the Malta Freeport terminal (MFT). TL manages more than 13 container terminals and is related with another 12 terminals (from CMA-CGM Terminals) and CMA-CGM shipping line (3rd largest shipping line in the world, with ~500 vessels). TL terminals, and CMA-CGM are third parties of the TL, with shareholding structure (MFLT is owned by the following holding of solid companies: 50% of Terminal Link and 50% of Yilport Holding Inc where the 50% of TL is split as follows: 51% CMA-CGM and 49% CMPort).
MFT is on the way between Suez and Gibraltar, the perfect spot to transfer cargo from one vessel to another o to transfer cargo from the main liners to the North Africa.Ready for 23000 TEU vessels size is ready to move 3.8M Teus per year with its 20 quayside cranes and 60 RTGs.
MFT is near to reach its maximum capacity in terms of yard occupation leading to a saturation of the terminal, with almost permanent congestion and occasional collapses (collapse is an event during which operation productivity dramatically falls down, and terminal service cannot be served neither executed) with an important impact in the business operations. The complexity of the operations where more than 2,000,000 containers are transshipped each year, from 130 origins to 130 destinations, makes every day a challenge task.
Optimising the RTG operations in the yard is a challenge and the main benefit expected from ASSIST-IoT technology.
Business Scenario P1-1: Tracking assets in terminal yard
Despite the latest digitisation of container terminals, the latest PDS/RFID/OCR/TOS operating systems still provide a general overview of where all the containers and CHEs in the terminal are located. Hence, current job orders assigned to both internal and external drivers only notify the bay number where the assigned container has to be loaded/unloaded, and with the help of which RTG. The lack of additional contextual information leads to lower operational efficiencies. A combination of real-time telemetry from fleet assets, located in the yard, OCR, QR codes, and image-based positioning, will be used to automatically report container’s location to operators in order to enhance the operational efficiency of terminal operators that cost to each container terminal, several million euros per year.
Following the actions of the CHEs when they move the containers, ASSIST-IoT solution will allow to get traceability of all the assets in the terminal (machines and containers). Since the amount of information moved per year to follow any asset is huge, as well as due to very challenging environment, the ASSIST-IoT’s edge-oriented and scalable platform will allow to reuse the gathered information on the edge, so that those actors that require this type of data from specific assets can obtained in a fast and secure manner. Three use cases will be tested under this scenario: (i) asset location management, (ii) CHE location tracking, (iii) Container handling operations reporting.
Business Scenario P1-2: Automated CHE cooperation
The objective pursued by this scenario will be to enhance the operational performance by the automation of complex tasks (e.g. alignment of CHE or optimal twist-locks operation) whose availability will help container terminals to increase the throughput volume and reduce human errors. To achieve this objective, ASSIST-IoT will resolve the M2M communication challenge between two devices (a truck and a crane) even if both do not belong to the same network (owner), allowing authentication, security, low latency and human-to-machine interface. Furthermore, today direct communications between cranes and external trucks is still forbidden due to security reasons. The ASSIST-IoT solution will allow not only internal but also external truck (manual or autonomous) to get data directly from the crane and to do a manual or automatic alignment with the crane. Hence, the scenario is split into two use cases: (i) RTG-Truck identification and authentication, (ii) RTG-Truck alignment
Business Scenario P1-3: RTG remote control with AR support
The operation efficiency of the RTGs is very low due to the nature of the operations, with long idle times and not high flexibility to displace the equipment from one area to another. Nowadays the RTGs just stay idle in one area waiting for the cargo of that area. This causes that the most expensive resources of a container terminal, i.e. crane drivers, keep doing nothing around 50-70% of the time. To enhance the operational efficiency of RTGs, the best possible solution is to enable the remote operation of a pull of RTGs by a pool of drivers, who can virtually jump from one machine to another to execute the operations within few seconds. However, full electrification, fiber optics, and wired network connectivity by busbar currently required for enabling remote operation in container terminals like MFT would cost around 20 M€, limiting its deployment. ASSIST-IoT will make possible to move diesel-driven fleets of RTGs to remote operation via fully wireless control and communication. The main challenge will be to ensure that the WiFi 802.11 or 4G struggling issues for automation and remote operation (especially available throughput, latency and roaming, and uptime) are fulfilled. Moreover, not only wireless remote operations will be supported on the remote operation of RTGs during ASSIST-IoT project, but also new human-to-machine interfaces will be developed and deployed, providing the remote crane drivers with visuals indicating which container is to be handled and where should it be placed afterwards. Therefore, two use cases will be tested in this scenario: (i) Wireless remote RTG operation, (ii) Target visualization during RTG operation.
Pilot 2: Smart Safety of workers Pilot
This pilot will be driven by the industrial partner Mostostal Warszawa S.A. (MOW) and will be deployed on its premises. It will take place at the construction site of the Marshal’s Office in Szczecin, Poland. Construction sites represent a constantly changing work environment: soil and weather conditions; concurrent activities; time pressure and tight schedules; workers-on-foot, personnel and bystanders operating or walking near heavy machines. All those factors make the construction industry around the globe demonstrate an unsatisfactory health and safety track record. The Smart safety of workers with NGIoT solutions pilot will present the benefits of the ASSIST-IoT approach increasing occupational safety and health (OSH) at the complex and unpredictable work environment of the construction site.
Business Scenario P2-1: Occupational safety and health monitoring
ASSIST-IoT will provide a safety net to each individual present at the construction site. The solutions to be developed in the context of this scenario are focused on human-centric safety aspects and involve connected wearables and near real-time monitoring of relevant health and safety information while emphasising personal data protection and user-friendliness. Its aim is to demonstrate the use of smart IoT devices (sensors and actuators) functioning together in a closed feedback loop of an OSH risk management process. The sensory part of the system will provide streams of real-time measurements of key workers’ health parameters (such as heart rate and skin temperature) performed by personal health trackers, and environmental factors (e.g. ambient temperature or UV radiation) measured by weather conditions and air quality monitoring station. Incidents and undesirable behaviour in the construction site will be notified to the OSH manager.
Safety at the construction site will be also increased by additional control of access to restricted zones, ensuring that only workers with relevant permissions, and valid safety trainings have access to dangerous locations. Moreover, workers in the vicinity of operating construction equipment will be better protected thanks to dynamic geofencing in the zone around the construction plant in operation.
Each person will be identifiable to prevent entry of unauthorised persons to critical zones, and to limit the access of suppliers/sellers. The permission to access a specific area of the construction site will be indicated in the BIM system for each construction worker.
Business Scenario P2-2: Fall arrest monitoring
Its aim is to identify any incidents related to the use of a smart fall arrest detector in order to provide help in a case of life-threatening situations. A between the fall arrest equipment and the anchor point. If the worker falls off the platform he is standing on, the fall arrest equipment will prevent him from falling onto the ground. If the worker is able to return to the safety zone of the platform, the incident will be detected and automatically reported, along with the location and the identity of the worker for further investigation. If the worker remains suspended on the fall arrest equipment, an alert will be raised and help will be sent to the location of the incident immediately.
Business Scenario P2-3: Safe navigation
The aim of this scenario is to provide the worker with navigation instructions along predefined routes ensuring safe evacuation in the case of an emergency. The walking paths to be followed by workers during the evacuation will lead through areas that the worker is authorised to access. The emergency routes will be updated by the OSH manager according to the evolving situation based on the routes followed by safely evacuated workers. All paths and routes will be indicated on BIM.
Business Scenario P2-4: Health and safety inspection support
The scenario includes providing an AR-based support to the OSH manager in conducting an inspection to verify whether or not health and safety regulations are followed at all times at the construction site. The inspector will be able to confirm that the necessary procedures are followed; to this end, they will be provided with information about the nature of the activity and the safety plan that needs to be followed. Any information will be securely logged and the site’s incident will be updated so that further action can be taken to mitigate potential hazards.
Pilot 3: Cohesive vehicle monitoring and diagnostics Pilot
This pilot is driven by the industrial partner FORD, that is a leader Original Equipment Manufacturers (OEMs) of passenger cars, with over a 5 million vehicles produced per year. FORD will provide a state-of the-art hybrid electrical vehicle with an open access Electronic Control Unit (ECU) which will be integrated into ASSIST-IoT reference architecture, thus allowing remote access to powertrain parameters and over-the-air update of diagnostics firmware. The Cohesive vehicle monitoring and diagnostics pilot will demonstrate the benefits of the ASSIST-IoT approach for the case of vehicle fleet diagnostics, where inputs coming from different sources are combined for providing an incremental and cohesive evaluation of the vehicle condition.
Business Scenario P3-1:Advanced powertrain monitoring and diagnostics
The scenario includes vehicle monitoring by the OEM for emission in-service conformity verification (ISC) and the over-the-air deployment of enhanced intelligent diagnostic functions. Expected stricter requirements in the context of Post-EU6 emission regulations are a major challenge for OEMs.
On this scenario, ASSIST-IoT will provide the ability to monitor emission on vehicle fleet level, opening the door for a wholesome, accurate, cost-sensitive and transparent statistical emission evaluation. ASSIST-IoT additionally will allow selective monitoring of powertrain parts and vehicle units to replace series recalls by partial recalls or remote ECU recalibration.
Also, with the help of ASSIST-IoT, enhanced AI/ML diagnostic software functions can be distributed to specific vehicles, in order to identify previously unknown defects and allow customer friendly vehicle repairs.
Business Scenario P3-2:Vehicle condition monitoring
This scenario deals with vehicle exterior inspection using TwoTronic’s vehicle scanning solution (indoor version), where daily multiple vehicles (i.e., 40-250 vehicles) visit a repair shop for typical maintenance services.
Aim of the scenario is the timely identification of potential vehicle surface damages and automated inspection of vehicle’s exterior conditions, to determine actual vehicle status at the takeover moments for car garages and car rent service organizations, facilitate driver/insurance liability assignment, and schedule maintenance and repair interventions in the case of fleet management.
ASSIST-IoT will support both ergonomic visualization of surface damages for the end-users as well as an AI-based, automated surface inspection via adaptive edge and cloud distribution of computing power and communication bandwidth, novel AI-techniques for machine learning and a multi-user protective usage of multiple system resources according to person data protection rules and business considerations.
Pilots Challenges and Technical Expectations
Pilot 1 – Port challenges
Several container terminals, like Malta Freeport Terminal, have almost reached their maximum capacity, with almost permanent congestion in the terminal area and occasional services collapse with an important impact on the business operations. Congestion or a collapse of terminals result in four major negative effects: (i) longer vessel stay times; (ii) increase in the vessels berthing-wait-time; (iii) vessels being diverted to other terminals; and (iv) increase of wait and turn-around times of landside trucks all causing increased environmental load, inefficiency of the logistics chain and cost of productive movements. Even though digital information is used in diverse stages of terminal and shipping lines processes, data exploitation potential is still far from optimal. The main hurdles in the data exploitation are caused mainly by the lack of computational resources, high latency of existing networks, throughput bottlenecks in the intermediate layers of current IoT architectures, and lack of trust/confidence among stakeholders of pertinent management systems. While data acquisition and availability is mostly solved, for current container terminals, there is lack of improvements in the following: (i) industry ready, reliable and cost-effective solutions for distributed data analysis and processing, (ii) decentralized combinations of multi-source heterogeneous data, (iii) low-latency actuation loops (especially for data processing from different endpoints, or synchronized cooperation of equipment), or (iv) trusted data sharing for supply chain inter-business integration. Solutions like the one provided in ASSIST-IoT will address the aforementioned industrial challenges, leading to a paradigm shift in the sector and an important evolution compared to the current state of the art technologies. In particular, ASSIST-IoT will help operators by means of smart devices to interact and to make better decisions in real time, by improving availability of information, and the way operators interact with it. These aspects, within the port environment, where several stakeholders coexist, have to interoperate, and share data in a secure and trustable fashion.
Pilot 2- Smart Safety of Workers challenges
Construction site represent a constantly changing multi-risk working environment as it includes work on a height, under time pressure, with a significant physical load, in various ambient conditions including exposure to high ambient temperature and UV radiation etc. Moreover, each construction site is unique and characterized by a dynamic nature due to progressing in time construction works. Construction sites also often occupy a large area what with involvement of subcontractors causes that supervision of work safety is challenging. At the same time, European statistics on accident at work indicate that even one fifth of all fatal accident at work in the EU-27 take place within the construction sector. These statistics confirm a need for undertaking new measures that will contribute to increasing safety in this sector. One of the areas for potential intervention is prevention from serious health issues resulting from e.g. physically demanding work in a hot microclimate. Such incidents are particularly dangerous while working on a height when a risk of falling exists. Moreover, in the case when a fall from the height is restricted and the worker is suspended in his equipment, the time to provide help is crucial due to the pressure of the harness on the femoral arteries. Therefore, detection of such incident, as well as precise location of the worker can save their life. Co-existence of construction workers with construction plants in close proximity, as well as entrance to either construction site or dangerous zone without authorisation are also causes of a potential threat to human life. Providing technical solutions aimed at increasing operator’s awareness about workers in the vicinity will enhance their protection. Information support about OSH-related irregularities for inspector in such a complex environment can also contribute to enhanced supervision and better risk mitigation in the future. Finally, dynamic nature of the construction site makes that safe evacuation of a huge group of workers in the emergency case is challenging taking into account e.g. dangerous zones (e.g. excavations), location of source of hazard (e.g. fire). Therefore, the ASSIST-IoT pursuits aimed at solving above mentioned challenges within the Smart Safety of Workers pilot are expected to significantly contribute to increasing safety at the construction site. However, in order to achieve this high-level goal, there are several technical challenges to be overcome including such areas as development and integration of smart devices with edge computing capabilities, providing AI-based algorithms for prediction and prevention of risky situations, as well as providing near real-time streams of data for decision making.
Pilot 3 – Cohesive vehicle monitoring and diagnostics
Pilot 3a Advanced powertrain monitoring and diagnostics Challenges
By the beginning of the XXI century, all major automotive markets require some sort of On-Board Diagnostics (OBD). OBD is based on a series of embedded software routines, which verify sensor signals and engine behaviour under certain known excitation. OBD procedures for propulsion systems are run in the Engine Control Unit (ECU) of the vehicle, which means that they must meet strict computation power and memory constraints. Thus, there is not possibility to analyse historic records of the considered vehicle and, obviously, getting access of the use data from other units in the fleet. While in its conception OBD is designed for detecting and identifying error sources, this identification is usually deficient, which forces time-consuming and expensive repair operations.
A second aspect which is addressed in Pilot 3a is the mechanism of emission In-Service Conformity (ISC), where a sample of vehicles is tested after several years of operation and may force a complete recall of the produced vehicles if the emission levels are significantly increased when compared with the certified emissions. From a technical standpoint this is not ideal and a close to real-time approach, monitoring and potentially also updating the fleet in total using OTA technologies, could provide benefits for all involved parties, namely the vehicle manufacturer, the vehicle owner and any trusted third party with well-founded interest.
This is where ASSIST-IoT has major role. ASSIST-IoT will allow monitoring of the fleet emissions levels as a whole: the system is expected to provide metrics of fulfillment of the certification levels and could even – as a technical vision – provide the tools to update vehicle calibrations in order to optimize the fleet emissions profile if necessary, decreasing the need for recalls and costly repair actions. Also, ASSIST-IoT can help identifying any given vehicle as an outlier of the fleet emission distribution with the help of advanced logging and diagnostics function, as the capability of deploying diagnostic routines would not be hindered by limited memory or computing power of current embedded diagnostic systems. It will also enable OEM engineers to deploy new diagnostic methods, based on real-life data provided by the system, including predictive maintenance and repair thanks to connected prognostics and AI/ML. Finally, all of the aforementioned innovative technologies will be safe, secure and transparent by ASSIST-IoT design.
Pilot 3b Vehicle condition monitoring
During the last years a fierce need for digitalisation has also faced the automotive industries not only on the production sector but also in the general market of vehicle usage. The need for process optimization based on big data has been also dramatically increased. Several procedures performed so far manually are not any more cost-effective as well as fast enough for the needs of the modern process chains. The exterior conditions of a vehicle represent a major asset to the vehicle ownership. In several business cases it determines an important asset value for the vehicle owners. Also, the timely changes of these conditions via damages and the reasons for those changes play an important role for several commercial processes. An example hereby is the evaluation of the value of a leased car at the end of his leasing contract, or the fleet value as an asset for a fleet management and services company, determining a lot of operational actions (technical and business) during the life cycle of the fleet vehicles. Further examples are insurance cases for everyday life and vehicle handovers from their owners to auto house workshops and repair garages for service and repair purposes.
Comprehensive machine-vision based vehicle scanning as well as AI-based automated inspections support the new needs of several business areas of the general vehicle markets. Novel ASSIST-IoT technology components shall support the creation, distribution, and usage in real time in a business-adapted, safe, and dedicated manner of useful information based on vast amount of primary data. Depending on the business application there are different damage categories with respect to needed sizes and automatic recognition performances, various acquisition and interaction times between users and machines during the business procedures. The balanced determination of the proper number of images per scanned vehicle and its associated data volume, needed edge computing power, necessary communication bandwidth, the interaction time of images reviewing and displaying on the edge and the coordination of AI-associated training methodologies to improve the necessary performance (recognition rate and correlated precision) impose a challenging design space with yet unknown results. Furthermore, having allowed also for outdoor-installed digital scanners the acquisition conditions in varying day time and seasons with different weather situations increase the challenges for a reasonably good AI-performance and digitalisation procedure. Intelligent buffering, pre-fetching, and compressing techniques are needed to reduce the required communication bandwidth. Additional measures must be taken to support the necessary General Personal Data Protection laws, with highest levels of face and license plate anonymization. Last but not least dynamic authorization of software modules with specific, added-value functionalities within a generic ecosystem of technology suppliers can be build-in with the intended DLT-technologies of the project.