Written by Isabel del Castillo (COO)

The autotuning is a visual tool that enables users to optimise the performance of Klepsydra AI for three different criteria: power consumption, throughput, and latency. 

This tool has been available since the end of 2021. It is a new product that can be used freely by Klesydra’s customers. So far, it has been very liked by our clients.

Flexible optimisation

Most AI inference engines that run AI models on the edge are like a one gear car. They run in one manner only. There is no possibility to tune them, they just perform it in the fixed capacity these tools are set to do. Klepsydra AI is more elastic and flexible, it is a tool built to optimise for different situations. With the auto tuning software, it is now possible to do it very easily with a graphical interface.

The innovation of autotuning

The autotuning is the first tool of this kind in the market. It is a quite unique feature as the user can have the flexibility to optimise a model for a specific computer in a very easy manner. Performance can be really squeezed to the maximum.

The original technical innovation in autotuning comes in Klepsydra AI that has a very advanced so-called threading model. This model consists in a multi-dimensional parallelization process that is very elastic and enables multiple types of optimizations. 

Applications

 There are many applications that could benefit from autotuning. Depending on the capability sought, autotuning can be set to enhance one function or another:

• Power consumption. In the case of a space where the computers run with very limited power, as the satellite must perform different tasks and cannot use all its power to feed the computer and simultaneously ensure better service (longer lasting batteries, less outages of power). It is necessary to have a strong control on power consumption. The autotuning will ensure that the power consumption is below the requirements, or close to below the requirements, of the application.
• Throughput. There are other cases for example in the automotive industry or the gaming industry where throughput is extremely important due to the large amount of data to be handled.
  In those cases, Klepsydra AI can be optimised to have the highest possible throughput and eliminate data losses.
• Latency. There are other applications where the real-time aspect is very important. For example, security and aerospace applications where reaction speed is critical.  Klepsydra autotuning will enable the user to optimise for the lowest possible latency. We have recently observed in real tests a reduction of up to 90% in latency. (From 100ms to 10ms).

The environmental benefits of using klepsydra are remarkable both directly and indirectly:

• Directly, by reducing CPU usage and extending the computer lifetime. On edge applications the benefits are twofold. First, battery lifetime can be increased for IoT, satellites, robots, etc. by reducing power consumption in the data processed on-board. And secondly, by extending the life of edge and on-board computers in the same environments.
• Indirectly, as an enabling technology. Klepsydra products have been used for green energy markets (e.g., wind turbine inspections), space debris removal, electric cars, and Earth Observation (fires prevention, drought detection, etc).