Considerations for Concurrent Processing in IoT Systems

by | June 30, 2019

Many IoT products are what we like to call real-time systems. This is in contrast with “software” system or applications that would run on your PC. These systems can take more time to perform a task in some instances than in others, without adverse overall performance effects, other than an annoyed user, because of a sluggish computer. A real-time system must respond to signals within explicit and bounded time requirements. A real-time system’s response time must be somewhat deterministic so we can know if it will meet the desired requirements. The timing requirements may be specified with a maximum value, a minimum value, or both. Note that real-time does not mean “faster” necessarily.

Most IoT systems are also required to perform more than a single task. This complicates the design because IoT CPU-based systems are sequential in nature. They can only run one task at a time, so the time spent executing one task will affect the timing of the other tasks. This is in contrast to parallel systems using more advanced parallel processors of hardware digital logic, which can perform multiple tasks in parallel. The complexity of a real-time system can range from a simple software delayed sequencer to a complex multitasking system with a full-featured real-time operating kernel.

To design real time projects, it is helpful to view the project as a single-processor-based input/output model. Consider a microcomputer that is running two tasks. These two tasks cannot run in parallel, but they can run concurrently by sharing the CPU. Hence the difference between “parallel” and “concurrently”, which yields the model of multitasked processes. For example you can type in your computer and listen to music in parallel, because you are using two distinct resources that you have available; your hands and your ears. On the other hand, if you are cooking, you cannot prepare two dishes in parallel using your hands, as per our definition. You can, however, prepare them concurrently, by doing a bit for each task, so that both are completed at the same time. In general, in parallel processing, the execution time does not increase, by adding tasks. You simply add resources. In concurrent processing or multitasking, the completion time of all tasks does increase as more tasks are added because the processor is switching between tasks and devoting some time to each task until they are all done. There is also overhead associated with switching tasks, so doing tasks concurrently can take in total more time than if the tasks were done sequentially. However, there are many instances when it is required that tasks be performed concurrently, even if some efficiency is lost. In our cooking example, it would take less total time to prepare a dish after the other, but then the first one would be cold (I know you have a warming plate, but bear with me, and assume you have no way to keep it warm).

Now let’s go back to our discussion of an IoT CPU running two tasks concurrently, and the single-processor-based input/output model. The difference between these two tasks and two normal functions is that their execution is asynchronous to each other. This means that one task does not control the other by calling it directly. Input signals can be sampled by the tasks, which in turn respond by generating output signals. In addition to input signals and output signals, the system may have signals between the tasks. These are called inter-task signals and are realized in software. For example, an alarm variable set by a sensor-scanning task and detected by a siren-output task would be considered an inter-task signal.

The sampling of signals and the generation of output signals are all controlled by the program flow of the CPU. We call this the program time. Events on the signals are asynchronous to the program flow, so they are called real-time events. This means that within the program, we cannot determine where a real-time event will occur so the input signals must be sampled periodically to detect the event. The sampling process of a task can be modeled as a synchronizer that is clocked when the program reads the inputs. In effect this synchronizes the real-time events to program time, so we know when the data are valid within the program flow.

So how do you know if a processor has enough bandwidth (Is it fast enough?) to support a given number of tasks required for a certain IoT system? This is a simple, yet effective, first order procedure to analyze load on a CPU in the context of a real-time system. One that, to be useful, must react within certain amount of time. Not like your web browser, which can load a page in half a second sometimes, and in five others, but it is still useful; you are just bit annoyed. 

By calculating the CPU load, we will be able to determine whether a system is realizable; real-time tasks are endless loops that sample sets of inputs periodically and provide sets of outputs periodically as well. Therefore, many programs we write for IoT systems are periodic. For a system with a single periodic task, the CPU load is 

L = Ttask / Tperiod

where Ttask is the task’s execution time and Tperiod is the tasks execution period. 

Note that Tperiod comes as a natural constraint of the project you are trying to implement, and Ttask is controlled mainly by how fast the CPU is. The faster the CPU, the smaller Ttask will be and the smaller L will be. Conversely, you can reduce the load by increasing Tperiod, which amounts to performing tasks in the system less frequently. 

For example, assume there is a CPU in an IoT system that interrogates a sensor every 30 seconds, and takes 30 mS to get the sensor data and process the data, such that a useful outcome is activated. In this case, Tperiod = 30 S and Ttask = 30 mS = 0.03 S. Therefore, the load L = Ttask / Tperiod = 0.03 / 30 = 0.001. This indicates that the CPU in this project is loaded at a 0.1% level (L x 100%). 

However, most useful projects require more than a single task, so in general the load for n tasks is
L = (Ttask_1 / Tperiod_1) + (Ttask_2 / Tperiod_2) + … + (Ttask_n / Tperiod_n) 

The value of the load parameter L can help determine whether an IoT system is realizable in a given CPU. An IoT system cannot possibly work if the load is greater than one. After all a project with a single task and a load greater than one means that the task takes longer to execute than the time between executions. Obviously, this cannot work. So, a necessary condition for a project to be realizable is L < 1 

A general rule of thumb is to keep the CPU load under 0.7. This allows for overhead for housekeeping, switching between tasks, and some room for future expansion. 

When designing a real-time IoT system, it is of the outmost importance to consider both the output response function and the output response times. Equally important is to understand the CPU load imposed by the system requirements. Then we can know if the system we need to implement is realizable with a given CPU early on during the system definition stage of the development.

Learn more about concurrent processing in IoT systems by contacting us today!