Discussing FreeRTOS architecture for sort of a data logger application
Ask QuestionAsked 3 days agoActive todayViewed 197 times2
I’m fairly new to RTOS architecture and been reading up on articles on FreeRTOS.org, but as it’s commonly agreed, practice clears things up even more.
I have a basic project idea which I’d ideally want to be RTOS-based as I’ll explain below along with some questions:
My understanding is RTOS comes in handy when you’re computing multiple tasks concurrently. Based on that, I came up with the idea which involves a BLE-supported MCU, sensors, BLE-supported app, and some sort of a display component be it LCD, and how it works is a request is sent over BLE from an app, which is processed by the MCU, and does the following concurrently:
- sending the data from the desired sensor back to the app over BLE
- updating the display with the value on an LCD
In an RTOS context, I’d then have:
- TaskA – constantly reads the requests coming over BLE from an app
- Process the request in either TaskA or a different task (?), and obtain the desired data and put it in queueB.
- TaskB – as soon as
queueBis no longer empty, send the data out to the app over BLE
- TaskC – as soon as
queueBis no longer empty, update the display with the data
I think with this approach, there isn’t any shared data that I need to worry about locking mechanisms…unless I’m missing something. Is the use of RTOS justified? If not, what else could be done?
I have a BLE-compatible phone (acting as a central) with an app that’s connected to, say, the BLE of nRF52 (acting as a peripheral). After the connection, the user wants to retrieve some information from
sensorA, so they send out “A”, which is received by the peripheral device, processes it, and responds back to the central with the desired data which was requested, and at the same time, the LCD display is updated with the data as wellembeddedfreertosrtosShareCiteEditFollowFlagedited Mar 29 at 5:19asked Mar 29 at 1:12Jazzy2933 bronze badges
- 3please ask a specific, answerable question … this site is not meant for braistorming – jsotola Mar 29 at 1:52
- 1I’m confused. You have inputs coming from BLE and then you “send it out” over BLE. What’s the point of just sending it right back? Or are these two different BLE source/sinks? It would help a lot if you would be willing to discuss more details about what exactly you are trying to do. And no, an RTOS is not a saws-all nor does what you describe necessarily require anything even similar to FreeRTOS. More details would help a great deal. – jonk Mar 29 at 1:54
- 1So, much as I like discussion and opinion, StackExchange wants a clear question which maps to a clear answer. So this isn’t the place for your question as posed (www.reddit.com/r/ece, however…). If you could rephrase your question into something that could have a definitive answer, that would work. (And: (1) while the app is pretty small for an RTOS, it’s not a bad thing to put under an RTOS; and (2) your TaskB and TaskC may both be reading data from the queue at the same time — so you’d need a mechanism to deal with that). – TimWescott Mar 29 at 2:49
- 1@jonk – sorry for the confusion. I have updated the description. Let me know if it’s still unclear – Jazzy Mar 29 at 2:55
- 1@TimWescott – thanks for the suggestion. My main concern being whether my idea and usage of RTOS in this context make sense and if not, what could be done instead – Jazzy Mar 29 at 2:56
- 1For the most part, what you describe is event driven. Depending on the timing requirements, i see no specific reason for using an RTOS. – Kartman Mar 29 at 2:58
- 1@Kartman any way this could be made so RTOS’ is justified? – Jazzy Mar 29 at 3:07
- 1If you want to get some mileage with FreeRTOS, go for it. In general, though, if everything has roughly the same response time you can just use a superloop. Where you really need an RTOS is if you have very different response times in play — e.g., a motor control loop that needs to respond in 1ms, running on a processor with a human interface that can take 50ms to respond. – TimWescott Mar 29 at 3:11
- 1from my understanding, as long as things are sequential, you’re fine without RTOS but if you want to be doing multiple things around the same time, you may wanna consider RTOS, no? – Jazzy Mar 29 at 3:24
- 1‘Same time’ – concurrency on a single cpu is an illusion- Only one task executes at a given time. Multitasking is achieved by swapping tasks co- operatively or pre-emptively. An RTOS gives you a franework for managing this and methods to schedule and manage inter-process communications. It is a common solution but there is a cost in overhead with memory (data and code) and execution time. For larger projects this is justified. It may be overkill for small projects. – Kartman Mar 29 at 4:13
- 1that’s why I said “around the same time”. You’re right. – Jazzy Mar 29 at 4:20
- 1@Jazzy Kartman is talking to you about the way I would. An O/S on a single cpu core will only execute one thing at a time. Ever. Perhaps the simplest and most important addition an O/S can offer is separate stacks as most languages don’t offer it. This allows two different threads to appear isolated in the sense of call parameters and return values. That’s NOT a win in terms of performance or real-time behavior. It IS a win in terms of avoiding spaghetti-code and improving the maintainability of your application. – jonk Mar 29 at 4:25
- 1@Jazzy The next most important thing an O/S might provide is pre-emption. Separate stacks alone can be handled in a “co-operative” way without pre-emption. Which is a lot safer and easier to implement as you don’t have to worry about static state of library functions, for example. But pre-emption provides the ability to interrupt a thread, suspend it for a moment, and run start a different thread at the point it was last interrupted. Pre-emption can be useful for time-critical threads. But again, the price is all of the static state saving and restoring time. So it’s still not faster. – jonk Mar 29 at 4:27
- 1@Jazzy Pretty much, most of what an O/S buys you is about keeping threads or processes relatively isolated (not co-mingled) and therefore simple to read, to write, and to maintain. Up to a point, anyway. At some point, the O/S itself has such a high learning curve and so many functions you need to call, that your code can become quite difficult, again. Microsoft’s .NET Core 5 would be such an example. Then, it’s about all the software stacks and massive libraries you get that make the trade-off worth the trouble, once again. An O/S is not a panacea. You just need to understand your needs well. – jonk Mar 29 at 4:34
- 1@jonk – yes, I understand that aspect and I never really meant or said that tasks execute at exactly the same time, and rather around the same time. Sorry, I should’ve been more clear but that wasn’t really a concern. – Jazzy Mar 29 at 4:34
- 1@Jazzy I’d recommend learning how to set up separate stacks and perform cooperative switching. This means one data structure and one assembly routine called switch(). It can take you easily less than a day to write (half-hour if you have the compiler manual’s section on mixed code already in front of you.) Perhaps the best book on the topic is this one by Douglas Comer. (DO NOT BUY A NEWER VERSION! ONLY THIS ONE!) – jonk Mar 29 at 4:45
- 1Many people get confused in this regard. That is why we’ve been explicit. In terms of tasks, think how a chworks- he’ll set the water to boil, whilst that is happening he’ll chop the carrots, when that is done, check the water otherwise prepare the peas. Other tasks are performed whilst waiting for something else. Eg a keypress. To read the key press takes microseconds, but the user might press it every 200ms. No use sitting in a loop to wait. – Kartman Mar 29 at 4:49
- @Jazzy, your “question” is interesting but confusing, though I agree with you saying that “practice clears things up”. I am curious which RTOS stuff you have been reading. Sometime ago I read this interesting article of placing FreeRTOS inside Rpi Pico: (1) Rust, RT-Thread OS & FreeRTOS come to the Raspberry Pi Pico By Jonathan Tan – Seeedstudio 2021jan seeedstudio.com/blog/2021/02/05/…, / to continue, … – tlfong01 Mar 29 at 5:14
- @Jazzy, I read the above comments and also found things confusing, because I have forgotten all the OS concepts I learned ages ago. So I googled for a free ebook to refresh my memory. I was happy to have googled one of my old favorite OS books, the “Dinosaur OS Book” . (2) Operating System Concepts, 8th Ed (The Free Electronic PDF “Dinosaur” OS book 980+ pages), Silberchatz 2009 uobabylon.edu.iq/download/M.S%202013-2014/…. / to continue, … – tlfong01 Mar 29 at 5:32
- @Jazzy, so I skimmed the dinosaur book and focused on the following: Ch 3, Process Concept (pipes, interprocess communication), Ch 4, Multithreaded Programming (for multi-core systems), Ch 5, Process Scheduling (virtual machine scheduling and multithreaded, multicore architectures etc), Ch 6, Synchronization (mutual exclusion locks etc). / to continue, … – tlfong01 Mar 29 at 5:50
- @Jazzy, now I have a brainstorming suggestion for the title of your question: “”How can a MCU based FreeRTOS talk to a smart phone through BlueTooth to get sensor data?. Ah, locking down lunch time. See you later. – tlfong01 Mar 29 at 6:07
- @Jazzy, and now I am thinking of which MCU to use. Just now I watch the following YT to get a rough idea of what might be a good choice: Rpi Pico vs ESP32 and STM32 – Andreas Spiess 2021jan31 youtube.com/watch?v=cVHCllbN3bQ, Contents: Intro, What we will compare, Competitors, The Ecosystem, The role of ARM (and RISC-V), Start of comparison, The Cores/PIO/Memory, The Pins/ADC/DMA, USB/MicroPython/Thonny/Debugging, 10:35 Wi-Fi and BLE, Power Consumption/Deep-Sleep/Powering Options, Price, 13:17 My Verdict, Outro. / to continue, … – tlfong01 Mar 29 at 6:27
- 1I think the importance of high speed concurent taks requires a RISC computer not RTOS specifically which is necessary high speed synchronous events events with large dedicated stacks and a wide range of priorities with a very short IRQ response unless the IRQ is synchronous and not asynchronous. – Tony Stewart Sunnyskyguy EE75 Mar 29 at 14:01
- I remember how in 1974 I built 24 automated recording devices for U of M Seismic research (Earth Sciences) with precision recorded Seismic waves 500 km away from the blast, with thumbwheel time and duration with WWVB binary signals with a real-time CMOS digital logic. The 1st was magic and the rest were tragic with race metastable conditions from asynchronous timers due to latency differences between NSC and Fairchild. A learning experience, but worked out in the end. (What no RTOS? just counters & logic and a cassette recorder modified with a DC response using FM. – Tony Stewart Sunnyskyguy EE75 14 hours ago
First off, good choice of using FreeRTOS, I’ve used several RTOS and FreeRTOS is by far the most stable and perhaps the best-written code of any embedded library I’ve seen so far.
Although the downside of an RTOS is that it will take you some processing from the MCU, and the task switch has a computing cost (delay). That is why it is better used on some fairly powerful MCUs.
Don’t imagine that an RTOS will make your MCU faster, especially if you need to handle a graphic library and a display. It allows, however, to write much cleaner and modular code and handle systems of larger complexity.
Don’t forget that despite you can have multiple tasks, only one can execute at a time, when a task switch occurs, the stack memory will be copied and this has a processing cost that is not negligible.
In your situation, you only have 3 things to achieve which are reading the sensor, updating the display, and sending the data over Bluetooth, which is not so many tasks, and is fairly simple, perhaps simpler to achieve without an RTOS. Using an RTOS may be overkill for that application and will introduce delays in your system. I personally wouldn’t use an RTOS for that application.
However, if you wish to use an RTOS, your general vision is fine, the tricky part is about the task priorities and when to switch tasks, this mostly depends on your application.
We can guess that the display update probably can be the lowest priority.
Then it really depends on what the data are used for. If you are doing some sort of monitoring and you don’t want to lose any data, you can set up the sensor read as the higher priority hand have a large queue. Depending on the sensor you use, if you can generate an interrupt from the sensor when you have data (like a Ready pin, or a timer) you can then use an interrupt outside the RTOS to write the data in a buffer or in DMA memory, then use a CRITICAL section in the RTOS to copy the data. You can also use double buffer implementation with a MUTEX and this sort of thing.
On the other hand, if you wish to have low latency to the app and don’t care about sometimes losing data, you can set the BLE task as the highest priority.
Implementation is really dependant of what you want to achieve.
Edit Add: When to use an RTOS: Basically when you have an MCU handling several logic tasks of different nature, and you want non-blocking code.
Before Rtos, this was mostly done by state machine architecture. However, state machine architecture is difficult to maintain, and easily ends up in disaster, like the microchip harmony stack which is all state machine based, it becomes a huge mess as you start to have several libraries. Rtos allows a much cleaner way to handle that kind of needs, and allows to cleanly separate the code and is way more manageable, especially when you have a lot of code.
It is also much simpler to write non blocking code with an rtos than state machine or callbacks as you can just write blocking code and call the task switch function on the blocking parts of the code, which allows to have a readable flow.ShareCiteEditFollowFlagedited Mar 29 at 17:06answered Mar 29 at 5:18Damien5,70711 gold badge99 silver badges2525 bronze badges
- 1thanks a lot for your clear explanation. I understand this application is fairly simple but my main goal is really to get a better grasp of RTOS. If you have any other ideas to justify RTOS, please feel free to share! For sensors, I’d most likely be using interrupts so every time there’s an interrupt, a callback would be invoked which would copy the data into a circular buffer and then passed into an RTOS queue. CRITICAL SECTION because the buffer is also being passed to the RTOS queue and we don’t want to corrupt the data while it’s being used inside, say,
xQueueGenericSend()? – Jazzy Mar 29 at 5:49
- 1FreeRTOS has specific functions to pass data to queues directly from interrupts xQueueSendFromISR freertos.org/a00119.html also in your case you can use queue peek function to avoid semaphores, you just need to check whether the queue has data. – Damien Mar 29 at 7:43
- determining whether the queue has the data could be achieved via
xQueueReceive()right? I could have that in a sensor task that waits on the queue that’s supposed to take in the user request. Can you also confirm whether CRITICAL SECTION is going to be needed to avoid the corruption of the buffer? What would be a potential scenario? – Jazzy Mar 29 at 16:38
- xQueueReceive() will pull data from the queue, peek will tell you if the queue has data, all depending how you write your code, both can be OK. If you use the rtos functions you do not need critical. – Damien Mar 29 at 16:43
- are you implying there’s no need to explicitly use
taskENTER_CRITICALbecause that’s already taken care of by RTOS APIs? – Jazzy Mar 29 at 18:59
- Yes @Jazzy. taskENTER_CRITICAL is kind of the nuclear option to disable task switch and interrupts. If you use the queues and semaphores, you don’t need to use it. You may want to use it if you copy data from non RTOS functions, or have sensitive communication that should not be interrupted. – Damien 2 days ago
How can a MCU based FreeRTOS talk to a smart phone through BlueTooth to get sensor data?
Part 1 – Short Answer
Part 2 – TL;DR Long Answer
Part 1 – Short Answer
1.1 Prerequisite and Recommended Reading
This answer is for newbies who has no previous experience with RTOS (Real Time Operating Systems) and very little knowledge and skills in the following:
(a) “process” concepts, such as multi-tasking/multi-threading/multi-processing,
(b) “processing synchronizing/locking concepts, such as seamaphore, critical section, mutex, coroutines etc.
I would recommend the newbies to read the following *free Operating Systems eBook:
Operating System Concepts, 8th Ed (The Free Electronic PDF “Dinosaur” OS book 980+ pages!), Silberchatz 2009,
paying particular attention on the following chapters:
Ch 3 – Process Concept (pipes, interprocess communication),
Ch 4 – Multi-threaded Programming (for multi-core systems),
Ch 5 – Process Scheduling (virtual machine scheduling and multithreaded, multicore architectures etc),
Ch 6 – Synchronization (mutual exclusion locks etc)
1.2 Practice makes thing clear
Multiprocessing and/or event programming stuff is very hard to develop and debug. For newbies, I strong recommend NOT to dive into any RTOS but be humble to start with writing python multi-processing toy programs. A good getting started is the python multiprocessing library/module, which you can import into the very newbie friendly Thonny Python IDE (more about this later).
1.3 Which MCU to use
For those who are not familiar with modern MCUs, I would recommend to watch the MCU comparison YouTube by the Swiss guy Andreas Spiess:
After watching the Swiss guy’s MCU review, I think it is a good idea for newbies to dip their fee wet into Rpi Pico + Thonny IDE + MicroPython. I would also recommend to use WinPC over Rpi/ESP as the host machine.
My recommendation is based on playing with Pico and MicroPython and learnt from the Rpi Foundation’s newbie tutorial on the following:
(1) Blinking a LED (using interrupt), reading a push button,
(2) Using on board 12 bit ADC and also external sensor DS18B20,
(3) Two two core (PIO) Programmable I/O programming with NeoPixel RGB LED strips.
(4) DC motor drivers
1.4 Incremental Learning Plan
The Swiss guy Spiess mentioned that he spent 4 hour struggling with Pico C++ SDK. The other RUST expert Jonathan Tan mention that he knew how to blink a LED. RUST base FreeRTOS seems not yet very mature in Pico. So I would recommend newbies not to start with Pico C++ or RUSt, but be humble and play with barebone python programs on multi-processing (yes, playing with interrupt to blink a LED is already multiprocessing). So I would recommend to eat the big elephant bite by bite, in three big bites.
(1) Win10 PC Thonny python IDE and Rpi Pico (for newbies)
(2) Rpi4B and Rpi Pico C++ SDK (ninjas only)
(3) Rpi Pcio RUST FreeRTOS (hackers only)
1.5 Learning Rpi Pico MicroPython Multiprocessing Programming
How to use the two Cores of the Pi Pico (for multi-processing)? And how fast are Interrupts? – Andreas Spiess
Another good newbie friendly video to learn Pico based FreeRTOS is also from the Swiss guy Spiess. This video shows how to use MicroPython to write a very simple two core/thread program to do multiprocessing using an interruptable button to blink a LED. In other words, you don’t need FreeRTOS to learn or appreciate the very basic idea of multiprocessing in RTOS, with the limitation of only two threads/processes are allowed in the demo program. Ah yes, 64-bit Rpi4B with 4 cores can do 4 processes in real time, and more than 4 processes “a bit slower” using a process queue/pool.
1.6 General Reading List Before Micro Python Multiprocessing Programming
I skimmed more general RTOS tutorials to make sure I have refresh my memory on all the necessary knowledge and skills required. I found Refs 9 ~ 13 very good in learning the concepts. The Arduino based C++ FreeRTOS is a bit tedious to follow. So I skipped them and start now the real Rpi Pico basic multiprocessing demo programs using Rpi Thonny Python 3.9 Multiprocessing Module (Not the old Multi-threading Module)
Note – I self taught all the basic OS concepts using the Dinosaur OS book, and also another good text book by HM Deitel.
Part 2 – Long TL;DR Answer
(1) Operating System Concepts, 8th Ed (The Free Electronic PDF “Dinosaur” OS book 980+ pages!), Silberchatz 2009
(2) The Python 3.9 Standard Library » Concurrent Execution » multiprocessing — Process-based parallelism
(3) Rpi Pico vs ESP32 and STM32 (15 min YouTube) – Andreas Spiess 2021jan31
(4) Rust, RT-Thread OS & FreeRTOS come to the Raspberry Pi Pico – Jonathan Tan, SeeedStudio 2021Jan
(6) FreeRTOS Real-time operating system for microcontrollers Home page and down load – FreeRTOS
(7) Rpi Pico vs ESP32 and STM32 – Andreas Spiess, 2021jan31, 190,293 views
(8) How to use the two Cores of the Pi Pico (15 minutes YouTube video)? And how fast are Interrupts? – Andreas Spiess
(9) Understanding Real Time Operating System (RTOS) and How to use it for your next Embedded Design – Sunakshi, CircuitDigest 2019aug29
(10) Arduino FreeRTOS Tutorial – How to use FreeRTOS to Multi-tasking in Arduino – SeeedStudio
(11) Arduino FreeRTOS Tutorial 1 – Creating a FreeRTOS task to Blink LED in Arduino Uno – Rishabh Jain, CircuitDigest, 2020mar26
(12) Arduino FreeRTOS Tutorial 2 – Using Queues in Arduino FreeRTOS – Rishabh Jain, CircuitDigest, 2020apr06
(13) Arduino FreeRTOS Tutorial 3 – Using Semaphore and Mutex in FreeRTOS with Arduino – Rishabh Jain, CircuitDigest, 2020apr09
(14) FreeRTOS Kernel Quick Start Guide – FreeRTOS
(15) Complete FreeRTOS Documentation – FreeRTOS
(16) Mastering the FreeRTOS Real Time Kernel – FreeRTOS
(17) The FreeRTOS V10.0.0 Reference Manual – FreeRTOS
(18) FreeRTOS Source Code for Book Examples Download – FreeRTOS
(19) System Analysis, Design, and Development Concepts, Principles, and Practices 1st ed – Charles S. Wasson, Wiley (free eBook) 2006
Appendix A – Wikipedia’s short introduction to FreeRTOS
FreeRTOS is a real-time operating system kernel for embedded devices that has been ported to 35 microcontroller platforms. It is distributed under the MIT License.
FreeRTOS is designed to be small and simple. The kernel itself consists of only three C files. To make the code readable, easy to port, and maintainable, it is written mostly in C, but there are a few assembly functions included where needed (mostly in architecture-specific scheduler routines).
FreeRTOS provides methods for multiple threads or tasks, mutexes, semaphores and software timers. A tick-less mode is provided for low power applications. Thread priorities are supported. FreeRTOS applications can be completely statically allocated. Alternatively RTOS objects can be dynamically allocated with five schemes of memory allocation.
3. Key features
Small memory footprint, low overhead, and fast execution.
Tick-less option for low power applications.
Intended for both hobbyists and professional developers working on commercial products.
Scheduler can be configured for both preemptive or cooperative operation.
Coroutine support (coroutines in FreeRTOS are simple and lightweight tasks with limited use of the call stack)
Trace support through generic trace macros, including task scheduling and kernel calls for semaphore and queue operations.
Appendix B – Getting Started with newbie friendly FreeRTOS tutorials
My learning plan is summarized below.
- Skim Arduino FreeRTOS tutorials to get a rough idea of what sort of multliprocessing stuff they usually recommend, eg. interrupt, semaphore, mutex etc. Then write some newbie demo programs using Rpi4B and Rpi Pico using the same stuff: semaphore, mutex etc
- Following the Swiss guy Andreas Spiess’s YouTube video tutorial, write a 2 core/two process Rpi Thonny Python/Rpi Pico MicroPython multiprocessing demo program, illustrating the use of (a) Interrupt, (b) Semaphore, (c) Process queue/pool to do the following event driven program:
(i) any one of two buttons pressed would cause an interrupt routine to setup a semaphore and blink a LED 3 times,
(ii) while the button service is in progress, no further interrupt from the other button will be serviced.
(iii) when the blinking service is completed, locked semaphore will be released, and all interrupt will be entertained again.
This two processes program using Interrupt and Semaphore can be seen as a very very oversimplified portion of RTOS. So when the real RUST FreeRTOs ofr Rpi Pico arrives, hopefully in a couple of months time, this demo program can be used as a starting started test of the FreeRTOS.
ShareCiteEditDeleteFlagedited 4 mins agoanswered Mar 29 at 7:11tlfong011,67811 gold badge66 silver badges1111 bronze badges
- 1I fail to appreciate those who down-vote considerable efforts without the slightest effort made to add comments or reasons or suggest improvements. Those users are simply lazy and judgemental with a problem on the mental side. – Tony Stewart Sunnyskyguy EE75 Mar 29 at 14:04
- 2@TonyStewartSunnyskyguyEE75 Multiple people have made multiple comments across multiple very similar posts, which aren’t being heeded. Eventually it gets boring. Considerable effort isn’t, in and of itself, worthwile. – awjlogan Mar 29 at 14:50
- 1Yet if one can be negatively judgemental, it doesn’t take much effort to comment – Tony Stewart Sunnyskyguy EE75 Mar 29 at 15:19
- 1@TonyStewartSunnyskyguyEE75 Of course, agreed 🙂 Shouldn’t have to comment for the n-th time either, but such is life! – awjlogan Mar 29 at 16:03
- 2This answer looks like copy paste of different random website and confuses multiprocessing and rtos which are not related, rtos generally does not support multiprocessing and this answer will add confusion to understanding what an rtos actually is. Rtos does not have processes, pipes and so forth… Author clearly does not know what an rtos is. – Damien Mar 29 at 16:46
- 2@TonyStewartSunnyskyguyEE75 You could upvote as [heresy of heresies !!!] I have. To a flagged comment I replied: “I’m aware of opinions on these. This is more pertinent than some. My approach is to let the voting process deal with them. They can be useful to some & can easily enough be ignored.” || I see these posts as being of potentially vast value to some – and as doing little harm overall as they (arguably) create a little noise which can easily be walked-on-by. | Anyone is welcome to open a related metapost. – Russell McMahon♦ 18 hours ago
- 1nice book review – Tony Stewart Sunnyskyguy EE75 14 hours ago
- “To continue” ????? – Mitu Raj 4 hours ago
- Sorry. Continue no more !!!!! – tlfong01 46 secs ago Edit
Leave a Reply