Part 9 - TwinCAT utilities library (RingBuffer, Profiler and NT_StartProcess)

Video

Table of Contents:

A) Objectives

In the last part of this tutorial we were looking at some basic functions and function blocks that you will with 100% certainty get use of. In this part we will continue on the same track and look at more stuff that I'm 100% certain that you will use.

We're going to look at the TwinCAT utilities library.

The utilities library contains a number of useful functions and function blocks which with TwinCAT or the operating system can be called. It's one of these libraries that have functionality for everything between heaven and earth. Let's get started!

B) TwinCAT libraries

Just as the various functions and function blocks that we went through in the last part resides in the Tc2_Standard library, the utilities that we will talk about reside in the Tc2_Utilities library

B.1) Tc2_Utilities library

There are various types of functions in the Tc2_Utilities library. We have for example:

operating system functions,
PLC functions,
STRING functions,
system functions,
scope functions
converting functions,
FLOAT functions,
licensing functions
and a lot more...

With the OS functions we can for example do shutdowns and reboots, get the local system time and start processes in the OS.

With the string functions we can convert all characters to upper or lower case, find a substring in a string and much more.

Using the system functionality we can do things like stopping or restarting TwinCAT, setting it to config mode or checking the CPU usage of a TwinCAT system.

With the conversion functions we can for instance convert between different data types

The PLC functions for example provides functionality to write persistent variables (which we haven't covered yet), and doing measurements of execution

In the category other there are some goodies like a basic PID regulator, a memory ring buffer and a real-time clock

and this list just continues on and on....

As you can see the utilities library is just a big box of tools that you can use in your daily workflow of developing PLC software.

In this part of the tutorial we will look a little bit closer on three function blocks.

The first one is FB_MemRingBuffer, which allows data records of varying lengths to be written into a [[ring buffer]].

The second one is the function block PROFILER which can be used to measure the execution time of PLC code.

The last one is the function block NT_StartProcess which can be used to start a windows application from the PLC.

Note: We have actually already looked at a function from the utilities library when we looked at time conversion functions in [[Part 5 - Structures & functions (IEC 61131-3)]].

|220

Back in this part of the tutorial we used a function called FILETIME_TO_DT, where we converted a file time format defined as a struct into a date and time format.

The Tc2_Utilities is one of the libraries that gets changed quite often. Maybe not every time there is a new release of TwinCAT3, but still often enough that you notice it. There is just more and more stuff added to this library.

I'll now demonstrate some of the capabilities of this library by showing some useful function blocks.

C) TUTORAL 18 - How to create a Ring Buffer //FILE -->MemRingBuffer ✍️

C.1) < Concept > - Other: MemRingBuffer FB explained

We will start with the function block FB_MemRingBuffer which acts as a ring buffer for whatever data we want to store.

It follows the principle of [[FIFO]], which stands for first in first out. It means that the data records are read from it in the same sequence in which they were first written into the ring buffer.

So the oldest entries are the first ones that are being read.

If we fill the buffer with one entry past the capacity of the buffer, the oldest entry gets deleted.

I will now demonstrate the concept of this function block.

Assume we have an instance of the FB_MemRingBuffer function block and in this example we will just store integers in it.

When you instantiate a [[ring buffer]] you do so with a certain size. Let's assume that we have storage for five data entries.

When we instantiate the function block it will have a read, and a write pointer both pointing to the first location in the buffer.

The write pointer points to where we will write data if we add an entry, whilst the read pointer points to the location of where we will get the data next time we do a read.

Now if we add a value, in this case the value 1, the value will be written into the first position of the buffer. And the write pointer will move to the next position, this means that the next value we insert will be stored in the second position.

If we add another value, the write pointer will move once again to the next position.

Now if we do a read, what happens?

The oldest value in the buffer (in this case the value 1 that was in the first position) will be removed and the read pointer will point to the second oldest value in the buffer.

Adding a new value moves the right pointer forward.

And again. Reading a value removes the oldest value from the buffer and moves the read pointer forward

Let's read again,

Now let's fill the buffer, in other words let's write values into the buffer until there is no space left in our buffer.

Let's add a '5',

then add a '6',

then add a '7',

and finally let's add an '8',

Now the buffer is full !! (and the write pointer points to the oldest entry in the buffer, the number '4')

We know it's the oldest entry as the read pointer is pointing to it.
The read pointer will always point to the oldest entry in the buffer.

Now let's see what happens if we add an additional entry into the buffer (even though it's full). 👀

What happens is that not only will the write pointer move to the next location, but the read pointer will too. The reason for this is that the ring buffer always has to point to the oldest entry for reading to fulfill the first in first out requirement.

This means that if we now again add one additional value to the buffer then the write and read pointer will move simultaneously again.

Adding another value, still same behavior.

It's not until we do a read that only the read pointer will move to the next location and the write pointer stays.

What happened here is that we read the oldest value, while the write pointer now points to the location where the oldest value was before.

Reading one additional value empties the newly read value and moves the read pointer one additional step forward.

Writing one value now fills the first empty slot with the value (and moves the pointer forward).

We will now look at how we can use this function block.

C.2) < Concept > - MemRingBuffer ACTIONS

The plan is now to make a memory ring buffer for our previous event handling project that we finished at [[Part 7 - Instructions in ST (IEC 61131-3)]] so we'll basically just create a function block, where we can store our events.

So the events are in other words these events, that are declared with the type ST_Event.

Previously we just had an array of these events, where we stored everything. Now we're going to use a memory ring buffer instead.

For this I put up the documentation for the memory buffer here.

The way you use this one is that you instantiate it.

Then to add information or get information from it you use something called ACTIONS

An action call is basically taking a bunch of code and giving it an [[alias]].

About actions...

Using actions is something I wouldn't recommend you to use if you develop new code.

The reason Beckhoff has selected to use actions in this case is because this is probably something very old that's been taken already from TwinCAT2, and where they probably used actions a lot and then that just followed along.

But there's actually no good reason whatsoever to use actions, just use methods instead (when you can).

But here unfortunately we have actions so we have to use them because they're part of this particular function block.

So we have different actions depending on what we want to do.

For example:

if we want to write a new data set we do it by calling the action A_AddTail.

But....

Before we work with the FB, I thought we'd simply wrap our whole memory buffer to store events into one function block. So we have a function block that takes care of storing these events in this memory buffer.

C.3) Create an Event Buffer FB

If we want a function block that takes care of storing the ST_Events in the memory buffer. First we need to create an name this FB.

I thought we could create a FB called EventBuffer for example,

|340

This guy will need to have an instance of the memory ring buffer. So we instantiate it.

Remember: this is a function block that resides in the Tc2_Utilities library, so we need to have a reference to that in our project.

The next thing that this memory buffer requires is an actual buffer where to store it, so it needs a location to store this. The buffer can be an arbitrary size, I mean the bigger the buffer the more data you can store in the buffer. But it still just needs a location, an array of bytes.

It's just a chunk of memory that we need to allocate where we can store these events.

C.3.1) Step 1 - Use Example: Memory ring FiFo (creating the buffer) 🖥️

Beckhoff already provides an example here

They create a MAX_BUFFER_SIZE, and use that.

And what they've done here is that they create an array buffer with the max size, so we can copy the code and give it a better name,

So this is what we have in our project,

To be honest, I don't like the solution so much. We are just creating a buffer size with 1000 bytes but that actually doesn't say us so much about how many events we can store.

All we know is that we can store 1001 bytes, but it doesn't say how many events we can store.

Every event takes up a certain amount of size,

So what I would do instead, is to simply declare a constant that we can call SIZE_OF_ST_EVENT,
and we can simply do use the operator SIZEOF() which will return the number of bytes of anything.

So we use SIZEOF(ST_Event) and then we can call another variable where we simply declare how many events we want to be able to store. We can call this MAX_NUMBER_OF_ST_EVENTS

And for now let's just store 5 events in the buffer,

Now we can create another constant that's called MAX_NUMBER_OF_BYTES_FOR_ALL_ST_EVENT, which is basically just a multiplication,

Then we can replace what we copied from the example.

Copy PLC Code:

FUNCTION BLOCK FB_EventBuffer
VAR
	MemRingBuffer : FB_MemRingBuffer; (* From Tc2_Utilities Library *)
	BufferArray : ARRAY [0..NUMBER_OF_BYTES_FOR_ALL_ST_EVENT] OF BYTE; (* Used by FB_MemRingBuffer*)
END_VAR

VAR CONSTANT
	MAX_NUMBER_OF_ST_EVENTS : UDINT := 5;
	
	SIZE_OF_ST_EVENT : UDINT := SIZEOF(ST_Event);
	
	NUMBER_OF_BYTES_FOR_ALL_ST_EVENT : UDINT := SIZE_OF_ST_EVENT * MAX_NUMBER_OF_ST_EVENTS; //Note: we are still missing padding bytes
	 
END_VAR
---------------------------------------------------------------------

Note - Overhead and Padding

It should be noted however that this doesn't actually guarantee that we will be able to store five events because every time we store something in this buffer there will probably be some overhead for padding.

I am assuming that the FB_MemRingBuffer will add some overhead on top of the actual event, so we might get five but i'm guessing we're probably gonna get less than five. But we'll see.

If the event data doesn't naturally align to these boundaries, the ring buffer might automatically add padding to ensure it does. This padding ensures efficient memory access but doesn't hold meaningful data.

#LearnMore about [[Overhead]] and [[Padding]]

Overhead: Overhead in computing generally refers to extra processing or memory that is used to manage data structures and operations, but isn't part of the actual data you're interested in. It's like the extra space or resources needed to keep things organized and running smoothly.
Padding: Padding, in the context of memory and data storage, means adding extra bytes to align data in memory. Computers often read and write data most efficiently at certain alignments (for example, multiples of 4 or 8 bytes).

Important Note

Notice how the BufferArray is now storing Bytes instead of ST_Events. This means that our Events will have to be converted into a sequence of Bytes. So we know that internally, the FB_MemRingBuffer will deal with the [[serialization]] of the Data Type we want to store.

#LearnMore about [[Serialization]]

Serialization is the process of converting a data structure (like stEvent) into a linear sequence of bytes. This is necessary when you want to store structured data in a byte array (BufferArray in our case), send it over a network, or save it to a file.
In our scenario, serialization would involve converting the contents of stEvent (which may include text, dates, etc.) into a continuous sequence of bytes, which then can be stored in BufferArray.

C.3.2) Step 2 - Making a Method to Write into the Buffer (AddEvent) 🖥️

Okay, the next thing I want to do is to add wrapper methods to wrap these actions.

We'll make a basic one, so we just make one where we can add data.

So we create a method called AddEvent. We don't have to return anything right now. We will actually return whether the event addition was successful or not. But we can add that in a separate output variable. And Make it PUBLIC.

And here we need for as input we need to have the actual event, so we just call it ST_Event (we can use REFERENCE TO as well).

Now we need to use this action of the memory ring buffer.

So we have the memory ring buffer function block and we need to call this action that's shown up here.

Understanding the VAR_INPUT:

What this guy has as inputs and outputs are these variables.

So it's pWrite which is the address of the PLCs variable of a buffer variable, so this is the data that we want to write so in our case it's the structure, the event structure.
We have the cbWrite variable that says the number of bytes that we want to write so that's actually the size of our structure.
Then we have pRead, this is something that we need to use when we read the buffer so when we're going to read a structure so this is where we should store the data when we have read a value from the buffer. So a pointer to a location that is a structure of the type ST_Event in our case.
Then we have cbRead which is the same as cbWrite but for reading, so the number of bytes that we should read so again the size of the event.
And then the pBuffer, that is an address to our buffer. So again the address to our buffer that's this buffer array.

And cbBufferis the size of the buffer (we need to provide that)

Implementing the Action inside the Method

Okay so for AddTail we'll simply need to provide the pWrite and cbWrite,

pWrite is the address of ST_Event and cbWrite is the SIZEOF the structure

And then we don't need to provide pRead and cbRead because this is when we actually get data.

But we need to provide the pBuffer, so where we're going to store the data. And cbBuffer that's the size of this buffer array.

We would also want to know whether this addition was successful or not so we should add an output variable called bError,

And we can use the output bOk to find any errors,

And this is how it looks,

That's AddEvent!

Copy PLC Code:

METHOD PUBLIC AddEvent
VAR_INPUT
	stEvent : REFERENCE TO ST_Event; //stEventToAdd, we have written the event here
END_VAR
VAR_OUTPUT
	bError : BOOL;
END_VAR
---------------------------------------------------------------------
MemRingBuffer.A_AddTail(pWrite := ADR(stEvent),
						cbWWrite := SIZEOF(stEvent),
						pBuffer := ADR(BufferArray),
						cbBuffer := SIZEOF(BufferArray));
bError := NOT MemRingBuffer.bOk;

C.3.3) Step 3 - Reading events from the buffer (GetAndRemoveEvent) 🖥️

Then we need to do the other thing that is to be able to get an event and for this we can use either one of these two:

A_GetHead reads the oldest data set in the ring buffer but does not remove it. Okay so we only read the data but we don't remove it. The
A_RemoveHead reads the data but also remove the oldest data set. (That's just as I've shown in my presentation before).

Okay so what we need to do is to add another method and let's now call it GetAndRemoveEvent because that's what we're doing,

We don't have any inputs but we have two outputs we're gonna provide. We're gonna provide latest event that we're reading (stEvent) but we also need to provide whether we got an error or not.

Now we're gonna do exactly the same thing but instead of writing we're gonna do reading,

Now finally what we want to do is check if there are errors,

That's GetAndRemoveEvent!

Copy PLC Code:

METHOD PUBLIC GetAndRemoveEvent
VAR_OUTPUT
	stEvent : ST_Event; //stLastReadEvent, here we visualize the event read from the buffer
	bError : BOOL;
END_VAR
---------------------------------------------------------------------
MemRingBuffer.A_RemoveHead(pRead := ADR(stEvent),
						   cbRead := SIZEOF(stEvent),
						   pBuffer := ADR(BufferArray),
						   cbBuffer := SIZEOF(BufferArray));
bError := NOT MemRingBuffer.bOk;

Can we improve our code?

I mean we could, and should probably also make a method that just returns the number of bytes available still in the buffer or number of events still available because maybe we want to provide it in an HMI or we want to do it for for error handling. I mean there's possibilities to improve on this I don't want to put too much time and effort on this right now.

Diagram:

C.4) Implementing our Ring Buffer 🖥️

We are now going to use our Buffer.

What I'm going to do, to do that, is I'm going to go to my main program here that's empty. I'm going to do an instance of our function block so now we have this function block that wraps all the handling of this memory ring buffer so it's specific for our events. So we have effectively created a memory ring buffer for events.

Let's instantiate this guy. And now we need some help variables because what we want to do now is to test this event buffer so we want to test by adding data into it and removing data from it or reading and removing data from it.

Now we're simply gonna do like this

Then we need to do exactly the same for ReadEvent so we need to do,

I think this is everything we need to run this code now, so we can test it, so we can just add some events and read them, so let's compile. Log in and run.

Copy PLC Code:

PROGRAM MAIN
VAR
	EventBuffer : FB_EventBuffer;
	
	bAddEvent : BOOL;
	stEventToAdd : ST_Event; // Here we write the event we want to save
	bEventToAddError : BOOL; 
	
	bReadAndRemoveEvent : BOOL; 
	stLastReadEvent : ST_EVENT; // Here we visualize the event read from the buffer
	bReadAndRemoveEventError: BOOL;	
END_VAR
---------------------------------------------------------------------
IF bAddEvent THEN
	EventBuffer.AddEvent(stEvent := stEventToAdd, bError => bEventAddError);
	bAddEvent := FALSE;
END_IF

IF bReadAndRemoveEvent THEN
	EventBuffer.GetAndRemoveEvent(stEvent => stLastReadEvent, bError => bReadAndRemoveEventError);
	bReadAndRemoveEvent := FALSE;
END_IF

Diagram:

C.5) Testing our Ring Buffer

Test1 - Adding Events to the Ring Buffer

First we need to have stEventToAdd, so we write some values to use,

Now we're gonna add the event so if you change the boolean to true then the event will be added,

And this we can easily check by simply reading from the buffer,

First we see these values in the buffer,

so let's set ReadAndRemoveEvent to TRUE and...

look, we get the data back!

Okay so it's working!

Test2 - Reading from an empty Ring Buffer

Let's try to read again, then it should not work because we don't have anything else. Now the buffer is empty and we get an error.

Test3 - Getting the Ring Buffer full

One final thing I want to do here is of course to add more events than we have in the buffer, we'll just add the same message over and over again.

So if we do that we will see that we can only store 4 events because we also require memory for the overhead. Once the buffer is full we get an error,

But that pretty much sums up how to use this memory ring buffer.

Note: Estimating the Size (including Padding and Alignment)

Below is a table that outlines the hypothetical size calculation of each member within the stEvent structure, including the assumed padding for alignment to 4-byte boundaries.

Component	Size (Bytes)	Padding (Bytes)	Total Size (Bytes)
`eEventType`	2	2	4
`eEventSeverity`	2	2	4
`nEventIdentity`	4	0	4
`sEventText`	256	0	256
`dtTimestamp`	8	0	8
Total			276

Padding bytes are added to align the data structure on memory boundaries, which is common in PLCs for faster access. For example, if each element is aligned to a 4-byte boundary, we will have to add those bytes to the total size.

Keep in mind that the actual size could be different based on the PLC's memory management. To get the exact size, you would consult the documentation or use the SIZEOF() function in TwinCAT 3, which will give you the size in bytes of the structure as it is laid out in memory by the system.

D) TUTORAL 19 - How to use the Profiler //FILE -->Profiler_Test ✍️

D.1) < Concept > - PLC: Profiler FB explained

The function block Profiler is used to measure the execution time of PLC code. This is quite convenient for example when you want to understand what parts of your code takes the most time to execute, so that you can optimize your software.

The function block takes the last 10 measurements and calculates the mean value and provides that into the output.

The function block is fairly easy in its construction.

INPUTS:

There is one input for starting the measurement of the execution time.
And one input to reset the function block, so that it's ready for a new measurement

OUTPUTS:

The output BUSY is true until the measurement is completed
and once the function block is complete with the calculation, the output PROFILERSTRUCT provides all the measurement data

To use the function block simply create an instance of it and optionally a variable to store the result

Then to execute it, start it with a rising edge of the START boolean and finish the measurement with a falling edge of the START boolean.

Put the actual code that you want to do the time measurement of between these two lines of code.

Let's play around a little bit with this function block now!

D.1) Step 1 - Writing a sample function to test (calculating the factorial of a number) 🖥️

We're going to test the function block "Profiler", to measure execution.

For this I prepared a tiny function that calculates the factorial of a number.

Recap: What is a Factorial?

D.1) Step 2 - Instantiating the Profiler

Now we're simply gonna call this factorial with the number and see the execution.

What we need to do is to first of course need to make sure we have a reference to Tc2_Utilities.

Next we need to instantiate the profiler, so we can just call it Profiler and instantiate the function block PROFILER.

Then we need a number to call with the factorial and I'll just call it nFactorial,

Note: We can play around with this and set a different number just to see how the execution time will be affected when we change this number.

The way we do the measurement is simply calling Profiler with the start flag set to true, reset to false. Everything between the lines of code will be measured.

D.3) Step 3 - Implementing the Profiler to measure our sample function 🖥️

Then we want to call our function Factorial so we simply call the Factorial with the number that we want to calculate, in this case nFactorial, that we will change online during execution.

It's that simple!

And then the output of the Profiler has a structure and we can also store this in our own variables, so let's do that. Let's just call this ProfilerResult and that's an instance of PROFILERSTRUCT.

This struct has some measurements like minimum execution time, maximum and average which is what we're interested in.

So, after we finished with the measurement, we simply store in ProfilerResult the output DATA.

Because the DATA is of type PROFILERSTRUCT.

so we can look at ProfilerResult while we're executing to see how long the execution time is and that's basically all there's to it.

Copy PLC Code:

PROGRAM MAIN
VAR
	Profiler : PROFILER; 
	ProfilerResult : PROFILERSTRUCT;
	nFactorial : ULINT;
END_VAR
---------------------------------------------------------------------
Profiler(START := TRUE, RESET := FALSE); // Start measuring the execution time

// Here we place the code to be measured
Factorial(n := nFactorial);

Profiler(START := FALSE); // Stop measuring the execution time

ProfilerResult := Profiler.DATA;

D.4) Step 4 - Testing the Profiler "online"

So we compile and run the code....

Note: This function block is extremely useful because sometimes you just want to look what parts of the code takes the longest time. There's actually more modern tools I'm not going to go through that in this particular tutorial but uh this is a good starting point at least for making the measurements.

Now we're executing we can see that basically have zeros here because the measurements are in microseconds.

The reason we get zeros is because simply this execution is very fast when we just use a factorial of zero. So it's faster than zero microseconds and again I'm running this off a virtual machine now on a quite fast CPU so this is not a big surprise and so we can start to do is to simply increase the factorial from 0 to something bigger, so for example 10,000?

Yeah, now we kind of can see that the last execution time is jumping around. Now we're actually starting to get into a number where we can actually measure using this function block.

What's interesting here is that now we can actually look at the PLC task and execution time.

Here we see that we have around 30 microseconds execution.

Let's increase this to 100,000 instead. Now we see we have...

Yeah it takes quite some time, now actually it takes around 150-200 microseconds to calculate the factorial with the number of one hundred thousand.

If you look at the CPU we can actually see there was a tiny jump there. Still far far far away from 10 milliseconds which is what we're running this PLC cycle on.

I mean that kind of says a little bit of how powerful a PLC is using PC based control

One thing to notice is that the results from big factorials don't probably fit in an unsigned long integer. We are taking about very big numbers, but at least it worked to show how with just a few lines of code we can do measurements very easy.

E) TUTORAL 20 - How to use NT_StartProcess //FILE -->Profiler_Test ✍️

E.1) < Concept > - OS: NT_StartProcess FB explained

The last function block in the utilities library that we will go through today is the NT_StartProcess function block. This can be used to start a windows application from the PLC which can be quite useful in many scenarios.

At a first glance this function block can look a little scary with all the inputs and outputs. But looks can be deceiving as this function block is actually very simple to use. Let's go through all the inputs and outputs.

INPUTS:

First we can supply the NETID of the PLC that should execute the windows program. This means default this will point to the local PLC of where TwinCAT 3 is running, but it can point to a NETID of another PLC. This means that we can execute a windows application on another TwinCAT 3 PLC
The next variable is PATHSTR, which is the complete path to the application that we want to execute.
Next we have the variable DIRNAME, which is the working directory of the application that is being executed. This means that we can execute the application from another directory than where the application resides
Next we have COMNDLINE, which gives us the possibility to add additional command line parameters to the application that we want to execute if it supports any
The parameter START is used to execute the function block and start the process of starting the windows application. The function block is executed with a rising edge of this parameter
The last parameter is the maximum TMOUT for the function block to execute its command.

OUTPUTS:

On the output side we first have the BUSY flag, which indicates whether TwinCAT is busy trying to launch the application.
The second and third output parameters give information of whether the execution was successful, and if not, an ERRID which can be used to do fault isolation.

To use the function block simply declare it and provide the various input parameters.

In this example we have provided the NETID and TMOUT already in the declaration. It should be noted that the values supplied in this example are exactly the same as the default values which means that they can be omitted as well.

To start the windows application from your PLC code, simply call it with a rising edge of the START input.

In this example we are starting the application Notepad, with the working directory of Temp, on the C drive and with the command line parameter of TextFile.txt. This would result in our PLC opening the file Textfile.txt in the folder Temp if it existed using Notepad. If a file didn't exist, it would instead create a new file with the name Textfile.txt in the Temp folder using Notepad.

Now it's time for some fun. Let's do a little bit of programming

E.2) Step 1 -

Okay, so we're going to demonstrate the NT_StartProcess function block and to do this I'm going to have a slightly different approach. What I'm gonna do is to use an open source project that I found that's done by a very nice chap called Ben He has some interesting open source project on his GitHub, I highly recommend you to look into it, that's called "tc3-ps-ping". What this does is to execute a PowerShell script

to call ping. So ping is a way to send a data packet to another host and see if you get a reply from that host. So you can ping Google or whatever site or whatever host and see whether you get a response. The problem with NT_StartProcess is that you can't get any response back from a process.

So you can call a process with parameters but there's no way to actually get a response from it. There's no support in the function block to do that But what Ben has done very smartly is that he has created a PowerShell script so you have NT_StartProcess calling the PowerShell script, and then the PowerShell script through ADS (which we'll talk about in the future), writes into a variable on our PLC, called back whether the ping was successful or not. So it's a two-stage rocket. You call the PowerShell script with NT_StartProcess