Wasmtime

Wasmtime is a high-performance WebAssembly runtime for applications written in C, C++, Rust, and other system programming languages. It provides a safe and efficient way to execute code in a sandboxed environment, making it ideal for running untrusted code like user-provided or LLM-generated scripts.

In the context of the plugin, Wasmtime is used to run Python scripts. The plugin defines the necessary Wasi configuration to provide a Python environment for the scripts to execute in.

Requirements

The plugin will require the following packages for C#: Microsoft.SemanticKernel and Wasmtime.
To run a Python CLI in WebAssembly, a Wasi implementation is needed. The directory structure could look like the following:

1
2
3
4
5
6
7
8
9
10
11

Project
├───Plugin.cs
├───Project.csproj
└───Resources
    └───Python
        ├───lib
        │   └───python3.13
        │       ├───asyncio
        │       ├───collections
        │       └───...
        └───python3.13.wasm

In this case, the folder Resources is copied to the output directory.

Plugin Implementation

The plugin can be organized in two separate functions: RunScriptAsync and RunCodeAsync. The former takes care of running a single script file in the Python interpreter running on Wasmtime and the latter is a Semantic Kernel function that receives the code from the LLM plan.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

private async Task<string> RunScriptAsync(string dir, string file, string stdout)
{
    using var engine = new Engine();
    using var store = new Store(engine);
    using var linker = new Linker(engine);
    linker.DefineWasi();

    using var module = Module.FromFile(engine, "Resources\\Python\\python3.13.wasm");

    var wasiConfig = new WasiConfiguration()
        .WithArgs("-c", "/scripts/" + file)
        .WithPreopenedDirectory("Resources/Python/lib", "/usr/local/lib")
        .WithPreopenedDirectory(dir, "/scripts")
        .WithInheritedStandardError()
        .WithStandardOutput(stdout);
    store.SetWasiConfiguration(wasiConfig);

    var instance = linker.Instantiate(store, module);
    instance.GetFunction("_start")!.Invoke();
    return stdout;
}

1. RunScriptAsync: This method handles the execution of a Python script using Wasmtime. It takes the script’s directory, file name, and the desired standard output file as input. The method performs the following steps:

   • Initializes a Wasmtime engine, store, and linker.
   • Defines the Wasi environment for the script to run in.
   • Loads the Python WebAssembly module (python3.13.wasm).
   • Configures the Wasi environment with the script’s directory, Python library paths, and standard output.
   • Instantiates the WebAssembly module and invokes the _start function to execute the script.
   • Returns the standard output of the script.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

[KernelFunction("run_code")]
[Description("Runs a script in Python language and returns the standard output")]
[return: Description("The standard output of the script execution")]
public async Task<string> RunCodeAsync(
    [Description("The code in Python language to be executed")]
    string code,
    CancellationToken cancellationToken)
{
    var temp = Path.GetTempFileName();
    var lines = code.Replace("\"", "'").Split("\\n");
    await File.WriteAllLinesAsync(temp, lines, cancellationToken);
    var tempDir = Path.GetDirectoryName(temp) ?? string.Empty;
    var tempFile = Path.GetFileName(temp);
    var outFile = Path.GetTempFileName();
    var fileOut = await RunScript(tempDir, tempFile, outFile);
    return await File.ReadAllTextAsync(fileOut, cancellationToken);
}

2. RunCodeAsync: This method is the entry point for the Semantic Kernel plugin. It takes a string of Python code as input, writes it to a temporary file, and then calls RunScriptAsync to execute the script. The method performs the following steps:

   • Replaces any double quotes in the input Python code with single quotes for safe file writing.
   • Splits the code into lines and writes it to a temporary file.
   • Retrieves the temporary file’s directory and name.
   • Calls RunScriptAsync to execute the script, passing the temporary directory, file name, and another temporary file for standard output.
   • Reads the standard output from the temporary file and returns it.

Result

Example output from a console application:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18

User >
What is the 10th number of the Fibonacci sequence? You can write and execute a Python script for the answer
Microsoft.SemanticKernel.KernelFunction: Information: Function Code-run_code invoking.
info: Microsoft.SemanticKernel.KernelFunction[0]
      Function Code-run_code invoking.
trce: Microsoft.SemanticKernel.KernelFunction[0]
      Function Code-run_code arguments: {"code":"def fib(n):\n    a, b = 0, 1\n    for _ in range(n):\n        a, b = b, a \u002B b\n    return a\n\nprint(fib(10))"}
Microsoft.SemanticKernel.KernelFunction: Trace: Function Code-run_code arguments: {"code":"def fib(n):\n    a, b = 0, 1\n    for _ in range(n):\n        a, b = b, a \u002B b\n    return a\n\nprint(fib(10))"}
Microsoft.SemanticKernel.KernelFunction: Information: Function Code-run_code succeeded.
info: Microsoft.SemanticKernel.KernelFunction[0]
      Function Code-run_code succeeded.
Microsoft.SemanticKernel.KernelFunction: Trace: Function Code-run_code result: 55
trce: Microsoft.SemanticKernel.KernelFunction[0]
      Function Code-run_code result: 55
info: Microsoft.SemanticKernel.KernelFunction[0]
      Function Code-run_code completed. Duration: 6.6005792s
Microsoft.SemanticKernel.KernelFunction: Information: Function Code-run_code completed. Duration: 6.6005792s
Assistant > The 10th number in the Fibonacci sequence is 55.

Conclusion

The plugin demonstrates how to use Wasmtime to execute Python code within a sandboxed environment as a Semantic Kernel plugin in C#. By leveraging Wasmtime’s WebAssembly support and Wasi configuration, the plugin enables secure and efficient execution of user-provided Python scripts, adhering to the principles of sandboxing and code isolation.

Next steps

• Avoid using files to communicate with the Python interpreter
• Have proper socket support in the Python interpreter on WebAssembly
• Create an agent that can use the plugin to iterate on code to fit the user request
• Consider using interpreters other than Python

Overview

The primary objective of this project is to create an efficient, low-cost, and easily manageable solution for sending and receiving emails. The project aims to leverage the capabilities of serverless platforms to bypass the intricate infrastructures traditionally associated with email servers, thus providing benefits to both developers and end-users.

Architecture

The following sections detail the essential components that enable the functionality of this serverless email stack.

Cloudflare Workers

Cloudflare Workers provide a serverless environment capable of running JavaScript code close to the end-user, allowing email-related functions to execute swiftly without the need for traditional server hosting. A major advantage of Cloudflare Workers is their seamless integration with KV Storage, a key-value database well-suited for storing small amounts of metadata related to emails.

• Receiving Emails:
  • Incoming emails are processed through Cloudflare Workers, using Email Routes.
  • Metadata associated with these emails is stored in Cloudflare KV, enabling quick categorization and retrieval.
• Authentication:
  • JWT based autentication with Azure Active Directory, integrated as middleware in Hono.

Resend

Resend is integrated to handle outbound emails, acting as a robust SMTP service that ensures reliable email delivery. This service manages the complexities of SMTP, deliverability, and compliance, making it an ideal choice for this serverless email stack.

• Sending Emails:
  • Cloudflare Workers trigger the Resend service to send emails.
  • Resend takes care of delivering emails to recipients’ inboxes.

JMAP Standard

A notable feature of this project is its compatibility with the JMAP (JSON Meta Application Protocol) standard. JMAP is a modern protocol that offers a more efficient and simpler approach to managing emails in comparison to traditional protocols like IMAP and SMTP.

• Advantages of JMAP:
  • Simplified API: Interaction with emails is made straightforward through JSON-based API calls.
  • Efficiency: Designed to minimize data transfer, making it ideal for serverless environments where performance and cost efficiency are prioritized.
  • Compatibility: Widespread support for JMAP across various email clients ensures broader accessibility.

Webmail client

To enhance the project’s usability, a simple email client was developed, specifically designed to integrate seamlessly with this email stack. This client demonstrates the ease with which developers and users can engage with the system, offering a familiar and intuitive interface.

• Client Features:
  • User-Friendly Interface: Facilitates the simple sending and receiving of emails.
  • Security Measures: Incorporates authentication mechanisms to protect user data.

Acknowledgments

The project draws inspiration from another open-source project known as elasticinbox-cloudflare, which explores similar concepts in leveraging Cloudflare for email functionalities.
It also leverages Jam as a strongly typed contract for both backend and frontend.

Conclusion

This serverless email stack exemplifies the transformative potential of modern cloud technologies in rethinking traditional services.

All the code is available in the repository.

Why

Developing cross-platform mobile apps can be done with a large variety of frameworks and tools, but, for native applications (or close-to-native performance), the selection is limited. Xamarin is one of the frameworks that is considered native, and it has evolved a lot over time to stay updated with the latest improvements and features of the mobile OSes.

Up until recently, Xamarin could only be developed with ease in the Visual Studio IDE. The downside is that someone might consider it limiting when it comes to license. On the other hand, VS Code is more widely used by a large group of developers.

The latest version of .NET 6 brought Xamarin inside the dotnet tooling in the form of workload and it’s now possible to get a fully functional development environment, that requires only the dotnet SDK.

What changes

Now it is possible to target Android in a C# project file using the standard SDK project. Here is an example file:

1
2
3
4
5
6
7
8
9

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net6.0-android</TargetFramework>
    <OutputType>Exe</OutputType>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Xamarin.AndroidX.Arch.Core.Runtime" Version="2.1.0.12" />
  </ItemGroup>
</Project>

VS Code setup

1. Have VS Code ready
2. Install dotnet SDK
3. Add dotnet workload for Android

   1	dotnet workload install android-aot

4. Open the sample project
5. Install the extensions
6. Start the emulator from the task

Running

1. Start the build task
2. Select the target Run
3. Wait until the emulator is loaded with the app and it’s showing a white screen
4. Attach the debugger to start the application

And more

Another area with huge improvements is binding libraries. Just change <OutputType>Library</OutputType> and add the jar or aar. Add Metadata.xml if needed. That’s it.
For more information, have a look here.

References

• Xamarin in .NET 6 Presentation

Basics of Three

Three JS is an interface library to WebGL and allows to render scenes defined in JS. Three allows to control the data structures down to the level of the buffer that is sent to WebGL. Geometries abstract the definition of the shapes in the space by the means of coordinates of the vertices. Other than the vertices, it is necessary to define the edges and the faces that together make the mesh. A vertex is a point in space and is composed by three spatial coordinates, and three points define a face.

The flexibility of Three allows to use many different common shapes or build geometries from scratch, by defining vertices, faces and so on.

Simple rope with built-in classes

To draw a rope, it is possible to use the TubeGeometry. This kind of geometry builds a tube along a curve in the 3D space. It is easy and straightforward to create a shape from a curve, but the main limitation is that there is no easy way to animate the curve without instantiating a new geometry.

Allocation of new objects is expensive and the execution will stutter.

Performant geometry for updates

To address the performance problem with the out-of-the box geometry, it is possible to define a new type of geometry that operates in the same way as the TubeGeometry, with a difference: this new geometry should allow to be updated if the path changes, without having to be reinstantiated.

Introducing the AnimatedTubeGeometry: this geometry exposes a method updatePath that recalculates the vertices’ positions. In the naive implementation, the geometry computation logic is repeated every time the update method is called. The performance is still far from satisfactory, mostly because the vertex math makes use of dynamic arrays, that are very expensive in terms of resources. The issue can be tackled by using buffers, and the class BufferedAnimatedTubeGeometry does just that. The vertices’ update logic does not use dynamic memory in order to be performant, and we can instantiate the arrays (buffers) that contain the geometry information once in the constructor and update the elements afterwards. In fact, the number of the vertices and faces is defined by the inital call to the constructor of the geometry, and, afterwards, only the position will change. Moreover, between all the attributes that define the geometry, only vertices and normals will be affected by a curve update, whereas UVs and indices (face definitions) stay the same.

The following scene shows the comparison in terms of performance between the “naive” AnimatedTubeGeometry and BufferedAnimatedTubeGeometry. Click the button to switch geometry and see the difference in the rendering time.

One last details is to make sure to update the path in an optimized way. It does not make sense to redraw the path if the changes are not rendered. So the guidance would be to call updatePath only after the curve changes, but not more than once between two different visible frames.

Animating

Having prepared the framework for animated tubes in Three, we can then leverage Theatre JS to set up an animation studio in the browser. Theatre allows to build transitions of properties, and also set up a sequence with interpolations, easings and so on. The rope is build on top of the class CatmullRomCurve3, that interpolates a list of positions in the space to build a curve. Each one of this position is shown as a ball in the studio, and we can connect the spheres’ positions to properties of a Theatre JS object.

1
2
3
4
5

const props = sheet.object(name, {
  posX: i,
  posY: 1,
  posZ: 0,
});

We can use OrbitControls and TransformControls to interact with the objects in the scene. The following is the relevant part of the code that stores the positions in Theatre JS when they the objects are moved manually.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

const orbitControls = new OrbitControls(camera, renderer.domElement);
const transformControls = new TransformControls(camera, renderer.domElement);

// We store an object name for each sphere so we can easily look up in a hash table
transformControls.addEventListener('objectChange', listener => {
  const scrub = studio.debouncedScrub(1000);
  const objName = listener.target.userData.selected;
  if (nodes[objName] === undefined)
    return;
  const { props, mesh } = nodes[objName];
  scrub.capture(({ set }) => {
    set(props.props, {
      posX: mesh.position.x,
      posY: mesh.position.y,
      posZ: mesh.position.z
    })
  });

  // This is used in the rendering method to call the updatePath on the geometry
  sceneState.geometryNeedsUpdate = true;
});

transformControls.addEventListener('dragging-changed', event => {
  orbitControls.enabled = !event.value;
});

Object.entries(nodes).map(([, value]) => {
  value.props.onValuesChange(newValues => {
    value.mesh.position.set(newValues.posX, newValues.posY, newValues.posZ);
    sceneState.geometryNeedsUpdate = true;
  });
  scene.add(value.mesh);
});

Final result

Here is the final result: use the button to toggle the animation.

The state of the conversations can be saved on the blob storage. Another Azure Function could be triggered and send messages to the conversations.

Quickstart

All the guidance to use it is in the README.

The font

To embed the new emoticons in the system, a specific font file, NotoColorEmoji.ttf, has to be replaced (under /system/fonts). Many choices are available: for example it is possible to use a font derived from Apple’s emoji font or the Android N font (actually, it is the latest version of NotoColorEmoji.ttf), which includes all the Unicode 9 emoticons. For this project a new font has been built from the sources of Noto emoji font.

Building

The main passages of the process are the following:

1. Set up the folder where the project has to be compiled, download the sources of noto-emoji from the previous link and install all the dependencies

2. Download the Twemoji project from the link provided above

3. Export the emoticons in PNG format from the svg folder. For example, the following code will export a SVG file to a PNG file with a width of 128px, which is enough for to make the font (play with bash to automate the process):

1

rsvg-convert -w 128 -a inputfile.svg -o outputfile.png

4. Copy all the PNG files into a directory in the noto-emoji project folder and rename accordingly to noto-emoji PNGs in the folder png/128
5. Change the makefile to exclude the generation of the flags (example).

Installing

The build process generates a file named NotoColorEmoji.ttf (here it is my compiled font), which has to be copied in the Android system in the folder /system/fonts. Root privileges are required to do so and system partition should be remounted as r/w-able. It is necessary that the permissions of the files are rw-r–r-- (644). This allows Android to use Twemoji instead of original Google emoticons system-wide.

The keyboard

Xperia system includes a good keyboard, but the emoticons coverage depends on the version of the system. By analyzing the apk, it can be found that there are many XML files (under res/xml) responsible for the emoticon list. The modification of these files leads to show the new emoticons in the keyboard.

Decompiling

The APK file can be found on many mirrors (for example here). The process of decompilation requires the use of apktool, which has to be set up: it is required to install the Xperia framework files framework-res.apk (from /system/framework) and SemcGenericUxpRes.apk (from /system/framework/SemcGenericUxpRes/). These files can be exported from an existing Xperia device with Marshmallow version (API 23), because the keyboard application had been compiled with such libraries. Here it is an example with ADB:

1
2

adb pull /system/framework/framework-res.apk
adb pull /system/framework/SemcGenericUxpRes/SemcGenericUxpRes.apk

Libraries can be installed in apktool with the commands apktool if framework-res.apk -t sony_m and apktool if SemcGenericUxpRes.apk -t sony_m. Then, the decompilation takes place:

1

apktool d xperia-keyboard.apk -t sony_m -o xperia-keyboard

Tweaking

The decompilation produces a folder (xperia-keyboard) containing all the resources of the original application. XML files of emoticons panels are res/xml/page_emoji*. Each emoticon is defined by a Cell tag such as the following one:

1

<Cell semc:label="\\U+1f600;" />

It is only necessary to add the rows of the new emoticons. The list of codepoints (e.g. U+1f600) can be obtained through Emojipedia. In this case a web crawler can be written to build a TXT file with all the new codes. This is one written in Python (requires BeautifulSoup):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

import BeautifulSoup
import urllib2
import Queue
import threading
categories = ['unicode-9.0']
before = '<Cell semc:label="'
after = '" />'
baseurl = 'http://emojipedia.org'
hdr = {'User-Agent':'Mozilla/5.0'}
def worker():
  while True:
    item = q.get()
    req = urllib2.Request(baseurl+item,headers=hdr)
    l.append({'name':item, 'source':urllib2.urlopen(req).read()})
    q.task_done()
    print 'done '+item[1:-1]
for k in categories:
  print k
  f = open(k+'.txt','w')
  url = baseurl+'/'+k
  req = urllib2.Request(url,headers=hdr)
  get = urllib2.urlopen(req).read()
  dom = BeautifulSoup.BeautifulSoup(get)
  data = dom.find('div',{'class':'content'}).findAll('ul')
  emojis = data[1].findAll('a');
  q = Queue.Queue()
  l = []
  for e in emojis:
    q.put(e['href'])
  for i in range(6):
    t = threading.Thread(target=worker)
    t.daemon = True
    t. start()
  q.join()
  print 'writing file'
  for p in l:
    dom = BeautifulSoup.BeautifulSoup(p['source'])
    h2 = dom.find(text="Codepoints")
    ul = h2.parent.findNextSibling('ul')
    codes = ul.findAll('li')
    f.write('<!--'+p['name'][1:-1]+'-->\n')
    complete = before
    for i in codes:
      complete += '\\\\'
      c = i.find('a').text.split(' ')[1]
      complete += c
      complete += ';'
    complete += after
    print complete
    f.write(complete+'\n')
  f.close()

With some work of copy-pasting these codes can be embedded in original XML files.

Recompiling

Recompilation takes place with apktool:

1

apktool b xperia-keyboard -t sony_m -o xperia-keyboard-uni9.apk

The package can not be installed yet because it lacks the signature, neverthless for personal and debugging purposes it is very easy to produce a certificate and sign the APK. Here follow the commands necessary to do so:

1
2

keytool -genkey -v -keystore my-release-key.keystore -alias mykey -keyalg RSA -keysize 2048 -validity 10000
jarsigner -verbose -sigalg SHA1withRSA -digestalg SHA1 -keystore my-release-key.keystore xperia-keyboard-uni9.apk mykey

Installing

Once the APK is signed it can be installed through ADB (adb install -r xperia-keyboard-uni9.apk) or copied to the device and then installed through Android’s package manager GUI.
The process will work only if the original package is not installed, otherwise the different signature will prevent the replacement of the application.

ISO 26262 documentation

Given the current vehicle speed V, and the distance D of the vehicle from the preceding vehicle, if $\frac{V^2}{100}>=D$ then the EBA activates the brake request and keeps braking the vehicle until $\frac{V^2}{100}<D$ The brake request is either 0 (indicating no brake is needed) or 1 (indicating that bake is needed). The EBA receives the vehicle speed V and the distance D every 100 msec, and decides whether a brake is needed or not.

Elements of the item

Microcontroller (CPU + embedded memory + CAN interface), radar.

Interaction of the items with other items

The item interacts with the Body Computer through the CAN for receiving the vehicle speed and for providing the brake signal.

Identification of provided functionalities to other items

The item provides the brake signal to the Body Computer through the CAN.

Identification of required functionalities from other items

The item requires the vehicle speed from the Body Computer through the CAN.

Identification of hazards

To identify correctly the hazards, we can firstly perform the FMEA (Failure Mode & Effect Analysis)

• FM1: radar breaks → distance is not correctly measured. Note that this failure can lead to different situations: distance measured is smaller than the real one or distance measured is higher than the real one. Anyway the main point is that the measure is not correct anymore due to the failure.
• FM2: microcontroller breaks → break signal is not reliable. Also for this failure we can have different situations, but the key point is that the microcontroller is not able to correctly perform the algorithm and as a consequence the brake signal is not correct.
• FM3: CAN interface breaks → information is not correctly exchanged with the CAN.

All these failures lead to two main hazards that we have to take into account for the Hazard Analysis and Risk Assessment of the item:

• H1: the item sends the brake signal when is not required (i.e. V²/100≥D).
• H2: the item does not send the brake signal when is required (i.e. V²/100<D).

Identification of operational situations

The key point in the identification of the operational situations is that we have to consider also the eventual presence of a following car. Hence we call:

• d1 the distance between the car under analysis and the following one
• d2 the distance between the car under analysis and the preceding one
  As a consequence, we can discriminate the operational situations by considering:
• Distance that can be higher or smaller with respect to safety distance (safey distance = V²/100).
• Speed that can be higher or smaller than a threshold value, we will consider 30 km/h as threshold (the speed will distinguish different degrees of severity).

Therefore, the operational situations are the following:

• OS1: V < 30 km/h, d1 < safety distance, d2 > safety distance;
• OS2: V > 30 km/h, d1 < safety distance, d2 > safety distance;
• OS3: V < 30 km/h, d1 > safety distance, d2 < safety distance;
• OS4: V > 30 km/h, d1 > safety distance, d2 < safety distance;
• OS5: V < 30 km/h, d1 < safety distance, d2 < safety distance;
• OS6: V > 30 km/h, d1 < safety distance, d2 < safety distance;
• OS7: vehicle moving and a pedestrian or a cycle crosses the road.
  Note that it is useless to consider the operational situations in which:
• OS8: V < 30 km/h, d1 > safety distance, d2 > safety distance;
• OS9: V > 30 km/h, d1 > safety distance, d2 > safety distance;
  In fact, in both of them, the distance is limit is respected and there is no risk.

ASIL determination

General notes

• The exposure is always 4 for the OS 1-6, in fact it is a common condition (> 10% of the driving time) to be driving either at low speed (V < 30 km/h) or at high speed (V > 30 km/h).
• The controllability has been set according to the following rules:
  • H1 is considered less controllable than H2 since the driver does not expect that behaviour and he cannot control it by breaking or accelerating. Hence:
    • If V < 30 km/h → C = 2;
    • If V > 30 km/h → C = 3;
  • H2 is considered more controllable than H1 since the driver needs only to brake to avoid dangerous situations. Then:
    • If V < 30 km/h → C = 1;
    • If V > 30 km/h → C = 2;
• The severity has been set according to the following rules:
  • If V < 30 km/h, then:
    • If only one of the two distances is not respected → S = 1;
    • If both the distances are not respected → S = 2;
  • If V > 30 km/h, then:
    • If only one of the two distances is not respected → S = 2;
    • If both the distances are not respected → S = 3;
• For what concerns OS7:
  • The exposure is set to 3 since we consider an urban scenario, then it is medium probable (1% - 10% of driving time) to have a pedestrian or a cycle crossing the road.
  • The severity is always set to 3.
  • The controllability depends on the hazard.

Definition of Safety goals

The item shall measure correctly the distance of the preceding vehicle and shall send the brake request only if the condition V²/100<D is verified.

Definition of Functional Safety Concepts

Since an ASIL D combination came out during the analysis, a lot of resources have to be spent in the functional safety concepts definition.

• FSC1: An ASIL D compatible microprocessor has to be used.
• FSC2: the item shall perform self-test in order to check the correct behaviour of all the components. If any misbehaviour is detected, then the item shall transit to safe state.
• FSC3: redundancy is crucial in case of ASIL D. Hence two radars shall be used to compare the different measurements and eventually two microcontrollers in master-slave configuration.
• FSC4: a bypass circuit which receives V and D and combines them through some logical ports shall be implemented.
• FSC5: an external check of the ECU correct working condition shall be implemented.

Definition of the Safe State

• If only one of the microcontrollers is working, then the other is disabled and the driver is informed.
• If both the microcontrollers are not working, the item is disabled and the driver is informed of the malfunction (e.g. through a led in the dashboard).

Implementation of redundancy

In this configuration two boards work in parallel to measure the distance independently and issue brake request. One board is the master and the other is the slave. Only the master can issue brake request. In case of malfunction of the master the slave becomes able to issue brake requests.

Critical safety application programming

As the ISO 26262 specifies, it is necessary to satisfy some software and hardware rules to operate in a critical safety environment. For this reason, part of the software is produced by automation tools such as Simulink. The hardware platform for the ECUs is Freedom K64F, connected to an ultrasonic sensor HC-04 for distance measurement. Furthermore, an Arduino Nano Pro is used as an external ECU which manages and coordinates the two sensoring ECUs.
The code of the K64F boards is composed by a discrete state part built in Stateflow, then exported in C code and included in the MBED project.

K64F code

The program is based on the source code exported from the Stateflow chart: the main code manages the inputs and the outputs depending on the ECU working state and tries to catch internal errors.
The following are possibile sources of errors:

• No signal from Arduino: the ECU can not operate because the connection with the manager is broken.
• No signal from body computer (simulated on PC): the ECU can not operate without speed value.
• Board error: in this case the ECU is not able to do a self check of this type, thus this error is recognized by the Arduino and notified to the ECU through serial communication.

Both the network checks (Arduino and body computer connections) are performed through a timer which is stopped and reset only when the signal is good. After a specific timeout, if no good signal is received, then the board put itself in error state.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26

void read_bc()
{
  // read speed from body computer (ASCII encoding)
  // - if malfunction set state to ERROR
  if (pc.readable())
  {
    bool malfunc = false;
    int i = 0;
    char buf[10];
    while (pc.readable() && i<10) {
      buf[i] = pc.getc();
      if (buf[i] == '\r') break;
      else if (!isdigit(buf[i])) malfunc = true;
      i++;
    }
    fflush(pc);
    if (!malfunc) {
      // the communication is working as expected then
      // the value is stored and the timer of network fault
      // check is reset
      buf[i] = '\0';
      controller_U.speed = atoi(buf);
      bc_t.reset();
    }
  }
}

The timer check is executed asynchronously to the main code using a scheduled routine.

1
2
3
4
5
6
7
8
9

void netcheck()
{
  bc_t.stop();
  if (bc_t.read_ms() > TIMEOUT) {
    // network error
    state = ERROR;
  }
  bc_t.start();
}

The routine that checks Arduino connection is implemented in a similar way.

Arduino code

Arduino is responsible for the detection of boards’ errors, as they are not completely able to detect a malfunction themself. The following checks are performed on each ECU port (connected to a K64F board):

• Presence of signal: if a port does not receive any signal in a defined number of cycles, then it means that the ECU is not working or that the communication cannot take place. After a certain amount of time the ECU is considered faulty.
• Coherence of signal: the signal coming from a port is analyzed for errors in the structure of information, in particular two checks are executed: the first three characters should represent the distance measured, hence they have to be digits, second, the last two characters should represent respectively the state and the role of ECU. If they are not consistent with what declared before or if the message has an unexpectd total lenght then the ECU could be faulty. However, some disturbs can occur along the transmission line, thus a reasonable number of errors in the stream has to occur in order to consider the ECU faulty.
• Coherence of measure: the distances measured by the ECUs are compared, and if the difference is higher than a certain threshold, then a malfunction could have happened in one ECU. In this case there is not any other way to check which is the correctly working ECU. Hence, we decided to assume that the master has a better hardware so the slave will be considered faulty after a certain amount of wrong measurements.
• Self check on ECU: the ECUs inform the manager about their state, because they can detect a malfunction also on their own (for example absence of signal from body computer which should provide vehicle current speed, as stated previously). If the master ECU notifies the manager that it is not working correctly, then the manager promotes the slave by sending a promote signal, so that it can issue brake requests to the vehicle.

The software implementation relies on SoftwareSerial library, which allows to build virtual serial ports, overcoming the limit imposed by a single UART connected only to pins 0 and 1. With SoftwareSerial, two software ports are created, but they share the same hardware infrastructure (buffer, clock, etc…) so they need to use the resources alternatively. During the main loop of Arduino code, the control over the serial port is swapped between the two software serial connections.
The message sent from each K64F board to the Arduino is composed as follows:

For each port, the following code is executed:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

// Listening for signals from ECU at port one
  i = 0;
  f = false;
  p_one.listen();
  delay(WAITFORDATA);
  if (p_one.available()){
    // A signal is received from ECU at port one
    while(p_one.available() && i < 5){
      if (i < 3){
        // First three characters should be digits
        buff_n[i] = p_one.read();
        // If character is not digit there is an error in the stream
        if (!isDigit(buff_n[i])) f = true;
      }
      else {
        // Other two alphabetic characters for state and role
        buff[i-3] = p_one.read();
        // First is state and should be E (error) or N (normal)
        if ((i == 3) && !((buff[i-3] == 'E') || (buff[i-3] == 'N'))) f = true;
        // Second is role and should be S (slave) or M (master)
        if ((i == 4) && !((buff[i-3] == 'S') || (buff[i-3] == 'M'))) f = true;
      }
      i++;
      delay(1);
    }
    // If the message lenght is not the one expected or there is a structure error
    // in the stream then the count of errors at port one is incremented
    if ((i != 5) || f ) { 
      p_one_errors++;
    }
    else {
      // If the message is good the count of errors is reset
      p_one_errors = 0;
      // The conversion of distance from characters to integer takes place
      // through the function atoi which accepts a string so the last character
      // has to be a null character
      buff_n[3] = '\0';
      dist_one = atoi(buff_n);
      // If the ECU is already in fault (due to previous communication from manager
      // or failed self check) then it is considered
      state_one = buff[0];
      if (buff[0] == 'E') fault_one = true;
      // The role of ECU at port one is saved
      if (buff[1] == 'M')
        role_one = 0;
      else
        role_one = 1;
    }
    // A message has arrived, then the counter of steps without message has to be
    // reset
    wait_one = 0;
  } else {
    // No message received, then the counter of steps without message has to be
    // incremented
    wait_one++;  
  }

After the reading of the second serial port, all the checks are performed, and response is sent to the boards:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

// If the distances are too much different the counter of measurement
  // errors is incremented, otherwise is reset
  if (THRESHOLD < abs(dist_two - dist_one)){
    dist_errors++;
  } else {
    dist_errors = 0;
  }
  // If the measurements differ for too long time the slave board if marked
  // as faulty if the master is not faulty
  if (dist_errors > MAXDISTER) {
    if ((role_one == 0) && !fault_one) fault_two = true;
    if ((role_two == 0) && !fault_two) fault_one = true;
  }
  // If the stream from an ECU is not coherent or is not present at all
  // for some time, the the ECU is marked as faulty
  if (p_one_errors > MAXERRORS || wait_one > MAXWAIT) fault_one = true;
  if (p_two_errors > MAXERRORS || wait_two > MAXWAIT) fault_two = true;
  // Messages to the ECUs are sent on the base of their state:
  // - E stands for error: if an ECU receives E it put itself in error
  //   state if is not already in that state
  // - P stands for promote: if an ECU receives P it put its role in
  //   master state if is not alreay in master state (i.e. able to issue
  //   brake requests)
  if (fault_one && !fault_two){
    p_one.print("E");
    if (role_two == 1)
      p_two.print("P");
  }
  if (!fault_one && fault_two){
    if (role_one == 1)
      p_one.print("P");
    p_two.print("E");
  }
  if (fault_one && fault_two){
    Serial.println("No ECUs available. System is deactivated.");
    // An infinite delay means that the component is deactivated
    // Error message is transmitted continuously to be sure that the boards
    // receive it
    while(true){
      p_one.print("E");
      delay(1);
      p_two.print("E");
      delay(1);
    }
    // Moreover a visual indicator, like a LED can be lighted in this
    // case by the manager through a DigitalWrite command and a suitable circuit.
  }

To improve the system, a bypass communicaton between master and slave can be added, because in case of manager fault the system is not able to work at all. Anyway, in this study, the manager is considered always working properly.

Source code

The code is available at the following location.

The agent

The software is implemented on a LEGO Mindstorms NXT in NXC language, which provides a good flexibility and a low level access to the hardware resources. Our agent is a single armed robot which has to learn how to move its arm to crawl. The feedback of its actions is acquired through the encoders in the servo motor connected to the wheels.
The arm is composed by two joints and two links: arm and hand. The hand has a free extremity, instead the arm is connected to the robot.
To provide better stability and improved torque, some reduction gears have been applied between motors and connected links.
Building instructions can be found here.

Q-learning

This algorithm has three main assumptions:

• a model of the environment is known
• the complete set of doable actions is known
• an environmental value used as reward is available

Model of the environment

In this particular case the robot acts on itself, thus the agent can be considered the environment. In particular it is necessary to define what are the possible states in which the agent can be. In the case of a path finding agent, the states are the possibile positions of the agent in the environment. Instead, for this robot, the states are all the possible position configurations that the arm and the hand can have.
At first glance, the number of total states is the cartesian product of hand states and arm states, nevertheless, arm and hand states are infinite, and the consequence is an infinite number of total states. The solution is to discretize arm and hand positions, for example using a step of 30°. Moreover, both the arm and the hand can not rotate by 360°: the total range, which is smaller, can be measured using the sensor functionality of NXT motors.
For example 4 states will be used for both the hand and the arm: the total number of states is 16. The position of each link will be encoded with a number between 0 and 3.

1
2
3
4

#define ARM_STATES        4
#define HAND_STATES       4
#define ARM_INIT_STATE    0
#define HAND_INIT_STATE   0

Set of actions

Given the structure of the agent, only two types of actions can be done for each joint: rotate in a sense or in the other. In conclusion there are 4 overall actions doable by the robot:

1
2
3
4
5

#define ARM_UP      0
#define ARM_DOWN    1
#define HAND_UP     2
#define HAND_DOWN   3
#define NONE       -1

The definition of NONE action is important because it is used for the initialization of the vector containing the sequence of actions.

Reward

The reward is the movement of the robot: it is measured through the wheels. Also, velocity can be used, but it is required to compute the ratio between the displacement of the robot and the number of iterations.

Algorithm structure

The algorithm starts with the initialization of all the variables:

• A matrix Q with dimension equal to the cartesian product between the number of states and the number of actions: its value measures the consequences in terms of reward of each action for a given state.

1

float Q[ARM_STATES][HAND_STATES][NUM_ACTIONS];

• A matrix E (eligibility) with the same dimension of Q: it is used to create a policy and makes Q higher for actions leading to a greater reward in successive steps.

1

float E[ARM_STATES][HAND_STATES][NUM_ACTIONS];

Both Q and E matrices should be initialized with zeros.

• Two vectors to store actions

1
2

unsigned int sequence[MAXSEQ];
unsigned int sequence_old[MAXSEQ];

The two vectors of actions should be initialized with action NONE (-1);

The complete initialization code is the following:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

// set Q and E to all zeros
for (int i=0; i<ARM_STATES; i++)
  for (int j=0; j<HAND_STATES; j++)
    for (int k=0; k<NUM_ACTIONS; k++){
  if (init){
    Q[i][j][k]=0;
    E[i][j][k]=0;
  } else {
    E[i][j][k]=0;
  }
}
// set policy to all none
for (int i = 0; i < MAXSEQ; i++){
if (init){
  sequence_old[i] = NONE;
} else {
  sequence_old[i] = sequence[i];
}
  sequence[i] = NONE;
}
// reset steps
steps = 0;
// initial reward
reward = 0;

The optimization part consists in a cycle which termines only when the sequence of actions is optimal, i.e. when it is the same of the previous cycle. The cycle is composed by the following steps:

• Between all the possible actions doable in the current state choose the one with the highest Q, if they all have the same value, then do a random action.
• Do the action and get the reward.
• Update the sequence vector and the matrices.
• If the reward is positive check the convergence of the sequence of actions:
  • if convergency happens then exit
  • otherwise reset the agent to initial position
• else do another action.

Choosing the action

The choice of the action to do is based on two opposite approaches: exploitation and exploration.

• Exploitation means to use the acquired knowledge of the environment to choose an action which maximizes the reward: it is a reasoned choice.
• Exploration means to choose arbitrarily the following action.

There are many criteria to select the right approach: in this case the exploration behavior is used only when acquired knowledge is missing or it is unuseful.
For example, a probabilistic approach could have been applied: the kind of approach adopted depends on the result of a random outcome.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

unsigned int chooseMovement(unsigned int arm, unsigned int hand){
  unsigned int choices[NUM_ACTIONS];
  unsigned int num_of_choices = 0;
    if (arm > 0){
      // arm can be moved down
    choices[num_of_choices] = ARM_DOWN;
    num_of_choices++;
  }
  if (arm < ARM_STATES){
    // arm can be moved up
    choices[num_of_choices] = ARM_UP;
    num_of_choices++;
  }
  if (hand > 0){
    // hand can be moved down
    choices[num_of_choices] = HAND_DOWN;
    num_of_choices++;
  }
  if (hand < HAND_STATES){
    // hand can be moved up
    choices[num_of_choices] = HAND_UP;
    num_of_choices++;
  }
  // have all the actions the same Q?
  bool eq = true;
  float curr_Q = Q[arm][hand][choices[0]];
  for (int i = 1; i < num_of_choices; i++){
    if (Q[arm][hand][choices[i]] != curr_Q){
      eq = false;
    }
  }
  if (!eq){
    // there's a max Q
    float bestQ = Q[arm][hand][choices[0]];
    int best_index = 0;
    for (int i = 1; i < num_of_choices; i++){
      if (Q[arm][hand][choices[i]] > bestQ){
        best_index = i;
        bestQ = Q[arm][hand][choices[i]];
      }
    }
    return choices[best_index];
  } else {
    // they are all equal
    return choices[Random(num_of_choices)];
  }
}

Actions execution

Executing the action is a matter of motor rotation, with some simple lines of code it is easily achieved. It is important to save the motor position before and after the action to compute the value of the reward.

Updating matrices

The core of the algorithm is made by Q and E matrices.
In general, for each value of the matrices

$$Q _{i+1} = (1 - \alpha) \cdot Q_i + \alpha \cdot \delta \cdot E_i$$

$$E _{i+1} = E_i \cdot \tau$$

where

$$\delta = (reward + ( \beta \cdot Q _{MAX}) - Q_P)$$

• Q_P is the Q value of the old state and the current action.
• reward does not need any explanation.
• Q_MAX is the best Q available for the current state considering all the possible actions. It is not important to filter only available actions in particular states, because the actions that are not possible to be done will have always a 0 value for Q as they can not be chosen.
• α, β, τ are coefficients between 0 and 1 and they tune the algorithm. In particular
  • α is the rate of learning: an higher value will produce a faster learning behavior
  • β is the weight of the policy (farsight): the higher it is, the more the following possible actions will influence the choice of the current action
  • τ is the forgetting factor

Before the computation of Q and E it is necessary to update the eligibility value of the action done:

$$E_P = E_P + 1$$

E_P is the eligibility of the previous state and the current action.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

void updateQ(unsigned int arm, unsigned int hand, unsigned int arm_old, unsigned int hand_old, unsigned int move, int rew){
  float max_Q = Q[arm][hand][0];
  for (int i=1; i<NUM_ACTIONS; i++){
    if (Q[arm][hand][i]>max_Q) max_Q=Q[arm][hand][i];
  }
  float old_Q = Q[arm_old][hand_old][move];
  float d = (rew + (BETA* max_Q) - old_Q);
  E[arm_old][hand_old][move]++;
  for (int i=0; i<ARM_STATES; i++)
    for (int j=0; j<HAND_STATES; j++)
      for (int k=0; k<NUM_ACTIONS; k++){
        Q[i][j][k]=(1-ALPHA)*Q[i][j][k]+ ALPHA*d*E[i][j][k];
        E[i][j][k]=E[i][j][k]* TAU;
      }
}

The main cycle

All these functions have to be used in the main cycle as described before: the following is an example of main function.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

task main(){
  reset(true); // if argument is true the vector storing the old sequence is reset, too
  while(ButtonPressed(BTNCENTER, false) == 0){
    // if terminal state
    if (reward > 0){
      // check for convergence
      if (checkConvergence()){
        // finished learning
        Wait(500);
        PlayTone(400,300);
        Wait(100);
        PlayTone(600,300);
        ClearLine(LCD_LINE3);
        TextOut(0, LCD_LINE3, "Convergence");
        Wait(1000);
        break;
      }
      else {
        // reset agent
        reset(false);
      }
    // if not a terminal state
    } else {
      // save old state
      arm_previous = arm_current;
      hand_previous = hand_current;
      // select move
      unsigned int chosen_move = chooseMovement(arm_current, hand_current);
      // perform movement and change state
      reward = performMovement(chosen_move);
      // update Q values
      updateQ(arm_current, hand_current, arm_previous, hand_previous, chosen_move, reward);
      // record to policy
      sequence[steps] = chosen_move;
      steps++;
    }
    ResetSleepTimer();
  }
}

Conclusion

The complete code can be found in my GitHub repo, it requires some parameter tuning. The learning is extremely slow, because the machine is intrinsically slow: if you try to simulate the agent with only software you can see results in short time. For code compilation and deployment, BricxCC is a good choice if you are on Windows, otherwise NBC is available for all platforms, but without IDE.

Sitography

• Q-learning applet with an one arm robot
• Q-learning overview
• Visual q-learning Processing sketch