In this article we will discuss some general principles and methods to adopted in different projects to get an idea of what to expect (and what to provide) in a project. Hopefully, this will in someday help you cross-compile without breaking a sweat.

Before we get into the specifics, we will take a detour and explain cross-compilation for the beginners. Those of you who are already familiar with the basics, feel free to skip the intro section.

What is cross-compilation?

It is the process of building an executable from source with a suitable compiler (cross-compiler) in one machine that can be directly executed in another. That is, the build and host machines are not of the same architecture (more on these terms later).

In other words, it is compilation of sources in one architecture that can be run on another with a suitable compiler.

What is the need for cross-compilation?

Predominantly, cross-compilation is used in the embedded world (it is used elsewhere as well) where the host platform (where the application will run) is severely limited in resources. On the other hand, the build machine (usually) has a lot more juice (CPU, GPU, RAM, etc.,) to produce machine code that can be executed directly in another architecture.

These limitations in the host system preclude the possibility of having a compiler on-board and ergo, the need for cross-compilation.

Can you get anything cross-compiled easily?

Short answer, no.

It depends on a lot of things. For starters, the software’s author must have had cross compilation in the road map for the project and refrained from using OS/architecture specific constructs.

There are other factors such as, assumptions made on endianness, alignment or word size, that can cause serious breakage in functionality when cross compiled. Going into the details on each of those topics and how exactly they surface is beyond the scope of this article, but Wikipedia has some nice write-ups if you are so inclined.

With that said, there are still a vast majority of small projects, that with little effort can be cross compiled successfully.

Typical cross-compilation process

In essence, cross-compilation is just a matter of export CC=some-arch-gcc followed by a make in the same shell. Although this can theoretically work (perhaps on small projects), the real world isn’t as straightforward as you might expect. Partly, due to the complexity in requirement of the system you are trying to cross-compile.

Dependency resolution

The biggest hurdle in cross compilation is meeting all dependencies of a given software package. In standard compilation, when you come across a build time dependency, it’s just a matter of apt-get/yum install pre-compiled version of that package. This also ensures all dependencies of that package is met: which is actually the most tricky part.

When cross compiling, the task of dependency resolution is left to us. Let’s consider the scenario where you are tasked with cross compiling curl to your target.

You will soon come to the realization that curl depends on openssl and zlib (and more depending on your need). Openssl in turn depends on pcre and some more packages. Now you have to cross compile three (or more) other packages to compile curl.

I’ve had times when I ended up cross compiling a dozen packages when I initially set out for one. Fret not, soon it won’t seem too hard and later it will be part and part of your development life. Now that we have covered the basics, let us jump into the actual topic.

Cross-Compilation techniques

You might be surprised at the number of ways in which you can get something cross-compiled. As explained above, its just a matter of exporting a bunch of variables to your shells environment and then initiating the compilation process. In the following sections we will see some of the popular methods.

Autotools based subsystems

Autotools does put some method to the madness. If you aren’t familiar with autotools, you can get an overall idea in this stackoverflow answer. The configure and config.guess scripts seem to have taken away much of the incoherence and standardized the system. Most of the time, it’s just ./cofigure CC=arm-linux-gnueabihf-gcc followed by the usual make. Obviously, it comes with a learning curve, that is made a lot simpler if you are familiar with pure make systems.

Understanding host, target, and build

To work effectively with autotools, you will need to clearly distinguish the terms, host, target and build. Do note that from the compilers perspective, there are some slight variations in the definitions of these terms. For the sake of simplicity we will leave that out for now.

host: The system that is going to run the software once it is built.

build: The system where the build process is being executed. When cross-compilation, this cannot be the same as host.

target: This is a rather confusing option: this specifies where the software being built will run on interacting with another architecture. Simply put, this option is used only when you are building toolchains/debuggers for the host. Confused? well you could read about the Canadian cross, to get a better picture.

This only exists, or rather has a meaning, when the software being built may interact specifically with a system that differs from the one it’s being executed on (our host). This is the case for compilers, debuggers, profilers and analyzers and other tools in general.

Install prefix

Of course, configure can do a lot more that that depending on what all the project has be setup for. One of the most useful argument to configure script is the --prefix=path/to/install/direcory/ this is useful if you are installing to a non-standard path (which you will most of the time in an embedded perspective).

Pure make based subsystems

Plain GNU Make recipes with no flavors is called pure make based subsystems. Pure make systems are probably the most cluttered method of cross-compilation. This is due to the fact that, the make subsystem doesn’t impose any rules on how things have to be done. As a result, each project came up it’s own way to skin the cat.

But don’t lose hope, it’s not as bad as it sounds. Somehow the community has settled down with some broad rules/methods. But there will still be the occasional outliers who will force you to dig deeper into the makefiles. Depending on the project, you will have to resort one or many of the methods below.

Exporting compiler prefix

Cross compiler binaries are usually named in the format arch-platform-abi-gcc. An example of that would be arm-linux-gnueabihf-gcc. Here we can see that the gcc is prefixed with arm-linux-gnueabihf-.

You are expected to export CROSS_COMPILE=arm-linux-gnueabihf- and then perform the build.

Pretty straightforward right? wait, now comes the annoying part. Some projects prefer to export CROSS_COMPILE=arm-linux-gnueabihf, ie., without the trailing - character.

Personally, I prefer the one with the hyphen in it as it allows me to do,

CC := $(CROSS_COMPILE)gcc
AR := $(CROSS_COMPILE)ar

This way, when I don’t export anything to CROSS_COMPILE, I end up compiling for host machine as CC would evaluate to just gcc.

Exporting toolchain path

In most places, you don’t really export toolchain path separately. You pass the compiler prefix and the underlying make system expects to find the toolchain in your system’s environment (usually the PATH variable). But this approach doesn’t scale very well when you have more than one compiler with the same name. Therefore, some projects expect you to explicitly state the toolchain path.

Usually its is called TOOLCHAIN_PATH but there is no said rule on this. it can vary from project to project you will have to look into the makefile and/or the documentation to see what it is expecting.

Exporting absolute path to cross-compiler:

In projects that do not take TOOLCHAIN_PATH, it can be particularly hard if you have more than one compiler of the same name installed in your computer. One way to deal with this, is to keep only the path to the compiler you plan on using in your environment’s PATH.

There is also another way to deal with such scenarios: you could pass an absolute path to the cross-compiler to the make system as,

export CC=/opt/arm/gcc-6.3.1linaro-6.3.1-2017.05-x86_64_arm-linux-gnueabihf/bin/arm-linux-gnueabihf-gcc

This approach (IMHO) is cleaner as you don’t have to fiddle with the systems PATH variable.

Now that we have a fair idea of the different practices that are followed, we can lay down some rules/guidelines to make a project cross compilation friendly.

Guidelines for cross-compilation

Before you make a project cross compilation friendly, think if the project needs to work on a machine (host) that is not the machine it which it was built for (build).

For example, the GLCD Emulator (GLEM) that I created is almost always meant to be operated on the build machine as it is a tool that helps creating embedded GUI during development phase. On the other hand, for a project like limpid, I knew cross-compilation has to be a design goal as the library is expected to be used for communication/integration with embedded applications.

Allow make variables to be overridden

In makefiles, use the conditional assignment operator ?= (assign only if not set) as apposed to regular assignments such as = and := wherever necessary. This give the users the ability to override some of your variables from command line like so, CC=arm-linux-gcc make all.

Don’t hard code anything

This is true for paths, compiler names, tool names, etc., If you need to access some tool, conditionally assign it to a variable and then use that instead.

Some people feel the need to hard code the PWD in their make recipes as they did not understand the difference between = and := and getting the PWD seemed impossible at that time. As it turn out, this is rookie mistake and turns off people. Just take the longer route and read and understand the difference.

Verbosely list all dependencies

Listing all external dependencies of a project is a good starting point for cross compilation. This gives your users an estimate of the effort needed to your code compiled to their host. It also reduces the number of emails you get asking for clarifications.

Stick to the generic stuffs

This is more of a note on portability. If you plan on having a wider coverage for your project, you should stick to things that are generic read about POSIX.

For instance, if you want your project to be able to run on a windows platform (sinful as it can be), you might want to stay aways from stuffs like Unix domain sockets.

Don’t over engineer

One of the common mistakes that people do when staring off a project is to start big, use your sense of proportion.

Don’t employ auto tools for a small project (sub 50k lines of code) it just adds too much clutter to your working directory. Don’t adopt a multi-makefile chaining solution for sub 10k lines of code project, a single makefile would do perfectly fine.

Provide for more than one way to skin the cat

People like to have more than one way to get a project cross compiled. This can be due to a variety of reasons, it may be just be that they have a personal preference (I do). Or they may already have a build system in place that provides some interface for cross compilation and would like to integrate with minimum to no code change.

This may not always come in handy given the number of ways there exists, but its certainly better than having only one.

Documentation

This is probably this single most important thing to do in your project if you expect others to work/contribute on top of it. A single README file form you on the top level of the repository explaining the working with a preliminary cross-compilation example will be a huge help for anyone attempting to use your project.

Never, ever, skimp of this.

Conclusion

To wrap it all up, I think I’ve covered most of what came to my mind about cross compilation. Like I said earlier, at the end of the day, it all boils down to a bunch of exports followed by a make. But surprisingly enough, a lot of people get stuck here. Hopefully this article helped some people get over their fear/fret for cross compilation. Leave your feedback/suggestions in the comments section below.

Edit History:
30 Aug 2018 - Correct heading levels to match new styles

We start by installing tftp client and server packages along with xinitd.

$ sudo apt-get install xinetd tftpd tftp

Create a directory to to act as your TFTP root (the place from which you serve your files) and set permissions so as to allow everyone to read-write-execute from there. Typically, this directory would be called tftpboot and placed at root level, and we will just stick to that convention.

$ sudo mkdir /tftpboot
$ sudo chmod -R 777 /tftpboot
$ sudo chown -R nobody /tftpboot

Create a new service in xinitd by creating /etc/xinetd.d/tftp with the following contents.

service tftp
{
    protocol        = udp
    port            = 69
    socket_type     = dgram
    wait            = yes
    user            = nobody
    server          = /usr/sbin/in.tftpd
    server_args     = /tftpboot
    disable         = no
}

Now that everything is in place, restart the xinetd service for the changes to kick-in and start your TFTP server.

sudo service xinetd restart

You can pass an additional -c option to server_args to allow remote file creation, but that is generally not preferred.

Now the TFTP server should be up and running. Now let’s test if its working,

$ echo "Test TFTP Server" > /tftpboot/test.txt
$ tftp <server-ip>
tftp> get test.txt
tftp> quit

After performing the above steps, you should have a file named test.txt in your current directory with the contents “Test TFTP Server”. If you did, your server is configured correctly.

But what if such tools are NOT an option?

This is not as remote as it appears at first glance. Coverity is not open source, ergo not at your disposal when you really need it. Valgrind may not have been ported on to your target platform or there may not be enough space—in a near production product—left to add a beast like Valgrind.

For all you know, it could just be that your delivery time lines are too rigid to accommodate the time taken to setup the necessary test bed to do a full scale memory analysis. This is particularly likely in memory leaks; they have a nasty habit of surfacing during the last minute when longevity test are conducted.

Nonetheless, it helps to understand what makes a leak, a leak. Even if you are using a tool, you need to understand how to discern the meaning of the logs they spit out (you are right, they don’t scream: there’s your leak go fix it!) and spot a leak, in an endless list of false positives.

Broad Categories of Memory Leaks

For most practical situations, you will be facing a deterministic, reproducible leak in one or more processes resulting in inflation of total used memory and eventually system failure due to starvation. We will get to that in a bit. But for now, let’s take a look at the board categories of the memory leak bugs.

Obviously, there are a lot of other remote places where a memory leak can happen too. The goal here is to be aware of most commonly occurring patterns (low hanging fruits) for a rainy day.

Early returns

This is perhaps the single most common type of leak that I have come across. The cause is pretty simple, code changes that introduce a return statement made by ignorant maintenance programmers. Generally, the software written by a programmer with full context in mind (usually the initial developer) doesn’t suffer such problems.

This can be better explained with an example. Let’s assume method print_model_name gets a cars model name (dynamically allocated pointer for some odd reason!) and prints it.

int get_print_model_name(car_t *car)
{
    char *model = malloc(sizeof(CAR_MODEL_STR));
    if (model == null)
        return -1;

    get_car_model_name(car->model_id, car->year, &model);
    printf("Model: %s\n", model);
    free(model);
    return 0;
}

So far so good. Later, a maintenance programmer drops by and find that get_car_model_name returns error for cars that were manufactured before 1999 (not relevant, but maybe they didn’t have a model number back them). He goes ahead and makes the changes necessary to handle this case. After the change, the code looked like this:

int get_print_model_name(car_t *car)
{
    char *model = malloc(sizeof(CAR_MODEL_STR));
    if (model == null)
        return -1;

    // get_car_model_name returns non-zero on errors.
    // propagate the error to caller.
    if (get_car_model_name(car->model_id, car->year, &model) != 0)
        return -1;

    printf("Model: %s\n", model);
    free(model);
    return 0;
}

As you must have noticed, the return that he added now, does not free the allocated pointer model.

This is a classical example of an early return causing a leak. In this example it may look seemingly obvious, as there are only a couple of lines in the entire method. Things may not look as obvious when functions are a 100 lines long.

Now, is the time you recollect those programming best practices class where you were instructed to break down your functions into smaller blocks so they fit one widow width?

A good practice is to stick to 30 or 40 lines as the upper limit for your methods.

Strdup pointer mix-up

strdup is a method that is used to duplicate a given string on to an allocated pointer. strdup will allocated a pointer sufficiently large to accommodate the string and the tailing null character. This comes in handy at times; (though I have no liking for this method) Very painful at times too.

Normally, we tend to subconsciously name the duped pointer with a similar name as that of the original pointer. Some prefer adding the suffix _dup or just a number to indicated its the same string. What ever the case, they end up calling free on the original pointer instead of the duped one.

void print_string(const char *p)
{
    char *p1 = strdup(p);
    print_char_pointer(p1);
    free(p);
}

The sad truth of the matter is, GCC doesn’t catch this sort of errors (even with -Wall and -Wextra).

Nesting allocated pointers

With complex problem statements, we are forced to use complex data structures. Often times, we allocate memory for a structure and then allocated memory for some members within the structure. For example,

struct sensor_data {
    int sendor_id;
    char *sensor_name;
    float data;
}

struct sensor_data *get_my_sensor_data()
{
    struct sensor_data *d = malloc(sizeof(struct sensor_data));
    if (d == null)
        return NULL;

    d->sensor_id = MY_COOL_SESOR_ID;
    d->sensor_name = strdup(SENSOR_NAME_STRING);
    d->data = get_sensor_data();

    return d;
}

Here the returned pointer to struct sensor_data has two allocated pointers instead of one. The caller will have to free d->sensor_name first and then free pointer to struct sensor_data.

The way this is done is create a dedicated destructor method to free all such child allocations and then free the parent itself.

void destroy_sensor_data(struct sensor_data *d)
{
    free(d->sensor_name);
    free(d);
}

At a lot of places where they don’t have such helper methods to release allocated memory, not all these child allocations are not freed before the parent is free-ed. The destructor method nicely scales as you add more pointers into the structure that needs to be free-ed.

Pointer arithmetic loss

Pointer arithmetics is so popular that, pointer without arithmetic operations on them is just unthinkable (unusable). It is also a popular way to loose an allocated pointer without a trace.

You allocate a pointer to a series of integers. Then, by force of habit, inside loops you increment the pointer dereferencing it at each iteration to store a value that that offset. The end result, you have an array of non-free-able integers.

int *get_numbers(int till)
{
    int i;
    int *p = malloc(sizeof(int) * till);
    for (i=1; i<=till; i++) {
        *(p++) = i;
    }
    return p;
}

Notice that the pointer that the caller will get will be pointing to the end of the allocated memory chunk in instead of the beginning. A free on p at this point will have no effect what-so-ever.

Abandoned list node

Lists (single or double) are a very common data structure. Often times, each node in the list dynamically allocated on demand. The problem is that, the nodes have to be removed from the list and then destroyed (remember nesting allocated pointers). Sometimes, we remove an item from the list and just forget to properly clean it up.

Again, this is a classic place where a destructor method for each node make a lot of sense.

Strategies for locating/fixing memory leaks

There are several strategies that you can employ to effectively locate and fix memory leaks. Some of them are common sense, while some come with work experience. I have listed down whatever I remember from my time fixing such issues. This is more of a reminder for myself when I have to get back to such issues.

Least tested/added code first

If your project is of a reasonable size, you will have a bunch of third-party/FOSS packages included in source tree. Generally (convention), stable releases of these software packages are ported into your applications. This is a good thing because it’s an audited code surface and you can bank on the fact that those packages have already gone through rigorous testing before their releases.

Now that you have ruled out those packages (be sure to git annotate those files to see if any other changes have been added on top of the downloaded packages), you are probably left with code that was developed in-house and those that has a lot of changes made in-house. You can be almost sure that majority of your leak is happening in this area.

Watch out for log flooding

This has been said over and over again. Most of the time, a log file that your application wrote to a RAMFS directory will let you go on a wild goose chase (in this case a leak, of course) for hours, if not days. A general good practice is to disable logging while looking for a leak in the system.

Also note that, deleting a file from RAMFS does not guarantee that the kernel will recollect that memory. That is, even if you delete the flooded log file, the total used memory doesn’t come back down. But don’t let that fool you into look elsewhere.

Accelerate activity for early reproduction

Initially, I mentioned that the leak has to be deterministic and reproducible. The idea is that, you should be able to perform a measured number of steps to reproduce the leak. In some cases, the leak happens even if you don’t do anything, and leave the device powered on for sometime. But that is not true, the device is doing a lot of things even when it is sting around doing nothing.

Now comes the deterministic part, for a set of actions, over a given period, the leak has to be deterministic. If this is the case, you are very close to finding and fixing it. Now that we know so much about it’s behavior, we can try to narrow down which subset of the action or actions are contributing more to the leak.

Let’s say you have a HTTP server that you suspect is leaking memory. You start by performing a lot of GET/POST requests on the server to see with code code path is causing/contributing to the leak. This will give you some hints on how to focus your search for the leak.

Selective patching

Although I mentioned that you can rely on testing done on external packages, that’s only for initial elimination to reduce the number of balls you have to juggle. You still cannot entirely rule out the possibility that there could be a leak in such packages as well.

This where you go into release notes of each packages and look for any hints on memory leak fixes in later releases. If there is one, you just got lucky. Sometimes it may not be possible to upgrade the entire package to later version. In such case you will have to selectively patch changes that correspond to the leak alone and see if that helps.

If employed correctly, this can turn out to be a life saver. It is a common practice to issue patches for critical fixes to older versions too. You could contact the package maintainers and ask them if they have a patch rolled out for your version.

Kill suspected process

This when you know that the total system memory is going up but you haven’t determined with process/processes are leaking memory. Typically you resort to doing free or top or even cat /proc/meminfo to over an extended range of time and then look at these logs to see if any process has steadily increased in memory consumption.

Once you have narrowed down a list of possible suspected processes, you can restart these process one at a time to see of the total available memory has increased. This method can also be effectly employed to see if the leak caused by a given process if significant enough and thereby re-prioritizing your goals.

Doing it the hard way

Armed with all these information, let’s see what it takes to create a tool to profile and log sufficient information so we can do some smartness over the dumped data to locate and fix memory leaks.

Lay down the ground rules for the tool:

• Solution should not take more than 5 to 10 minutes to setup.
• Should try to cover most common types of leaks (low hanging fruits discussed here).
• Must do some bounds check while we are at it.
• Should catch free errors (free on unallocated pointers).

With all these in mind, I wrote a simple but effective memory profiler that helped me sort of lot of memory leaks. I know there are a dozen such tools that try to address the exact same thing. For sub-1000 lines of programs, (IMHO, of course) it it is far easier to write it yourself then to look and understand someone’s code. It takes almost the same time, and gives far better control over the software.

Hope that was helpful. Did I miss something? leave you comments below. I will gladly take your inputs into this post.

In lay man’s terms, it’s a life saver when you have to work on remote devices over flaky networks. It is a simple tool that does one thing and does it extremely very well.

So far the only annoyance I have had with screen is its behavior when home directory encryption is enabled. But this is not all that big a deal breaker, as most people don’t turn on home directory encryption (except may be accidentally, or when they need to share the machine with some other users), they encrypt the whole drive.

I spend a lot of my time on remote machines. Be it my development server, or the deployment environment all of it is done through a SSH connection to the remote host. At times, these remote hosts are across the globe on another continent. This is really where it gets super annoying, as you have to type something expect it to be there in the next 2 seconds and typos are an expensive mistake.

What are we trying to solve?

To give you an idea of what I am taking about, let’s say you have a development server, where you initiate a build. A clean build on my current project (when I last timed) takes little over 40 minutes and incremental builds take up to 2 minutes (for this, I believe the poor make rules are fault).

Now, lets say you start a SSH connection and initiate a clean build. For the sake of this discussion we will assume that you don’t know how to keep an SSH connection alive indefinitely and your connection is interrupted. Or you were moving your laptop around and your wireless card roamed away to another SSID and you lost connection.

You will notice that the make process that you initiated receives a hang-up signal and dies with your connection.

One obvious and not so great way to deal with this is to start is with nohup and push it to background with a & like this.

$ nohup make &

This does work, in the sense that it will discard the hangup signal and persist even after the connection has ended. But it doesn’t actually help if you were editing some file or if you needed more than just one TTY. This is where screen comes into picture and far exceeds expectations.

GNU Screen

Your work flow will be like this, login to remote machine and then start off a screen session and then start doing your works in that session. If your connection gets interrupted, you don’t have to give a damn about it. Screen is still running in that machine. You can always ssh back to that machine and then reconnect that session to pickoff from where you left.

Screen Basics

The good thing about screen is, you just need to know a handful of things initially to get going. Once you’ve gotten a hang of these basic stuffs, you can always pick up more things as and when you need them. So let’s dive in.

Screen is not installed by default in most distributions, so you must use apt or yum or brew to install it.

Not surprisingly, you start a screen session by using the screen command on a shell.

$ screen

Screen key bindings

Once you hit screen, you are inside a screen session. The key binding prefix for screen is Ctrl + a. Any character followed by this sequence is interpreted as a command to screen.

For example, to detach from the session, you will have to press Ctrl + a, and then press d. The screen documents refer to this sequence with C-a d. This is of the form, C-a <command-character>

List, attach and detach screen sessions

You can You can use screen -ls to see the screen sessions that are active. Since we have started only one session so far, your output will list only one session. The following is an example of a screen listing.

There is a screen on:
    13866.pts-7.extrops (Sunday 11 June 2017 04:13:31  IST) (Detached)
1 Socket in /var/run/screen/S-siddharth.

Currently there is only one screen running in my computer, so the listing returns only one socket. You can create as many screen sessions as you wish and when you do a screen -ls you will notice that there are more sockets listed in /var/run/screen/.

To re attach to the session we earlier detached from, you can do,

$ screen -x

If there are more than one screen sessions, you will not be able to do screen -x directly to attach to a session. You have to add the screen’s name to the reattach command to explicitly state which screen you want to attach to. In this case you will have to do a screen -x 13866 to reattach to that session.

Screen window management

The best thing about screen is, it’s ability to spawn of more than one TTY. In the same session, you can start off may windows. Screen starts one window when you create a session. After this you can use C-a c to create another. And move between those windows by using C-a C-a.

Now you can be in one directory in one and another in the other window. To see the list of windows, you hit C-a " and move up and down in the list to choose one. Once you are done, you can kill that window with C-a k, you will be asked for a confirmation to kill the current window. Here is a small subset of key bindings that you need to know to get started with screen.

This is not a full list of key bindings. There is an overwhelming number of key bindings in the screen manual do have a look at it once you have internalised the basics.

Customizing screen with screenrc

Like vim and bash, the runtime behavior of screen can be customized with a screenrc file in the home directory. The following code block is the contents of my screenrc file, I have been using for years and it works like a charm.

If you are a screen beginner, then this is a good starting point. Copy the following lines to ~/.screenrc and checkout its effects.

startup_message off
term screen-256color
setenv LC_CTYPE en_US.UTF-8
defutf8 on
setenv DISPLAY ':0'
nonblock on
vbell on
msgwait 3
defscrollback 10000
altscreen on

bind = resize =
bind + resize +1
bind - resize -1
bind _ resize max
bind x remove

hardstatus on
sorendition kw
hardstatus alwayslastline
hardstatus string "%{= ky}%-Lw%{=r}%20>%n %t%{= ky}%+Lw %{= ky}%-=| %{= kw}%M%d %c%{-} %{=r} ${USER}@%H "

What to start screen soon after login?

As you familiarize yourself with screen, you will notice that most of the time screen is the first command that you run after ssh-ing into the machine. The easiest way to achieve this is to add an exec screen -d -RR as the last line in ~/.bashrc. That works, but if there is some issue with screen and for some reason it won’t start, then adding this line prevents you from logging into your machine, causing a lot of trouble.

Alternatively, you could add the machine details to your ~/.ssh/config and add a RemoteCommand section to indicate that you would like to attach-to/create a screen session after login.

host my_machine
	HostName 10.10.10.1
	User sid
	RequestTTY yes
	RemoteCommand screen -d -RR

Now you could just do ssh my_machine to login to sid@10.10.10.1. Neat right?

A word on tmux

There is one other tool—tmux—which is gaining popularity lately. I have tried using it and felt like a fish out of the pond, but I’m sure that was highly opinionated.

IMHO, tmux is trying to reinvent the wheel. Why spend all the developer effort in recreating screen when such energy can be channeled to perfect it? All they have managed to achieve is to cause fragmentation in community and user base. That said, I think screen is an amazing tool, being a screen user, I still can’t think of one good reason to switch.

A lot of action is happening around Zephyr in the past few months and it is going to play a major role in the embedded/IoT space in the forthcoming days. Support for Tensilica’s Xtensa core is also in the progress, this will be a game changer if we can run Zephyr on our favourite ESP32 in the near future.

Let’s take a look at zephyr and how we can use it to create amazing applications.

Why Zephyr?

The Linux Foundation, backed by companies like Linaro, Intel, ST, and NXP are spearheading Zephyr development. Zephyr is released under Open Source Apache license, needless to say, MIT and Apache are some of the most desirable licenses for Free and Open Source Software (FOSS) as they don’t pose much regulation. This means you can use it on, pretty much any commercial product without having to worry about licensing related implications.

Zephyr is being developed by the Linux developer community. So the source code organisation is very well done and resembles the Linux kernel source tree in a lot of sense. The make system has been adopted from the Linux kernel, it kind off makes you feel at home.

I have used FreeRTOS in the past, and during my initial days, have had great difficulty in locating where a given module fell ie., it wasn’t very intuitive to begin with. Also, it had the macro soup (#ifdefs literally everywhere) issue that Zephyr doesn’t (currently) suffer.

To follow this article, you will need the following,

• STM32 Minimum Development Board
• STLink V2
• USB to UART converter

STM32 Minimum Development Board

Since Zephyr is a fairly new RTOS, there are very few boards to which it has already been ported to. Most of the boards which already have a port are either not available or outright expensive. In this post I will take up the “STM32 Minimum Development Board” - a cheaper alternative and try to run Zephyr on it.

Well, this is not exactly a development board in the strictest of senses (all it has is an on board LED). This is more of a breakout board with just the bare minimum to get the CPU to boot. Somehow, the term “STM32 Minimum Development Board” seems to have caught on to it and for the sake of consistency with the existing SEO, we will refer to this board in the same way. More details on the board cab be found at zephyr’s board document page.

I have been having my eye on the STM32F103C8T6 series SoC for some time. Reason being, they were inexpensive and had sufficient juice to do some intermediate level stuffs. So I went ahead and bought one of these minimum system development board from eBay for like $4 including shipment.

STLink V2

Although STM32 can be bootloaded without the need for any external hardware, I bought one of this STLink V2 to flash and debug mu code. It was dirt cheap and I know it will come in handy at some point in time.

Hardware Setup

My setup as the following, STM32 breakout board connected to STLink and an UART to USB converter connected to the STM32 to get the console out.

STLink Connections:

UART to USB Connections:

Zephyr - Board Support Package

To my disappointment, Zephyr wouldn’t run on the board out of the box as the SoC wasn’t ported and the board itself didn’t have a port ie., there was no Board Support Package (BSP) for this board as yet.

Then I read through Zephyr’s docs and looked at other supported hardware. After some time, I branched off to start making changes of my own. Within the first 5 or 6 hours of effort, I was able to get a make shift BSP port working with the my board (although I wasn’t able to get the UART to work at that point).

After a bunch of email exchanges with Erwan Gouriou (Zephyr developer), here and here, I was able to get a fully working BSP for the STM32F103C8T6 breadboard breakout board. I raised a pull request sometime back and expecting it to be merged sometime soon is now available upstream.

Build and Flash

Now before you start building with zephyr, you will need to setup the zephyr toolchain and kernel sources. This is fairly straight forward procedure and Zephyr’s Development Environment Setup page does an excellent job at it. If you run into any issues, leave a comment and I will try to sort it out.

I am going to assume you choose the defaults when setting up your zephyr SDK and that you would be using west. If you made modification to the paths during setup, be sure to alter the below guide accordingly. That said, lets go ahead and build the embedded equivalent of a hello-world project — blink-an-led.

For our app to build correctly, you need to source zephyr-env.sh from the root level of the zephyr repository.

cd ~/zephyrproject/zephyr && source zephyr-env.sh

If you plan on workring with zephyr frequently, I found that adding an alias to source the zephyr environment file comes in pretty handy.

echo "alias get_zephyr='source ~/zephyrproject/zephyr/zephyr-env.sh'" >> ~/.bashrc

Zephyr allows your application to be built outside of the kernel and it’s dependencies — shaddow building. It is a good practice to keep your source code out of the upstream repos (to maintain a clean git tree). So let’s copy the blinky samples into your workspace and then build them.

get_zephyr # (optional) this is to source zephyr-env.sh
cp -r $ZEPHYR_BASE/samples/basic/blinky ~/workspace/
mkdir ~/workspace/blinky/build && cd ~/workspace/blinky/build
cmake -DBOARD=stm32_min_dev_black ..
make

If you did everything right, this make should build without any warnings or errors. This build triggers a recursive make process that walks up the Zephyr source tree and builds all required modules. Since this is a Zephyr primer, we won’t get into the details on how to configure the kernel. This will be tasked in a separate post.

Now, the final step, sending the built binary into the board to see the output. To do this, all you have to do is, invoke the flash target and the rest of the work is done for you.

make flash

You will notice that Zephyr invokes open On Chip Debugger (openOCD) to flash the board. You could also invoke the debug target to setup a GDB session with the board. Again, this is a huge topic and hence merits a separate post.

Once this succeeds, you should see the only LED on the board, blink away to eternity.

If you run into some issues with make flash, it porbably due to permission issues. Add the following udev rules to fix them.

wget https://raw.githubusercontent.com/zephyrproject-rtos/openocd/master/contrib/60-openocd.rules
mv 60-openocd.rules /etc/udev/rules.d/
udevadm control --reload-rules
udevadm trigger

Once the rules are loaded, you must add yourself to the group plugdev. If the group doesn’t already exist, just crate it. After this, you should be able to successfully flash the device. In my next post we will explore some other interesting features of Zephyr that could come in handy.

Edit History:
28 May 2017 - Initial draft
14 Jul 2019 - Update "Build and Flash" section to that in latest upstream.
28 Sep 2019 - Add note on udev rules for make flash.

In my previous post on Linux kernel compilation for Beaglebone black, I had used pre-built RFS for booting the kernel. Also, I mentioned that the RFS could be built from scratch using a utility called Busybox. In this post, we’ll see, how to create a Custom RFS using Busybox and what are all the additional files required to boot the kernel. The RFS which we’re going to create contains only the bare minimum stuff required to boot the kernel, so you can’t expect it to behave like your distribution’s rootfs.

Before getting our hands dirty by working with Busybox, let’s acquire some basic theory to get things organized!

What is RFS?

RFS is the Root File System (/), the place where the kernel acts upon. All the applications will reside inside this root file system. Usually RFS is created and placed in the Flash memory of the device, which could be either your Android phone or your Personal computer. There is also one filesystem called initramfs, which is used in the boot process of the Linux based desktops/servers. But, initramfs is a RAM based filesystem which contains the entire root file system directories often compressed and passed along with the kernel image. Embedded Linux devices don’t necessarily need initramfs for booting.

During the last stage of the Linux legacy booting process, the kernel executes the /sbin/init which in turn looks for the inittab file in /etc directory. It is based upon the SysV init process. But, most of the modern linux distros now switched from SysV init to Systemd, which is more flexible.

How do I create a RFS?

Now, we know that rootfs is mandatory inorder to boot the linux kernel. But how do I create one? Have you tried “ls /” in your linux machine… Yes, there are lot of directories under ‘/’. Usually not all the directories are needed to get your linux system up and running. Only a fair amount of directories are needed by the kernel, but it entirely depends upon the end application your linux system is using. It could be either a server or an Embedded linux system for a dedicated application.

In our case we’ll consider the latter one, because our target platform is Beaglebone black, which is mostly used for an Embedded application. Let’s discuss the list of directories needed and their uses.

Mandatory directories:

/bin directory contains the commands used by the normal linux user for day to day activities like ls, cp, rm. It also contains the commands needed during boot process like systemd etc…

/sbin directory contains the binaries used by the super user for system administration. Some of the commands are insmod, lsmod, ipconfig. Normal users can’t use the commands in this directory without administrative privileages.

/etc directory contains the files needed for system configuration like init scripts, network conf files, bootloader init scripts, application conf files etc…

/dev directory contains the special files which represent the devices present in the system like char, block and net devices. It also contains the files needed to interact with the device drivers for a particular hardware using generic read/write calls. For all types of devices attached to the target, appropriate device nodes will be created in this directory.

/lib directory contains the shared libraries used by the applications in the system. Often it contains the glibc/klibc, ld-linux shared libraries. It also contains the loadable kernel modules under /lib/modules which could be inserted into the system dynamically using modprobe/insmod commands. List of modules should be built while building the kernel using ‘make modules’ command. Modules could also be loaded automatically when the devices are attached to the system using some utilities.

/usr directory contains the userspace programs and data. In older unix implementations, this is the place where the home directories of all users were placed. It contains the necessary data, headers, libraries and also some programs like telnet, git etc…

Nice to have:

/proc directory is based on procfs filesystem. It is a type of virtual file system which contains files based on the processes existing in the system. There is no need to create any files under this directory, all files will be created once you mount the procfs in this directory.

/sys directory is based on sysfs filesystem. Like procfs, this is also a type of virtual file system based on the Kernel objects and its attributes. It is most widely used to interact with the device drivers like /dev directory. Drivers need to create sysfs entry, then it may contain files to send/receive data from the driver. For instance LED’s in Beaglebone black could be configured using sysfs/class/leds.

/config directory is based on configfs filesystem. Major use of the configfs is to manage the Kernel objects from userspace. Unlike sysfs, which just acts on the Kernel objects, this one can modify it in runtime.

Alright, we have seen the list of directories needed and their uses. So, are we going to create all these directories and its contents by hand? It would be an over kill, isn’t it?

For this scenario, we’re going to use a utility called Busybox, which will make our life easier.

Busybox Cross compilation

Here is what the busybox creators had to say about it,

BusyBox combines tiny versions of many common UNIX utilities into a single small executable. It provides replacements for most of the utilities you usually find in GNU fileutils, shellutils, etc. The utilities in BusyBox generally have fewer options than their full-featured GNU cousins; however, the options that are included provide the expected functionality and behave very much like their GNU counterparts. BusyBox provides a fairly complete environment for any small or embedded system.

Busybox is focused mainly on Embedded platforms as the size optimization is vastly required. It could be built as the binary requiring shared libraries (default option) or a single static binary requiring no external shared libraries. We are going to use the latter one.

Working with Busybox

Download the Busybox source from here.

Extract the tarball with tar -xvf busybox-1.24.1.tar.bz2. Then, cross compile the source for ARM platform using the following commands.

Note: This assumes that you have the arm cross compilation toolchain configured in your system. If not please go through my previous post to see how to get it done.

$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- defconfig
$ make ARCH=arm CRSOO_COMPILE=arm-linux-gnueabihf- menuconfig

Select Busybox Settings -> Build Options -> Build Busybox as a static binary (no shared libs). Press y for selecting that option and save it. Then execute the following commands for building.

$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- CONFIG_PREFIX=/path/to/RFS install

After the successful completion of the above commands, you can see 3 directories (bin, sbin, usr) and one file (linuxrc) created in your RFS directory. Path to RFS should be like /media/user/rfs which resides in SD card as ext3 file system. Apart from these, we need few more directories mentioned above to boot the kernel. So, move to the RFS location and create the following files and directories.

/dev:

Create /dev and some special files under this directory.

$ mkdir dev
$ mknod dev/console c 5 1
$ mknod dev/null c 1 3
$ mknod dev/zero c 1 5

• /dev/console, is the place where the kernel uses for interaction.
• /dev/null behaves like an empty file and is used to suppress the output from commands/programs.
• /dev/zero contains sequence of zeros used to fill up the memory regions.

/lib and /usr/lib:

For the static libraries, copy from the ARM cross compiler toolchain path.

$ mkdir lib usr/lib
$ rsync -a /opt/arm-linux-gnueabihf/lib/ ./lib/
$ rsync -a /opt/arm-linux-gnueabihf/lib/ ./usr/lib/

/proc, /sys, /root:

Create directories for mounting the virtual filesystems (procfs, sysfs) and root directory.

$ mkdir proc sys root

/etc:

Create /etc and then, create additional files inside this directory.

$ mkdir etc
$ cat >> etc/inittab
null::sysinit:/bin/mount -a
null::sysinit:/bin/hostname -F /etc/hostname
null::respawn:/bin/cttyhack /bin/login root
null::restart:/sbin/reboot
[ctrl-D]

Create another file called fstab and populate it. This file will mount the virtual file systems.

$ cat >> etc/fstab
proc  /proc proc  defaults  0 0
sysfs /sys  sysfs defaults  0 0
[ctrl-D]

Also, create files called hostname and passwd.

$ cat >> etc/hostname
embedjournal
[ctrl-D]

$ cat >> etc/passwd
root::0:0:root:/root:/bin/sh
[ctrl-D]

Busybox init will first look for /etc/init.d/rcS script, if it can’t find that then it will look for /etc/inittab. Inittab file will mount the virtual filesystem using fstab. Also, it will have the command for getting login prompt and shell.

/sbin/init -> /bin/cttyhack -> /bin/login -> /bin/sh.

Here, we don’t use password for login. So, after logging in, set the password by executing ‘passwd’ command.

Note: You may encounter ‘read only filesystem error’. This is due to the kernel parameters which Uboot has passed. You can change it by modifying the uEnv.txt file as:

root=/dev/mmcblk0p2 rw

This will mount the RFS as read/write file system.

That’s it…

Voila! You have successfully created one Custom RFS using Busybox for Beaglebone Black. Just insert the SD card into BBB’s slot and hold the Boot switch (SW2) while powering up. This will boot your linux kernel using Custom built RFS.

As we always say, if you get stuck at any point kindly post your questions in comments in somewhat detailed manner (No need to post the entire log…), we’ll help you out.

The unusual thing was majority of them seemed to give out similar track and robot description. Then I asked one of them to send me an image of the track, thats when I got to know about Mesh Flare, an Event in the TechFest-2016 a technical symposium organized by IIT Bombay. (Here is the full problem statement just incase they pull it down)

After looking at it for sometime, I had to get up and search for my college ID card (so that I too can take part in it), hoping that my year of graduation wouldn’t have been printed on the front.To my dismay, they did have it (very well pronounced).

Well that meant, I can’t actually take part in the event. Nevertheless I had spent some time thinking about it the previous day ,so I thought I will just make a small post on my ideas on the problem statement and how I would have approached it.

Of course I won’t spoil the fun for you. There will be no code at the bottom of this write up. So if you are looking for one, no luck here.

Here is the problem statement,

As always, you start at START and move through this maze and try to find your way to the big white square with the END below it. Now if you look at the track for a few seconds, you will observe that it has more than one loop in the track. This means, if you opt to use the all famous default-left or default right strategy, your robot is probably going to go in circles till it dies or the event coordinators kick you out. Honestly, the latter sounds more probable.

Now that can’t use conventional methods, how do we tackle this seemingly impossible problem statement?

Well it’s not going to be easy. But I can try to make it seem possible to a certain extent.

Your robot must,

• be able to follow a line (duh!)
• be able to detect 189, 90 and 45 deg direction changes. (not so tricky)
• be able keep a track of how far in space it has moved between any two points. (tricky)

Now let’s take a closer look at the sample track that they have provided. There are certain points in the track where the robot has to make a decision. We will call these nodes ‘vertices’. Lets mark out all the vertices in the track.

Yes, dead ends are decision making points. START and END are also decision making points. Now, some of you may have already observed that there is some niceness in the way the nodes are arranged. This is not just coincidence. Therein lies your secret. Now we will try to overlay a grid on the above image and see if we can make any sense out of it.

So, you can see that all the vertices can be made to fall on a grid and all motions happen from one of the 4 corners of each check on the grid. This check that we have defined here is going to be a single ‘unit-length’ on the arena. It’s up to you to define the resolution of this grid. Keep in mind that too much resolution is almost as bad as too little.

Now let’s think about what we have done so far. We managed to discretize the track into ‘unit-length’ checks and represent each check with an offset in space with (0,0) being the start vertex and (x,y) being the end vertex. This is a classic weighted directed graph.

Now you just have to perform a Depth First Search (DFS) on the graph to arrive at the destination node. If you have made it to this point successfully, choosing the least weighted path to the destination vertex should be a cake walk.

All the best for anyone participating. Let me know if this was helpful.

I for one, hated (note past tense) glcd work. I always found reasons to postpone or push it off to one my co workers. Now don’t get me wrong here, I’m referring only to embedded UI design. I had two main reasons for this dislike,

1. The roundtrip time involved in flashing the device, and reflashing with the updated layout. This time may be significantly long if your code base is large.
2. How ever good a programmer you are (IMHO), GLCD layer almost always looks bad after speed optimizations.

Now I know I can’t do much about the second point. But with some thinking, I figured point one is actually addressable with only a couple of my Sundays!

So I spent some time to create an emulator that can take the glcd back plane and display it on a window on your desktop. So the round trip time is totally eliminated from the equation. I can’t say I really like it now, but I certainly don’t dislike it as much as I used too.

How it works?

A typical software architecture for GLCD will be as follows (at least that’s how I would have it). The app layer gets new data that has to be pushed into the GLCD. Then it calls refresh routine to make changes to a backplane. The physical layer then processes this information into a bitmap and then calls a glcdWrite routine to flush the new data into the display RAM.

We will now introduce a small change in code flow that will do the magic. Just before the call to glcdWrite, a code macro is used to send the same data to the emulator instead of the actual GLCD.

The sequence is illustrated in the following image.

Data Encoding

Most commercially available GLCDs are 1 bit per pixel packed into bytes in two different ways.

1. Row Major - MSB of first byte is pixel (0,0) and LSB is pixel (7,0). Second byte represents pixels (8,0) to (15,0) and so on.
2. Column Major - MSB of first byte is pixel (0,0) and LSB is pixel (0,7). Second byte represents pixels (1,0) to (1,7) and so on.

Therefore, the glcd buffer length would be, buf_len = glcd_width/8 * glcd_height;This buffer is what your embedded device sends out to the GLCD through the SPI/I2C bus and the glem server expects same buffer to render the image.

Installation:

If you have any experience installing software in Linux, installing GLEM should be little/no trouble at all.

You can find the git upstream at https://github.com/EmbedJournal/glem.Clone this repository into your machine and follow the instructions in the README.md file to install it.

Once you have made the project, you will have to start the GLEM server with your GLCD’s width and height as arguments. For example if you have a 128x64 1bit GLCD, you should run,

$ ./glem serve 128 64

Now you should see a new window looking something like this (may not be identical as I may have made changes at some point that deviates from this article).

The vertical lines have appeared because I wrote 0x55 to all display RAM. You can also run the following to test if GLEM server is working,

$ ./glem test 128 64

To understand how to use it in your project, have a look at example.c in the src directory. It includes glcd.c which behaves as the physical layer for GLEM. You macro switch should redirect your SPI/I2C calls to this file.

Hope this was helpful. Leave your feedbacks in the comment sections.

A The shopkeeper agreed that the RPM on one of the motors was of a different RPM and other ‘slightly-off’ but the others where just fine. It took me a lot of time to convince him that his definition of ‘slight’ was just unacceptable.

Anyway, I brought that up because, external event counters can be used for a variety of reasons. One of them is to make your own tachometer (and I did it) to prove your argument in a street fight. But of-course you need to be jobless to start with.

In my previous post we discussed the basic interface seven segment displays and how the concept of persistence of vision can be used to used to decrease the pin count of the embedded device.

Counters can either count up from zero or count down to zero. In the embedded space, down counter gives a marginal improvements in performance as most processors have a decrement and jump if not zero kind of instructions. But don’t worry, that was just “fun facts” you don’t have to worry about it. Your optimizing C compiler will take care that for you.

In this post, we will discuss a basic application of the concept that we discussed earlier and proceed along to make a physical counter that keeps track of an external event (key press).

First, we will take up the task to write different data into all the 4 digits of the 7 segment display. Let’s say we want to print 1234 on the display. Here is a flow chart that will help you better understand the concept of persistence of vision (POV).

So if you write 1 to the data bus and enable the first segment, you will have to do the first step. Likewise, write 2 to data bus and enable the second segment. Similarly do the third and fourth step. Once this cycle is finished, repeat the above steps at a frequency, for the sake of argument, lets say 50 Hz.

If you did it correctly, you should see a static 1234 appear over the 4 digits of the 7 segment display. This is how you should program the above logic.

#include <xc.h>
#include <stdint.h>
#include "delays.h"

#pragma config OSC=HS,WDT=OFF,FCMEN=ON,XINST=OFF,IESO=OFF,LVP=OFF

/*
* Hardware connetions,
* PORT B is connected to the 7 segment display
* PORT c Pins 0, 1, 2 , 3, are used as enable pins for segments
*/

void main()
{
    uint8_t lookup[] = {
        0x3f,0x06,0x5b,0x4f,0x66,
        0x6d,0x7d,0x07,0x7f,0x6f
    };
    int idx,idy;
    ADCON1=0x0F;
    TRISB = 0x00;
    TRISC = 0x00;
    while(1)
    {
        idy=0x08;
        for(idx=1;idx<=4;idx++){
            LATB = lookup[idx];
            LATC = idy;
            idy = idy >> 1;
            dealy_ms(5);
        }
    }
}

Here is a small video that I made to demonstrate the working of the above procedure.

Event Counter

Now that the persistence of vision section has been dealt with, we can get started with the counter. For this post we will use a micro switch for providing the input to the microcontroller. But in practical application any digital input (such as that from a IR interruption system) can be used to increment the counter.

Here is a flowchart to help you understand the working of the counter. Since the controller runs endlessly in a while(1) loop, there is not Stop block in the flowchart.

In the main function, the controller first checks if there is a counter overflow condition. If there is overflow, it will reset it back to 0. After this it tests the state of the input switch. If the switch is pressed, it increments the counter. After this, it send the value of the counter to the seg_wrt() function to write the data to the display.

Here is a program to implement the above logic in embedded C.

#include <xc.h>
#include <stdint.h>
#include "delays.h"

#pragma config OSC = HS,WDT=OFF,FCMEN=ON,XINST=OFF,IESO=OFF,LVP=OFF

/*Macro For Segment Enable Pins*/
#define SEG_EN1 LATCbits.LATC1
#define SEG_EN2 LATCbits.LATC0
#define SWITCH PORTDbits.RD0

void seg_wrt(unsigned char);

void main()
{
    unsigned char ctr = 0; // Counter variable
    ADCON1 = 0x0F; // Make all pins Digital
    TRISC = 0x00; // Enable Pins
    TRISB = 0x00;
    TRISD = 0x0F; // 7 segment data Port
    while(1) {
        if (ctr%99 == 0)
            ctr = 0;        // reset the counter if it reaches 100
        if (SWITCH == 1) {
            // In here if switch pressed.
            ctr++;          // increment the counter.
            delay_ms(20);   // bebounce the press.
        }
        seg_wrt(ctr);
    }
}

/*Function to write data in the 7 segment display*/
void seg_wrt(unsigned char val)
{
    uint8_t lookup[] = {
        0x3f,0x06,0x5b,0x4f,0x66,
        0x6d,0x7d,0x07,0x7f,0x6f
    };
    int unit,ten;
    int idx = 0;
    unit = val/10; // separate the two digit data into units and tens
    ten = val%10;
    for(idx=0;idx<1000; idx++) {
        LATB = lookup[unit]; // write the units place
        SEG_EN1 = 1; // Segment 1 ON
        delay_ms(10);
        SEG_EN1 = 0; // Segment 1 OFF

        LATB = lookup[ten]; // write the tens place
        SEG_EN2 = 1; // Segment 2 ON
        delay_ms(10);
        SEG_EN2 = 0; // Segment 2 OFF
    }
}

Here is a demonstration of the above procedure,

I hope this post was helpful in understanding the interface of 7 Segment Displays and how they can be used to display numeric data. In the upcoming posts we will start working with these displays to make some real projects.

Edit History:

• Article first published 15th Aug 2013
• Code updated to xc8 compiler and republished 6th Dec 2015

For the sake of this discussion,it is assumed that you have some working knowledge of the Linux operating systems (at least as a user). Needless to say you should have a working bone to follow this tutorial.

BeagleBone Black

BeagleBone Black (BBB), is a popular Single Board Computer (SBC) which was released as a successor to the BeagleBone or BeagleBone White. If you don’t have a BBB, just order one to dive into the world of Embedded Linux. I’m sure that BBB will occupy a special place in your electronics hardware inventory :-).

Why deploy a custom kernel?

Well, I know that this question will be itching your mind. Instead of using the pre-built image, why should we use this method of building our own image and RFS? The answer for this question is, you have to do this in order to get some fun out of BeagleBone. Electronics is fun when you start doing things of your own and also you will learn a lot of things while doing this.

For starters, I would strongly recommend to use the pre-built image for working with BeagleBone. But, as I already stated, this post is for the intermediate level users of BBB. This will be cool when you do this and I’m sure this will guide you into the real world of Embedded Linux.

Tools needed to get started

For building linux kernel you will need several tools other than BeagleBone. The tools which are required is listed below:

• A PC/Laptop with a flavor of Linux (Preferably Ubuntu 14.04)
• Linux kernel source for BeagleBone Black
• ARM cross compiler
• U-Boot(optional)
• mkimage

Installing ARM cross compiler toolchain

The first and foremost thing in compiling kernel is installing ARM gcc cross compiler. Since, BeagleBone Black is based on AM335x Sitara Processor, we need to compile the kernel for that environment. There are numerous compilers available online for free but it is important to install a stable one for proper compilation. For instance gcc-arm-linux-gnueabihf compiler available in standard Ubuntu package is an unstable one. So, download a stable compiler. The preferred one is Linaro cross compiler.

You can download the compiler here

After downloading, extract the compiler using the following command.

$ sudo tar xvf gcc-linaro-arm-linux-gnueabihf-4.8-2014.04_linux.tar.xz -C /opt/

The compiler will be extracted to /opt/ directory. opt is nothing but the optional directory. Next, step is to add the compiler to the PATH variable, in order to direct the shell to find our compiler.

Go to /opt/ directory and change the directory name for adaptivity. Then, add the compiler to PATH variable.

$ cd /opt/
$ sudo mv gcc-linaro-arm-linux-gnueabihf-4.8-2014.04_linux/ gcc-arm-linux
$ export PATH=$PATH:/opt/gcc-arm-linux/bin

After installing the compiler you can verify it using the following command,

$ arm-linux-gnueabihf-gcc --version
arm-linux-gnueabihf-gcc (crosstool-NG linaro-1.13.1-4.8-2014.04 - Linaro GCC 4.8-2014.04) 4.8.3 20140401 (prerelease)

U-boot

U-boot is an open source universal bootloader for Linux systems. It supports features like TFTP, DHCP, NFS etc… In order to boot the kernel, a valid kernel image (uImage) is required.

It is not possible to explain u-boot here, as it is beyond the scope of this post. So, we will see how to produce a bootable image using U-boot. You can either follow the below steps to compile your own U-boot image or you could just get a prebuilt image.

Clone U-boot (Optional)

Clone u-boot using the following command

$ git clone git://git.denx.de/u-boot.git u-boot/

Compile U-boot (Optional)

Before, compiling U-Boot we need to configure it. Thanks to the availability of configuration files in the configs/ directory under u-boot. We can configure using the am332x_boneblack_defconfig file. All the configuration will be written to .config file located in u-boot/ directory. By default you will not be able to view the .config file. To view give ls -a command.

$ cd u-boot
$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- am335x_boneblack_defconfig

After configuring, u-boot can be cross compiled using the following command.

$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf-

It will take around 10 to 15 minutes depending on the system configuration. Mine is Pentium dual core processor and it took 10 minutes for compilation. After successful compilation, several files will be produced in u-boot/ directory. Our prime concern is MLO and u-boot.img files.

For now, we will not use the above mentioned files for booting. But, during later stages those will be needed.

Building a kernel for beaglebone black

Building a custom kernel for beaglebone black involves the following steps. Feel free to jump to the step you are currently in if you are not looking for a step-by-step guide.

Cloning the Kernel

After installing the compiler, clone the kernel source for BeagleBone Black from GitHub using

$ git clone https://github.com/beagleboard/linux.git

Go to the linux directory and ensure that you have cloned the correct repo by executing the following command

$ cd linux
$ git remote -v
origin https://github.com/beagleboard/linux.git (fetch)
origin https://github.com/beagleboard/linux.git (push)

Formatting the SD card

For deploying kernel from sd card, we need to format it and place the files accordingly. For this process, “Gparted” tool is needed. Install Gparted by the following command.

$ sudo apt-get install gparted

Insert your sd card by means of card reader and open Gparted. Select your sd card from the top right corner. it will be something like this, “/dev/sdb”

Note: Always use sd card of size greater than 2 GB. Although, 500 MB is more than enough for our task, having large free space will come handy at times.

Right click on the rectangular area showing your sd card name and select unmount as we need to unmount the existing partitions. Then, delete the existing partitions by again right clicking and selecting “delete”. This will delete all your files in sd card, so make sure you backed up any important files. We need two partitions in order to boot the kernel, one is for storing the bootable images and another one is for storing the minimal RFS(Root File System). Select new option by right clicking the partitions and provide the following details:

• New size: 50MBembed journal
• File System: FAT32
• Label: BOOT

Click Add button. Then, create another partition for storing RFS by entering the options below

• New Size: 1000MB
• File System: EXT3
• Label: RFS

Finally, click the green tick mark at the menu bar. The partition will be created and you can see two partitions created as BOOT and RFS.

Compile Linux kernel

Before compiling the kernel we need to configure it. It will be hard for the newbies. Once again, thanks to the kernel developers for providing all configurations in a single file. Go to the kernel directory and issue the following command.

$ sudo make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- bb.org_defconfig

This will write the configurations in file bb.org_defconfig to .config file.

Then compile the kernel.

$ sudo make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- uImage dtbs LOADADDR=0x80008000 -j4

The above command will compile the kernel using the arm cross compiler having the load address as 80008000. “j4” corresponds to the number of process to be run during the compilation. Give the value as twice that of your cpu core. Mine is dual core, so I it gave as 4.

After compiling you can find the image files in “arch/arm/boot/” directory. Copy “uImage” file from this directory and also “am335x-boneblack.dtb” file from “arch/arm/boot/dts/” directory to the BOOT partition.

Then, create a file named as “uEnv.txt” in BOOT partition and copy the following code to it.

console=ttyS0,115200n8
netargs=setenv bootargs console=ttyO0,115200n8 root=/dev/mmcblk0p2 ro rootfstype=ext4 rootwait debug earlyprintk mem=512M
netboot=echo Booting from microSD ...; setenv autoload no ; load mmc 0:1 ${loadaddr} uImage ; load mmc 0:1 ${fdtaddr} am335x-boneblack.dtb ; run netargs ; bootm ${loadaddr} - ${fdtaddr}
uenvcmd=run netboot

This will be the file in which uboot will look upon while booting. The instructions in this file will make the uboot to boot from our kernel.

After completing the above steps you can find the following files in BOOT partition of sd card.

• uImage
• am335x-boneblack.dtb
• uEnv.txt

Note: Make sure you have installed mkimage tool and mounted the sd card.

Root File System (RFS)

You can download RFS here.

Instead of downloading RFS, we can create our own custom RFS using BusyBox, which is an elaborate process and hence merits the need for a separate post. For the sake of simplicity, we can download and uncompress the RFS.

$ sudo tar -xvf rootfs.tar.xz -C /media/mani/RFS/
$ cd /media/mani/RFS/rootfs/
$ sudo mv ./* ../
$ cd ../
$ sudo rmdir rootfs

The above command will uncompress the tar file and will place it in the RFS partition of SD card. Just replace “mani” with your username in the above command.

Install Kernel Modules

We need modules for proper working of the kernel. So, install the kernel modules by the following command.

$ sudo make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- -j4 modules
$ sudo make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- INSTALL_MOD_PATH=/media/mani/RFS/ modules_install

That’s it. After completing the above steps, remove the SD card and place it in your BeagleBone. Connect the Bone to your PC via USB to serial converter and open the serial console using minicom in PC. (Set baud rate as 115200). After ensuring all things are correct, power on your BBB while holding the Boot switch (SW2). It will boot from your own custom kernel. Now you can cherish that you have created your own kernel image and deployed it in BeagleBone Black!!!

In my next post I will show you how to create custom RFS using BusyBox. As always, if you encountered any troubles on the way, just throw it in the comments, we will try to figure it out.