Adding new target platform and new tool chain

Check this page to remember the porting effort overview.

Back to Porting engineer
Back to SBuild manual home

• Step 1: Support in the SBuild C/C++ toolkit
  • New tgtplatform
  • New toolchain
  • New bldopt
  • Notes
• Step 2: Support as SCons Tools
  • Making a new SCons Tool, SCons Construction Environment
  • Automatic detection of the installed executables
• Step 3: Support in the product build description
  • Support in targets
  • Support in prebuilt externals (a.k.a. C/C++ SBuild SDKs)
  • Default platform support

Step 1: Support in the SBuild C/C++ toolkit

Let's first locate the C/C++ toolkit. It is stored in a subdirectory of the SBuild installation directory that is called tk_ccpp (it is a deep subdirectory, use file search by name).

The SBuild C/C++ toolkit stores the information about what to build for and what to build with in three global variants:

• tgtplatform = Variant modeling the target platform that is supposed to run the software that you build with this toolchain

• toolchain = Variant modeling the executables of your tool chain on your build-hosting platform

• bldopt = Variant modeling the degree of instrumentation and the degree of optimization of your software

You don't have to learn first what a SBuild variant is (by the way, the 3 above are the most complex variants, way more complex than the average variant). You just have to make addition to them and that may be just 1-2 lines of code. The relationship between these 3 variants is the following:

• One target platform has one tool chain building for it (called the preferred toolchain) and possibly more (for example, historical versions of the same toolchain: v2, v3, v4, etc.)

• One tool chain allows two or more build optimization levels. Most notably it accepts bldopt=dbg and bldopt=rls

First thing first, you have to choose names for your new target platform and your new toolchain. The name of the toolchain is usually the name of the provider/seller and the name of the platform is usually in the form <processor_operatingsystem>. These names will become new allowed values for the respective variants.

Step 1 requires you to make additions to the file ccpp_vars.py. This is the file that stores the Python descriptions of the 3 variants (location on your machine is probably <InstallDir>\scons-local-0.96.1\SCons\SBuild\tk_ccpp\ccpp_vars.py). In the future, we may decide to split this file in several different files (for several practical reasons).

Running any build script with argument explain=sum will tell you, among other things, which are the already supported platforms and tool chains (by listing the allowed values of all variants, including tgtplatform and toolchain).

Back to top

New tgtplatform value

You will notice that a variant has a number of attributes. Most of them are Python dictionaries keyed on the allowed values that a variant may take. For example, one attribute on the variant tgtplatform is descriptions and it is a dictionary containing a short, human readable text description for each platform name.

There are several attributes in tgtplatform variant that you need to look up and extend. Do not extend just yet, only take note. Important attributes in tgtplatform:

• supported_toolchains: The list of the allowed toolchains for this target platform (allowed values of the toolchain variant that may be used with a given value of tgtplatform). Most of the time there is only one; sometimes several just because of historical versions. The first one in the list is the preferred one.

• inhibatePDCvsPICcheck: a Boolean set to 1 if the distinction between position independent code and position dependent code is not relevant for this platform. More on that sibject in this set of frequently asked questions.

• scons_constr_var_to_set: a list of setting that SBuild prepares for SCons

You should be informed at this point that SCons is NOT using the shell environment where you start the build script when building something. It is using instead it's own environment named the SCons Construction Environment. This attribute, scons_constr_var_to_set, is used to enrich the construction environment of SCons with variables and variables values that you want to be specific to your new target platform. More on this in Step 2.

The good news is that you don't have to touch directly all attributes of a variant. Adding a new value to a variant can be achieved in two alternative ways:

• the long method: add a new key to all the dictionary attributes of the variant directly in the variant description

• the quick method: at the end of the variant description, clone one existing allowed value and only change what is different with respect to that base. This is very handy if the new platform is only slightly different from an existing one.

It is always a good idea to take a look first and check if you can find a similar platform already supported. Moreover, if that platform has been already cloned then, looking at that older clone, you already have a quick list of what you likely need to provide for your new platform.

Back to top

Important attributes in toolchain:

• scons_tools_to_set: The list of the SCons Tools to be used. Precisely, this list holds the file names of the Python files that implement the SCons Tools. It usually reads like the list of command line executables part of that tool chain (because of the convention to give to SCons Tool the same file name as the executable that it wraps).

• scons_constr_var_to_set: a list of setting that SBuild prepares for SCons

Note that the SCons tools have access to the SCons Construction Environment. Therefore, you may and you will put in this attribute, scons_constr_var_to_set, things that you want to "pass over" to your SCons Tools.

Back to top

After you changed toolchain and tgtplatform you should also change the variant bldopt. This variant contains command line options to make a debug build or a release build with any toolchain. It is not about introducing a new bldopt value, but about changing what happens for the already existing allowed values dbg and rls. The important attributes of bldopt are:

• ccflags_per_toolchain

• ccpp_defines_per_toolchain

• linkflags_per_toolchain

As the name says, these are dictionaries keyed with the current value of the toolchain variant. You can use the local function clone_bldopt() to "copy" the options of one toolchain to another toolchain in 1 line of code.

Note that you should NOT stuff up, for example, all options to the compiler in ccflags_per_toolchain just because you can put them there. Here only those options that are different between the debug and the release build have their place. Those that are common may be placed in the attribute ccflags of the variant toolchain (or of tgtplatform, if specific to a target platform).

Note also that the variant bldopt has more allowed values, not only dbg and rls:

• rld_dbginfo : used to make a build where the C code is optimized and the C debugger information is still provided in the compiled object files. This kind of build is instrumental, for example, to eliminate a program crash that doesn't show up in the dbg build

• rls_noopt : used to make a build that doesn't have the debugger info like the rls build but where the C code is not optimized. This is used to circumvent bugs in the compiler backend of older compilers (and frequently used in conjunction with the feature of SBuild that allows a special bldopt value for individual source files).

It is possible to introduce new bldopt allowed values for your toolchain (just provide them, the other toolchains will ignore them). It may be useful when you want to separate different kind of optimization (ex. for speed versus for space). But we don't recommend that path: multiplying the flavors also means multiplying the testing burden. Better decide here for only 1 kind of optimization and map that to rls, you may latter still customize it for individual products (in the SBuild project of that product).

Back to top

• The file ccpp_vars.py is supposed to be independent of any build machine. So you are not allowed to place file paths in it. That information belongs to the SCons Tools (see also Step 2 below).
• When you never cross-compile, you may have a harder time to discriminate what information belongs to the toolchain variant and what belongs to tgtplatform. It certainly becomes clearer as soon as you have one toolchain on your build machine that you have to use to build for several target platforms. Rest assured, if you never cross-compile and you do misplace settings between tgtplatform and toolchain you will not be affected in the end.
• As a side note, since SBuild is using the variants to compute output paths, your different ports of the same product will be available in separate directories on disk, ready to be compared.

Back to top

Step 2: Support in SCons

SCons knows how to run an individual external executable from a tool chain (SBuild doesn't know, SCons knows). That means SCons knows how to make up the command line to run that. SCons stores this information in a Python file per external executable. That file is called a SCons Tool (a.k.a. SCons wrapper)

How can you check what SCons Tools are already available? SCons stores all the Python Tool files in one directory: <InstallDir>\scons-local-0.96.1\SCons\Tool. You will find there all kind of tools, some concerning C/C++ compilations (Microsoft, Borland, GNU, Sun, IBM, SGI, HP, etc) as well as tools completely unrelated to C/C++ (Java, PDF compilation, Zip making, etc.). There are more than 60 tools, most of them not used yet by SBuild.

But if you are here it is likely that you will have to write new SCons tools. Don't panic, "new" tools may be not that new. New tools can reuse existing tools; they can amount in the end to just a few lines of code carrying the genuine new information. Take a look, for example, to how the SCons Tool named "suncc.py" is reusing the tool named "cc.py".

The one-to-one relation between SCons Tools and wrapped executable is not mandatory (there may be a Tool wrapping several executables or several Tools wrapping different functionality of the same executable). Know that no external executable at all is possible (just Python code that you provide). It only happens that, for C/C++ builds, you always wrap existing external executables.

Back to top

Making a new SCons tool, SCons Construction Environment

The structure of a SCons Tool is very simple. An SCons Tool is a Python script that is required to provide two file scope level functions

• generate(env)
• exists(env)

The function generate() is called during SCons initialization (if SBuild decides that it needs to start SCons, of course). It has only one in-out parameter: the so-called SCons Construction Environment. It's typically written as env, so this is how we will refer to it hereafter. Only one input parameter that is in fact a large namespace makes a quite loose contract between the SCons Tool and SCons, meaning a lot of freedom plus a lot of responsibility. The env is a dictionary and we call the entries in that dictionary construction variables.

Think of env as mainly an output parameter. The function generate() is supposed to add to the env some construction variables (and their values) specific to the executable that this SCons Tool is describing. Such a variable is, for example, the file name of the C compiler. More precisely, the function generate() for all the tools listed in the attribute scons_tools_to_set of the toolchain variant will be called in order to set up the SCons Construction Environment that will be used for most important actions, like building.

Here is how you set in the SCons Construction Environment the name of the compiler inside the generate(env) function:

        env['CC'] = "my_great_compiler"

        env.PrependENVPath('PATH', path_where_toolchain_executables_are_installed)

        env['CC'] = path_where_toolchain_executables_are_installed+os.sep+"my_great_compiler"

The important thing in both cases is that it is your code that will somehow compute this path_where_toolchain_executables_are_installed. That's one chief mission of the SCons Tool and it may well be that the code for finding the wrapped executable and providing then the related file paths becomes larger than the code for putting construction variables in the env. See the paragraph below on automatic detection.

The construction environment is also an input parameter in the generate() function. That means that inside the function you can test construction variables and act accordingly. These construction variables are set on the env either by SCons Tools that have been initialized before this one or by SBuild from the scons_constr_var_to_set attribute of the variants tgtplatform and toolchain. For example, you can have in scons_constr_var_to_set PROCESSOR_ARCHITECTURE set to "armv4i" and then use that in your SCons Tool to set a compiler argument to compile for that kind of processor or choose the name of the compiler that compiles for that kind of processor, etc.

Note that the construction environment of SCons can be manipulated also from other places, not only from the SCons tools. One place is the SCons build platform description (look in embedasr\build\scons\scons-local-0.96.1\SCons\Platform).

Note also that the construction variables listed in the scons_constr_var_to_set are set BOTH before and after the generate() function of the tools is called by SCons. This way, what you put in scons_vars_to_set can be used both to control things inside the SCons Tools generate() and to overwrite things after all generate() calls.

You have to know that the value you set for the construction variables can be computed. This computation uses the syntax of the Perl string interpolation. Here is an example:

        env['CCCOM']     = '$CC $CCFLAGS -c -o $TARGET $SOURCES'

The code above says that the compiler command line is made by expanding the construction variables CC, CCFLAGS, TARGET and SOURCES. If you wonder, know that the variables TARGET and SOURCES are automatically filled-in by SCons for each target. The construction variables CC and CCFLAGS will be provided most likely by your SCons tool. When working to get a command line right, don't forget that the -P option to a build script dumps all the command lines (without running any of them) for that build.

Know that the SCons reference manual gives an extensive list of construction environment variables with their purpose. The two SCons manuals (reference and user guide) are provided as HTML pages with your SBuild installation in <InstallDir>\scons-local-0.96.1\man. Know that you are also free to invent new variables in the Construction Environment (as long as not conflicting with documented ones, of course).

Suppose that your external executable, "my_great_compiler", is only slightly different from the ubiquitous Unix "cc" compiler. Than you will create an SCons tool named "my_great_compiler.py" that reuses the "cc.py" SCons tool (through a Python import statement). Here is your complete SCons tool:

import cc
def generate(env):
    "Add construction variables for  my_great_compiler"
    cc.generate(env)
    env['CCFLAGS']        = "my_special_compiler options"

Note also that the example above is purely theoretical because if this is the only difference than you don't need a new SCons tool. An addition to the ccflags attribute of the toolchain variant would be enough. 

Back to top

Adding code for the automatic detection of the installed executables

You probably want to release the developers from the chore of telling SBuild/SCons where did they put on their development machine the executables of the toolchain. This requires some code addition to your SCons tools. It can be as simple as adding one fixed, hard coded directory to env['ENV']['PATH'] (the path used at build time to search for executables) or it can be complex code that scans the Windows registry and/or all the hard disks inside out. It is up to you to choose how you support your developers

In an enterprise environment, it also makes sense to not detect anything and enforce instead one unique location for the compiler, linker, etc. across all computers of all developers in the team (a shared location on a file server or a same path location on all individual machines). This avoids that a "non-official" compiler is selected by mistake. That is usually done by manipulating construction variables in the file _sb_custom.py of _sb_users.py of an SBuild project (they are executed before the generate() of the SCons Tools). It is of course possible to combine policies (for some platforms enforce one single official toolchain while for other platforms leave it to whatever is installed on the developer's machine).

It is also possible to provide in your SCons Tool some interactive code. That will start a dialog with the user if all attempts to detect failed, asks the user for the path to the installation and then save it in a local file (so that next run doesn't fire the dialog anymore).

Whatever you choice for specifying the relevant locations of the installed tool chain, don't forget to assert that each directory exists on disk and provide adequate error message if not. It is understandable if, under terrible dead line pressure, you just hardcode in your SCons tool some absolute path. But is not allowed to skip the code that tells to the next user that the installation of the toolchain in that fixed location is a hard requirement of your SCons Tool. Here is an example:

root_dir = r"c:\Program Files\ARM\rvct"
assert os.path.isdir(root_dir), "We expect RVCT to be installed in: "+root_dir 

If you hardcode, don't forget to hardcode in only one place, by assigning to a Python string variable that you will use afterwards where you need the same path. That variable will hold the root directory of the installation (even if you don't need the root directly but some subdirectory of it).

Back to top

Step 3: Support in the product build description

This last step depends largely on what C/C++ product you want to port. Some may require no change in the build scripts, some may require a lot of changes. Ideally there is no change needed (meaning the port becomes a mere build) but that remains theoretical for real life products.

The task is to spot all the places in the build description that are customized per target platform. You need to make sure that in all those places, the build description does the right thing for your new target platform.  As an example of such place, let's say that executable A is linked with some system library to support audio output. This audio library is target platform specific and also target specific (you don't want any executable you build to link with that audio library). So, you will need to specify what to link with for your executable A for your new platform in the target specification of that executable.

If you miss one such place, don't worry. SBuild will detect the missing information and it will stop at that place reporting what attribute (target attribute or prebuilt external attribute) hasn't been configured for the current target platform. This feature may be disabled by using the so-called "default platform support" for your new platform, see below.

Back to top

As a reminder, the target specifications are those constructs in the build script introduced as sb.T(). The fact that the build scripts for one product are all in one directory comes in handy at this point. You can scan the build description for targets with attributes mentioning the platform (they all have a name like ***_per_platform, for example ccppdefines_per_platform) and you have a draft list of the places that you may need to touch to finish your porting.

To extend a target specification you have to change directly the target description and add an entry to all attributes with a name like *_per_platform. You have to add an entry even if you have nothing to do in particular for this target and this target platform. You then add an empty list, []. If a list is not present,  empty or not, SBuild will complain that this is an customization point that was not adapted to all supported target platforms. This mechanism is somewhat heavy but it is precise and clear. Note that empty lists may be added automatically by SBuild using the "default platform support" described below.

Back to top

Prebuilt externals or, with more words, SBuild externals of type "sdk", are parts of your product that, for some reason, you don't built from sources. They are explained in more detail on this page: How to use C/C++ SDKs.  As a reminder, the prebuilt externals are introduced as sb.E().

Since these prebuilt externals contain code that is already compiled for one target platform, you will obviously need to extend their specification with information about where to find the code compiled for the new target platform that you introduce now. You can search for "sb.E(" in all Python files in the SBuild project and quickly get the list of these specifications (although that will show also other SBuild externals, not only prebuilt externals). The sb.E() items are often present in just one file: _sb_vars.py.

To extend prebuilt externals to new values of tgtplatform, you may use the same "cloning" that you used for extending a variant. This way, assuming that your prebuilt doesn't need any single change for the new platform compared to an already existing platform, you only need 1 line of code: a call to the clone() method.

Back to top

You can instruct SBuild that, throughout this SBuild project, it should provide automatically something where customization per target platform is expected. This default platform support is of two kinds:

• empty platform support : This means that an empty list will be provided by SBuild in all places. This is achieved by just adding the tgtplatform value (as Python string) to some special list, _default_platform_support
• clone platform support : This means that some entry is copied from another tgtplatform. That other entry must exist. This is achieved by adding to some special list a pair of values, the new tgtplatform and the older tgtplatform to close, both as Python strings

Know that whatever the default platform provides, empty list or copied list, has lower priority than what you give explicitly in the target specification. This mechanism will change a target only if the entry is not present already. For prebuilt externals also what is provided by clone() method call takes precedence over the default platform support.

As a general note, the default platform support mechanism is sometimes the solution (for example when a new platform is very similar to an already supported one). But some other times is no real help: it shuts off all SBuild errors related to not-yet-ported targets  it and leaves you with link-time errors or, even worse, run-time errors. Therefore, it is always a good idea to review all places where customization per platform has been provided before. That suggests that your entry in the default platform support list shouldn't be the very first step of your port.

Back to top

In all build descriptions that are reused for several platforms, there is some danger that a change for one platform breaks the build for another platform. SBuild is no exception. It is good to know that the use of the default platform support can greatly reduce this risk.

Back to Porting engineer
Back to SBuild manual home