## Tools and generators

This section will describe how Boost.Build can be extended to support new tools.

For each additional tool, a Boost.Build object called generator must be created. That object has specific types of targets that it accepts and produces. Using that information, Boost.Build is able to automatically invoke the generator. For example, if you declare a generator that takes a target of the type `D` and produces a target of the type `OBJ`, when placing a file with extention `.d` in a list of sources will cause Boost.Build to invoke your generator, and then to link the resulting object file into an application. (Of course, this requires that you specify that the `.d` extension corresponds to the `D` type.)

Each generator should be an instance of a class derived from the `generator` class. In the simplest case, you don't need to create a derived class, but simply create an instance of the `generator` class. Let's review the example we've seen in the introduction.

    import generators ;
    generators.register-standard verbatim.inline-file : VERBATIM : CPP ;
    actions inline-file
    {
        "./inline-file.py" $(<) $(>)
    }

We declare a standard generator, specifying its id, the source type and the target type. When invoked, the generator will create a target of type `CPP` with a source target of type `VERBATIM` as the only source. But what command will be used to actually generate the file? In bjam, actions are specified using named "actions" blocks and the name of the action block should be specified when creating targets. By convention, generators use the same name of the action block as their own id. So, in above example, the "inline-file" actions block will be used to convert the source into the target.

There are two primary kinds of generators: standard and composing, which are registered with the `generators.register-standard` and the `generators.register-composing` rules, respectively. For example:

    generators.register-standard verbatim.inline-file : VERBATIM : CPP ;
    generators.register-composing mex.mex : CPP LIB : MEX ;

Standard generators take a single source of type `VERBATIM` and produces a result. The second generator takes any number of sources, which can have either the `CPP` or the `LIB` type. Composing generators are typically used for generating top-level target type. For example, the first generator invoked when building an `exe` target is a composing generator corresponding to the proper linker.

You should also know about two specific functions for registering generators: `generators.register-c-compiler` and `generators.register-linker`. The first sets up header dependecy scanning for C files, and the seconds handles various complexities like searched libraries. For that reason, you should always use those functions when adding support for compilers and linkers.

(Need a note about UNIX)

### Custom generator classes

The standard generators allows you to specify source and target types, an action, and a set of flags. If you need anything more complex, you need to create a new generator class with your own logic. Then, you have to create an instance of that class and register it. Here's an example how you can create your own generator class:

    class custom-generator : generator
    {
        rule __init__ ( * : * )
        {
            generator.__init__ $(1) : $(2) : $(3) : $(4) : $(5) : $(6) : $(7) : $(8) : $(9) ;
        }

    }

    generators.register
      [ new custom-generator verbatim.inline-file : VERBATIM : CPP ] ;

This generator will work exactly like the `verbatim.inline-file` generator we've defined above, but it's possible to customize the behaviour by overriding methods of the `generator` class.

There are two methods of interest. The `run` method is responsible for the overall process - it takes a number of source targets, converts them to the right types, and creates the result. The `generated-targets` method is called when all sources are converted to the right types to actually create the result.

The `generated-target` method can be overridden when you want to add additional properties to the generated targets or use additional sources. For a real-life example, suppose you have a program analysis tool that should be given a name of executable and the list of all sources. Naturally, you don't want to list all source files manually. Here's how the `generated-targets` method can find the list of sources automatically:

    class itrace-generator : generator {
    ....
        rule generated-targets ( sources + : property-set : project name ? )
        {
            local leaves ;
            local temp = [ virtual-target.traverse $(sources[1]) : : include-sources ] ;
            for local t in $(temp)
            {
                if ! [ $(t).action ]
                {
                    leaves += $(t) ;
                }
            }
            return [ generator.generated-targets $(sources) $(leafs)
              : $(property-set) : $(project) $(name) ] ;
        }
    }
    generators.register [ new itrace-generator nm.itrace : EXE : ITRACE ] ;

The `generated-targets` method will be called with a single source target of type `EXE`. The call to `virtual-target.traverse` will return all targets the executable depends on, and we further find files that are not produced from anything. The found targets are added to the sources.

The `run` method can be overriden to completely customize the way the generator works. In particular, the conversion of sources to the desired types can be completely customized. Here's another real example. Tests for the Boost Python library usually consist of two parts: a Python program and a C++ file. The C++ file is compiled to Python extension that is loaded by the Python program. But in the likely case that both files have the same name, the created Python extension must be renamed. Otherwise, the Python program will import itself, not the extension. Here's how it can be done:

    rule run ( project name ? : property-set : sources * )
    {
        local python ;
        for local s in $(sources)
        {
            if [ $(s).type ] = PY
            {
                python = $(s) ;
            }
        }

        local libs ;
        for local s in $(sources)
        {
            if [ type.is-derived [ $(s).type ] LIB ]
            {
                libs += $(s) ;
            }
        }

        local new-sources ;
        for local s in $(sources)
        {
            if [ type.is-derived [ $(s).type ] CPP ]
            {
                local name = [ $(s).name ] ;    # get the target's basename
                if $(name) = [ $(python).name ]
                {
                    name = $(name)_ext ;        # rename the target
                }
                new-sources += [ generators.construct $(project) $(name) :
                  PYTHON_EXTENSION : $(property-set) : $(s) $(libs) ] ;
            }
        }

        result = [ construct-result $(python) $(new-sources) : $(project) $(name)
                     : $(property-set) ] ;
    }

First, we separate all source into python files, libraries and C++ sources. For each C++ source we create a separate Python extension by calling `generators.construct` and passing the C++ source and the libraries. At this point, we also change the extension's name, if necessary.