Building MacPerl Extensions with SWIG

Contents

Introduction

Terminology

The Basics of SWIG

Obtaining and Installing SWIG for MacOS

Running SWIG

Building MacPerl Shared Libraries from SWIG Wrappers

Example One: Simple Datatypes and Functions

A Digression on What to Wrap, and How

Summary

References

Author

Legal Stuff

Introduction

A Perl extension is a combination shared library, with the source written in C and/or C++, and a Perl module. Special interface code exists in both parts of the extension, more so in the shared library.

The build process comprises the following steps:
  1. Writing the interface;
  2. Compiling the interface file into a C language source;
  3. Compiling and linking to create a shared library;
  4. Installation and testing.

There are two mechanisms for writing the interface - one is to use the XS language, and the other is to use SWIG (there is actually a third: writing the required C directly, but it's more work and I don't recommend it). SWIG stands for Simplified Wrapper Interface Generator; this particular tutorial is all about using SWIG with MacPerl.

This tutorial is a companion to the MacPerl XS Tutorial (www.macperl.com/depts/Tutorials/XS), which despite its resounding title actually focuses more on what happens after Step One in the list above, that is, compiling and linking, and more or less assumes the use of XS. Here we will concentrate on using SWIG - for details on what to do after that refer to the other tutorial.

It so happens that once you run SWIG, and produce C wrapper code, that the essential steps of building a shared library are the same as they for XS. Hence, apart from describing the use of the MPW swig tool, and the SWIG console app, most of this tutorial will be devoted to examples. All have been built and tested on MacOS using either the swig tool or the application, MPW or the Codewarrior IDE, and MacPerl. Enjoy.


Terminology

Shared Libraries
For shorthand, this tutorial will occasionally refer to shared libraries as DLL's (dynamically-loaded libraries).
module
A module will normally mean the entire set of files that makes up a proper CPAN module distribution. From time to time the word module may also just refer to the .pm file with the same basename.
extension
For our purposes here an extension module contains at least one source file with C code. Use of the terms extension and extension module is interchangeable.
XS files
Writing the C source is simplified if you prepare a file using XS macros, and run xsubpp on it to write a C source file. A reference to an .xs file means this initial file that must have xsubpp run on it. If you feel bold, you may always write the C file directly.
(C) function
The term function will be considered to refer specifically to a function in the unadulterated C source, differentiating it from a...
(SWIG) wrapper
which is the function wrapper prepared by SWIG, and contains the extra stuff required to make itself known to Perl, and allow the original function to be called with proper arguments, and return proper results.
(Perl) subroutine
A Perl function. This subroutine may include extra code, and therefore act as a wrapper in its own right. (By convention, a wrapper calls another routine, and its primary role is to condition input and output).

The Basics of SWIG

SWIG is a tool, available for UNIX, Windows and MacOS, which reads C source or header files, or preferably specially-prepared interface files, and generates C wrapper code. This wrapper code contains all the constructs required to identify C datatypes and functions (which are usually contained in an existing C library) to the target scripting language (I use the term scripting language under protest - as far as I'm concerned a programming language is or isn't - but everyone seems to use the term, so I will also). The target scripting languages include Perl, Tcl and Python, and there is also support for Guile, Eiffel and Java.

I made reference to "existing C libraries", and preparing interfaces for these is really what SWIG is designed for. If you have a situation where you've identified a particular bottleneck in Perl code, and want to write some C to address the problem, you're most likely better off writing the interface in XS.

Before we start, let us put things in perspective. We will assume that there exists some C code (which exists in a library or as source), that has one or more C header files describing the API (Application Programming Interface), and we want to use this library from Perl. The basic procedure involves writing a SWIG interface file, which describes exactly what we want the corresponding Perl API to include. We invoke SWIG on this interface file, using options which at a minimum tell SWIG to tailor our code for Perl, and the output is a C wrapper file.

This wrapper file is the exact counterpart to the C file that we get after running xsubpp on an XS file. In other words, with reference to the build procedures described in the MacPerl XS Tutorial, once we have one or the other, the subsequent steps are exactly the same.

Wrapper code prepared by SWIG is completely platform independent, although obviously the C library and the interface file may not be. In fact, SWIG itself is also written in very portable C, and the only reason separate MacOS versions exist is to handle the problem of input - how to specify options and arguments.


Obtaining and Installing SWIG for MacOS

The existing official port of SWIG for MacOS is called MacSWIG and is based on the 1.2p2 patchlevel. It has a Tcl interface, and most of it works. In order to bring SWIG support on MacOS up to the latest standard, and fix some MacOS bugs, the 1.1p5 patchlevel of SWIG has been built and is available both as a console application and also as an MPW tool. Both of these are packaged in the distribution which accompanies this tutorial.

If you decide to use these updated versions, be aware that you will not have the full SWIG distribution. You must also download the UNIX SWIG distribution from www.swig.org. This will supply a full set of documentation and the indispensable SWIG libraries.

Installation procedures for both the console app and the MPW tool are described in an included README file.


Running SWIG

There are no differences in functionality between the MPW tool and the console application. For MPW, as long as the swig tool is in one of the folders specified by the {Commands} environment variable, entering a command line exactly as you would for UNIX will work. Example:

swig -python -dlatex newpy.i

When running the console app, which may be anywhere, a console pops up. Ignore all the radio buttons; just type in the command line arguments and press OK. Do not type in swig. Example:

-perl5 -shadow -make_default -module SomeMod :examples:example1.i

Although SWIG has an option for specifying include directories, I recommend relying on the default search paths. SWIG must know the location of swig_lib: this folder contains standard library files. The only default which makes sense in the case of the console app is that the swig_lib folder is in the same folder as the app. There is no default which really makes sense for the MPW tool.

However, a user-defined default can be set up for both. For the console app, modify the SWIG Prefs file in the :System:Preferences: folder. It should contain a single line which specifies the full path for the swig_lib folder, e.g. HD:SWIG Folder:swig_lib.

The MPW tool looks for an environment variable named SWIG_LIB. You can either edit the User•SWIG Startup file in the MPW Startup folder, to set this environment variable there, or just set it in MPW prior to running SWIG.


Building MacPerl Shared Libraries from SWIG Wrappers

The main thing to remember is, that if you use SWIG, you cannot avail yourself of the

perl Makefile.PL
BuildProgram dynamic
BuildProgram install_dynamic

MPW build commands. These apply (at present) to XS builds only.

However, once you've run SWIG and produced the C wrapper, the build process is exactly the same as it would be for MPW (Apple or Codewarrior) or the Codewarrior IDE. Just substitute the wrapper file produced by SWIG for the C file produced by xsubpp. Please refer to either the examples in this tutorial, or the MacPerl XS tutorial.


Example One: Simple Datatypes and Functions

Before I direct you to an existing third-party library that we're going to use for a more complicated example, we are going to basically write our own C and then interface it.

I'm partial to mathematical and scientific stuff, because my background is physics and scientific programming (among other things), so we'll grab something interesting from Numerical Recipes In C. Create a file called graycode.c which contains:

#include "graycode.h"

unsigned long igray(unsigned long n, int is) {
    int ish;
    unsigned long ans, idiv;
    
    if (is >= 0)
	return n ^ (n >> 1);
	
    ish = 1;
    ans = n;
    for (;;) {
        ans ^= (idiv = ans >> ish);
        if (idiv <= 1 || ish == 16) return ans;
        ish <<= 1;
    }
}

Also write a little file called graycode.h which consists of:

unsigned long igray(unsigned long n, int is);

and finally, write the SWIG interface file, graycode.i:

%module Graycode
%{
#include "graycode.h"
%}

unsigned long igray(unsigned long n, int is);

As you can see, this interfacing business can be pretty straightforward. Now we invoke SWIG on the .i file. If using the console app, launch it, and enter the following:

-perl5 "full_path_to_interface_file:graycode.i"

and if using the MPW tool, Set Directory to where your .i file is, and just type

swig -perl5 graycode.i

In both cases SWIG chugs away (it's really fast) and with these options will spit out the following files:

What to do now? Well, refer to the MacPerl XS Tutorial for specifics on how to built with your development environment of choice. In any case, you need to compile and link both graycode_wrap.c and graycode.c; other than that everything is completely routine.

For example, using Codewarrior MPW and building for PPC, I'll do

Set MPSrc "Tallinn:MacPerl_Src"
Set MPWLibs "Tallinn:Metrowerks:CodeWarrior¶ MPW:MPW:Interfaces&Libraries"
    
MWCPPC -nosyspath -sym on -d MULTIPLICITY -w off -i- -i : ¶
    -i {MPSrc}:sfio:include: ¶
    -i {MPSrc}:GUSI:include: ¶
    -i {MPW}:Interfaces:MWCIncludes:" ¶
    -i {MPSrc}:perl: -i- -i : -w off -traceback -opt all -t -ext .PPC.o ¶
    :graycode.c :graycode_wrap.c -o :
    
MWLinkPPC -xm sharedlibrary -sym on -msg nodup -export boot_Graycode ¶
    -name Graycode -o Graycode.shlb.PPC ¶
    graycode.c.PPC.o graycode_wrap.c.PPC.o ¶
    "{MPW}:Libraries:MWPPCLibraries:MathLib" ¶
    {MPSrc}:perl:PerlStub ¶
    "{MPW}:Libraries:MWPPCLibraries:InterfaceLib" ¶
    "{MPW}:Libraries:MWPPCLibraries:MSL RuntimePPC.Lib" ¶
    "{MPW}:Libraries:MWPPCLibraries:MSL C.PPC.Lib"

followed by these commands to set up a local installation:

NewFolder lib
NewFolder :lib:MacPPC
NewFolder :lib:MacPPC:auto
NewFolder :lib:MacPPC:auto:Graycode
    
Duplicate -y Graycode.shlb.PPC :lib:MacPPC:auto:Graycode:Graycode
Duplicate -y Graycode.pm :lib:Graycode.pm

If I then prepare a little Perl script containing:

#!perl -w
    
use Graycode;
    
foreach $num (0..7) {
    my $gc = igray($num, 0);
    my $gcbin = dec2bin($gc);
    print "igray( $num ) = $gc : $gcbin\n";
}
    
# taken from page 48 of The Perl Cookbook
sub dec2bin {
    my $str = unpack("B32", pack("N", shift));
    $str =~ s/^0+(?=\d)//;
    $str;
}

running it results in:

perl -I ':lib' graycode.plx

igray( 0 ) = 0 : 0
igray( 1 ) = 1 : 1
igray( 2 ) = 3 : 11
igray( 3 ) = 2 : 10
igray( 4 ) = 6 : 110
igray( 5 ) = 7 : 111
igray( 6 ) = 5 : 101
igray( 7 ) = 4 : 100

which is as we expect - the values differ by one bit in the binary representation. Voilá!! Your first SWIG-assisted MacPerl extension.

Discussion

Pretty simple stuff, right? Sure, but imagine having a utility library containing fifty, or one hundred, functions like this. We'll assume for the moment that these functions are all using basic datatypes (and typedefs are fine, as long as you put them in the interface file). You can run SWIG right on the header file, or better, copy and paste the whole kit and caboodle into your .i file, add the few lines at the top to make sure the literal "#include" statement gets into the wrapper and to identify the module, and invoke SWIG right awy.

To be fair, the h2xs alternative (the standard Perl way) isn't much worse in these cases. The editing required to make the function declarations conform to XS standards, if you have to copy and paste, is removed entirely on a platform (e.g. UNIX) with a working C::Scan module, which is invoked by h2xs using the -x switch, and parses header files to produce an XS template containing the proper XSUB declarations already. But unless I miss my guess, we (us MacPerl types) don't have a functional C::Scan. So SWIG already looks more attractive.

Note: I should point out that there are MPW scripts which duplicate the functionality of h2xs -x. These are found in the :ext:Mac:ExtUtils folder in the MacPerl_Src distribution.


A Digression on What to Wrap, and How

The default SWIG behaviour is to pass basic C datatypes by value, and everything else by reference. "Everything else" includes arrays, structs, and so forth.

Furthermore, by default, the return value of the Perl subroutine will be the return value of the C function. If the desired return value happens to be an argument of the function, you won't get it back unless you take steps to "make it so", as Patrick Beardsley puts it. Or perhaps there are multiple return values.

Before wrapping a C library, then, consider the following:

  1. What C functions do I need? There is no point in converting hundreds of functions if you only need three of them, particularly if most of the ones you don't need are nasty;
  2. What inputs do I need to supply, and what output(s) do I want?
  3. What are the complex datatypes called for, if any?
  4. How will I return my outputs?

You can exercise a good deal of personal choice. There are no code police telling you how to use the various SWIG possibilities. As long as you can correctly return a desired output to Perl, don't worry too much about how you did it.

Using Object Methods

Let's look at an example. The function mktime in time.h has this prototype:

time_t mktime(struct tm *);

mktime() is a handy little function - it takes a C struct tm, loaded with some or all of the values you're familiar with from using Perl's localtime and gmtime, and returns the seconds since epoch. Furthermore, and this is a useful side-effect, it updates the weekday and yearday fields based on the others. This latter may in fact be exactly what you'd like to use mktime for. The same thing can be accomplished in Perl, but it's more tortuous.

So let's say that I need to declare struct tm's in a Perl script, with a view to having access to each field (get and set), being able to run mktime on a struct tm, and being able to make deep copies. Rather than mess with typemaps, I'll use object methods:

%module StructTM

%{
#include 
 %}
    
struct tm {
    int tm_sec;
    int tm_min;
    int tm_hour;
    int tm_mday;
    int tm_mon;
    int tm_year;
    int tm_wday;
    int tm_yday;
    int tm_isdst;
    %addmethods {
        tm() { return (struct tm *)malloc(sizeof(struct tm)); }

        ~tm() { free(self); }

        struct tm * mktime() {
            if (mktime(self) == -1) { return NULL; }
            else { return self; }
        }

        struct tm *copy() {
            struct tm* stm;
                
            tm1 = (struct tm *)malloc(sizeof(struct tm));
                
            stm->tm_sec = self->tm_sec; stm->tm_min = self->tm_min;
            stm->tm_hour = self->tm_hour; stm->tm_mday = self->tm_mday;
            stm->tm_mon = self->tm_mon; stm->tm_year = self->tm_year;
            stm->tm_wday = self->tm_wday; stm->tm_yday = self->tm_yday;
            stm->tm_isdst = self->tm_isdst;
                
            return stm;
        }
    }
};

I'll run SWIG on this with the -shadow option. This option generates an extra layer of code, both in the C interface and also in the Perl module, that allows Perl OO syntax to be used for accessor methods and so forth. In effect, we'll get Perl wrappers for our C wrappers.

The command line is:

swig -perl5 -shadow struct_tm.i

The resulting struct_tm_wrap.c file is compiled and linked into a shared library exactly as we did for the Graycode example.


As a result, I can instantly write and run the following Perl script:
#!perl -w
    
use StructTM;
    
$tm1 = tm->new();
    
$tm1->{'tm_mon'} = 6;
$tm1->{'tm_mday'} = 25;
$tm1->{'tm_year'} = 99;
$tm1->{'tm_sec'} = 0;
$tm1->{'tm_min'} = 0;
$tm1->{'tm_hour'} = 0;
    
$tm2 = $tm1->copy();
    
$tm2->{'tm_mon'} = 0;
$tm2->{'tm_mday'} = 13;
$tm2->{'tm_year'} = 98;
    
$tm3 = $tm1->mktime();
print "TM3: Yearday = ", $tm3->{'tm_yday'}, "\n";
print "TM3: Weekday = ", $tm3->{'tm_wday'}, "\n";
    
$tm4 = $tm2->mktime();
print "TM4: Yearday = ", $tm4->{'tm_yday'}, "\n";
print "TM4: Weekday = ", $tm4->{'tm_wday'}, "\n";

with output

TM3: Yearday = 205
TM3: Weekday = 0
TM4: Yearday = 12
TM4: Weekday = 2

and you see that both the deep copying and the mktime side-effect are working.

Using Helper Functions

Helper functions are wrappers of wrappers, loosely speaking. They more or less correspond to XSUB's that include code to condition input arguments and/or output. Helper functions are created using the SWIG %inline directive, and show up in the Perl API. (There are a number of SWIG directives for inserting code - %{ %}, %inline, %wrapper etc).

Helper functions are useful for customizing how an existing C function is used, for adding or restricting functionality, and for returning output in a form better suited to Perl.

In the following example we'll look at converting the header file Scrap.h. This is a small header, which is why I chose it, and it also happens to lend itself to demonstration of helper functions.

The portion of the header which we are concerned with boils down to

struct ScrapStuff {
    SInt32      scrapSize;
    Handle      scrapHandle;
    SInt16      scrapCount;
    SInt16      scrapState;
    StringPtr   scrapName;
};
typedef struct ScrapStuff   ScrapStuff;

typedef ScrapStuff *PScrapStuff;
typedef ScrapStuff *ScrapStuffPtr;

EXTERN_API( ScrapStuffPtr ) InfoScrap(void);
EXTERN_API( SInt32 ) UnloadScrap(void);
EXTERN_API( SInt32 ) LoadScrap(void);
EXTERN_API( SInt32 ) GetScrap(  Handle hDest,
                                ResType theType,
                                SInt32 *offset);
EXTERN_API( SInt32 ) ZeroScrap(void);
EXTERN_API( SInt32 ) PutScrap(  SInt32 length,
                                ResType theType,
                                const void * source);

Right off the bat we'll get rid of the EXTERN_API macros. For MacOS these translate to extern pascal - SWIG doesn't grok pascal, and we don't want to keep the extern keyword either (SWIG leaves the C declarations in if we do that, but we're already including the header anyhow).

We can extract the appropriate typedefs from MacTypes.h, and put those in. I also anticipate the fact that the ScrapStuff struct is read-only, so I'll put in the SWIG directives to enforce that.

At this point we've got an interface that looks something like:

%module Scrap
%{
#include
#include 
%}
    
typedef long SInt32;
typedef short SInt16;
typedef char **Handle;
typedef char *StringPtr;
typedef unsigned long ResType;
    
typedef struct ScrapStuff {
%readonly
    SInt32      scrapSize;
    Handle      scrapHandle;
    SInt16      scrapCount;
    SInt16      scrapState;
    StringPtr   scrapName;
%readwrite
} ScrapStuff;
    
typedef ScrapStuff *ScrapStuffPtr;

ScrapStuffPtr InfoScrap(void);
SInt32 UnloadScrap(void);
SInt32 LoadScrap(void);
SInt32 GetScrap(Handle hDest, ResType theType, SInt32 *offset);
SInt32 ZeroScrap(void);
SInt32 PutScrap(SInt32 length, ResType theType, const void * source);

A naïve approach is to immediately run SWIG on this. Well, what are the problems associated with doing this? First off, what do we actually want to do? I'm going to assume that the Perl script(s) in question are only dealing with scrap data of type 'TEXT', and so really the user wants to supply or receive Perl strings. No mucking around with handles.

Furthermore, InfoScrap() assigns to a pointer. It doesn't allocate that memory itself. But it's not the Perl way to write

$scrap_ptr = new ScrapStuff();
$scrap_ptr = InfoScrap();

We really would like to be able to just say

$scrap_ptr = InfoScrap();

Note: setting up an object method which ultimately calls InfoScrap() is more Perlish, and I suppose this would look more like

$scrap_ptr = new ScrapStuff();
$scrap_ptr->info();

but we already talked about object methods above, and so I won't use that approach here.

What we're leading to is setting up helper functions for InfoScrap(), GetScrap() and PutScrap(). When these are in place, the interface file will look like this:

%module Scrap
%{
#include
#include 
#include 
%}

typedef long SInt32;
typedef short SInt16;
typedef char **Handle;
typedef char *StringPtr;

typedef struct ScrapStuff {
%readonly
    SInt32      scrapSize;
    Handle      scrapHandle;
    SInt16      scrapCount;
    SInt16      scrapState;
    StringPtr   scrapName;
%readwrite
} ScrapStuff;

typedef ScrapStuff *ScrapStuffPtr;

%inline %{
ScrapStuffPtr InfoScrapH(void) {
    ScrapStuffPtr ssptr;

    ssptr = (ScrapStuffPtr)malloc(sizeof(ScrapStuff));
    ssptr = InfoScrap();
    return ssptr;
}

char *GetScrapText(void) {
    Handle h;
    SInt32 size, offset;
    char *text;

    if (GetScrap(NULL, 'TEXT', &offset) > 0) {
        h = NewHandle(0);
        HLock(h);
        size = GetScrap(h, 'TEXT', &offset);

        text = malloc(size+1);
        memcpy(text, *h, size);
        text[size] = '\0';

        HUnlock(h);
        DisposeHandle(h);
        return text;
    } else { return NULL; }
}

SInt32 PutScrapText(char *text) {
    return PutScrap((SInt32)strlen(text), 'TEXT', text);
}
%}

SInt32 UnloadScrap(void);
SInt32 LoadScrap(void);
SInt32 ZeroScrap(void);

Run SWIG with the usual command line:

swig -perl5 -shadow Scrap.i

and compile, link and install as described above.

This allows us to write a script like:

#!perl -w

use Scrap;

$scrap = Scrap::InfoScrapH();
print "Scrap Size = ", $scrap->{'scrapSize'}, "\n";
print "Scrap Handle = ", $scrap->{'scrapHandle'}, "\n";
print "Scrap Count = ", $scrap->{'scrapCount'}, "\n";
print "Scrap State = ", $scrap->{'scrapState'}, "\n";
print "Scrap Name = ", $scrap->{'scrapName'}, "\n";

$text = Scrap::GetScrapText();
print "Text = $text\n";

Scrap::ZeroScrap();
Scrap::PutScrapText("Just some routine garbage\n");

If you run this script, you'll see that it prints out whatever is in the Clipboard, along with scrap statistics, and if you choose Paste afterwards, the sentence "Just some routine garbage\n" will be inserted at the selection point.

Using Typemaps

Both XS and SWIG use typemaps, and the idea is exactly the same - to specify how Perl variables are translated to C variables, and vice versa. I recommend reading the sections in the SWIG Manual pertaining to typemaps, both the general discussion and the discussion specific to Perl5. Also, look at the file :swig_lib:perl5:typemaps.i to get an idea of existing typemaps which may be imported using the

%include typemaps.i

statement in an interface file. Finally, look at the examples in the Examples folder in the SWIG distribution which illustrate the use of typemaps. Note that typemaps, by their very nature, are language specific, so be careful to look at Perl5 examples.

Let's now turn our attention to C pointers. We'll assume the case of a header file pointers.h:

void modify(double *num);
void stats(double *data, double *min, double *max, double *avg);

and an accompanying source file pointers.c:

#include 
#include 
#include "pointers.h"

void modify(double *num) {
    *num = -*num;
}

void stats(double *data, double *min, double *max, double *avg) {
    int num = 0;

    *min = *max = *data;
    *avg = 0.0;
    while (!isnan(*data)) {
        *avg += *data;
        if (*data < *min)
            *min = *data;
        else if (*data > *max)
            *max = *data;
        data++;
        num++;
    }
    *avg /= num;
}
In this case I'll present the desired Perl code first:
#!perl -w

use Pointers;

$num = $ARGV[0];
print "modify( $num ) = ", Pointers::modify($num), "\n";

@nums = (
	1.1, -4.33, 11.8, 17.5, 0.06, -92.7, -15.0, 48.8
);

($min, $max, $avg) = Pointers::stats(\@nums);
print "MIN = $min, MAX = $max, AVG = $avg\n";

To handle the first function, modify(), I'm doing exactly what is suggested in the Perl5 library file typemaps.i. We're using the double *BOTH typemap, which allows the Perl value to be passed in as a number, and exit as a numerical return value, but is handed to the actual C function as a pointer.

The second function is being handled using a named typemap which is an adaptation of the SWIG Manual example for handling the C char ** datatype as a Perl array of strings. In that example, the array is terminated with a zero value. In my typemap we'll use a NaN value, which is defined in , and can be tested for using the isnan() function.

Furthermore, the three arguments for maximum, minimum and average of the input list of numbers are designated as output values only using the double *OUTPUT typemap. This already exists in the file typemaps.i. As such they are returned out the left side as a list.

%module Pointers
%{
#include 
#include 
#include "pointers.h"
%}

void modify(double *BOTH);

// This tells SWIG to treat 'double *data' as a special case
%typemap(perl5,in) double *data {
	AV *tempav;
	I32 len;
	int i;
	SV **tv;
	
	if (!SvROK($source))
		croak("$source is not a reference.");
	if (SvTYPE(SvRV($source)) != SVt_PVAV)
		croak("$source is not an array.");
		
	tempav = (AV*)SvRV($source);
	len = av_len(tempav);
	$target = (double *) malloc((len+1)*sizeof(double));
	for (i = 0; i <= len; i++) {
		tv = av_fetch(tempav, i, 0);
		$target[i] = (double) SvNV(*tv);
	}
	$target[i] = NAN;
}

// This cleans up our 'double *data' array after the function call
%typemap(perl5,freearg) double *data {
	free($source);
}

%apply double *OUTPUT { double *min, double *max, double *avg };
void stats(double *data, double *min, double *max, double *avg);
%clear double *min, double *max, double *avg;

Again, we compile, link and install in the usual manner.

Here Be Dragons

I'll conclude the examples by presenting the interface file for a portion of the DES library by Eric Young - version 4.0.1 to be exact. Caution: I am including this code neither to encourage you to go to such lengths in your typemap development, nor to recommend the use of SWIG for this particular case. More on this at the end of the section.

The instructive feature of this particular library is that most of the functions accept and return arrays, and pointers to arrays. The following two typedef's are of central importance:

typedef unsigned char des_cblock[8];
typedef struct des_ks_struct
	{
	union	{
		des_cblock _;
		/* make sure things are correct size on machines with
		 * 8 byte longs */
		DES_LONG pad[2];
		} ks;
#undef _
#define _	ks._
	} des_key_schedule[16];

DES_LONG is a #define for unsigned long, so in essence we are ensuring that a struct des_ks_struct is 64 bytes. The essential point is that a des_cblock is an array of 8 unsigned char's, and that a des_key_schedule is an array of 128 unsigned char's.

SWIG does not like typedefs of the form

typedef datatype name[num];

and hence the first approach is to use the two typedefs

typedef unsigned char *des_cblock;
typedef unsigned char *des_key_schedule;

To a first approximation these aren't bad, as generally speaking unsigned char * and unsigned char[] are interchangeable.

Let's examine the wrapper code produced by invoking SWIG on a very simple interface file, namely des3.i:

typedef unsigned char *des_cblock;
typedef unsigned char *des_key_schedule;

void somefunc(des_cblock *key, des_key_schedule dks);

The idea here is that key is an input argument, and dks is an output. Using the command line

swig -perl5 -module DES3 des3.i

produces a wrapper file des3_wrap.c, where the wrapper function corresponding to the C function somefunc() is

XS(_wrap_somefunc) {

    des_cblock * _arg0;
    des_key_schedule  _arg1;
    int argvi = 0;
    dXSARGS ;

    cv = cv;
    if ((items < 2) || (items > 2)) 
        croak("Usage: somefunc(key,dks);");
    if (SWIG_GetPtr(ST(0),(void **) &_arg0,"des_cblockPtr")) {
        croak("Type error in argument 1 of somefunc. Expected des_cblockPtr.");
        XSRETURN(1);
    }
    if (SWIG_GetPtr(ST(1),(void **) &_arg1,"des_key_schedule")) {
        croak("Type error in argument 2 of somefunc. Expected des_key_schedule.");
        XSRETURN(1);
    }
    somefunc(_arg0,_arg1);
    XSRETURN(argvi);
}

So far so good, because the variable declarations are correct. Keep in mind that the typedefs which count are the ones in the C header file des.h, and have been reproduced above. However, we need to tackle the following deficiencies:

  1. allocating storage for the des_cblock variable;
  2. ensuring that _arg1 is passed back out, and does not have to be provided as an input;
  3. deciding how a des_cblock will be constructed in Perl.

The last is easiest to solve. The pack() function with a "C*" template will suit our purposes admirably. After all, no matter what you call these variables they are basically arrays of unsigned char's.

The other points are addressed with typemaps. Rewriting the interface as:

%typemap(perl5,in) des_cblock *INPUT(des_cblock dcb)
{
	memcpy(dcb, SvPV($source, na), sizeof(des_cblock));
	$target = &dcb;
}

%typemap(perl5,in) des_key_schedule
{
	memcpy($target, SvPV($source, na), sizeof(des_key_schedule));
}

%typemap(perl5,ignore) des_key_schedule OUTPUT
{
}

%typemap(perl5,argout) des_key_schedule OUTPUT
{
	if (argvi >= items)
		EXTEND(sp,1);
	$target = sv_newmortal();
	sv_setpvn($target, $source, sizeof(des_key_schedule));
	argvi++;
}

typedef unsigned char *des_cblock;
typedef unsigned char *des_key_schedule;

void somefunc(des_cblock *INPUT, des_key_schedule OUTPUT);

and running SWIG with the same options produces the wrapper function

XS(_wrap_somefunc) {

    des_cblock * _arg0;
    des_key_schedule  _arg1;
    int argvi = 0;
    des_cblock  dcb;
    dXSARGS ;

    cv = cv;
{
}
    if ((items < 1) || (items > 1)) 
        croak("Usage: somefunc(INPUT);");
{
	memcpy(dcb, SvPV(ST(0), na), sizeof(des_cblock));
	_arg0 = &dcb;
}
    somefunc(_arg0,_arg1);
{
	if (argvi >= items)
		EXTEND(sp,1);
	ST(argvi) = sv_newmortal();
	sv_setpvn(ST(argvi), _arg1, sizeof(des_key_schedule));
	argvi++;
}
    XSRETURN(argvi);
}

Depending on what somefunc() does, after building the shared library, we could call this function from Perl as follows:

#!perl -w

use DES3;

@data = (0x00,0x01,0x02,0x03,0x04,0x05,0x06,0x07);
$data = pack("C8", @data);

$dks = DES3::somefunc($data);

I have built upon this example in the DES example, which is found in the tutorial distribution. In particular, there are also typemaps for handling des_cblocks as INOUT arguments and for handling arrays of des_cblocks. Enough is developed to illustrate the use of three separate encryption routines in the DES library.

One motivation for doing this particular example is to illustrate the technique, but more importantly, I wanted to reassure you that all of this stuff works perfectly well with the MacOS SWIG tool or app, and the resulting code builds and works with MacPerl. There is some array support in SWIG, but in general I would discourage the use of SWIG with libraries such as this one. In particular, if you decide to build this baby, and lay your hands on the DES distribution, take a look at the des_xcbc_encrypt() portion of the destest.c source file (incidentally, when you build the DES library, also build destest as an app - it works just fine); you'll see code like this:

des_xcbc_encrypt((C_Block *)cbc_data,(C_Block *)cbc_out,
	(long)strlen((char *)cbc_data)+1,ks,
	(C_Block *)iv3,
	(C_Block *)cbc2_key, (C_Block *)cbc3_key, DES_ENCRYPT);

This functionality is part of the example Perl script, and looks like this:

($cbc_out, $ivec) = DES::des_xcbc_encrypt(
	$cbc_data,40,$ks,$cbc_iv,$cbc2_key,$cbc3_key,1
);

The above allows me to present my rule of thumb for undertaking SWIG conversions:

Sandström's Rule Of Thumb for SWIG (When Not To) - if, when observing a typical usage for a C function, you cannot identify the fundamental datatype of each variable, and determine whether it is an IN, INOUT or OUT variable, and you cannot decide whether a *var is a pointer or an array, do not consider SWIGing the function.

In other words, if pointers are involved, find out if arrays are being used. If not, you can get by with supplied typemaps for pointers to primitive datatypes. Otherwise, give it a miss unless you feel comfortable with writing typemaps. Blithely running SWIG on function prototypes that you don't even know the typical usage for carries with it a medium to high risk of having bad pointers crash your application.

Also, consider using helper functions before you use typemaps. In the case of the DES library, this is contra-indicated: every function would need a helper. Usually, though, helper functions are the better option.


Summary

I also run SWIG 1.1p5 on MkLinux. However, everything in this tutorial has been developed and tested on MacOS (8.1 on a PM 6100/60, with the latest MacPerl) only. In other words, what I've really wanted to demonstrate here is that the full functionality of SWIG is available to MacPerl extension writers (MacTcl and MacPython extension writers, too, for that matter).

Ultimately there are no guidelines I can offer that will help you decide between using XS and using SWIG for any given task. I will suggest, at a minimum, that for a generic C library your first thought might be for SWIG, and that for writing code specifically to supplement Perl that you might first consider XS. And that's as far as I'll commit myself.

Ultimately you should try both, and develop facility with both.


References

  1. Mac XS Tutorial www.macperl.com/depts/Tutorials/XS
  2. MacPerl SWIG Tutorial distribution www.macperl.com/depts/Tutorials/MacSWIG
  3. Standard MacSWIG distribution: start at www.swig.org
  4. Standard UNIX SWIG distribution: start at www.swig.org

Author

Arved Sandström

Email: Arved_37@chebucto.ns.ca


Legal Stuff

This tutorial: Arved Sandström © 1999.

SWIG is written by David Beazley, and you should read the documents accompanying the full SWIG distribution, in order to ascertain your rights with respect to code produced using SWIG. I have also included the original Copyright and Readme.

The DES library is copyrighted by Eric Young. It is free, but if you use portions of the library in your code, you must attribute him properly. Please read the Copyright in the libdes distribution. This is easy enough to locate on the Web.