Tuesday, February 2, 2016

Swift for Linux part 2 – Using C libraries with Swift

Swift developers coming from Apple’s iOS and OS X environments are use to using the Cocoa and Cocoa Touch frameworks however these frameworks are not available in the Linux environment.  When we develop Linux applications and utilities with Swift we need to use the system libraries provided by the Linux environment.  In this post will look at how we can use these system libraries with our Swift code to build useful applications and utilities. 

In this post we will look at the Glibc module that Apple provides for us which includes the majority of the Linux Standard Library.  We will also look at how we could create our own modules to add additional system libraries not included in the Glibc module.  To demonstrate the concepts discussed we will conclude this post by building a simple command line utility that will list the IP addresses of the device that it is run on.

Modules


A module in Swift is code that is distributed as a single unit that can then be imported into other modules using Swift’s import keyword.  Frameworks and applications are examples of modules in Swift. In this post we are going to be looking at a special kind of module that can be used to map system libraries so we can import and use them into our Swift code. 

The Linux port of Swift comes with a predefined module named Glibc that contains most of the Linux standard library however there are numerous headers that have not been imported in it.  This module is similar to the Darwin module on Apple platforms.  Lets start off by looking at tthe Glibc module and then we will look at defining our own modules to map other system libraries that we may need.

To see what headers are defined in the Glibc module view the module.map file located in the usr/lib/swift/glibc directory of your Swift installation.  Don’t worry if you do not fully understand the format of this file at this time, we will be looking at how to create module.map files later on in this post.  For right now, knowing that any header that is defined in this file will be included simply by importing the Glibc module is enough.

In my previous post, Swift for Linux part 1, we saw an example of how to use the Glibc module.  In that example we created an extension to the Array type that would randomly select an element from the array.  We used the random() function provided by the system to generate the random number in the extension.  Lets take a look at the code for this example again so we can see how it worked.  The following code shows the Array extension:

import Foundation
import Glibc

extension Array {
   
    func getRandomElement() -> Element {
           let index = Int(random() % self.count)
           return self[index]
    }
} 

In this example the code starts off with two import statements.  The first import statement imports the Foundation framework.  The Foundation framework defines the basic functionality that is needed for most applications.  Most if not all of your Swift source files will need to import the Foundation framework.

The second import statement imports the Glibc framework.  This import statement allows us to use the system libraries that are defined in the Glibc framework.  If we tried to build the code without importing the Glibc framework we would receive the following error.

/xxx/xxx/xxx/xxx/MakeFtpFile/Sources/ArrayExtension.swift:6:19: error: use of unresolved identifier 'random'                
let index = Int(random() % self.count)
                 ^~~~~~

What this error tells us is the compiler does not know anything about the random() function.  If we look at the man page for the random() function (command: man random) we see that we need to import the stdlib.h header if we want to use this function.  If we look at the headers that are imported in the Glibc framework we will see that the Glibc framework does include the stdlib.h header.  By importing the Glibc Framework we are essentially importing all of the header files defined within it therefore we are importing the stdlib.h header which defines the random() function.  This will allows us to use the random() function within our code.

If you are new to developing applications and utilities in the Linux environment you will want to get use to using the Linux man pages to retrieve information about the system libraries and the functionality they provide.  These man pages will give you a wealth of knowledge about the functions that you are using.

Earlier we mentioned that the Glibc framework contains most of the Linux standard library so what do we do if we want include libraries that are not in the Glibc framework?  These libraries could be part of the Linux standard library that are not currently defined in the Glibc framework or libraries that are not part of the Linux standard library itself.  Lets look at how to include these libraries and also how to use the functionality they provide in our code by creating a custom module.

Creating a custom module

To create a custom module we will begin by creating a directory to put the files for the module in.  This will be the module’s main directory.  Within this directory we will need two files.  The first is an empty file named Package.swift and the second is named module.modulemap. 

Within the module.modulemap file we will define the headers we want to import and the libraries that contain the functionality defined in the headers.  The example below shows the format of the module.modulemap file

module CMyModule [system] {
    header "/usr/include/mylibheader.h"
    link "mylib"
    export *
}

The first line defines the name for our module.  This name is what we will import in our Swift files.  In this sample the module’s name is CMyModule.  The next line defines the full path to the header file that we want to import.  The third line tells the compiler that the functionality defined in the header can be found in the mylib library so we will need to link it.  The last line says to export all of the functionality.

The Swift package manager uses git and git tags to manage packages and modules therefore once we create both files we will want to create a git repository for our module.  To do this we run the following commands in the main directory for the module.

git init
git add .
git commit -m "Initial Import"
git tag 0.1.0

Before we show how to use a module, lets go ahead and create the module needed for our example.

Creating the Cifaddrs module

In this post we will be creating a utility that will list the network addresses of the device it is running on.  For this utility we will use the getifaddrs() function.  The man page for the getiffaddrs() function shows that we will need to import sys/types.h and ifaddrs.h header files.  In addition to these two headers we will also need to import the netdb.h, sys/socket.h and arpa/inet.h headers for other functions that we will be using in our code.  

Since some of the headers that we need for our project are not defined in the Glibc framework we will create our own module so we can import them.  Lets begin by creating a directory name Cifaddrs and the two files that we need:

mkdir  Cifaddrs
cd Cifaddrs
touch Package.swift
touch module.modulemap

Now we will need to define the five headers in the module.modulemap file.  To do this we would put the following code into module.modulemap file.

module Cifaddrs [system] {
    module types {
        header "/usr/include/x86_64-linux-gnu/sys/types.h"
        export *
    }

    module ifaddrs {
        header "/usr/include/ifaddrs.h"
        export *
    }

    module Socket {
        header "/usr/include/x86_64-linux-gnu/sys/socket.h"
        export *
    }
   
    module inet {
        header "/usr/include/arpa/inet.h"
        export *
    }

    module netdb {
        header "/usr/include/netdb.h"
        export *
    }
}

Now we need to create our git repository by running the following commands in the module’s main directory.

git init
git add .
git commit -m "Initial Import"
git tag 0.1.0

Since the swift package manager uses the tag for versioning, you will want to update the tag whenever you update code.  To make it easier to create the initial structure for the module, I created a shell scripted named createmodule.sh.  This script is very similar to the createproject.sh that I created in my last post to create the structure for a project.  The following is the code for the createmodule.sh script.

#!/bin/bash
#title:        createmodule.sh
#author:       Jon Hoffman
#description:  Creates the directories and files need for a Swift module
#date:         012816
#version:      1.0
#usage:        createmodule.sh {module name} {optional: dir name}

MODULENAME=""
DIRNAME=""
PACKAGEFILENAME="Package.swift"
MODULEFILENAME="module.modulemap"

#Check to make sure at least one command
#line arg is present otherwise exit script
if [ $# -le 0 ]; then
    echo "Usage: creatproject {Name for Module} {Optional directory name}"
    exit 1
fi

#Assign the value of the first command line arg to the module name
#if a second command line arg is present assign that value to the
#the directory name otheerwise use the module name
MODULENAME=$1
if [ "$1" != "" ]; then
    DIRNAME=$1
else
    DIRNAME=$MODULENAME
fi

#Check to see if the directory exists and if so display an error
#and exit
if [ -d "$DIRNAME" ]; then
    echo "Directory already exists, please choose another name"
    exit 1
fi

#Make the directory structue and create the neccessary files
mkdir -p $DIRNAME

cd $DIRNAME
touch $PACKAGEFILENAME
touch $MODULEFILENAME

echo "module $MODULENAME [system] {" >> $MODULEFILENAME
echo "" >> $MODULEFILENAME
echo "}" >> $MODULEFILENAME

git init
git add .
git commit -m "Initial Import"
git tag 0.0.1

The createmodule.sh script takes one required and one optional command line argument.  The first (required) command line argument is the name of the module and is used to create the module.modulemap file.    The second (optional) command line argument is the name for the module directory.  If the second command line argument is not present then we use the name of the module (first command line argument) for the directory name.  The following examples show how we would use the createmodule.sh script to create a module named Clib in a directory named Clib.

./createmodule.sh Clib

The previous command would create a directory named Clib.  It would also create the Package.swift and module.modulemap files.  The following code shows what the newly created module.modulemap file would look like.

import PackageDescription 

module Clib [system] {

}

We used the module name (first command line argument) to define the module name in this module.modulemap file.  You can find the code for the createmodule.sh file on my Scripts for Swift Linux development github page.

Now that we have created our module, lets see how to use it in a project.

Using the Cifaddrs module


Now that we have our module created, lets look at how we would use it in a project.  The first thing we will want to do is to create the project.  For this I will use my createproject.sh script like this:
./createproject.sh getifaddrs
This will create the directory structure and files needed to start the project with.  To tell the compiler to use our newly created module we will need to add a dependency to the Package.swift file.  We would add this dependency as shown with the following code:
import PackageDescription

let package = Package(
    name:  "getifaddrs",
    dependencies: [.Package(url: "../Cifaddrs", majorVersion: 0, minor: 1)]
)

The url defines the path to the module.  This can be the full file system path as shown in our example or an Internet path to a github repository.  We can also define multiple dependencies by separating the packages by a comma as shown here:

let package = Package(
    name:  "getifaddrs",
    dependencies: [.Package(url: "../modOne", majorVersion: 0, minor: 1),
 dependencies: [.Package(url: "../modTwo", majorVersion: 0, minor: 1)]
)

We are now ready to use the libraries defined in the module within our application.  The following code shows the main.swift file that will retrieve the list of IP addresses and print them out.

import Foundation
import Cifaddrs

// Get list of all interfaces
var ifaddr : UnsafeMutablePointer<ifaddrs> = nil
if getifaddrs(&ifaddr) == 0 {

    // Loop through all interfaces
    var ptr = ifaddr
    while (ptr != nil) {
       
        // Get address and interface name
       var addr = ptr.memory.ifa_addr.memory
        var ifname = String.fromCString(ptr.memory.ifa_name)

        // If addr is IPv4 or IPv6
        if addr.sa_family == UInt16(AF_INET)
|| addr.sa_family == UInt16(AF_INET6) {

            // Convert interface address to a string and print it
            var ad = [CChar](count: Int(NI_MAXHOST),
repeatedValue: 0)      
            if (getnameinfo(&addr, socklen_t(32), &ad,
socklen_t(ad.count), nil, socklen_t(0), NI_NUMERICHOST) == 0) {
                   
                if let address = String.fromCString(ad) {
                      print("\(ifname):  \(address)")
                }
            }
        }

        ptr = ptr.memory.ifa_next
    }
    freeifaddrs(ifaddr)
}

Notice that in the second line we import the Cifaddrs module using the import keyword.   This essentially imports all of the headers that are defined in the Cifaddrs modeul.modulemap file.
The code is commented so you can see what is going on but I do want to point out a couple of items that will help you when it comes to use Linux system libraries with swift.  The first item is how to use C pointers.  The following code show how we would use the getifaddrs() function in normal C code:

struct ifaddrs *ifap;
getifaddrs (&ifap);

In this code we create a pointer to an ifaddrs structure.   In Swift we would write this same code like this:

var ifap : UnsafeMutablePointer<ifaddrs> = nil
if getifaddrs(&ifap)

Notice in Swift we use the UnsafeMutablePointer structure to declare a pointer to an object type in memory.  In this case the object type is the ifaddrs structure. 

The other item that I want to point is how we are accessing the information within the ifaddrs structure using pointers.  In C we would access the information like this:

ifa->ifa_addr

In Swift we access the information like this:

var addr = ifa.memory.ifa_addr.memory

It took me a little bit of time to wrap my brain around this.  Having a pretty good C background this line of code seemed just wrong to me but once I really wrapped my brain around the basic concepts here it really made since.  Basically in Swift we want to avoid using pointers if we can so there isn’t a simple interface to use them.  Notice in our Swift line of code we have ifa.memory.ifa_addr.memory.  The ifa.memory part of this line gets the value in memory that the ifa pointer is pointing too.  Then the second part of this line ifa_addr.memory gets the value in memory that the ifa_addr pointer is pointing too.

If we wanted to get the pointer rather than the actual value, we would use this line instead:

var addr = ifa.memory.ifa_addr

In this line the addr variable would contain an UnsafeMutablePointer.

It does take a little bit of work to include the Linux system libraries in our Swift code but overall I think Apple did a great job making it as easy as possible while avoid a lot of the complexity of C.  I will hopefully writing more posts on how to use the Swift port for Linux.



Saturday, January 30, 2016

Swift for Linux part 1 – Building Applications

At the end of last year Apple open sourced Swift and released a port for the Linux operating system.  At the time of the release I really wanted to try the Linux port of Swift however I was right in the middle of writing my new book on Protocol-Oriented programming so I was unable to really spend any time with it.  Now that I am finishing up the new book, I am able to spend some quality time with the Swift Linux port.  These next few posts will show what I have discovered.

In this first post we will look at several examples that will demonstrate how to write and build applications with the Swift port for Linux.  We will also create a shell scripts that we can use to create the directory structure and minimum files needed to use Swift’s package manager to build our applications.

We will not go over installing Swift on Linux because Apple has very good documentation on how to do this.  You can find the documentation on the Swift.org  

Using Swiftc

Once we install Swift on our system and set up the path we should be able to run the Swift compile.  To run the compiler we would use the swiftc command.  To see how we could use swiftc lets create a file named helloWorld.swift and put the following code in it.

import Foundation
print(“Hello World”)

Now run the command swiftc helloWorld.swift.  If all went well, we would have an application named helloWorld that we can run like this:  ./helloWorld.  This application will (obviously) print Hello World to the console. 

There are numerous options with the swiftc command and we can see them by using the –help option like this:  swiftc –help. 

Setting up the directory structure for an application

We could use the Swift command line compiler to compile our applications but if we had multiple files and/or dependencies, our compile command could get very complicated and hard to maintain.  Anyone that has used Make files or other similar utilities to build C projects can verify that building applications in this manner can get pretty complicated.

Apple has given us a much better approach for developing applications and modules.  This approach does require us to set up a specific directory structure and also a manifest file named Package.swift.
The following diagram shows how we would set up the directory structure and also the required files:

{project dir}
     |
     |---- Package.swift {file}
     |
     |------ Sources {directory}
                |
                |----main.swift {file}

This diagram shows that we have one subdirectory below our main project directory named Sources.  It also shows that we need two files.  The first file is the Package.swift file which is located in the main project directory and the main.swift file which is located in the Sources directory.

The Package.swift file is a manifest file that tells the compiler about our project and any dependencies that it may have.  At minimum we need to define a name for our project.  The following example shows the minimum manifest file that simply defines the name of the project which is HelloWorld.

import PackageDescription 

let package = Package(   
    name:  "HelloWorld"
)

The main.swift file is a special file that is the entry point for our application.  It is also the only file that is allowed to have top-level code in it.  Top-level code is code that is not encapsulated in a function or type.

To simplify the process of starting a project I created a shell script named createPoject.sh that will create the directory structure, manifest file and main swift file for a project.  The Package.swift and main.swift files are created with the minimum code needed for our project.  You can find this and other scripts that I use with my Swift Linux development on my Scripts for Linux development github page. The following shows the code for the createproject.sh script.

#!/bin/bash

PROJECTNAME=""
DIRNAME=""
PACKAGEFILENAME="Package.swift"
MAINFILENAME="main.swift"
SOURCESDIRNAME="Sources"

#Check to make sure at least one command
#line arg is present otherwise exit script
if [ $# -le 0 ]; then
    echo "Usage: creatproject {Name for project} {Optional directory name}"
    exit 1
fi

#Assign the value of the first command line arg to the project name
#if a second command line arg is present assign that value to the
#the directory name otheerwise use the project name
PROJECTNAME=$1
if [ "$1" != "" ]; then
    DIRNAME=$1
else
    DIRNAME=$PROJECTNAME
fi

#Check to see if the directory exists and if so display an error
#and exit
if [ -d "$DIRNAME" ]; then
    echo "Directory already exists, please choose another name"
    exit 1
fi

#Make the directory structure
mkdir -p $DIRNAME/$SOURCESDIRNAME

#Change to the project's directory and create the Package.swift file
cd $DIRNAME
touch $PACKAGEFILENAME

echo "import PackageDescription" >> $PACKAGEFILENAME
echo "" >> $PACKAGEFILENAME
echo "let package = Package(" >> $PACKAGEFILENAME
echo "    name:  \"$PROJECTNAME\"" >> $PACKAGEFILENAME
echo ")" >> $PACKAGEFILENAME

#Change to the Sources directory and create the main.swift file
cd $SOURCESDIRNAME
touch $MAINFILENAME

echo "import Foundation" >> $MAINFILENAME
echo ""  >> $MAINFILENAME
echo "print(\"Hello from Swift\")" >> $MAINFILENAME

#Done

The createPoject.sh script takes one required and one optional command line argument.  The first (required) command line argument is the name of the project and is used to create the Package.swift file.    The second (optional) command line argument is the name for the project directory.  If the second command line argument is not present then we use the name of the project (first command line argument) for the directory name.  The following examples show how we would use the createPoject.sh script to create a project named Hello in a directory named Hello.

./createproject Hello

The previous command would create a directory named Hello and also the Sources subdirectory.  It would also create the Package.swift and main.swift files.  The following code shows what the newly created Package.swift file would look like.

import PackageDescription 

let package = Package(    
     name:  "Hello"
)

We used the project name (first command line argument) to define the package name in this 
Package.swift file.  The newly created main.swift file would look like this.

import Foundation 
print("Hello from Swift")

As we mentioned earlier, the main.swift file is the entry point for our application therefore the code that is in this file is run when our application starts.

If we wanted the main project directory to have a different name from our project name then we would use the second command line argument. This next example shows how we would create a project named Hello in a directory named HelloDirectory.

./createproject Hello HelloDirectory

The createPoject.sh script actually creates a full project that can be built as is.  Lets see how we would build this new project.

Building a project

To build our project we would use the following command; swift build.  To see the options with this command we would use the --help option like this:  swift build --help.  Notice the two dashes, all of the other help options for the other swift commands (IE:  swiftc and swift commands) use the single dash.

Let build our project that we created in the last section.  Change to the project directory that was created by the createPoject.sh script and then run swift build.  If all went well you should see output similar to this:

Compiling Swift Module 'hello' (1 sources)
Linking Executable:  .build/debug/hello

If we see this output, we will have an executable named hello in the .build/debug directory.  You can execute it like this:  .build/debug/hello

Creating a project with multiple files

Recently I had the need to create a number of files that were of a specific size (I was testing sftp transfer speeds over different connection types).  I decided that this would be a good project to do in Swift.  The requirements that I had for this project was to create a number of files that contained random characters and were of specific sizes.

What I decided to do was to create an array that contained each character of the alphabet and then randomly select a character from the array until the file was the size I needed.  I also decided that I would create an extension to the array type which would randomly select an element from the array.  I created a file name ArrayExtension.swift in the Sources directory that contained the following code:

import Foundation
import Glibc


extension Array {
    func getRandomElement() -> Element {
          let index = Int(random() % self.count)
          
return self[index]
  
}
}

Don’t worry too much about how this code works at this time.  My next post will be about using C libraries with Swift and will explain more about modules and using C functions.  You can however see that generating a random number with the Swift port for Linux is a little different than with Swift for OS X or iOS.

Now in our main.swift file we can put the following code:

import Foundation
import Glibc

let num = 1024
var str = ""
let alpha = ["a","b","c","d","e","f","g","h","i","j","k",
"l","m","n","o","p","q","r","s","t","u","v","w","x","y","z"]


srandom(UInt32(NSDate().timeIntervalSince1970))

for i in 0..<num {   
str += alpha.getRandomElement()
}

print(str)
var filename = "test\(num)file.txt"

do {
   
try str.writeToFile(filename, atomically: true,encoding:
NSUTF8StringEncoding)
} catch let e {
   
print("Error: \(e)")
}

Once again, we are not worried about how this code works.  We are mainly focused on creating and building a project with multiple files.  Now we should have two files in our Sources directory named main.swift and ArrayExtension.swift.  Now if we go back to the main project directory (the one with the Package.swift file) we can run the swift build command and our project should compile to an executable. 

What we just saw is the swift build command will compile all of the files in the Sources directory and include them in our project.  This is a lot easier than creating complex Make files with C.  I would recommend that unless there is a specific requirement to use the swiftc command that you use the swift build command as we saw in this post.

Developers that are use to using Swift to build iOS and/or OS X applications are also use to using the Cocoa and Cocoa Touch frameworks however these frameworks are not present in the Linux environment.  Instead we need to use the C libraries that are provided with Linux.  In my next post I will show how to create modules that will expose those libraries so we can use them with our Swift applications.