Published on June 8, 2023

Building a Convolutional Neural Network for Image Classification with Swift API for TensorFlow

Swift for TensorFlow (S4TF) is a high-level API developed by Google that allows you to use the TensorFlow machine learning framework in the Swift programming language. It provides a seamless integration of TensorFlow's powerful features while leveraging Swift's safety, expressiveness, and ease of use.

In this tutorial, we will guide you step-by-step through the process of building a Convolutional Neural Network (CNN) for image classification using the Swift API for TensorFlow. You will learn how to import necessary modules, load and preprocess image data, define the CNN architecture, create the model, train it, and evaluate its performance. Throughout the tutorial, we will provide code examples to help you understand and implement each step effectively.

Importing necessary modules and libraries:
- Import the TensorFlow module to access the Swift API for TensorFlow functionalities:
```
import TensorFlow
```

Loading and preprocessing image data:

Load the image data and preprocess it using techniques like resizing, normalizing pixel values, and converting images to tensors. For example:

let dataset = Dataset(...)
let resizedImages = dataset.images.resized(to: (224, 224))
let normalizedImages = resizedImages / 255.0
let tensorImages = Tensor<Float>(normalizedImages)
let labels = Tensor<Int32>(dataset.labels)
let trainDataset = Dataset(elements: (tensorImages, labels))

Defining the CNN architecture:

Define the structure of the CNN model using the building blocks provided by the Swift API for TensorFlow. Configure convolutional layers, pooling layers, and fully connected layers. For example:

struct CNN: Layer {
    var conv1 = Conv2D<Float>(filterShape: (3, 3, 3, 32), padding: .same, activation: relu)
    var conv2 = Conv2D<Float>(filterShape: (3, 3, 32, 64), padding: .same, activation: relu)
    var flatten = Flatten<Float>()
    var dense = Dense<Float>(inputSize: 7 * 7 * 64, outputSize: 10, activation: softmax)
 
    @differentiable
    func callAsFunction(_ input: Tensor<Float>) -> Tensor<Float> {
        let convolved1 = conv1(input)
        let convolved2 = conv2(convolved1)
        let flattened = flatten(convolved2)
        return dense(flattened)
    }
}

Creating the model:
- Create an instance of the CNN model using the defined architecture. Initialize the model, specify the optimizer, and choose an appropriate loss function for image classification. For example:
```
var model = CNN()
let optimizer = Adam(for: model)
let loss = softmaxCrossEntropy(logits:reduction:)
 
```

Training the model:

Train the CNN model using labeled training data. Perform forward and backward propagation, apply gradient descent optimization, and update the model's parameters. For example:

let epochs = 10
for epoch in 1...epochs {
    var epochLoss: Float = 0
    for batch in trainDataset.batched(batchSize) {
        let (images, labels) = (batch.first, batch.second)
        let gradients = gradient(at: model) { model -> Tensor<Float> in
            let logits = model(images)
            let batchLoss = loss(labels: labels, logits: logits)
            epochLoss += batchLoss.scalarized()
            return batchLoss
        }
        optimizer.update(&model.allDifferentiableVariables, along: gradients)
    }
    print("Epoch \(epoch): Loss: \(epochLoss)")
}

Evaluating the model:

Evaluate the performance of the trained model using a separate set of labeled test data. Calculate metrics like accuracy, precision, and recall to assess how well the model generalizes to new, unseen images.

For example:

let testImages = loadTestImages()
let testLabels = loadTestLabels()
let testTensorImages = Tensor<Float>(testImages)
let testTensorLabels = Tensor<Int32>(testLabels)
let testDataset = Dataset(elements: (testTensorImages, testTensorLabels))
 
var correctPredictions = 0
var totalPredictions = 0
for batch in testDataset.batched(batchSize) {
    let (images, labels) = (batch.first, batch.second)
    let logits = model(images)
    let predictions = logits.argmax(squeezingAxis: 1)
    correctPredictions += predictions .== labels
    totalPredictions += predictions.shape[0]
}
let accuracy = Float(correctPredictions) / Float(totalPredictions)
print("Test Accuracy: \(accuracy)")

By following this tutorial and implementing the provided code examples, you will gain hands-on experience in building a Convolutional Neural Network for image classification using the Swift API for TensorFlow. You will have a solid foundation to further explore and develop your own computer vision models in Swift.

See all posts