Wed. May 25th, 2022

# Convolutional Neural-Network on Zynq –part 00: Convolution in Caffe

May 26, 2017

Recently, I have started to used FPGA (e.g. Zynq) to run neural-networks (NNs) defined in Caffe. My first step is performing the NN inference on FPGA. To do this and to be able to integrate the FPGA platform into Caffe, I have started to understand the Caffe C++ code and structure. Through a series of blogs, I am trying to explain my understanding using a few simple examples.

The goal of these blogs is not using the Caffe in efficient way to implement an application but get familiar with the Caffe code. Therefore, this blog is written for code developers not for application developers. In addition, I assume that reader is already familiar with the basic concepts in Caffe such as net, blob, layer and so on that can be found in its website.

In this first blog, I am going to define convolutional neural network (CNN). Although there are many books and articles explaining CNNs, theirÂ concepts and applications, here I am trying to keep everything simple just enough to be used in understanding the Caffe structure and how to add FPGA back-end for it.

Almost all articles explaining CNN start from neural-network (NN) concept, however,Â here I decided to start withÂ convolution. This approach helps people who do not have background knowledge of NN start having early real experiments.

### What is an image convolution?

First, what is aÂ convolution? In general, convolution is a binary operator which combines two inputÂ functions and generates a new function highlighting a feature in one of theÂ input function. Â The function whose features are going to be highlighted is called the main function and the second function is called the kernel.

In image processing, convolution is used to apply different filters on an image such as blurring,Â sharpen, edge detection and so on.

The following figure shows how to apply a kernel of size Â on anÂ input image of size .

### How to write a simple convolution in Caffe?

```#include &lt;caffe/caffe.hpp&gt;
#include &lt;opencv2/highgui/highgui.hpp&gt;
```

Step 01â€”Select CPU or GPU

```#ifdef CPU_ONLY
Caffe::set_mode(Caffe::CPU);
#else
Caffe::set_mode(Caffe::GPU);
#endif
```

Step 02: Define a network

```shared_ptr&lt;Net&lt;float&gt; &gt; net_;
```

Step 03: Load the network from a file

```net_.reset(new Net&lt;float&gt;(model_file, TEST));
```

Step 04: assign weights for the Sobel filter

```shared_ptr&lt;Layer&lt;float&gt; &gt; conv_layer = net_-&gt;layer_by_name(&quot;conv&quot;);
float* weights = conv_layer-&gt;blobs()[0]-&gt;mutable_cpu_data();

weights[0] = -1;Â Â Â Â  weights[1] = 0; Â Â Â Â  weights[2] = 1;
weights[3] = -2;Â Â Â Â  weights[4] = 0;Â  Â Â Â  weights[5] = 2;
weights[6] = -1;Â Â Â Â  weights[7] = 0; Â Â Â Â  weights[8] = 1;
```

Step 05: read the input image

```string image_file = argv[2];
```

Step 06: reshape the input blob to the size of the input image

```shared_ptr&lt;Blob&lt;float&gt; &gt; input_blob = net_-&gt;blob_by_name(&quot;data&quot;);
num_channels_ = input_blob-&gt;channels();
input_blob-&gt;Reshape(1, num_channels_, img.rows, img.cols);
```

Step 07: reshape the whole network correspondingly

```net_-&gt;Reshape();
```

Step 08: copy the input image to the network input blob

```int width = input_blob-&gt;width();
int height = input_blob-&gt;height();

float* input_data = input_blob-&gt;mutable_cpu_data();
cv::Mat channel(height, width, CV_32FC1, input_data);
img.convertTo(channel, CV_32FC1);
```

Step 09: run the NN inference

```net_-&gt;Forward();
```

Step 10: get the output and save in a file

```Blob&lt;float&gt;* output_layer = net_-&gt;output_blobs()[0];
int num_out_channels_ = output_layer-&gt;channels();

width = output_layer-&gt;width();
height = output_layer-&gt;height();
float* output_data = output_layer-&gt;mutable_cpu_data();
cv::Mat outputImage(height, width, CV_32FC1, output_data);
imwrite(&quot;outout_Image.jpg&quot;, outputImage);
```

If the input image is

Then the output would be

The complete code can be found atÂ here.