Today, we’re going to build our own toolchain using crosstool-NG . There are so many cases that we want to build a particular toolchain for a specific embedded system development board. One MOST important reason for that is probably because the specific development board has limited resource to build upon, therefore, the time-consumption to build some software on the development board is VERY inefficient . In the other way around, cross compiling on the host PC is MORE efficient with respective to the time-consumption.

In this blog, for simplicity, we take Raspberry Pi 3B as our demo development board, for which we are building the cross compiler . Some references can be found at:

1. Installation

How to install crosstool-NG are thoroughly summarized at its official website . In my case, I had it installed under folder /opt/compilers/crosstool-ng . Let’s take a look:

longervision-GT72-6QE% pwd
/opt/compilers/crosstool-ng
longervision-GT72-6QE% ls
bin  libexec  share
longervision-GT72-6QE% ls bin
ct-ng

And make sure /opt/compilers/crosstool-ng/bin is under environment variable PATH .

2. Configuration

Under any directory that you want to save your .config file, we can configure our target cross compiler .

1	longervision-GT72-6QE% ct-ng menuconfig

According to elinux: RPi Linaro GCC Compilation , we need to do the following selections:

Paths and misc options :

Try features marked as EXPERIMENTAL : ticked
Prefix directory : input the full path where you want to save the built toolchains
Number of parallel jobs : 4. Sorry that I’ve NO idea if this is the number of cores on Raspberry Pi 3B , but Raspberry Pi 3B does have 4 cores.

Target options :

Target Architecture : arm
Default instruction set mode : arm
Use EABI : ticked
append hf to the tuple (EXPERIMENTAL) : ticked
Endianness : Little endian
Bitness : 32-bit
Emit assembly for CPU : cortex-a53
Use specific FPU : vfp
Floating point : hardware (FPU)

Toolchain options :

Tuple’s vendor string : rpi

Operating System

Target OS : linux

Version of Linux : 4.20.8. Currently, I flashed Raspbian Buster with desktop and recommended software, 2019-07-10 onto my Raspberry Pi 3B , it comes with kernel of version 4.19.66 , which is NOT in the list. That’s why I decided to select 4.20.8 and try my luck.

1 2	➜ ~ uname -a Linux raspberrypi 4.19.66-v7+ #1253 SMP Thu Aug 15 11:49:46 BST 2019 armv7l GNU/Linux

Binary utilities :

Binary format : ELF

Version of binutils : 2.31.1

➜  ~ apt show binutils
Package: binutils
Version: 2.31.1-16+rpi1
Priority: optional
Section: devel
Maintainer: Matthias Klose <[email protected]>
Installed-Size: 95.2 kB
Provides: binutils-gold, elf-binutils
Depends: binutils-common (= 2.31.1-16+rpi1), libbinutils (= 2.31.1-16+rpi1), binutils-arm-linux-gnueabihf (= 2.31.1-16+rpi1)
Suggests: binutils-doc (>= 2.31.1-16+rpi1)
Conflicts: binutils-multiarch (<< 2.27-8), modutils (<< 2.4.19-1)
Homepage: https://www.gnu.org/software/binutils/
Download-Size: 56.9 kB
APT-Manual-Installed: no
APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
Description: GNU assembler, linker and binary utilities
The programs in this package are used to assemble, link and manipulate
binary and object files.  They may be used in conjunction with a compiler
and various libraries to build programs.

C-library :

C library : glibc

Version of glibc : 2.28

➜  ~ apt show libc-bin
Package: libc-bin
Version: 2.28-10+rpi1
Priority: required
Essential: yes
Section: libs
Source: glibc
Maintainer: GNU Libc Maintainers <[email protected]>
Installed-Size: 3,015 kB
Depends: libc6 (>> 2.28), libc6 (<< 2.29)
Recommends: manpages
Homepage: https://www.gnu.org/software/libc/libc.html
Download-Size: 657 kB
APT-Manual-Installed: yes
APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
Description: GNU C Library: Binaries
This package contains utility programs related to the GNU C Library.
.
* catchsegv: catch segmentation faults in programs
* getconf: query system configuration variables
* getent: get entries from administrative databases
* iconv, iconvconfig: convert between character encodings
* ldd, ldconfig: print/configure shared library dependencies
* locale, localedef: show/generate locale definitions
* tzselect, zdump, zic: select/dump/compile time zones

C compiler :

Show gcc versions from : GNU
Version of gcc : 8.3.0
gcc extra config : –with-float=hard
Link libstdc++ statically into the gcc binary : tick

C++ : tick

➜  ~ apt show gcc
Package: gcc
Version: 4:8.3.0-1+rpi2
Priority: optional
Section: devel
Source: gcc-defaults (1.181+rpi2)
Maintainer: Debian GCC Maintainers <[email protected]>
Installed-Size: 46.1 kB
Provides: c-compiler, gcc-arm-linux-gnueabihf (= 4:8.3.0-1+rpi2)
Depends: cpp (= 4:8.3.0-1+rpi2), gcc-8 (>= 8.3.0-1~)
Recommends: libc6-dev | libc-dev
Suggests: gcc-multilib, make, manpages-dev, autoconf, automake, libtool, flex, bison, gdb, gcc-doc
Conflicts: gcc-doc (<< 1:2.95.3)
Download-Size: 5,200 B
APT-Manual-Installed: no
APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages
Description: GNU C compiler
This is the GNU C compiler, a fairly portable optimizing compiler for C.
.
This is a dependency package providing the default GNU C compiler.

After we save the configuration to file .config , we Exit the crosstool-NG configuration dialog.

3. Build

Please remember to :


                    unset LD_LIBRARY_PATH

before building. Otherwise, you’ll meet some ERROR messages.


                    mkdir ~/src

before building. Otherwise, whenever you tried to rerun


                    ct-ng build

, you’ll have to download ALL required packages from scratch.

longervision-GT72-6QE% ct-ng build
[INFO ]  Performing some trivial sanity checks
[WARN ]  Number of open files 1024 may not be sufficient to build the toolchain; increasing to 2048
[INFO ]  Build started 20190906.014128
[INFO ]  Building environment variables
[EXTRA]  Preparing working directories
[EXTRA]  Installing user-supplied crosstool-NG configuration
[EXTRA]  =================================================================
[EXTRA]  Dumping internal crosstool-NG configuration
[EXTRA]    Building a toolchain for:
[EXTRA]      build  = x86_64-pc-linux-gnu
[EXTRA]      host   = x86_64-pc-linux-gnu
[EXTRA]      target = arm-rpi-linux-gnueabihf
[EXTRA]  Dumping internal crosstool-NG configuration: done in 0.07s (at 00:01)
[INFO ]  =================================================================
[INFO ]  Retrieving needed toolchain components' tarballs
[EXTRA]    Retrieving 'ncurses-6.1'
[EXTRA]    Verifying SHA512 checksum for 'ncurses-6.1.tar.gz'
[EXTRA]    Saving 'ncurses-6.1.tar.gz' to local storage
[EXTRA]    Retrieving 'libiconv-1.15'
[EXTRA]    Verifying SHA512 checksum for 'libiconv-1.15.tar.gz'
[EXTRA]    Saving 'libiconv-1.15.tar.gz' to local storage
[EXTRA]    Retrieving 'gettext-0.19.8.1'
[EXTRA]    Verifying SHA512 checksum for 'gettext-0.19.8.1.tar.xz'
[EXTRA]    Saving 'gettext-0.19.8.1.tar.xz' to local storage
[EXTRA]    Retrieving 'binutils-2.31.1'
[EXTRA]    Verifying SHA512 checksum for 'binutils-2.31.1.tar.xz'
[EXTRA]    Saving 'binutils-2.31.1.tar.xz' to local storage
[EXTRA]    Retrieving 'gcc-8.3.0'
[EXTRA]    Verifying SHA512 checksum for 'gcc-8.3.0.tar.xz'
[EXTRA]    Saving 'gcc-8.3.0.tar.xz' to local storage
[EXTRA]    Retrieving 'glibc-2.28'
[EXTRA]    Verifying SHA512 checksum for 'glibc-2.28.tar.xz'
[EXTRA]    Saving 'glibc-2.28.tar.xz' to local storage
[INFO ]  Retrieving needed toolchain components' tarballs: done in 130.35s (at 02:12)
[INFO ]  =================================================================
[INFO ]  Extracting and patching toolchain components
[EXTRA]    Extracting linux-4.20.8
[EXTRA]    Patching linux-4.20.8
[EXTRA]    Extracting zlib-1.2.11
[EXTRA]    Patching zlib-1.2.11
[EXTRA]    Extracting gmp-6.1.2
[EXTRA]    Patching gmp-6.1.2
[EXTRA]    Extracting mpfr-4.0.2
[EXTRA]    Patching mpfr-4.0.2
[EXTRA]    Extracting isl-0.20
[EXTRA]    Patching isl-0.20
[EXTRA]    Extracting mpc-1.1.0
[EXTRA]    Patching mpc-1.1.0
[EXTRA]    Extracting ncurses-6.1
[EXTRA]    Patching ncurses-6.1
[EXTRA]    Extracting libiconv-1.15
[EXTRA]    Patching libiconv-1.15
[EXTRA]    Extracting gettext-0.19.8.1
[EXTRA]    Patching gettext-0.19.8.1
[EXTRA]    Extracting binutils-2.31.1
[EXTRA]    Patching binutils-2.31.1
[EXTRA]    Extracting gcc-8.3.0
[EXTRA]    Patching gcc-8.3.0
[EXTRA]    Extracting glibc-2.28
[EXTRA]    Patching glibc-2.28
[INFO ]  Extracting and patching toolchain components: done in 33.39s (at 02:45)
[INFO ]  =================================================================
[INFO ]  Installing ncurses for build
[EXTRA]    Configuring ncurses
[EXTRA]    Building ncurses
[EXTRA]    Installing ncurses
[INFO ]  Installing ncurses for build: done in 14.62s (at 03:00)
[INFO ]  =================================================================
[INFO ]  Installing zlib for host
[EXTRA]    Configuring zlib
[EXTRA]    Building zlib
[EXTRA]    Installing zlib
[INFO ]  Installing zlib for host: done in 1.12s (at 03:01)
[INFO ]  =================================================================
[INFO ]  Installing GMP for host
[EXTRA]    Configuring GMP
[EXTRA]    Building GMP
[EXTRA]    Installing GMP
[INFO ]  Installing GMP for host: done in 30.12s (at 03:31)
[INFO ]  =================================================================
[INFO ]  Installing MPFR for host
[EXTRA]    Configuring MPFR
[EXTRA]    Building MPFR
[EXTRA]    Installing MPFR
[INFO ]  Installing MPFR for host: done in 17.35s (at 03:48)
[INFO ]  =================================================================
[INFO ]  Installing ISL for host
[EXTRA]    Configuring ISL
[EXTRA]    Building ISL
[EXTRA]    Installing ISL
[INFO ]  Installing ISL for host: done in 17.73s (at 04:06)
[INFO ]  =================================================================
[INFO ]  Installing MPC for host
[EXTRA]    Configuring MPC
[EXTRA]    Building MPC
[EXTRA]    Installing MPC
[INFO ]  Installing MPC for host: done in 5.47s (at 04:12)
[INFO ]  =================================================================
[INFO ]  Installing ncurses for host
[EXTRA]    Configuring ncurses
[EXTRA]    Building ncurses
[EXTRA]    Installing ncurses
[INFO ]  Installing ncurses for host: done in 13.64s (at 04:25)
[INFO ]  =================================================================
[INFO ]  Installing libiconv for host
[EXTRA]    Skipping (included in GNU C library)
[INFO ]  Installing libiconv for host: done in 0.01s (at 04:25)
[INFO ]  =================================================================
[INFO ]  Installing gettext for host
[EXTRA]    Skipping (included in GNU C library)
[INFO ]  Installing gettext for host: done in 0.01s (at 04:25)
[INFO ]  =================================================================
[INFO ]  Installing binutils for host
[EXTRA]    Configuring binutils
[EXTRA]    Building binutils
[EXTRA]    Installing binutils
[INFO ]  Installing binutils for host: done in 46.89s (at 05:12)
[INFO ]  =================================================================
[INFO ]  Installing pass-1 core C gcc compiler
[EXTRA]    Configuring core C gcc compiler
[EXTRA]    Building gcc
[EXTRA]    Installing gcc
[EXTRA]    Housekeeping for core gcc compiler
[EXTRA]       '' --> lib (gcc)   lib (os)
[INFO ]  Installing pass-1 core C gcc compiler: done in 285.36s (at 09:58)
[INFO ]  =================================================================
[INFO ]  Installing kernel headers
[EXTRA]    Installing kernel headers
[EXTRA]    Checking installed headers
[INFO ]  Installing kernel headers: done in 6.47s (at 10:04)
[INFO ]  =================================================================
[INFO ]  Installing C library headers & start files
[INFO ]    =================================================================
[INFO ]    Building for multilib 1/1: ''
[EXTRA]      Configuring C library
[EXTRA]      Installing C library headers
[EXTRA]      Installing C library start files
[INFO ]    Building for multilib 1/1: '': done in 7.65s (at 10:12)
[INFO ]  Installing C library headers & start files: done in 7.72s (at 10:12)
[INFO ]  =================================================================
[INFO ]  Installing pass-2 core C gcc compiler
[EXTRA]    Configuring core C gcc compiler
[EXTRA]    Building gcc
[EXTRA]    Installing gcc
[EXTRA]    Housekeeping for core gcc compiler
[EXTRA]       '' --> lib (gcc)   lib (os)
[INFO ]  Installing pass-2 core C gcc compiler: done in 354.87s (at 16:07)
[INFO ]  =================================================================
[INFO ]  Installing C library
[INFO ]    =================================================================
[INFO ]    Building for multilib 1/1: ''
[EXTRA]      Cleaning up start files
[EXTRA]      Configuring C library
[EXTRA]      Building C library
[EXTRA]      Configuring C library
[EXTRA]      Building C library
[EXTRA]      Installing C library
[INFO ]    Building for multilib 1/1: '': done in 184.69s (at 19:12)
[INFO ]  Installing C library: done in 184.78s (at 19:12)
[INFO ]  =================================================================
[INFO ]  Installing final gcc compiler
[EXTRA]    Configuring final gcc compiler
[EXTRA]    Building final gcc compiler
[EXTRA]    Installing final gcc compiler
[EXTRA]    Housekeeping for final gcc compiler
[EXTRA]       '' --> lib (gcc)   lib (os)
[INFO ]  Installing final gcc compiler: done in 339.13s (at 24:51)
[INFO ]  =================================================================
[INFO ]  Finalizing the toolchain's directory
[INFO ]    Stripping all toolchain executables
[EXTRA]    Installing the populate helper
[EXTRA]    Installing a cross-ldd helper
[EXTRA]    Creating toolchain aliases
[EXTRA]    Removing installed documentation
[EXTRA]    Collect license information from: ~/Crosscompile/.build/arm-rpi-linux-gnueabihf/src
[EXTRA]    Put the license information to: ~/....../RPi/share/licenses
[INFO ]  Finalizing the toolchain's directory: done in 1.80s (at 24:52)
[INFO ]  Build completed at 20190906.020619
[INFO ]  (elapsed: 24:51.86)
[INFO ]  Finishing installation (may take a few seconds)...
[24:52] / %                                                                                                                                                              longervision-GT72-6QE%

This process may take a while. I’m going to sleep tonight. Continue tomorrow…

Alright, let’s continue today. Glad to know it’s built successfully.

Let’s FIRST take a look at what’s under the current folder.

1 2	longervision-GT72-6QE% ls build.log rpi.config

And then, let’s take a look at what’s built under the specified destination folder.

longervision-GT72-6QE% ls ~/....../CrossCompile/RPi
arm-rpi-linux-gnueabihf  bin  build.log.bz2  include  lib  libexec  share
longervision-GT72-6QE% ls ~/....../CrossCompile/RPi/bin
arm-rpi-linux-gnueabihf-addr2line  arm-rpi-linux-gnueabihf-cpp           arm-rpi-linux-gnueabihf-gcc-ar      arm-rpi-linux-gnueabihf-gprof    arm-rpi-linux-gnueabihf-objdump   arm-rpi-linux-gnueabihf-strip
arm-rpi-linux-gnueabihf-ar         arm-rpi-linux-gnueabihf-ct-ng.config  arm-rpi-linux-gnueabihf-gcc-nm      arm-rpi-linux-gnueabihf-ld       arm-rpi-linux-gnueabihf-populate
arm-rpi-linux-gnueabihf-as         arm-rpi-linux-gnueabihf-elfedit       arm-rpi-linux-gnueabihf-gcc-ranlib  arm-rpi-linux-gnueabihf-ld.bfd   arm-rpi-linux-gnueabihf-ranlib
arm-rpi-linux-gnueabihf-c++        arm-rpi-linux-gnueabihf-g++           arm-rpi-linux-gnueabihf-gcov        arm-rpi-linux-gnueabihf-ldd      arm-rpi-linux-gnueabihf-readelf
arm-rpi-linux-gnueabihf-cc         arm-rpi-linux-gnueabihf-gcc           arm-rpi-linux-gnueabihf-gcov-dump   arm-rpi-linux-gnueabihf-nm       arm-rpi-linux-gnueabihf-size
arm-rpi-linux-gnueabihf-c++filt    arm-rpi-linux-gnueabihf-gcc-8.3.0     arm-rpi-linux-gnueabihf-gcov-tool   arm-rpi-linux-gnueabihf-objcopy  arm-rpi-linux-gnueabihf-strings
longervision-GT72-6QE%

Finally, let’s take a look at the version of our built cross compilers for Raspberry Pi 3B .

Additional issue: It seems current crosstool-NG does NOT officially support Python . Please refer to my issue at crosstool-NG .

4. Compile/Build a Package with Generated Toolchain

For most of the packages nowadays, they are

either supported by make : ./configure -> make -> make install
or supported by CMake : mkdir build -> cd build -> ccmake ../ -> make -> make install

How to use our generated toolchain to compile/build our target packages?

For the FIRST option, you can follow the crosstool-NG Using the toolchain .
For the SECOND option, you are welcome to follow CMake Toolchains .

Today, we’re going to take the package flann as an example, which are to be built with CMake .

4.1 CMakeLists.txt Modification

4.1.1 CMake Toolchains - Cross Compiling for Linux

By following CMake Toolchains - Cross Compiling for Linux , we FIRST modify flann CMakeLists.txt a bit by adding the following lines before project(flann) .

cmake_minimum_required(VERSION 2.6)

if(COMMAND cmake_policy)
    cmake_policy(SET CMP0003 NEW)
endif(COMMAND cmake_policy)

set(CMAKE_SYSTEM_NAME Linux)
set(CMAKE_SYSTEM_PROCESSOR arm)

set(CMAKE_SYSROOT ....../arm-rpi-linux-gnueabihf/sysroot)
set(CMAKE_STAGING_PREFIX ....../stage)

set(tools /opt/compilers/RPi)
set(CMAKE_C_COMPILER ${tools}/bin/arm-rpi-linux-gnueabihf-gcc)
set(CMAKE_CXX_COMPILER ${tools}/bin/arm-rpi-linux-gnueabihf-g++)

set(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE ONLY)

set(CMAKE_TRY_COMPILE_TARGET_TYPE "STATIC_LIBRARY")

project(flann)
string(TOLOWER ${PROJECT_NAME} PROJECT_NAME_LOWER)

CMAKE_SYSROOT specify the sysroot directory which emulate your target environment, here, Raspberry Pi 3B
CMAKE_STAGING_PREFIX is where the STAGING results store, for the reason that final results may require to be built in multiple stages. You may refer to Linux From Scratch for further background knowledge about that.
tools actually specify the building tool directory. Making sure all generated cross compiling tools are under folder ${tools}/bin .

4.1.2 Ignore hdf5

In addition, for the emulated Raspberry Pi 3B sysroot, hdf5 is NOT supported. Therefore, let’s simply comment out the following line in flann CMakeLists.txt .

1	#find_hdf5()

4.2 Cross Compile

Now, let’s start cross-compiling flann .

1
2
3

longervision-GT72-6QE% mkdir build
longervision-GT72-6QE% cd build
longervision-GT72-6QE% ccmake ../

and press c and then t , you’ll see the cross compiling toolchains have been automatically configured as:

BUILD_CUDA_LIB                  *OFF
BUILD_C_BINDINGS                *ON
BUILD_DOC                       *OFF
BUILD_EXAMPLES                  *OFF
BUILD_MATLAB_BINDINGS           *OFF
BUILD_PYTHON_BINDINGS           *ON
BUILD_TESTS                     *OFF
CMAKE_AR                        */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ar
CMAKE_BUILD_TYPE                *RelWithDebInfo
CMAKE_COLOR_MAKEFILE            *ON
CMAKE_CXX_COMPILER_AR           */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-gcc-ar
CMAKE_CXX_COMPILER_RANLIB       */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-gcc-ranlib
CMAKE_CXX_FLAGS                 *-I/usr/include
CMAKE_CXX_FLAGS_DEBUG           *-g
CMAKE_CXX_FLAGS_MINSIZEREL      *-Os -DNDEBUG
CMAKE_CXX_FLAGS_RELEASE         *-O3 -DNDEBUG
CMAKE_CXX_FLAGS_RELWITHDEBINFO  *-O2 -g -DNDEBUG
CMAKE_C_COMPILER_AR             */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-gcc-ar
CMAKE_C_COMPILER_RANLIB         */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-gcc-ranlib
CMAKE_C_FLAGS                   *-I/usr/include
CMAKE_C_FLAGS_DEBUG             *-g
CMAKE_C_FLAGS_MINSIZEREL        *-Os -DNDEBUG
CMAKE_C_FLAGS_RELEASE           *-O3 -DNDEBUG
CMAKE_C_FLAGS_RELWITHDEBINFO    *-O2 -g -DNDEBUG
CMAKE_EXE_LINKER_FLAGS          *
CMAKE_EXE_LINKER_FLAGS_DEBUG    *
CMAKE_EXE_LINKER_FLAGS_MINSIZE  *  
CMAKE_EXE_LINKER_FLAGS_RELEASE  *     
CMAKE_EXE_LINKER_FLAGS_RELWITH  *
CMAKE_EXPORT_COMPILE_COMMANDS   *OFF
CMAKE_INSTALL_PREFIX            */usr/local
CMAKE_LINKER                    */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ld
CMAKE_MAKE_PROGRAM              */usr/bin/make
CMAKE_MODULE_LINKER_FLAGS       *    
CMAKE_MODULE_LINKER_FLAGS_DEBU  *  
CMAKE_MODULE_LINKER_FLAGS_MINS  *   
CMAKE_MODULE_LINKER_FLAGS_RELE  *    
CMAKE_MODULE_LINKER_FLAGS_RELW  * 
CMAKE_NM                        */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-nm
CMAKE_OBJCOPY                   */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-objcopy
CMAKE_OBJDUMP                   */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-objdump
CMAKE_RANLIB                    */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ranlib
CMAKE_SHARED_LINKER_FLAGS       *    
CMAKE_SHARED_LINKER_FLAGS_DEBU  *
CMAKE_SHARED_LINKER_FLAGS_MINS  *   
CMAKE_SHARED_LINKER_FLAGS_RELE  *
CMAKE_SHARED_LINKER_FLAGS_RELW  *   
CMAKE_SKIP_INSTALL_RPATH        *OFF
CMAKE_SKIP_RPATH                *OFF
CMAKE_STATIC_LINKER_FLAGS       *
CMAKE_STATIC_LINKER_FLAGS_DEBU  *
CMAKE_STATIC_LINKER_FLAGS_MINS  * 
CMAKE_STATIC_LINKER_FLAGS_RELE  *
CMAKE_STATIC_LINKER_FLAGS_RELW  *
CMAKE_STRIP                     */opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-strip
CMAKE_VERBOSE_MAKEFILE          *ON
CPACK_SOURCE_RPM                *OFF
CPACK_SOURCE_TBZ2               *ON
CPACK_SOURCE_TGZ                *ON
CPACK_SOURCE_TXZ                *ON 
CPACK_SOURCE_TZ                 *ON
CPACK_SOURCE_ZIP                *OFF
DPKG_PROGRAM                    */usr/bin/dpkg
NSIS_PROGRAM                    */usr/bin/makensis
NVCC_COMPILER_BINDIR            *
PACKAGE_MAKER_PROGRAM           *PACKAGE_MAKER_PROGRAM-NOTFOUND
PKG_CONFIG_EXECUTABLE           */usr/bin/pkg-config
PYTHON_EXECUTABLE               */usr/bin/python
RPM_PROGRAM                     *RPM_PROGRAM-NOTFOUND
USE_MPI                         *OFF
USE_OPENMP                      *OFF

The LAST step before make is to modify some of the parameters accordingly, in my case:

BUILD_PYTHON_BINDINGS : ON -> OFF
CMAKE_CXX_FLAGS : -I/usr/include (for lz4.h from my host Ubuntu 19.04 )
CMAKE_C_FLAGS : -I/usr/include (for lz4.h from my host Ubuntu 19.04 )
CMAKE_VERBOSE_MAKEFILE : OFF -> ON
CMAKE_INSTALL_PREFIX : /usr/local/ -> under which directory you want to install, which can be IGNORED for now

Now, press c , you’ll see:

1 2	CMake Warning at CMakeLists.txt:99 (message): hdf5 library not found, some tests will not be run

A warining of missing hdf5 is clearly reasonable and acceptable. Then press g .

Finally, it’s the time for us to cross build flann .

longervision-GT72-6QE% make -j8
/usr/bin/cmake -S....../flann -B....../flann/build --check-build-system CMakeFiles/Makefile.cmake 0
/usr/bin/cmake -E cmake_progress_start ....../flann/build/CMakeFiles ....../flann/build/CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make[1]: Entering directory '....../flann/build'
make -f src/cpp/CMakeFiles/flann_s.dir/build.make src/cpp/CMakeFiles/flann_s.dir/depend
make -f src/cpp/CMakeFiles/flann_cpp_s.dir/build.make src/cpp/CMakeFiles/flann_cpp_s.dir/depend
make[2]: Entering directory '....../flann/build'
cd ....../flann/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" ....../flann ....../flann/src/cpp ....../flann/build ....../flann/build/src/cpp ....../flann/build/src/cpp/CMakeFiles/flann_s.dir/DependInfo.cmake --color=
make[2]: Entering directory '....../flann/build'
cd ....../flann/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" ....../flann ....../flann/src/cpp ....../flann/build ....../flann/build/src/cpp ....../flann/build/src/cpp/CMakeFiles/flann_cpp_s.dir/DependInfo.cmake --color=
Scanning dependencies of target flann_cpp_s
make[2]: Leaving directory '....../flann/build'
make -f src/cpp/CMakeFiles/flann_s.dir/build.make src/cpp/CMakeFiles/flann_s.dir/build
make[2]: Entering directory '....../flann/build'
make[2]: Leaving directory '....../flann/build'
make -f src/cpp/CMakeFiles/flann_cpp_s.dir/build.make src/cpp/CMakeFiles/flann_cpp_s.dir/build
make[2]: Entering directory '....../flann/build'
[ 12%] Building CXX object src/cpp/CMakeFiles/flann_s.dir/flann/flann.cpp.o
cd ....../flann/build/src/cpp && /opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot  -DFLANN_STATIC -D_FLANN_VERSION=1.9.1 -I....../flann/src/cpp  -I/usr/include -O2 -g -DNDEBUG   -Wall -Wno-unknown-pragmas -Wno-unused-function -fPIC -o CMakeFiles/flann_s.dir/flann/flann.cpp.o -c ....../flann/src/cpp/flann/flann.cpp
[ 25%] Building CXX object src/cpp/CMakeFiles/flann_cpp_s.dir/flann/flann_cpp.cpp.o
cd ....../flann/build/src/cpp && /opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot  -DFLANN_STATIC -DFLANN_USE_CUDA -D_FLANN_VERSION=1.9.1 -I....../flann/src/cpp  -I/usr/include -O2 -g -DNDEBUG   -Wall -Wno-unknown-pragmas -Wno-unused-function -fPIC -o CMakeFiles/flann_cpp_s.dir/flann/flann_cpp.cpp.o -c ....../flann/src/cpp/flann/flann_cpp.cpp
In file included from ....../flann/src/cpp/flann/util/params.h:33,
                 from ....../flann/src/cpp/flann/flann.hpp:42,
                 from ....../flann/src/cpp/flann/flann.h:603,
                 from ....../flann/src/cpp/flann/flann.cpp:31:
....../flann/src/cpp/flann/util/any.h: In member function 'virtual void flann::anyimpl::big_any_policy<T>::static_delete(void**)':
....../flann/src/cpp/flann/util/any.h:81:9: warning: this 'if' clause does not guard... [-Wmisleading-indentation]
         if (* x) delete (* reinterpret_cast<T**>(x)); *x = NULL;
         ^~
....../flann/src/cpp/flann/util/any.h:81:55: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the 'if'
         if (* x) delete (* reinterpret_cast<T**>(x)); *x = NULL;
                                                       ^
In file included from ....../flann/src/cpp/flann/util/params.h:33,
                 from ....../flann/src/cpp/flann/flann.hpp:42,
                 from ....../flann/src/cpp/flann/flann_cpp.cpp:30:
....../flann/src/cpp/flann/util/any.h: In member function 'virtual void flann::anyimpl::big_any_policy<T>::static_delete(void**)':
....../flann/src/cpp/flann/util/any.h:81:9: warning: this 'if' clause does not guard... [-Wmisleading-indentation]
         if (* x) delete (* reinterpret_cast<T**>(x)); *x = NULL;
         ^~
....../flann/src/cpp/flann/util/any.h:81:55: note: ...this statement, but the latter is misleadingly indented as if it were guarded by the 'if'
         if (* x) delete (* reinterpret_cast<T**>(x)); *x = NULL;
                                                       ^
[ 37%] Linking CXX static library ../../lib/libflann_cpp_s.a
cd ....../flann/build/src/cpp && /usr/bin/cmake -P CMakeFiles/flann_cpp_s.dir/cmake_clean_target.cmake
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_link_script CMakeFiles/flann_cpp_s.dir/link.txt --verbose=1
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ar qc ../../lib/libflann_cpp_s.a  CMakeFiles/flann_cpp_s.dir/flann/flann_cpp.cpp.o
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ranlib ../../lib/libflann_cpp_s.a
make[2]: Leaving directory '....../flann/build'
[ 37%] Built target flann_cpp_s
make -f src/cpp/CMakeFiles/flann_cpp.dir/build.make src/cpp/CMakeFiles/flann_cpp.dir/depend
make[2]: Entering directory '....../flann/build'
cd ....../flann/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" ....../flann ....../flann/src/cpp ....../flann/build ....../flann/build/src/cpp ....../flann/build/src/cpp/CMakeFiles/flann_cpp.dir/DependInfo.cmake --color=
Scanning dependencies of target flann_cpp
make[2]: Leaving directory '....../flann/build'
make -f src/cpp/CMakeFiles/flann_cpp.dir/build.make src/cpp/CMakeFiles/flann_cpp.dir/build
make[2]: Entering directory '....../flann/build'
[ 50%] Building CXX object src/cpp/CMakeFiles/flann_cpp.dir/empty.cpp.o
cd ....../flann/build/src/cpp && /opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot  -DFLANN_EXPORTS -D_FLANN_VERSION=1.9.1 -I....../flann/src/cpp  -I/usr/include -O2 -g -DNDEBUG -fPIC   -Wall -Wno-unknown-pragmas -Wno-unused-function -o CMakeFiles/flann_cpp.dir/empty.cpp.o -c ....../flann/src/cpp/empty.cpp
[ 62%] Linking CXX shared library ../../lib/libflann_cpp.so
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_link_script CMakeFiles/flann_cpp.dir/link.txt --verbose=1
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot -fPIC -I/usr/include -O2 -g -DNDEBUG  -shared -Wl,-soname,libflann_cpp.so.1.9 -o ../../lib/libflann_cpp.so.1.9.1 CMakeFiles/flann_cpp.dir/empty.cpp.o -Wl,-whole-archive ../../lib/libflann_cpp_s.a -Wl,-no-whole-archive 
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_symlink_library ../../lib/libflann_cpp.so.1.9.1 ../../lib/libflann_cpp.so.1.9 ../../lib/libflann_cpp.so
make[2]: Leaving directory '....../flann/build'
[ 62%] Built target flann_cpp
In file included from ....../flann/src/cpp/flann/algorithms/all_indices.h:35,
                 from ....../flann/src/cpp/flann/flann.hpp:45,
                 from ....../flann/src/cpp/flann/flann.h:603,
                 from ....../flann/src/cpp/flann/flann.cpp:31:

VERY Important:

If you build flann from source on a Raspberry Pi 3B , your system is going to hang HERE, possibly due to lack of memory..
Raspberry Pi 3B has 4 cores ONLY , but now you can use MORE cores on your host PC, which is clearly one advantage of using cross compiling.

Finally, you’ll see flann has been successfully cross compiled as:

....../flann/src/cpp/flann/algorithms/autotuned_index.h: In member function 'void flann::AutotunedIndex<Distance>::optimizeKMeans(std::vector<flann::AutotunedIndex<Distance>::CostData>&) [with Distance = flann::L2<double>]':
....../flann/src/cpp/flann/algorithms/autotuned_index.h:379:31: note: parameter passing for argument of type 'flann::L2<double>' changed in GCC 7.1
         KMeansIndex<Distance> kmeans(sampledDataset_, cost.params, distance_);
                               ^~~~~~
[ 75%] Linking CXX static library ../../lib/libflann_s.a
cd ....../flann/build/src/cpp && /usr/bin/cmake -P CMakeFiles/flann_s.dir/cmake_clean_target.cmake
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_link_script CMakeFiles/flann_s.dir/link.txt --verbose=1
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ar qc ../../lib/libflann_s.a  CMakeFiles/flann_s.dir/flann/flann.cpp.o
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-ranlib ../../lib/libflann_s.a
make[2]: Leaving directory '....../flann/build'
[ 75%] Built target flann_s
make -f src/cpp/CMakeFiles/flann.dir/build.make src/cpp/CMakeFiles/flann.dir/depend
make[2]: Entering directory '....../flann/build'
cd ....../flann/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" ....../flann ....../flann/src/cpp ....../flann/build ....../flann/build/src/cpp ....../flann/build/src/cpp/CMakeFiles/flann.dir/DependInfo.cmake --color=
Scanning dependencies of target flann
make[2]: Leaving directory '....../flann/build'
make -f src/cpp/CMakeFiles/flann.dir/build.make src/cpp/CMakeFiles/flann.dir/build
make[2]: Entering directory '....../flann/build'
[ 87%] Building CXX object src/cpp/CMakeFiles/flann.dir/empty.cpp.o
cd ....../flann/build/src/cpp && /opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot  -DFLANN_EXPORTS -D_FLANN_VERSION=1.9.1 -I....../flann/src/cpp  -I/usr/include -O2 -g -DNDEBUG -fPIC   -Wall -Wno-unknown-pragmas -Wno-unused-function -o CMakeFiles/flann.dir/empty.cpp.o -c ....../flann/src/cpp/empty.cpp
[100%] Linking CXX shared library ../../lib/libflann.so
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_link_script CMakeFiles/flann.dir/link.txt --verbose=1
/opt/compilers/RPi/bin/arm-rpi-linux-gnueabihf-g++ --sysroot=....../RPi/arm-rpi-linux-gnueabihf/sysroot -fPIC -I/usr/include -O2 -g -DNDEBUG  -shared -Wl,-soname,libflann.so.1.9 -o ../../lib/libflann.so.1.9.1 CMakeFiles/flann.dir/empty.cpp.o -Wl,-whole-archive ../../lib/libflann_s.a -Wl,-no-whole-archive -Wl,--push-state,--no-as-needed -Wl,--pop-state 
cd ....../flann/build/src/cpp && /usr/bin/cmake -E cmake_symlink_library ../../lib/libflann.so.1.9.1 ../../lib/libflann.so.1.9 ../../lib/libflann.so
make[2]: Leaving directory '....../flann/build'
[100%] Built target flann
make[1]: Leaving directory '....../flann/build'
/usr/bin/cmake -E cmake_progress_start ....../flann/build/CMakeFiles 0

You can now:

make install to install the built/generated libraries installed under CMAKE_INSTALL_PREFIX
copy and paste the built/generated libraries onto Raspberry Pi 3B and use it directly.

BTW : do NOT forget to install the header files.

5. Additional Issues

Multilib/multiarch seems to be problematic nowadays. Please pay attention to Multilib/multiarch . Some related issues are enumuated as the end of this blog.

Multilib caveats from Official Notes on specific toolchain features
fatal error: gnu/stubs-hard.h: No such file or directory from crosstool-NG
fatal error: gnu/stubs-hard.h: No such file or directory from gcc-cross-compiler
Build error: “features.h: No such file or directory” from risc-v tools
crosstool-NG risc-v linux multilib issue
raspberrypi toolchain issue

Alright, that’s all for today. Let me go to bed. Good bye…