Build systems

Conan .gitignore for executable projects

This is the .gitignore I use for most of my conan-built executable projects:

CMakeCache.txt
CMakeFiles/
Makefile
bin/
cmake_install.cmake
conan.lock
conanbuildinfo.cmake
conanbuildinfo.txt
conaninfo.txt
graph_info.json

It is debatable whether you should include lock files such as conan.lock. My opinion is that in the development phase they should be ignored but reconsidered when entering into production service.

Posted by Uli Köhler in Conan, git

How to initialize new C++ executable project using conan

In the directory where you want to create the project, run (obviously, replace mypackageby the name of everywhere in our code!)

conan new mypackage/1.0.0 -s

-s is important here. It tells conan that the source is available in the project directory and it should not be pulled using git etc.

This will initialize a shared library project, which we will now modify to build an executable.

First, open src/CMakeLists.txt and change add_library() to add_executable(), for example

add_library(mypackage mypackage.cpp)

to

add_executable(mypackage mypackage.cpp)

Now we shall modify conanfile.py to properly build & install the executable:

    def package(self):
        self.copy("*.h", dst="include", src="src")
        self.copy("*.lib", dst="lib", keep_path=False)
        self.copy("*.dll", dst="bin", keep_path=False)
        self.copy("*.dylib*", dst="lib", keep_path=False)
        self.copy("*.so", dst="lib", keep_path=False)
        self.copy("*.a", dst="lib", keep_path=False)
    def package_info(self):
        self.cpp_info.libs = ["NoxecoDB"]

with

    def package(self):
        self.copy("mypackage", src="bin", dst="bin", keep_path=False)

    def package_info(self):
        self.env_info.PATH = os.path.join(self.package_folder, "bin")

    def deploy(self):
        self.copy("mypackage", dst="bin", src="bin")

and add

import os.path

to the top of the file

Now let’s add a main() function to  src/mypackage.cpp. Replace the file’s content with

#include <iostream>

int main(int argc, char** argv) {
    std::cout << "Hello World!" <<std::endl;
}

Now it’s time to install the dependencies & build the project using

conan install . && conan build .

The executable will be in

bin/mypackage

You can run it using

./bin/mypackage

which will print

Hello World!

I also recommend to add this .gitignore:

CMakeCache.txt
CMakeFiles/
Makefile
bin/
cmake_install.cmake
conan.lock
conanbuildinfo.cmake
conanbuildinfo.txt
conaninfo.txt
graph_info.json

 

Posted by Uli Köhler in C/C++, Conan

How to fix conan install ERROR: Missing binary

Problem:

When running

conan install .

you see  ERROR: Missing binary error messages like this:

Installing (downloading, building) binaries...
ERROR: Missing binary: cpr/1.6.2:76fe60a9d5c829a2384b0f578695b5a6357b8acc
ERROR: Missing binary: gtest/cci.20210126:c37dc23725d84483088a68c29e2dd7122c328359
ERROR: Missing binary: jsoncpp/1.9.4:718278bac9f92b77a9a44bfbe1aa00d1c8344d51
ERROR: Missing binary: libcurl/7.78.0:d287dc966931230205222ceabf47400733e72fa3
ERROR: Missing binary: openssl/1.1.1l:c3baf9fae083edda2e0c5ef3337b6a111016a898
ERROR: Missing binary: zlib/1.2.11:c3baf9fae083edda2e0c5ef3337b6a111016a898

Solution:

Use the --build=missing flag for conan install to enable building the dependencies for which no binaries are available:

conan install . --build=missing

Complete error log example

Configuration:
[settings]
arch=armv8
arch_build=armv8
build_type=Release
compiler=gcc
compiler.libcxx=libstdc++
compiler.version=9
os=Linux
os_build=Linux
[options]
[build_requires]
[env]

openssl/1.1.1l: Not found in local cache, looking in remotes...
openssl/1.1.1l: Trying with 'conancenter'...
Downloading conanmanifest.txt completed [0.26k]                                          
Downloading conanfile.py completed [40.47k]                                              
Downloading conan_export.tgz completed [0.25k]                                           
Decompressing conan_export.tgz completed [0.00k]                                         
openssl/1.1.1l: Downloaded recipe revision 0
cpr/1.6.2: Not found in local cache, looking in remotes...
cpr/1.6.2: Trying with 'conancenter'...
Downloading conanmanifest.txt completed [0.25k]                                          
Downloading conanfile.py completed [9.62k]                                               
Downloading conan_export.tgz completed [0.30k]                                           
Decompressing conan_export.tgz completed [0.00k]                                         
cpr/1.6.2: Downloaded recipe revision 0
WARN: cpr/1.6.2: requirement libcurl/7.80.0 overridden by elasticlient/0.2 to libcurl/7.78.0 
libcurl/7.78.0: Not found in local cache, looking in remotes...
libcurl/7.78.0: Trying with 'conancenter'...
Downloading conanmanifest.txt completed [0.70k]                                          
Downloading conanfile.py completed [27.11k]                                              
Downloading conan_export.tgz completed [0.23k]                                           
Decompressing conan_export.tgz completed [0.00k]                                         
libcurl/7.78.0: Downloaded recipe revision 0
jsoncpp/1.9.4: Not found in local cache, looking in remotes...
jsoncpp/1.9.4: Trying with 'conancenter'...
Downloading conanmanifest.txt completed [0.65k]                                          
Downloading conanfile.py completed [4.90k]                                               
Downloading conan_export.tgz completed [0.25k]                                           
Decompressing conan_export.tgz completed [0.00k]                                         
jsoncpp/1.9.4: Downloaded recipe revision 0
gtest/cci.20210126: Not found in local cache, looking in remotes...
gtest/cci.20210126: Trying with 'conancenter'...
Downloading conanmanifest.txt completed [0.39k]                                          
Downloading conanfile.py completed [8.14k]                                               
Downloading conan_export.tgz completed [0.32k]                                           
Decompressing conan_export.tgz completed [0.00k]                                         
gtest/cci.20210126: Downloaded recipe revision 0
conanfile.py (elasticlient/0.2): Installing package
Requirements
    cpr/1.6.2 from 'conancenter' - Downloaded
    jsoncpp/1.9.4 from 'conancenter' - Downloaded
    libcurl/7.78.0 from 'conancenter' - Downloaded
    openssl/1.1.1l from 'conancenter' - Downloaded
    zlib/1.2.11 from 'conancenter' - Cache
Packages
    cpr/1.6.2:76fe60a9d5c829a2384b0f578695b5a6357b8acc - Missing
    jsoncpp/1.9.4:718278bac9f92b77a9a44bfbe1aa00d1c8344d51 - Missing
    libcurl/7.78.0:d287dc966931230205222ceabf47400733e72fa3 - Missing
    openssl/1.1.1l:c3baf9fae083edda2e0c5ef3337b6a111016a898 - Missing
    zlib/1.2.11:c3baf9fae083edda2e0c5ef3337b6a111016a898 - Missing
Build requirements
    gtest/cci.20210126 from 'conancenter' - Downloaded
Build requirements packages
    gtest/cci.20210126:c37dc23725d84483088a68c29e2dd7122c328359 - Missing

Installing (downloading, building) binaries...
ERROR: Missing binary: cpr/1.6.2:76fe60a9d5c829a2384b0f578695b5a6357b8acc
ERROR: Missing binary: gtest/cci.20210126:c37dc23725d84483088a68c29e2dd7122c328359
ERROR: Missing binary: jsoncpp/1.9.4:718278bac9f92b77a9a44bfbe1aa00d1c8344d51
ERROR: Missing binary: libcurl/7.78.0:d287dc966931230205222ceabf47400733e72fa3
ERROR: Missing binary: openssl/1.1.1l:c3baf9fae083edda2e0c5ef3337b6a111016a898
ERROR: Missing binary: zlib/1.2.11:c3baf9fae083edda2e0c5ef3337b6a111016a898

 

Posted by Uli Köhler in Conan

How to create conan debug profile

Run this command to create a debug profile from your default profile:

cp ~/.conan/profiles/default ~/.conan/profiles/debug && sed -i -e 's/Release/Debug/g' ~/.conan/profiles/debug

Example output from conan profile show debug:

[settings]
os=Linux
os_build=Linux
arch=x86_64
arch_build=x86_64
compiler=gcc
compiler.version=9
compiler.libcxx=libstdc++
build_type=Debug
[options]
[conf]
[build_requires]
[env]

 

Posted by Uli Köhler in C/C++, Conan

How to list local conan packages

In order to list local conan packages, run

conan search

Example output:

$ conan search
Existing package recipes:

cpr/1.6.2
elasticlient/0.2
elasticlient/[email protected]/testing
gtest/cci.20210126
jsoncpp/1.9.4
libcurl/7.69.1
libcurl/7.78.0
openssl/1.1.1l
zlib/1.2.11

 

Posted by Uli Köhler in Conan

How to fix Conan CMake “Could NOT find OpenSSL”

Problem:

When building your conan package, you see this CMake error:

CMake Error at /usr/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:146 (message):
  Could NOT find OpenSSL, try to set the path to OpenSSL root folder in the
  system variable OPENSSL_ROOT_DIR (missing: OPENSSL_CRYPTO_LIBRARY
  OPENSSL_INCLUDE_DIR)
Call Stack (most recent call first):
  /usr/share/cmake-3.16/Modules/FindPackageHandleStandardArgs.cmake:393 (_FPHSA_FAILURE_MESSAGE)
  /usr/share/cmake-3.16/Modules/FindOpenSSL.cmake:447 (find_package_handle_standard_args)
  ../_deps/curl-src/CMakeLists.txt:365 (find_package)

even though your conanfile.py declares OpenSSL as a dependency:

class MyPackageConan(ConanFile):
    # ...
    requires = ("openssl/1.1.1l", )

Solution:

Make sure that the project’s CMakeLists.txt contains these lines:

include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup()

Typically, the way to do this is to use this code which is automatically generated by conan new:

        # This small hack might be useful to guarantee proper /MT /MD linkage
        # in MSVC if the packaged project doesn't have variables to set it
        # properly
        tools.replace_in_file("hello/CMakeLists.txt", "PROJECT(HelloWorld)",
                              '''PROJECT(HelloWorld)
include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup()''')

in source() like this:

    def source(self):
        self.run("git clone https://github.com/conan-io/hello.git")
        # This small hack might be useful to guarantee proper /MT /MD linkage
        # in MSVC if the packaged project doesn't have variables to set it
        # properly
        tools.replace_in_file("hello/CMakeLists.txt", "PROJECT(HelloWorld)",
                              '''PROJECT(HelloWorld)
include(${CMAKE_BINARY_DIR}/conanbuildinfo.cmake)
conan_basic_setup()''')

Note that you need to replace both hello/CMakeLists.txt with the correct filename (typically, just replace hello by). Also, you need to replace both instances PROJECT(HelloWorld) by the actual line from your CMakeLists.txt in order for the replace command to work.

Posted by Uli Köhler in C/C++, CMake, Conan

How to add build-only dependencies in Conan recipe

Just add build_requires = ('dep1/version1', 'dep1/version2') to your conanfile.py:

class MyPackageConan(ConanFile):
    # ...
    requires = ("openssl/1.1.1l", )
    build_requires = ("jsoncpp/1.9.4", )

 

Posted by Uli Köhler in C/C++, Conan

How to run ‘git submodule update’ in Conan recipe

If you want to run git submodule update in your conanfile.py, you can simple add a self.run() command after the git clone command:

self.run("git clone https://github.com/conan-io/hello.git")
self.run("cd hello && git submodule update --init --recursive")

Full example:

self.run("git clone https://github.com/seznam/elasticlient.git -b version-0.2")
self.run("cd elasticlient && git submodule update --init --recursive")

Note that you need to cd <directory> && git submodule update

Posted by Uli Köhler in C/C++, Conan

How to add conan CMake build definition

When building a library using conan, this is the default conanfile.py definition for building using CMake

cmake = CMake(self)
cmake.configure(source_folder="hello")
cmake.build()

You can easily add definitions like this:

cmake.definitions["MY_FLAG"] = "1"

This will generate -DMY_FLAG=1 when calling CMake

Full example:

cmake = CMake(self)
cmake.definitions["USE_SYSTEM_CPR"] = "1
cmake.configure(source_folder="elasticlient")"
cmake.build()

Note that you need to add all cmake.definitions calls before calling cmake.configure()

Posted by Uli Köhler in C/C++, Conan

How to fix CMake Protobuf “FindThreads only works if either C or CXX language is enabled”

Problem:

When trying to configure your CMake project using

find_package(Protobuf REQUIRED)

you see an error message like

CMake Error at /usr/share/cmake-3.16/Modules/FindThreads.cmake:49 (message):
  FindThreads only works if either C or CXX language is enabled
Call Stack (most recent call first):
  /usr/share/cmake-3.16/Modules/FindProtobuf.cmake:420 (find_package)
  CMakeLists.txt:7 (find_package)

-- Configuring incomplete, errors occurred!

Solution:

You need to put your

project(MyProject)

line before the

find_package(Protobuf REQUIRED)

line. If you don’t have a project() line, create one.

Posted by Uli Köhler in C/C++, CMake

How to compile boost unit test program using CMake

Add this to your CMakeLists.txt:

#
# Compile test suite
#
find_package(Boost COMPONENTS system filesystem unit_test_framework REQUIRED)

add_executable(test_myprogram
    tests/MyTest.cpp
)
target_include_directories(test_myprogram PUBLIC "${CMAKE_CURRENT_LIST_DIR}/include")
target_compile_features(test_myprogram PRIVATE cxx_std_17)

target_link_libraries(test_myprogram
                      ${Boost_FILESYSTEM_LIBRARY}
                      ${Boost_SYSTEM_LIBRARY}
                      ${Boost_UNIT_TEST_FRAMEWORK_LIBRARY}
                      )
add_test(test_myprogram test_myprogram)
# make "make test" run the test program
add_custom_target(test
     DEPENDS myprogram
     COMMAND ./test_myprogram
)

In order to build the program and the test, run

make

In order to run the tests, use

make test
Posted by Uli Köhler in Boost, C/C++, CMake

How to fix CMake error Could not find a package configuration file provided by “boost_unit_test_framework”

Problem:

When compiling your CMake-based C++

CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:117 (find_package):
  Could not find a package configuration file provided by
  "boost_unit_test_framework" (requested version 1.71.0) with any of the
  following names:

    boost_unit_test_frameworkConfig.cmake
    boost_unit_test_framework-config.cmake

  Add the installation prefix of "boost_unit_test_framework" to
  CMAKE_PREFIX_PATH or set "boost_unit_test_framework_DIR" to a directory
  containing one of the above files.  If "boost_unit_test_framework" provides
  a separate development package or SDK, be sure it has been installed.
Call Stack (most recent call first):
  /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component)
  /usr/share/cmake-3.16/Modules/FindBoost.cmake:443 (find_package)
  OCCUtils/CMakeLists.txt:58 (find_package)


-- Configuring incomplete, errors occurred!

Solution:

The error message is trying to tell you that you don’t have the boost unit test library installed (or CMake can’t find it).

On Ubuntu, for example, install it using

sudo apt -y install libboost-test-dev

On Windows you can install it using the prebuilt boost binaries for Windows.

Posted by Uli Köhler in C/C++, CMake

How to fix CMake error Could not find a package configuration file provided by “boost_filesystem”

Problem:

When compiling your CMake-based C++

CMake Error at /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:117 (find_package):
  Could not find a package configuration file provided by "boost_filesystem"
  (requested version 1.71.0) with any of the following names:

    boost_filesystemConfig.cmake
    boost_filesystem-config.cmake

  Add the installation prefix of "boost_filesystem" to CMAKE_PREFIX_PATH or
  set "boost_filesystem_DIR" to a directory containing one of the above
  files.  If "boost_filesystem" provides a separate development package or
  SDK, be sure it has been installed.
Call Stack (most recent call first):
  /usr/lib/x86_64-linux-gnu/cmake/Boost-1.71.0/BoostConfig.cmake:182 (boost_find_component)
  /usr/share/cmake-3.16/Modules/FindBoost.cmake:443 (find_package)
  CMakeLists.txt:58 (find_package)

Solution:

The error message is trying to tell you that you don’t have the boost filesystem library installed (or CMake can’t find it).

On Ubuntu, for example, install it using

sudo apt -y install libboost-filesystem-dev

On Windows you can install it using the prebuilt boost binaries for Windows.

Posted by Uli Köhler in C/C++, CMake

CMake executable minimal example

This CMakeLists.txt builds an executable called main from main.cpp:

add_executable(main main.cpp)

 

Posted by Uli Köhler in CMake

Towards a docker-based build of C/C++ applications

Note: Based on this post I have published buildock on GitHub.

Many C/C++ programmers and project managers know the pain of creating a reproducible build environment for all developers: Works for me is a common meme not without a reason.

My approach is to dockerize not neccessarily the application itself but the build system, encapsulating both the specific compiler version and the system around it plus all required system-level libraries in a Docker image.

Take the obligatory Hello World in C++:

// main.cpp
#include <iostream>
using namespace std;

int main() {
    cout << "Hello, World!" << endl;
    return 0;
}

and the corresponding Makefile:

all:
        g++ -o helloworld main.cpp

How can we compile this simple project without installing a compiler and GNU make on the local computer (no cheating by using build servers allowed)?

Actually it’s rather simple:

docker run --user $(id -u):$(id -g) -v $PWD:/app -it ulikoehler/ubuntu-gcc-make make

Breaking it down:

  • docker run: Create a new docker container and run a command in it
  • --user $(id -u):$(id -g): This makes the docker container run with the current user’s ID and the current user’s group – both for preventing the compiler to create output files as root and to safeguard against some IT security risks. Also see How to run docker container as current user & group
  • -v $PWD:/app: Mount the current directory ($PWD) on /app in the container. Since the Dockerfile used to build the container contains the WORKDIR /app directive, whatever command we run in the container will by default be executed in the /app directory – and therefore in the current local directory on the host.
  • -it runs the container in interactive mode, i.e. keypressed are passed down to the command you are running. Also, this means that our command will only finish when the container has finished executing.
  • ulikoehler/ubuntu-gcc-make: This is the image we’re using for that example. It’s nothing more than an ubuntu:18.04 base image with build-essentials and make installed and WORKDIR set to /app
  • make: This is the command we’ll run in the container. You can use any command here, even no command is possible (in which case the container’s default command will be used – in case of ulikoehler/ubuntu-gcc-make that is CMD [ "/usr/bin/make" ])

Here’s the complete Dockerfile used to generate ulikoehler/ubuntu-gcc-make:

FROM ubuntu:18.04
RUN apt update && apt -y install build-essential make && rm -rf /var/lib/apt/lists/*
WORKDIR /app
CMD [ "/usr/bin/make" ]
Posted by Uli Köhler in Build systems, C/C++, Container, Docker

What is the CMake equivalent to AC_CHECK_FUNCS()?

With automake/m4 in configure.ac you use syntax like

AC_CHECK_FUNCS(atan2)

which defines HAVE_FDATASYNC if the function is available.

In CMake, you can use check_symbol_exists() for the same purpose like this:

cmake_minimum_required(VERSION 3.0)
include(CheckSymbolExists)

# Define executable
add_executable(myexe main.c)

# atan2 requires the math library to be linked
list(APPEND CMAKE_REQUIRED_LIBRARIES m)
check_symbol_exists(atan2 math.h HAVE_ATAN2)

# Add compile definitions if we have the library
if(HAVE_ATAN2)
    target_compile_definitions(myexe PRIVATE -DHAVE_ATAN2)
endif()

Note that check_symbol_exists() does not automatically add a preprocessor define but you have to do that manually (see the last block in the code shown above). While this might seem less comfortable at first, this approach provides you with much more flexibility on how to handle missing or available functions.

See the CMake check_symbol_exists() documentation for more details.

Posted by Uli Köhler in CMake

CMake check_symbol_exists() minimal example

cmake_minimum_required(VERSION 3.0)
include(CheckSymbolExists)

# Define executable
add_executable(myexe main.c)

# atan2 requires the math library to be linked
list(APPEND CMAKE_REQUIRED_LIBRARIES m)
check_symbol_exists(atan2 math.h HAVE_ATAN2)

# Add compile definitions if we have the library
if(HAVE_ATAN2)
    target_compile_definitions(myexe PRIVATE -DHAVE_ATAN2)
endif()
Posted by Uli Köhler in CMake

How to fix ‘Unknown CMake command “check_symbol_exists”‘

If you have CMake code using check_symbol_exists(...) like

list(APPEND CMAKE_REQUIRED_LIBRARIES m)
check_symbol_exists(atan2 math.h HAVE_ATAN2)

but you get an error message like

CMake Error at CMakeLists.txt:8 (check_symbol_exists):
  Unknown CMake command "check_symbol_exists".

-- Configuring incomplete, errors occurred!

you need to add

include(CheckSymbolExists)

near the top of your CMakeLists.txt (before your first call to check_symbol_exists()).

Posted by Uli Köhler in CMake