C/C++

How to recursively delete directory using C++17 filesystem library

To remove a file or directory (for example my-directory) use remove_all from the C++17 filesystem library:

remove_all("my-directory");

This will remove my-directory and all its sub-directories and files recursively.

Full example:

#include <experimental/filesystem>
using namespace std::experimental::filesystem;

int main() {
    remove_all("my-directory");
}

In case you are using GCC, you need to compile the file like this:

g++ -o delete-cpp17 delete-cpp17.cpp -lstdc++fs

You need to link the stdc++fs library so the functions from the C++17 filesystem library are available to your program.

If you just want to remove a file and don’t want to risk deleting an entire directory tree, use remove instead of remove_all or see our previous post How to delete file using C++17 filesystem library

Posted by Uli Köhler in C/C++

How to delete file using C++17 filesystem library

To remove a file (for example test.txt) use remove from the C++17 filesystem library:

remove("test.txt");

Full example:

#include <experimental/filesystem>
using namespace std::experimental::filesystem;

int main() {
    remove("test.txt");
}

In case you are using GCC, you need to compile the file like this:

g++ -o delete-cpp17 delete-cpp17.cpp -lstdc++fs

You need to link the stdc++fs library so the functions from the C++17 filesystem library are available to your program.

Note that remove does not recursively remove directories! Use remove_all or see How to recursively delete directory using C++17 filesystem library

Posted by Uli Köhler in C/C++

Is it a file or directory? Using C++17 filesystem library

You can use is_regular_file to check any path (either a C++17 path object or just a string). Similarly you can use is_directory to check if the given path belongs to a directory.

Note that these just return false if the file or directory does not exist!

// Check if something is a file
bool isTestTxtAFile = is_regular_file("test.txt"); // true
bool isMyDirectoryAFile = is_regular_file("my-directory"); // false
bool isDoesNotExistAFile = is_regular_file("does-not-exist"); // false
// Check if something is a directory
bool isTestTxtADirectory = is_directory("test.txt"); // false
bool isMyDirectoryADirectory = is_directory("my-directory"); // true
bool isDoesNotExistADirectory = is_directory("does-not-exist"); // false
Posted by Uli Köhler in C/C++

How to recursively list a directory using C++17 filesystem library

Note: To find out how to list a directory non-recursively, just replace recursive_directory_iterator by directory_iterator or see our full post How to list a directory using C++17 filesystem library.

To recursively list a directory using the C++17 filesystem library use this snippet:

#include <experimental/filesystem>

using namespace std::experimental::filesystem;

for(const directory_entry& entry : recursive_directory_iterator("my-directory")) {
    const auto& path = entry.path();
    // ...
}

Full example:

#include <iostream>
#include <string>
#include <experimental/filesystem>

using namespace std;
using namespace std::experimental::filesystem;

int main() {
    for(const directory_entry& entry : recursive_directory_iterator("my-directory")) {
        // Is it a file / directory?
        bool isNormalFile = is_regular_file(entry);
        bool isDirectory = is_directory(entry);

        auto path = entry.path();
        // Path: my-directory/test.txt
        string pathString = path.string();

        // Filename: test.txt
        string filenameString = path.filename().string();

        // Extension: txt
        string extensionString = path.extension().string();

        // NOTE: You can also "cout << path" directly
    }
}

Compile like this:

g++ -o filesystem-example filesystem-example.cpp -lstdc++fs

For this directory:

my-directory
├── subdirectory
│   └── subdir-test.txt
└── test.txt

this will list subdirectory , subdirectory/subdir-test.txt and test.txt.

Posted by Uli Köhler in C/C++

How to list a directory using C++17 filesystem library

To list a directory (non-recursively) using the C++17 filesystem library use this snippet:

#include <experimental/filesystem>

using namespace std::experimental::filesystem;

for(const directory_entry& entry : directory_iterator("my-directory")) {
    const auto& path = entry.path();
    // ...
}

Full example:

#include <iostream>
#include <string>
#include <experimental/filesystem>

using namespace std;
using namespace std::experimental::filesystem;

int main() {
    for(const directory_entry& entry : directory_iterator("my-directory")) {
        // Is it a file / directory?
        bool isNormalFile = is_regular_file(entry);
        bool isDirectory = is_directory(entry);

        auto path = entry.path();
        // Path: my-directory/test.txt
        string pathString = path.string();

        // Filename: test.txt
        string filenameString = path.filename().string();

        // Extension: txt
        string extensionString = path.extension().string();

        // NOTE: You can also "cout << path" directly
    }
}

Compile like this:

g++ -o filesystem-example filesystem-example.cpp -lstdc++fs

For this directory:

my-directory
├── subdirectory
│   └── subdir-test.txt
└── test.txt

this will only list subdirectory and test.txt

Posted by Uli Köhler in C/C++

Are changes made to mmap MAP_PRIVATE visible to the current process?

The mmap mapage tells us that whether changes in memory made to a MAP_PRIVATE-memory-mapped file are visible to the process mapping the file is unspecified (they will not be written to the mapped file nor will they be visible to other processes mapping the same file).

However, we can verify if changes are actually visible on any given system / kernel version using this test program:

#include <string>
#include <iostream> 
#include <fstream> 
#include <sys/mman.h>
#include <sys/types.h>
#include <fcntl.h>
#include <unistd.h>
#include <assert.h>
#include <sys/stat.h>

using namespace std;

int main() {
    string filename = "deleteme.txt";
    // Create file
    ofstream fout(filename.c_str());
    fout << "ABCD";
    fout.close();
    // Create & open underlying file
    int fd = open(filename.c_str(), O_RDONLY);
    assert(fd != -1);
    // mmap with MAP_PRIVATE and allow writes to the mmapped pages
    char* mmappedData = (char*)mmap(NULL, 4, PROT_WRITE, MAP_PRIVATE | MAP_POPULATE, fd, 0);
    assert(mmappedData != MAP_FAILED);
    // Insert data into the MAP_PRIVATE area
    mmappedData[0] = 0;
    mmappedData[1] = 1;
    mmappedData[2] = 2;
    mmappedData[3] = 3;
    // Overwrite once more
    mmappedData[0] = 3;
    mmappedData[1] = 2;
    mmappedData[2] = 1;
    mmappedData[3] = 0;
    // Check data
    if(mmappedData[0] == 3 &&
        mmappedData[1] == 2 &&
        mmappedData[2] == 1 &&
        mmappedData[3] == 0) {
        cout << "Congrats, MAP_PRIVATE changes are reflected in memory!" << endl;
    } else {
        cout << "Nope, MAP_PRIVATE changes are NOT reflected in memory!" << endl;
    }
}

Download, compile and run using:

wget https://techoverflow.net/scripts/mmap-private-check.cpp && g++ -o mmap-private-check mmap-private-check.cpp && ./mmap-private-check

This will either print

Congrats, MAP_PRIVATE changes are reflected in memory!

(I only got this result for every Linux system I tested on, e.g. on Ubuntu 18.04) or

Nope, MAP_PRIVATE changes are NOT reflected in memory!

Note that this program only tests the visibility of the changes directly after writing the data. In principle, the Kernel is allowed to just discard your changes at any later point in time. So note that you are living on the edge here, doing changes to MAP_PRIVATE memory is not inherently safe but in practice it often works very well.

Posted by Uli Köhler in C/C++, Linux

How to get file size using boost::filesystem library

To get the filesize (in bytes) of any file (test.xml in our example) using the boost::filesystem library, use this snippet:

#include <experimental/filesystem>
#include <iostream>

using namespace std;
using namespace std::experimental::filesystem;

int main() {
    size_t filesize = file_size("test.xml");
    cout << filesize << endl;
}

You need to link the boost_filesystem and the boost_system libraries, i.e. compile like this:

g++ -o test test.cpp -lboost_filesystem -lboost_system
Posted by Uli Köhler in C/C++

How to get file size using C++17 filesystem library

To get the filesize (in bytes) of any file (test.xml in our example) using only the C++17 filesystem library, use this snippet:

#include <experimental/filesystem>
#include <iostream>

using namespace std;
using namespace std::experimental::filesystem;

int main() {
    size_t filesize = file_size("test.xml");
    cout << filesize << endl;
}

With GCC/G++ you need to link the stdc++fs library, i.e. compile like this:

g++ -o test test.cpp -lstdc++fs
Posted by Uli Köhler in C/C++

How to fix GCC undefined reference to std::experimental::filesystem::…

Problem:

You want to compile an executable using GCC/G++ that uses functionality from std::experimental::filesystem but you get an error message like this during compilation / linking:

test.cpp:(.text+0x33): undefined reference to `std::experimental::filesystem::v1::file_size(std::experimental::filesystem::v1::__cxx11::path const&)'
/tmp/ccSXeCVb.o: In function `std::experimental::filesystem::v1::__cxx11::path::path<char [9], std::experimental::filesystem::v1::__cxx11::path>(char const (&) [9])':
test.cpp:(.text._ZNSt12experimental10filesystem2v17__cxx114pathC2IA9_cS3_EERKT_[_ZNSt12experimental10filesystem2v17__cxx114pathC5IA9_cS3_EERKT_]+0x73): undefined reference to `std::experimental::filesystem::v1::__cxx11::path::_M_split_cmpts()'
collect2: error: ld returned 1 exit status

Solution:

You need to link the stdc++fs library, i.e. append -lstdc++fs to your g++ command.

For example, a working command looks like this:

g++ -o myprogram main.cpp -lstdc++fs
Posted by Uli Köhler in C/C++, GCC errors

Minimal RapidXML file reader example

XML:

<?xml version="1.0" encoding="UTF-8"?>
<root-element>Test text</root-element>

C++:

#include <rapidxml/rapidxml_utils.hpp>
#include <iostream> 

using namespace rapidxml;
using namespace std;

int main() {
    file<> xmlFile("test.xml");
    // Create & parse document
    xml_document<> doc;
    doc.parse<0>(xmlFile.data());
    // Get root node
    xml_node<> *root = doc.first_node("root-element");
    // Print content
    if(root == nullptr) {
        cerr << "Element not found!" << endl;
    } else { // We found the root element
        cout << root->value() << endl;
    }
}

 

Posted by Uli Köhler in C/C++

How to fix std::wcout printing question marks (?) on Linux

Problem:

You are trying to print a wstring from a wstring literal using std::wcout (with an UTF-8-encoded source file):

wstring w = L"Test: äöü";
wcout << w << endl;

but when you run this program, you see

Test: ???

Solution:

Use setlocale() to set a UTF-8 locale:

setlocale( LC_ALL, "en_US.utf8" );
wstring w = L"Test: äöü";
wcout << w << endl;

This will print

Test: äöü

as expected.

Full example

#include <string>
#include <iostream>

using namespace std;

int main() {
    setlocale( LC_ALL, "en_US.utf8" );
    wstring w = L"Test: äöü";
    wcout << w << endl;
}

Compile like this:

g++ -o main main.cpp
Posted by Uli Köhler in C/C++, Linux

How to cout a wstring or wchar_t in C++

Problem:

You have a std::wstring that you want to print using cout

wstring w = L"Test: äöü";
cout << w << endl;

but you see a long error message that ends like this:

/usr/include/c++/7/ostream:682:5: note:   template argument deduction/substitution failed:
/usr/include/c++/7/ostream: In substitution of 'template<class _Ostream, class _Tp> typename std::enable_if<std::__and_<std::__not_<std::is_lvalue_reference<_Tp> >, std::__is_convertible_to_basic_ostream<_Ostream>, std::__is_insertable<typename std::__is_convertible_to_basic_ostream<_Tp>::__ostream_type, const _Tp&, void> >::value, typename std::__is_convertible_to_basic_ostream<_Tp>::__ostream_type>::type std::operator<<(_Ostream&&, const _Tp&) [with _Ostream = std::basic_ostream<char>&; _Tp = std::__cxx11::basic_string<wchar_t>]':
test.cpp:9:13:   required from here
/usr/include/c++/7/ostream:682:5: error: no type named 'type' in 'struct std::enable_if<false, std::basic_ostream<char>&>'

Solution:

You need to use std::wcout instead of std::cout:

wstring w = L"Test: äöü";
wcout << w << endl

 

Posted by Uli Köhler in C/C++

How to declare wstring or wchar_t literal in C++

In order to declare a wstring literal, use the L prefix:

wstring w = L"Test: äöü";

In order to declare a wchar_t literal, you can also use the L prefix:

wchar_t wc = L'ö';
Posted by Uli Köhler in C/C++

How to get attribute value in RapidXML

Problem:

You have a rapidxml::xml_node instance for which you want to access a specific attribute,  e.g. my-attribute

Solution:

Use first_attribute with the name argument set to a string:

rapidxml::xml_attribute<>* attr = node->first_attribute("my-attribute");

Remember that first_attribute() returns nullptr if no such attribute exists so be sure to check for that to avoid segmentation faults!

Full example:

XML:

<?xml version="1.0" encoding="UTF-8"?>
<root-element>
    <child my-attr="foo"></child>
</root-element>

C++:

#include <rapidxml/rapidxml_utils.hpp>
#include <string>
#include <iostream> 

using namespace rapidxml;
using namespace std;

int main() {
    rapidxml::file<> xmlFile("test.xml");
    // Create & parse document
    rapidxml::xml_document<> doc;
    doc.parse<0>(xmlFile.data());
    
    // Get root node
    rapidxml::xml_node<> *root = doc.first_node("root-element");
    rapidxml::xml_node<> *child = root->first_node("child");

    // Get & print attribute
    rapidxml::xml_attribute<>* attr = child->first_attribute("my-attr");
    if(attr == nullptr) {
        cout << "No such attribute!" << endl;
    } else {
        cout << attr->value() << endl;
    }
}

Or you can use this snippet function to get either the attribute value or a default value:

#include <rapidxml/rapidxml_utils.hpp>
#include <string>
#include <iostream> 

using namespace rapidxml;
using namespace std;

/**
 * Return either the ->value() of attr or default_value if attr == nullptr
 */
inline string attr_value_or_default(rapidxml::xml_attribute<>* attr, string default_value="") {
    if(attr == nullptr) {
        return default_value;
    } else {
        return attr->value();
    }
}

int main() {
    rapidxml::file<> xmlFile("test.xml");
    // Create & parse document
    rapidxml::xml_document<> doc;
    doc.parse<0>(xmlFile.data());
    
    // Get root node
    rapidxml::xml_node<> *root = doc.first_node("root-element");
    rapidxml::xml_node<> *child = root->first_node("child");

    // Get & print attribute+
    rapidxml::xml_attribute<>* attr = child->first_attribute("my-attr");
    cout << attr_value_or_default(attr, "No such attribute!") << endl;
}
Posted by Uli Köhler in C/C++

How to fix fatal error: rapidxml_utils.hpp: No such file or directory

If you are seeing an error message like fatal error: rapidxml_utils.hpp: No such file or directory in a line like

#include "rapidxml_utils.hpp"

or

#include <rapidxml_utils.hpp>

you likely need to replace that line by

#include <rapidxml/rapidxml_utils.hpp>

If this also doesn’t work, you might not have RapidXML installed. See How to install RapidXML on Ubuntu for details on how to do that on Ubuntu.

Posted by Uli Köhler in C/C++

How to install RapidXML on Ubuntu

If you want to use RapidXML to parse XML data on Ubuntu, run this command:

sudo apt install librapidxml-dev -y

Since RapidXML is a header-only library, this will install the header files and no native libraries.

Posted by Uli Köhler in C/C++, Linux

AutoBenchmark: Automatic multi-interval benchmarking in C++ using std::chrono

Problem:

Some part of your C++ code is suffering from performance issues. You are looking for a lightweight solution that allows you to easily record different time points and adaptively print the results (i.e. you don’t want to know something ran for 1102564643 nanoseconds, you just want to now that it took 1.102 seconds)

Solution:

I wrote AutoBenchmark so you can have the most hassle-free C++11 experience possible for your micro-benchmarking needs.

AutoBenchmark allows you to record different points in time, each with a label. The first time point is recorded when this instance is constructed. AutoBenchmark supports an arbitrary number of time points.
When an instance of this class is destructed, it will automatically print all the benchmark results, but only if a configurable amount of time has passed since its construction – this is extremely handy especially if you have multiple exit points in your function that would otherwise require calling Print() multiple times.
It allows you to ignore the benchmark when some performance goal is passed (e.g. if you have a for loop that is slow only for some datapoints, you can configure AutoBenchmark to print infos only for the slow runs).
The default behaviour (i.e. constructor with default parameters) is to disable automatic printing – in that case, you can call Print() yourself.

Header (AutoBenchmark.hpp):

/**
 * AutoBenchmark v1.0
 * Written by Uli Köhler
 * https://techoverflow.net
 * 
 * Published under CC0 1.0 Universal
 */
#pragma once

#include <chrono>
#include <string>
#include <limits>
#include <vector>

using namespace std;

/**
 * Automatic benchmark: Allows you to record different points in time,
 * each with a label. The first time point is recorded when this instance
 * is constructed. This class supports an arbitrary number of time points.
 * 
 * When an instance of this class is destructed, it will automatically
 * print all the benchmark results, but only if a configurable amount
 * of time has passed since its construction, allowing you to automatically
 */
class AutoBenchmark {
  public:
    /**
     * Initialize a benchmark that automatically prints its records
     * on destruction if the total time consumed is >= autoPrintThreshold
     * at the time of destruction. Only the time up until the last Record()ed
     * label is printed.
     * @param autoPrintThreshold: How many seconds will need to have passed
     * so that the destructor will automatically print. Default is to never print.
     * @param benchmarkLabel: A label that will be printed once, before all the results
     * @param lineLabel: A label (e.g. indent) that will be printed before each result line
     */
    AutoBenchmark(double autoPrintThreshold=std::numeric_limits::max(), const std::string& benchmarkLabel = "", const std::string& lineLabel = "    ");
    ~AutoBenchmark();
    /**
     * Record a datapoint
     */
    void Record(const std::string& label = "");
    void Record(const char *label = "");
    /**
     * Print all time deltas
     */
    void Print();
    
    /**
     * Reset the benchmark, as if it were a new instance.
     */
    void Reset();

    /**
     * Return now() - first timepoint in seconds.
     */
    double TotalSeconds();

  private:
    vector times;
    vector labels;
    double autoPrintThreshold;
    std::string benchmarkLabel;
    std::string lineLabel;
};

Source (AutoBenchmark.cpp):

#include "AutoBenchmark.hpp"

#include <iostream>

AutoBenchmark::AutoBenchmark(double autoPrintThreshold, const std::string& benchmarkLabel, const std::string& lineLabel)
    : autoPrintThreshold(autoPrintThreshold), benchmarkLabel(benchmarkLabel), lineLabel(lineLabel) {
    times.push_back(chrono::system_clock::now());
    labels.emplace_back("Begin"); // Just to keep indices the same
}

AutoBenchmark::~AutoBenchmark() {
    if(TotalSeconds() >= autoPrintThreshold) {
        Print();
    }
}

void AutoBenchmark::Record(const std::string &label) {
    times.push_back(chrono::system_clock::now());
    labels.emplace_back(label); // Just to keep indices the same
}

void AutoBenchmark::Record(const char *label) {
    Record(string(label));
}

void AutoBenchmark::Print() {
    if(benchmarkLabel.length()) {
        cout << benchmarkLabel << '\n';
    }
    for (size_t i = 1; i < times.size(); i++) {
        // Compute time interval for size comparison
        chrono::duration<double, std::nano> ns = times[i] - times[i - 1];
        chrono::duration<double, std::micro> us = times[i] - times[i - 1];
        chrono::duration<double, std::milli> ms = times[i] - times[i - 1];
        chrono::duration s = times[i] - times[i - 1];
        chrono::duration<double, std::ratio<60>> min = times[i] - times[i - 1];
        chrono::duration<double, std::ratio<3600>> hrs = times[i] - times[i - 1];
        // Print
        if(ns.count() < 1000.0) {
            cout << lineLabel << labels[i] << " took " << ns.count() << " ns\n";
        } else if(us.count() < 1000.0) {
            cout << lineLabel << labels[i] << " took " << us.count() << " μs\n";
        } else if (ms.count() < 1000.0) {
            cout << lineLabel << labels[i] << " took " << ms.count() << " ms\n";
        } else if (s.count() < 60.0) {
            cout << lineLabel << labels[i] << " took " << s.count() << " seconds\n";
        } else if (min.count() < 1000.0) {
            cout << lineLabel << labels[i] << " took " << min.count() << " minutes\n";
        } else {
            cout << lineLabel << labels[i] << " took " << hrs.count() << " hours\n";
        }
    }
    cout << flush;
}

void AutoBenchmark::Reset() {
    times.clear();
    labels.clear();
    times.push_back(chrono::system_clock::now());
    labels.emplace_back("Begin");
}

double AutoBenchmark::TotalSeconds() {
    chrono::duration s = chrono::system_clock::now() - times[0];
    return s.count();
}

Usage example:

#include "AutoBenchmark.hpp"

void MySlowFunction() {
    // Every run that takes >= 0.3 seconds will auto-print
    AutoBenchmark myBenchmark(0.3, "Results of running MySlowFunction():");
    // .. do task 1 ...
    myBenchmark.Record("Running task 1"); // will print as: Running task 1 took 1.2ms
    // .. do task 2 ...
    myBenchmark.Record("Running task 2");

    // Loop example
    for(size_t i = 0; ....) {
        // ... do loop iteration task here ...
        myBenchmark.Record("Loop iteration " + std::to_string(i));
    }

    // myBenchmark will be destructed here, so if MySlowFunction() took
    // more than 0.3s to run until it returned, the result will be printed
    // to cout automatically.
}

If MySlowFunction() took more than 0.3s to run overall, AutoBenchmark will print the results when it is destructed – i.e. when MySlowFunction( ) returns:

Results of running MySlowFunction():
    Running task 1 took 260.826 ms
    Running task 2 took 36.148 μs
    Loop iteration 0 took 2.5522 seconds
    Loop iteration 1 took 664.059 ms
    Loop iteration 2 took 22.2772 ms
    Loop iteration 3 took 57.4024 ms
    Loop iteration 4 took 16.9928 ms
    Loop iteration 5 took 14.0497 ms
    Loop iteration 6 took 62.5218 ms

 

Posted by Uli Köhler in C/C++, Performance

ISO8601 UTC time as std::string using C++11 chrono

You want to use the C++11 standard’s chrono library to generate a ISO8601-formatted timestamp as a std::string, e.g. 2018-03-30T16:51:00Z

Solution:

You can use this function which uses std::put_time with a std::ostringstream to generate the resulting std::string.

#include <iostream>
#include <chrono>
#include <iomanip>
#include <sstream>

/**
 * Generate a UTC ISO8601-formatted timestamp
 * and return as std::string
 */
std::string currentISO8601TimeUTC() {
  auto now = std::chrono::system_clock::now();
  auto itt = std::chrono::system_clock::to_time_t(now);

  std::ostringstream ss;
  ss << std::put_time(gmtime(&itt), "%FT%TZ");
  return ss.str();
}

// Usage example
int main() {
    std::cout << currentISO8601TimeUTC() << std::endl;
}

 

Posted by Uli Köhler in C/C++

How to fix GCC error ‘the lambda has no capture-default’

When encountering a GCC error like this:

error: the lambda has no capture-default

fixing it is usually quite easy. Look for a Lambda function that captures some variable like this

[&myVar] (/* ... */) {/* ... */}

&myVar means “capture myVar by reference”.

In most cases you can just capture all local variables by using a capture default:

[&] (/* ... */) {/* ... */}

In rare cases this will have unintended side-effects as you now are capturing all variables by reference where you might want to capture some by copy – so be sure to check your code.

Note that this error is GCC version-dependent. For me using GCC 7.2 fixed the error.

Posted by Uli Köhler in C/C++, GCC errors

How to fix GCC error: invalid use of incomplete type ‘class …’

Problem:

You are compiling a C/C++ program using GCC. You get an error message similar to this:

error: invalid use of incomplete type ‘class SomeType’

Solution:

There are multiple possible issues, but in general this error means that GCC can’t find the full declaration of the given class or struct.

The most common issue is that you are missing an #include clause. Find out in which header file the declaration resides, i.e. if the error message mentions class Map, look for something like

class Map {
   // ...
};

Usually the classes reside in header files that are similar to their name, e.g. MyClass might reside in a header file that is called MyClass.h, MyClass.hpp or MyClass.hxx, so be sure to look for those files first. Note that you might also be looking for a type from a library. Often the best approach is to google C++ <insert the missing type here> to find out where it might be located.

Another possible reason is that you have your #include clause after the line where the error occurs. If this is the case, ensure that all required types are included before they are used.

For other reasons, see StackOverflow, e.g. this post

Posted by Uli Köhler in C/C++, GCC errors