Fixing docker ‘unable to delete …- image is being used by running container’

Problem:

You want to delete a docker image using a command like

docker image rm c91b419ac445

but you see an error message like

Error response from daemon: conflict: unable to delete c91b419ac445 (cannot be forced) - image is being used by running container 3477a4dcdce2

Solution:

There is currently a container running that uses the image you are trying to delete. We will solve this issue by first stopping the container and then deleting the image

Warning: Deleting the image is dangerous since you cannot undo deleting the image ! Also note that force-stopping a running container might result in data loss if that container is doing something important !

Run these commands to stop the container and delete the image:

# Force-stop the container
docker container rm --force <container ID>
# Delete the image
docker image rm <image ID>

Copy <container ID> from the end of your original error message (3477a4dcdce2 in my example).

Copy <image ID> from the beginning of your error message. This is the same image ID you originally intended to delete (c91b419ac445) in my example.

In my example, the command would be

# Force-stop the container
docker container rm --force 3477a4dcdce2
# Delete the image
docker image rm c91b419ac445

Note that there might be multiple containers running using this image, so if you keep getting a similar error message, you might need to repeat this command.

Background information:

Docker will not allow you to force-delete the image using

docker image rm c91b419ac445 --force

as you can also see from the (cannot be forced) clause of your original error message. This behaviour makes sense since the container would crash in an undefineable manner if the underlying image is deleted.

Note that we could use docker image rm --force after stopping the container but this is typically not required and might result in additional risks for other containers, e.g. if other images depend on said image. Since docker uses layered images. Read the background information section of our post Docker: Remove all images and containers to learn more about how docker images work from an image management perspective.

Posted by Uli Köhler in Container, Docker

How to get path to the wp_content directory in WordPress plugin

Problem:

You are writing a wordpress plugin in which you need the path to the wp-content directory on the filesystem.

Solution:

Use the WP_CONTENT_DIR constant.

$path_to_wp_content = WP_CONTENT_DIR; // e.g. "/var/sites/techoverflow.net/wp-content"

Note that WP_CONTENT_DIR has no trailing slash.

Use WP_CONTENT_DIR . "/" like this

$path_to_wp_content = WP_CONTENT_DIR . "/"; // e.g. "/var/sites/techoverflow.net/wp-content/"

to get the path to wp_content including a trailing slash.

Posted by Uli Köhler in PHP, Wordpress

How to fix WP_Query not listing any posts

Problem:

In your custom WordPress plugin or theme you have code like

$query = new WP_Query( );
while ( $query->have_posts() ) {
        $query->the_post();
        // ... your code to process the post ...
}

But $query->have_posts() is always false and the loop is never executed.

Solution:

WP_Query works only if you use an appropriate query. Using no query at all is not equivalent to list all posts!

This is likely what you want to do:

$query = new WP_Query(array('post_type' => 'post'));
Posted by Uli Köhler in PHP, Wordpress

How to recursively delete directory using C++17 filesystem library

To remove a file or directory (for example my-directory) use remove_all from the C++17 filesystem library:

remove_all("my-directory");

This will remove my-directory and all its sub-directories and files recursively.

Full example:

#include <experimental/filesystem>
using namespace std::experimental::filesystem;

int main() {
    remove_all("my-directory");
}

In case you are using GCC, you need to compile the file like this:

g++ -o delete-cpp17 delete-cpp17.cpp -lstdc++fs

You need to link the stdc++fs library so the functions from the C++17 filesystem library are available to your program.

If you just want to remove a file and don’t want to risk deleting an entire directory tree, use remove instead of remove_all or see our previous post How to delete file using C++17 filesystem library

Posted by Uli Köhler in C/C++

How to delete file using C++17 filesystem library

To remove a file (for example test.txt) use remove from the C++17 filesystem library:

remove("test.txt");

Full example:

#include <experimental/filesystem>
using namespace std::experimental::filesystem;

int main() {
    remove("test.txt");
}

In case you are using GCC, you need to compile the file like this:

g++ -o delete-cpp17 delete-cpp17.cpp -lstdc++fs

You need to link the stdc++fs library so the functions from the C++17 filesystem library are available to your program.

Note that remove does not recursively remove directories! Use remove_all or see How to recursively delete directory using C++17 filesystem library

Posted by Uli Köhler in C/C++

Is it a file or directory? Using C++17 filesystem library

You can use is_regular_file to check any path (either a C++17 path object or just a string). Similarly you can use is_directory to check if the given path belongs to a directory.

Note that these just return false if the file or directory does not exist!

// Check if something is a file
bool isTestTxtAFile = is_regular_file("test.txt"); // true
bool isMyDirectoryAFile = is_regular_file("my-directory"); // false
bool isDoesNotExistAFile = is_regular_file("does-not-exist"); // false
// Check if something is a directory
bool isTestTxtADirectory = is_directory("test.txt"); // false
bool isMyDirectoryADirectory = is_directory("my-directory"); // true
bool isDoesNotExistADirectory = is_directory("does-not-exist"); // false
Posted by Uli Köhler in C/C++

How to recursively list a directory using C++17 filesystem library

Note: To find out how to list a directory non-recursively, just replace recursive_directory_iterator by directory_iterator or see our full post How to list a directory using C++17 filesystem library.

To recursively list a directory using the C++17 filesystem library use this snippet:

#include <experimental/filesystem>

using namespace std::experimental::filesystem;

for(const directory_entry& entry : recursive_directory_iterator("my-directory")) {
    const auto& path = entry.path();
    // ...
}

Full example:

#include <iostream>
#include <string>
#include <experimental/filesystem>

using namespace std;
using namespace std::experimental::filesystem;

int main() {
    for(const directory_entry& entry : recursive_directory_iterator("my-directory")) {
        // Is it a file / directory?
        bool isNormalFile = is_regular_file(entry);
        bool isDirectory = is_directory(entry);

        auto path = entry.path();
        // Path: my-directory/test.txt
        string pathString = path.string();

        // Filename: test.txt
        string filenameString = path.filename().string();

        // Extension: txt
        string extensionString = path.extension().string();

        // NOTE: You can also "cout << path" directly
    }
}

Compile like this:

g++ -o filesystem-example filesystem-example.cpp -lstdc++fs

For this directory:

my-directory
├── subdirectory
│   └── subdir-test.txt
└── test.txt

this will list subdirectory , subdirectory/subdir-test.txt and test.txt.

Posted by Uli Köhler in C/C++

How to list a directory using C++17 filesystem library

To list a directory (non-recursively) using the C++17 filesystem library use this snippet:

#include <experimental/filesystem>

using namespace std::experimental::filesystem;

for(const directory_entry& entry : directory_iterator("my-directory")) {
    const auto& path = entry.path();
    // ...
}

Full example:

#include <iostream>
#include <string>
#include <experimental/filesystem>

using namespace std;
using namespace std::experimental::filesystem;

int main() {
    for(const directory_entry& entry : directory_iterator("my-directory")) {
        // Is it a file / directory?
        bool isNormalFile = is_regular_file(entry);
        bool isDirectory = is_directory(entry);

        auto path = entry.path();
        // Path: my-directory/test.txt
        string pathString = path.string();

        // Filename: test.txt
        string filenameString = path.filename().string();

        // Extension: txt
        string extensionString = path.extension().string();

        // NOTE: You can also "cout << path" directly
    }
}

Compile like this:

g++ -o filesystem-example filesystem-example.cpp -lstdc++fs

For this directory:

my-directory
├── subdirectory
│   └── subdir-test.txt
└── test.txt

this will only list subdirectory and test.txt

Posted by Uli Köhler in C/C++

How to fix ElasticSearch ‘[match] query doesn’t support multiple fields, found […] and […]’

Problem:

You want to run an ElasticSearch query like

{
    "query": {
        "match" : {
            "one_field" : "one_value",
            "another_field": "another_value"
        }
    }
}

but you only see an error message like

elasticsearch.exceptions.RequestError: RequestError(400, 'parsing_exception', "[match] query doesn't support multiple fields, found [one_field] and [another_field]")

Solution:

Match queries only support one field. You should use a bool query with a must clause containing multiple match queries instead:

{
    "query": {
        "bool": {
            "must": [
                {"match": {"one_field" : "one_value"}},
                {"match": {"another_field" : "another_value"}},
            ]
        }
    }
}

Also see the official docs for the MultiMatch query.

Posted by Uli Köhler in Databases, ElasticSearch

How to fix ElasticSearch ‘no [query] registered for [missing]’

Problem:

You are trying to run an ElasticSearch query like

{
    "query": {
        "missing" : { "field" : "myfield" }
    }
}

to find documents that do not have myfield.

However you only see an error message like this:

elasticsearch.exceptions.RequestError: RequestError(400, 'parsing_exception', 'no [query] registered for [missing]')

Solution:

As the ElasticSearch documentation tells, us, there is no missing query! You instead need to use an exists query inside a must_not clause:

{
    "query": {
        "bool": {
            "must_not": {
                "exists": {
                    "field": "myfield"
                }
            }
        }
    }
}

 

Posted by Uli Köhler in Databases, ElasticSearch

How to fix Google Cloud Build ignoring .dockerignore

Problem:

You want to run a docker image build on Google Cloud Build, but the client is trying to upload a huge context image to Google Cloud even though you have added all your large directories to your .dockerignore and the build works fine locally.

Solution:

Google Cloud Build ignores .dockerignore by design – the equivalent is called .gcloudignore.

You can copy the .dockerignore behaviour for gcloud by running

cp .dockerignore .gcloudignore

 

Posted by Uli Köhler in Cloud, Container, Docker

Are changes made to mmap MAP_PRIVATE visible to the current process?

The mmap mapage tells us that whether changes in memory made to a MAP_PRIVATE-memory-mapped file are visible to the process mapping the file is unspecified (they will not be written to the mapped file nor will they be visible to other processes mapping the same file).

However, we can verify if changes are actually visible on any given system / kernel version using this test program:

#include <string>
#include <iostream> 
#include <fstream> 
#include <sys/mman.h>
#include <sys/types.h>
#include <fcntl.h>
#include <unistd.h>
#include <assert.h>
#include <sys/stat.h>

using namespace std;

int main() {
    string filename = "deleteme.txt";
    // Create file
    ofstream fout(filename.c_str());
    fout << "ABCD";
    fout.close();
    // Create & open underlying file
    int fd = open(filename.c_str(), O_RDONLY);
    assert(fd != -1);
    // mmap with MAP_PRIVATE and allow writes to the mmapped pages
    char* mmappedData = (char*)mmap(NULL, 4, PROT_WRITE, MAP_PRIVATE | MAP_POPULATE, fd, 0);
    assert(mmappedData != MAP_FAILED);
    // Insert data into the MAP_PRIVATE area
    mmappedData[0] = 0;
    mmappedData[1] = 1;
    mmappedData[2] = 2;
    mmappedData[3] = 3;
    // Overwrite once more
    mmappedData[0] = 3;
    mmappedData[1] = 2;
    mmappedData[2] = 1;
    mmappedData[3] = 0;
    // Check data
    if(mmappedData[0] == 3 &&
        mmappedData[1] == 2 &&
        mmappedData[2] == 1 &&
        mmappedData[3] == 0) {
        cout << "Congrats, MAP_PRIVATE changes are reflected in memory!" << endl;
    } else {
        cout << "Nope, MAP_PRIVATE changes are NOT reflected in memory!" << endl;
    }
}

Download, compile and run using:

wget https://techoverflow.net/scripts/mmap-private-check.cpp && g++ -o mmap-private-check mmap-private-check.cpp && ./mmap-private-check

This will either print

Congrats, MAP_PRIVATE changes are reflected in memory!

(I only got this result for every Linux system I tested on, e.g. on Ubuntu 18.04) or

Nope, MAP_PRIVATE changes are NOT reflected in memory!

Note that this program only tests the visibility of the changes directly after writing the data. In principle, the Kernel is allowed to just discard your changes at any later point in time. So note that you are living on the edge here, doing changes to MAP_PRIVATE memory is not inherently safe but in practice it often works very well.

Posted by Uli Köhler in C/C++, Linux

How to get file size using boost::filesystem library

To get the filesize (in bytes) of any file (test.xml in our example) using the boost::filesystem library, use this snippet:

#include <experimental/filesystem>
#include <iostream>

using namespace std;
using namespace std::experimental::filesystem;

int main() {
    size_t filesize = file_size("test.xml");
    cout << filesize << endl;
}

You need to link the boost_filesystem and the boost_system libraries, i.e. compile like this:

g++ -o test test.cpp -lboost_filesystem -lboost_system
Posted by Uli Köhler in C/C++

How to get file size using C++17 filesystem library

To get the filesize (in bytes) of any file (test.xml in our example) using only the C++17 filesystem library, use this snippet:

#include <experimental/filesystem>
#include <iostream>

using namespace std;
using namespace std::experimental::filesystem;

int main() {
    size_t filesize = file_size("test.xml");
    cout << filesize << endl;
}

With GCC/G++ you need to link the stdc++fs library, i.e. compile like this:

g++ -o test test.cpp -lstdc++fs
Posted by Uli Köhler in C/C++

How to fix GCC undefined reference to std::experimental::filesystem::…

Problem:

You want to compile an executable using GCC/G++ that uses functionality from std::experimental::filesystem but you get an error message like this during compilation / linking:

test.cpp:(.text+0x33): undefined reference to `std::experimental::filesystem::v1::file_size(std::experimental::filesystem::v1::__cxx11::path const&)'
/tmp/ccSXeCVb.o: In function `std::experimental::filesystem::v1::__cxx11::path::path<char [9], std::experimental::filesystem::v1::__cxx11::path>(char const (&) [9])':
test.cpp:(.text._ZNSt12experimental10filesystem2v17__cxx114pathC2IA9_cS3_EERKT_[_ZNSt12experimental10filesystem2v17__cxx114pathC5IA9_cS3_EERKT_]+0x73): undefined reference to `std::experimental::filesystem::v1::__cxx11::path::_M_split_cmpts()'
collect2: error: ld returned 1 exit status

Solution:

You need to link the stdc++fs library, i.e. append -lstdc++fs to your g++ command.

For example, a working command looks like this:

g++ -o myprogram main.cpp -lstdc++fs
Posted by Uli Köhler in C/C++, GCC errors

Minimal RapidXML file reader example

XML:

<?xml version="1.0" encoding="UTF-8"?>
<root-element>Test text</root-element>

C++:

#include <rapidxml/rapidxml_utils.hpp>
#include <iostream> 

using namespace rapidxml;
using namespace std;

int main() {
    file<> xmlFile("test.xml");
    // Create & parse document
    xml_document<> doc;
    doc.parse<0>(xmlFile.data());
    // Get root node
    xml_node<> *root = doc.first_node("root-element");
    // Print content
    if(root == nullptr) {
        cerr << "Element not found!" << endl;
    } else { // We found the root element
        cout << root->value() << endl;
    }
}

 

Posted by Uli Köhler in C/C++

How to fix std::wcout printing question marks (?) on Linux

Problem:

You are trying to print a wstring from a wstring literal using std::wcout (with an UTF-8-encoded source file):

wstring w = L"Test: äöü";
wcout << w << endl;

but when you run this program, you see

Test: ???

Solution:

Use setlocale() to set a UTF-8 locale:

setlocale( LC_ALL, "en_US.utf8" );
wstring w = L"Test: äöü";
wcout << w << endl;

This will print

Test: äöü

as expected.

Full example

#include <string>
#include <iostream>

using namespace std;

int main() {
    setlocale( LC_ALL, "en_US.utf8" );
    wstring w = L"Test: äöü";
    wcout << w << endl;
}

Compile like this:

g++ -o main main.cpp
Posted by Uli Köhler in C/C++, Linux

How to cout a wstring or wchar_t in C++

Problem:

You have a std::wstring that you want to print using cout

wstring w = L"Test: äöü";
cout << w << endl;

but you see a long error message that ends like this:

/usr/include/c++/7/ostream:682:5: note:   template argument deduction/substitution failed:
/usr/include/c++/7/ostream: In substitution of 'template<class _Ostream, class _Tp> typename std::enable_if<std::__and_<std::__not_<std::is_lvalue_reference<_Tp> >, std::__is_convertible_to_basic_ostream<_Ostream>, std::__is_insertable<typename std::__is_convertible_to_basic_ostream<_Tp>::__ostream_type, const _Tp&, void> >::value, typename std::__is_convertible_to_basic_ostream<_Tp>::__ostream_type>::type std::operator<<(_Ostream&&, const _Tp&) [with _Ostream = std::basic_ostream<char>&; _Tp = std::__cxx11::basic_string<wchar_t>]':
test.cpp:9:13:   required from here
/usr/include/c++/7/ostream:682:5: error: no type named 'type' in 'struct std::enable_if<false, std::basic_ostream<char>&>'

Solution:

You need to use std::wcout instead of std::cout:

wstring w = L"Test: äöü";
wcout << w << endl

 

Posted by Uli Köhler in C/C++