Fixing VTiger „Illegal request“ for links from other domains

Uli Köhler Allgemein

Problem:

You’ve got a link to your VTiger installation from another domain, but any time you open it, you get an Illegal request error message, even though you are logged in correctly.

Solution:

The reason for this error message is that vtiger validates the Referer (i.e. source URL of the request) as a protection layer against certain security issues, for example CSRF (cross-site request forgery). We will disable the referer check. Be sure to understand the implications before you do as suggested.

Disabling involves only editing a single code line. I tested this with VTiger 6.5.0, but likely only minor adjustments have to be made for other versions.

Steps to fix:

  • Open <your vtiger directory>/includes/http/Request.php in a text editor
  • In the editor. search for Illegal request. You will see a code block like this:
protected function validateReferer() {
$user=  vglobal('current_user');
        // Referer check if present - to over come 
        if (isset($_SERVER['HTTP_REFERER']) && $user) {//Check for user post authentication.
                global $site_URL;
                if ((stripos($_SERVER['HTTP_REFERER'], $site_URL) !== 0) && ($this->get('module') != 'Install')) {
                        throw new Exception('Illegal request');
                }
        }
        return true;
}
  • Comment out throw new Exception('Illegal request'); with // (results in //throw new Exception('Illegal request');)
  • The code block should now look like this:
protected function validateReferer() {
$user=  vglobal('current_user');
        // Referer check if present - to over come 
        if (isset($_SERVER['HTTP_REFERER']) && $user) {//Check for user post authentication.
                global $site_URL;
                if ((stripos($_SERVER['HTTP_REFERER'], $site_URL) !== 0) && ($this->get('module') != 'Install')) {
                        //throw new Exception('Illegal request');
                }
        }
        return true;
}
  • Save the file
  • The fix should be in effect immediately, else restart your webserver.

How to resolve fatal error: bytecode stream generated with LTO version … instead of the expected …

Uli Köhler GCC errors

Problem:

When compiling a program or library with GCC, you get an error message similar to this:

fatal error: bytecode stream generated with LTO version 5.1 instead of the expected 5.2

Solution:

This error basically means that you’ve got some binary build results (mostly object files) which are incompatible with other binary build results, but you’re trying to link them together. The LTO part just means that the incompatibility is due to you having link-time optimization enabled – but you don’t need to worry about that, you can just treat it like any other incompatibility.

In almost all cases, there is a very simple reason for the error: You have built your project with an older GCC version, then updated GCC and re-compiled (partially, as not all files have been changed) later. In this case, there is a simple solution: Just clean your build and re-build from scratch.

If you use make (or CMake), this is usually as simple as

make clean # Remove old, incompatible files
make # Rebuild

For other buildsystems, lookup how to clean your build accordingly – or just delete your build or dist directory if that doesn’t cause any unintended side effects.

In some very rare cases, an issue in the build system configuration causes the software to be built with two different compilers. If you think that might be the case (i.e. if the cleaning process does not help), I suggest trying to look at the verbose output of your build system — however, keep in mind that it’s more likely that you’re trying to link a stray object file that has been built with an older version of the compiler.

If you’re happy with just fixing the symptom while ignoring the possibility of hard-to-debug incompatibilities, you can just omit the -flto flag in the build system config. This hides the LTO incompatibility as it disables the link-time optimizer altogether, but even if no new error codes are shown, this approach is generally not recommended.

Disabling SSL certificate checking in unirest (NodeJS)

Uli Köhler Javascript

Problem:

You want to make a HTTP request with unirest like this:

const unirest = require('unirest');
unirest.get("https://mydomain.net").end(console.log)

but you encounter the following error:

{ error: 
   { Error: unable to verify the first certificate
       at TLSSocket.<anonymous> (_tls_wrap.js:1088:38)
       at emitNone (events.js:86:13)
       at TLSSocket.emit (events.js:188:7)
       at TLSSocket._finishInit (_tls_wrap.js:610:8)
       at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:440:38) code: 'UNABLE_TO_VERIFY_LEAF_SIGNATURE' } }

 Solution:

You can work around this problem by using strictSSL(false) like this:

const unirest = require('unirest');

unirest.get("https://mydomain.net")
.strictSSL(false)
.end(console.log)

Note however that this might have negative effects on the security of your application as this request will be vulnerable to man-in-the-middle attacks.

How to use concurrent.futures map with a tqdm progress bar

Uli Köhler Python

Problem:

You have a concurrent.futures executor, e.g.

import concurrent.futures

executor = concurrent.futures.ThreadPoolExecutor(64)

Using this executor, you want to map a function over an iterable in parallel (e.g. parallel download of HTTP pages).

In order to aid interactive execution, you want to use tqdm to provide a progress bar, showing the fraction of futures

Solution:

You can use this function:

from tqdm import tqdm
import concurrent.futures

def tqdm_parallel_map(executor, fn, *iterables, **kwargs):
    """
    Equivalent to executor.map(fn, *iterables),
    but displays a tqdm-based progress bar.
    
    Does not support timeout or chunksize as executor.submit is used internally
    
    **kwargs is passed to tqdm.
    """
    futures_list = []
    for iterable in iterables:
        futures_list += [executor.submit(fn, i) for i in iterable]
    for f in tqdm(concurrent.futures.as_completed(futures_list), total=len(futures_list), **kwargs):
        yield f.result()

Note that internally, executor.submit() is used, not executor.map() because there is no way of calling concurrent.futures.as_completed() on the iterator returned by executor.map().

Script user input

Yann Spöri Allgemein

Problem

You want to remote control a program but unfortunately this program has only a „klick&gaudy“(*) interface.
(*) Okok – only has a graphical user interface (short GUI).

Solution

You may use xdotool in order to script user actions. To install this tool, use:

sudo apt-get install xdotool

Now you can get the position of the mouse pointer with:

xdotool getmouselocation

or set it via:

xdotool mousemove 400 300 (This means set the mouse to position x=400, y=300; Point of origin is the top left corner of the screen.)

In order to click use:

xdotool click 1

And in order to type a text (e.g. into a control field of the GUI)

xdotool type 'Hello World'

(P.S.: xdotool has much more options … Once this tool got installed type man xdotool in order to see them all.)

Exporting campaign contact CSVs from VTiger using SQL

Uli Köhler Databases

In Exporting campaign account CSVs from VTiger using SQL we investigated how to use direct SQL database access to export names and emails for organizations for a given campaign.

This post shows how to do the same thing. Please refer to the previous post for a detailed explanation.

SELECT CONCAT(cont.firstname, " ", cont.lastname) AS "Name", cont.email AS "EMail" FROM vtiger_campaigncontrel AS ccr JOIN vtiger_contactdetails cont ON (cont.contactid = ccr.contactid) WHERE `campaignid` =(SELECT postvalue FROM `vtiger_modtracker_detail` WHERE fieldname = "record_id" AND `id` = (SELECT `id` FROM `vtiger_modtracker_detail` WHERE postvalue = (SELECT campaign_no FROM vtiger_campaign WHERE `campaignname` = "MyCampaign") and fieldname = "campaign_no")) AND cont.email <> ''

Besides joining on vtiger_contactdetails and vtiger_campaigncontrel instead of the account-related equivalent, the only major difference is in assembling the name. For contacts, first name and last name are stored separatedly. Therefore we use CONCAT(cont.firstname, " ", cont.lastname) to get a single name string out of that information.

As shown in the previous post, you can use phpmyadmin to export the result of the query to CSV or to Excel directly. While this might seem like a tedious process, it is much less tedious than copy-and-pasting hundreds of contacts manually.

Play a sound in a Webbrowser

Yann Spöri Allgemein

Problem:

You want to play some sounds in a webbrowser.

Solution:

Modern Browsers have a fancy integrated AudioContext that allows you to play sounds. Here is an example (JavaScript Code):

// get the AudioContext
window.AudioContext = window.AudioContext || window.webkitAudioContext;

// Initialize audio context
var context = new AudioContext();

// Create an oscillator ... via this oscillator we can then play different sounds
var oscillator = context.createOscillator();
oscillator.frequency.value = 440; // this is an "A"
oscillator.type = "square";

// attach the oscillator to the sound output
oscillator.connect(context.destination);

oscillator.start(0); // start the oscillator (0=now) ...
oscillator.stop(1);  // stop playing this sound after 1 second

Exporting campaign account CSVs from VTiger using SQL

Uli Köhler Databases

Update: Related post for exporting contacts instead of accounts

Although VTiger works great as CRM, its campaign management feature is missing one of most important features: It doesn’t support exporting

This post aims to provide technically oriented users a guide on how to use SQL queries to perform that task. We assume basic knowledge of SQL syntax, MySQL and PHPMyAdmin.

The goal for our example is to export a table of Account name and E-Mail. For the email, the primary email will be used, unless it’s empty in which case the secondary email is used.

The input for the query is the campaign name (in our example, this will be MyCampaign).

Querying the campaign number

The campaign number is a string, e.g. CAM4 or CAM9 which is stored in vtiger_campaign.

SELECT campaign_no FROM vtiger_campaign WHERE `campaignname` = "MyCampaign"

 Finding the campaign record ID

In the campaign account relations n-to-n relations table, the campaign number is not used directly – instead, the campaign record ID is used. We need to query that from vtiger_modtracker_detail, which contain metadata key-value-data about the campaigns (among other things). One of the K-V pairs contain the campaign identifier (key campaign_no) whereas another one contains the record ID (key record_id).

SELECT postvalue FROM `vtiger_modtracker_detail` WHERE `id` = (SELECT `id` FROM `vtiger_modtracker_detail` WHERE postvalue = <<campaign number>> and fieldname = "campaign_no") AND fieldname = "record_id"

Full query:

SELECT postvalue FROM `vtiger_modtracker_detail` WHERE `id` = (SELECT `id` FROM `vtiger_modtracker_detail` WHERE postvalue = (SELECT campaign_no FROM vtiger_campaign WHERE `campaignname` = "MyCampaign") and fieldname = "campaign_no") AND fieldname = "record_id"

The result of this query is the numeric campaign record ID, e.g. 315.

Querying campaign-to-account relations

We can now use the campaign record ID to query the n-to-n table vtiger_campaignaccountrel. There are other, similar tables for leads (vtiger_campaignleadrel) and contacts (vtiger_campaigncontrel).

SELECT accountid FROM vtiger_campaignaccountrel WHERE `campaignid` = <<campaign record ID>>

Full query:

SELECT accountid FROM vtiger_campaignaccountrel WHERE `campaignid` = (SELECT postvalue FROM `vtiger_modtracker_detail` WHERE fieldname = "record_id" AND `id` = (SELECT `id` FROM `vtiger_modtracker_detail` WHERE postvalue = (SELECT campaign_no FROM vtiger_campaign WHERE `campaignname` = "MyCampaign") and fieldname = "campaign_no"))

The result of this query is a list of account IDs related to the campaign. The order is usually the same as displayed in VTiger.

Querying account information

Using the list of account IDs, we can JOIN on vtiger_account to query the account details. Note that you need to use the appropriate table for leads or contacts if you used an alternative relations table in the previous step.

SELECT acc.accountname AS "Name", COALESCE(acc.email1, acc.email2) AS "EMail" FROM vtiger_campaignaccountrel AS car JOIN vtiger_account acc ON (acc.accountid = car.accountid) WHERE `campaignid` =(SELECT postvalue FROM `vtiger_modtracker_detail` WHERE fieldname = "record_id" AND `id` = (SELECT `id` FROM `vtiger_modtracker_detail` WHERE postvalue = (SELECT campaign_no FROM vtiger_campaign WHERE `campaignname` = "MyCampaign") and fieldname = "campaign_no")) AND (acc.email1 <> '' OR acc.email2 <> '')

Note that besides the SELECT column list there are some aspects which are specific to the Name and Email query:

  1. COALESCE(acc.email1, acc.email2) uses email2 if email1 is empty
  2. AND (email1 <> '' OR email2 <> '') ignores accounts that have neither email1 nor email2 set

If you want a custom list of columns, you need to modify that appropriately.

Exporting CSV

When entering the query into a tool like PHPMyAdmin, you can click the Export button in order to export the result to different formats, including CSV and Excel.

While this might not be the most convenient option, it’s a very flexible way of generating and exporting the data. Any more convenient ways are likely to be less flexible.

Fixing PPA Unable to identify ‚package‘: user@mycomputer in launchpad

Uli Köhler Linux

Problem:

You’ve uploaded a DEB package to a Launchpad PPA (e.g. using dput), but you get an error message similar to this:

Solution:

You need to use a proper email address (which must be registered in Launchpad) in debian/changelog .

In order to do this, set the $DEBEMAIL environment variable before running dch

Example:

export DEBEMAIL=myemail@example.com
dch [...]

If $DEBEMAIL is not set, [username]@[hostname] will be used

Graph layouting via Graphviz

Yann Spöri Allgemein

Problem:

You want to display a Graph.

Solution:

Create a simple text file describing your graph and save it with a .dot file extension:

graph {
node1 -- node2;
node2 -- node3;
node3 -- node4;
node4 -- node1;
}

Afterwards you can use a program of the graphviz package (sudo apt-get install graphviz) in order to visualize the graph. This package contains different layouting programs like dot, neato, fdp (all from the GraphViz project) etc. Simply call one of these programs in order to visualize the graph:

neato -Tsvg yourFile.dot -o outputFile.svg

Output from this command: