%%sitename%%

Aug

2019

Javascript equivalent to Python’s re.findall()

The equivalent to this Python code which is using re.findall()

import re

hashtag_regex = r"(\B#\w\w+)"
hits = re.findall(hashtag_regex, "This is a string #with #hashtags")
print(hits) # prints ['#with', '#hashtags']

in Javascript is

const string = "This is a string #with #hashtags";
const re = /(\B#\w\w+)/g;
const hits = [];
// Iterate hits
let match = null;
do {
    match = re.exec(string);
    if(match) {
        hits.push(match[0]);
    }
} while (match);

console.log(hits); // Prints [ '#with', '#hashtags' ]

You need to assign your regular expression to a variable like re! If you use

match = /(\B#\w\w+)/g.exec(string); // WRONG ! Don't do this !

you will create an infinite loop which always generates the first hit in the string, if any!

Aug

2019

How to fix Puppeteer ‘Error: Protocol error (Emulation.setDeviceMetricsOverride): Invalid parameters width: integer value expected; height: integer value expected’

Problem:

You are trying to set the viewport size in Puppeteer using

const browser = await puppeteer.launch({
    defaultViewport: '1920x1080'
});

but you see an error message like

(node:15692) UnhandledPromiseRejectionWarning: Error: Protocol error (Emulation.setDeviceMetricsOverride): Invalid parameters width: integer value expected; height: integer value expected
    at /home/uli/dev/myproject/node_modules/puppeteer/lib/Connection.js:183:56
    at new Promise (<anonymous>)
    at CDPSession.send (/home/uli/dev/myproject/node_modules/puppeteer/lib/Connection.js:182:12)
    at EmulationManager.emulateViewport (/home/uli/dev/myproject/node_modules/puppeteer/lib/EmulationManager.js:41:20)
    at Page.setViewport (/home/uli/dev/myproject/node_modules/puppeteer/lib/Page.js:808:54)
    at Page.<anonymous> (/home/uli/dev/myproject/node_modules/puppeteer/lib/helper.js:112:23)
    at Function.create (/home/uli/dev/myproject/node_modules/puppeteer/lib/Page.js:49:18)
    at processTicksAndRejections (internal/process/task_queues.js:85:5)
    at async Browser._createPageInContext (/home/uli/dev/myproject/node_modules/puppeteer/lib/Browser.js:177:18)
    at async /home/uli/dev/myproject/index.js:8:16
  -- ASYNC --
    at Target.<anonymous> (/home/uli/dev/myproject/node_modules/puppeteer/lib/helper.js:111:15)
    at Browser._createPageInContext (/home/uli/dev/myproject/node_modules/puppeteer/lib/Browser.js:177:31)
    at processTicksAndRejections (internal/process/task_queues.js:85:5)
    at async /home/uli/dev/myproject/index.js:8:16
  -- ASYNC --
    at Browser.<anonymous> (/home/uli/dev/myproject/node_modules/puppeteer/lib/helper.js:111:15)
    at /home/uli/dev/myproject/index.js:8:30
    at processTicksAndRejections (internal/process/task_queues.js:85:5)
(node:15692) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:15692) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

Solution:

You need to use

{defaultViewport: {width: 1920, height: 1080}}

instead of

{defaultViewport: '1920x1080'}

Full example:

// Minimal puppeteer example
const puppeteer = require('puppeteer');
(async () => {
  const browser = await puppeteer.launch({
      defaultViewport: {width: 1920, height: 1080}
  });
  const page = await browser.newPage();
  await page.goto('https://techoverflow.net', {waitUntil: 'domcontentloaded'});
  // Make a screenshot
  await page.screenshot({path: 'screenshot.png'});
  await browser.close();
})();

Aug

2019

How to set screenshot size in Puppeteer?

Problem:

You are trying to make a screenshot using Puppeteer like this:

await page.screenshot({path: 'screenshot.png'});

but the screenshot is always 800×600 pixels. How can you increase the size of the screenshot?

Solution:

You can’t set the size for the screenshot, you need to set the size for the viewport in puppeteer.launch:

const browser = await puppeteer.launch({
    defaultViewport: {width: 1920, height: 1080}
});

Full example to produce a 1920x1080 screenshot:

const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch({
      defaultViewport: {width: 1920, height: 1080}
  });
  const page = await browser.newPage();
  await page.goto('https://techoverflow.net', {waitUntil: 'domcontentloaded'});
  // Wait until page has loaded completely
  // Make a screenshot
  await page.screenshot({path: 'screenshot.png'});

  await browser.close();
})();

Aug

2019

Puppeteer minimal example

You can use this example as a starting point for your puppeteer application.

// Minimal puppeteer example
const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch();
  const page = await browser.newPage();
  await page.goto('https://techoverflow.net', {waitUntil: 'domcontentloaded'});
  //
  // YOUR CODE GOES HERE!
  //
  await browser.close();
})();

The screenshot is 800x600px by default and could look like this:

Aug

2019

Pure Javascript equivalent to jQuery $(window).load()

The pure Javascript equivalent to

$(window).load(function() {
    // Your code goes here!

})

window.addEventListener("load", function() { 
    // Your code goes here!
});

Note that it’s not supported by IE8, but that should not be an issue in 2019.

Aug

2019

What is a SELV power supply?

SELV means Safety extra low voltage.

This means that

the voltage at the output of the power supply is so low that is isn’t considered a safety risk (less than 60V DC or 35VAC).
the secondary side is isolated from the primary side by double or reinforced insulation, so that the output terminals can’t become electrically dangerous by short-circuiting to the primary side (which is connected e.g. to 230VAC mains)
the secondary side is isolated from earth so if there are faults in other devices connected to the same earth conductor, the output of the SELV power supply can’t get unsafe.

What does 60V DC mean in practice?

Typically the 60V value is defined as ripple-free DC. This means that the peak value of the waveform is not more than 10% higher than the maximum allowed voltage, e.g. it must not be more than 70V for a 60V system.

For 35VAC, the peak value has to be considered.

The values are typically measured across a 50kΩ resistance, refer e.g. to IEC61347-1 Annex A (the standard for safety of power supply for lighting purposes).

What does double or reinforced insulation mean?

This is an insulation that is so thick that even under extreme, abnormal conditions (like power surges due to lightning) it will not cause a discharge through the isolation.

This can be achieved by either using double insulation, which is just two layers of basic (non-reinforced) insulation, or using a much thicker type of insulation, reinforced isolation. In almost all cases, using reinforced isolation is more economical than using double insulation.

While this depends on the material, typically 0.4mm of insulating plastic foil is sufficient to count as reinforced insulation, however most standards require the insulator in use to be tested for its safety.

Can the primary side be connected to Earth?

Yes, the primary side may or may not be connected to Earth (as a third conductor). As long as the secondary side is not connected to earth, this does not affect the SELV rating, however note that this might have other implications since a power supply with primary connected to Earth is a Class I power supply whereas a power supply without any Earth connection is a Class II power supply.

There are many differences in how Class I and Class II power supplies are treated, so you need to check your applicable standards for details.

Aug

2019

How to make strcmp/strncmp case-insensitive in C/C++

To make strcmp case-insensitive, use strcasecmp from #include <strings.h>. strcasecmp can be used in exactly the same way as strcmp.

To make strncmp case-insensitive, use strncasecmp from #include <strings.h>. strncasecmp can be used in exactly the same way as strncmp.

Note that both of these will not deal with unicode characters correctly, but will work just fine in most applications.

Aug

2019

What is a transformer construction according to clause 26.2.4.1 of IEC 61558-1?

In transformer specifications or tests, you will often find sentences like

Construction according to clause 26.2.4.1 of IEC 61558-1

This clause refers to hermetically sealed construction by impregnation potting (even if only parts of the transformer are potted), i.e. the transformer is (partially) filled with epoxy or similar potting materials in order to prevent moisture or dust from influencing its performance.

This usually means:

the transformer can be smaller since the safety regulations (IEC61558-1) requires less creepage distances compared to non-potted transformers
the transformer manufacturer has to perform additional tests according to IEC61558-1 (compared to non-potted transformers) to prove that the potting material provides sufficient isolation
the transformer is typically more expensive than an equivalent non-potted transformer since it needs to be potted in the factory and additional safety tests need to be performed on transformer specimens.

Also see What is a transformer construction according to clause 19.12.3 of IEC 61558-1?

Aug

2019

What is a transformer construction according to clause 19.12.3 of IEC 61558-1?

In transformer specifications or tests, you will often find sentences like

Construction according to clause 19.12.3 of IEC 61558-1

This clause refers to isolated winding wire construction, i.e. a way of making a transformer where the isolation of the winding wires themselves is the only isolation.

This usually means:

that the winding wires will have multiple layers of isolation (usually two or three to satisfy safety requirements from IEC61558-1)
that there is no additional isolation foil since the isolation of the wound wires is sufficient to fulfil the safety requirements
that manufacturer will need to perform additional safety tests for each and every transformer (100% production test) to prove that the transformer is safe.

Also see What is a transformer construction according to clause 26.2.4.1 of IEC 61558-1?

Aug

2019

How to intercept AJAX JSON response in Pyppeteer

This example shows you how to intercept and print the content of a JSON response requested via any AJAX request on a web page by using Pyppeteer:

import asyncio
import json
from pyppeteer import launch

async def intercept_network_response(response):
    # In this example, we care only about responses returning JSONs
    if "application/json" in response.headers.get("content-type", ""):
        # Print some info about the responses
        print("URL:", response.url)
        print("Method:", response.request.method)
        print("Response headers:", response.headers)
        print("Request Headers:", response.request.headers)
        print("Response status:", response.status)
        # Print the content of the response
        try:
            # await response.json() returns the response as Python object
            print("Content: ", await response.json())
        except json.decoder.JSONDecodeError:
            # NOTE: Use await response.text() if you want to get raw response text
            print("Failed to decode JSON from", await response.text())

async def main():
    browser = await launch()
    page = await browser.newPage()
    
    page.on('response', intercept_network_response)
            
    await page.goto('https://instagram.com')
    await browser.close()

asyncio.get_event_loop().run_until_complete(main())

Aug

2019

Pyppeteer minimal network response interception example

Using Javascript (puppeteer)? Check out Minimal puppeteer response interception example

This example shows you how to intercept network responses in pyppeteer.

Note: This intercepts the response, not the request! This means you can abort the request before it is actually sent to the server, but you can’t read the content of the response! See Pyppetteer minimal network request interception example for an example on how to intercept requests.

import asyncio
from pyppeteer import launch

async def intercept_network_response(response):
    # In this example, we only care about HTML responses!
    if "text/html" in response.headers.get("content-type", ""):
        # Print some info about the responses
        print("URL:", response.url)
        print("Method:", response.request.method)
        print("Response headers:", response.headers)
        print("Request Headers:", response.request.headers)
        print("Response status:", response.status)
        # Print the content of the response
        print("Content: ", await response.text())
        # NOTE: Use await response.json() if you want to get the JSON directly

async def main():
    browser = await launch()
    page = await browser.newPage()
    
    page.on('response', intercept_network_response)
            
    await page.goto('https://techoverflow.net')
    await browser.close()

asyncio.get_event_loop().run_until_complete(main())

Aug

2019

How to fix Python NameError: name ‘reponse’ is not defined

If you see this error message in Python:

NameError: name 'reponse' is not defined

you likely just mis-spelled response !

reponse  <-- your code, you're missing an 's'!
response <-- correct

Aug

2019

Pyppeteer minimal network request interception example

Using Javascript (puppeteer)? Check out Minimal puppeteer request interception example

This example shows you how to intercept network requests in pyppeteer:

Note: This intercepts the request, not the response! This means you can abort the request made, but you can’t read the content of the response! See Pyppetteer minimal network response interception example for an example on how to intercept responses.

import asyncio
from pyppeteer import launch

async def intercept_network_request(request):
    # Print some info about the request
    print("URL:", request.url)
    print("Method:", request.method)
    print("Headers:", request.headers)
    # NOTE: You can also await request.abort() to abort the requst1
    await request.continue_()

async def main():
    browser = await launch()
    page = await browser.newPage()
    await page.setRequestInterception(True)
    
    page.on('request', intercept_network_request)
            
    await page.goto('https://techoverflow.net')
    await browser.close()

asyncio.get_event_loop().run_until_complete(main())

Aug

2019

How to fix pyppeteer.errors.NetworkError: Request interception is not enabled.

Note: Also see Pyppetteer minimal network request interception example

Problem:

You are trying to intercept a request in Pyppeteer using

page.on('request', my_intercept_request)

but you’re getting an error message like this:

Traceback (most recent call last):
  File "/usr/lib/python3.6/asyncio/events.py", line 145, in _run
    self._callback(*self._args)
  File "/usr/local/lib/python3.6/dist-packages/pyee/_compat.py", line 62, in _callback
    self.emit('error', exc)
  File "/usr/local/lib/python3.6/dist-packages/pyee/_base.py", line 106, in emit
    self._emit_handle_potential_error(event, args[0] if args else None)
  File "/usr/local/lib/python3.6/dist-packages/pyee/_base.py", line 83, in _emit_handle_potential_error
    raise error
  File "run.py", line 6, in intercept
    await request.continue_()
  File "/usr/local/lib/python3.6/dist-packages/pyppeteer/network_manager.py", line 481, in continue_
    raise NetworkError('Request interception is not enabled.')
pyppeteer.errors.NetworkError: Request interception is not enabled.

Solution:

Add

await page.setRequestInterception(True)

directly after your call to

page = await browser.newPage()

This will enable request interception and your code will run just fine.

Aug

2019

How to fix PyVISA “No module named ‘serial.tools'”

Problem:

You want to use an ASRL (serial) instrument in PyVISA, but when you run

python3 -m visa info

you get this output even though you have serial installed:

ASRL INSTR:
   Please install PySerial (>=3.0) to use this resource type.
   No module named 'serial.tools'

Solution:

You have installed serial but you need to install pyserial – they are not the same!

First you need to remove the system package python3-serial if installed. Example for Ubuntu/Debian:

sudo apt remove python3-serial

and also remove the pip serial package if installed

sudo pip3 uninstall serial

Then install pyserial:

sudo pip3 install pyserial

You can check if PySerial is installed properly using

python3 -m visa info

It should show you

ASRL INSTR: Available via PySerial (3.4)

once pyserial is installed correctly!

Note: The commands above are for Python 3.x. In case you are still using Python 2.x use pip2 or pip instead of pip3 and use python-serial instead of python3-serial as APT package name.

Aug

2019

How to fix PyVISA not finding any ASRL (serial port) instruments

Problem:

You are trying to connect to a USB instrument using PyVISA & pyvisa-py, but the PyVISA resource manager doesn’t find any instruments:

#!/usr/bin/env python3
import visa
rm = visa.ResourceManager()
print(rm.list_resources()) # Prints "()" => No instruments found!

Solution:

Install PySerial 3.0+:

First you need to remove the system package python3-serial if installed. Example for Ubuntu/Debian:

sudo apt remove python3-serial

and also remove the pip serial package if installed (we need to install pyserial, not serial!)

sudo pip3 uninstall serial

Then install pyserial:

sudo pip3 install pyserial

You can check if PySerial is installed properly using

python3 -m visa info

It should show you

ASRL INSTR: Available via PySerial (3.4)

if pyserial is installed correctly!

Note: The commands above are for Python 3.x. In case you are still using Python 2.x use pip2 or pip instead of pip3 and use python-serial instead of python3-serial as APT package name.

Aug

2019

How to fix PyVISA ‘Found a device whose serial number cannot be read. The partial VISA resource name is: USB0::[…]::[…]::???::0::INSTR’

Problem:

You are trying to list available resources using PyVISA e.g. using

import visa
rm = visa.ResourceManager()
print(rm.list_resources())

But when you try to run it, you see an output like

Found a device whose serial number cannot be read. The partial VISA resource name is: USB0::6833::3601::???::0::INSTR
()

Solution:

Even though PyVISA doesn’t tell you that exactly, this is just the bog-standard Linux USB permission problem. We already provided a generic solution in How to fix ALL USB permission issues on Linux once and for all.

Excerpt from this post (see there for details on why it works):

Run this in your favourite shell:

wget https://techoverflow.net/scripts/udev-install-usbusers.sh | sudo bash -s $USER

This will print:

SUBSYSTEM=="usb", MODE="0666", GROUP="usbusers"
USB device configuration has been installed. Please log out and log back in or reboot

then log out and log back in (or close your SSH session and log back in).

In case this doesn’t work, reboot!

After that, your PyVISA script should work as intended and should print e.g.

('USB0::6833::3601::DL3A204800938::0::INSTR')

Aug

2019

How to fix ALL USB permission issues on Linux once and for all

On Linux users often have the issue that normal users can’t access some USB devices while root can. Most pages on the internet try to address this issue individually for each device, but most users don’t need that granularity, they just want it to work.

This post provides a method

Installation

Run this in your favourite shell:

wget https://techoverflow.net/scripts/udev-install-usbusers.sh | sudo bash -s $USER

This will print:

SUBSYSTEM=="usb", MODE="0666", GROUP="usbusers"
USB device configuration has been installed. Please log out and log back in or reboot

then log out and log back in (or close your SSH session and log back in).

In case this doesn’t work, reboot!

How it works

It creates a group called usbusers
It adds your user ($USER) to the usbusers group. You might need to sudo usermod -a -G usbusers $USER for additional users that should have access to USB devices!
Then it creates an udev config file /etc/udev/rules.d/99-usbusers.rules with the following content:
```
SUBSYSTEM=="usb", MODE="0666", GROUP="usbusers"
```
It then tries to reload & trigger udev using udevadm. This usually means you don’t have to reboot

In effect, it sets the group to usbusers for every USB device, no matter what type and ensures the group has write access. This is why this solution is so generic – it’s not limited to a specific type of USB device.

Aug

2019

How to pad string with zeroes & right-justify in bash using sed

Pad any string using this sed command:

sed -e :a -e 's/^.\{1,3\}$/0&/;ta'

This pads every string to length 4, usage example:

$ echo x | sed -e :a -e 's/^.\{1,3\}$/0&/;ta'
000x

In case you want to pad to a different length, replace 3 in the script by (your desired length - 1).

You can also use a bash function like this:

# Zero pad to length 4, right-justified
function zero_pad4 { echo $1 | sed -e :a -e 's/^.\{1,3\}$/0&/;ta' ; }

Usage example:

$ zero_pad4 1x
001x

Original source, modified: The Unix School

Aug

2019

How to fix PyVISA not finding any USB instruments

Problem:

You are trying to connect to a USB instrument using PyVISA & pyvisa-py, but the PyVISA resource manager doesn’t find any instruments:

#!/usr/bin/env python3
import visa
rm = visa.ResourceManager()
print(rm.list_resources()) # Prints "()" => No instruments found!

Solution:

In order for pyvisa-py to be able to connect to USB instruments, you need to install the Python usb library!

On Debian or Ubuntu, install it using

sudo apt-get -y install python3-usb

or, if you are still using Python 2.x

sudo apt-get -y install python-usb

Now, re-run the script – you should see an output like

('USB0::6833::3601::DL3A204800938::0::INSTR',)

In case you still don’t see the output, run python3 -m visa info or python -m visa info (for Python 2.x).

It should show an output like this:

Machine Details:
   Platform ID:    Linux-4.19.0-5-686-i686-with-debian-10.0
   Processor:      

Python:
   Implementation: CPython
   Executable:     /usr/bin/python3
   Version:        3.7.3
   Compiler:       GCC 8.3.0
   Bits:           32bit
   Build:          Apr  3 2019 05:39:12 (#default)
   Unicode:        UCS4

PyVISA Version: 1.9.1

Backends:
   ni:
      Version: 1.9.1 (bundled with PyVISA)
      Binary library: Not found
   py:
      Version: 0.3.1
      ASRL INSTR: Available via PySerial (3.4)
      USB INSTR: Available via PyUSB (1.0.2). Backend: libusb1
      USB RAW: Available via PyUSB (1.0.2). Backend: libusb1
      TCPIP INSTR: Available 
      TCPIP SOCKET: Available 
      GPIB INSTR:
         Please install linux-gpib to use this resource type.
         No module named 'gpib'

Check Backends -> py -> USB INSTR: If it’s not Available via PyUSB, check the information message for hints what might be the issue. For example, if it says

USB INSTR:
   Please install PyUSB to use this resource type.
   No module named 'usb'

that means that the Python USB library has not been installed properly.

If USB is Available via PyUSB but PyVISA still doesn’t find the instrument, check if it is connected properly using

lsusb

which should show a line related to your instrument’s manufacturer, e.g.

Bus 001 Device 002: ID 1ab1:0e11 Rigol Technologies

Also unplug and re-plug your instrument so Linux tries to reconnect to the USB device and check the end of the output of sudo dmesg which could list e.g.

[19427.230120] usb 1-2: new high-speed USB device number 2 using ehci-pci
[19427.425464] usb 1-2: config 1 interface 0 altsetting 0 bulk endpoint 0x82 has invalid maxpacket 64
[19427.425469] usb 1-2: config 1 interface 0 altsetting 0 bulk endpoint 0x3 has invalid maxpacket 64
[19427.425947] usb 1-2: New USB device found, idVendor=1ab1, idProduct=0e11, bcdDevice= 0.02
[19427.425950] usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[19427.425953] usb 1-2: Product: DL3000 Serials
[19427.425955] usb 1-2: Manufacturer: Rigol Technologies. 
[19427.425957] usb 1-2: SerialNumber: DL3A204800938
[19429.525745] usbcore: registered new interface driver usbtmc

usbtmc in the last line means that the USB device has been recognized as USB Test & Measurement class device, and hence you should be able to connect to it using PyVISA as USB INSTR.

Javascript equivalent to Python’s re.findall()

How to fix Puppeteer ‘Error: Protocol error (Emulation.setDeviceMetricsOverride): Invalid parameters width: integer value expected; height: integer value expected’

Problem:

Solution:

How to set screenshot size in Puppeteer?

Problem:

Solution:

Puppeteer minimal example

Pure Javascript equivalent to jQuery $(window).load()

What is a SELV power supply?

What does 60V DC mean in practice?

Can the primary side be connected to Earth?

How to make strcmp/strncmp case-insensitive in C/C++

What is a transformer construction according to clause 26.2.4.1 of IEC 61558-1?

What is a transformer construction according to clause 19.12.3 of IEC 61558-1?

How to intercept AJAX JSON response in Pyppeteer

Pyppeteer minimal network response interception example

How to fix Python NameError: name ‘reponse’ is not defined

Pyppeteer minimal network request interception example

How to fix pyppeteer.errors.NetworkError: Request interception is not enabled.

Problem:

Solution:

How to fix PyVISA “No module named ‘serial.tools'”

Problem:

Solution:

How to fix PyVISA not finding any ASRL (serial port) instruments

Problem:

Solution:

How to fix PyVISA ‘Found a device whose serial number cannot be read. The partial VISA resource name is: USB0::[…]::[…]::???::0::INSTR’

Problem:

Solution:

How to fix ALL USB permission issues on Linux once and for all

Installation

How it works

How to pad string with zeroes & right-justify in bash using sed

How to fix PyVISA not finding any USB instruments

Problem:

Solution:

Search

Categories