Programming languages

Aug

2019

Pyppeteer minimal network request interception example

Using Javascript (puppeteer)? Check out Minimal puppeteer request interception example

This example shows you how to intercept network requests in pyppeteer:

Note: This intercepts the request, not the response! This means you can abort the request made, but you can’t read the content of the response! See Pyppetteer minimal network response interception example for an example on how to intercept responses.

import asyncio
from pyppeteer import launch

async def intercept_network_request(request):
    # Print some info about the request
    print("URL:", request.url)
    print("Method:", request.method)
    print("Headers:", request.headers)
    # NOTE: You can also await request.abort() to abort the requst1
    await request.continue_()

async def main():
    browser = await launch()
    page = await browser.newPage()
    await page.setRequestInterception(True)
    
    page.on('request', intercept_network_request)
            
    await page.goto('https://techoverflow.net')
    await browser.close()

asyncio.get_event_loop().run_until_complete(main())

Aug

2019

How to fix pyppeteer.errors.NetworkError: Request interception is not enabled.

Note: Also see Pyppetteer minimal network request interception example

Problem:

You are trying to intercept a request in Pyppeteer using

page.on('request', my_intercept_request)

but you’re getting an error message like this:

Traceback (most recent call last):
  File "/usr/lib/python3.6/asyncio/events.py", line 145, in _run
    self._callback(*self._args)
  File "/usr/local/lib/python3.6/dist-packages/pyee/_compat.py", line 62, in _callback
    self.emit('error', exc)
  File "/usr/local/lib/python3.6/dist-packages/pyee/_base.py", line 106, in emit
    self._emit_handle_potential_error(event, args[0] if args else None)
  File "/usr/local/lib/python3.6/dist-packages/pyee/_base.py", line 83, in _emit_handle_potential_error
    raise error
  File "run.py", line 6, in intercept
    await request.continue_()
  File "/usr/local/lib/python3.6/dist-packages/pyppeteer/network_manager.py", line 481, in continue_
    raise NetworkError('Request interception is not enabled.')
pyppeteer.errors.NetworkError: Request interception is not enabled.

Solution:

Add

await page.setRequestInterception(True)

directly after your call to

page = await browser.newPage()

This will enable request interception and your code will run just fine.

Aug

2019

How to fix PyVISA “No module named ‘serial.tools'”

Problem:

You want to use an ASRL (serial) instrument in PyVISA, but when you run

python3 -m visa info

you get this output even though you have serial installed:

ASRL INSTR:
   Please install PySerial (>=3.0) to use this resource type.
   No module named 'serial.tools'

Solution:

You have installed serial but you need to install pyserial – they are not the same!

First you need to remove the system package python3-serial if installed. Example for Ubuntu/Debian:

sudo apt remove python3-serial

and also remove the pip serial package if installed

sudo pip3 uninstall serial

Then install pyserial:

sudo pip3 install pyserial

You can check if PySerial is installed properly using

python3 -m visa info

It should show you

ASRL INSTR: Available via PySerial (3.4)

once pyserial is installed correctly!

Note: The commands above are for Python 3.x. In case you are still using Python 2.x use pip2 or pip instead of pip3 and use python-serial instead of python3-serial as APT package name.

Aug

2019

How to fix PyVISA not finding any ASRL (serial port) instruments

Problem:

You are trying to connect to a USB instrument using PyVISA & pyvisa-py, but the PyVISA resource manager doesn’t find any instruments:

#!/usr/bin/env python3
import visa
rm = visa.ResourceManager()
print(rm.list_resources()) # Prints "()" => No instruments found!

Solution:

Install PySerial 3.0+:

First you need to remove the system package python3-serial if installed. Example for Ubuntu/Debian:

sudo apt remove python3-serial

and also remove the pip serial package if installed (we need to install pyserial, not serial!)

sudo pip3 uninstall serial

Then install pyserial:

sudo pip3 install pyserial

You can check if PySerial is installed properly using

python3 -m visa info

It should show you

ASRL INSTR: Available via PySerial (3.4)

if pyserial is installed correctly!

Note: The commands above are for Python 3.x. In case you are still using Python 2.x use pip2 or pip instead of pip3 and use python-serial instead of python3-serial as APT package name.

Aug

2019

How to fix PyVISA ‘Found a device whose serial number cannot be read. The partial VISA resource name is: USB0::[…]::[…]::???::0::INSTR’

Problem:

You are trying to list available resources using PyVISA e.g. using

import visa
rm = visa.ResourceManager()
print(rm.list_resources())

But when you try to run it, you see an output like

Found a device whose serial number cannot be read. The partial VISA resource name is: USB0::6833::3601::???::0::INSTR
()

Solution:

Even though PyVISA doesn’t tell you that exactly, this is just the bog-standard Linux USB permission problem. We already provided a generic solution in How to fix ALL USB permission issues on Linux once and for all.

Excerpt from this post (see there for details on why it works):

Run this in your favourite shell:

wget https://techoverflow.net/scripts/udev-install-usbusers.sh | sudo bash -s $USER

This will print:

SUBSYSTEM=="usb", MODE="0666", GROUP="usbusers"
USB device configuration has been installed. Please log out and log back in or reboot

then log out and log back in (or close your SSH session and log back in).

In case this doesn’t work, reboot!

After that, your PyVISA script should work as intended and should print e.g.

('USB0::6833::3601::DL3A204800938::0::INSTR')

Aug

2019

How to pad string with zeroes & right-justify in bash using sed

Pad any string using this sed command:

sed -e :a -e 's/^.\{1,3\}$/0&/;ta'

This pads every string to length 4, usage example:

$ echo x | sed -e :a -e 's/^.\{1,3\}$/0&/;ta'
000x

In case you want to pad to a different length, replace 3 in the script by (your desired length - 1).

You can also use a bash function like this:

# Zero pad to length 4, right-justified
function zero_pad4 { echo $1 | sed -e :a -e 's/^.\{1,3\}$/0&/;ta' ; }

Usage example:

$ zero_pad4 1x
001x

Original source, modified: The Unix School

Aug

2019

How to fix PyVISA not finding any USB instruments

Problem:

You are trying to connect to a USB instrument using PyVISA & pyvisa-py, but the PyVISA resource manager doesn’t find any instruments:

#!/usr/bin/env python3
import visa
rm = visa.ResourceManager()
print(rm.list_resources()) # Prints "()" => No instruments found!

Solution:

In order for pyvisa-py to be able to connect to USB instruments, you need to install the Python usb library!

On Debian or Ubuntu, install it using

sudo apt-get -y install python3-usb

or, if you are still using Python 2.x

sudo apt-get -y install python-usb

Now, re-run the script – you should see an output like

('USB0::6833::3601::DL3A204800938::0::INSTR',)

In case you still don’t see the output, run python3 -m visa info or python -m visa info (for Python 2.x).

It should show an output like this:

Machine Details:
   Platform ID:    Linux-4.19.0-5-686-i686-with-debian-10.0
   Processor:      

Python:
   Implementation: CPython
   Executable:     /usr/bin/python3
   Version:        3.7.3
   Compiler:       GCC 8.3.0
   Bits:           32bit
   Build:          Apr  3 2019 05:39:12 (#default)
   Unicode:        UCS4

PyVISA Version: 1.9.1

Backends:
   ni:
      Version: 1.9.1 (bundled with PyVISA)
      Binary library: Not found
   py:
      Version: 0.3.1
      ASRL INSTR: Available via PySerial (3.4)
      USB INSTR: Available via PyUSB (1.0.2). Backend: libusb1
      USB RAW: Available via PyUSB (1.0.2). Backend: libusb1
      TCPIP INSTR: Available 
      TCPIP SOCKET: Available 
      GPIB INSTR:
         Please install linux-gpib to use this resource type.
         No module named 'gpib'

Check Backends -> py -> USB INSTR: If it’s not Available via PyUSB, check the information message for hints what might be the issue. For example, if it says

USB INSTR:
   Please install PyUSB to use this resource type.
   No module named 'usb'

that means that the Python USB library has not been installed properly.

If USB is Available via PyUSB but PyVISA still doesn’t find the instrument, check if it is connected properly using

lsusb

which should show a line related to your instrument’s manufacturer, e.g.

Bus 001 Device 002: ID 1ab1:0e11 Rigol Technologies

Also unplug and re-plug your instrument so Linux tries to reconnect to the USB device and check the end of the output of sudo dmesg which could list e.g.

[19427.230120] usb 1-2: new high-speed USB device number 2 using ehci-pci
[19427.425464] usb 1-2: config 1 interface 0 altsetting 0 bulk endpoint 0x82 has invalid maxpacket 64
[19427.425469] usb 1-2: config 1 interface 0 altsetting 0 bulk endpoint 0x3 has invalid maxpacket 64
[19427.425947] usb 1-2: New USB device found, idVendor=1ab1, idProduct=0e11, bcdDevice= 0.02
[19427.425950] usb 1-2: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[19427.425953] usb 1-2: Product: DL3000 Serials
[19427.425955] usb 1-2: Manufacturer: Rigol Technologies. 
[19427.425957] usb 1-2: SerialNumber: DL3A204800938
[19429.525745] usbcore: registered new interface driver usbtmc

usbtmc in the last line means that the USB device has been recognized as USB Test & Measurement class device, and hence you should be able to connect to it using PyVISA as USB INSTR.

Aug

2019

How to fix PyVISA ‘ValueError: Could not locate a VISA implementation. Install either the NI binary or pyvisa-py.’

Problem:

You are trying to use PyVISA to connect to an instrument, but you see an error message like

Traceback (most recent call last):
  File "TestPyVISA.py", line 3, in <module>
    rm = visa.ResourceManager()
  File "/usr/local/lib/python3.7/dist-packages/pyvisa/highlevel.py", line 1526, in __new__
    visa_library = open_visa_library(visa_library)
  File "/usr/local/lib/python3.7/dist-packages/pyvisa/highlevel.py", line 1493, in open_visa_library
    wrapper = _get_default_wrapper()
  File "/usr/local/lib/python3.7/dist-packages/pyvisa/highlevel.py", line 1470, in _get_default_wrapper
    raise ValueError('Could not locate a VISA implementation. Install either the NI binary or pyvisa-py.')
ValueError: Could not locate a VISA implementation. Install either the NI binary or pyvisa-py.

Solution:

Install the pyvisa-py Python package:

sudo pip3 install pyvisa-py

or, if you are still using Python 2.x:

sudo pip2 install pyvisa-py

Aug

2019

How to fix pyaudio ‘fatal error: portaudio.h: No such file or directory’

Problem:

You are trying to install pyaudio using sudo pip install pyaudio or a similar command but you see an error message like

    src/_portaudiomodule.c:29:23: fatal error: portaudio.h: No such file or directory
     #include "portaudio.h"
                           ^
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-jgxnwixs/pyaudio/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-c3blzlv5-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-jgxnwixs/pyaudio/

Solution:

You need to install the portaudio library. On Debian/Ubuntu you can do that using

sudo apt install portaudio19-dev

on other systems either search for portaudio in your package manager or download the library from http://www.portaudio.com/

Aug

2019

What is ‘…’ in HTML?

… is the HTML entity code for the ellipsis dots:

…

If you see it on a webpage outside the HTML source code, it usually means that the author or content management system has encoded the HTML entity incorrectly. You should only ever see … on the page except in literal HTML sourcecode.

I recommend using the unicode character HORIZONTAL ELLIPSIS (see the code block above) instead of the HTML entity …. Ensure that you have

<meta charset="utf-8" />

in your HTML <head> for compatibility.

Aug

2019

How to fix LaTeX missing footnotes in tabular environment

Problem:

You have LaTeX code with a \footnote inside a \tabular like this:

\documentclass{scrartcl}

\begin{document}
\begin{tabular}{p{2cm}|p{2cm}|}
\textbf{Column A\footnote{This is a footnote!}} & \textbf{Column B}\\
1 & 2\\
3 & 4\\
\end{tabular}
\end{document}

but LaTeX/PDFLaTeX is not showing any of the footnotes you declared inside the \tabular environment.

Solution:

Use this code just after your \documentclass declaration:

\usepackage{footnote}
\makesavenoteenv{tabular}

Then recompile your LaTeX code. The footnotes inside your tabular should now appear as expected.

Note that if you are using other environments than \begin{tabular} you might need to add more \makesavenoteenv declarations for the correct environments. The tabularx environment from the tabularx package works with footnotes out-of-the box without any additional packages!

Jul

2019

How to fix LaTeX \glqq or \grqq error ‘Undefined control sequence’

Problem:

You have LaTeX code containing \glqq and/or \grqq like

\documentclass{scrartcl}

\begin{document}
\glqq Test\grqq
\end{document}

but when you try to compile it, you see an error message like this:

! Undefined control sequence.
l.5 \glqq
          Test\grqq
?

Solution:

In order to use \glqq or \grqq you need to include

\usepackage[ngerman]{babel}

after your \documentclass declaration.

In case you want to use another language, use the correct language specifier instead of ngerman.

Jul

2019

How to create a self-deleting temporary directory in Python

Starting from Python 3.2 your can use tmpfile.TemporaryDirectory like this:

from tempfile import TemporaryDirectory

with TemporaryDirectory(prefix="myapp-") as tmpdir:
    print(tmpdir) # e.g. "/tmp/myapp-fevhzh93"

# Once you are outside the "with" block, the directory will be deleted!

In case you are stuck with using older Python versions, check out How to create temporary directory in Python.

Jul

2019

How to insert € symbol in LaTeX using UTF8

Note: This is the recommended method for using € and other symbols in LaTeX!

In order to insert the € symbol into your LaTeX document, first add

\usepackage[official]{eurosym}
\usepackage[utf8x]{inputenc}

to the top of your document, e.g. directly after your \documentclass line.

After that, you can use the normal € symbol anywhere in your document like this:

€

to insert an € symbol.

Jul

2019

How to insert € symbol in LaTeX using the eurosym package

Note: This method is no longer recommended and only provided for compatibility reasons! See How to insert Euro symbol in LaTeX using UTF8 for the recommended method instead!

In order to insert the € symbol into your LaTeX document, first add

\usepackage[official]{eurosym}

to the top of your document, e.g. directly after your \documentclass line.

After that, you can use

\euro{}

to insert an € symbol anywhere in your document.

Jul

2019

How to reduce page margins in LaTeX using the geometry package

To set or reduce the page margin (white space between the border of the page and the text) you can use the geometry package.

Set margin of all sides to 2cm:

\usepackage[left=20mm, right=20mm, top=20mm, bottom=20mm]{geometry}

To set just the left and right margin to 2cm, use

\usepackage[left=20mm, right=20mm]{geometry}

Jul

2019

How to convert a DVI file to SVG on the command line

If you want to convert my.dvi to my.svg, use this command

dvi2ps my.dvi | ps2eps - > my.eps && eps2svg my.eps

This produces my.svg – note that if my.svg already exists, eps2svg will create my_1.svg, my_2.svg and so on and will not overwrite my.svg!

You can also use this shell function:

function dviToSVG { dvi2ps "$1" | ps2eps - > "${1%.*}.eps" && eps2svg "${1%.*}.eps" "${1%.*}.svg" ; }

Usage example:

dviToSVG my.dvi # Produces my.svg

Jul

2019

Minimal LaTeX booktabs example

This is a minimal example using the LaTeX booktabs package which you can use & modify to create your own table.

\documentclass{scrartcl}
\usepackage{booktabs}
\usepackage[utf8x]{inputenc}

\begin{document} 

\begin{table}[h!]
    \begin{center}
      \caption{Minimal booktabs example.}
      \label{tab:table1}
      \begin{tabular}{l|c|r}
        \toprule % <-- Toprule here
        \textbf{Column 1} & \textbf{Column 2} & \textbf{Column 3}\\
        $\alpha$ & $\beta$ & $\gamma$ \\
        \midrule % <-- Midrule here
        A & 10.23 & a\\
        B & 45.678 & b\\
        C & 99.987 & c\\
        \bottomrule % <-- Bottomrule here
      \end{tabular}
    \end{center}
  \end{table}
\end{document}

Render it using

pdflatex booktabs.tex

The table looks like this:

Jul

2019

How to fix LaTeX error File `siunitx.sty’ not found on Ubuntu

Problem:

You want to compile your LaTeX file on Ubuntu using latex or pdflatex but you see an error message like this:

! LaTeX Error: File `siunitx.sty' not found.

Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: sty)

Solution

sudo apt -y install texlive-science

This will install, among many other LaTeX files, siunitx.sty.

Jul

2019

Inductive reactance online calculator & Python code

Use this online calculator to compute the reactance of an inductor in Ω at a specific frequency given its inductance.

Also see Capacitive reactance online calculator & Python code

TechOverflow calculators:
You can enter values with SI suffixes like 12.2m (equivalent to 0.012) or 14k (14000) or 32u (0.000032).
The results are calculated while you type and shown directly below the calculator, so there is no need to press return or click on a Calculate button. Just make sure that all inputs are green by entering valid values.

H ⚠

Hz ⚠

Formula:

X_L = 2\pi fL

Python code:

The preferred way is to use UliEngineering’s UliEngineering.Electronics.Reactance.inductive_reactance:

from UliEngineering.Electronics.Reactance import *
# You can either pass strings like "150 uH" or values like 150e-6

inductive_reactance("150 uH", "10 MHz") # returns 9424.77796076938

Or get a human-readable value:

from UliEngineering.Electronics.Reactance import *
from UliEngineering.EngineerIO import *

# Compute value as a string
xc = auto_format(inductive_reactance, "150 uH", "10 MHz") # "9.42 kΩ"

# ... or print directly
auto_print(inductive_reactance, "150 uH", "10 MHz") # prints "9.42 kΩ"

In case you can’t use UliEngineering and you want to do it manually, here’s a minimal example:

import math
def inductive_reactance(f, l):
    """Compute the inductive reactance"""
    return 2*math.pi*f*l

Problem:

Solution:

Problem:

Solution:

Problem:

Solution:

Problem:

Solution:

Problem:

Solution:

Problem:

Solution:

Problem:

Solution:

Problem:

Solution:

Problem:

Solution:

Problem:

Solution

Formula:

Python code:

Search

Categories