To emulate a keypress to the Page up
key in Puppeteer, use
await page.keyboard.press("PageUp");
To emulate a keypress to the Page down
key in Puppeteer, use
await page.keyboard.press("PageDown");
To emulate a keypress to the Page up
key in Puppeteer, use
await page.keyboard.press("PageUp");
To emulate a keypress to the Page down
key in Puppeteer, use
await page.keyboard.press("PageDown");
Install argparse using
npm i --save argparse
Optional argument:
const ArgumentParser = require('argparse').ArgumentParser; const parser = new ArgumentParser({}); parser.addArgument(['-u', '--url'], {help: "The URL to use"}); const args = parser.parseArgs(); // Example usage console.log(args.url) // null if no such argument
If you want to make an argument mandatory, use required: true
:
const ArgumentParser = require('argparse').ArgumentParser; const parser = new ArgumentParser({}); parser.addArgument(['-u', '--url'], {help: "The URL to use", required: true}); const args = parser.parseArgs(); // Example usage console.log(args.url)
In case the user does not provide the argument, it will print
usage: CLI.js [-h] -u URL CLI.js: error: Argument "-u/--url" is required
This positional argument will always be required:
const ArgumentParser = require('argparse').ArgumentParser; const parser = new ArgumentParser({}); parser.addArgument(['url'], {help: "The URL to use"}); const args = parser.parseArgs(); // Example usage console.log(args.url)
When you try to run your puppetteer application, e.g. under docker, you see this error message:
Note: Unless you are running in a Docker or similar container, first consider running the application as non-root-user!
You have to pass the --no-sandbox
option to puppeteer.launch()
:
const browser = await puppeteer.launch({ headless: true, args: ['--no-sandbox'] });
We recommend to use this slightly more complex solution to pass it only if the process is being run as root
:
/** * Return true if the current process is run by the root user * https://techoverflow.net/2019/11/07/how-to-check-if-nodejs-is-run-by-root/ */ function isCurrentUserRoot() { return process.getuid() == 0; // UID 0 is always root } const browser = await puppeteer.launch({ headless: true, args: isCurrentUserRoot() ? ['--no-sandbox'] : undefined });
This ensures Chromium is run in the most secure mode possible with the current user.
Use this function to determine if the current NodeJS process is being run by the root
user:
function isCurrentUserRoot() { return process.getuid() == 0; // UID 0 is always root }
Usage example:
if(isCurrentUserRoot()) { // TODO Your code for root user goes here } else { // TODO Your code for NON-root user goes here! }
This works since the UID of the root
user is always 0
on Linux/Unix systems.
Note: You don’t need to
const process = require("process");
since the process
object is automatically imported in any NodeJS environment
You can use process.env.NODE_ENV
to check the value of the NODE_ENV
environment variable in Node.JS:
if(process.env.NODE_ENV == "development") { // TODO your code goes here }
or
if(process.env.NODE_ENV == "production") { // TODO your code goes here }
Note that by default NODE_ENV
is undefined
so remember to handle that case appropriately.
This is the minimal Koa.JS application which I use as a template for new NodeJS webserver applications.
#!/usr/bin/env node const router = require('koa-router')(); const koaBody = require('koa-body'); const Koa = require('koa'); const app = new Koa(); app.use(koaBody()); router.get('/', async ctx => { ctx.body = "Hello world"; }); app.use(router.routes()); if (!module.parent) app.listen(3000);
Install the requirements using
npm i --save koa koa-router koa-body
and run using
node index.js
assuming you have saved our code from above in index.js
.
Now (with the node index.js
command still running) go to http://localhost:3000 . You should see Hello world
there. Now it’s your turn to continue on your endeavour to develop the world’s greatest webservers 🙂
In NodeJS applications you often see code like
if (!module.parent) { app.listen(3000); }
This means: Run app.listen(3000)
only if you are running the file
Suppose this code is in index.js
. In this case, the code will only be executed if you run index.js
directly (i.e. using node index.js
) and not if index.js
is require
d from another file (by require('./index.js');
).
If index.js
is required from another Javascript module (i.e. file), module.parent
will be set to that module.
To emulate the user typing something on the keyboard, use
await page.keyboard.type("the text");
This will type the text extremely fast with virtually no delay between the characters.
In order to simulate the finite typing speed of real users, use
await page.keyboard.type("the text", {delay: 100});
instead. The delay betwen characters in this example is 100 Milliseconds, i.e. the emulated user types 10 characters per second.
You can take a screenshot in Puppeteer using
await page.screenshot({path: 'screenshot.png'});
The path is relative to the current working directory.
Want to have a screenshot in a size different to 800×600? See How to set screenshot size in Puppeteer?
Full example:
// Minimal puppeteer example const puppeteer = require('puppeteer'); (async () => { const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.goto('https://techoverflow.net', {waitUntil: 'domcontentloaded'}); // Screenshot to verify result await page.screenshot({path: 'screenshot.png'}); // Cleanup await browser.close(); })();
The easiest way to sleep using async/await
in NodeJS is the sleep-promise package:
npm install --save sleep-promise
Using that package you can simply use await sleep(milliseconds)
syntax like this:
const sleep = require('sleep-promise'); // In any async function: await sleep(2000); // Wait 2000 ms
Note that this sleep()
variant is fully asynchronous, IO and other asynchronous operations will still be able to continue in the background – sleep()
will not block the NodeJS process.
Full example:
const sleep = require('sleep-promise'); (async () => { console.log("This prints immediately"); await sleep(2000); console.log("This prints 2 seconds later"); })();
Install the sleep-promise package:
npm install --save sleep-promise
then you can use
const sleep = require('sleep-promise'); // In any async function: await sleep(2000); // Wait 2000 ms
Full example:
const sleep = require('sleep-promise'); (async () => { console.log("This prints immediately"); await sleep(2000); console.log("This prints 2 seconds later"); })();
You can use this example as a starting point for your puppeteer application.
// Minimal puppeteer example const puppeteer = require('puppeteer'); (async () => { const browser = await puppeteer.launch(); const page = await browser.newPage(); await page.goto('https://techoverflow.net', {waitUntil: 'domcontentloaded'}); // // YOUR CODE GOES HERE! // await browser.close(); })();
The screenshot is 800x600px by default and could look like this:
Run these shell commands on your Ubuntu computer to install NodeJS 14.x:
curl -sL https://deb.nodesource.com/setup_14.x | sudo -E bash - sudo apt-get install -y nodejs
Instead of setup_12.x
you can also choose other versions like setup_8.x
. However, using this method, you can’t install multiple versions of NodeJS in parallel.
You want to run a NodeJS app using
npm start
but you only see this error message:
npm ERR! missing script: start
You need to tell npm
what to do when you run npm start
explicitly by editing package.json
.
First, identify the main file of your application. Most often it is called index.js
, server.js
or app.js
. When you open package.json
in an editor, you can also often find a line like
"main": "index.js",
In this example, index.js
is the main file of the application.
Now we can edit package.json
to add a start
script.
In package.json
, find the "scripts"
section. If you have a default package.json
, it will look like this:
"scripts": { "test": "echo \"Error: no test specified\" && exit 1" },
Now add a comma to the end of the "test"
line and add this line after it:
"start": "node index.js"
and replace index.js
by the main file of your application (e.g. app.js
, server.js
, index.js
etc).
The "scripts"
section should now look like this:
"scripts": { "test": "echo \"Error: no test specified\" && exit 1", "start": "node index.js" },
Now save and close package.json
and run
npm start
to start your application.
You want to run a db.adminCommand()
in NodeJS using the node-mongodb-native client, e.g. you want to run the NodeJS equivalent of
db.adminCommand({setParameter: 1, internalQueryExecMaxBlockingSortBytes: 100151432});
Use conn.executeDbAdminCommand()
where db
is a MongoDB database object.
db.executeDbAdminCommand({setParameter: 1, internalQueryExecMaxBlockingSortBytes: 100151432});
// To install, use npm i --save mongodb const MongoClient = require('mongodb').MongoClient; async function configureMongoDB() { // Connect to MongoDB const conn = await MongoClient.connect('mongodb://localhost:27017/', { useNewUrlParser: true }); const db = await conn.db('mydb'); // Configure MongoDB settings await db.executeDbAdminCommand({ setParameter: 1, internalQueryExecMaxBlockingSortBytes: 100151432 }); // Cleanup return conn.close(); } // Run configureMongoDB() configureMongoDB().then(() => {}).catch(console.error)
You want to run a command like file my.pdf
using NodeJS child-process.exec
and get its stdout after it’s finished.
TL;DR: (await exec('file my.pdf')).stdout
We’re using child-process-promise here in order to simplify our implementation. Install it using npm i --save child-process-promise
!
const { exec } = require('child-process-promise'); async function run () { const ret = await exec(`file my.pdf`); return ret.stdout; } run().then(console.log).catch(console.error);
You can also use .stderr
instead of .stdout
to get the stderr
output as a string
Note: Also see How to install NodeJS 12.x on Ubuntu in 1 minute
Run these shell commands on your Ubuntu computer to install NodeJS 10.x:
curl -sL https://deb.nodesource.com/setup_10.x | sudo -E bash - sudo apt-get install -y nodejs
Instead of setup_10.x
you can also choose other versions like setup_8.x
. However, using this method, you can’t install multiple versions of NodeJS in parallel.
You are trying to build your webpack project, but you see an error message like this:
/home/uli/project/node_modules/webpack-cli/bin/config-yargs.js:89 describe: optionsSchema.definitions.output.properties.path.description, ^ TypeError: Cannot read property 'properties' of undefined at module.exports (/home/uli/project/node_modules/webpack-cli/bin/config-yargs.js:89:48) at /home/uli/project/node_modules/webpack-cli/bin/webpack.js:60:27 at Object.<anonymous> (/home/uli/project/node_modules/webpack-cli/bin/webpack.js:515:3) at Module._compile (internal/modules/cjs/loader.js:723:30) at Object.Module._extensions..js (internal/modules/cjs/loader.js:734:10) at Module.load (internal/modules/cjs/loader.js:620:32) at tryModuleLoad (internal/modules/cjs/loader.js:560:12) at Function.Module._load (internal/modules/cjs/loader.js:552:3) at Module.require (internal/modules/cjs/loader.js:659:17) at require (internal/modules/cjs/helpers.js:22:18)
This is a known bug in webpack 4.20.0
– you can circumvent this issue by using webpack 4.19.0
.
Look for a line like
"webpack": "^4.7.0",
in your package.json
. The caret (^
) allows npm to use any 4.x.x version – including the broken 4.20.0
.
Replace the aforementioned line by
"webpack": "4.19.0",
to use only webpack 4.19.0
.
After that, run npm install
and retry building your application.
NodeJS starting from version v9.x supports the ES6 Intl.DateTimeFormat
When you use it with the ‘en-US’ locale, it works properly and prints "August 13, 2018"
:
const df = new Intl.DateTimeFormat('en-US', {day: 'numeric', month: 'long', year: 'numeric', timeZone: 'UTC'}); console.log(df.format(new Date("2018-08-13T04:00:00.000Z")));
However, using it with a different locale fails:
const df = new Intl.DateTimeFormat('de', {day: 'numeric', month: 'long', year: 'numeric', timeZone: 'UTC'}); console.log(df.format(new Date("2018-08-13T04:00:00.000Z")));
While you would expect this to print "13. August 2018"
, it will print "2018 M08 13"
By default, NodeJS is only built with small-icu
support, thereby only installing the en-US
locale in order to reduce the installation filesize.
You can use the intl polyfill module to completely replace NodeJS’s implementation of intl:
npm i --save intl
// Replace Intl by polyfill Intl = require("intl") const df = new Intl.DateTimeFormat('de', {day: 'numeric', month: 'long', year: 'numeric', timeZone: 'UTC'}); console.log(df.format(new Date("2018-08-13T04:00:00.000Z")));
This will print 13. August 2018
as expected.
You can use the full-icu package to continue using the NodeJS ICU implementation (i.e. no polyfill), but just install the ICU data.
While this reduces the total installation filesize, installation is slow and the exact method depends on the NodeJS version and requires more work than just using the intl polyfill.
In order to install, use
npm i --save full-icu
This will take some time to compile the data and then will print instructions like this:
√ icudt62l.dat (link) Node will use this ICU datafile if the environment variable NODE_ICU_DATA is set to “node_modules/full-icu” or with node --icu-data-dir=node_modules/full-icu YOURAPP.js For package.json: {"scripts":{"start":"node --icu-data-dir=node_modules/full-icu YOURAPP.js"}} By the way, if you have full data, running this in node: > new Intl.DateTimeFormat('es',{month:'long'}).format(new Date(9E8)); ... will show “enero”. If it shows “January” you don't have full data. News: Please see https://github.com/icu-project/full-icu-npm/issues/6
In order to actually use full-icu, you need to use the --icu-data-dir=node_modules/full-icu
argument every time you run node
. In order to run node interactively, use
node --icu-data-dir=node_modules/full-icu
If you use scripts in your application (e.g. the start
script, i.e. what gets executed if you run npm start
), you need to adjust the configuration in package.json
:
Instead of
// [...] "scripts": { "start": "node index.js" } // [...]
use
// [...] "scripts": { "start": "node --icu-data-dir=node_modules/full-icu index.js" } // [...]
Depending on your application, you might need to use a different script name than index.js – common names include server.js
and start.js
When you run npm install
and it tries to install a native package like bcrypt and you see an error message like this:
gyp ERR! build error gyp ERR! stack Error: not found: make gyp ERR! stack at getNotFoundError (/usr/lib/node_modules/npm/node_modules/which/which.js:13:12) gyp ERR! stack at F (/usr/lib/node_modules/npm/node_modules/which/which.js:68:19) gyp ERR! stack at E (/usr/lib/node_modules/npm/node_modules/which/which.js:80:29) gyp ERR! stack at /usr/lib/node_modules/npm/node_modules/which/which.js:89:16 gyp ERR! stack at /usr/lib/node_modules/npm/node_modules/isexe/index.js:42:5 gyp ERR! stack at /usr/lib/node_modules/npm/node_modules/isexe/mode.js:8:5 gyp ERR! stack at FSReqWrap.oncomplete (fs.js:182:21)
you simple need to install GNU Make. On Ubuntu, the easiest way of doing this is to run
sudo apt install build-essential
This will not only install make
but also related tools like gcc
and some standard header files and tools.