I use this code for sending a GET request:
(async() => {
const browser = await puppeteer.launch();
const page = await browser.newPage();
await page.goto('https://www.example.com/search');
const data = await page.content();
browser.close();
res.send(data);
})();
How do I send a POST request?
Getting the "order" right can be a bit of a challenge. Documentation doesn't have that many examples... there are some juicy items in the repository in the example folder that you should definitely take a look at.
https://github.com/GoogleChrome/puppeteer/tree/main/examples
Here is the example; place the following into an async block:
// Create browser instance, and give it a first tab
const browser = await puppeteer.launch();
const page = await browser.newPage();
// Allows you to intercept a request; must appear before
// your first page.goto()
await page.setRequestInterception(true);
// Request intercept handler... will be triggered with
// each page.goto() statement
page.on('request', interceptedRequest => {
// Here, is where you change the request method and
// add your post data
var data = {
'method': 'POST',
'postData': 'paramFoo=valueBar¶mThis=valueThat'
};
// Request modified... finish sending!
interceptedRequest.continue(data);
});
// Navigate, trigger the intercept, and resolve the response
const response = await page.goto('https://www.example.com/search');
const responseBody = await response.text();
console.log(responseBody);
// Close the browser - done!
await browser.close();
"Content-Type": "application/x-www-form-urlencoded" header from Dmitry Sheiko's answer below for this to work.
There's a quirk with the way setRequestInterception and the 'request' event work. Once activated, Puppeteer will send the POST data to every resource on the page, not just the original requested page. I was having the issue that all of my page resources (scripts, CSS) were failing to load once I added POST data in Puppeteer.
Since I only want to apply POST data to the first request, this code worked for me:
// Used for serializing POST parameters from an object
const querystring = require('querystring');
// ...
const browser = await puppeteer.launch();
const page = await browser.newPage();
let postData = {a: 1, b: 2};
await page.setRequestInterception(true);
page.once('request', request => {
var data = {
'method': 'POST',
'postData': querystring.stringify(postData),
'headers': {
...request.headers(),
'Content-Type': 'application/x-www-form-urlencoded'
},
};
request.continue(data);
// Immediately disable setRequestInterception, or all other requests will hang
page.setRequestInterception(false);
});
const response = await page.goto('https://www.example.com/');
You can use the Page.evaluate() method to run Fetch API inside the browser. For example:
const postResponse = await page.evaluate( async (param1, param2, param3) => {
const response = await fetch("https://www.example.com/add", {
"headers": {
"content-type": "application/x-www-form-urlencoded; charset=UTF-8"
},
"body": `param1=${param1}¶m2=${param2}¶m3=${param3}`,
"method": "POST"
});
const data = await response.json();
return data;
}, param1, param2, param3);
Here the complete example with Puppeteer 2.0.0 :
const puppeteer = require("puppeteer");
const devices = require("puppeteer/DeviceDescriptors");
async function main() {
const browser = await puppeteer.launch({
args: ["--enable-features=NetworkService", "--no-sandbox"],
ignoreHTTPSErrors: true
});
const page = await browser.newPage();
await page.setRequestInterception(true);
page.once("request", interceptedRequest => {
interceptedRequest.continue({
method: "POST",
postData: "foo=FOO&bar=BAR",
headers: {
...interceptedRequest.headers(),
"Content-Type": "application/x-www-form-urlencoded"
}
});
});
const response = await page.goto("https://postman-echo.com/post");
console.log({
url: response.url(),
statusCode: response.status(),
body: await response.text()
});
await browser.close();
}
main();
Note that if you check response.request().method() it won't be updated (still GET)
const devices being defined and never used when pasting this sample. Then, I can't run it because Error: Cannot find module 'puppeteer/DeviceDescriptors'.
Commented
May 24, 2021 at 15:09
