CONTENTUM.COMA specialist. A plan. A result.

Vulnerabilities · 2021

CVE-2021-41124
Zyte Scrapy-splash vulnerability

Scrapy-splash is a library which provides Scrapy and JavaScript integration. In affected versions users who use [`HttpAuthMiddleware`](http://doc.scrapy.org/en/latest/topics/downloader-middleware.html#module-scrapy.downloadermiddlewares.httpauth) (i.e. the `http_user` and `http_pass` spider attributes) for Splash authentication will have any non-Splash request expose your credentials to the request target. This includes `robots.txt` requests sent by Scrapy when the `ROBOTSTXT_OBEY` setting is set to `True`. Upgrade to scrapy-splash 0.8.0 and use the new `SPLASH_USER` and `SPLASH_PASS` settings instead to set your Splash authentication credentials safely. If you cannot upgrade, set your Splash request credentials on a per-request basis, [using the `splash_headers` request parameter](https://github.com/scrapy-plugins/scrapy-splash/tree/0.8.x#http-basic-auth), instead of defining them globally using the [`HttpAuthMiddleware`](http://doc.scrapy.org/en/latest/topics/downloader-middleware.html#module-scrapy.downloadermiddlewares.httpauth). Alternatively, make sure all your requests go through Splash. That includes disabling the [robots.txt middleware](https://docs.scrapy.org/en/latest/topics/downloader-middleware.html#topics-dlmw-robots).

Affected products

Scrapy-splash. The exact versions are in the references.

What to do now

  • Check your exposureFind out whether Zyte Scrapy-splash runs anywhere in your environment, including systems of suppliers.
  • UpdateInstall the version in which the vendor fixed the vulnerability. The references lead to the advisory.
  • Limit access until thenIf you cannot update at once, take the system off the internet or restrict who can reach it.

References

Data: CVE list and the NVD. This product uses data from the NVD API but is not endorsed or certified by the NVD.