HTTP authentication credential leak to target websites in scrapy
Key facts
- Attack vector
- Network
- Privileges required
- Low
- User interaction
- None
- Attack complexity
- Low
- Impact type
- Information disclosure
- Confidentiality · integrity · availability
- High · None · None
- Fix status
- Fixed
Attack profile from CVSS 3.1 · NVD. Fix status from CNA and GitHub. Impact type from CWE-200.
Summary
Scrapy is a high-level web crawling and scraping framework for Python. If you use `HttpAuthMiddleware` (i.e. the `http_user` and `http_pass` spider attributes) for HTTP authentication, all requests will expose your credentials to the request target. This includes requests generated by Scrapy components, such as `robots.txt` requests sent by Scrapy when the `ROBOTSTXT_OBEY` setting is set to `True`, or as requests reached through redirects. Upgrade to Scrapy 2.5.1 and use the new `http_auth_domain` spider attribute to control which domains are allowed to receive the configured HTTP authentication credentials. If you are using Scrapy 1.8 or a lower version, and upgrading to Scrapy 2.5.1 is not an option, you may upgrade to Scrapy 1.8.1 instead. If you cannot upgrade, set your HTTP authentication credentials on a per-request basis, using for example the `w3lib.http.basic_auth_header` function to convert your credentials into a value that you can assign to the `Authorization` header of your request, instead of defining your credentials globally using `HttpAuthMiddleware`.
Why it matters now
SSVC decision points
Not available: CISA has not published SSVC decision points for this record.
Affected products
| Vendor | Product | Affected versions | Fixed in | Status | Source |
|---|---|---|---|---|---|
| scrapy | scrapy | before 1.8.1; 2.0.0 to before 2.5.1 | 1.8.1, 2.5.1 | Patch available | CNA |
| — | Scrapy (PyPI) | before 1.8.1 | 1.8.1 | Patch available | GHSA-jwqp-28gf-p498 |
| — | Scrapy (PyPI) | 2.0.0 to before 2.5.1 | 2.5.1 | Patch available | GHSA-jwqp-28gf-p498 |
- scrapy Vendorscrapy Affected before 1.8.1; 2.0.0 to before 2.5.1 Fixed in 1.8.1, 2.5.1 Patch available CNA
- Scrapy (PyPI) Vendor— Affected before 1.8.1 Fixed in 1.8.1 Patch available GHSA-jwqp-28gf-p498
- Scrapy (PyPI) Vendor— Affected 2.0.0 to before 2.5.1 Fixed in 2.5.1 Patch available GHSA-jwqp-28gf-p498
Remediation
- Fix · CNA scrapy (scrapy) Fixed in 1.8.1, 2.5.1
- Fix · GHSA-jwqp-28gf-p498 Scrapy (PyPI) Fixed in 1.8.1 Advisory →
- Fix · GHSA-jwqp-28gf-p498 Scrapy (PyPI) Fixed in 2.5.1 Advisory →
Vendor assessments
Timeline
- 2026-10-02 Threatist record created from CVE-2021-41125
- 2026-10-03 Alias added: GHSA-jwqp-28gf-p498
References
Advisory 5
- https://github.com/scrapy/scrapy/security/advisories/GHSA-jwqp-28gf-p498
- https://w3lib.readthedocs.io/en/latest/w3lib.html#w3lib.http.basic_auth_header
- http://doc.scrapy.org/en/latest/topics/downloader-middleware.html#module-scrapy.downloadermiddlewares.httpauth
- [debian-lts-announce] 20220316 [SECURITY] [DLA 2950-1] python-scrapy security update
- https://nvd.nist.gov/vuln/detail/CVE-2021-41125
More Scrapy vulnerabilities
- TI-2024-03333 Authorization Header Leak During Cross-Domain Redirect in scrapy/scrapy CVSS 3.0HIGH 7.5
- TI-2024-03331 XML External Entity (XXE) Vulnerability in scrapy/scrapy CVSS 3.0HIGH 7.5
- TI-2024-01831 Authorization Header Leakage in scrapy/scrapy on Scheme Change Redirects CVSS 3.0HIGH 7.5
- TI-2017-10313 Scrapy 1.4 allows remote attackers to cause a denial of service (memory consumption) via… CVSS 3.0HIGH 7.5
- TI-2024-01761 ReDoS Vulnerability in scrapy/scrapy's XMLFeedSpider CVSS 3.1MEDIUM 6.5
- TI-2022-00495 Exposure of Sensitive Information to an Unauthorized Actor in scrapy/scrapy CVSS 3.1MEDIUM 6.5
Sources
- CVE-2021-41125 · CVE Program
- CVE-2021-41125 · NVD
- FIRST EPSS
- GHSA-jwqp-28gf-p498 · GitHub Advisory Database · CC BY 4.0
- CVE-2021-41125 · Red Hat security data · CC BY 4.0
This product uses the NVD API but is not endorsed or certified by the NVD. Sources and licences