Releases: RafaelWO/unparallel
0.4.0
What's Changed
This new release of Unparallel comes with a major fix that improves the performance and reduces the number of timeout errors massively if you do thousands of web requests with Unparallel! ⚡
This is achieved by synchronizing the async tasks with a semaphore object having the same value as the maximum connections (benchmarking tests showed that setting the semaphore to a higher value gives worse results; see also #197).
Additionally, you can now initialize (or reuse) your own httpx.AsyncClient
and pass it to Unparallel:
client = httpx.AsyncClient(...)
await up(..., client=client)
See also #186 or check the related section in the docs.
🚀 Features
🔧 Fixes & Refactoring
📦 Build System & CI/CD
- Use ruff instead of black and isort (#165) @RafaelWO
- Refactor tests and mock URLs (#169) @RafaelWO
- Use
bump-my-version
(#141) @RafaelWO
📝 Documentation
⬆️ Dependencies updates
Update dev dependencies
- ⬆️ Bump bandit from 1.7.7 to 1.7.8 (#152) @dependabot
- ⬆️ Bump mypy from 1.8.0 to 1.9.0 (#149) @dependabot
- ⬆️ Bump pytest from 8.0.2 to 8.1.1 (#148) @dependabot
- ⬆️ Bump mkdocs-material from 9.5.11 to 9.5.13 (#147) @dependabot
- ⬆️ Bump mkdocs-material from 9.5.13 to 9.5.14 (#153) @dependabot
- ⬆️ Bump coverage from 7.4.3 to 7.4.4 (#154) @dependabot
- ⬆️ Bump mkdocstrings-python from 1.8.0 to 1.9.0 (#156) @dependabot
- ⬆️ Bump tj-actions/verify-changed-files from 18 to 19 (#143) @dependabot
- ⬆️ Bump pytest-cov from 4.1.0 to 5.0.0 (#158) @dependabot
- ⬆️ Bump respx from 0.20.2 to 0.21.1 (#162) @dependabot
- ⬆️ Bump ruff from 0.3.4 to 0.3.5 (#167) @dependabot
- ⬆️ Bump safety from 3.0.1 to 3.1.0 (#164) @dependabot
- ⬆️ Bump pytest-asyncio from 0.23.5 to 0.23.6 (#160) @dependabot
- ⬆️ Bump mkdocs-material from 9.5.14 to 9.5.18 (#171) @dependabot
- ⬆️ Bump mkdocstrings-python from 1.9.0 to 1.10.0 (#172) @dependabot
- ⬆️ Bump ruff from 0.3.5 to 0.4.1 (#173) @dependabot
- ⬆️ Bump ruff from 0.4.1 to 0.4.3 (#182) @dependabot
- ⬆️ Bump coverage-badge from 1.1.0 to 1.1.1 (#177) @dependabot
- ⬆️ Bump mypy from 1.9.0 to 1.10.0 (#176) @dependabot
- ⬆️ Bump coverage from 7.4.4 to 7.5.1 (#185) @dependabot
- ⬆️ Bump pytest from 8.1.1 to 8.2.0 (#179) @dependabot
- ⬆️ Bump mkdocs-material from 9.5.18 to 9.5.21 (#183) @dependabot
- ⬆️ Bump tj-actions/verify-changed-files from 19 to 20 (#184) @dependabot
- ⬆️ Bump mkdocs-material from 9.5.21 to 9.5.25 (#196) @dependabot
- ⬆️ Bump pytest-asyncio from 0.23.6 to 0.23.7 (#194) @dependabot
- ⬆️ Bump bump-my-version from 0.21.0 to 0.21.1 (#193) @dependabot
- ⬆️ Bump mike from 2.0.0 to 2.1.1 (#192) @dependabot
- ⬆️ Bump coverage from 7.5.1 to 7.5.3 (#191) @dependabot
- ⬆️ Bump safety from 3.1.0 to 3.2.0 (#190) @dependabot
- ⬆️ Bump pytest from 8.2.0 to 8.2.1 (#189) @dependabot
- ⬆️ Bump mkdocstrings-python from 1.10.0 to 1.10.3 (#188) @dependabot
- ⬆️ Bump ruff from 0.4.3 to 0.4.7 (#187) @dependabot
- ⬆️ Bump types-tqdm from 4.66.0.20240106 to 4.66.0.20240417 (#174) @dependabot
👥 List of contributors
@RafaelWO and @dependabot
Full Changelog: 0.3.0...0.4.0
0.3.0
What's Changed
This new release of Unparallel comes with two main new features:
- Support different URLs in one call to
up()
- The new parameter
response_fn
allows you to configure your custom response function
Support different URLs
The first feature is also a breaking change because the first parameter of up()
is no longer the base URL but the list of URLs (or paths if you specify base_url
additionally). Before, you could use Unparallel as follows:
async def main():
url = "https://httpbin.org"
paths = [f"/get?i={i}" for i in range(5)]
results = await up(url, paths)
To migrate the code above to work with version 0.3.0
you have to pass the base URL as a keyword argument base_url=url
:
async def main():
url = "https://httpbin.org"
paths = [f"/get?i={i}" for i in range(5)]
results = await up(paths, base_url=url) # <--- CHANGE
Or you use full URLs as the first parameter:
async def main():
urls = [f"https://httpbin.org/get?i={i}" for i in range(5)]
results = await up(urls)
Configure a custom response function
You can configure custom response processing using the new parameter response_fn
. Per default, the .json()
method of the Response
object is called. If you e.g. want to return just the status codes, you can pass response_fn=lambda x: x.status_code
to up()
. You can find more details in the docs.
💥 Breaking Changes
🚀 Features
- Feature: Support different URLs per request (#108) @RafaelWO
- Feature: Add parameter to configure custom response function (#109) @RafaelWO
🔧 Fixes & Refactoring
📦 Build System & CI/CD
📝 Documentation
⬆️ Dependencies updates
Click for details
- ⬆️ Bump pytest from 7.4.3 to 7.4.4 (#110) @dependabot
- ⬆️ Bump tj-actions/verify-changed-files from 16 to 17 (#111) @dependabot
- ⬆️ Bump pytest-asyncio from 0.23.2 to 0.23.3 (#112) @dependabot
- ⬆️ Bump mkdocstrings-python from 1.7.5 to 1.8.0 (#114) @dependabot
- ⬆️ Bump actions/cache from 3.3.2 to 3.3.3 (#113) @dependabot
- ⬆️ Bump mkdocs-material from 9.5.3 to 9.5.4 (#115) @dependabot
- ⬆️ Bump actions/cache from 3.3.3 to 4.0.0 (#116) @dependabot
- ⬆️ Bump bandit from 1.7.6 to 1.7.7 (#117) @dependabot
- ⬆️ Bump coverage from 7.4.0 to 7.4.1 (#118) @dependabot
- ⬆️ Bump pytest from 7.4.4 to 8.0.0 (#119) @dependabot
- ⬆️ Bump pytest-asyncio from 0.23.3 to 0.23.4 (#120) @dependabot
- ⬆️ Bump mkdocs-material from 9.5.4 to 9.5.6 (#122) @dependabot
- ⬇️ Downgrade pytest to 7.4.4 to match pytest-asyncio requirement (#123) @RafaelWO
- ⬆️ Bump tj-actions/verify-changed-files from 17 to 18 (#124) @dependabot
- ⬆️ Bump release-drafter/release-drafter from 5.25.0 to 6.0.0 (#125) @dependabot
- ⬆️ Bump mkdocs-material from 9.5.6 to 9.5.7 (#126) @dependabot
- ⬆️ Bump tqdm from 4.66.1 to 4.66.2 (#131) @dependabot
- ⬆️ Bump actions/cache from 3 to 4 (#130) @dependabot
- ⬆️ Bump pytest-asyncio from 0.23.4 to 0.23.5 (#132) @dependabot
- ⬆️ Bump pytest from 7.4.4 to 8.0.0 (#127) @dependabot
- ⬆️ Bump mkdocs-material from 9.5.7 to 9.5.9 (#134) @dependabot
- ⬆️ Bump safety from 2.4.0b2 to 3.0.1 (#133) @dependabot
- Upgrade multiple packages manually (#133) @RafaelWO
- ⬆️ Bump pytest from 8.0.1 to 8.0.2 (#139) @dependabot
👥 List of contributors
@RafaelWO and @dependabot
Full Changelog: 0.2.0...0.3.0
0.2.0
The second release of Unparallel comes with some new features for configuring your HTTP requests (timeouts, limits, payload). Additionally, the docs are extended with more details on the usage (including examples) - see here.
🚀 Features
- Feature: Configure HTTPX timeouts and limits (#87) @RafaelWO
- Feature: Add support for single payload for multiple paths (#101) @RafaelWO
📦 Build System & CI/CD
💥 Breaking Changes
To be in line with HTTPX's options, the parameter connection_limit
was renamed to max_connections
(same as in HTTPX) in the function up()
(see #87).
📝 Documentation
- Improve docs: Add section "Why?" with GIF to README (#82) @RafaelWO
- Docs: Add more detailed usage examples (#105) @RafaelWO
- Docs: Add versioning via mike (#106) @RafaelWO
- Docs: Add a changelog (#107) @RafaelWO
⬆️ Dependencies updates
- ⬆️ Bump mypy from 1.7.0 to 1.7.1 (#76) @dependabot
- ⬆️ Bump pytest-asyncio from 0.21.1 to 0.23.2 (#78) @dependabot
- ⬆️ Bump mkdocs-material from 9.5.1 to 9.5.2 (#92) @dependabot
- ⬆️ Bump coverage from 7.3.2 to 7.3.3 (#91) @dependabot
- ⬆️ Bump actions/setup-python from 4 to 5 (#83) @dependabot
- ⬆️ Bump httpx from 0.25.2 to 0.26.0 (#97) @dependabot
- ⬆️ Bump mypy from 1.7.1 to 1.8.0 (#96) @dependabot
- ⬆️ Bump mkdocs-material from 9.5.2 to 9.5.3 (#95) @dependabot
- ⬆️ Bump coverage from 7.3.3 to 7.3.4 (#94) @dependabot
- ⬆️ Bump black from 23.11.0 to 23.12.1 (#93) @dependabot
- ⬆️ Bump isort from 5.12.0 to 5.13.2 (#89) @dependabot
- ⬆️ Bump pylint from 3.0.2 to 3.0.3 (#88) @dependabot
👥 List of contributors
@RafaelWO, @dependabot and @dependabot[bot]
Full Changelog: 0.1.0...0.2.0
0.1.0
This is the initial release of unparallel
! 🚀
Check out the README for examples and features.
What's Changed
🚀 Features
📦 Build System & CI/CD
- Add docs via MkDocs (#80) @RafaelWO
- Final improvements for initial release (#79) @RafaelWO
- 👷 Auto-update coverage badge via CI (#17) @RafaelWO
📝 Documentation
- Add docs via MkDocs (#80) @RafaelWO
- Final improvements for initial release (#79) @RafaelWO
- Add code for MVP (#72) @RafaelWO
⬆️ Dependencies updates
Click to expand
- ⬆️ Bump mypy from 1.6.1 to 1.7.0 (#74) @dependabot
- ⬆️ Bump black from 23.10.1 to 23.11.0 (#73) @dependabot
- ⬆️ Bump black from 23.10.0 to 23.10.1 (#70) @dependabot
- ⬆️ Bump identify from 2.5.30 to 2.5.31 (#71) @dependabot
- ⬆️ Bump filelock from 3.12.4 to 3.13.0 (#69) @dependabot
- ⬆️ Bump pytest from 7.4.2 to 7.4.3 (#68) @dependabot
- ⬆️ Bump virtualenv from 20.24.5 to 20.24.6 (#67) @dependabot
- ⬆️ Bump ruamel-yaml from 0.17.40 to 0.18.3 (#66) @dependabot
- ⬆️ Bump typing-extensions from 4.7.1 to 4.8.0 (#38) @dependabot
- ⬆️ Bump coverage from 7.3.1 to 7.3.2 (#54) @dependabot
- ⬆️ Bump identify from 2.5.27 to 2.5.30 (#44) @dependabot
- ⬆️ Bump pre-commit from 3.4.0 to 3.5.0 (#56) @dependabot
- ⬆️ Bump smmap from 5.0.0 to 5.0.1 (#32) @dependabot
- ⬆️ Bump filelock from 3.12.3 to 3.12.4 (#35) @dependabot
- ⬆️ Bump setuptools from 68.2.0 to 68.2.2 (#34) @dependabot
- ⬆️ Bump actions/setup-python from 4.7.0 to 4.7.1 (#48) @dependabot
- ⬆️ Bump rich from 13.5.2 to 13.6.0 (#47) @dependabot
- ⬆️ Bump ruamel-yaml-clib from 0.2.7 to 0.2.8 (#49) @dependabot
- ⬆️ Bump platformdirs from 3.10.0 to 3.11.0 (#51) @dependabot
- ⬆️ Bump gitdb from 4.0.10 to 4.0.11 (#57) @dependabot
- ⬆️ Bump charset-normalizer from 3.2.0 to 3.3.1 (#58) @dependabot
- ⬆️ Bump pylint from 2.17.5 to 3.0.2 (#59) @dependabot
- ⬆️ Bump gitpython from 3.1.35 to 3.1.40 (#60) @dependabot
- ⬆️ Bump mypy from 1.5.1 to 1.6.1 (#61) @dependabot
- ⬆️ Bump black from 23.9.1 to 23.10.0 (#62) @dependabot
- ⬆️ Bump ruamel-yaml from 0.17.32 to 0.17.40 (#63) @dependabot
- ⬆️ Bump urllib3 from 2.0.4 to 2.0.7 (#64) @dependabot
- ⬆️ Bump release-drafter/release-drafter from 5.24.0 to 5.25.0 (#65) @dependabot
- ⬆️ Bump pytest from 7.4.1 to 7.4.2 (#27) @dependabot
- ⬆️ Bump coverage from 7.3.0 to 7.3.1 (#25) @dependabot
- ⬆️ Bump gitpython from 3.1.34 to 3.1.35 (#24) @dependabot
- ⬆️ Bump setuptools from 68.1.2 to 68.2.0 (#26) @dependabot
- ⬆️ Bump virtualenv from 20.24.4 to 20.24.5 (#28) @dependabot
- ⬆️ Bump black from 23.7.0 to 23.9.1 (#29) @dependabot
- ⬆️ Bump actions/checkout from 3 to 4 (#30) @dependabot
- ⬆️ Bump actions/cache from 3.3.1 to 3.3.2 (#31) @dependabot
- ⬆️ Bump pre-commit from 3.3.3 to 3.4.0 (#20) @dependabot
- ⬆️ Bump filelock from 3.12.2 to 3.12.3 (#21) @dependabot
- ⬆️ Bump virtualenv from 20.24.3 to 20.24.4 (#22) @dependabot
- ⬆️ Bump gitpython from 3.1.32 to 3.1.34 (#23) @dependabot
- ⬆️ Bump pytest from 7.4.0 to 7.4.1 (#19) @dependabot
- ⬆️ Bump mypy from 1.0.1 to 1.5.1 (#18) @dependabot
- ⬆️ Bump actions/setup-python from 2.2.2 to 4.7.0 (#4) @dependabot
- ⬆️ Bump release-drafter/release-drafter from 5.15.0 to 5.24.0 (#2) @dependabot
- ⬆️ Bump mypy-extensions from 0.4.4 to 1.0.0 (#14) @dependabot
- ⬆️ Bump packaging from 21.3 to 23.0 (#15) @dependabot
- ⬆️ Bump actions/checkout from 2 to 3 (#3) @dependabot
- ⬆️ Bump black from 22.12.0 to 23.7.0 (#6) @dependabot
- ⬆️ Bump coverage from 6.5.0 to 7.3.0 (#7) @dependabot
- ⬆️ Bump pytest-cov from 3.0.0 to 4.1.0 (#8) @dependabot
- ⬆️ Bump pyupgrade from 2.38.4 to 3.8.0 (#9) @dependabot
- ⬆️ Bump mypy from 0.910 to 1.0.1 (#10) @dependabot
- ⬆️ Bump pre-commit from 2.21.0 to 3.3.3 (#11) @dependabot
- ⬆️ Bump safety from 1.10.3 to 2.3.5 (#13) @dependabot
- ⬆️ Bump pytest from 6.2.5 to 7.4.0 (#12) @dependabot
- ⬆️ Bump actions/cache from 2.1.6 to 3.3.1 (#1) @dependabot