Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Check PageAttention #704

Closed
Show file tree
Hide file tree
Changes from 21 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
e7e121e
check model pattern changes
wgzintel Jul 30, 2024
ed5369d
clone tests/model_hub_tests
wgzintel Jul 30, 2024
ab1fb17
PagedAttention Test
wgzintel Jul 30, 2024
e106c8d
Remove redundant commands
wgzintel Jul 30, 2024
9767955
suppress warning summary
wgzintel Jul 30, 2024
ea218b1
Generate a new file to check model changes
wgzintel Jul 31, 2024
95af9be
updated to latest of master
wgzintel Jul 31, 2024
429cc1b
modify requirements.txt
wgzintel Jul 31, 2024
45d7a9b
test triggering check model pattern changes
wgzintel Jul 31, 2024
bdc738b
test triggering check model pattern changes
wgzintel Jul 31, 2024
4886e01
test triggering check model pattern changes
wgzintel Jul 31, 2024
a332959
revert requirements.txt
wgzintel Jul 31, 2024
3653581
revert benchmark.py
wgzintel Jul 31, 2024
6c8c361
modify check_model_pattern_changes.yml
wgzintel Jul 31, 2024
7241335
modify requirement.txt
wgzintel Jul 31, 2024
715eccc
modify check_model_pattern_changes.yml
wgzintel Jul 31, 2024
7aa0c47
modify benchmark.py
wgzintel Jul 31, 2024
2237c2b
revert benchmark.py
wgzintel Jul 31, 2024
5c2a422
revert requirements.txt
wgzintel Jul 31, 2024
3e12d31
resolve conflict
wgzintel Jul 31, 2024
8d91eec
modify benchmark.py
wgzintel Jul 31, 2024
518576d
revert benchmark.py
wgzintel Aug 1, 2024
3007837
modify requirements.txt
wgzintel Aug 1, 2024
1583514
revert requirements.txt
wgzintel Aug 1, 2024
11f292b
Merge branch 'master' of https://github.com/openvinotoolkit/openvino.…
wgzintel Aug 1, 2024
63c63e4
modify benchmark.py
wgzintel Aug 1, 2024
2bcdc28
revert benchmark.py
wgzintel Aug 1, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions .github/workflows/check_model_pattern_changes.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-python

name: check model pattern changes

env:
LLM_BENCH_PYPATH: llm_bench/python

on:
pull_request:
paths:
- 'requirements.txt'
- .github/workflows/check_model_pattern_changes.yml

permissions: read-all # Required by https://github.com/ossf/scorecard/blob/e23b8ad91fd6a64a0a971ca4fc0a4d1650725615/docs/checks.md#token-permissions

jobs:
build:

runs-on: ubuntu-latest

strategy:
fail-fast: false
matrix:
python-version: ["3.9"]

steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install pytest
GIT_CLONE_PROTECTION_ACTIVE=false pip install -r ${{ env.LLM_BENCH_PYPATH }}/requirements.txt
pip install openvino-nightly

- name: PagedAttention Test
run: |
git clone --depth 1 --filter=blob:none --no-checkout https://github.com/openvinotoolkit/openvino
cd openvino
git checkout master tests/model_hub_tests/
cd tests/model_hub_tests
python3 -m pytest transformation_tests/test_pa_transformation.py -m precommit -v --tb=short -Wignore
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need such tests here? Looks like they can be run on OpenVINO side.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ilya-lavrenov The intention is to test when optimum-intel and transformers (or may be even NNCF?) changes.
We should think about the overlap with test
https://github.com/openvinotoolkit/openvino/blob/master/.github/workflows/job_pytorch_models_tests.yml

env:
TEST_DEVICE: CPU
USE_SYSTEM_CACHE: False
2 changes: 1 addition & 1 deletion llm_bench/python/benchmark.py
Original file line number Diff line number Diff line change
Expand Up @@ -762,7 +762,7 @@ def main():
model_precision,
)
except Exception:
log.error('An exception occurred')
log.error('An exception occurred ....')
log.info(traceback.format_exc())
exit(1)
finally:
Expand Down
Loading