Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keras3 tests #2152

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 14 additions & 18 deletions .github/workflows/actions.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,13 @@ permissions:
contents: read

jobs:
keras2:
name: Test the code with tf.keras
keras_2:
name: Test the code with Keras 2
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Set up Python 3.9
uses: actions/setup-python@v1
uses: actions/setup-python@v4
with:
python-version: 3.9
- name: Get pip cache dir
Expand All @@ -34,7 +34,7 @@ jobs:
${{ runner.os }}-pip-
- name: Install dependencies
run: |
pip install tensorflow>=2.13.0
pip install tensorflow~=2.14
pip install torch>=2.0.1+cpu
pip install "jax[cpu]"
pip install keras-core
Expand All @@ -44,17 +44,17 @@ jobs:
TEST_CUSTOM_OPS: false
run: |
pytest keras_cv/ --ignore keras_cv/models/legacy/ --durations 0
multibackend:
name: Test the code with Keras Core
keras_3:
name: Test the code with Keras 3
strategy:
fail-fast: false
matrix:
backend: [tensorflow, jax, torch]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Set up Python 3.9
uses: actions/setup-python@v1
uses: actions/setup-python@v4
with:
python-version: 3.9
- name: Get pip cache dir
Expand All @@ -71,11 +71,7 @@ jobs:
${{ runner.os }}-pip-
- name: Install dependencies
run: |
pip install tensorflow>=2.13.0
pip install "jax[cpu]"
pip install torch>=2.0.1+cpu
pip install torchvision>=0.15.1
pip install keras-core
pip install -r reqirements.txt
pip install -e ".[tests]" --progress-bar off --upgrade
- name: Test with pytest
env:
Expand All @@ -99,9 +95,9 @@ jobs:
name: Check the code format
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- uses: actions/checkout@v3
- name: Set up Python 3.9
uses: actions/setup-python@v1
uses: actions/setup-python@v4
with:
python-version: 3.9
- name: Get pip cache dir
Expand All @@ -118,7 +114,7 @@ jobs:
${{ runner.os }}-pip-
- name: Install dependencies
run: |
pip install tensorflow>=2.13.0
pip install tensorflow~=2.14
pip install -e ".[tests]" --progress-bar off --upgrade
- name: Lint
run: bash shell/lint.sh
Expand All @@ -128,4 +124,4 @@ jobs:
source: '.'
extensions: 'h,c,cpp,hpp,cc'
clangFormatVersion: 14
style: google
style: google
2 changes: 1 addition & 1 deletion .github/workflows/devcontainer.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
steps:

- name: Checkout (GitHub)
uses: actions/checkout@v2
uses: actions/checkout@v4

- name: Build and run dev container task
uses: devcontainers/ci@v0.2
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/nightly.yml
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ jobs:
pip install -r requirements.txt --progress-bar off
- name: Build wheel file
run: |
export BUILD_WITH_CUSTOM_OPS=false
python pip_build.py --nightly
- name: Publish to PyPI
uses: pypa/gh-action-pypi-publish@release/v1
Expand Down
133 changes: 33 additions & 100 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,105 +8,38 @@ permissions:
contents: read

jobs:
deploy-with-custom-ops:
# This job is currently skipped until we cut a release with custom ops.
if: false
name: Build and deploy release wheels with custom ops
runs-on: ${{ matrix.os }}
strategy:
matrix:
# To switch on windows-2022/latest, please verify the bazel version:
# https://github.com/bazelbuild/bazel/issues/14232#issuecomment-1011247429
os: ['macos-12', 'windows-2019', 'ubuntu-18.04']
py-version: ['3.9', '3.10', '3.11']
tf-version: ['2.13.0']
use-macos-arm: [false]
include:
- os: 'macos-12'
tf-version: '2.13.0'
py-version: '3.9'
use-macos-arm: true
- os: 'macos-12'
tf-version: '2.13.0'
py-version: '3.10'
use-macos-arm: true
fail-fast: false
steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v1
with:
python-version: ${{ matrix.py-version }}
- name: Get pip cache dir
id: pip-cache
run: |
python -m pip install --upgrade pip setuptools wheel auditwheel twine
echo "::set-output name=dir::$(pip cache dir)"
- name: pip cache
uses: actions/cache@v2
with:
path: ${{ steps.pip-cache.outputs.dir }}
key: ${{ runner.os }}-pip-${{ hashFiles('setup.py') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Install dependencies
run: |
python -m pip install tensorflow-cpu==${{ matrix.tf-version }}
python -m pip install -e ".[tests]" --progress-bar off --upgrade
- name: Configure Build Environment
run: |
python build_deps/configure.py
- name: Reinstall TensorFlow (MacOS ARM)
if: ${{ matrix.os == 'macos-12' && matrix.use-macos-arm}}
run: |
python -m pip uninstall -y tensorflow-cpu
python -m pip install --platform=macosx_12_0_arm64 --no-deps --target=$(python -c 'import site; print(site.getsitepackages()[0])') --upgrade tensorflow-macos==${{ matrix.tf-version }}
- name: Bazel Build
if: ${{ ! matrix.use-macos-arm }}
run: |
export BUILD_WITH_CUSTOM_OPS=true
bazel build build_pip_pkg
- name: Bazel Build (MacOS ARM)
if: ${{ matrix.use-macos-arm}}
run: |
bazel build --cpu=darwin_arm64 --copt -mmacosx-version-min=12.0 --linkopt -mmacosx-version-min=12.0 build_pip_pkg
- name: Build wheels
run: |
export BUILD_WITH_CUSTOM_OPS=true
bazel-bin/build_pip_pkg wheels
- name: Repair wheels (manylinux)
if: ${{ matrix.os == 'ubuntu-18.04' }}
run: |
python -m pip install --upgrade patchelf==0.14
bash build_deps/tf_auditwheel_patch.sh
python -m auditwheel repair --plat manylinux2014_x86_64 wheels/*.whl
rm wheels/*.whl
mv wheelhouse/* wheels/
- name: Upload wheels
env:
TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
run: |
twine upload wheels/*
deploy-without-custom-ops:
name: Build and deploy release wheels without custom ops
run-test-for-release:
uses: ./.github/workflows/actions.yml
release:
name: Build and publish to PyPI
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up Python
uses: actions/setup-python@v1
with:
python-version: 3.9
- name: Build wheels
run: |
pip install tensorflow==2.13.0
python -m pip install --upgrade setuptools wheel twine
python -m pip install --upgrade -r requirements.txt
export BUILD_WITH_CUSTOM_OPS=false
python pip_build.py
- name: Upload wheels
env:
TWINE_USERNAME: ${{ secrets.PYPI_USERNAME }}
TWINE_PASSWORD: ${{ secrets.PYPI_PASSWORD }}
run: |
twine upload dist/*.whl
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: 3.9
- name: Get pip cache dir
id: pip-cache
run: |
python -m pip install --upgrade pip setuptools
echo "::set-output name=dir::$(pip cache dir)"
- name: pip cache
uses: actions/cache@v2
with:
path: ${{ steps.pip-cache.outputs.dir }}
key: ${{ runner.os }}-pip-${{ hashFiles('setup.py') }}
restore-keys: |
${{ runner.os }}-pip-
- name: Install dependencies
run: |
pip install -r requirements.txt --progress-bar off
- name: Build a binary wheel and a source tarball
run: |
export BUILD_WITH_CUSTOM_OPS=false
python pip_build.py
- name: Publish distribution to PyPI
if: startsWith(github.ref, 'refs/tags')
uses: pypa/gh-action-pypi-publish@master
with:
password: ${{ secrets.PYPI_API_TOKEN }}
50 changes: 31 additions & 19 deletions keras_cv/backend/random.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,23 +16,14 @@

if keras_3():
from keras.random import * # noqa: F403, F401
# SeedGenerator is imported from `keras.random`
else:
from keras_core.random import * # noqa: F403, F401


class SeedGenerator:
def __init__(self, seed=None, **kwargs):
if keras_3():
self._seed_generator = keras.random.SeedGenerator(
seed=seed, **kwargs
)
else:
class SeedGenerator:
def __init__(self, seed=None, **kwargs):
self._current_seed = [seed, 0]

def next(self, ordered=True):
if keras_3():
return self._seed_generator.next(ordered=ordered)
else:
def next(self, ordered=True):
self._current_seed[1] += 1
return self._current_seed[:]

Expand All @@ -50,7 +41,6 @@ def make_seed(seed=None):


def normal(shape, mean=0.0, stddev=1.0, dtype=None, seed=None):
init_seed = make_seed(seed)
kwargs = {}
if dtype:
kwargs["dtype"] = dtype
Expand All @@ -59,7 +49,7 @@ def normal(shape, mean=0.0, stddev=1.0, dtype=None, seed=None):
shape,
mean=mean,
stddev=stddev,
seed=init_seed,
seed=seed,
**kwargs,
)
else:
Expand All @@ -69,13 +59,12 @@ def normal(shape, mean=0.0, stddev=1.0, dtype=None, seed=None):
shape,
mean=mean,
stddev=stddev,
seed=init_seed,
seed=make_seed(seed),
**kwargs,
)


def uniform(shape, minval=0.0, maxval=1.0, dtype=None, seed=None):
init_seed = make_seed(seed)
kwargs = {}
if dtype:
kwargs["dtype"] = dtype
Expand All @@ -84,7 +73,7 @@ def uniform(shape, minval=0.0, maxval=1.0, dtype=None, seed=None):
shape,
minval=minval,
maxval=maxval,
seed=init_seed,
seed=seed,
**kwargs,
)
else:
Expand All @@ -94,7 +83,30 @@ def uniform(shape, minval=0.0, maxval=1.0, dtype=None, seed=None):
shape,
minval=minval,
maxval=maxval,
seed=init_seed,
seed=make_seed(seed),
**kwargs,
)

def randint(shape, minval=0.0, maxval=1.0, dtype="int32", seed=None):
kwargs = {}
if dtype:
kwargs["dtype"] = dtype
if keras_3():
return keras.random.randint(
shape,
minval=minval,
maxval=maxval,
seed=seed,
**kwargs,
)
else:
import tensorflow as tf

return tf.random.uniform(
shape,
minval=minval,
maxval=maxval,
seed=make_seed(seed),
**kwargs,
)

Expand Down
8 changes: 4 additions & 4 deletions keras_cv/layers/preprocessing/aug_mix.py
Original file line number Diff line number Diff line change
Expand Up @@ -127,20 +127,20 @@ def _sample_from_beta(self, alpha, beta):
return sample_alpha / (sample_alpha + sample_beta)

def _sample_depth(self):
return random.uniform(
return random.randint(
shape=(),
minval=self.chain_depth[0],
maxval=self.chain_depth[1] + 1,
dtype=tf.int32,
dtype="int32",
seed=self._seed_generator,
)

def _loop_on_depth(self, depth_level, image_aug):
op_index = random.uniform(
op_index = random.randint(
shape=(),
minval=0,
maxval=8,
dtype=tf.int32,
dtype="int32",
seed=self._seed_generator,
)
image_aug = self._apply_op(image_aug, op_index)
Expand Down
15 changes: 15 additions & 0 deletions requirements-common.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# Library deps.
absl-py
regex
pandas
keras-core>=0.1.6
tensorflow-datasets
pycocotools
# Tooling deps.
packaging
flake8
isort
black
pytest
build
namex
Loading
Loading